Security Reports in M365
This blog post is about reports from Microsoft 365 Security also known as Microsoft 365 Defender. As a prerequisite, one or, in the best case, several Defender products should be set up.
The objective
My initial objective was to obtain a high-level management summary of our Microsoft Defender cloud environment, that we would share with our IT responsibles on a regular basis. It should be effective with its contents but also straight forward. The data should mainly come from Microsoft 365 Defender.
Please note that this is especially recommended when only utilizing the Microsoft Defender Suite. (not Azure Sentinel SIEM, as there more unified reports would be available)
Purpose of a (scheduled) security report
A personalized summary of the current and past security landscape can help to really understand the impact of your security products between the end users and security as a perimeter. It discloses information about your security posture and tells different parties about their contemporary and future strategy to follow. It could range from high- to low level, but often a management summary is required to be self explaining and straight forward. The key points of a security report consist of:
- Understand security products and their impact
- Summarize data from different event sources to successfully manage your environment and set the future course
- Enlighten decision makers and the management
But how do you get and summarize all the signals and outputs from your various security systems? Well, you need to know the right places to find reportings, have Graph interfaces and sometimes even need to go to a separate portal.
Part 1: where to find reports
Report types in Microsoft 365 Defender
Of course Microsoft 365 Defender houses a built-in report section:
| Category | Name | Content | Source |
|---|---|---|---|
| General | Security report | Identity, data, devices, apps lateral summary | Microsoft 365 Defender https://oceanleaf.ch/microsoft-cloud-security-correlation/ |
| Endpoints | Threat protection | Alert trends with classification and source | Defender for Endpoint https://oceanleaf.ch/defender-for-endpoint-configuration/ |
| Device health and compliance | Device information and Antiviurs health state | ||
| Vulnerable devices | Device vulnerability overview across OS & versions | ||
| Web protection | Web content information | ||
| Firewall | Firewall status and events (inbound/outbound and apps) | ||
| Device control | Device media usage (USB, Plug and Play etc.) | ||
| Attack surface reduction rules (ASR) | ASR detection, configuration and exclusions | ||
| Email & collaboration | Email & collaboration reports | All email & collaboration reports include: mailflow summary, email security (spoofing, attachments), threat protection, URL protection, and technical reports (transport rules, latency etc.) | Defender for Office 365 https://oceanleaf.ch/defender-for-office-365-fundamentals/ |
| Manage schedules | Configure scheduled reports | ||
| Reports for download | Download reports | ||
| Exchange mail flow reports | Technical mail flow reports |
Threat and vulnerability management (TVM) with Defender for Endpoint
For endpoint security, Defender for Endpoint provides massive insight into threat and vulnerability management (TVM). This mainly focuses on devices and software and their CVE addressing.
Other sources
Let's link some more information to product reports. This could also be helpful to find more relevant data.
Defender for Endpoint
To gather more report types, I would consider PowerBI with the: Microsoft's Defender for Endpoint Github repository or this one. Next to that, the portal can give more insight, with the device inventory and the information related to each device. Additionally Microsoft Intune (if in place) has an own reports and analytics section.
Defender for Identity
Scheduled reports can be set up in the Defender for Identity external portal (legacy).
- Summary
- Modifications to sensitive groups
- Passwords exposed in clear text
- Lateral movements paths to sensitive accounts
Defender for Office 365
For Defender for Office 365 I can recommend the built-in reports of Microsoft 365 and the reports from Exchange admin center.
One very important thing is missing at the time which constitutes of SPF and DKIM for DMARC. A DMARC policy supports aggregate and forensic reporting addresses. Unfortunately Microsoft does not provide a product for that use case yet. DMARC Analyzer from Mimecast is a suitable solution, to email security visibility and governance in terms of authentication.
Defender for Cloud Apps
Cloud discovery from Defender for Cloud Apps is perfect to raise SaaS collaboration information. Defender for Endpoint integration is the best and most easy way to fill it with data. There you also have the option to have app connectors or Firewall logs that deliver powerful data.
Azure AD
I have already written a whole blog post about Monitoring Azure AD. Summarized: platform logs come with good filter options and workbooks are even better for reporting.
Conditional Access
The service insights from Azure AD Conditional Access is what I would go for to get MFA and CA reports.
Do not forget: Asset management and ADDS
An extra source in addition to the already mentioned places would be an independent asset management for: more device information, device security and compliance stats - Antivirus software, status and if it is running an up to date intelligence version or any irregularities.
Next to that, do not forget about the on-premises Active Directory for: all Hybrid joined devices, hybrid/lateral events originated from ADDS, local identities and their sync state.
Part 2: Structure a management summary
The management summary which I wrote, has the following requirements and is structured with its contents like the following:
Requirements
- Understandable for a non IT person with explanations and on point
- Showing the big picture of the Microsoft 365 environment over the last 30 days
- Regularly bring attention on the IT, how the organizations posture in terms of Microsoft 365 is
Content list
| Name | Use |
|---|---|
| 1 Title & introduction | Explain the use of this document and the timespan of the reports (usually last 7-30 days) |
| 1.1 Document information | List information about: author, date created, validation through, classification (top secret), version, application, target audience |
| 1.2 Table of contents | List all of your chapters or report sections, I prefer to order after products (Defender for ...) |
| 1.3 Products in use | List all utilized products for security operations and their application in your environment |
| 2. Microsoft 365 Defender Incidents & Alerts | |
| 2.1 Management Summary | List a count of low, medium and high severity incidents from Microsoft 365 Defender (over all products) Additional: list top incident occurrences (type), events of special interest (unusual SecOps events) with explanation |
| 3. Defender for Cloud Apps | |
| 3.1 Dashboard overview | A screenshot of the dashboard includes the most relevant information about apps that your organization uses and how they are rated (blur the identities for keeping everything anonymous) |
| 4. Defender for Office 365 | |
| 4.1 Mail type summary | List mail type quantity (mails goodmail, emailphish, spamdetections, emailmalware, edgeblockspam) |
| 3.2 Mail flow | A screenshot of the mail flow, explain the different types (total email, edge block, processed, rule block, delivered) |
| 5. DMARC Analyzer from Mimecast | In my case, I create a screenshot of the domains dashboard in DMARC Analyzer from Mimecast and explain what DMARC, SPF and DKIM are used for |
| 6. Defender for Endpoint | |
| 5.1 Organization exposure score | Show the organization exposure score and the development over time, explain that this shows an overall security posture |
| 5.2 Organizational weakness | Weakness scores in the organization, CVEs that are exploitable, critical, or zero-day, those that have no security updates yet, or those with some |
| 5.3 Operating systems overview | List all operating system types and quantity |
| 5.4 Operating system version distribution | List operating system versions (especially Windows, e. g. Win21H1, 21H2) and quantity |
| 5.5 Antivirus health | Health state of Defender (active, impaired communications, inactive, no sensor data) |
| 5.6 Antivirus up to date | Defender version up to date state (updated, not updated, disabled, unknown) |
| 5.7 Device exposure level | Summarize quantities of low, medium or high classified devices and explain that these address to CVEs |
| 6. Threat Protection | Summarize all threat actions across your landscape |
| 6.1 Detection source | A screenshot of the detection sources, with explanation (EDR, Antivirus, SmartScreen, Custom TI, MDO, Automated Investigation, Microsoft Threat Experts, Custom Detection, 3rd Party sensors |
| 6.2 Threat categories | A screenshot of the various threat categories (command and control, execution, initial access, malware, suspicious activity, unwanted software |
| 7. Asset management reports | Individual - I use: Antivirus software type, Antivirus enabled, a list of machines with Antivirus disabled, Antivirus up to date |
| 8. Active Directory reports | Individual - for example Pingcastle |
If you want to dive deeper I can recommend the following sources: