UAL = Unaligned Activity Logs

The unified audit log is a centralized repository for M365 user and admin activities. The activities originate from different applications, such as Exchange, Teams, SharePoint, Azure, OneDrive and Defender XDR.

In this blog, we compare four different ways to acquire and investigate the unified audit logs (UAL):

• Purview Audit Search
• Defender For Cloud Apps CloudAppEvents Logs
• Sentinel OfficeActivity Logs
• Invictus Incident Response Microsoft Extractor Suite

There is another approach to acquire the UAL logs, which is not included in this blog. The UAL can also be retrieved via the Office 365 Management Activity API.

There are many interesting differences between the different acquisition methods, resulting in Unaligned Activity Logs. At this moment, you cannot rely on a single source to collect all activities; no logs does not mean it did not happen. These results are key if you are performing incident response, forensics, or want to detect malicious activities based on UAL logs.

If you use the UAL for Incident Response do not rely solely on the OfficeActivity and CloudAppEvents tables, always perform data acquisition to get a complete picture of the performed activities!

Activity Comparison

Before we dive into the specific (dis)advantages of all the individual acquisition methods we compare the overall coverage of the products. The coverage is based on 191 unique activities performed in a single tenant with default configuration. None of the tools log all the 191 actions performed during the research, meaning you need to combine acquisition methods to get a complete overview.

The dataset used for this research is available on GitHub.

The image below pivots the number of unique operations identified per application.

Findings

1. None of the acquisition methods get 100% coverage on the performed activities, which is concerning.
2. You cannot solely rely on UAL from CloudAppEvents and OfficeActivity, for incident response additional data acquisition is needed.
3. While not visible in the image below there is a disadvantage in the schema that the OfficeActivity table uses, the OfficeActivity table does not include the OperationCount field for operations. As Microsoft describes in their documentation This field is used for aggregated actions: The number of bind operations that were aggregated in the record is displayed in the OperationCount field in the AuditData property. The result of this is that users with only OfficeActivity tables cannot distinguish if a row is aggregated or not. The image below shows the importance of this field since more than 1/3th of the events are aggregated in my test environment.
4. Defender For Cloud Apps logs significantly less Exchange logs than the other acquisition methods. The activities that were not included are: Set-AdminAuditLogConfig, Set-TransportConfig,Enable-AddressListPaging, Set-MailboxPlan, Set-TenantObjectVersion, Set-ExchangeAssistanceConfig,Set-OwaMailboxPolicy, Add-MailboxPermission, New-ExchangeAssistanceConfig,Install-AdminAuditLogConfig, Set-RecipientEnforcementProvisioningPolicy,Install-DefaultSharingPolicy, Install-ResourceConfig and Install-DataClassificationConfig.
5. Purview Audit Search and Invictus Incident Response Microsoft Extractor Suite are the only two that include almost all activities (performed during this test). The interesting part is that both are not available for active querying.
6. The activities logged for Microsoft Teams differentiate. The activity TeamsSessionStarted is not logged by Defender For Cloud Apps, while WriteUserObjPreference and TeamsAdminAction are not logged by Sentinel OfficeActivity.
7. Defender For Cloud Apps has a gap in the SharePoint activities, the activities SiteLocksChanged, SiteDeleted and SiteIBModeChanged are included in the others but not in Defender For Cloud Apps.
8. The logs for Azure Active Directory (Entra ID) are not complete in the CloudAppEvents table. The actions UserLoggedIn and UserLoginFailed are not included. This is not a significant issue, because the AADSignInEventsBeta in Defender XDR and the SigninLogs table in Sentinel contain this information as well to cover the gap.
9. The event that is not included in the Purview Audit Search and the Invictus Incident Response Microsoft Extractor Suite is the Broke sharing inheritance operation for OneDrive. This event was only found in the Defender For Cloud Apps logs, thus also the OfficeActivity table is blind here.
10. Sentinel OfficeActivity Logs only cover Exchange, Microsoft Teams, OneDrive and Sharepoint. All other entries in the UAL are not included as per the configuration of the Microsoft 365 Sentinel connector.

Detailed Comparison

The detailed comparison describes the advantages and disadvantages of all the acquisition methods we have listed above.

Purview Audit Search

Advantage

• Best coverage for logged actions
• Easiest solution
• Default 180 days retention (More on retention and licences)
• Export easily imported in Azure Data Explorer
• Enabled by default

Disadvantages

• 50.000 export limit
• Not streamed to Sentinel or Defender XDR
• OneDrive, Broke sharing inheritance not logged (and presumably more)

Defender For Cloud Apps CloudAppEvents Logs

Advantage

• Streamed to Defender XDR
• Can be streamed to Sentinel for additional retention/enrichment
• Broad application scope

Disadvantages

• Gaps in logs, additional logs required to investigate all activities

Sentinel OfficeActivity Logs

Advantage

• Almost all activities logged for the products it supports
• Can be streamed to Sentinel
• Free Sentinel ingestion (More info)

Disadvantages

• Small scope of applications, additional logs required to investigate all activities
• OperationCount is missing from the schema
• Gap in OneDrive Logs

Invictus Incident Response Microsoft Extractor Suite

Advantage

• Best coverage for logged actions
• Retention based on Default Purview Retention (180 days)
• Leverages the by default enabled Purview Audit Logs
• Export easily imported in Azure Data Explorer

Disadvantages

• Not streamed to Sentinel or Defender XDR
• OneDrive, Broke sharing inheritance not logged (and presumably more)

Conclusion

With this research, we have concluded that the UAL is an Unaligned Audit Log. All four solutions their advantages and disadvantages you have to be aware of before you consider any of the options for a UAL strategy. With that being said the logs contain a lot of valuable content that should not be overlooked. I highly recommend choosing the proactive approach with this data source; build detections and hunt based on the data you have available!

It is important to realize that none of the four acquisition methods used in this blog covered 100% of the activities. Hence you need to combine the data sources to get a complete picture. This also means that you should not rely on one source only, but validate with another UAL acquisition method if the results are the same and to fill the gaps between the products.

From an incident response perspective, it is always worth it to do the data acquisition with Invictus Incident Response Microsoft Extractor Suite (Preferred due to Purview Limitations) to overcome the gaps in the Purview Search and the CloudAppEvents and OfficeActivity logs. These logs already provide a good basis for the start of the incident, pointing you to the user(s) to investigate. With the right entities you can refine your search, which massively reduces the time it takes to acquire the logs.

This research has been performed in a small test environment, the acquisition times and coverage might differ for big tenants. If you want to collaborate and test this on a bigger scale feel free to reach out! :)

Questions? Feel free to reach out to me on any of my socials.