Manage data tiers and retention in Microsoft Sentinel (preview)

Data you collect into Microsoft Sentinel (SIEM) and Microsoft Defender XDR is stored in tables. The Microsoft Defender portal lets you manage the retention period and the store costs associated with your data. You can manage retention and costs when you:

• Configure data connectors to send data to Microsoft Sentinel or Microsoft Defender XDR.
• Manage your existing tables and data.

This article explains how to manage table retention and tier options in the Microsoft Defender portal to optimize security operations and reduce costs in Microsoft Sentinel and Microsoft Defender XDR.

Which tables can you manage in the Defender portal?

This section describes the table types you can manage in the Defender portal.

How data tiers and retention work

You can retain data in Microsoft Sentinel in one of two tiers:

• Analytics tier: This tier makes data available for alerting, hunting, workbooks, and all Microsoft Sentinel features. It retains data in two states:

  • Analytics retention: In this "hot" state, data is fully available for real-time analytics - including high-performance queries and analytics rules - and threat hunting. By default, Microsoft Sentinel and Microsoft Defender XDR retain data in this tier for 30 days. You can extend the retention period of all tables to up to two years at a prorated monthly long-term retention charge. You can extend the retention period of Microsoft Sentinel solution tables to 90 days for free.
  • Total retention: By default, all data in the analytics tier is mirrored to the data lake for the same retention period. You can extend the retention of your data in the lake beyond the analytics retention, for up to 12 years of total retention at a low cost.

• Data lake tier: In this low-cost "cold" tier, Microsoft Sentinel retains your data in the lake only. Data in the data lake tier isn't available for real-time analytics features and threat hunting. However, you can access data in the lake whenever you need it through KQL jobs, analyze trends over time by running scheduled KQL or Spark jobs, and aggregate insights from incoming data at a regular cadence by using summary rules.

For more information about the differences between these two retention types, see Compare the analytics and data lake tiers.

By default, Microsoft Defender XDR retains threat hunting data in the XDR default tier, which includes 30 days of analytics retention, included in the XDR license. You can extend the retention period of supported Defender XDR tables beyond 30 days. When you extend the retention period of supported Microsoft Defender XDR tables Microsoft automatically creates the table in your Microsoft Sentinel workspace in the analytics tier.

This diagram shows the retention components of the analytics, data lake, and XDR default tiers, and which table types apply to each tier:

For more information about the Microsoft Sentinel data lake, see What is Microsoft Sentinel data lake.

Compare the analytics and data lake tiers

This table compares the two analytics and data lake tiers and their key characteristics:

What happens when you modify table settings

You can switch a table's tier and retention settings at any time.

When you change a table's tier from analytics to data lake, all real-time analytics and hunting queries stop working.

When you shorten a table's total retention, Microsoft waits 30 days before removing the data, so you can revert the change and avoid data loss if you made an error in configuration.

When you increase total retention, the new retention period applies to all data that was already ingested into the table and wasn't yet removed.

When you change the analytics retention settings of a table with existing data, the change takes effect immediately.

Example:

• You have a table in the analytics tier with 180 days of analytics retention. By default, the Total retention is also set to 180 days.
• You change the analytics retention to 90 days without changing the Total retention period of 180 days.
• Microsoft Sentinel removes the last 90 days of data from in analytics retention automatically, but continues to store data that's 90-180 days in the data lake.

Preview limitations

As part of the public preview:

• Microsoft Sentinel tables with the Basic plan can only be managed from the Log Analytics workspace. For more information, see Manage tables in a Log Analytics workspace.

  To manage retention and tiering from the Microsoft Defender portal, change the table plan to analytics from the Log Analytics workspace.

• Some Microsoft Defender XDR tables can only be viewed in the Microsoft Defender portal. Currently, the Microsoft Defender portal supports managing these Microsoft Defender XDR tables:

  • AlertEvidence
  • AlertInfo
  • CampaignInfo
  • CloudAppEvents
  • DeviceEvents
  • DeviceFileCertificateInfo
  • DeviceFileEvents
  • DeviceImageLoadEvents
  • DeviceInfo
  • DeviceLogonEvents
  • DeviceNetworkEvents
  • DeviceNetworkInfo
  • DeviceProcessEvents
  • DeviceRegistryEvents
  • EmailAttachmentInfo
  • EmailEvents
  • EmailPostDeliveryEvents
  • EmailUrlInfo
  • FileMaliciousContentInfo
  • IdentityDirectoryEvents
  • IdentityLogonEvents
  • IdentityQueryEvents
  • SecurityAlert
  • SecurityIncident
  • UrlClickEvents

Next steps

Learn more about:

• Configuring log table tiers and retention settings
• Creating a search job to retrieve data matching particular criteria