Edit

Share via

Microsoft Sentinel data lake (preview) service parameters and limits

The following service parameters and limits apply to the Microsoft Sentinel data lake service.

Service parameters and limits for tables, data management, and ingestion

Note

During preview Microsoft Sentinel data lake uses a single region. Your primary and other workspaces must be in the same region as your tenant’s home region. Only workspaces in the same region as your tenant’s home region can be attached to the data lake.

Important

If your organization uses Customer-Managed Keys (CMK) for data encryption, be aware that CMK isn't fully supported for data stored in the Microsoft Sentinel data lake during the preview period. Any data ingested into the data lake, such as custom tables or transformed data is encrypted using Microsoft-managed keys. Onboarding to the Microsoft Sentinel data lake during the preview period may not fully align with your organization's encryption policies or data protection standards.

The following table lists the service parameters and limits for the Microsoft Sentinel data lake (preview) service related to table management, data ingestion, and retention. These limits include, but aren't limited to, Azure Resource Graph data, Microsoft 365 data, and data mirroring.

Category Parameter/limit
Workspaces per tenant 20 workspaces during preview
Data ingestion per minute to a data collection endpoint 50 GB
Default ingestion volume rate threshold in LALog Analytics workspaces 6 GB/min uncompressed
Ingestion requests per minute to a data collection endpoint 15,000
Lake Retention (Asset data) 12 years
Lake Retention (Aux) 12 years
Maximum size for field values (Log Analytics) 32 KB (truncated above the limit)
Table setup latency during onboarding 90-120 minutes
New table setup latency 90-120 minutes
Switching data between tiers latency 90-120 minutes

Service parameters and limits for VS Code Notebooks

The following section lists the service parameters and limits for Microsoft Sentinel data lake (Preview) when using VS Code Notebooks.

Category Parameter/limit
Custom table in the analytics tier Custom tables in analytics tier can't be deleted from a notebook; Use Log Analytics to delete these tables. For more information, see Add or delete tables and columns in Azure Monitor Logs
Gateway web socket timeout 2 hours
Interactive query timeout 2 hours
Interactive session inactivity timeout 20 minutes
Language Python
Max concurrent notebook jobs 3, subsequent jobs are queued
Max concurrent users on interactive querying 8-10 on Large pool
Session start-up time Spark compute session takes about 5-6 minutes to start. You can view the status of the session at the bottom of your VS Code Notebook.
Supported libraries Only Azure Synapse libraries 3.4 and the Microsoft Sentinel Provider library for abstracted functions are supported for querying the data lake. Pip installs or custom libraries aren't supported.
VS Code UX limit to display records 100,000 rows

Service parameters and limits for KQL queries in the lake tier

The following service parameters limitations apply when writing queries in Microsoft Sentinel data lake (Preview).

Category Parameter/limit
Concurrent interactive queries 45 per minute
Query result data 64 MB
Query result rows 30,000 rows
Query Scope Single workspace
Query timeout 8 minutes
Queryable time range Up to 12 years, depending on data retention.

Service parameters and limits for KQL jobs

The following table lists the service parameters and limits for KQL jobs in the Microsoft Sentinel data lake (Preview).

Category Parameter/limit
Concurrent job execution per tenant 3
Job query execution timeout 1 hour
Jobs per tenant (enabled jobs) 100
Number of output tables per job 1
Query scope Single workspace
Query time range Up to 12 years