How to use existing cache for external table when acceleration in progress

Question

How to use existing cache for external table when acceleration in progress

Kunpeng Fan 0 Microsoft Employee

I enabled query acceleration for my external table which binds a delta table (1TB) on ADLS. But acceleration progress needs 1.5 hours to complete.

I found during acceleration in progress, querying on the table is quite slower than case when acceleration completed.

How can I use the existing acceleration cache/index and after acceleration completed, Kusto will switch to new index?

Thanks.

Smaran Thoomu 28,225 Reputation points Microsoft External Staff Moderator

2025-08-05T17:12:01.92+00:00

Kunpeng Fan Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Your answer

Smaran Thoomu 28,225 Reputation points Microsoft External Staff Moderator

2025-08-05T17:12:01.92+00:00

Kunpeng Fan Following up to see if the below answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

Smaran Thoomu 28,225 Microsoft External Staff Moderator

Hi Kunpeng Fan
Thanks for your question.

When query acceleration is enabled on an external Delta table in ADLS, Azure Data Explorer (Kusto) starts building a new acceleration index in the background. During this index build phase, queries will not use the old index - they temporarily fall back to full table scans, which is why you're seeing slower performance.

Currently, Kusto does not support using an existing index or cache while the new acceleration is being built. Once the acceleration build is complete, the system automatically switches to the new optimized index, and you'll see improved performance again.

If the performance during the build is a concern, one workaround is to run heavier queries after the acceleration is done, or stage queries during off-peak hours if possible.
For more information refer the below articles.

I hope this information helps. Please do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Kunpeng Fan 0 Reputation points Microsoft Employee

2025-08-01T05:06:10.4666667+00:00

Currently, Kusto does not support using an existing index or cache while the new acceleration is being built.

Do you have and link to approve that?
Smaran Thoomu 28,225 Reputation points Microsoft External Staff Moderator

2025-08-01T08:39:08.7233333+00:00

Hi Kunpeng Fan
At this time, Azure Data Explorer does not support fallback to an existing acceleration index or cache while a new acceleration index is being built. During the rebuild, queries fall back to scanning the underlying external table directly, which can lead to noticeably slower performance.

This fallback limitation is not explicitly called out in documentation, but it is implied through observed behavior during index refresh cycles. The articles I shared earlier outline how acceleration behaves during index builds, which aligns with what you're seeing.
I hope this information helps. Please do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Kunpeng Fan 0 Reputation points Microsoft Employee

2025-08-07T05:46:36.54+00:00

does maxage help this issue?
Smaran Thoomu 28,225 Reputation points Microsoft External Staff Moderator

2025-08-07T14:05:52.09+00:00

Hi Kunpeng Fan

Thanks for your question.

The MaxAge setting does not help while the acceleration is still in progress. It only works after the acceleration index is fully built.

What MaxAge does is tell the system how long it should keep using the accelerated data. For example, if the index is already built and still fresh (within the MaxAge limit), Kusto will use it for faster queries. But if the index is old, it won’t use it.

In your case, since the acceleration is still building, Kusto doesn’t use any index at all, it just scans the whole external table directly. So, even if you set MaxAge, it won’t speed things up during this time.

Once the acceleration is done, MaxAge becomes useful to control how long the system uses the new index.

Hope this clears it up! Let me know if you have any other questions.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

How to use existing cache for external table when acceleration in progress

1 answer

Your answer