0

I have a KQL query (which includes a join too). The task is to join 2 tables in a Kusto database and dump into a new table.

When I run the query in its entirety in Azure Data Explorer UI, it runs successfully. However, when I run the same in ADF using a "Azure Data Explorer Command" block, I get this error :

    "error": {
        "code": "LowMemoryCondition",
        "message": "Request is invalid and cannot be executed.",
        "@type": "Kusto.Data.Exceptions.KustoServicePartialQueryFailureLowMemoryConditionException",
        "@message": "Query execution lacks memory resources to complete (80DA0007): hr: '2161770503' 'Partial query failure: Low memory condition (E_LOW_MEMORY_CONDITION). (message: 'bad allocation (E_LOW_MEMORY_CONDITION)', details: '')'."
    }

What I've tried:

  1. Created batches using count + foreach so that the Kusto job has to process 1000 rows at a time only.
  2. Kept the smaller dataset on the left as per Kusto best practices.

I fail to understand this difference between the memory limit between ADX UI and ADF. And is there any way to increase the memory limit for the ADF command?

Improve this question
Related questions
0
How to execute a .purge command in ADX using Azure Data Factory?
0
Incremental Data Load in Azure Kusto
2
Does Azure Data Explorer require Azure storage to work?
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.