Can we pass partition key as a parameter in a generic/dynamic azure dataflow.

Question

Can we pass partition key as a parameter in a generic/dynamic azure dataflow.

Batta, Uha Kanakavalli 0

Operation on target Load Entity failed: Operation on target Load Entity failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: com.microsoft.dataflow.Issues: DF-TX-114 - Key partitioning does not allow computed columns - [564 737],\nsource(\n) ~> DLExtract,EXE-0001,Dataflow cannot be analyzed as a graph\nDF-TX-114 - Key partitioning does not allow computed columns - [564 737],EXE-0001,Dataflow cannot be analyzed as a graph,\nsource(\n) ~> DBExtract\nDF-TX-114 - Key partitioning does not allow computed columns - EXE-0001,Dataflow cannot be analyzed as a graph,\nInsertRows sink(\n) ~> Insert2DB,[564 737]\nDF-TX-114 - Key partitioning does not allow computed columns - \nMarkasUpdate sink(\n) ~> Update2DB,EXE-0001,Dataflow cannot be analyzed as a graph,[564 737]","Details":""}

QuantumCache 20,681 Reputation points Moderator

2023-10-05T17:30:23.0866667+00:00

Hello @Batta, Uha Kanakavalli Just checking in if you have had a chance to see the previous response?

1 answer

Your answer

QuantumCache 20,681 Reputation points Moderator

2023-10-05T17:30:23.0866667+00:00

Hello @Batta, Uha Kanakavalli Just checking in if you have had a chance to see the previous response?

Answer 1

Subashri Vasudevan 11,306 Volunteer Moderator

Hi Batta, Uha Kanakavalli

There is a limitation in Key Paritioning in data flow. It doesnt allow dynamic expressions in key field.

If you want to use such dynamic expressions, you can use a derived column and use that in the paritition key field. But not directly on the key field.

Below thread discusses a similar issue where we use data flow parameter for key column.

https://stackoverflow.com/questions/75225600/azure-data-factory-dataflow-error-key-partitioning-does-not-allow-computed-colu

Below are my test data.

Screenshot 2023-10-05 at 6.11.08 PM

If i try to use month(toDate(date)) in the partition key column directly, i am getting same error as yours. So I add a derived column and then use the expression there as follows.

Screenshot 2023-10-05 at 6.11.20 PM

Screenshot 2023-10-05 at 6.11.34 PM

Please check and let us know for any question.

Thanks

QuantumCache 20,681 Reputation points Moderator

2023-10-12T18:26:57.77+00:00

Hi Batta, Uha Kanakavalli Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you!

Please click "Accept Answer" . So that we can close this thread.
Sidharth 0 Reputation points

2026-03-21T04:41:48.0333333+00:00

I am facing similar issue, Just checking if the issue still persists that key paritioning in data flow does not allow parameters or expressions in key field.

Share via

Can we pass partition key as a parameter in a generic/dynamic azure dataflow.

1 answer

Your answer