Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
2 answers
ADF Logging Strategy – Expert Insights Wanted!
ADF Logging Strategy – Expert Insights Wanted! I'm currently building a reusable logging framework in Azure Data Factory (ADF) to capture execution metadata for key activities like Data Flows, Copy Activities, and Stored Procedures — and I’d love to get…
Azure Data Factory
asked 2025-08-07T17:31:45.4033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 79%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
sudadigitus az
0
Reputation points
commented 2025-08-12T07:37:38.7933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 16%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Kalyani Kondavaradala
405
Reputation points Microsoft External Staff
Moderator
1 answer
Azure Data Factory: Inquiry on New Undocumented Property 'trustModeClaimForMi'
It looks like PG added a new undocumented property on data factory resource properties: "trustModeClaimForMi": { "value": "Enabled", "editable": true } Any ideas about this and any ETA when…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-08T09:06:38.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 45%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJK%3C/text%3E%3C/svg%3E)
Janne Kujanpää
256
Reputation points
edited an answer 2025-08-12T06:38:08.46+00:00
Amira Bedhiafi
35,766
Reputation points
Volunteer Moderator
1 answer
Backup of ADF pipelines
Hi Everyone, I am looking for backup of ADF including pipelines, datasets and dataflows. Please find the snapshot, how to take backup of this so that I can restore it when required.
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-05T07:45:38.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 4%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKR%3C/text%3E%3C/svg%3E)
Kundan Ray
0
Reputation points
commented 2025-08-11T18:20:06.4966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
28,225
Reputation points Microsoft External Staff
Moderator
0 answers
create around 800 helper tables with ADF
We have a requirement in our project to create around 800 helper tables in DB2, each with specific columns (row_hash calculation, table_name, primary column, partition column, partition value). We have tried two approaches so far: Using a stored…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-11T16:36:53.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 56.00000000000001%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Janice Chi
580
Reputation points
commented 2025-08-11T18:00:58.7+00:00
Smaran Thoomu
28,225
Reputation points Microsoft External Staff
Moderator
0 answers
Managed Airflow scheduler not working
We have a managed Airflow instance in our subscription, and the UI shows that the last heartbeat was a day ago (08/09). I tried resetting the Airflow instance multiple times, either by changing a variable or importing dags from a storage, and nothing…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-11T14:39:01.2266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 93%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Milica Batinic
0
Reputation points Microsoft Employee
edited the question 2025-08-11T17:01:41.5466667+00:00
Smaran Thoomu
28,225
Reputation points Microsoft External Staff
Moderator
1 answer
CDC set up in Azure Data Factory for Onprem Sql to Blob Storage
Hello, We are looking to use Azure Data Factory new feature CDC to load data from On prem Sql Server to Data lake Blob Storage. We have already both Self hosted and Managed Integration Run time exists for other process and we have also brand new Self…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 78%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 17%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
1 answer
Error upgrading Mysql connector(s) for Azure Data Factory/Synapse
We are unable to successfully upgrade our MySql connector in Synapse (Linked service) to a newer version before the termination date of 30 September 2025. We have created a ticket in the past and the outcome of the ticket was: "Resolution: There…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-08T07:09:09.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 49%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERN%3C/text%3E%3C/svg%3E)
Renald Noordam
0
Reputation points
commented 2025-08-11T11:48:56.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 50%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Venkat Reddy Navari
5,330
Reputation points Microsoft External Staff
Moderator
3 answers
One of the answers was accepted by the question author.
Automatically cancelling a copy activity when no data is being read.
I have a data source that sometimes causes copy activities to get stuck - no data is being read or copied. ADF just shows the time to first byte as continually increasing until timeout is reached. Ideally I want to catch this as early as possible, but…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-07T09:30:26.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 84%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELA%3C/text%3E%3C/svg%3E)
Lockyer, Aaron (SGRE COG B MS SYS BSPM)
45
Reputation points
accepted 2025-08-11T07:52:32.2766667+00:00
Lockyer, Aaron (SGRE COG B MS SYS BSPM)
45
Reputation points
1 answer
Row Count Difference between the Web and Copy activity using Salesforce V2 connector
Hi! I am using the Salesforce V2 connector to extract data from my salesforce account. When I use the Copy activity, I get around 1200 rows for a certain object. And when I use the Web activity to pass a query that does a Count of Id from that same…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-06T22:06:12.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 22%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Manjeet Acharya
20
Reputation points
commented 2025-08-11T07:10:53.2366667+00:00
Kalyani Kondavaradala
405
Reputation points Microsoft External Staff
Moderator
1 answer
Salesforce to Dynamic CRM migration using ADF.
Hi Team , i am working on Salesforce to Dynamic CRM migration using ADF . while creating the New Linked service using Salesforce V2 in ADF , we have encounter below issue. Below are the screenshots. Could you please help us to resolve the issue ? Error…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-05T15:04:17.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(313.6, 68%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEP%3C/text%3E%3C/svg%3E)
Earala, Prasad
0
Reputation points
commented 2025-08-11T03:14:25.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 13%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPV%3C/text%3E%3C/svg%3E)
Pratyush Vashistha
975
Reputation points Microsoft External Staff
Moderator
0 answers
Not able to create ADF due regional constraint in student free tier account
Not able to create Data Factory due to the below error-
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-10T05:47:26.1366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 10%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
#AVINASH CHAURASIYA#
0
Reputation points
commented 2025-08-11T03:07:46.7966667+00:00
Pratyush Vashistha
975
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
ADF Pipeline failed with error "An insufficient number of arguments were supplied for the procedure or function cdc.fn_cdc_get_net_changes"
I have a pipeline setup to copy set of tables from a source Azure DB into another Azure SQL DB. The pipeline has CDC enabled dataflow. The tables that I am trying to copy/sync have the CDC enabled. I have a setup/metadatatable where I have columns -…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-07-31T18:04:05.8066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 81%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAW%3C/text%3E%3C/svg%3E)
Aaron Wong
40
Reputation points
accepted 2025-08-08T16:16:52.8333333+00:00
Aaron Wong
40
Reputation points
0 answers
ADF Pipeline fails in trigger mode with error: no client credentials user enabled"},Source=Microsoft.Connectors.Salesforce,, but runs successfully in debug mode
I have an ADF Pipeline that is connected to Salesforce. It runs fine in debug mode. But when I trigger it, it fails with user credential error: Failure happened on 'Source' side.…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-03-18T17:12:02.85+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(214.4, 77%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
SW
20
Reputation points
edited the question 2025-08-07T16:05:08.86+00:00
Gérard Oomens
105.1K
Reputation points
Volunteer Moderator
1 answer
IR Upgrade to Spark 3.4 - data flow performance issues
Hello, We have a data flow that retrieves data from an API endpoint. The pipeline has not been changed in months, however on 7/9/2025, our pipeline time tripled in run time. We have tried to create a new Azure IR and increased the TTL option as well as…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-07-18T15:44:49.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 94%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Melissa Lord
0
Reputation points
edited a comment 2025-08-07T17:35:21.42+00:00
Melissa Lord
0
Reputation points
1 answer
QuickBooks Online Connector v2.0 in ADF Failing for Tables with Underscores
Hello ADF Support Team, We are encountering a consistent issue with the QuickBooks Online connector in Azure Data Factory (ADF) following the upgrade from version 1.0 to 2.0. The connector works successfully for most tables. However, we’ve observed that…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-07-25T16:44:10.6033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 9%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMZ%3C/text%3E%3C/svg%3E)
Mandar Zope
0
Reputation points
commented 2025-08-08T05:01:02.1766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 4%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESM%3C/text%3E%3C/svg%3E)
SSingh-MSFT
16,381
Reputation points
Moderator
1 answer
One of the answers was accepted by the question author.
How to look up values in another table based on multiple columns?
I have a dataset that needs to reference another table to look up and replace values. I'm not quite sure how to set this up in the data flow, since there's no common columns in the dataset or reference tables. There are multiple columns in the dataset…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 46%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
Unable to stop Change Data Capture
I'm following the official documentation on how to setup CDC: https://learn.microsoft.com/en-us/azure/data-factory/how-to-change-data-capture-resource I'm able to create a CDC but it fails to start: When trying to start it again I get this error: Unable…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-06T08:24:47.15+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 4%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESJ%3C/text%3E%3C/svg%3E)
Stark, John
20
Reputation points
accepted 2025-08-07T04:49:24.1866667+00:00
Stark, John
20
Reputation points
1 answer
Adf script activity inner join from different linked service
I have two Iinked service1 - table " Ext_emp" and linked service2 - table "Int_emp". Now I want to combine both this table using inner join and update table "Ext_emp" column " Processed_Flag* based on matching row…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-06T14:47:23.52+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 35%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Pankaj Joshi
391
Reputation points
commented 2025-08-07T12:35:02.6433333+00:00
Venkat Reddy Navari
5,330
Reputation points Microsoft External Staff
Moderator
2 answers
One of the answers was accepted by the question author.
Partial Inserts in ADF Copy Activity and Stored Procedure — Is it Possible?
Hi community, I’m working on a data pipeline in Azure Data Factory (ADF) where: I copy data from ADLS (Azure Data Lake Storage) into an Azure SQL RAW layer table using a Copy Activity. After that, I use a Stored Procedure Activity to move data from the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-07T07:07:52.0066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 57.00000000000001%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYS%3C/text%3E%3C/svg%3E)
YERNAIDU SIRAPARAPU
85
Reputation points
accepted 2025-08-08T16:14:56.9166667+00:00
YERNAIDU SIRAPARAPU
85
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure Data factory - executepipeline timeout ?
In one of my processes, one of the ExecutePipeline activity fails after 1 hour with a timeout issue. I would like to set-up a longer time-out for this parameter, but the option timeout is not available on the interactive screen. When I try to update…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,759 questions
asked 2025-08-08T04:22:52.8866667+00:00
Thierry Tropée
40
Reputation points
commented 2025-08-08T07:57:27.73+00:00
Pratyush Vashistha
975
Reputation points Microsoft External Staff
Moderator