Issue with Oracle Connector Ver. 2.0 in ADF writing to Parquet fles
As recommended I have updated our Oracle connectors in our ADF linked services from ver 1.0 to 2.0 since ver 1.0 will be deprecated by 31st July 2025. Since using ver 2.0 we are getting errors when saving files to parquet format. Below is the error we…
Azure Data Factory
ADF Inner Pipeline Still Runs Disabled Stored Procedure Activity via Triggered Parent Pipeline
Hi, I'm facing an issue in Azure Data Factory where a disabled stored procedure activity is still being executed when a pipeline is triggered. Here's the setup: I have a pipeline (let's call it InnerPipeline) that contains a stored procedure activity…
Azure Data Factory

Read varchar(max) field from sql server DB and export to datalake as actual document
Hi, I have a table in sql server with the below fields. ActivityId, DocumentBody(images/documents), filename, MimeType(application/octet-stream) I need to read this table and export to the blob container as an image/document depending on the filetype. I…
Azure Data Factory

How to determine partition column for large table extraction from source systems during Azure-based data migration?
In our data migration project (source: IBM DB2 → Azure), we are designing a control-table-based ingestion framework to handle large source tables (10–1500 GB range). For performance and checkpointing, each table is split into partitions, and data is…
Azure Data Factory

My on-prem data gateway is not showing in the Installed Instance list in the Azure Portal
I installed an on-prem data gateway on my on-prem server. I then tried creating a corresponding data gateway in my Azure portal but the local installation is not showing in the list to select. I used the same login for both the local installation and the…
Azure Data Factory

dataset showing warning even after updating linked service version from 1 to 2
ADF dataset showing warning even after updating linked service version from 1 to 2 also tried to refresh the page tried to reopen the ADF ports but still the same warning also the test connection is successful in linked service after updating…
Azure Data Factory
How to trigger an Azure Data Factory pipeline through API using praameters
Hello All, I have a use case where I want to trigger an Azure Data Factory pipeline through API. Right now I am calling the API in Databricks and using Service Principal(token based) to connect to ADF from Databricks.The ADF pipeline has some parameters…
Azure Data Factory
How to copy Azure database from Elastic Business Critical to Standard edition using ADF or T-SQL?
I'm trying to copy an Azure SQL Database from an Elastic Pool in the Business Critical tier to a Standard (DTU-based) edition using T-SQL. However, the following attempt fails with the error: Msg 40808, Level 16, State 1, Line 10: The edition…
Azure Data Factory
Sync feature between live mode and git mode Data Factory
Is There an Official Way to Synchronize Live Mode and Git Mode in Azure Data Factory? I'm currently using Azure Data Factory with Git integration enabled. However, I was curious about what happens if I start making changes in Live mode instead of Git…
Azure Data Factory
Export application/octet-stream field from sql server to blob container
Hi, I have a table in sql server with the below fields. ActivityId, DocumentBody(images/documents), filename, MimeType(application/octet-stream) I need to read this table and export to the blob container as an image/document depending on the filetype. I…
Azure Data Factory
How to load the data from Sql server DB to Snowflake DB by ADF without using any external stage?
How to load the data from Sql server DB to Snowflake DB by ADF without using any external stage? We have PII data which is not approved by our security to stage in any cloud storage (ADLS) Hence the requirement is to read the data from the source and…
Azure Data Factory
How to extract header row in a file containing column definitions and compare these against a control table
What is the best way to design a pipeline in ADF where you can dynamically capture the header row (only) from a source file and compare this row containing column names against a control table with column definitions. If they match proceed to further…
Azure Data Factory
While executive the pipeline in ADF taking long time
While executive the pipeline in ADF taking long time more then 2hr,Previous same pipeline run with in 1.30 hrs. showing mentioned error. Pipeline format, copy activity run successfully but scrip activity taking long time and pipeline fails showing…
Azure Data Factory
ADF Pipeline Still Uses Old Linked Service After Updating Sink Dataset and Publishing
Hi, I'm encountering an issue in Azure Data Factory where my pipeline seems to ignore an updated linked service for the sink dataset. Here’s the scenario: I have a pipeline with a Copy Data activity. The sink dataset was originally using Linked…
Azure Data Factory
ADF error 'The object has not yet been initialized'
Hello, We have a pipeline that loads multiple data objects from SAP CDC source which runs okay for the past few weeks. This error only pops up in the last few days. Error: "The object has not yet been initialized." We have asked SAP team, they…
Azure Data Factory
How to fix ClientIPNotAuthorized on ADF
How do I fix the ClientIDNotAuthorized error message? I am on VPN as well.
Azure Data Factory
ADF triggers failing after migrating to new resource group in same subscriptioin
Hi Team, I have recently migrated some basic ADF pipelines with data flow activity which has source & sinck with minimal transformation (derived column). Migration was successful. when i debug, they runs successful. However, when i trigger manually…
Azure Data Factory
Copy activity convert NUMBER to DOUBLE format
Hi we use the ADF copy activity to copy data from an oracle source to azure gen2 container as parquet file. The source data contain data type NUMBER, but the copy activity converts it to DOUBLE format. Could you please provide a solution for this…
Azure Data Factory
'json' parameter is not valid after updating secret key in ADF web activity
Hello, We have a pipeline with two web activities: The first retrieves a secret key from the key vault. The second uses that secret key to obtain a bearer token. Recently, we updated the expired secret key retrieved in the first web activity. The…
Azure Data Factory
The ADF Databricks job Activity doesn't seem to pass parameters
I have created an ADF pipeline that uses the databricks job activity. The linked service works well. When I run the pipeline, the databricks job fails - the ADF component fails, and I see a failed job run in databricks too. The problem seems to be the…