Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
0 answers
@microsoft/azure-data-factory-utilities: purpose of the factoryId parameter
Hi, I have recently started using the @microsoft/azure-data-factory-utilities package to generate ARM templates in an Azure DevOps pipeline. It is working, and I am able to generate the ARM templates successfully. :) However, I have a question about the…
Azure Data Factory
asked 2026-09-04T07:55:42.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 75%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBN%3C/text%3E%3C/svg%3E)
Bánhidy Attila
20
Reputation points
asked 2026-09-04T07:55:42.2233333+00:00
Bánhidy Attila
20
Reputation points
2 answers
Trying to filter csv file based on content
I have an existing ADF pipeline that is creating an error due to some bad (or not acceptable data) displayed on the csv file. Basically, instead of having proper header on the first row with certain amount columns (with data), csv file displays data like…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-07-22T21:45:14.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 39%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJD%3C/text%3E%3C/svg%3E)
Justin Doh
1,000
Reputation points
edited a comment 2026-09-03T23:45:11.4+00:00
Justin Doh
1,000
Reputation points
2 answers
How to handle BigQuery mirroring when the source has more than 1,000 tables?
Hi, We are setting up database mirroring in a Microsoft Fabric workspace, with BigQuery as the source. Currently, our BigQuery source contains 1,193 tables, but a single mirrored database supports a maximum of 1,000 tables. Because of this limitation, we…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 87%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVG%3C/text%3E%3C/svg%3E)
1 answer
enable the access to local machine under self hosted integration runtime
enable the access to local machine under self hosted integration runtime
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-09-02T06:35:00.2133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 39%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
adula amala
0
Reputation points
commented 2026-09-03T14:30:12.1833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(25.6, 64%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHY%3C/text%3E%3C/svg%3E)
Himaja Y
280
Reputation points Microsoft External Staff
Moderator
3 answers
Copy data from Sharepoint using Authorization: Bearer token getting status code 401 Unauthorized
ErrorCode=HttpRequestFailedWithUnauthorizedError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with status code 401 Unauthorized, usually this is caused by invalid credentials, please check your activity…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-09-03T11:20:09.3133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 28.000000000000004%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Prakash Palanivelu
0
Reputation points
answered 2026-09-03T12:03:51.4566667+00:00
kagiyama yutaka
5,165
Reputation points
3 answers
Checkpoint Key on Azure Data Factory (ADF)
Hi, We have created Azure Data Flow with Azure SQL Database Managed Instance as source and Azure Data Lake Storage Gen 2 (ADLS) as a sink (parquet format). Change Data Capture is enabled at the source with SQL Server CDC type and "Full on the first…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-09-02T16:03:08.1133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 28.999999999999996%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Anil Kestur
0
Reputation points
commented 2026-09-03T11:08:05.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 57.99999999999999%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESY%3C/text%3E%3C/svg%3E)
SIVASANKAR YEDDULA
150
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
@microsoft/azure-data-factory-utilities fails with downloaded main.js
Hi, I've found a similar problem here: https://learn.microsoft.com/en-us/answers/questions/1483654/@microsoft-azure-data-factory-utilities-fails-to-d. My company's network is highly restricted, so the pipeline is not able to download the main.js…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-26T06:37:38.2966667+00:00
Bánhidy Attila
20
Reputation points
accepted 2026-08-31T10:42:42.9033333+00:00
Bánhidy Attila
20
Reputation points
1 answer
In-place upgrade Windows Server 2022 → 2025 on Azure VM hosting Self-Hosted Integration Runtime (ADF) — impact on SHIR configuration/credentials?
We're planning an in-place OS upgrade of an Azure VM from Windows Server 2022 Datacenter to Windows Server 2025 Datacenter, following the documented procedure at learn.microsoft.com/en-us/azure/virtual-machines/windows-in-place-upgrade. Environment…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-26T13:14:45.8433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 19%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKY%3C/text%3E%3C/svg%3E)
K Yogesh
0
Reputation points
commented 2026-08-31T02:20:53.4766667+00:00
Allan Solomon Mejia
7,585
Reputation points
1 answer
Azure Data Factory Data Flow CDC Source Not Refreshing When Using Dynamic Table Names
Environment Azure Data Factory running in a Visual Studio subscription SQL Server source using CDC-enabled tables AdventureWorks sample database used for testing Mapping Data Flow with: Source (SQL CDC) Sink (Parquet or JSON) Pipeline…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-25T09:07:29.6933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 49%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKK%3C/text%3E%3C/svg%3E)
Kishor kumar Reddy Akki
50
Reputation points
commented 2026-08-28T17:37:29.0166667+00:00
SIVASANKAR YEDDULA
150
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
Azure Data Factory - self-hosted integration runtime - missing metrics in Data Factory (v2) overview
We just noticed, that the metrics from the self hosted integration runtime, like Integration Runtime CPU or Integration Runtime Memory are missing. The last metrics, we have are from Apr. 29 of this year (around 6:00 CEST) - before this time, we have…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-18T12:15:02.3233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 46%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVE%3C/text%3E%3C/svg%3E)
von Euw Barbara
21
Reputation points
accepted 2026-08-28T04:45:17.1566667+00:00
von Euw Barbara
21
Reputation points
1 answer
How to copy image from sqlserver in varbinary to Azure Blob storage using azure data factory
Hi all, “We are looking for a sample Copy Activity in Azure Data Factory to copy image data from Oracle/SQL databases into Azure Blob Storage as image files. The challenge we face is that when copying from relational databases, we are unable to perform a…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-13T10:11:50.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 20%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
manish verma
516
Reputation points
commented 2026-08-27T08:55:15.5366667+00:00
Himaja Y
280
Reputation points Microsoft External Staff
Moderator
0 answers
Join: Microsoft Azure Q&A Champions Program
The Microsoft Azure Q&A Champions Program recognizes and empowers a global community of internal and external Azure experts who help customers succeed by providing high-quality, trusted answers on Microsoft Q&A. The program scales expert led…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-26T19:35:44.28+00:00
Ashley Swearingen
0
Reputation points Microsoft External Staff
Moderator
4 answers
One of the answers was accepted by the question author.
@microsoft/azure-data-factory-utilities fails to download script from internet in securized private build agent
In this link (https://www.npmjs.com/package/@microsoft/azure-data-factory-utilities?activeTab=readme) it has been mentioned that the package @microsoft/azure-data-factory-utilities downloads a JavaScript (https://adf.azure.com/assets/cmd-api/main.js)…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2024-01-08T14:34:35.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 50%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
Jean-Christophe Gauvreau
20
Reputation points
commented 2026-08-25T18:13:11.5566667+00:00
Erland Sommarskog
137.1K
Reputation points MVP
Volunteer Moderator
1 answer
ADF Power Query cannot evaluate Azure SQL source with system-assigned managed identity
Issue In Azure Data Factory Power Query (Data Wrangling), an Azure SQL Database dataset cannot be evaluated in the Power Query Online editor. The following error is displayed: Credentials are invalid or missing, so this query could not be…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-21T08:38:53.87+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 26%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
豊田 明
0
Reputation points
commented 2026-08-25T17:20:16.66+00:00
SIVASANKAR YEDDULA
150
Reputation points Microsoft External Staff
Moderator
1 answer
One of the answers was accepted by the question author.
Example for Fabric Data Pipeline CICD with variable substitution.
Hi All, I am hoping someone can help me figure out how to implement CICD with connection substitution for a Fabric Data Factory pipeline. An example would be awesome - All I want to do is to promote a Data Factory pipeline to another environment and…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-19T17:30:31.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(28.799999999999997, 57.00000000000001%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJA%3C/text%3E%3C/svg%3E)
John Aherne
176
Reputation points
edited a comment 2026-08-25T07:13:51.99+00:00
SIVASANKAR YEDDULA
150
Reputation points Microsoft External Staff
Moderator
3 answers
Azure Data Factory Data Flow Trigger Runs Successfully but Source File Is Not Updated (Debug Works Fine)
We are currently experiencing an issue with Azure ADF data flows. While the data flow functions as expected in debug mode and correctly updates the storage file in sink, it fails to update the storage file when triggered, even though the logs indicate…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-14T10:32:32.24+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 68%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Saksham Dubey
0
Reputation points
answered 2026-08-24T15:39:49.6966667+00:00
SIVASANKAR YEDDULA
150
Reputation points Microsoft External Staff
Moderator
1 answer
ADF ODBC Linked Service Snowflake JWT key-pair connection string exceeds 1024-char limit secure workaround needed
Environment Azure Data Factory with Self-Hosted Integration Runtime Snowflake Reader Account owned by an external team (no CREATE STAGE permission available) Must use ODBC connector Snowflake V2 connector not viable because it requires COPY INTO and…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-24T08:25:35.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 63%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Singh, Gunveen
0
Reputation points
answered 2026-08-24T10:20:13.1+00:00
Himaja Y
280
Reputation points Microsoft External Staff
Moderator
0 answers
The specified network name is no longer available - error code - 22852
Hello Team, Today I faced an error, when i am trying to do test connection. Please resolved as soon as possible. I need to resolve please help me to resolve. 
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2024-09-20T12:10:29.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 96%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Balwant Singh
60
Reputation points
edited a comment 2026-08-20T13:22:39.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 16%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBV%3C/text%3E%3C/svg%3E)
BIRJU VERMA
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Vulnerable libcurl.dll detected in Microsoft Integration Runtime 5.0 bundled ODBC drivers - supported remediation?
Hello, Our security vulnerability scanner has identified vulnerable versions of libcurl.dll within the Microsoft Integration Runtime 5.0 installation directory: C:\Program Files\Microsoft Integration Runtime\5.0\Shared\ODBC Drivers\ We verified that the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-08-18T05:13:51.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 64%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPB%3C/text%3E%3C/svg%3E)
Pawan Bhagat
20
Reputation points
accepted 2026-08-20T08:07:33.5233333+00:00
Pawan Bhagat
20
Reputation points
3 answers
Azure Data Factory Private endpoint for Portal Endpoint
Hi Team, We are looking for recommendations and best practices around configuring the portal private endpoint for Azure Data Factory in a multi-VNET environment. Current setup: We have multiple ADF instances owned by separate teams. Each ADF has its…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
12,198 questions
asked 2026-06-05T12:04:07.2533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 54%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Sumit Gaur
475
Reputation points
commented 2026-08-19T17:57:07.2933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 5%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Alicja Rudnicka
30
Reputation points