I have a path to DataLakeStorage, which may or may not exist. I want to iterate over the contents of that folder, if it exists. In C# I would arrange to have a children collection, that was empty i...

I'm trying to decompress a zip file (with multiple files inside) using Azure Data Factory v2. The zip file is located in Azure File Storage. The ADF Copy task just copies the original zip file with...

Step to reproduce: I created a Copy Data first in the pipeline to simple transfer CSV files frol Azure VM to Azure Blob storage. I always use IRPOC1 as a connection via integration runtime and con...

I am trying to use two forEach activities to iterate on subfolders of folders with parameters to get metadata of subfolders. I have forEach1 and forEach2 with their own items array. Within the seco...

I am developing an ARM template for Azure Data Factory with managed private endpoints to SQL Server and Azure Datalake. However, when the ARM template completes execution, the managed private endpo...

Need to insert a parameter into a string using the dynamic data function from the pipeline parameters. Have tried backslash, double backslash, double single quote,@, @@, and other such nonsense. Ju...

Can some one please elaborate on when to use Polybase versus bulk insert in azure datafactory, what are the differences between these two copy methods?

We've been using the "new" CICD setup for Azure Data Factory as descibed by the documentation page: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-im...

{ "errorCode": "2200", "message": "ErrorCode=FailedDbOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A database operation failed with the following error: ...

I am not able to open Azure data factory in any browser, it just keep loading from past 1 hour. I have tried refreshing and using other browser, is there any specific reason why it happens? All oth...

I have a boolean expression: equals(myStringValue, targetStringValue) I have an array expression which might or might not be valid, depending on the boolean condition myArrayExpression. I want to w...

I was trying to invoke data factory pipeline from azure function programmatically. Its throwing following error. link: http://eatcodelive.com/2016/02/24/starting-an-azure-data-factory-pipeline-...

I am having an error when trying to create a parameterised Mapping Data Flow. More specifically, I have the following error. Anyone have suggestions on how to fix it or what the error may be? { "...

I am using below Terraform code to deploy Data factory Azure IR in managed virtual network: resource "azurerm_data_factory_integration_runtime_azure" "ManagedIR" { name = &quot...

So I am testing each of the activities of the pipeline and I want to disable some of the activities in it. Essentially there is an activity of sending emails which I want to disable as I wanted to ...

I have a really frustrating error trying to parse basic Json read from Blob Storage using a data set within ADF My Json is below [{"Bid":0.197514880839,"BaseCurrency":"AED&...

I am trying to create my Azure DevOps release pipeline for Azure Data Factory. I have followed the rather cryptic guide from Microsoft (https://learn.microsoft.com/en-us/azure/data-factory/continu...

I am not sure whether Azure Data Factory project is supported on Visual Studio 2017 at the moment. I have just installed VS 2017 but cannot open our solution as there is one azure data factory pr...

I am new to Azure Data Factory and have an interesting requirement. I need to move files from Azure Blob storage to Amazon S3, ideally using Azure Data Factory. However S3 isnt supported as a s...

I will give some context regarding our inconvenience in Azure Synapse: We created a Stored Procedure (it creates a VIEW which reads all the parquet files in a certain folder) on a Develop script, ...

The last step of my pipeline in Azure Data Factory executes another pipeline with the flow in which there is a Notebook step. As part of my requirement I need to capture the details of the error me...

I have an ADF that takes a Dataset input from Azure Data Lake Storage, it then has a pipeline with a custom .NET activity. The activity moves files from their Import folder into a custom folder loc...

I have some Excel files stored in SharePoint online. I want copy files stored in SharePoint folders to Azure Blob storage. To achieve this, I am creating a new pipeline in Azure Data factory usin...

When creating storage event trigger, path to the file that triggered the event is found in @triggerBody().folderPath. However, the path also contains container name. I would like to set my dataflow...

I am trying to select the last string after splitting it in Azure Data Factory. My file name looks like this: s = "cloudboxacademy/covid19/main/ecdc_data/hospital_admissions.csv" With Pyt...