How to move sharepoint list or excel file to azure sql dw?

Question

I want to copy data from sharepoint to microsoft azure sql DW using azure datafactory or alternative service. Can I do this. Please anyone help me with this.

Kalynkam

Answer 1

You can do this by setting up a data pipeline using Azure Data Factory to Azure blob storage. Afterwards you can use Azure's fast PolyBase technology to load the data from blob to your SQL Data Warehouse instance.

Can I ask how much data you intend on loading into the DW? Azure Data Warehouse is intended for use with at least terabyte level data up to petabyte compute and storage. I only ask because each SharePoint list or Excel file has a maximum of 2GB per file.

Impoverished

Thanks for your response. The list or excel file would not exceed 2GB. But I have a doubt like does azure data factory has support for accessing sharepoint files. Because I am not seeing any option while doing copy activity in azure. – Kalynkam

Can you expand more on what you're trying to do? Are you trying to set up an automatic process? If you're wondering whether CSV files can be put into DW with ease then you are correct. See here – Impoverished

Hi @hirokibutterfield ..could you please provide a link with example on how to transfer data from Sharepoint to Azure blob ? – Pare

Hi @user2713075..did you get a solution for this ? If so, could you please post it? – Pare

You would want to export the data to csv somehow. There are various methods online. I think I found one here that may be of use: codeproject.com/Articles/8411/…. From there, you can move the file to blob and then to DW. The biggest challenge will be finding out how to change the data to a flat file schema programmatically which is independent of DW as a service. – Impoverished