Explore Courses Blog Tutorials Interview Questions
0 votes
in Azure by (17.6k points)

I have a metadata activity to get the files in one of my azure data factory pipeline and its linked to a data lake. Is there any method available in the Azure data factory for sorting the available files based on the file name in the metadata activity?

Sample output for the Metadata activity is given below

"childitems" :[ { "name":"File_20200101.csv", "type": "File" }, { "name":"File_20200501.csv", "type": "File" }, { "name":"File_20200301.csv", "type": "File" }, { "name":"File_20200201.csv", "type": "File" } ]

I need to get the files in the below-given order.

"childitems" :[ { "name":"File_20200101.csv", "type": "File" }, { "name":"File_20200201.csv", "type": "File" }, { "name":"File_20200301.csv", "type": "File" }, { "name":"File_20200501.csv", "type": "File" } ]

1 Answer

0 votes
by (47.2k points)
edited by

We can use SQL server table to store the array values and then used a lookup activity with a order by file name query inside another loop to get the sorted filenames. This helped me to solve the sorting problem

You can check out Azure Data Factory Course which will help you to upgrade your skills in Azure Data Factory.

Browse Categories