Question

Is there some way to do just a simple string manipulation in Azure Data Factory?

Something as simple as I have a storage blob with a tab-separated file, and I want to move it over into a storage table ... but make some

And I want to say, convert the tabs to commas, merge columns 4 through to the last column

Answer 1

Basically, Azure Data Factory is meant for automating data movement and data transformation. When we talk about data transformation, it comes under the concept of Azure Data Lake Analytics where we use a software called USQL or any other for the transformation of data. I just mentioned the sample file below:

 

DECLARE @inputFilepath string = "input/input67.tsv";

DECLARE @outputFilepath string = "output/output67.csv";

@input =

    EXTRACT rowId int,

            col1 int,

            col2 int,

            col3 int,

            col4 int

    FROM @inputFilepath

    USING Extractors.Tsv(skipFirstNRows : 1);

// Concat the four columns

@output =

    SELECT rowId,

           string.Concat(col1.ToString(), col2.ToString(), col3.ToString(), col4.ToString()) AS col5

    FROM @input;

// Export as csv

OUTPUT @output

TO @outputFilepath

USING Outputters.Csv(quoting:false);