Using SqlReaderQuery in Data Pipeline

Question

I'm copying data from an Azure SQL DB to blob by the means of a query.

Here is the script of the activity:

{
    "type": "Copy",
    "typeProperties": {
        "source": {
            "type": "SqlSource",
            "sqlReaderQuery": "select distinct a.*, b.Name from [dbo].[Transactxxxxxxx] a join dbo.Anxxxxx b on a.[Clixxxxx] = b.[Fixxxxxx] where b.Name = 'associations'"
        },
        "sink": {
            "type": "BlobSink",
            "writeBatchSize": 0,
            "writeBatchTimeout": "00:00:00"
        }
    },
    "inputs": [
        {
            "name": "Txnsxxxxxxxxxxx"
        }
    ],
    "outputs": [
        {
            "name": "Txnxxxxxxxxxxxx"
        }
    ],
    "policy": {
        "timeout": "01:00:00",
        "concurrency": 1,
        "retry": 3
    },
    "scheduler": {
        "frequency": "Hour",
        "interval": 1
    },
    "name": "Copyxxxxxxxxxx"
}

The activity seems to work but it does not put any file in the sink.

The dataset points to the correct container.

Yingqin · Accepted Answer · 2016-06-08 02:39:49Z

1

Per your provided information, I found successful run logs in our service. I noticed that the target blob is specified like "experimentinput/Inxxx_To_xx_Associations.csv/Inxxx_To_xx.csv". The blob name is static, and multiple slice runs will overwrite the same blob file. You could leverage the partitionBy property to have a dynamic blob name. Refer to this article for more details: https://azure.microsoft.com/en-us/documentation/articles/data-factory-azure-blob-connector/#azure-blob-dataset-type-properties.

answered Jun 8, 2016 at 2:39

Yingqin

2051 silver badge6 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Using SqlReaderQuery in Data Pipeline

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related