ADFv2

  • Synapse Script Activity Error “Argument {0} is null or empty.\r\nParameter name: paraKey”

    Came across this issue when trying to get a range of values that are calculated dynamically at runtime. Using the script activity in ADF, I wanted a result set that I could reference as parameters for a notebook later in the pipeline but was experiencing an error I had not seen before referencing null arguments. The Problem The script is simple – get a few key dates from the last few months and set these as variables. Finally, return these…

    » Read more
  • Azure Data Factory Pricing – How much is my pipeline actually costing me?

    Has a client ever asked you how much it actually costs to run a single pipeline in Azure Data Factory? Have you ever thought ADF pricing is just a black box? Well, hopefully my latest blog post will give you an indication on how you can start calculating the cost of a pipeline run! I will base my analysis on a sample pipeline containing the following activities as shown below: 1 x Lookup Activity (Pipeline Activity) 1 x Copy Data Activity (Data…

    » Read more
  • Azure SSIS – How to Setup, Deploy, Execute & Schedule Packages

    Welcome back to work in 2018! 🙂 Let’s get stuck in with a hot topic. How do we actually use our beloved SQL Server Integration Services (SSIS) packages in Azure with all this new platform as a service (PaaS) stuff? Well, in this post I’m going to go through it end to end. Post Contents Azure Services Setup Azure Data Factory v2 Azure SQL Instance Azure SSIS IR Creating & Deploying the SSIS Package SSIS Azure Feature Pack…

    » Read more
  • What’s New in Azure Data Factory Version 2 (ADFv2)

    I’m sure for most cloud data wranglers the release of Azure Data Factory Version 2 has been long overdue. Well good news friends. It’s here! So, what new features does the service now offer for handling our Azure data solutions?… In short, loads! In this post, I’ll try and give you an overview of what’s new and what to expect from ADFv2. However, I’m sure more questions than answers will be raised here. As developers we…

    » Read more