Azure

  • ADF: Where have my connections gone?

    If you’ve been doing any development work in ADF this week you might have noticed that “Connections” has moved. But where has it gone? When you click onto “Connections” now you’ll receive the following message: Clicking this button takes you to the new Management Hub, the new fourth icon which goes alongside the existing three. Within this area you now have access to your Linked Services, Integration Runtimes…

    » Read more
  • Azure Data Factory (ADF) Parent Activities don’t trigger ON ERROR OUTPUT when Child Activities Error… What happens?

    So that title is a big question that a client of ours came across that required some additional investigation… What does happen and how do we get around it? In the below example we’ll use a simple IF Activity as our Parent Activity. To explain briefly how it functions, you provide it with a query that evaluates to either True of False. This then passes down to a set of Activities If True and a different set of Activities If False. We…

    » Read more
  • SSAS Tabular Deployment Wizard fails Newtonsoft.Json error

    The Scenario Deploying an Analysis Services Tabular model to SSAS Azure using the Analysis Services Deployment Wizard. Both Visual Studio 2017 & SQL Server 2017 installed on the client. Try and click on the ellipses to change the data source connection string or impersonation information results in a Newtonsoft.json error: “Could not load file or assembly ‘Newtonsoft.Json, Version 6.0.0.0, Culture=neutral, ……”…

    » Read more
  • Lookups and If Conditions in Azure Data Factory v2 (ADFv2)

    Azure Data Factory v2 (ADFv2) has some significant improvements over v1, and we now consider ADF as a viable platform for most of our cloud based projects. But things aren’t always as straightforward as they could be. I’m sure this will improve over time, but don’t let that stop you from getting started now. This post provides a walk through of using the ‘Lookup’ and ‘If Condition’ activities to do some…

    » Read more
  • Azure SSIS – How to Setup, Deploy, Execute & Schedule Packages

    Welcome back to work in 2018! 🙂 Let’s get stuck in with a hot topic. How do we actually use our beloved SQL Server Integration Services (SSIS) packages in Azure with all this new platform as a service (PaaS) stuff? Well, in this post I’m going to go through it end to end. Post Contents Azure Services Setup Azure Data Factory v2 Azure SQL Instance Azure SSIS IR Creating & Deploying the SSIS Package SSIS Azure Feature Pack…

    » Read more
  • Business Intelligence in Azure – SQLBits 2018 Precon

    What can you expect from my SQLBits pre conference training day in February 2018 at the London Olympia? Well my friends, in short, we are going to take whirlwind tour of the entire business intelligence stack of services in Azure. No stone will be left unturned. No service will be left without scalability. We’ll cover them all and we certainly aren’t going to check with the Azure bill payer before turning up the compute on our data…

    » Read more
  • RDP to Azure Batch Service Compute Nodes

    Did you know it’s now possible to RDP to your Azure Batch Service compute nodes? I’ve used the batch service to handle the compute for my Azure Data Factory custom activities for a while now. Which I’ve basically been doing blindly because the code execution and logging is provided to ADF, with no visibility to the underlying pool of VM’s doing the work. Well, no more is this the case! In the Azure portal go to your Batch…

    » Read more
  • Azure Data Lake – The Services. The U-SQL. The C# (Reference Guide)

    This post is a reference guide to support an event talk or webinar. The content is intended to assist the audience only. Thank you. Abstract How do we implement Azure Data Lake? How does a lake fit into our data platform architecture? Is Data Lake going to run in isolation or be part of a larger pipeline? How do we use and work with USQL? Does size matter?! The answers to all these questions and more in this session as we immerse ourselves in…

    » Read more
  • What’s New in Azure Data Factory Version 2 (ADFv2)

    I’m sure for most cloud data wranglers the release of Azure Data Factory Version 2 has been long overdue. Well good news friends. It’s here! So, what new features does the service now offer for handling our Azure data solutions?… In short, loads! In this post, I’ll try and give you an overview of what’s new and what to expect from ADFv2. However, I’m sure more questions than answers will be raised here. As developers we…

    » Read more
  • Chaining Azure Data Factory Activities and Datasets

    As I work with Azure Data Factory (ADF) and help others in the community more and more I encounter some confusion that seems to exist surrounding how to construct a complete dependency driven ADF solution. One that chains multiple executions and handles all of your requirements. In this post I hope to address some of that confusion and will allude to some emerging best practices for Azure Data Factory usage. First a few simple questions: Why is…

    » Read more