Call today 0845 643 6463

Blog

  • Azure Data Factory Pricing – How much is my pipeline actually costing me?

    December 1st, 2022 in ADFv2, Azure, Blog, Data Factory

    Has a client ever asked you how much it actually costs to run a single pipeline in Azure Data Factory? Have you ever thought ADF pricing is just a black box? Well, hopefully my latest blog post will give you an indication on how you can start calculating the cost of a pipeline run! I will base my analysis on a sample pipeline containing the following activities as shown below: 1 x Lookup Activity (Pipeline Activity) 1 x Copy Data Activity (Data…

    » Read more

  • Part 3: Natural Language Processing – Sentiment Analysis and Opinion Mining

    November 24th, 2022 in AI, Azure, Big Data, Blog, Business Intelligence, Python, Synapse

    If you remember in part 2 we discussed what Key Word Analysis is and how this can be implemented to gain deeper insight from textual data. But we can go one step deeper and extract feelings and opinions from the same data. We can do this through Sentiment Analysis and Opinion mining! In this blog I will talk you through what they are and how we can implement them using Microsoft’s Cognitive Services. What is Sentiment Analysis? We should…

    » Read more

  • What are SQL Server Index Fragmentation and Index Fill Factor?

    November 9th, 2022 in Azure, Blog, SQL Server, Uncategorised

    What is index fragmentation and how does it occur? It is important to know that SQL Server data is stored in data pages, with each page holding 8KB of data. There are two types of fragmentation, both are a result of these pages not being used as efficiently as possible. When you UPDATE or INSERT data on a page that is already full, SQL Server creates a new page. The information from the original page will be split 50/50 with half being added to…

    » Read more

  • ADF Data flow string split

    October 28th, 2022 in Blog, Data Factory, Synapse

    This is a quick blog showing how to do a string split to get particular items in ADF data flows. Consider the following data where names and colours are combined into the FullName and Colours columns respectively. Note the delimiter for FullName is a space and the delimiter for Colours is a comma. To get each individual item and create new columns for this data use the split function in a Derived column transformation. The syntax for this…

    » Read more

  • Previous Row Value DAX

    October 18th, 2022 in Blog, Business Intelligence, DAX, Power BI

    In this quick blog post I’m going to show you how to use a hidden DAX function to get a previous row value with just a few lines of code. It’s that hidden that intellisense doesn’t even recognise it! I’ll use an example of some fictional race results from some well-known cartoon characters! Lets assume that in this scenario we wanted to analyse the time difference between 1st and 2nd place, 2nd and 3rd place etc in the race scenario…

    » Read more

  • Part 2 : Natural Language Processing- Key Word Analysis

    August 10th, 2022 in AI, Azure, Big Data, Blog, Machine Learning, Python, Uncategorized

    Here we are with part 2 of this blog series on web scraping and natural language processing (NLP). In the first part I discussed what web scraping was, why it’s done and how it can be done. In this part I will give you details on what NLP is at a high level, and then go into detail of an application of NLP called key word analysis (KWA). What is NLP? NLP is a form of artificial intelligence which deals with the interactions between humans…

    » Read more

  • Query Store Forced Plan Failures

    July 28th, 2022 in Azure, Blog, SQL Server, Uncategorized

    Query Store is a fantastic feature of both SQL Server and Azure SQL DB. It allows you to monitor how queries execute against the database which is invaluable for troubleshooting performance issues. More than that though, it gives you the option to force an erratic query to use a particular execution plan, this helps avoid queries from running with inefficient plans and provides predictability and stability on the server. Here we can see an…

    » Read more

  • Part 1: Web Scraping and Natural Language Processing- Web Scraping

    July 18th, 2022 in AI, Blog, Machine Learning, Python

    In this multi blog series I will go through what web scraping is, what Natural Language processing is as a general term as well as diving into some constituent techniques we are interested in; key word extraction, sentiment analysis and its derivative opinion mining. The last few parts will then go through a coded example of scraping the popular review site Trust pilot for reviews of the popular supermarket chain ‘Lidl’. We will then…

    » Read more

  • ADF breaking out of a ForEach activity

    July 14th, 2022 in Azure, Blog, Data Factory, Synapse

    Currently, the ForEach activity in ADF (or Synapse) does not break out when an inner activity fails. There are several ways you can force the ForEach to break out, the most common is to cancel the pipeline run when an inner activity fails and there are already many blogs out there that cover this. My requirement is slightly different and that is what I will show here. Consider a scenario where you have a ForEach activity which has multiple inner…

    » Read more

  • Upload Power BI Reports using PowerShell

    July 8th, 2022 in Azure, Blog, Uncategorized

    In this blog post, I will be sharing a PowerShell script that allows multiple Power BI reports to be uploaded at once. In a previous blogpost, I shared a PowerShell script that allowed users to downloaded multiple Power BI reports. Combined you could move several reports from one workspace to another in a few seconds. The script is downloadable at the bottom of the page as a txt file. To use the script there are three steps to take. The first…

    » Read more