Call today 0845 643 6463

Data Warehouse

  • How to Handle Deleted Rows in an Incremental Load

    September 4th, 2026 in Big Data, Blog, Data Lakehouse, Data Warehouse, ETL, SQL, SQL Server, SSIS, Uncategorised

    Most data warehouse issues are apparent: a pipeline fails, an alert fires, you get a phone call at an unsociable hour. Deleted source data doesn’t do this. A row is removed from the source, and no one notices. It sits in the warehouse silently skewing measures and KPIs, until the finance director asks why a cancelled order is still in the revenue report. What’s causing this? Incremental loads use watermarks to avoid re-extracting…

    » Read more

  • SQL Server vs Fabric Warehouse: T-SQL gotchas nobody warns you about

    SQL Server vs Fabric Warehouse: T-SQL gotchas nobody warns you aboutJuly 23rd, 2026 in Blog, Data Warehouse, Fabric, Microsoft Fabric, SQL, SQL Server

    Fabric Warehouse looks just like SQL Server. It uses T-SQL, has schemas, stored procs and more. Yet, you migrate your first proper workload and see “that’s not supported” errors you never had to think about before. Fabric Warehouse isn’t SQL Server. Under the bonnet it’s storing everything as Delta Parquet in OneLake, and that storage format shapes what T-SQL can and can’t do. Here’s what actually trips…

    » Read more

  • Baking Up Better Data: Microsoft Fabric and the Fabric Toolbox

    May 8th, 2026 in Blog, Community, Data Factory, Data Warehouse, Fabric, Microsoft Fabric

    Recently, I presented a talk at the Birmingham Data User Group, titled “Stop Hand-Whisking Your Data: Faster Fabric Development with the fabric-toolbox”. On the back of that, I wanted to summarise the talk into a blog and share how you can use fabric-toolbox to enhance your Fabric development. Data engineering can feel abstract. Layers upon layers of transformation, movement, and logic that are hard to visualise at a glance. So, when talking…

    » Read more

  • Time Travel in Microsoft Fabric Warehouses

    May 7th, 2026 in Blog, Data Warehouse, Fabric, Warehouse

    Have you ever wished you could see what your data looked like last Tuesday? With time travel in Microsoft Fabric Warehouse, you can! Time travel in Microsoft Fabric Warehouse provides the ability to query your data exactly as it existed at any point within the last 30 days, without maintaining separate historical snapshots or copies. Fabric Warehouse is built on Delta Lake, which records every insert, update, and delete as a new version in the…

    » Read more

  • Why DuckDB Is the Data Engineer’s New Secret Weapon

    August 6th, 2025 in Blog, Data Warehouse, Python, SQL, Uncategorised

          Most databases need a server. DuckDB doesn’t. It’s a zero‑config, in‑process OLAP database you can run directly using Python. It’s fast, SQL‑friendly, and speaks Parquet like a native language — no clusters, no heavyweight setup, no waiting for provisioning. Think of DuckDB as SQLite for analytics — but faster, more column‑aware, and built for modern data engineering.   What Makes DuckDB Special DuckDB…

    » Read more

  • Overcoming Direct Lake Connection Hurdles

    January 7th, 2025 in Blog, Data Lakehouse, Data Warehouse, Fabric, Microsoft Fabric, PowerBI

    I love organising. When it comes to Microsoft Fabric, I am no different. The addition of files was a heaven-sent for me as I could finally clear up the clutter and make a Fabric solution user friendly to navigate. This allows my end users to access the correct information when they need it, quickly and accurately. Majority of the time, the end user does not need to access the Data Warehouse or Lakehouse that is storing their data, but rather the…

    » Read more

  • My Experience at Fabric Community Conference Europe 2024

    October 1st, 2024 in Blog, Data Lakehouse, Data Warehouse, Fabric, Power BI

    Stockholm: A city of charm and innovation Let’s start with the location, Stockholm. As a seasoned traveller, Stockholm is one of the few major European cities that I have not yet visited. So, when given the chance to go to FabCon Europe, I jumped at the chance. Stockholm is a great city, with beautiful historic architecture mixed with modern innovation and the people there were friendly and warm. The conference centre, the Stockholmsmässan,…

    » Read more

  • Power BI Dataset Gives Mashup Timeout Error While Refreshing

    July 26th, 2024 in Blog, Data Warehouse, Power BI, PowerBI, SQL, SQL Server, Uncategorised

    Help – my Power BI dataset is giving mashup timeout error on refresh The mashup timeout error Incremental refreshes are a great benefit to have in order to reduce refresh times – but not when they give a mashup timeout error. Although they are meant to be a performance enhancement, they aren’t always the best approach as they fire parallel queries to the source and apply date time filters. Despite the default timeout being 2 hours, or 5…

    » Read more

  • Ownership Made Easy: Microsoft Fabric Warehouse Takeovers

    June 19th, 2024 in Blog, Data Lakehouse, Data Warehouse, Fabric

    Fabric Warehouse Ownership Takeover As part of your project you have created and developed a Fabric Warehouse and now it is time to transfer the ownership of it away from your account to a production Service Principal Name or someone else in the organisation to maintain. There’s no simple method in the GUI so how do you do it? You use the Takeover API. Let’s go through the process step-by-step but beware, there is one caveat, you MUST…

    » Read more

  • Microsoft Fabric Mirroring: Quick Start Guide

    April 17th, 2024 in Blog, Data Factory, Data Warehouse, Fabric

    Mirroring provides a modern approach to accessing and ingesting data continuously and seamlessly from any database or data warehouse into the Data Warehousing experience within Microsoft Fabric. Here are the key points about Mirroring: Near Real-Time Data Access: Mirroring allows users to access changes in the source data almost instantly. It ensures that the data in Fabric’s OneLake remains up-to-date without complex setup or manual…

    » Read more