Call today 0845 643 6463

Tag Archive: Blog

  • How to Handle Deleted Rows in an Incremental Load

    Python Dictionaries; what do I need to know?September 4th, 2026 in Big Data, Blog, Data Lakehouse, Data Warehouse, ETL, SQL, SQL Server, SSIS, Uncategorised

    Most data warehouse issues are apparent: a pipeline fails, an alert fires, you get a phone call at an unsociable hour. Deleted source data doesn’t do this. A row is removed from the source, and no one notices. It sits in the warehouse silently skewing measures and KPIs, until the finance director asks why a cancelled order is still in the revenue report. What’s causing this? Incremental loads use watermarks to avoid re-extracting…

    » Read more

  • Part 2: Using the Companies House API

    Part 2: Using the companies house API, by CallumJune 26th, 2026 in Blog, Business Intelligence, Jupyter, Python

    Part 2: Using the Companies House API Welcome back following part 1… Getting a list of companies via Python – Advanced Filtering. This example will look at the transport sector, leveraging the SIC codes in companies house. This can be altered accordingly to suit the business nature you are looking for. The libraries allows you to do more with Python, in this example that will included API calls, referencing the current date time and…

    » Read more

  • My Broken Query Runs Successfully in Synapse, Not That I’m Complaining, But How?

    My Broken Query Runs Successfully in Synapse, Not That I'm Complaining, But How?May 14th, 2026 in Blog, SQL, SQL Server, Synapse

    I had one of those moments this week, I spent too long trying to figure out how something that I knew was broken, was running fine and throwing no errors. Can it be called troubleshooting if it isn’t causing trouble? I have a Synapse pipeline that’s been running just fine for months, due to some migration work, I needed to take a look at the source query and found it was broken. I had it open in SSMS on the other monitor, ran it,…

    » Read more

  • Time Travel in Microsoft Fabric Warehouses

    May 7th, 2026 in Blog, Data Warehouse, Fabric, Warehouse

    Have you ever wished you could see what your data looked like last Tuesday? With time travel in Microsoft Fabric Warehouse, you can! Time travel in Microsoft Fabric Warehouse provides the ability to query your data exactly as it existed at any point within the last 30 days, without maintaining separate historical snapshots or copies. Fabric Warehouse is built on Delta Lake, which records every insert, update, and delete as a new version in the…

    » Read more

  • Businesses with purpose, not just profit – Jo Tutchener-Sharp’s incredible Scamp & Dude story

    Businesses with purpose, not just profit - Jo Tutchener-Sharp's incredible Scamp & Dude storyApril 17th, 2026 in Blog, Community, News, Purple Frog Systems

    Businesses with purpose, not just profit – Jo Tutchener-Sharp’s incredible Scamp & Dude story This morning’s FSB Big Networking event gave us something truly special. Alongside my co-host David Bell, I had the privilege of sitting down with Jo Tutchener-Sharp, founder of Scamp & Dude, a brand that proves a great business can also do extraordinary good in the world. Where it all began Scamp & Dude wasn’t born…

    » Read more

  • Incremental Refresh in Power BI

    January 23rd, 2026 in Blog, Power BI

    The aim of an incremental refresh is to reduce refresh time by avoiding unnecessary reprocessing of historical data. Rather than reloading an entire table on every refresh, Power BI refreshes only the most recent period where new or corrected records are likely to appear, while retaining older data that is unchanged. Incremental refresh is based on two time windows: Retention (historical data): how far back data is kept in the model. Refresh…

    » Read more

  • AI Warnings, Real Risks, and Why Businesses Should Not Panic

    January 19th, 2026 in AI, Artificial Intelligence, Blog

    You may have seen recent headlines quoting former Google CEO Eric Schmidt warning that future AI systems (https://www.thecrimson.com/article/2025/12/2/google-ceo-ai-self-improvement/) could become difficult for humans to fully understand or control. These comments reflect genuine long-term considerations around autonomy, speed and governance, but much of the media coverage amplifies them into dramatic worst-case scenarios. For most organisations…

    » Read more

  • Is ETL now a Circular Process?

    December 24th, 2025 in Blog, Business Intelligence, Data analytics, Microsoft Fabric, Power BI, PowerBI

    Traditionally, ETL (Extract, Transform, Load) has been a one-way journey. Data is extracted from operational systems, it’s transformed into a clean, structured format and then it’s loaded into a data store (typically a warehouse or lakehouse), rinse and repeat. But what’s after that? Business users build reports and dashboards and make a decisions from their insights, but the data itself never flows back into the systems where the work…

    » Read more

  • Back Up and Restore a Fabric Lakehouse

    Jon wearing a black and purple Purple Frog Systems polo shirt, standing in front of a graphic background featuring hexagonal patterns and an image of a lakeside house surrounded by trees. Text on the image reads: “Back Up and Restore a Fabric Lakehouse.”November 27th, 2025 in Backup, Blog, Fabric, Restore Point

    Fabric Lakehouses have been around for a while, but they still offer no native way to back them up or restore them. Fortunately, the way Lakehouses are structured makes it possible to build your own backup process. A Lakehouse is split into two areas: Files and Tables. The Files area is straightforward, you can download and back up these items directly. The Tables are powered by Delta tables, which are made up of two key components: the…

    » Read more

  • Underutilised Power BI Visuals (and When to Use Them)

    James wearing a black and purple Purple Frog Systems polo shirt, standing in front of a graphic background with hexagonal patterns and an image of hands working on a tablet. Text on the image reads: “Underutilised Power BI Visuals (and When to Use Them).October 3rd, 2025 in Blog, Microsoft Fabric, Power BI

    Introduction Power BI provides many built-in visuals, plus custom options from the marketplace. Yet many reports still rely heavily on bar, column, and line charts. While those are effective, a few underutilised visuals can help tell richer stories and surface deeper insights. This post highlights some of them, explains when they add value, and offers guidance on using them thoughtfully. Decomposition Tree What it does: Lets you dynamically…

    » Read more