Tag Archive: Data Warehouse

  • Connecting Your Fabric Data Warehouse to Spark Notebooks

    When using Spark notebooks in Fabric, it is very easy to connect directly to a Lakehouse. From here, you can see all of the data that is stored in tables and click and drop to insert code snippets to extract the data. Here at Purple Frog, it is common to have projects using Data Warehouses as their data store. When looking at ways to connect Notebooks to a warehouse, there is currently ¬†little information available online; One way that is…

    » Read more
  • Fabric Data Warehouse – ALTER TABLE workaround

    You may be aware that the ALTER TABLE T-SQL command is currently not supported when working with data warehouses in Fabric. This isn’t much of a problem if the table you’re working with is empty, you can simply drop and recreate the table with the new structure. But if you do have data in a table and you want to make a change to the table, how can you do this without the ALTER TABLE command and without losing your data? My solution…

    » Read more
  • Microsoft Fabric: Should I Use a Data Warehouse or Lakehouse?

    Fabric is the newest data analytics platform from Microsoft. It brings together the entire analytics process onto the Power BI service, meaning that you no longer need to be moving your data around from different services. The introduction of OneLake means that all data that you import whether from Cloud servers, On-Premises servers or more is available in one central location. With Fabric, you need to choose how you will store the data that you…

    » Read more
  • SQLBits X Video available

    SQLBits X Video Now available The video of my talk at SQLBits X is now available on the SQLBits website here. The talk was focused on presenting the results of my MSc Business Intelligence dissertation, comparing the performance of different methods of using SSIS to load data warehouse dimensions, specifically type 2 SCDs. The talk also covers a comparison of the performance between traditional hard disks and solid state storage systems such as…

    » Read more
  • Automating T-SQL Merge to load Dimensions (SCD)

    This is the 3rd post in the Frog-Blog series on the awesomeness of T-SQL Merge. Post 1: Introduction to T-SQL merge basics Post 2: Using T-SQL merge to load data warehouse dimensions In this post we’ll be looking at how we can automate the creation of the merge statement to reduce development time and improve reliability and flexibility of the ETL process. I discussed this in the 2nd half of a talk I gave at the UK technical launch of SQL…

    » Read more
  • Using T-SQL Merge to load Data Warehouse dimensions

    In my last blog post I showed the basic concepts of using the T-SQL Merge statement, available in SQL Server 2008 onwards. In this post we’ll take it a step further and show how we can use it for loading data warehouse dimensions, and managing the SCD (slowly changing dimension) process. Before we start, let’s have a quick catch up on what an SCD is… What is a Slowly Changing Dimension (SCD)? If you want a full explanation of…

    » Read more
  • Data Warehousing: Kimball vs Inmon

    Anyone involved in the Business Intelligence space has had their head in the sand if they are not aware of the long running, and more often than not misunderstood, debate between the two conceptual models of data warehouse design. Bill Inmon has recently posted an article on www.b-eye-network.com discussing the matter, and to his credit, has tried to put forward a number of balanced pros and cons of each methodology. I’ll state now that…

    » Read more
  • SQL User Group Session 24 June 2010

    I’m excited to be presenting another session to the South Wales SQL Server User Group. On Thursday 24th June 2010, Eversheds in Cardiff are kindly hosting the event, to run from 18:45 to 21:00. The event is free, and you’ll even get pizza thrown in – what more can you ask for? Oh yes, some BI content… My session will cover data warehouse modelling, including a number of hands on business case studies including…

    » Read more