Blog

  • Fabric Data Warehouse – ALTER TABLE workaround

    You may be aware that the ALTER TABLE T-SQL command is currently not supported when working with data warehouses in Fabric. This isn’t much of a problem if the table you’re working with is empty, you can simply drop and recreate the table with the new structure. But if you do have data in a table and you want to make a change to the table, how can you do this without the ALTER TABLE command and without losing your data? My solution…

    » Read more
  • How to combine images using Python

    How to combine images using Python

    In this blog post, I will be showing how to combine images using Python. I used this code in a for loop to combine hundreds of different images for one of our clients. These are the two images that we will combine. Pillow is a PIL fork, install Pillow using pip and import PIL into the Python script. Next, set the two variables. These images are saved in the same folder as the script, therefore only the filenames are required to reference them.…

    » Read more
  • Power BI: Grouping & Binning using Data Groups

    In this blog, we explore Power BI’s Data Groups feature, enhancing data visualization and interaction for report users. Data Groups in Power BI: Power BI’s Data Groups efficiently organizes data. Compared to lengthier alternatives like grouping in Power Query or DAX Calculated Columns. Binning Data for Visualization: Let’s discuss data binning for clearer visuals. Binning categorizes data into bands or ranges. For instance in…

    » Read more
  • How are you doing? No really?

    Something that we at Purple Frog take very seriously is the mental health and well-being of the staff. The emphasis is very much on helping each other as a team. We all rely on each other in different ways and for different things which means we all need to keep an eye on and an eye out for each other. Various factors in the workplace can cause stress or increase the risk of developing or worsening existing mental health problems. These include…

    » Read more
  • Microsoft Fabric: Should I Use a Data Warehouse or Lakehouse?

    Fabric is the newest data analytics platform from Microsoft. It brings together the entire analytics process onto the Power BI service, meaning that you no longer need to be moving your data around from different services. The introduction of OneLake means that all data that you import whether from Cloud servers, On-Premises servers or more is available in one central location. With Fabric, you need to choose how you will store the data that you…

    » Read more
  • How do I make a multi-line string in Python (so I can ctrl+C my SQL code)

    This should be a quick one, but it is something that took me a little while to work out, but now I know I use it pretty much every day in my projects to save me a lot of time. This is the ability to copy and paste my SQL code into my python code as a multi-line string without having to reduce it to be all on one line or using escape characters. Both of which can take a bit of time depending on how big your SQL is. You usually need to have SQL…

    » Read more
  • Automate Power BI single table refresh using Azure Synapse Analytics and Power BI APIs

    In a blog I posted back in June 2022, I explained how you could trigger a Power BI dataset refresh via Azure Synapse Analytics. https://www.purplefrogsystems.com/2022/06/how-to-trigger-a-power-bi-dataset-refresh-via-azure-synapse-analytics-in-3-simple-steps/ However, what if you just wanted to refresh a select number of tables or objects rather than a full dataset? This might be the case if you are performing intraday loads in your ETL. There…

    » Read more
  • AI – Will robot overlords take over the world

    Everyone assumes that AI is the Hollywood version with robot overlords taking over the world. In reality, AI is already here and we are using it in society. Phone apps such as Uber using an AI to figure out the closest car to your destination, Netflix suggesting films for you to watch based on your history, Amazon prompting ‘customers also bought’ options on checkout, digital assistants like Alexa and Siri all of which are helping to…

    » Read more
  • Synapse Copy Activity Fails Over Certain File Size – ADF

    Copy Activity Issue in ADF / Synapse Analytics Recently, when trying to copy a .csv file from an FTP source to a Azure Data Lake using a Copy Activity in Azure Synapse, I had an issue where files > 16MB in size would fail. To overcome this, I took the first 13k rows and created another file from this, which resulted in a 4MB file. I tested this extra small file and it worked in the copy activity no issues. I multiplied these same 13k rows out…

    » Read more
  • What are Meta Threads? Should I be using them?

    In the ever-evolving landscape of social media, there have been some notable shifts and surprises (specifically in the last 48 hours). We’re about to dive into the world of Threads, where threads aren’t just for sewing anymore…   One of the most intriguing developments has been the transformation of Twitter since Elon Musk took the helm. With his ‘unconventional’ approach, the platform has experienced quite a rollercoaster…

    » Read more