March was a big month for Microsoft Fabric and its users as there were two major conferences discussing all aspects of Microsoft Fabric, from functionality to security and governance. SQLBits, a community conference, came first with a wealth of knowledge from experts using Microsoft Fabric from a beginner to experienced level. FabCon immediately followed, which was a conference from Microsoft to announce new features and to hear from those closest to the inner workings of the technology.

In a previous blog post, I discussed some of the features Microsoft had set out for release on their roadmap. Now that we have reached the end of Q1 in 2024, Microsoft have released a whole new set of features and enhancements to extend the Microsoft Fabric platform experience.

SQLBits:

Purple Frog Team at SQLBits

Purple Frog Team at SQLBits!

The entire Purple Frog team were able to attend SQLBits this year, and it was an amazing opportunity to learn something new and enhance my existing knowledge. As a community conference, there were so many talks to attend and a real variety of topics – but here are some of the most popular and exciting areas spoken about:

Real-Time Analytics

The general buzz in the room whenever there was a demonstration of Real-Time Analytics made it clear that this will be a game-changing and heavily adopted feature of Microsoft Fabric. The ability to query and transform live data as it comes into a KQL database, and then to be able to report on this data as it comes in, with refreshes as frequent as every second!

Real-time data, combined with a Power BI dashboard, has the ability to provide powerful reporting and insights to many business departments across many industries. These include:

  • Live sales reporting
  • Marketing campaign response
  • Order & Shipment tracking map
  • Tracking the ISS

Data Governance & Security

Data Governance was a key area discussed at SQLBits as it is very important for businesses to do right. Governance is important in your Microsoft Fabric estate to monitor and manage your item, workspace and capacity security. Using features such as permissions, sensitivity labels and RLS/OLS allow you to create a secure data storage on OneLake, where the right level of access is given to each person.

There are multiple tools that can be used with Microsoft Fabric to increase your security and enable you to govern data. Microsoft Purview is a built-in tool to help audit assets within the tenant on Microsoft Fabric; Checking data lineage, access and usage. Power BI Sentinel is external tool that has been developed to fully aid security and governance for a whole Microsoft Fabric capacity, giving you in depth Power BI dashboards to monitor lineage, access and usage. Using these features can help you reduce your costs by preventing data loss and leaks, and also monitor where redundant resources are using up compute.

Data Architectures

Since day one of Microsoft Fabric, there has been the question of Data warehouse vs Data lakehouse. Speakers at SQLBits had sessions on comparing different data architectures, deep dives and some on the evolution of the data world. As Microsoft Fabric expands more, and is adopted by more businesses, best practices are becoming known, and where better than from a community of experts to learn this from!

FabCon:

FabCon is a Microsoft hosted conference, where product managers and industry professionals speak about the platform. Here are some of the new features announce to mark the end of Q1 2024:

Workspace Folders

It’s finally here! Say goodbye to a workspace with all items in one big list – now you can organise your items to easier access them. Folders can be nested as deep as 10 subfolders and folder structure will be maintained when using deployment pipelines.

Mirroring

Mirrored Databases are now in Public preview. Using a mirrored DB, you can create a sychronised replication of an Azure SQL, Snowflake or Cosmos DB into a data warehouse.

On-Prem Gateways for Data Pipelines

Gateways for on-prem data are now available in the data connections panel in Microsoft Fabric. Data from the gateway can be imported using either pipelines or dataflows Gen2. For a guide on setting up your on-prem gateway, check out this blog from Joe!

AutoML and Hyperparameter Tuning

As part of the Data Science experience, code-first AutoML has been included to notebooks. Allowing you to pass prepared data into a trial that with run various different ML models on the data to find the best fit and outcomes. A guide on how to set up an ML trial can be found here.

Other enhancements such as…

  • Pipeline activity limit increased to 80
  • Pipeline activity to refresh semantic models
  • Ability to cancel a dataflow when it runs

 

If you have any other features that you want covering, or want advice on setting up for your business, feel free to reach out to Purple Frog Systems and speak to one of our experts!

Tags: , , ,