This post is a reference guide to support an event talk or webinar. The content is intended to assist the audience only. Thank you.
Abstract
How do we implement Azure Data Lake? How does a lake fit into our data platform architecture? Is Data Lake going to run in isolation or be part of a larger pipeline? How do we use and work with USQL? Does size matter?! The answers to all these questions and more in this session as we immerse ourselves in the lake, that’s in a cloud. We’ll take an end to end look at the components and understand why the compute and storage are separate services. For the developers, what tools should we be using and where should we deploy our USQL scripts. Also, what options are available for handling our C# code behind and supporting assemblies. We’ll cover everything you need to know to get started developing data solutions with Azure Data Lake. Finally, let’s extend the U-SQL capabilities with the Microsoft Cognitive Services!
Links
What is Azure Data Lake? The Microsoft version.
https://azure.microsoft.com/en-gb/solutions/data-lake/
Understanding the ADL Analytics Unit
https://blogs.msdn.microsoft.com/azuredatalake/2016/10/12/understanding-adl-analytics-unit/
Why use Azure Data Lake? The Microsoft version.
https://azure.microsoft.com/en-gb/solutions/data-lake/
Comsuming Data Lake with Power – Cross tenant data refreshes.
https://www.purplefrogsystems.com/2017/06/connecting-power-bi-to-azure-data-lake-store-across-tenants/
U-SQL String Data Type 128KB Limit
https://feedback.azure.com/forums/327234-data-lake/suggestions/13416093-usql-string-data-type-has-a-size-limit-of-128kb
Creating a U-SQL Merge Statement
https://www.purplefrogsystems.com/2016/12/writing-a-u-sql-merge-statement/
U-SQL Looping
https://www.purplefrogsystems.com/2017/05/recursive-u-sql-with-powershell-u-sql-looping/
U-SQL Date Dimension
https://www.purplefrogsystems.com/2017/02/creating-a-u-sql-date-dimension-numbers-table-in-azure-data-lake/
Further Reading
Microsoft Blog – An Introduction to U-SQL in Azure Data Lake
https://blogs.msdn.microsoft.com/robinlester/2016/01/04/an-introduction-to-u-sql-in-azure-data-lake/
Microsoft Documentation – U-SQL Programmability Guide
https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-programmability-guide
Microsoft MSDN – U-SQL Language Reference
https://msdn.microsoft.com/en-US/library/azure/mt591959(Azure.100).aspx
SQL Server Central – Stairway to U-SQL
http://www.sqlservercentral.com/stairway/142480/
Stack Overflow – U-SQL Tag
http://stackoverflow.com/questions/tagged/u-sql
Cognitive services with U-SQL in Azure Data Lake
https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-cognitive
