Azure HDInsight on Ubuntu Linux brings Microsoft closer to hybrid cloud

The analytics system for Data Lake Store – named appropriately enough Apache Data Lake Analytics – stores data in the same HDFS format as Hadoop itself, but also allows data to be pulled in from other Azure sources, such as Azure SQL Database and Azure SQL Data Warehouse.

Microsoft has decided to make the offering available publicly following a wildly successful period of customer and partner adoption since its launch as a public preview at Strata + Hadoop World back in February. These three expansions will make Azure Data Lake more accessible and easier to use.

Azure Data Lake Analytics is a new service built on the Apache Yarn cluster management tool. We’ve faced the same problems inside Microsoft and that’s why we introduced, U-SQL, a new query language that unifies the ease of use of SQL with the expressive power of C#. Maybe, it will be this: Microsoft has selected Ubuntu for its first Linux-based software-as-a service offering, HDInsight, Microsoft’s big data Hadoop-on-Azure service. SQL’s still one of the most common ways to perform self-service data requests in Hadoop, so this extends that, akin to the way Microsoft juiced up T-SQL with the.Net CLR. The cloud-based big data analytics service has been available to Windows customers for some time and after it was unveiled as a public preview in February it is now fully available to Linux users. Millions of SQL and.NET developers can now process and analyze all of their data with the skills they already have. “Leading Hadoop applications that span security, governance, data preparation and analytics can be easily deployed from the Azure Marketplace on top of Azure Data Lake”, said Rengarajan.

“The analytics service for Azure Data Lake is cost-efficient because you only pay for your job when it is running, and support for Azure Active Directory lets you manage access and roles simply and integrates with your on-premises identity system”, he wrote on Microsoft’s SQL Server blog. Azure Data Lake Store helps set the stage for enterprise Internet of things (IoT) initiatives, according to Rengarajan. Microsoft says it is now on general release with a 99.9 per cent uptime service level agreement.

Microsoft’s Visual Studio Tools, meanwhile, have been updated to build, debug and tune for Hive queries and Storm topologies running in HDInsight.