Azure Data Lake Analytics, is a powerful on-demand analytics job service on Microsoft Azure. By reading this article, you can learn Azure Data Analytics by Example.
The Example
Since this is an introductory article on Azure Data Lake Analytics, the example that will be presented, will be a simple one just to showcase the technology. The rest is left to your imagination and creativity!
More specifically, in this simple example, I will create and submit for execution 2 jobs:
Job 1
Create a sample database and table
Populate the table with sample data
Job 2
Query the sample table and extract the data into a .csv file
Learn more about Azure SQL Database: Enroll to our Online Course!
The first step, is to create an Azure Data Lake Analytics account, in case you don’t already have one.
You can easily do this, by creating a new resource – Data Lake Analytics:
Then, you will need to specify some details, as shown in the below screenshot:
Right after the above steps, within a few minutes, the Azure Data Lake Analytics account is ready for use:
Step 2: Configure firewall
The next step is to configure the Firewall rules for the newly created Azure Data Lake Analytics account. In my example, I have just added the IP of my client.
Creating and Submitting Jobs on Azure Data Lake Analytics
Now that the Azure Data Lake Analytics account is ready to accept jobs for execution, let’s create and submit a new job.
Define First Job
As mentioned earlier, in this example I will create and submit for execution 2 simple jobs.
The first job is to create a sample database and table, and then populate it with sample data.
So, in order to create the job, within the Azure Data Lake Analytics account, you click on the link “New job”.
Then, in this example, I used U-SQL, in order to define the job:
Submit First Job for Execution
After submitting the job for execution, I can see that it was successfully executed.
Below, you can see the relevant job graph that illustrates the job’s execution steps, as created by the powerful engine behind Azure Data Lake Analytics.
Learn all about MySQL on Azure – Enroll to our Online Course!
Enroll to our online course on Udemy titled “Introduction to Azure Database for MySQL” and get lifetime access to high-quality lessons and hands-on guides about all the aspects of MySQL database service on the Azure cloud platform!
(Lifetime Access, live guides, certificate of completion, downloadable resources and more!)
Moreover, in the “AU analysis” tab, you can get more statistics and shortcuts to further optimize your job (i.e. for performance, balanced execution, etc.).
Data Explorer
Now, let’s check also via Data Explorer, to see our table:
Define Second Job
Now, let’s define the second which will extract the data from the sample table into a .csv file.
This time, I have created the job right from within Data Explorer:
Submit Second Job for Execution
After submitting the second job for execution, again, I can see that it was successfully executed.
Below, you can see the relevant job graph that illustrates the job’s execution steps:
Check the Second Job’s Outcome
Let’s check the second job’s outcome. As you can see in the below screenshot, the .csv file has been successfully created, and we can check its contents:
Job Insights
When we click on “Job insights”, we can see useful information and statistics about our jobs executed in Data Lake Analytics.
Azure Data Lake Tools
Azure Data Lake Analytics, integrate with many tools that you can use to create and submit different jobs.
For example, if you click on the “Tools” link, you will easily get started with the below tools:
Data Lake Tools for Visual Studio
Data Lake Tools for Visual Studio Core
Azure PowerShell
Azure CLI
Conclusion
Azure Data Lake Analytics, is a powerful engine, which allows you to create and execute heavy jobs on Microsoft Azure. Depending on the job type, Azure Data Lake Analytics automatically scale, thus making efficient use of its powerful engine, in order to execute the job. Moreover, it provides in-depth insights and statistics, which help you assess each job’s execution, or even change how each jobs should execute.
With Azure Data Lake Analytics, you can process any data (relational, unstructured, big data, and more) through its job system fast and efficiently.
See More!
Learn more about Azure SQL Database and get started fast and easy! Through our online course, “Introduction to Azure SQL Database for Beginners“, you will get lifetime access to a comprehensive set of lectures and live demonstrations that will get you started with Azure SQL Database in no time! The course is regularly updated with additional educational material!
(Lifetime Access, Certificate of Completion and more!)
Artemakis Artemiou is a seasoned Senior Database and AI/Automation Architect with over 20 years of expertise in the IT industry. As a Certified Database, Cloud, and AI professional, he has been recognized as a thought leader, earning the prestigious Microsoft Data Platform MVP title for nine consecutive years (2009-2018). Driven by a passion for simplifying complex topics, Artemakis shares his expertise through articles, online courses, and speaking engagements. He empowers professionals around the globe to excel in Databases, Cloud, AI, Automation, and Software Development. Committed to innovation and education, Artemakis strives to make technology accessible and impactful for everyone.
Views:6,526
We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent. Read More
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
Cookie
Duration
Description
cookielawinfo-checkbox-analytics
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional
11 months
The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance
11 months
This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy
11 months
The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.