Azure Data Factory Cookbook: Build ETL, Hybrid ETL, and ELT pipelines using ADF, Synapse Analytics, Fabric and Databricks 2nd Edition

Azure Data Factory Cookbook: Build ETL, Hybrid ETL, and ELT pipelines using ADF, Synapse Analytics, Fabric and Databricks 2nd Edition book cover

Azure Data Factory Cookbook: Build ETL, Hybrid ETL, and ELT pipelines using ADF, Synapse Analytics, Fabric and Databricks 2nd Edition

Author(s): Dmitry Foshin (Author), Tonya Chernyshova (Author), Dmitry Anoshin (Author), Xenia Ireton (Author)

  • Publisher: Packt Publishing
  • Publication Date: 28 Feb. 2024
  • Edition: 2nd
  • Language: English
  • Print length: 532 pages
  • ISBN-10: 1803246596
  • ISBN-13: 9781803246598

Book Description

Solve real-world data problems and create data-driven workflows for easy data movement and processing at scale with Azure Data Factory

Key Features

  • Learn how to load and transform data from various sources, both on-premises and on cloud
  • Use Azure Data Factory’s visual environment to build and manage hybrid ETL pipelines
  • Discover how to prepare, transform, process, and enrich data to generate key insights

Book Description

This new edition of the Azure Data Factory Cookbook, fully updated to reflect ADS V2, will help you get up and running by showing you how to create and execute your first job in ADF.

You’ll learn how to branch and chain activities, create custom activities, and schedule pipelines, as well as discovering the benefits of Cloud Data Warehousing, Azure Synapse Analytics, and Azure Data Lake Storage Gen2.

With practical recipes, you’ll learn how to actively engage with analytical tools from Azure’s data services and leverage your on-premises infrastructure with cloud-native tools to get relevant business insights. As you advance, you’ll be able to integrate the most commonly used Azure services into ADF and understand how Azure services can be useful in designing ETL pipelines. You’ll familiarize yourself with the common errors that you may encounter while working with ADF and find out how to use the Azure portal to monitor pipelines. You’ll also understand error messages and resolve problems in Connectors and Data flows with the debugging capabilities of ADF.

Two new chapters covering Azure Data Explorer and key best practices have been added, along with new recipes throughout.

By the end of this book, you’ll be able to use ADF as the main ETL and orchestration tool for your Data Warehouse or Data Platform projects.

What you will learn

  • Create an orchestration and transformation job in ADF
  • Develop, execute, and monitor Data Flows using Azure Synapse Analytics
  • Create Big Data pipelines using Databricks and Delta tables
  • Work with Big Data in Azure Data Lake Storage Gen2 using Spark pools
  • Migrate on-premises SSIS jobs to ADF
  • Integrate ADF with commonly used Azure services such as Azure ML, Azure Logic Apps, and Azure Functions
  • Run big data compute jobs within HDInsight and Azure Databricks
  • Copy data from AWS S3 and Google Cloud Storage to Azure Storage using ADF’s built-in connectors

Who this book is for

This book is for ETL developers, data warehouse and ETL architects, software professionals, and anyone else who wants to learn about the common and not-so-common challenges faced while developing traditional and hybrid ETL solutions using Microsoft’s Azure Data Factory. You’ll also find this book useful if you are looking for recipes to improve or enhance your existing ETL pipelines. Basic knowledge of data warehousing is a prerequisite.

Table of Contents

  1. Getting Started with ADF
  2. Orchestration and Control Flow
  3. Setting up Synapse Analytics
  4. Working with Data Lake and Spark Pools
  5. Working with Big Data and Databricks
  6. Data Migration – Azure Data Factory and Other Cloud Services
  7. Extending Azure Data Factory with Logic Apps and Azure Functions
  8. Microsoft Fabric and Power BI, Azure ML and Cognitive Services
  9. Managing Deployment Processes with Azure DevOps
  10. Monitoring and Troubleshooting Data Pipelines
  11. Working with Azure Data Explorer
  12. The Best Practices of Working with ADF

Editorial Reviews

Review

“This book is for ETL developers, data warehouse and ETL architects, and any other professional looking to learn about the common challenges faced while developing ETL solutions using Azure Data Factory (ADF), Synapse Analytics, and Fabrics. The book comes with 12 chapters, including a chapter on setting up your ADF deployment process using CI/CD with Azure DevOps which is a trending topic.”

Elkhan Yusubov

Azure MVP

“I am committed to continuing to give my best and contributing to the success of our team and organization. Thank you once again for this recognition.

Overall, the book gives a good perspective to the Azure Data Factory together with labs, configurations and solution-driven chapters. With this book that’d be easy to have a better understanding of the environment and the needs that are going to be configured.”

Hamid

Microsoft MVP

“As someone interested in data management, I was thrilled to dive into the updated edition of the Azure Data Factory Cookbook. One of the main strengths of the book is its clear, step-by-step explanation of loading and transforming data from various sources using ADF. The readers will also learn how to branch and chain activities, create custom activities, and schedule pipelines. This book is a goldmine of practical solutions for real-world data problems and creating data-driven workflows.”

Jiadong Chen

Azure MVP

“I recently read the second edition of this book and found it to be a fairly comprehensive treatment on Azure Data Factory, Synapse Analytics, Spark pools, Databricks, and even unexpected topics such as Azure AI Services, Azure Machine Learning, and Fabric.

The book features many step-by-step instructions on accomplishing specific tasks and includes copious amounts of supporting screenshots. If you have a task you’re trying to accomplish and have not found a good reference on that task, this will be a great resource for you.”

Matt Eland

Azure MVP

“Definitely recommended! Really enjoyed reading and working through the recipes in Azure Data Factory Cookbook by Dmitry Foshin, Tonya Chernyshova, Dmitry Anoshin, and Xenia H. Being more of a Power BI and Power Query kind of guy, this book allowed me to get up to speed on ADF quickly. The recipes were clear, covering Synapse Analytics, Fabric, Data Lake, ML and Cognitive Services, and more! I also appreciated the treatment of CI/CD processes using Azure DevOps and the best practices chapter.”

Greg Deckler, Vice President, Fusion Alliance

“This book is an excellent resource whether you are new to Azure Data Factory or an experienced user. You will find lots of great step-by-step guides on topics ranging from getting started with ADF, right through to working with Databricks, working with other cloud solutions and CI/CD pipelines, and everything in between. […] I recommend this book to anyone who needs to work with Azure Data Factory. You can download the sample code and work through the many different samples. This book covers every scenario I can think of when working with ADF.”

Gregor Suttie, Azure MVP

About the Author

Dmitry Foshin is a business intelligence team leader, whose main goals are delivering business insights to the management team through data engineering, analytics, and visualization. He has led and executed complex full-stack BI solutions (from ETL processes to building DWH and reporting) using Azure technologies, Data Lake, Data Factory, Data Bricks, MS Office 365, PowerBI, and Tableau. He has also successfully launched numerous data analytics projects – both on-premises and cloud – that help achieve corporate goals in international FMCG companies, banking, and manufacturing industries.

Tonya Chernyshova is an experienced Data Engineer with over 10 years in the field, including time at Amazon. Specializing in Data Modeling, Automation, Cloud Computing (AWS and Azure), and Data Visualization, she has a strong track record of delivering scalable, maintainable data products. Her expertise drives data-driven insights and business growth, showcasing her proficiency in leveraging cloud technologies to enhance data capabilities.

Dmitry Anoshin is a data-centric technologist and a recognized expert in building and implementing big data and analytics solutions. He has a successful track record when it comes to implementing business and digital intelligence projects in numerous industries, including retail, finance, marketing, and e-commerce. Dmitry possesses in-depth knowledge of digital/business intelligence, ETL, data warehousing, and big data technologies. He has extensive experience in the data integration process and is proficient in using various data warehousing methodologies. Dmitry has constantly exceeded project expectations when he has worked in the financial, machine tool, and retail industries. He has completed a number of multinational full BI/DI solution life cycle implementation projects. With expertise in data modeling, Dmitry also has a background and business experience in multiple relation databases, OLAP systems, and NoSQL databases. He is also an active speaker at data conferences and helps people to adopt cloud analytics.

Xenia Ireton is a Senior Software Engineer at Microsoft. She has extensive knowledge in building distributed services, data pipelines and data warehouses.

View on Amazon

电子书代发PDF格式价格30我要求助
未经允许不得转载:Wow! eBook » Azure Data Factory Cookbook: Build ETL, Hybrid ETL, and ELT pipelines using ADF, Synapse Analytics, Fabric and Databricks 2nd Edition