Deep Leaing at Scale: At the Intersection of Hardware, Software, and Data

by: Suneeta Mall (Author)

Publisher: O’Reilly Media

Edition: 1st

Publication Date: 2024/7/23

Language: English

Print Length: 448 pages

ISBN-10: 1098145283

ISBN-13: 9781098145286

Book Description

Bringing a deep-leaing project into production at scale is quite challenging. To successfully scale your project, a foundational understanding of full stack deep leaing, including the knowledge that lies at the intersection of hardware, software, data, and algorithms, is required. This book illustrates complex concepts of full stack deep leaing and reinforces them through hands-on exercises to arm you with tools and techniques to scale your project. A scaling effort is only beneficial when it’s effective and efficient. To that end, this guide explains the intricate concepts and techniques that will help you scale effectively and efficiently. You’ll gain a thorough understanding of: How data flows through the deep-leaing network and the role the computation graphs play in building your model How accelerated computing speeds up your training and how best you can utilize the resources at your disposal How to train your model using distributed training paradigms, i.e., data, model, and pipeline parallelism How to leverage PyTorch ecosystems in conjunction with NVIDIA libraries and Triton to scale your model training Debugging, monitoring, and investigating the undesirable bottlenecks that slow down your model training How to expedite the training lifecycle and streamline your feedback loop to iterate model development A set of data tricks and techniques and how to apply them to scale your training model How to select the right tools and techniques for your deep-leaing project Options for managing the compute infrastructure when running at scale

About the Author

Bringing a deep-leaing project into production at scale is quite challenging. To successfully scale your project, a foundational understanding of full stack deep leaing, including the knowledge that lies at the intersection of hardware, software, data, and algorithms, is required. This book illustrates complex concepts of full stack deep leaing and reinforces them through hands-on exercises to arm you with tools and techniques to scale your project. A scaling effort is only beneficial when it’s effective and efficient. To that end, this guide explains the intricate concepts and techniques that will help you scale effectively and efficiently. You’ll gain a thorough understanding of: How data flows through the deep-leaing network and the role the computation graphs play in building your model How accelerated computing speeds up your training and how best you can utilize the resources at your disposal How to train your model using distributed training paradigms, i.e., data, model, and pipeline parallelism How to leverage PyTorch ecosystems in conjunction with NVIDIA libraries and Triton to scale your model training Debugging, monitoring, and investigating the undesirable bottlenecks that slow down your model training How to expedite the training lifecycle and streamline your feedback loop to iterate model development A set of data tricks and techniques and how to apply them to scale your training model How to select the right tools and techniques for your deep-leaing project Options for managing the compute infrastructure when running at scale