Join Us for an Insightful Talk on Distributed Training! 🚀

We are thrilled to host Suman Debnath, Technical Lead in Machine Learning at Anyscale, for a hands-on and engaging session on distributed training in modern AI.

Talk Overview:

As AI models become increasingly sophisticated, relying solely on single-GPU training is becoming obsolete. Distributed training is now a necessity! However, scaling these models brings forth unique challenges that necessitate a solid grasp on communication patterns, system bottlenecks, and essential trade-offs.

In this enlightening session, we will:

• Decode the principles of distributed training.
• Discuss why single-GPU training reaches its limits.
• Analyze how transformer models manage memory effectively.
• Explore techniques such as gradient accumulation, checkpointing, and data parallelism.
• Simplify key communication primitives and delve into ZeRO-1, ZeRO-2, ZeRO-3, and FSDP.
• Discover strategies to overlap compute and communication for optimized efficiency.
• Link these concepts to real-world tools found in frameworks like Ray and PyTorch.

Attendee Benefits:

Participants will walk away with a robust understanding of distributed training and practical knowledge on when to deploy various strategies.

About the Speaker:

Suman Debnath boasts a wealth of experience in large-scale distributed training, fine-tuning, and inference optimization in cloud environments. His expertise spans domains including Natural Language Processing, Large Language Models, and Retrieval-Augmented Generation.

Having presented at over a hundred conferences globally—such as PyCon, PyData, and ODSC—he has also developed performance benchmarking tools for distributed storage systems.

We look forward to welcoming you!

📅 Date & Time: October 10, 17:00 - 19:00
📍 Location: 101 Howard St, San Francisco, CA 94105-1616, United States
🔗 More Info: Event Link

Let’s set the stage for a future built on cutting-edge AI! 🌐
#DataScience #MachineLearning #DistributedTraining #Ray #PyTorch #LLM #RAG #DeepLearning #USFCA #USFMSDSAI #DataInstitute #AIEngineering #TechTalk