- Presencial
- 21 horas
- Aula Virtual
- No disponible
DeepSpeed is a deep learning optimization library that makes it easier to scale deep learning models on distributed hardware. Developed by Microsoft, DeepSpeed integrates with PyTorch to provide better scaling, faster training, and improved resource utilization.
This instructor-led, live training (online or onsite) is aimed at beginner to intermediate-level data scientists and machine learning engineers who wish to improve the performance of their deep learning models.
By the end of this training, participants will be able to:
- Understand the principles of distributed deep learning.
- Install and configure DeepSpeed.
- Scale deep learning models on distributed hardware using DeepSpeed.
- Implement and experiment with DeepSpeed features for optimization and memory efficiency.
Introduction
- Overview of deep learning scaling challenges
- Overview of DeepSpeed and its features
- DeepSpeed vs. other distributed deep learning libraries
Getting Started
- Setting up the development environment
- Installing PyTorch and DeepSpeed
- Configuring DeepSpeed for distributed training
DeepSpeed Optimization Features
- DeepSpeed training pipeline
- ZeRO (memory optimization)
- Activation checkpointing
- Gradient checkpointing
- Pipeline parallelism
Scaling Models with DeepSpeed
- Basic scaling using DeepSpeed
- Advanced scaling techniques
- Performance considerations and best practices
- Debugging and troubleshooting techniques
Advanced DeepSpeed Topics
- Advanced optimization techniques
- Using DeepSpeed with mixed precision training
- DeepSpeed on different hardware (e.g. GPUs, TPUs)
- DeepSpeed with multiple training nodes
Integrating DeepSpeed with PyTorch
- Integrating DeepSpeed with PyTorch workflows
- Using DeepSpeed with PyTorch Lightning
Troubleshooting
- Debugging common DeepSpeed issues
- Monitoring and logging
Summary and Next Steps
- Recap of key concepts and features
- Best practices for using DeepSpeed in production
- Further resources for learning more about DeepSpeed
Otros cursos bonificables de Nobleprog:
Nosotros nos encargamos de gestionar la bonificación de estos cursos. Para información sobre la inscripción, CONTACTA CON EL CENTRO.
