Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Core Performance Concepts and Metrics
- Latency, throughput, power consumption, and resource utilization
- Distinguishing between system and model-level bottlenecks
- Profiling techniques for inference versus training
Profiling on Huawei Ascend
- Leveraging CANN Profiler and MindInsight
- Diagnosing kernels and operators
- Understanding offload patterns and memory mapping
Profiling on Biren GPU
- Exploring Biren SDK performance monitoring features
- Kernel fusion, memory alignment, and execution queue management
- Profiling with awareness of power and thermal constraints
Profiling on Cambricon MLU
- Utilizing BANGPy and Neuware performance tools
- Interpreting kernel-level visibility and logs
- Integrating MLU profilers with deployment frameworks
Optimization at the Graph and Model Level
- Strategies for graph pruning and quantization
- Operator fusion and restructuring computational graphs
- Standardizing input sizes and tuning batch parameters
Memory and Kernel Optimization
- Improving memory layout and data reuse
- Managing buffers efficiently across different chipsets
- Platform-specific kernel-level tuning methods
Best Practices for Cross-Platform Environments
- Ensuring performance portability through abstraction strategies
- Developing shared tuning pipelines for multi-chip setups
- Case study: tuning an object detection model across Ascend, Biren, and MLU
Wrap-Up and Future Directions
Requirements
- Practical experience with AI model training or deployment pipelines
- A solid grasp of GPU/MLU compute principles and model optimization concepts
- Familiarity with fundamental performance profiling tools and key metrics
Target Audience
- Performance engineers
- Machine learning infrastructure teams
- AI system architects
