Building Custom AI Operators with CANN TIK and TVM Training Course

Introduction to Custom Operator Development

• Rationale for custom operators: specific use cases and system constraints.
• CANN runtime architecture and key operator integration points.
• Overview of TBE, TIK, and TVM within the Huawei AI ecosystem.

Low-Level Operator Programming with TIK

• Detailed examination of the TIK programming model and its API surface.
• Strategies for memory management and tiling within TIK.
• Process for creating, compiling, and registering custom ops with CANN.

Testing and Verification of Custom Ops

• Performing unit and integration testing of ops within the execution graph.
• Troubleshooting kernel-level performance bottlenecks.
• Visualising op execution flows and buffer dynamics.

Scheduling and Optimisation via TVM

• Understanding TVM as a compiler framework for tensor operations.
• Developing custom schedules for ops using TVM.
• Executing TVM tuning, benchmarking, and code generation for Ascend.

Framework and Model Integration

• Registering custom ops for compatibility with MindSpore and ONNX.
• Ensuring model integrity and managing fallback behaviours.
• Handling multi-operator graphs that utilise mixed precision.

Case Studies and Specialised Optimisations

• Case study: achieving high-efficiency convolution for small input shapes.
• Case study: optimising attention operators with memory awareness.
• Best practices for deploying custom ops across various devices.

Summary and Future Directions