Accelerated C++ Programming with CUDA Training Course

Introduction to GPU-Accelerated Computing

• Heterogeneous computing concepts and the interaction between CPU and GPU architectures.
• Understanding CUDA execution flows and distinct memory spaces.
• Compiling CUDA C++ code using nvcc and CMake build systems.
• Verifying the readiness of the GPU development environment.

Parallel Algorithms with Thrust and CUB

• Accelerating sorting, reduction, and transformation tasks on the GPU.
• Adapting STL algorithms for efficient GPU execution.
• Utilizing Thrust device vectors and specific execution policies.
• Applying CUB device-wide primitives for specialized pipelines.

GPU Memory Architecture and Management

• Differentiating between global, constant, and texture memory types.
• Handling explicit device memory allocation and data transfers.
• Using Unified Memory to streamline data access patterns.
• Optimizing memory coalescing and access strategies.

Asynchronous Execution with CUDA Streams

• Creating and managing parallel CUDA streams.
• Concurrent execution of kernels and data transfers to maximize throughput.
• Using CUDA events to manage task dependencies.
• Adjusting stream priorities and concurrency for optimal performance.

Developing Custom CUDA Kernels

• Understanding the SIMT model and warp-level execution.
• Configuring kernel launches and implementing grid-stride loops.
• Mastering thread indexing and multidimensional grid structures.
• Implementing robust error handling and CUDA runtime API checks.

Thread Hierarchy and Execution Model

• Structuring grids, blocks, and threads within device code.
• Leveraging warp-level primitives and ballot operations.
• Managing block-level synchronization and barriers.
• Analyzing occupancy and resource utilization metrics.

Cooperative Groups for Flexible Parallelism

• Exploring the cooperative_groups API and various group types.
• Implementing thread-block tiles and tiled_partition techniques.
• Executing grid-level cooperative launches.
• Applying multi-grid synchronization patterns.

Shared Memory Optimization Techniques

• Managing shared memory banks and avoiding bank conflicts.
• Applying tiling strategies for efficient matrix operations.
• Utilizing shared memory as a user-managed cache layer.
• Using cuda::shared_memory_mdspan for multidimensional data views.

Kernel Fusion and Advanced Parallel Patterns

• Merging multiple kernels to minimize launch overhead.
• Implementing scan, reduce-by-key, and segmented algorithms.
• Utilizing atomic operations and lock-free data structures.
• Aggregating warp-level atomic operations for increased throughput.

Profiling and Optimization with Nsight Systems

• Analyzing CPU and GPU activity via timeline visualization.
• Identifying and resolving memory transfer bottlenecks.
• Profiling kernel performance and occupancy levels.
• Performing iterative optimization using Nsight Compute.

Modern C++ Features in CUDA Device Code

• Incorporating lambdas, constexpr, and auto within kernels.
• Using C++17 parallel algorithms and execution policies.
• Applying C++20 concepts and ranges on the device.
• Leveraging C++23 support in nvcc and CCCL 3.x.

CUDA Graphs and Advanced Asynchrony

• Defining and launching CUDA graphs for efficient execution.
• Capturing graphs directly from stream execution.
• Updating graphs and managing conditional execution nodes.
• Reducing launch latency in iterative workloads.

Integration Patterns for Existing Applications

• Encapsulating GPU code behind standard C++ interfaces.
• Managing multi-GPU and NUMA-based system architectures.
• Integrating CUDA builds seamlessly with CMake.
• Debugging device code effectively with cuda-gdb.

Summary and Best Practices

• Deciding between Thrust, CUB, and custom kernel implementations.
• Ensuring performance portability across various GPU architectures.
• Structuring code and applying RAII for CUDA resource management.
• Exploring next steps and advanced CUDA learning paths.