Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Differences between CPU and GPU architectures
• NVIDIA GPU streaming multiprocessor model
• Overview of the CUDA programming model
• Heterogeneous computing and the host-device paradigm

Configuring the CUDA Development Environment

• Installation of CUDA Toolkit 13.x
• NVCC compiler and build workflows
• Environment verification through device queries
• Integration with IDEs and development tools

Creating and Executing CUDA Kernels

• Kernel function syntax and qualifiers
• Launch configuration and execution details
• Vector addition and fundamental data-parallel patterns
• Implementation of CUDA error checking macros

CUDA Thread Hierarchy and Execution Model

• Organization of grids, blocks, and threads
• Thread indexing and global ID computation
• Warp execution and the SIMT model
• Occupancy and resource utilization metrics

GPU Memory Architecture and Management

• Memory types: global, shared, constant, and registers
• Allocation and release of device memory
• Transfers between host and device
• Use of shared memory for intra-block coordination

Unified Memory and Data Migration

• Unified memory model and managed allocations
• Page migration and on-demand paging mechanisms
• Asynchronous prefetching via cudaMemPrefetchAsync
• Memory advice hints for managing access patterns

System-Wide Profiling with Nsight Systems

• Timeline analysis in Nsight Systems
• Detection of CPU-GPU synchronization points
• Visualization of kernel execution and memory transfers
• Interpretation of system-level performance metrics

Kernel Optimization with Nsight Compute

• Interactive kernel profiling with Nsight Compute
• Analysis of memory throughput and bandwidth
• Compute utilization and warp state statistics
• Guided analysis and optimization rules

Concurrent Streams and Asynchronous Operations

• CUDA streams and the concept of the default stream
• Overlapping kernel execution with data transfers
• Stream synchronization and CUDA events
• Design patterns for multi-stream pipelines

Error Handling and Debugging Utilities

• CUDA API error codes and recovery tactics
• Using compute-sanitizer for memory access verification
• Kernel debugging with cuda-gdb
• Assertions and synchronous error detection

Profile-Driven Optimization Workflow

• Methodology for iterative profiling
• Identification and prioritization of bottlenecks
• Performance regression testing
• Documentation of optimization decisions

End-to-End Accelerated Application Project

• Designing a comprehensive GPU-accelerated solution
• Integrating profiling throughout the development cycle
• Performance benchmarking and reporting
• Deployment considerations for production environments