Occupancy Is a Latency-Hiding Resource, Not the Goal

Core Concept

Occupancy is the ratio of active warps to the architectural maximum. It is limited by threads, registers per thread, shared memory per block, and architectural limits.

Why It Affects Performance

More resident warps can supply ready work while another warp waits, but increasing occupancy does not improve performance once bandwidth or an execution pipeline is already saturated.

Example or Pseudocode

int blocks;
cudaOccupancyMaxActiveBlocksPerMultiprocessor(
    &blocks, kernel, threads_per_block, dynamic_smem_bytes);

Benchmark 128, 256, and 512 threads per block rather than assuming the largest block wins.

Profiling and Performance Judgment

Correlate theoretical and achieved occupancy with registers/thread, blocks/SM, eligible warps, and actual kernel duration.

Five-Minute Exercise

A 256-thread block uses 64 registers per thread. On an SM with 65,536 registers, how many blocks fit based on registers alone?