Grid Tail Waves: Performance Steps at Shape Boundaries
REFERENCE CONTEXT · CUDA 12.2; RTX 2070 SUPER; Turing sm_75
Core Concept
With S SMs and R resident blocks per SM, approximate wave capacity is C=S*R. A grid requires ceil(B/C) waves, and an incomplete final wave leaves many SMs idle.
Why It Affects Performance
Adding one block can move execution from two waves to three. Occupancy on the remaining active SM may still be high even while most SMs have no work.
Example or Pseudocode
S=40, R=2, C=80
B=160 -> 2 waves
B=161 -> 3 waves; the final wave uses only 1 of 80 slotsApproximate wave-slot utilization is B/(C*ceil(B/C)).
Profiling and Performance Judgment
Inspect launch__waves_per_multiprocessor, grid size, blocks/SM, SM throughput, and CUDA Event latency around C-1, C, C+1, 2C-1, 2C, and 2C+1.
Five-Minute Exercise
For C=80, what is utilization at B=161? Should the grid always be padded to 240?
Show answer
Utilization is 161/240=67.1%. Do not pad unconditionally: it remains three waves while adding work unless padding unlocks a more efficient algorithm or hardware path.