Nsight and Roofline: Classify Before Optimizing
REFERENCE CONTEXT · CUDA 12.2; RTX 2070 SUPER; Turing sm_75
Core Concept
The Roofline bound is P <= min(P_compute_peak, BW_peak * AI). Low arithmetic intensity tends toward memory limits; high intensity tends toward compute limits; points far below both roofs often indicate latency or parallelism problems.
Why It Affects Performance
The model eliminates poor optimization directions and distinguishes algorithmic minimum traffic from measured hardware traffic. DRAM, L2, and L1 can each have different roofs.
Example or Pseudocode
def stats(n, ms, bytes_per_elem, flops_per_elem):
s = ms * 1e-3
return n*flops_per_elem/s/1e9, n*bytes_per_elem/s/1e9Fused SAXPY has AI=2/12; a decomposed version is approximately 2/20.
Profiling and Performance Judgment
Use CUDA Events for final latency, Nsight Systems for end-to-end gaps, and Nsight Compute Roofline/SpeedOfLight/MemoryWorkloadAnalysis for single-kernel diagnosis. Do not use profiler replay time as the production benchmark.
Five-Minute Exercise
If effective bandwidth remains about 247 GB/s while runtime falls from 1.35 ms to 0.82 ms after fusion, what caused the gain?
Show answer
The gain came primarily from removing the intermediate and reducing bytes per element from roughly 20 to 12, not from increasing hardware bandwidth.