Notebookverified
GPU synchronization changed the ranking
Adding synchronization reversed an apparent latency advantage between two inference paths.
Context
Two inference paths were timed with perf_counter_ns. The supposedly faster path launched more asynchronous work.
Observation
Without device synchronization, the benchmark mostly measured host dispatch. Synchronizing before and after each measured region reversed the ranking.
Questions
- Should end-to-end latency and device-only latency appear as separate metrics?
- Which synchronization overhead belongs to the real request path?
Next steps
Repeat with device events, preserve all samples, and document the measured boundary in the result manifest.