Table of contents

Get trace-wide statistics for custom groups of kernels

1. Open a Trace File in the Trace Analyzer 0:32

Start with a trace generated for a model
Use the NC Trace Analyzer to open the trace

2. Create a Custom Kernel Group 1:17

Highlight a group of kernels of interest.
Use the 'NC Group Kernels' button to create a custom group.
Edit the group name (e.g., 'my custom group') and run analysis.

3. Analyze the Custom Group 1:43

The analysis identifies 24 instances of the selected kernels across the trace.
Navigate to events and use 'show in trace' to view all instances.
Jump between instances and observe their locations in the overall trace.

4. Gather Statistics for the Group 2:32

Obtain system-wide statistics for the custom group of kernels.
View total time, mean, and p95 time for the group across all instances.

5. Focus on a specific region in the code 3:01

If interested in a single decode token, update the analysis range to focus on it.
Rerun the analysis to see if the custom group appears in the new range.

https://www.loom.com/share/3931ac6b42834527a85c130f06e2da68

Auto-extract repeated patterns of kernels [In Beta]

This feature extends the previous one by automatically detecting repeated patterns of kernels in a trace and displaying trace wide statistics for these kernels. This could be things like detecting repeated patterns that make up a layer or even an entire token.

1. Open a Trace File in the Trace Analyzer 0:32

To use the Trace Analyzer:
- Start with a trace generated for a model
- Use the NC Trace Analyzer to open the trace

Run analysis to detect patterns 0:44
- The analyzer detects patterns of kernel launches within the trace:
Analyzing Statistics 1:22
- After detecting patterns, users can view various statistics:
  - Total occurrences of the pattern.
  - Variance, mean, median, and 95th percentile (p95) of execution times.
  - Time taken for each element in the detected pattern.
- They can also view each instance of the pattern throughout the trace

https://www.loom.com/share/ce7bc7b50b764c938b19064cee0f83e0