test(cuda.core): cover PDL same-stream overlap via GraphBuilder capture by lijinf2 · Pull Request #2524 · NVIDIA/cuda-python

lijinf2 · 2026-08-06T19:50:08Z

Description

Add GraphBuilder coverage for programmatic_stream_serialization: programmatic dependency-edge mapping and same-stream primary/secondary overlap (Hopper+, opportunistic → xfail).
Share the overlap protocol in helpers/pdl_kernels.py so direct and graph-capture paths stay in sync.
Tweak 1.2.0 notes to use the runtime PDL attribute name.

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

Cover GraphBuilder stream-capture for programmatic_stream_serialization: functional launch, programmatic dependency edge mapping, and Hopper+ overlap (xfail if opportunistic), plus 1.2.0 release notes.

Refer to cudaLaunchAttributeProgrammaticStreamSerialization instead of the driver-style CU_LAUNCH_ATTRIBUTE_* spelling.

copy-pr-bot · 2026-08-06T19:50:11Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

lijinf2 · 2026-08-06T20:02:11Z

/ok to test 4ca415b

The test writes host buffers from np.from_dlpack, which are read-only before NumPy 2.2.5 (GH #28632).

Align with other graph builder tests so context setup and teardown stay consistent.

Drop fragile Programming Guide section numbers and note Driver vs Runtime enum name equivalence at the edge asserts.

Extract kernels and the same-stream / graph-capture overlap check into run_pdl_overlap_check so launcher and GraphBuilder tests stay in sync.

Both direct and graph-capture paths are same-stream; update names and docs accordingly.

lijinf2 · 2026-08-07T18:58:52Z

/ok to test fdfdf8b

isVoid · 2026-08-13T19:58:30Z

+    return mod.get_kernel("primary_kernel"), mod.get_kernel("secondary_kernel")
+
+
+def run_pdl_overlap_check(device, *, via_graph: bool = False):


Why do we need to test with graph launch? This seems like an additional requirement for programmatic stream.

Yeah, that is a request in #1334
per "PDL launches are graph-compatible and this use case should be tested and showcased"

isVoid · 2026-08-13T20:13:55Z

+    mod = compile_common_kernels()
+    producer = mod.get_kernel("add_one")
+    consumer = mod.get_kernel("add_one")


So... I'm curious. add_one is a simple kernel that doesn't have programmatic dependency like the kernel primary_kernel. Setting programmatic_stream_serialization between for the second launch still create a graph dependency between these two kernels?

Yeah, here just needs two dummy kernels as input so that we can check things in _assert_programmatic_dependency_edge.

Renamed to dummy_kernel.

…tests Keep the stream overlap case self-contained in test_launcher, and inline the graph capture/overlap checks in test_graph_builder so each path stays simple.

Avoid implying an in-kernel producer/consumer dependency; the test only needs two launches for the programmatic edge assert.

lijinf2 · 2026-08-13T21:19:48Z

/ok to test 8f922e6

isVoid · 2026-08-14T21:41:30Z

+        from cuda.bindings import driver
+
+        h_graph = graph_definition.handle
+        err, _, _, _, num_edges = driver.cuGraphGetEdges(h_graph)


This API seems to return different number of results based on the major version so you can do:

Suggested change

err, _, _, _, num_edges = driver.cuGraphGetEdges(h_graph)

if driver.CUDA_VERSION >= 13000:

err, _, _, _, num_edges = driver.cuGraphGetEdges(h_graph)

else:

err, _, _, _, num_edges = driver.cuGraphGetEdges_v2(h_graph)

Though what stands out here is that cuda.core graph APIs discards the edge data to prevent implementing the whole helper solely with cuda.core. A feature gap we can consider adding?

Good point — yeah, without edge metadata on the cuda.core graph APIs this helper has to go through cuda.bindings.

isVoid · 2026-08-14T22:36:44Z

/ok to test ee0e37f

Published cuda-bindings <13.3.0 (or <12.9.7 on 12.x) returns dangling CUgraphEdgeData from cuGraphGetEdges; also fix release-note RST indent.

lijinf2 · 2026-08-17T20:42:57Z

/ok to test b98324c

lijinf2 · 2026-08-17T20:49:42Z

@isVoid Thanks for the commit that handles the versioned cuGraphGetEdges / cuGraphGetEdges_v2 API!

I dug into the remaining CI failures on the CUDA 13.0.x rows. They weren’t from the versioned-edge-API handling — those jobs install published cuda-bindings==13.0.* from PyPI, and that line still has the use-after-free in cuGraphGetEdges where the returned CUgraphEdgeData wrappers point at a freed scratch buffer (#1804). The fix only shipped in 13.3.0 (and was backported to 12.9.7 on the 12.x branch), so 13.0.x / 13.1.x / 13.2.x published wheels don’t have it.

I pushed a follow-up that skips the edge-data assertion on those older bindings.

lijinf2 · 2026-08-18T00:05:49Z

pre-commit.ci run

github-actions · 2026-08-18T00:19:56Z

Doc Preview CI
Preview removed because the pull request was closed or merged.

lijinf2 added 2 commits August 6, 2026 19:45

test(cuda.core): verify PDL GraphBuilder capture and overlap

c007afc

Cover GraphBuilder stream-capture for programmatic_stream_serialization: functional launch, programmatic dependency edge mapping, and Hopper+ overlap (xfail if opportunistic), plus 1.2.0 release notes.

docs(cuda.core): use runtime PDL attribute name in 1.2.0 notes

90325d7

Refer to cudaLaunchAttributeProgrammaticStreamSerialization instead of the driver-style CU_LAUNCH_ATTRIBUTE_* spelling.

lijinf2 self-assigned this Aug 6, 2026

lijinf2 marked this pull request as draft August 6, 2026 19:50

lijinf2 changed the title ~~Fea/1334 pdl launch config graph~~ Cover GraphBuilder stream-capture for programmatic dependent launch (PDL) Aug 6, 2026

lijinf2 added cuda.core Everything related to the cuda.core module CI/CD CI/CD infrastructure labels Aug 6, 2026

lijinf2 added this to the cuda.core 1.2.0 milestone Aug 6, 2026

resolve pre-commit errors

4a3cb00

lijinf2 force-pushed the fea/1334-pdl-launch-config-graph branch from 7ebea54 to 4ca415b Compare August 6, 2026 20:01

lijinf2 changed the title ~~Cover GraphBuilder stream-capture for programmatic dependent launch (PDL)~~ Cover GraphBuilder in programmatic dependent launch (PDL) Aug 7, 2026

lijinf2 added 6 commits August 7, 2026 18:04

test(cuda.core): skip PDL overlap graph capture on NumPy < 2.2.5

973a9a6

The test writes host buffers from np.from_dlpack, which are read-only before NumPy 2.2.5 (GH #28632).

test(cuda.core): use init_cuda in PDL overlap graph capture test

552b20a

Align with other graph builder tests so context setup and teardown stay consistent.

test(cuda.core): clarify PDL graph capture doc references

ef9ecb8

Drop fragile Programming Guide section numbers and note Driver vs Runtime enum name equivalence at the edge asserts.

test(cuda.core): share PDL overlap protocol via helper runner

daa6d49

Extract kernels and the same-stream / graph-capture overlap check into run_pdl_overlap_check so launcher and GraphBuilder tests stay in sync.

test(cuda.core): rename PDL overlap tests to emphasize same-stream

0371a37

Both direct and graph-capture paths are same-stream; update names and docs accordingly.

test(cuda.core): tidy PDL overlap helper import order and docstring

2e0d2e6

lijinf2 changed the title ~~Cover GraphBuilder in programmatic dependent launch (PDL)~~ test(cuda.core): cover PDL same-stream overlap via GraphBuilder capture Aug 7, 2026

lijinf2 marked this pull request as ready for review August 7, 2026 18:58

This comment has been minimized.

Sign in to view

isVoid reviewed Aug 13, 2026

View reviewed changes

lijinf2 force-pushed the fea/1334-pdl-launch-config-graph branch from fdfdf8b to 2e0d2e6 Compare August 13, 2026 20:43

lijinf2 added 2 commits August 13, 2026 21:08

test(cuda.core): drop shared PDL overlap helper for clearer per-path …

4d1c049

…tests Keep the stream overlap case self-contained in test_launcher, and inline the graph capture/overlap checks in test_graph_builder so each path stays simple.

test(cuda.core): rename PDL graph-capture kernels to dummy_kernel

8f922e6

Avoid implying an in-kernel producer/consumer dependency; the test only needs two launches for the programmatic edge assert.

isVoid approved these changes Aug 14, 2026

View reviewed changes

isVoid reviewed Aug 14, 2026

View reviewed changes

test(cuda.core): handle versioned graph edge API

ee0e37f

test(cuda.core): skip PDL edge assert on bindings with NVIDIA#1804 UAF

b98324c

Published cuda-bindings <13.3.0 (or <12.9.7 on 12.x) returns dangling CUgraphEdgeData from cuGraphGetEdges; also fix release-note RST indent.

lijinf2 enabled auto-merge (squash) August 17, 2026 21:03

lijinf2 merged commit e0a2d1a into NVIDIA:main Aug 18, 2026
109 checks passed

lijinf2 deleted the fea/1334-pdl-launch-config-graph branch August 18, 2026 00:06

		return mod.get_kernel("primary_kernel"), mod.get_kernel("secondary_kernel")


		def run_pdl_overlap_check(device, *, via_graph: bool = False):

Uh oh!

Conversation

lijinf2 commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

copy-pr-bot Bot commented Aug 6, 2026

Uh oh!

lijinf2 commented Aug 6, 2026

Uh oh!

lijinf2 commented Aug 7, 2026

Uh oh!

This comment has been minimized.

isVoid Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

lijinf2 Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

isVoid Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

lijinf2 Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lijinf2 commented Aug 13, 2026

Uh oh!

isVoid Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

lijinf2 Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

isVoid commented Aug 14, 2026

Uh oh!

lijinf2 commented Aug 17, 2026

Uh oh!

lijinf2 commented Aug 17, 2026

Uh oh!

lijinf2 commented Aug 18, 2026

Uh oh!

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lijinf2 commented Aug 6, 2026 •

edited

Loading

lijinf2 Aug 13, 2026 •

edited

Loading