Ext_proc filter by kannanjgithub · Pull Request #12792 · grpc/grpc-java

kannanjgithub · 2026-05-04T11:35:49Z

Implements ext_proc filter from A93 (internal design doc)
Metrics aren't implemented yet.
Includes commits from unmerged Filter API enhancements and channel caching PRs.
Only the ExternaProcessingFilter.java, ExternaProcessingFilterTest.java and the envoy xds proto import and generated code need to be reviewed.
Rebasing commit history caused all received and merged commits to show my name as the committer, ignore all commits for which I'm not shown as the author.

…xtProcStreamSendsMetadata to use real dataPlaneChannel

…sHeadersAndCallIsBuffered to use real dataPlaneChannel

…henMutationsAreAppliedAndCallIsActivated to use real dataPlaneChannel

…sActivatedImmediately to use real dataPlaneChannel and ServerInterceptor

…entToExtProc to use real dataPlaneChannel

…henMessagesAreDiscarded to use real dataPlaneChannel

…ExtProcAndSuperHalfCloseIsDeferred to use real dataPlaneChannel

…nSuperHalfCloseIsCalled to use real dataPlaneChannel

…False to use real dataPlaneChannel

…sTrue to use real dataPlaneChannel

…OnReady to use real dataPlaneChannel

…Ready to use real dataPlaneChannel

…stsAreForwardedImmediately to use real dataPlaneChannel

…sHeadersAndCallIsBuffered to use real dataPlaneChannel

…henMutationsAreAppliedAndCallIsActivated to use real dataPlaneChannel

…sActivatedImmediately to use real dataPlaneChannel

…entToExtProc to use real dataPlaneChannel

…henMutatedBodyIsForwardedToDataPlane to use real dataPlaneChannel

…eCallFailsOpen to use real dataPlaneChannel

…lizingExecutor to use real dataPlaneChannel

…henMutatedBodyIsForwardedToDataPlane to use real dataPlaneChannel

…henMessagesAreDiscarded to use real dataPlaneChannel

…ExtProcAndSuperHalfCloseIsDeferred to use real dataPlaneChannel

…nSuperHalfCloseIsCalled to use real dataPlaneChannel

…ntToExtProc to use real dataPlaneChannel

…thenMutatedBodyIsDeliveredToClient to use real dataPlaneChannel

…thenClientListenerCloseIsPropagated to use real dataPlaneChannel

# Conflicts: # xds/src/main/java/io/grpc/xds/ExternalProcessorFilter.java

Fix style and warning as errors.

sauravzg · 2026-05-06T06:43:24Z

Reviewed approximately ~500 LOC of source(not tests), which currently mostly only covers the config part of things. Sending comments incrementally to kick start progress, since it seems that I'll need probably about 3 days to review all of source.

…iew comments.

sauravzg · 2026-05-07T06:25:14Z

Reviewed about another 300 LOC, which covers static utilities and the interceptors. Speed was slower than expected, but will try to catch up.

TAG=agy

sauravzg · 2026-05-11T05:48:27Z

+      private long serverTrailersStartNanos;
+
+      private volatile Metadata requestHeaders;
+      final AtomicBoolean activated = new AtomicBoolean(false);


Should we be using an atomic reference enum with a state machine driver here?

The current set of 10 atomic bools, results in potentially 2^10 representable states. I don't think we have that many and should try to compress the state machine.

Introduced state machine states now:

private enum ExtProcStreamState { ACTIVE, DRAINING, COMPLETED, FAILED } private enum DataPlaneCallState { IDLE, ACTIVE, CLOSED }

sauravzg · 2026-05-11T05:52:42Z

+        this.backendService = checkNotNull(backendService, "backendService");
+      }
+
+      private void activateCall() {


Optional: Somewhat echoing go/java-practices/class-structure - Maybe we should try to put the public member that uses a private method before the private method to enhance readability.

That would make more sense when there is a single caller of the private methods. But there are many calling sites for activateCall.

sauravzg · 2026-05-11T05:54:34Z

+        }
+      }
+
+      private boolean checkCompressionSupport(BodyResponse bodyResponse) {


This function name feels slightly misleading. Given that it starts with check and returns a boolean, I'd expect it to not have any side effects, but it does.

My rec would be to either rename this method appropriately (optionally documenting the side effects) or split this method to reflect check and side effect behavior separately.

Renamed to validateCompressionSupport and added Javadoc to explain the side-effects if validation fails.

sauravzg · 2026-05-11T05:56:19Z

+            StatusRuntimeException ex = Status.UNAVAILABLE
+                .withDescription("gRPC message compression not supported in ext_proc")
+                .asRuntimeException();
+            if (!extProcStreamCompleted.get() && extProcClientCallRequestObserver != null) {


This member seems to be synchronized in other places. Should we have it here as well or is this a conscious decision?

You are correct, it needs to be synchronized as well. Done.

sauravzg · 2026-05-11T05:59:30Z

+          }
+          headersToModify.add(HeaderValueOption.create(
+              headerValue,
+              HeaderValueOption.HeaderAppendAction.valueOf(protoOption.getAppendAction().name()),


This may be slightly risky IMO.
I am not sure we want to couple our internal enum to xds proto enum, so I'd prefer an explicit switch case if possible.

valueOf will fail with an IllegalArgumentException if the enum with the specified name is not present, so it will be caught by unit test failures if a mismatch occurs in the future.
Besides, with backward compatibility in mind, enum names are not subject to change.

sauravzg · 2026-05-11T06:13:39Z

+          @Override
+          public void beforeStart(ClientCallStreamObserver<ProcessingRequest> requestStream) {
+            synchronized (streamLock) {
+              extProcClientCallRequestObserver = requestStream;


Couple of things here.

It seems like we aren't respecting control flow before calling on next for the stream and processing pending buffer

It seems like this is dead code. BeforeStart is called synchronously so pendingRequests should in theory always be empty.

Why buffering is required - Buffering is still required when grpc creates a DelayedClientCall for the ext_proc call: While the interceptor execution starts on the application thread, the extProcStub is created using a channel from cachedChannelManager.getChannel(...). If the external processor's gRPC channel is not yet connected, or name resolution/load balancing is still in progress, gRPC-Java wraps the call in a DelayedClientCall. A DelayedClientCall buffers and delays the actual stream start. In this case, stub.process(...) will return before the stream is created and before beforeStart() is called.
The application thread can resume execution and can immediately start calling sendMessage() or halfClose().
At this point, extProcClientCallRequestObserver is still null. Without the pendingProcessingRequests buffer, all request headers, body messages, and half-closes would be completely lost or trigger a crash.

About respecting ext_proc flow control:
The isReady behavior of the data plane call uses also the ext_proc call's isReady state, and it would apply in this buffering case as well. In fact the any buffering into pendingProcessingRequests will already be a violation of flow control by the calling application because isReady would be returning false in that period because extProcClientCallRequestObserver would be null.

sauravzg · 2026-05-11T06:18:54Z

+          public void beforeStart(ClientCallStreamObserver<ProcessingRequest> requestStream) {
+            synchronized (streamLock) {
+              extProcClientCallRequestObserver = requestStream;
+              while (!pendingProcessingRequests.isEmpty()) {


I have some questiosns on of why we need pendingProcessing.... The design doc seems to mention a couple of cases

observability mode ext proc slowness

request drain.

But the grfc for observability mode requests flow control over buffering: https://github.com/markdroth/proposal/blob/6ee533dac7d9d1b71ad46ba8826d2a3b3cdba313/A93-xds-ext-proc.md#flow-control . This should be somewhat easy to enforce, where we push problem to ext proc server, if our caller doesn't respect flow control.

Similarly, flow control is recommended solution for draining as well at https://github.com/markdroth/proposal/blob/6ee533dac7d9d1b71ad46ba8826d2a3b3cdba313/A93-xds-ext-proc.md#early-termination-of-ext_proc-stream . I don't really have a good solution for this if the caller doesn't respect flow control, we'll have to buffer to avoid out of order messages. Maybe let's discuss with Mark on the grfc about this.

Flow control is already implemented via isReady and onReady overriding of the dataplane rpc to take into account ext_proc stream's existence and readiness.
I have now also implemented buffering request messages during the ext_proc draining time period.

sauravzg · 2026-05-11T06:19:25Z

+                        .asRuntimeException());
+                    return;
+                  }
+                  Metadata target = wrappedListener.trailersOnly.get()


encapsulation instead of member access.

sauravzg · 2026-05-11T06:19:49Z

+                }
+              }
+              // 3. Client Trailers
+              else if (response.hasRequestTrailers()) {


Should we be handling this? The grfc doesn't talk about this.

Would this also make proceedWithClose dead code?

Removed.
There are other usages of proceedWithClose.

sauravzg · 2026-05-11T06:27:47Z

+            if (extProcStreamCompleted.compareAndSet(false, true)) {
+              synchronized (streamLock) {
+                if (extProcClientCallRequestObserver != null) {
+                  extProcClientCallRequestObserver.onError(t);


For my education is this valid behavior?

I'd assume, when the stream has error, it calls our onError , but it seems we are propagating it back to the request stream. Am I missing something here?

No, you are right, it is both unnecessary and illegal since the call is already closed. Removed it now.
I have now introduced a new private method internalOnError for cases that do require calling onError on the ext_proc request observer, such as during ext_proc protocol violations.

sauravzg · 2026-05-11T06:29:05Z

Reviewed the extproc response handling. Will be reviewing rest of the client call private methods today.

…lers - Separate internally triggered protocol errors from server-side gRPC stream onError callbacks. - Introduce `internalOnError(Throwable)` to propagate client cancellation only when the error is triggered internally by the client filter. - Avoid calling `extProcClientCallRequestObserver.onError(t)` inside the StreamObserver's `onError(t)` callback since the stream is already terminated by the gRPC framework. - Remove unused and dead `response.hasRequestTrailers()` handling block.

… machines Migrates the ExternalProcessorFilter's state management in DataPlaneClientCall from 10+ independent AtomicBoolean flags (such as activated, notifiedApp, extProcStreamCompleted, extProcStreamFailed, and drainingExtProcStream) to two disciplined AtomicReference state machines. This eliminates invalid concurrent state combinations, prevents race conditions during cleanup, and simplifies overall lifecycle readability. - Introduced ExtProcStreamState (IDLE, DRAINING, COMPLETED, FAILED) and DataPlaneCallState (IDLE, ACTIVE, CLOSED) enums. - Implemented atomic CAS transition helpers (markExtProcStreamCompleted, markExtProcStreamFailed, markDataPlaneCallClosed) and query methods. - Updated activateCall, internalOnError, and onClose callbacks to coordinate lifecycle events and seamlessly support fail-open (failureModeAllow) behavior without redundant cancellations.

1. Introduced buffering queue: Added pendingDrainingMessages inside DataPlaneClientCall. 2. Buffered during draining: Updated sendMessage(InputStream message) to detect if isExtProcStreamDraining() is true, buffering any outgoing application messages into pendingDrainingMessages rather than forwarding them to ext_proc. 3. Drained upon completion: Added drainPendingDrainingMessages() which drains the queue directly to the upstream raw call. This method is invoked synchronously during stream completion (handleFailOpen). Added unit test givenDrainingStream_whenAppSends_thenBufferedAndDelivered. This test verifies that: 1. Outgoing data plane messages sent while the ext_proc stream is in the DRAINING state are not forwarded to the ext_proc observer. 2. Once the ext_proc stream reaches completion, all buffered messages are successfully drained and received by the upstream data plane receiver.

# Conflicts: # xds/src/main/java/io/grpc/xds/Filter.java # xds/src/test/java/io/grpc/xds/StatefulFilter.java

…anges for parseFilterConfig.

kannanjgithub added 30 commits April 9, 2026 10:05

Migrate givenGrpcServiceWithInitialMetadata_whenCallIntercepted_thenE…

d78e448

…xtProcStreamSendsMetadata to use real dataPlaneChannel

Migrate givenRequestHeaderModeSend_whenStartCalled_thenExtProcReceive…

e8b773c

…sHeadersAndCallIsBuffered to use real dataPlaneChannel

Migrate givenRequestHeaderModeSend_whenExtProcRespondsWithMutations_t…

ef4961e

…henMutationsAreAppliedAndCallIsActivated to use real dataPlaneChannel

Migrate givenRequestHeaderModeSkip_whenStartCalled_thenDataPlaneCallI…

7ad1dab

…sActivatedImmediately to use real dataPlaneChannel and ServerInterceptor

Migrate givenRequestBodyModeGrpc_whenSendMessageCalled_thenMessageIsS…

b0d8430

…entToExtProc to use real dataPlaneChannel

Fix halfClose idempotency and migrate mutated body test

849c10c

Migrate givenExtProcSignaledEndOfStream_whenClientSendsMoreMessages_t…

946e9fc

…henMessagesAreDiscarded to use real dataPlaneChannel

Migrate givenRequestBodyModeGrpc_whenHalfCloseCalled_thenSignalSentTo…

54e93b1

…ExtProcAndSuperHalfCloseIsDeferred to use real dataPlaneChannel

Migrate givenDeferredHalfClose_whenExtProcRespondsWithEndOfStream_the…

1075563

…nSuperHalfCloseIsCalled to use real dataPlaneChannel

Migrate givenObservabilityModeTrue_whenExtProcBusy_thenIsReadyReturns…

058fd86

…False to use real dataPlaneChannel

Migrate givenObservabilityModeFalse_whenExtProcBusy_thenIsReadyReturn…

2dfddc2

…sTrue to use real dataPlaneChannel

Migrate givenCongestionInExtProc_whenExtProcBecomesReady_thenTriggers…

c88e706

…OnReady to use real dataPlaneChannel

Migrate givenDrainingStream_whenExtProcStreamCompletes_thenTriggersOn…

f9878da

…Ready to use real dataPlaneChannel

Migrate givenExtProcStreamCompleted_whenAppRequestsMessages_thenReque…

41cc10a

…stsAreForwardedImmediately to use real dataPlaneChannel

Fix halfClose idempotency in ExternalProcessorFilter

31eaa5b

Use SerializingExecutor for sidecar calls and migrate first test

277b910

Migrate givenRequestHeaderModeSend_whenStartCalled_thenExtProcReceive…

ec38666

…sHeadersAndCallIsBuffered to use real dataPlaneChannel

Migrate givenRequestHeaderModeSend_whenExtProcRespondsWithMutations_t…

db46290

…henMutationsAreAppliedAndCallIsActivated to use real dataPlaneChannel

Migrate givenRequestHeaderModeSkip_whenStartCalled_thenDataPlaneCallI…

d99f8e5

…sActivatedImmediately to use real dataPlaneChannel

Migrate givenRequestBodyModeGrpc_whenSendMessageCalled_thenMessageIsS…

ad94cce

…entToExtProc to use real dataPlaneChannel

Migrate givenRequestBodyModeGrpc_whenExtProcRespondsWithMutatedBody_t…

2b0cadc

…henMutatedBodyIsForwardedToDataPlane to use real dataPlaneChannel

Migrate givenFailureModeAllowTrue_whenExtProcStreamFails_thenDataPlan…

ae9a25b

…eCallFailsOpen to use real dataPlaneChannel

Migrate givenInterceptor_whenCallIntercepted_thenExtProcStubUsesSeria…

ab394b7

…lizingExecutor to use real dataPlaneChannel

Migrate givenRequestBodyModeGrpc_whenExtProcRespondsWithMutatedBody_t…

5b1f07c

…henMutatedBodyIsForwardedToDataPlane to use real dataPlaneChannel

Migrate givenExtProcSignaledEndOfStream_whenClientSendsMoreMessages_t…

21bca29

…henMessagesAreDiscarded to use real dataPlaneChannel

Migrate givenRequestBodyModeGrpc_whenHalfCloseCalled_thenSignalSentTo…

9240095

…ExtProcAndSuperHalfCloseIsDeferred to use real dataPlaneChannel

Migrate givenDeferredHalfClose_whenExtProcRespondsWithEndOfStream_the…

c9dc587

…nSuperHalfCloseIsCalled to use real dataPlaneChannel

Migrate givenResponseBodyModeGrpc_whenOnMessageCalled_thenMessageIsSe…

b6b89b9

…ntToExtProc to use real dataPlaneChannel

Migrate givenResponseBodyModeGrpc_whenExtProcRespondsWithMutatedBody_…

8c963db

…thenMutatedBodyIsDeliveredToClient to use real dataPlaneChannel

Migrate givenResponseBodyModeGrpc_whenExtProcRespondsWithEndOfStream_…

94e7e6f

…thenClientListenerCloseIsPropagated to use real dataPlaneChannel

nit: Undo whitespace changes

a28cc57

kannanjgithub marked this pull request as draft May 4, 2026 11:39

Merge remote-tracking branch 'origin/ext-proc' into ext-proc

3a9b96f

# Conflicts: # xds/src/main/java/io/grpc/xds/ExternalProcessorFilter.java

kannanjgithub force-pushed the ext-proc branch from 6dcc263 to 57f216b Compare May 4, 2026 12:39

kannanjgithub marked this pull request as ready for review May 4, 2026 12:39

kannanjgithub requested a review from sauravzg May 4, 2026 12:39

Fix style and warning as errors.

aac9d87

Fix style and warning as errors.

kannanjgithub force-pushed the ext-proc branch from 57f216b to aac9d87 Compare May 4, 2026 18:33

sauravzg requested changes May 6, 2026

View reviewed changes

Created separate config for the override config and address other rev…

a6e61a1

…iew comments.

sauravzg requested changes May 7, 2026

View reviewed changes

sauravzg reviewed May 7, 2026

View reviewed changes

Comment thread xds/src/main/java/io/grpc/xds/ExternalProcessorFilter.java

Comment thread xds/src/main/java/io/grpc/xds/ExternalProcessorFilter.java

Comment thread xds/src/main/java/io/grpc/xds/ExternalProcessorFilter.java

kannanjgithub added 3 commits May 7, 2026 10:14

Address review comments.

ca05e90

xds: Rename FilterContext to FilterConfigParseContext

cd72421

TAG=agy

xds: Introduce FilterContext and pass it to newInstance

08d9464

TAG=agy

kannanjgithub force-pushed the ext-proc branch 3 times, most recently from c75ba11 to 64010fd Compare May 8, 2026 16:13

xds: Add client-side latency metrics for ext_proc HTTP filter

e94ca66

TAG=agy

kannanjgithub force-pushed the ext-proc branch from 64010fd to e94ca66 Compare May 8, 2026 17:19

sauravzg reviewed May 11, 2026

View reviewed changes

kannanjgithub added 6 commits May 11, 2026 10:02

Address review comments.

4578a41

Merge remote-tracking branch 'upstream/master' into ext-proc

74b240a

# Conflicts: # xds/src/main/java/io/grpc/xds/Filter.java # xds/src/test/java/io/grpc/xds/StatefulFilter.java

Resolving merge conflicts after merge from upstream for Filter API ch…

f920c70

…anges for parseFilterConfig.

Conversation

kannanjgithub commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sauravzg commented May 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sauravzg commented May 7, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sauravzg commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

kannanjgithub commented May 4, 2026 •

edited

Loading

sauravzg commented May 11, 2026 •

edited

Loading