Skip to content

Fix the Bazel 9.1 build and make sure it doesn't silently break again - #6

Draft
jdcormie wants to merge 2 commits into
masterfrom
bazel-9-1
Draft

jdcormie wants to merge 2 commits into
masterfrom
bazel-9-1

Conversation

@jdcormie

Copy link
Copy Markdown
Owner

No description provided.

Upgrading fixes a rules_go incompatibility with Bazel 9.1 (as mentioned
in the Bazel 9.1 release notes), although this specific version was
selected simply because it was the newest.

Googleapis greatly reduced their dependencies, so we no longer have
transitive dependencies on grpc (C repo) and rules_go, which cleans up
our dependencies considerably.
This backs up our claim that we support the two most recent Bazel versions.
jdcormie pushed a commit that referenced this pull request Sep 16, 2026
…orClientInterceptor (grpc#13022)

Prevent concurrent cancellations of the underlying ClientCall in
ExternalProcessorClientInterceptor:
- Wrap rawCall with SimpleForwardingClientCall using an AtomicBoolean to
ensure the underlying ClientCall.cancel() is executed at most once, even
if invoked concurrently across threads or from DelayedListener.
- In DataPlaneClientCall.cancel(), atomically transition
extProcStreamState to FAILED, catch exceptions during onError(), and
clear extProcClientCallRequestObserver.
- In sendToExtProc(), return early if the ext-proc stream is already
completed or the observer is null, and catch unexpected onNext()
exceptions to trigger internalOnError() rather than letting exceptions
escape into listener callbacks.
- Safely complete and clear extProcClientCallRequestObserver in
closeExtProcStream() and halfCloseExtProcStream().
- Route all rawCall.cancel() calls in sendMessage(),
handleImmediateResponse(), and DataPlaneListener through
cancelDownstream().

Fixes the TSAN detected race condition below. The previous fix in commit
cf92f2d introduced `AtomicBoolean downstreamCancelled` in
`cancelDownstream()`. However, the TSAN data race still reproduced under
postsubmit because `DelayedListener` exception path bypasses
`cancelDownstream()`: When `DelayedClientCall.start(wrappedListener,
headers)` is called, it registers an internal `DelayedListener`. If
`wrappedListener.onHeaders()` throws any exception,
`DelayedListener.deliverHeaders()` catches it and calls
`exceptionThrown()`, which invokes `delayedCall.cancel()` directly. Once
`delayedCall` has been activated `(realCall != null)`,
`DelayedClientCall.cancel()` directly calls `realCall.cancel()` on
whatever thread triggered it without any de-duplication.
```
Details
==================
WARNING: ThreadSanitizer: data race (pid=7774)
  Read of size 1 at 0x000089d8a086 by thread T113:
    #0 io.grpc.internal.ClientCallImpl.cancelInternal(Ljava/lang/String;Ljava/lang/Throwable;)V ClientCallImpl.java:464 
    #1 io.grpc.internal.ClientCallImpl.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ClientCallImpl.java:455 
    #2 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    #3 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    #4 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    #5 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    #6 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    #7 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    grpc#8 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    grpc#9 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    grpc#10 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    grpc#11 io.grpc.internal.DelayedClientCall$2.run()V DelayedClientCall.java:269 
    grpc#12 io.grpc.internal.DelayedClientCall.delayOrExecute(Ljava/lang/Runnable;)V DelayedClientCall.java:292 
    grpc#13 io.grpc.internal.DelayedClientCall.cancel(Lio/grpc/Status;Z)V DelayedClientCall.java:266 
    grpc#14 io.grpc.internal.DelayedClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V DelayedClientCall.java:242 
    grpc#15 io.grpc.xds.ExternalProcessorClientInterceptor$DataPlaneClientCall.internalOnError(Ljava/lang/Throwable;)V ExternalProcessorClientInterceptor.java:715 
    grpc#16 io.grpc.xds.ExternalProcessorClientInterceptor$DataPlaneClientCall$1.onNext(Lio/envoyproxy/envoy/service/ext_proc/v3/ProcessingResponse;)V ExternalProcessorClientInterceptor.java:457 
    grpc#17 io.grpc.xds.ExternalProcessorClientInterceptor$DataPlaneClientCall$1.onNext(Ljava/lang/Object;)V ExternalProcessorClientInterceptor.java:426 
    grpc#18 io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(Ljava/lang/Object;)V ClientCalls.java:568 
    grpc#19 io.grpc.ForwardingClientCallListener.onMessage(Ljava/lang/Object;)V ForwardingClientCallListener.java:33 
    grpc#20 io.grpc.ForwardingClientCallListener.onMessage(Ljava/lang/Object;)V ForwardingClientCallListener.java:33 
    grpc#21 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal()V ClientCallImpl.java:662 
    grpc#22 io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext()V ClientCallImpl.java:649 
    grpc#23 io.grpc.internal.ContextRunnable.run()V ContextRunnable.java:37 
    grpc#24 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133 
    grpc#25 io.grpc.internal.SerializingExecutor.run()V SerializingExecutor.java:133 
    grpc#26 java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V ThreadPoolExecutor.java:1090 
    grpc#27 java.util.concurrent.ThreadPoolExecutor$Worker.run()V ThreadPoolExecutor.java:614 
    grpc#28 java.lang.Thread.runWith(Ljava/lang/Object;Ljava/lang/Runnable;)V Thread.java:1560 
    grpc#29 java.lang.Thread.run()V Thread.java:1547 
    grpc#30 (Generated Stub) <null> 

  Previous write of size 1 at 0x000089d8a086 by thread T29:
    #0 io.grpc.internal.ClientCallImpl.cancelInternal(Ljava/lang/String;Ljava/lang/Throwable;)V ClientCallImpl.java:467 
    #1 io.grpc.internal.ClientCallImpl.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ClientCallImpl.java:455 
    #2 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    #3 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    #4 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    #5 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    #6 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    #7 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    grpc#8 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    grpc#9 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    grpc#10 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    grpc#11 io.grpc.internal.DelayedClientCall$2.run()V DelayedClientCall.java:269 
    grpc#12 io.grpc.internal.DelayedClientCall.delayOrExecute(Ljava/lang/Runnable;)V DelayedClientCall.java:292 
    grpc#13 io.grpc.internal.DelayedClientCall.cancel(Lio/grpc/Status;Z)V DelayedClientCall.java:266 
    grpc#14 io.grpc.internal.DelayedClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V DelayedClientCall.java:242 
    grpc#15 io.grpc.PartialForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V PartialForwardingClientCall.java:39 
    grpc#16 io.grpc.ForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:22 
    grpc#17 io.grpc.ForwardingClientCall$SimpleForwardingClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ForwardingClientCall.java:44 
    grpc#18 io.grpc.xds.ExternalProcessorClientInterceptor$DataPlaneClientCall.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V ExternalProcessorClientInterceptor.java:887 
    grpc#19 io.grpc.xds.XdsNameResolver$RawMessageClientInterceptor$2.cancel(Ljava/lang/String;Ljava/lang/Throwable;)V XdsNameResolver.java:1182 
    grpc#20 io.grpc.xds.ExternalProcessorClientInterceptorTest.clientInterceptor_contextPropagatedToStartCall()V ExternalProcessorClientInterceptorTest.java:13411 
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants