Skip to content

HTTP/2 HPACK inflate failure (-523) with gRPC native clients behind Envoy proxy on v26.2.0 #388

Description

@kiloumap

Description

After upgrading from OpenSwoole 22.1.2 to 26.2.0, gRPC native mobile clients (Android/iOS using okhttp/5.3.2) fail with HPACK header decompression errors when connecting through an Envoy proxy. Direct connections to the OpenSwoole server (bypassing Envoy) work fine. The same setup worked perfectly on 22.1.2.

Environment

  • OpenSwoole: 26.2.0 (compiled with --enable-openssl --enable-http2 --enable-json --enable-curl)
  • PHP: 8.5.3 (NTS)
  • OS: Ubuntu 24.04 (ARM64 / aarch64)
  • Envoy proxy: 1.34.1
  • gRPC client: okhttp/5.3.2 (Android native gRPC)

Error

WARNING http2_parse_header(): inflate failed, Error: Header compression/decompression error[-523]

Error code -523 is NGHTTP2_ERR_HEADER_COMP from the bundled nghttp2 library.

Reproduction

Architecture

Mobile App (okhttp/5.3.2) → Envoy 1.34.1 (TLS termination, port 8443) → OpenSwoole (plaintext HTTP/2, port 50051)

What works

  • grpcurl (20 parallel requests) through Envoy → OpenSwoole: works
  • curl --http2-prior-knowledge directly to OpenSwoole port 50051: works
  • Same mobile clients against OpenSwoole 22.1.2 (PHP 8.3): works
  • REST API (not using OpenSwoole): works

What fails

  • Any gRPC call from Android/iOS native clients through Envoy → OpenSwoole 26.2.0: fails
  • Errors appear in bursts across multiple worker PIDs simultaneously
  • Envoy logs show 0 DC (Downstream Connection terminated) for all mobile requests

Envoy cluster config

- name: grpc-api
  typed_extension_protocol_options:
    envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
      "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
      explicit_http_config:
        http2_protocol_options:
          max_concurrent_streams: 100

Workarounds attempted (none worked)

  • USE_ZEND_ALLOC=0 (disable Zend memory allocator) — no effect
  • Increase http2_header_table_size to 8192 — no effect
  • Increase http2_max_header_list_size to 32768 — no effect
  • Increase http2_max_frame_size to 32768 — no effect
  • Increase Envoy max_concurrent_streams from 1 to 100 — no effect
  • Set Envoy hpack_table_size: 0 (disable dynamic table) — no effect

Investigation

We compared the HTTP/2 source code between v22.1.2 and v26.2.0:

  • http2_parse_header() function: identical (only SW_ → OSW_ prefix renames)
  • php_nghttp2_mem() allocator: identical
  • Bundled nghttp2 library: unchanged
  • HTTP/2 default constants (header_table_size, max_frame_size, etc.): identical

The only relevant changes between versions are:

  • Migration to llhttp for HTTP/1.x parsing (should not affect HTTP/2)
  • Native PHP Fiber context backend for coroutines (coroutines are disabled in our config: enable_coroutine: false)

Expected behavior

gRPC native clients connecting through Envoy should work the same as on v22.1.2.

Additional context

Downgrading to 22.1.2 is not possible because it doesn't compile with PHP 8.5 (php_shutdown_function_entry.fci removed). v25.2.0 doesn't support PHP 8.5 either. v26.2.0 is the only option for PHP 8.5.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions