Skip to content

Commit 324b4ad

Browse files
committed
fix(transport): preserve behavior across the gRPC boundary
Signed-off-by: Yordis Prieto <yordis.prieto@gmail.com>
1 parent 53e5942 commit 324b4ad

402 files changed

Lines changed: 2153 additions & 39409 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build-container-ubuntu-lts.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,6 @@ jobs:
3535
fail-fast: true
3636
matrix:
3737
include:
38-
- test-group-name: core-clientapi-persistent
39-
- test-group-name: core-clientapi-security
40-
- test-group-name: core-clientapi-streams
4138
- test-group-name: core-http
4239
- test-group-name: core-services
4340
- test-group-name: core-cluster-services

‎.gitignore‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,6 @@ ipch/
3333
.DS_Store
3434

3535
src/EventStore/EventStore.Common/Properties/AssemblyVersion.cs
36-
src/EventStore/EventStore.ClientAPI/Properties/AssemblyVersion.cs
37-
3836
*.o
3937
*.ii
4038
*.s

‎Dockerfile‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@ FROM mcr.microsoft.com/dotnet/sdk:10.0-${CONTAINER_RUNTIME} AS test
4242
WORKDIR /build
4343
COPY --from=build ./build/published-tests ./published-tests
4444
COPY --from=build ./build/ci ./ci
45-
COPY --from=build ./build/src/EventStore.Core.Tests/Services/Transport/Tcp/test_certificates/ca/ca.crt /usr/local/share/ca-certificates/ca_eventstore_test.crt
4645
COPY ./scripts/test.sh /build/test.sh
4746
RUN mkdir ./test-results
4847
RUN chmod +x /build/test.sh
@@ -84,12 +83,11 @@ RUN mkdir -p /var/lib/eventstore && \
8483

8584
USER eventstore
8685

87-
RUN printf "NodeIp: 0.0.0.0\n\
88-
ReplicationIp: 0.0.0.0" >> /etc/eventstore/eventstore.conf
86+
RUN printf "NodeIp: 0.0.0.0" >> /etc/eventstore/eventstore.conf
8987

9088
VOLUME /var/lib/eventstore /var/log/eventstore
9189

92-
EXPOSE 1112/tcp 1113/tcp 2113/tcp
90+
EXPOSE 2113/tcp
9391

9492
HEALTHCHECK --interval=5s --timeout=5s --retries=24 \
9593
CMD curl --fail --insecure https://localhost:2113/-/liveness || curl --fail http://localhost:2113/-/liveness || exit 1

‎docker-compose.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ services:
1717
- shared.env
1818
environment:
1919
- EVENTSTORE_GOSSIP_SEED=172.30.240.12:2113,172.30.240.13:2113
20-
- EVENTSTORE_REPLICATION_IP=172.30.240.11
20+
- EVENTSTORE_NODE_IP=172.30.240.11
2121
- EVENTSTORE_CERTIFICATE_FILE=/etc/eventstore/certs/node/node.crt
2222
- EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/etc/eventstore/certs/node/node.key
2323
- EVENTSTORE_ADVERTISE_HOST_TO_CLIENT_AS=127.0.0.1
@@ -44,7 +44,7 @@ services:
4444
- shared.env
4545
environment:
4646
- EVENTSTORE_GOSSIP_SEED=172.30.240.11:2113,172.30.240.13:2113
47-
- EVENTSTORE_REPLICATION_IP=172.30.240.12
47+
- EVENTSTORE_NODE_IP=172.30.240.12
4848
- EVENTSTORE_CERTIFICATE_FILE=/etc/eventstore/certs/node/node.crt
4949
- EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/etc/eventstore/certs/node/node.key
5050
- EVENTSTORE_ADVERTISE_HOST_TO_CLIENT_AS=127.0.0.1
@@ -71,7 +71,7 @@ services:
7171
- shared.env
7272
environment:
7373
- EVENTSTORE_GOSSIP_SEED=172.30.240.11:2113,172.30.240.12:2113
74-
- EVENTSTORE_REPLICATION_IP=172.30.240.13
74+
- EVENTSTORE_NODE_IP=172.30.240.13
7575
- EVENTSTORE_CERTIFICATE_FILE=/etc/eventstore/certs/node/node.crt
7676
- EVENTSTORE_CERTIFICATE_PRIVATE_KEY_FILE=/etc/eventstore/certs/node/node.key
7777
- EVENTSTORE_ADVERTISE_HOST_TO_CLIENT_AS=127.0.0.1

‎docs/README.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,12 @@ surfaces for running a node or cluster.
1010

1111
TrogonEventStore keeps the database node focused on the durable event log:
1212

13-
- Application event access is gRPC-first.
13+
- Database client APIs, cluster replication, and follower-to-leader forwarding
14+
use gRPC over the node HTTP(S) endpoint.
1415
- HTTP is reserved for the Admin UI, health probes, metrics, and other
1516
infrastructure-level concerns.
17+
- The server does not open a separate legacy EventStore TCP protocol listener
18+
or support its TCP transport configuration.
1619
- The project is FOSS-only. The documentation does not describe unsupported
1720
proprietary server features.
1821
- Rich read models, user-defined query engines, connector runtimes, and
@@ -37,7 +40,7 @@ For a production node, review:
3740

3841
## Protocols and clients
3942

40-
The supported application protocol is gRPC. Existing TrogonEventStore-compatible
43+
The supported database protocol is gRPC. Existing TrogonEventStore-compatible
4144
gRPC clients can be useful while the TrogonDB client libraries continue to
4245
evolve, but the server documentation should be treated as authoritative for this
4346
repository.

‎docs/admin-ui.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ The TrogonEventStore Admin UI is available at `http://SERVER_IP:2113/ui` and hel
1010

1111
The dashboard opens at `/ui` and combines the daily operational view in one place:
1212

13-
- _Cluster status_: live gossip membership, node state, checkpoints, TCP and HTTP endpoints, replica status, and a copy-friendly snapshot.
13+
- _Cluster status_: live gossip membership, node state, checkpoints, HTTP(S) endpoints, replica status, and a copy-friendly snapshot.
1414
- _Queue pressure_: live queue length, throughput, processing time, and currently processed messages.
1515
- _Node probes_: inline Ping, Node info, and Gossip checks rendered inside the UI.
1616

@@ -24,7 +24,8 @@ The _Observability_ page focuses on runtime diagnostics:
2424

2525
- queue groups and individual queue rows
2626
- current and last processed messages
27-
- TCP connection statistics
27+
- active connections on the shared HTTP and gRPC endpoint
28+
- live gRPC replication sessions, byte totals, pending bytes, and send queue depth
2829
- snapshot output for copy-paste debugging
2930

3031
## Configuration

‎docs/architecture.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,16 @@ and sharding work. Features that need their own compute model, query model, or
2222
serving model should be separate components that consume the database through
2323
subscriptions or reads.
2424

25+
## Network protocol boundary
26+
27+
gRPC carries database client APIs, cluster replication, and follower-to-leader
28+
forwarding over the node HTTP(S) endpoint. Ordinary HTTP routes on that endpoint
29+
serve the Admin UI, health probes, metrics, and other operator workflows.
30+
31+
Keeping one advertised endpoint gives clients and cluster members the same TLS,
32+
identity, and network-policy boundary. The node therefore has no separate legacy
33+
EventStore TCP protocol listener or configuration surface.
34+
2535
## Projection execution
2636

2737
Projection execution is future external component work by default.

‎docs/cluster.md‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -55,15 +55,21 @@ The multi-address DNS name cluster discovery only works for clusters that use ce
5555

5656
### Internal communication
5757

58-
When setting up a cluster, the nodes must be able to reach each other over both the HTTP channel, and the internal TCP channel. You should ensure that these ports are open on firewalls on the machines and between the machines.
58+
Cluster nodes use gRPC over each node's HTTP(S) endpoint for replication and
59+
follower-to-leader request forwarding. Ensure every node can reach every other
60+
node at its advertised HTTP(S) endpoint. There is no separate replication
61+
listener or TCP port to expose.
5962

60-
Learn more about [internal TCP configuration](networking.md#replication-protocol) and [HTTP configuration](networking.md#http-configuration) to set up the cluster properly.
63+
Learn more about the shared [HTTP(S) configuration](networking.md#http-configuration)
64+
before configuring cluster firewall or network policy rules.
6165

6266
## Cluster with DNS
6367

6468
When you tell TrogonEventStore to use DNS for its gossip, the server will resolve the DNS name to a list of IP addresses and connect to each of those addresses to find other nodes. This method is very flexible because you can change the list of nodes on your DNS server without changing the cluster configuration. The DNS method is also useful in automated deployment scenarios when you control both the cluster deployment and the DNS server from your infrastructure-as-code scripts.
6569

66-
To use DNS discovery, you need to set the `ClusterDns` option to the DNS name that allows making an HTTP call to it. When the server starts, it will attempt to make a gRPC call using the `https://<cluster-dns>:<gossip-port>` URL (`http` if the cluster is insecure).
70+
To use DNS discovery, set the `ClusterDns` option to a DNS name that resolves to
71+
the cluster nodes. When the server starts, it attempts a gRPC call over
72+
`https://<cluster-dns>:<gossip-port>` (`http` if the cluster is insecure).
6773

6874
When using a certificate signed by a publicly trusted CA, you'd normally use the wildcard certificate. Ensure that the cluster DNS name fits the wildcard, otherwise the request will fail on SSL check.
6975

@@ -107,7 +113,8 @@ The setting accepts a comma-separated list of IP addresses or host names with th
107113

108114
TrogonEventStore uses a quorum-based replication model. When working normally, a cluster has one node known as a leader, and the remaining nodes are followers. The leader node is responsible for coordinating writes while it is the leader. Cluster nodes use a consensus algorithm to determine which node should be the leader and which should be followers. TrogonEventStore bases the decision as to which node should be the leader on a number of factors.
109115

110-
For a cluster node to have this information available to them, the nodes gossip with other nodes in the cluster. Gossip runs over HTTP interfaces of cluster nodes.
116+
For a cluster node to have this information available to them, the nodes gossip
117+
with other nodes in the cluster over the shared HTTP(S) endpoint.
111118

112119
The gossip protocol configuration can be changed using the settings listed below. Pay attention to the settings related to time, like intervals and timeouts, when running in a cloud environment.
113120

@@ -229,7 +236,7 @@ candidate.
229236

230237
### Follower
231238

232-
A cluster assigns the follower role based on an election process. A cluster uses one or more nodes with the follower role to form the quorum, or the majority of nodes necessary to confirm that the write is persisted.
239+
A cluster assigns the follower role based on an election process. A cluster uses one or more nodes with the follower role to form the quorum, or the majority of nodes necessary to confirm that the write is persisted. When a follower accepts a request that must run on the leader, it forwards the request to the leader over gRPC on the leader's HTTP(S) endpoint.
233240

234241
### Read-only replica
235242

‎docs/diagnostics/README.md‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,17 +49,6 @@ type `$statsCollected`.
4949
"proc-gc-largeHeapSize": 0,
5050
"proc-gc-timeInGc": 0.0,
5151
"proc-gc-totalBytesInHeaps": 0,
52-
"proc-tcp-connections": 0,
53-
"proc-tcp-receivingSpeed": 0.0,
54-
"proc-tcp-sendingSpeed": 0.0,
55-
"proc-tcp-inSend": 0,
56-
"proc-tcp-measureTime": "00:00:19.0534210",
57-
"proc-tcp-pendingReceived": 0,
58-
"proc-tcp-pendingSend": 0,
59-
"proc-tcp-receivedBytesSinceLastRun": 0,
60-
"proc-tcp-receivedBytesTotal": 0,
61-
"proc-tcp-sentBytesSinceLastRun": 0,
62-
"proc-tcp-sentBytesTotal": 0,
6352
"es-checksum": 1613144,
6453
"es-checksumNonFlushed": 1613144,
6554
"sys-drive-/System/Volumes/Data-availableBytes": 545628151808,
@@ -104,7 +93,7 @@ type `$statsCollected`.
10493
"es-queue-MonitoringQueue-lengthLifetimePeak": 0,
10594
"es-queue-MonitoringQueue-totalItemsProcessed": 14,
10695
"es-queue-MonitoringQueue-inProgressMessage": "<none>",
107-
"es-queue-MonitoringQueue-lastProcessedMessage": "GetFreshTcpConnectionStats",
96+
"es-queue-MonitoringQueue-lastProcessedMessage": "GetFreshStats",
10897
"es-queue-PersistentSubscriptions-queueName": "PersistentSubscriptions",
10998
"es-queue-PersistentSubscriptions-groupName": "",
11099
"es-queue-PersistentSubscriptions-avgItemsPerSecond": 1,

‎docs/installation.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,9 @@ Before running a durable node or cluster:
7676
- Store data, index, and logs on durable volumes.
7777
- Expose `/-/liveness`, `/-/readiness`, and `/-/metrics` to the platform.
7878
- Use gRPC clients for application reads and writes.
79+
- Expose only the node HTTP(S) endpoint. Database client APIs, replication, and
80+
follower-to-leader forwarding use gRPC on that endpoint; no separate legacy
81+
EventStore TCP protocol listener is required or supported.
7982

8083
## Linux service notes
8184

0 commit comments

Comments
 (0)