Skip to content

Expose natsjskv Watcher#142

Open
kobergj wants to merge 6 commits into
go-micro:mainfrom
kobergj:NatsjskvWatcher
Open

Expose natsjskv Watcher#142
kobergj wants to merge 6 commits into
go-micro:mainfrom
kobergj:NatsjskvWatcher

Conversation

@kobergj

@kobergj kobergj commented Jul 16, 2024

Copy link
Copy Markdown
Contributor

This PR bundles two related changes to the nats-js-kv store.

1. Expose natsjskv Watcher

Adds a Watch method to the nats-js-kv store. This exposes the underlying natsjs watcher and allows reacting to changes in the store.

2. Fix: treat a closed connection as no connection

hasConn() previously only checked n.conn != nil. Once the NATS client exhausts its reconnect attempts, the connection is permanently closed but n.conn stays non-nil. Since initConn() is gated on !hasConn(), it never re-runs, so every subsequent KV operation fails with nats: connection closed until the calling process is restarted.

func (n *natsStore) hasConn() bool {
    return n.conn != nil && !n.conn.IsClosed()
}

A closed connection is now treated as "no connection", so the next operation re-initializes it. This also adds a ClosedCB/resetConn() handler.

Note: the ClosedCB handler alone is not sufficient for consumers that pass nats.Options via context — Init() calls setOption() again, reassigning n.nopts from the context value before Connect(), which drops the callback. The IsClosed() check in hasConn() is therefore the load-bearing fix; resetConn() is complementary for direct callers.

@kobergj

kobergj commented Jul 16, 2024

Copy link
Copy Markdown
Contributor Author

Pipelines broken unrelated?

go: open /home/runner/work/plugins/plugins/v5/acme/certmagic/go.mod: no such file or directory

Comment thread v4/store/nats-js-kv/nats.go Outdated
@kobergj kobergj requested a review from butonic July 16, 2024 13:46
@kobergj

kobergj commented Jul 23, 2024

Copy link
Copy Markdown
Contributor Author

@asim is this something we would like to be merged?

@kobergj kobergj marked this pull request as draft July 23, 2024 09:43
@asim asim marked this pull request as ready for review July 23, 2024 10:36
@asim

asim commented Jul 23, 2024

Copy link
Copy Markdown
Contributor

If the test pass and it's been go fmt then sure, why not

@kobergj kobergj force-pushed the NatsjskvWatcher branch 2 times, most recently from 89027d7 to 4bc93ff Compare July 24, 2024 10:27
@kobergj

kobergj commented Jul 24, 2024

Copy link
Copy Markdown
Contributor Author

If the test pass and it's been go fmt then sure, why not

@asim tests seem broken due to v5 update. Not sure what is going wrong. Unrelated to this PR:

go: open /home/runner/work/plugins/plugins/v5/acme/certmagic/go.mod: no such file or directory

@asim

asim commented Jul 24, 2024

Copy link
Copy Markdown
Contributor

I think the acme dir is gone?

@kobergj

kobergj commented Jul 25, 2024

Copy link
Copy Markdown
Contributor Author

I think the acme dir is gone?

I do not know to be honest. I was not involved in v5 update and also didn't see any PRs. So I do not know what the change between v4 and v5 is and neither do I know how this affects v4 builds.

@asim

asim commented Jul 25, 2024

Copy link
Copy Markdown
Contributor

I don't know what you're doing in your branch but you need to merge main otherwise this won't work.

@kobergj

kobergj commented Jul 25, 2024

Copy link
Copy Markdown
Contributor Author

It is up-to-date with main. Previous commit before this one is cc7e2f8

@asim

asim commented Jul 25, 2024

Copy link
Copy Markdown
Contributor

I'm not near a computer but I'd do 'grep -r plugins/v5/acme' to see what's referencing this non existent path.

@gambitier

Copy link
Copy Markdown

@asim I am having same issue. Other ones are also missing

./v5/acme/certmagic
./v5/codec/bsonrpc
./v5/sync/consul
./v5/sync/etcd
./v5/sync/memory

If the test pass and it's been go fmt then sure, why not

@asim tests seem broken due to v5 update. Not sure what is going wrong. Unrelated to this PR:

go: open /home/runner/work/plugins/plugins/v5/acme/certmagic/go.mod: no such file or directory

@asim

asim commented Jul 25, 2024

Copy link
Copy Markdown
Contributor

So it's a go work issue then. Fixing in a separate PR #146

Signed-off-by: jkoberg <jkoberg@owncloud.com>
@kobergj

kobergj commented Jul 30, 2024

Copy link
Copy Markdown
Contributor Author

There still seems to be something wrong:

go: downloading github.com/Microsoft/go-winio v0.5.2
     | pattern ./...: directory prefix . does not contain modules listed in go.work or their selected dependencies

I tried go mod tidy but it doesn't help.

@asim

asim commented Jul 30, 2024

Copy link
Copy Markdown
Contributor

Try delete and reinitialise go work

Signed-off-by: jkoberg <jkoberg@owncloud.com>
@kobergj

kobergj commented Jul 30, 2024

Copy link
Copy Markdown
Contributor Author

Seems to be even more broken after the go work sync.

go: github.com/micro/go-micro@v1.18.0 requires
	github.com/micro/protoc-gen-micro@v1.0.0: reading github.com/micro/protoc-gen-micro/go.mod at revision v1.0.0: git ls-remote -q origin in /home/runner/go/pkg/mod/cache/vcs/28cfbca4f5e41691076780b2017d5a1000b007b1d3d37b7a9d3b562c7adc9377: exit status 128:
	fatal: could not read Username for 'https://github.com/': terminal prompts disabled

I think this is somehow related to the v5 update. Pipelines were green before it. Did we change build mechanics with v5?

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
kobergj and others added 3 commits August 7, 2024 15:01
do not try to unmarshal on deletes
hasConn() only checked n.conn != nil. Once the NATS client exhausts its
reconnect attempts the connection is permanently closed but stays non-nil,
so initConn() (gated on !hasConn()) never re-runs and every subsequent KV
operation fails with 'nats: connection closed' until the pod is restarted.

Also bring over the ClosedCB-based resetConn() handler from the
NatsjskvConnClose branch. Note that the callback alone is insufficient for
go-micro consumers that pass nats.Options via context (e.g. reva's
pkg/store): NewStore registers the ClosedCB, but Init() calls setOption()
again which re-assigns n.nopts from the context value and drops the
callback before Connect(). The IsClosed() check in hasConn() is therefore
the load-bearing fix; resetConn() is complementary for direct callers.

Refs OCISDEV-834
fix(store/nats-js-kv): treat closed connection as no connection
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants