Commit Graph

206 Commits

Author SHA1 Message Date
8a6c1335e2 v3rpc: run health notifier to listen on online defrag state change
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-28 17:49:24 -07:00
ea035471ce online defrag notifies gRPC health server to expose NOT_SERVING status
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-25 08:58:33 -07:00
aea1cd0077 feat: enable unparam lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-17 21:24:13 +08:00
7a57e06eca etcdserver: add livez and ready http endpoints for etcd.
Add two separate probes, one for liveness and one for readiness. The liveness probe would check that the local individual node is up and running, or else restart the node, while the readiness probe would check that the cluster is ready to serve traffic. This would make etcd health-check fully Kubernetes API complient.

Signed-off-by: Siyuan Zhang <sizhang@google.com>
2023-10-14 22:32:16 -07:00
867faa1924 etcdserver: remove redundant len check in health check
From the Go specification [1]:

  "1. For a nil slice, the number of iterations is 0."

`len` returns 0 if the slice or map is nil [2]. Therefore, checking
`len(v) > 0` around a loop is unnecessary.

[1]: https://go.dev/ref/spec#For_range
[2]: https://pkg.go.dev/builtin#len

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-10-13 18:39:38 +08:00
c25f1dff82 http health check bug fixes
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-12 16:59:34 -07:00
70a3205506 fix broken unit test in server_test.go
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-09-28 20:07:06 +01:00
9c9804399e do not update RaftCluster.members and RaftCluster.removed if the v3store is ahead of the current replayed WAL entry index
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-09-28 20:06:12 +01:00
4704a5af3a *: fix unused issue
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-25 19:37:18 +08:00
896d161d16 *: fix stylecheck linter
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-25 19:14:01 +08:00
07effc4d0a *: fix revive linter
Remove old revive_pass in the bash scripts and migirate the revive.toml
into golangci linter_settings.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-24 14:21:11 +08:00
aa97484166 *: enable goimports in verify-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:14:09 +08:00
5e3910d96c *: fix govet-shadow lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-19 20:24:01 +08:00
189f709a5d Apply patches on the generated *.pb.gw.go files to convert v1 messages to v2 messages
Signed-off-by: Benjamin Wang <wachao@vmware.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-18 11:22:16 +01:00
4ec581a6a3 re-generate all the *.pb.gw.go files
Run ./scripts/genproto.sh

Signed-off-by: Benjamin Wang <wachao@vmware.com>
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-18 11:22:15 +01:00
522b698928 upgrade grpc-gateway from v1 to v2
Signed-off-by: Wei Fu <fuweid89@gmail.com>
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-09-18 11:22:15 +01:00
c0aa3b613b Use any instead of interface{}
Signed-off-by: chenyahui <cyhone@qq.com>
2023-09-17 17:41:58 +08:00
f432c1cf20 chore: pkg import more than once
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-08-21 10:19:05 +08:00
10c7e81cac Merge pull request #16358 from ahrtr/remove_creds_bundle_20230802
clientv3: remove the experimental gRPC API grpccredentials.Bundle
2023-08-04 08:46:02 +01:00
979102f895 clientv3: remove the experimental gRPC API grpccredentials.Bundle
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-08-02 19:35:51 +01:00
e5b7dde17e Add a method to export membership info to v2 store from RaftCluster
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-07-28 16:55:41 +00:00
abbfc2964a Fix goword issue
Fix `make verify` issues after updating golangci-lint

Signed-off-by: iuriatan <iuriatan@gmail.com>
2023-07-14 16:46:26 -03:00
eff9517a90 etcdserver: add cluster id check for hashKVHandler
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-05 14:09:40 -05:00
e9fa3d30d7 Enable test to verify membership recovery from backend
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-13 18:33:03 +00:00
0c919dc212 use the more efficient strings.Builder
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-05-19 10:44:58 +08:00
1ba577e499 server/etcdserver: togRPCError for maintenance API
It's to deflake TestAuthMemberRemove.

When the client has multiple endpoints, the client might send a request
with valid token to the follower member which hasn't received token
replicated log yet. The member will reject the request.

For instance, the maintenance.Status API will return "auth: invalid auth
token". But the client doesn't identify the error. The client won't retry to
refresh auth token. The maintenance.Status should togRPCError before return
so that the client can reflesh token. It's align with existing API.

Since the maintenance client always creates one connection to target
member, the member will have the token after refresh auth.

Maybe we can introduce a sync to wait for member is ready with token,
instead of refreshing.

Fixes: #15758

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-04-22 18:35:53 +08:00
dae1d70189 test: workaround the breaking change in jonboulle/clockwork
See
- https://github.com/jonboulle/clockwork/pull/55
- https://github.com/jonboulle/clockwork/blob/v0.3.0/clockwork.go#L42

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-11 12:01:09 +08:00
74feb229c7 etcdserver: Guarantee order of requested progress notifications
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.

This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.

This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.

1. Generate a message with WatchId -1, indicating the revision for
   *all* watchers in the stream

2. Guarantee that a response is (eventually) sent

The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.

Signed-off-by: Peter Wortmann <peter.wortmann@skao.int>
2023-04-05 11:54:10 +01:00
c767f429f0 rafthttp: replace inline code with existing function
The isMsgApp function implements the judgment of the MsgApp message,
use the isMsgApp function instead.

Signed-off-by: xakdwch <xakdwch5@gmail.com>
2023-03-03 09:50:14 +08:00
b17b9c1428 chore: Use http constants to replace numbers as parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2023-02-20 11:53:41 +08:00
9abc895122 Goimports: Apply automated fixing to test files as well.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 13:04:45 +01:00
9e1abbab6e Fix goimports in all existing files. Execution of ./scripts/fix.sh
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 09:41:31 +01:00
394956ca4e doc: cleanup etcd/raft in all documents
TODO:
1. Update Documentation/contributor-guide/modules.svg;
2. Update bill-of-materials.json when raft and raftexample are removed in future;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 14:13:18 +08:00
faff80a2b3 etcdserve: format the source code
gofmt -w ./server

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:00:59 +08:00
e9aa275b36 etcdserver: update etcdserver to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory server.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 09:33:45 +08:00
2feec4fe68 comments: fix comments as per goword in go test files
Comments fixed as per goword in go test files that shell
function go_srcs_in_module lists as per changes on #14827

Helps in #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 23:05:42 +05:30
f656fa0f49 add missing copyright headers
Signed-off-by: Andrew Sims <andrew.cameron.sims@gmail.com>
2022-11-23 19:13:43 +11:00
c3b6cbdb73 all: goimports -w .
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-17 19:07:04 +03:00
2c9c209eb6 all: Changing Printf and friends to Print if there is no formatting
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-15 22:11:23 +03:00
006e747a44 all: Change time unit
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-11-15 01:15:01 +03:00
f77b8a735f etcdserver: populate HashRevision when responding to leader or client's HashKV request
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-14 08:33:44 +08:00
0f9d7a4f95 raft: make Message.Snapshot nullable, halve struct size
This commit makes the rarely used `raftpb.Message.Snapshot` field nullable.
In doing so, it reduces the memory size of a `raftpb.Message` message from
264 bytes to 128 bytes — a 52% reduction in size.

While this commit does not change the protobuf encoding, it does change
how that encoding is used. `(gogoproto.nullable) = false` instruct the
generated proto marshaling logic to always encode a value for the field,
even if that value is empty. `(gogoproto.nullable) = true` instructs the
generated proto marshaling logic to omit an encoded value for the field
if the field is nil.

This raises compatibility concerns in both directions. Messages encoded
by new binary versions without a `Snapshot` field will be decoded as an
empty field by old binary versions. In other words, old binary versions
can't tell the difference. However, messages encoded by old binary versions
with an empty Snapshot field will be decoded as a non-nil, empty field by
new binary versions. As a result, new binary versions need to be prepared
to handle such messages.

While Message.Snapshot is not intentionally part of the external interface
of this library, it was possible for users of the library to access it and
manipulate it. As such, this change may be considered a breaking change.

Signed-off-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
2022-11-09 17:35:52 +00:00
c967715d93 auth: protect all maintainence APIs when auth is enabled
All maintenance APIs require admin privilege when auth is enabled,
otherwise, the request will be rejected. If auth isn't enabled,
then no such requirement any more.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-03 04:39:42 +08:00
f215cd89d2 Merge pull request #14612 from spacewander/azq
chore: commit the change generated by scripts/genproto.sh
2022-10-24 20:00:52 +02:00
3a63a0d5e3 chore: commit the change generated by scripts/genproto.sh
TODO: ensure the generated code is up-to-date in the CI.
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
2022-10-23 21:13:55 +08:00
b58f9c27e4 Refactoring code to remove duplicate code test.
Signed-off-by: Samuele Resca <sr7@ad.datcon.co.uk>
Signed-off-by: Samuele Resca <samuele.resca@gmail.com>
2022-10-23 13:46:10 +01:00
3d9c5c6166 Adding fuzz test on v3rpc interfaces.
Signed-off-by: Samuele Resca <sr7@ad.datcon.co.uk>
Signed-off-by: Samuele Resca <samuele.resca@gmail.com>
2022-10-23 13:46:10 +01:00
1c20ed2cc5 Merge pull request #14521 from lovehhf/remove_pick_peer_url
membership: Remove PickPeerURL Method
2022-09-27 02:10:35 +08:00
f6d808736c membership: Remove PickPeerURL Method
PickPeerURL only used by unit test

Signed-off-by: Hongfei Huang <853885165@qq.com>
2022-09-26 23:21:10 +08:00
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
2022-09-26 20:30:33 +08:00