Commit Graph

63 Commits

Author SHA1 Message Date
8a6c1335e2 v3rpc: run health notifier to listen on online defrag state change
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-28 17:49:24 -07:00
ea035471ce online defrag notifies gRPC health server to expose NOT_SERVING status
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-10-25 08:58:33 -07:00
aa97484166 *: enable goimports in verify-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:14:09 +08:00
c0aa3b613b Use any instead of interface{}
Signed-off-by: chenyahui <cyhone@qq.com>
2023-09-17 17:41:58 +08:00
979102f895 clientv3: remove the experimental gRPC API grpccredentials.Bundle
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-08-02 19:35:51 +01:00
1ba577e499 server/etcdserver: togRPCError for maintenance API
It's to deflake TestAuthMemberRemove.

When the client has multiple endpoints, the client might send a request
with valid token to the follower member which hasn't received token
replicated log yet. The member will reject the request.

For instance, the maintenance.Status API will return "auth: invalid auth
token". But the client doesn't identify the error. The client won't retry to
refresh auth token. The maintenance.Status should togRPCError before return
so that the client can reflesh token. It's align with existing API.

Since the maintenance client always creates one connection to target
member, the member will have the token after refresh auth.

Maybe we can introduce a sync to wait for member is ready with token,
instead of refreshing.

Fixes: #15758

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-04-22 18:35:53 +08:00
74feb229c7 etcdserver: Guarantee order of requested progress notifications
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.

This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.

This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.

1. Generate a message with WatchId -1, indicating the revision for
   *all* watchers in the stream

2. Guarantee that a response is (eventually) sent

The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.

Signed-off-by: Peter Wortmann <peter.wortmann@skao.int>
2023-04-05 11:54:10 +01:00
9abc895122 Goimports: Apply automated fixing to test files as well.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 13:04:45 +01:00
9e1abbab6e Fix goimports in all existing files. Execution of ./scripts/fix.sh
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 09:41:31 +01:00
faff80a2b3 etcdserve: format the source code
gofmt -w ./server

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:00:59 +08:00
e9aa275b36 etcdserver: update etcdserver to use the new raft module go.etcd.io/raft/v3
Just replaced all go.etcd.io/etcd/raft/v3 with go.etcd.io/raft/v3
under directory server.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 09:33:45 +08:00
f656fa0f49 add missing copyright headers
Signed-off-by: Andrew Sims <andrew.cameron.sims@gmail.com>
2022-11-23 19:13:43 +11:00
f77b8a735f etcdserver: populate HashRevision when responding to leader or client's HashKV request
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-14 08:33:44 +08:00
c967715d93 auth: protect all maintainence APIs when auth is enabled
All maintenance APIs require admin privilege when auth is enabled,
otherwise, the request will be rejected. If auth isn't enabled,
then no such requirement any more.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-11-03 04:39:42 +08:00
b58f9c27e4 Refactoring code to remove duplicate code test.
Signed-off-by: Samuele Resca <sr7@ad.datcon.co.uk>
Signed-off-by: Samuele Resca <samuele.resca@gmail.com>
2022-10-23 13:46:10 +01:00
3d9c5c6166 Adding fuzz test on v3rpc interfaces.
Signed-off-by: Samuele Resca <sr7@ad.datcon.co.uk>
Signed-off-by: Samuele Resca <samuele.resca@gmail.com>
2022-10-23 13:46:10 +01:00
f1d4935e91 *: avoid closing a watch with ID 0 incorrectly
Signed-off-by: Kafuu Chino <KafuuChinoQ@gmail.com>

add test
2022-09-26 20:30:33 +08:00
2441a24cee Merge pull request #14493 from demoManito/style/format-import-order
etcd: format import order
2022-09-21 06:03:31 +08:00
f67ec10779 etcd: format import order
golang CodeReviewComments:
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: demoManito <1430482733@qq.com>
2022-09-20 18:41:39 +08:00
2dcfa83094 *: handle auth invalid token and old revision errors in watch
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2022-09-17 21:51:36 +09:00
508ce517e0 update according to the review
Signed-off-by: spacewander <spacewanderlzx@gmail.com>
2022-08-17 09:25:37 +08:00
bebefd8b80 chore: log when an invalid watch request is received
As protobuf doesn't have required field, user may send an empty
WatchRequest by mistake. Currently, etcd will ignore the invalid request
and keep the stream opening. If we don't reject the invalid request by
closing the stream, it would be better to leave a log there.

This commit also fixes a typo in the comment.

Signed-off-by: spacewander <spacewanderlzx@gmail.com>
2022-08-16 11:33:01 +08:00
6220174687 support custom grpc.MaxConcurrentStreams
There is no update on the original PR (see below) for more then 2
weeks. So Benjamin(@ahrtr) continues to work on the PR. The first
step is to rebase the PR, because there are lots of conflicts with
the main branch.

The change to go.mod and go.sum reverted, because they are not needed.
The e2e test cases are also reverted, because they are not correct.

```
https://github.com/etcd-io/etcd/pull/14081
```

Signed-off-by: nic-chen <chenjunxu6@gmail.com>
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-07-06 03:43:46 +08:00
2b090e86a6 server: Extract hasher to separate interface
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
80828b593a server: Remove duplicated compaction revision
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:18 +02:00
34a02ba621 server: Return revision range that hash was calcualted for
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-06-13 18:20:17 +02:00
5097b33ab9 Rename etcdserver/etcderrors package to etcdserver/errors. 2022-05-20 14:32:04 +02:00
c78bf655a7 Simplify imports and improve comments. 2022-05-20 14:32:04 +02:00
47a771871b Move apply to its own package (no dependency on etcdserver). 2022-05-20 14:32:04 +02:00
fc6a6c3c27 Move etcdserver/errors.go to sepatate package to avoid cyclic dependencies. 2022-05-20 14:32:04 +02:00
b073129d03 Applier does not depend on EtcdServer any longer.
All the depencies are explicily passed to the UberApplier factory method.
2022-05-20 14:32:04 +02:00
651de5a057 Rename EtcdServer.Id with EtcdServer.MemberId.
It was misleading and error prone vs. ClusterId.
2022-05-20 14:32:04 +02:00
1b3d6cb0c8 set an separate applyTimeout for the waitAppliedIndex 2022-04-10 14:44:55 +08:00
d659403955 Update server/etcdserver/api/v3rpc/maintenance.go
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2022-04-08 18:45:13 +01:00
4c8ef011e0 *: drop use of humanize.Time() in favour of time.Duration.String()
humanize.Time() drops precision resulting in some events reporting they took
"now" time to complete. Using time.Duration.String() results in accurate
duration being reported.

Fixes #13905
2022-04-07 23:24:35 +01:00
edce939f6e add one more field storageVersion into StatusResponse
When performing the downgrade operation, users can confirm whether each member
is ready to be downgraded using the field 'storageVersion'. If it's equal to the
'target version' in the downgrade command, then it's ready to be downgraded;
otherwise, the etcd member is still in progress of processing the db file.
2022-03-18 07:04:44 +08:00
8681888012 fix typo, renamed ErrGPRCNotSupportedForLearner to ErrGRPCNotSupportedForLearner 2022-02-21 14:46:58 +08:00
15568f4c00 add protection code for Range when the sortTarget is an invalid value 2022-01-18 07:46:37 +08:00
f8aafea504 add protection code to prevent etcd from panic when the client api version is not valid UTF-8 2022-01-17 06:21:22 +08:00
e47c3c22d2 server: Move downgrade API logic into version package 2021-10-08 12:01:51 +02:00
90932324b1 client: Add grpc authority header integration tests 2021-09-29 12:42:16 +02:00
2a750a8dba *: implement a retry logic for auth old revision in the client 2021-09-05 01:13:52 +09:00
83a325ac46 server: Move all functions needed for storage bootstrap to storage package
This is prerequestite to move storage bootstrap, splitted to separate PR
to make it easier to review.
2021-08-03 13:09:15 +02:00
23b742cfd3 server: Remove Quota direct dependency on EtcdServer 2021-08-03 12:48:41 +02:00
2a26f7ae4c etcdserver: configure "expensive" requests duration
When a unary request takes more than predefined duration, this request
is defined as "expensive" and a warning is printed. The expensive request
duration is hard-coded to 300 ms. It can be not enough for example
for transactions with a lot of operations. The warnings just blow up
the log files and reduce throughput.

This fix allows user to configure the "expensive" request duration.

Signed-off-by: Alexey Roytman <roytman@il.ibm.com>
2021-07-27 08:33:44 +03:00
5b6f4579fb server: Rename buckets to schema 2021-07-12 15:37:21 +02:00
5e40a8b00c server: Create storage package and move mvcc files to it 2021-07-12 15:37:21 +02:00
bf3e7033e9 etcdserver: Move Read/Update methods on Meta bucket to one place
There are still some left like compact keys, but they will require more
work to avoid circular dependency.
2021-07-05 13:23:53 +02:00
e2740b4afa server,etcdutl: Preserve etcd version in backend allowing etcdutl to read it from snapshot 2021-06-25 14:06:56 +02:00
e1b1d93548 *: Snapshot returns local etcd version
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
2021-06-14 16:36:50 +02:00