Commit Graph

627 Commits

Author SHA1 Message Date
eff9517a90 etcdserver: add cluster id check for hashKVHandler
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-07-05 14:09:40 -05:00
ffe73f9a15 add tests for uber applier
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-30 22:03:29 -05:00
22f9dac7b1 Merge pull request #15708 from chaochn47/confchange_raft_node_notifies_apply
raft node notifies configure when confChanged
2023-06-28 10:03:50 +01:00
bda68d8d06 Merge pull request #16074 from geetasg/pr2
Enable test to verify membership recovery from backend
2023-06-27 20:24:51 +01:00
6cdc9ae4fe server/etcdserver/raft.go:
1. rename confChangeCh to raftAdvancedC
2. rename waitApply to confChanged
3. add comments and test assertion

Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-06-26 22:42:44 -07:00
ad3b6ee4c6 etcdserver: wait for raft is notified on confChange before responding to client
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-06-26 13:40:51 -07:00
1967b8e5e6 Merge pull request #16086 from CaojiamingAlan/applier_test
etcdserver: add tests for apply_auth.go
2023-06-26 12:20:05 +02:00
11aa59c42d etcdserver: add tests for apply_auth.go
Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-23 10:58:30 -05:00
c8247731a2 Merge pull request #16099 from chaochn47/enable_failpoint_in_integration_test
Enable failpoint in integration test
2023-06-23 13:15:08 +01:00
6d79b86219 Enable failpoint by default in integration tests
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-06-21 23:13:46 -07:00
550aa152a7 Verify consistent index is latest at the time of snapshot
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-19 16:00:04 +00:00
cb3730a30f Merge pull request #16005 from tjungblu/putauthshort
Early exit auth check on lease puts
2023-06-16 08:33:57 +02:00
84a9af17cc Add first unit test for authApplierV3
This contains a slight refactoring to expose enough information
to write meaningful tests for auth applier v3.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-15 15:57:44 +02:00
e9fa3d30d7 Enable test to verify membership recovery from backend
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-13 18:33:03 +00:00
2b81483103 Adding test for version update function used in 3.6
Signed-off-by: Geeta Gharpure <geetagh@amazon.com>
2023-06-13 17:38:04 +00:00
b9e30bf878 etcdserver: add e2e test to reproduce the incorrect hash issue when resuming scheduled compaction.
check ScheduledCompactKeyName and FinishedCompactKeyName
before writing hash to hashstore. If they do not match, then it means this compaction has once been interrupted and its hash value is invalid. In such cases, we won't write the hash values to the hashstore, and avoids the incorrect corruption alarm.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-06-07 19:54:09 -05:00
dfbe2038f3 Early exit auth check on lease puts
Mitigates #15993 by not checking each key individually for permission
when auth is entirely disabled or admin user is calling the method.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2023-06-06 10:23:46 +02:00
3413f2e08d Merge pull request #15908 from cuishuang/main
*: use strings.Builder instead of bytes.Buffer
2023-05-26 09:41:01 +08:00
9e1e378e9e update code comments
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-05-24 12:54:27 -07:00
0c919dc212 use the more efficient strings.Builder
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-05-19 10:44:58 +08:00
05b663fbe8 Merge pull request #15828 from chaochn47/add_leadership_transfer_coverage
tests/e2e: add graceful shutdown test
2023-05-11 07:39:25 +08:00
f31d0eafb9 tests/e2e: add graceful shutdown test
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-05-09 17:08:53 -07:00
a0cffb6a1f server/etcdserver: swap order of return types
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
2023-05-08 22:53:04 +09:00
49b59cc8e5 Merge pull request #15656 from mitake/lease-timetolive-auth
protect LeaseTimeToLive with RBAC
2023-05-02 23:02:29 +09:00
975854f07f etcdserver: protect lease timetilive with auth
Signed-off-by: Hitoshi Mitake <h.mitake@gmail.com>
Co-authored-by: Benjamin Wang <wachao@vmware.com>
2023-04-26 20:35:20 +09:00
1ba577e499 server/etcdserver: togRPCError for maintenance API
It's to deflake TestAuthMemberRemove.

When the client has multiple endpoints, the client might send a request
with valid token to the follower member which hasn't received token
replicated log yet. The member will reject the request.

For instance, the maintenance.Status API will return "auth: invalid auth
token". But the client doesn't identify the error. The client won't retry to
refresh auth token. The maintenance.Status should togRPCError before return
so that the client can reflesh token. It's align with existing API.

Since the maintenance client always creates one connection to target
member, the member will have the token after refresh auth.

Maybe we can introduce a sync to wait for member is ready with token,
instead of refreshing.

Fixes: #15758

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-04-22 18:35:53 +08:00
9d14ae43c2 fix review: remove if on error check
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-18 10:43:13 -03:00
ecc7441ba1 fix review: use assert lib
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-18 10:02:03 -03:00
3654552fe8 fix review: move setup and vars
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-17 08:19:10 -03:00
7a7e09d3a5 fix review: group calls
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 19:59:47 -03:00
41b1d36bd9 fix review
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 17:44:46 -03:00
a1a2f43f2f fix review
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 17:41:59 -03:00
8fb839eac1 Update server/etcdserver/txn/txn_test.go
Signed-off-by: Marcondes Viana <marju10@gmail.com>
Co-authored-by: Marek Siarkowicz <marek.siarkowicz@protonmail.com>
2023-04-16 17:09:02 -03:00
4c63611768 tests: cover txn.CheckTxnAuth logic with unit tests
Signed-off-by: Marcondes Viana <marju10@gmail.com>
2023-04-16 13:24:12 -03:00
dae1d70189 test: workaround the breaking change in jonboulle/clockwork
See
- https://github.com/jonboulle/clockwork/pull/55
- https://github.com/jonboulle/clockwork/blob/v0.3.0/clockwork.go#L42

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-11 12:01:09 +08:00
8b1cd036ff security: remove password after authenticating the user
fix https://nvd.nist.gov/vuln/detail/CVE-2021-28235

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-04-06 17:11:54 +08:00
74feb229c7 etcdserver: Guarantee order of requested progress notifications
Progress notifications requested using ProgressRequest were sent
directly using the ctrlStream, which means that they could race
against watch responses in the watchStream.

This would especially happen when the stream was not synced - e.g. if
you requested a progress notification on a freshly created unsynced
watcher, the notification would typically arrive indicating a revision
for which not all watch responses had been sent.

This changes the behaviour so that v3rpc always goes through the watch
stream, using a new RequestProgressAll function that closely matches
the behaviour of the v3rpc code - i.e.

1. Generate a message with WatchId -1, indicating the revision for
   *all* watchers in the stream

2. Guarantee that a response is (eventually) sent

The latter might require us to defer the response until all watchers
are synced, which is likely as it should be. Note that we do *not*
guarantee that the number of progress notifications matches the number
of requests, only that eventually at least one gets sent.

Signed-off-by: Peter Wortmann <peter.wortmann@skao.int>
2023-04-05 11:54:10 +01:00
caed563e08 fix flaking auth member remove test
Signed-off-by: Chao Chen <chaochn@amazon.com>
2023-04-03 17:41:08 -07:00
22bdc91302 server/etcdserver: add log for terminating monitors
Adding log for terminating monitors is to make the debug easier.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-11 15:07:17 +08:00
275e10bcf7 Return default snapshot count to 10,000.
The huge (100k+) value was justified when storev2 was being dumped completely with every snapshot.

With storev2 being decomissioned we can checkpoint more frequently for faster recovery.

Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-06 20:21:03 +13:00
c767f429f0 rafthttp: replace inline code with existing function
The isMsgApp function implements the judgment of the MsgApp message,
use the isMsgApp function instead.

Signed-off-by: xakdwch <xakdwch5@gmail.com>
2023-03-03 09:50:14 +08:00
b17b9c1428 chore: Use http constants to replace numbers as parameters
Signed-off-by: xin.li <xin.li@daocloud.io>
2023-02-20 11:53:41 +08:00
e2b8e86624 etcdserver: correct the old name of notifyc in comments
The old name(raftDone) of the channel(notifyc) which indicates the apply has been
completed is left unchanged in the comments, resulting in confusion when reading
the source code.

Signed-off-by: caojiamingalan <alan.c.19971111@gmail.com>
2023-01-27 10:02:22 +08:00
0f4d7a7483 Cleanup
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-01-21 15:57:48 +03:00
de8d6b3792 etcdserver: use time.Ticker instead of time.After
Using time.After will create a new Timer in each cycle, In these cases
, it is better to use time.Ticker.

Signed-off-by: guozhao <guozhao@360.cn>
2023-01-17 16:58:13 +08:00
8ed20e85d2 etcdserver: return membership.ErrIDNotFound when the memberID not found
When promoting a learner, we need to wait until the leader's applied ID
catches up to the commitId. Afterwards, check whether the learner ID
exist or not, and return `membership.ErrIDNotFound` directly in the API
if the member ID not found, to avoid the request being unnecessarily
delivered to raft.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-17 06:18:15 +08:00
bd9f1584d4 process the scenaro of the last WAL record being partially synced to disk
We need to return io.ErrUnexpectedEOF in the error chain, so that
etcdserver can repair it automatically.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-08 04:46:51 +08:00
9abc895122 Goimports: Apply automated fixing to test files as well.
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 13:04:45 +01:00
6f899a7b40 Merge pull request #15052 from ptabor/20221228-goimports-fix
./scripts/fix.sh: Takes care of goimports across the whole project.
2022-12-29 11:31:22 +01:00
9e1abbab6e Fix goimports in all existing files. Execution of ./scripts/fix.sh
Signed-off-by: Piotr Tabor <ptab@google.com>
2022-12-29 09:41:31 +01:00