Commit Graph

55 Commits

Author SHA1 Message Date
037a898ba0 tests: Unify TestCompactionHash and extend it to also Delete keys and Defrag
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:55 +02:00
d3db3bc454 tests: Add integration tests for compact hash
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2022-09-07 15:11:55 +02:00
8c5f110b59 Fix the failure in TestEndpointSwitchResolvesViolation
Refer to a0bdfc4fc9

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-08-07 07:17:27 +08:00
50978d5b25 clientv3: disable mirror auth test with proxy 2022-04-13 12:41:24 +00:00
5d44f2242f cv3/mirror: Fetch the most recent prefix revision
When a user sets up a Mirror with a restricted user that doesn't have
access to the `foo` path, we will fail to get the most recent revision
due to permissions issues.

With this change, when a prefix is provided we will get the initial
revision from the prefix rather than /foo. This allows restricted users
to setup sync.
2022-04-13 12:39:06 +00:00
0bac49bda4 tests: Cleanup member interface by exposing Bridge directly 2021-09-30 12:05:10 +02:00
f324894e8f tests: Make using bridge optional 2021-09-30 12:05:04 +02:00
994bd08723 tests: Rename grpcAddr to grpcURL to imply that it includes schema 2021-09-30 12:04:57 +02:00
3f13d3a2d5 integration.BeforeTest can be run without leak-detection. 2021-05-28 10:01:36 +02:00
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
3acf6e35b2 Finish migration zap.NewExample -> zaptest.NewLogger(t) in integration tests 2021-04-29 22:32:26 +02:00
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
9a4b2bdccc Errors: context cancelled or context deadline exceeded are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown' 2021-04-22 14:35:24 +02:00
b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
2021-04-12 09:43:48 +02:00
e776efbb2a Merge pull request #12828 from ptabor/20210404-embed-etcd
embed: etcd.Close() is closing Errc() channel as well.
2021-04-08 01:20:07 +02:00
5da9cac193 embed: etcd.Close() is closing Errc() channel as well.
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
2021-04-08 01:19:13 +02:00
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
f290ab2e60 Update dependecies:
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
d0962f10ae Adopt grpc_testing to grpc-1.36. 2021-03-24 23:31:03 +01:00
8469108548 Merge pull request #12782 from ptabor/20210316-fixes
Integration: Test flakiness fixes
2021-03-19 20:10:11 +01:00
18321a0c85 Merge pull request #12786 from ptabor/20210318-debug-env
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
809e7629ed Add integration.BeforeTest to all missing tests. 2021-03-16 23:50:01 +01:00
a57e967d84 Integration test flakes fixes. 2021-03-16 16:08:18 +01:00
5ddabfdb24 tests: Make tests operate in /tmp director instead of src.
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
2021-03-09 18:19:52 +01:00
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest().
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
87258efd90 Integration tests: Use zaptest.Logger based testing.TB
Thanks to this the logs:
  - are automatically printed if the test fails.
  - are in pretty consistent format.
  - are annotated by 'member' information of the cluster emitting them.

Side changes:
  - Set propert default got DefaultWarningApplyDuration (used to be '0')
  - Name the members based on their 'place' on the list (as opposed to
'random')
2021-03-09 18:19:51 +01:00
5e10d12996 tests: Fixes a few recently spotted test-flakes
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.039s
```

```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
    server_test.go:110: took 128.026085ms with no latency
    server_test.go:125: took 62.980988ms with latency 50ms��5ms
    server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```

https://github.com/etcd-io/etcd/issues/12372
2021-03-01 18:07:38 +01:00
a7f340216d Reformat code according to 'gotip' rules.
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
45b1e6b470 ClientV3: Ordering: Fix the ordering test such it does not fail.
The test depended on very subtle timing semantic and on properties of
'copied' clients.

https://travis-ci.com/github/etcd-io/etcd/jobs/486191449

Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
    ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
2021-02-25 22:15:13 +01:00
1a9c81abda Update grpc dependency to 1.32.
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
4a1c24556c clientv3: PS: Replace balancer with upstream grpc solution
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
2021-02-23 10:03:15 +01:00
a836a8045b Get rid of legacy client/v3/naming API.
Update grpcproxy to use the new abstractions.
2021-02-09 11:56:28 +01:00
8feb55f65c *: implement Endpoint Watch and new Resolver 2021-02-08 20:05:45 +08:00
571ed502d4 endpoints: implement Update method for EndpointManager.
- Add integration test for endpoints and resolver.
2021-02-04 23:30:22 +08:00
d6d03beaea Merge pull request #12538 from lzhfromustc/12_9_GoroutineLeak
test: change channel operations to avoid potential goroutine leaks
2021-02-01 21:16:43 +01:00
0d7a671c75 Tests: Fix vet warnings about Fatal in sub-goroutines.
% (cd tests && go vet ./...)
stderr: # go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency_test
stderr: integration/clientv3/concurrency/election_test.go:74:6: call to (*T).Fatal from a non-test goroutine
stderr: integration/clientv3/concurrency/mutex_test.go:57:4: call to (*T).Fatal from a non-test goroutine
2021-01-31 00:00:26 +01:00
5d7c1db3a9 Introduce grpc-1.30+ compatible client/v3/naming API.
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652

We propose here 3 packages:
 - clientv3/naming/endpoints ->
    That is abstraction layer over etcd that allows to write, read &
    watch Endpoints information. It's independent from GRPC API. It hides
    the storage details.

 - clientv3/naming/endpoints/internal ->
    That contains the grpc's compatible Update class to preserve the
    internal JSON mashalling format.

 - clientv3/naming/resolver ->
   That implements the GRPC resolver API, such that etcd can be
   used for connection.Dial in grpc.

Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.
2021-01-30 12:32:19 +01:00
351bdb33c5 Split intengration/clientv3 tests into multiple packages
They used to take >10min with coverage, so were causing interrupted
Travis runs.

Know thay fit in 100-150s (together), thanks also to parallel
execution.
2021-01-23 11:12:44 +01:00
f2a912a4e6 test: change channel operations to avoid potential goroutine leaks
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
2020-12-09 22:23:21 -05:00
aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
45b007b8b4 contrib,clientv3: Move contrib/recipies to clientv3/experimental/recipies/...
Recipies is set of patterns / primitives implementation on top of clientv3.
It's used by integration tests. It shouldn't be considered "server" code.
2020-10-22 11:10:07 +02:00
0a830c871e clientv3test: fix network partition flaky test 2020-10-20 23:13:23 +08:00
e33c6dd9df client/v3: Rename of imports 2020-10-20 10:13:06 +02:00
de55bb6331 pkg: Rename imports after making 'pkg' a module
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
30811a06aa etcdctl, clientv3/snapshot: snapshot (Restore,Status) code out of client
"snapshot" Restore/Status code was the only remaining dependency of client on 'server'
code. The code is solelly used by etcdctl. Long-term the snapshot code
should be migrated to 'etcdadm' style of tool such that we can
distinguish tool solelly depending on networking API vs. tools that
operation on etcd files directly.

We left snapshot.Save() code in clientv3.snapshot package, such that
clients can benefits from automated download&safe to file snapshot
functionality over the wire.
2020-10-10 14:14:57 +02:00
f67956cb7a clientv3: Expose clientv3/examples close to the code.
Many of the tests had missing '// Output:' comment, so were not
runnable. They required fining.
2020-10-08 14:27:32 +02:00