f290ab2e60
Update dependecies:
...
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
d0962f10ae
Adopt grpc_testing to grpc-1.36.
2021-03-24 23:31:03 +01:00
8469108548
Merge pull request #12782 from ptabor/20210316-fixes
...
Integration: Test flakiness fixes
2021-03-19 20:10:11 +01:00
18321a0c85
Merge pull request #12786 from ptabor/20210318-debug-env
...
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
9312d1b077
clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
...
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
dac6e37ea1
*: over 20 staticcheck fixes
2021-03-18 15:06:17 +01:00
725a8c5e02
Enable configuring delegated zap-logging for embed server.
2021-03-17 08:17:36 +01:00
6657d5907c
Make sure all integration tests have BeforeTest.
...
The CL disallows to create NewCluster in tests without BeforeTest.
2021-03-16 23:50:01 +01:00
809e7629ed
Add integration.BeforeTest to all missing tests.
2021-03-16 23:50:01 +01:00
a84bd093b0
Integration with grpc-settable logger.
2021-03-16 22:50:41 +01:00
a57e967d84
Integration test flakes fixes.
2021-03-16 16:08:18 +01:00
a47c18d30a
Fix 2 remaining 'defer AfterTest' calls.
2021-03-13 23:41:29 +01:00
b406647dd7
Fix/remove broken: TestMetricDbSizeDefragDebugging
2021-03-12 23:05:53 +01:00
fd7fed1511
Move config (ServerConfig) out of etcdserver package.
...
Motivation:
- ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
- EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
2021-03-11 20:56:22 +01:00
b6c2e87a74
Testing: Integration tests does not check whether t==nil
2021-03-09 18:19:52 +01:00
5ddabfdb24
tests: Make tests operate in /tmp director instead of src.
...
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
2021-03-09 18:19:52 +01:00
bfe02c0526
tests: Cluster creation that failed shouldn't leak goroutines.
2021-03-09 18:19:52 +01:00
41f6cc7234
Tests: Better isolation between store_v2v3 integration tests.
2021-03-09 18:19:51 +01:00
fb1d48e98e
Integration tests: Use BeforeTest(t) instead of defer AfterTest().
...
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
87258efd90
Integration tests: Use zaptest.Logger based testing.TB
...
Thanks to this the logs:
- are automatically printed if the test fails.
- are in pretty consistent format.
- are annotated by 'member' information of the cluster emitting them.
Side changes:
- Set propert default got DefaultWarningApplyDuration (used to be '0')
- Name the members based on their 'place' on the list (as opposed to
'random')
2021-03-09 18:19:51 +01:00
102c198444
Merge pull request #12705 from astromechza/bm_etcd_peer_server_cert
...
etcdmain: added peer-client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
2021-03-02 09:03:35 +01:00
5e10d12996
tests: Fixes a few recently spotted test-flakes
...
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL go.etcd.io/etcd/tests/v3/integration/client 0.039s
```
```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
server_test.go:110: took 128.026085ms with no latency
server_test.go:125: took 62.980988ms with latency 50ms��5ms
server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```
https://github.com/etcd-io/etcd/issues/12372
2021-03-01 18:07:38 +01:00
3d44f5bf80
*: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
...
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
flag and config file field for peer TLS.
Signed-off-by: Ben Meier <ben.meier@oracle.com >
2021-02-28 14:37:56 +00:00
a7f340216d
Reformat code according to 'gotip' rules.
...
In practices adds annotations in the new syntax:
```
+//go:build !linux
// +build !linux
```
Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
45b1e6b470
ClientV3: Ordering: Fix the ordering test such it does not fail.
...
The test depended on very subtle timing semantic and on properties of
'copied' clients.
https://travis-ci.com/github/etcd-io/etcd/jobs/486191449
Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
2021-02-25 22:15:13 +01:00
1a9c81abda
Update grpc dependency to 1.32.
...
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
4a1c24556c
clientv3: PS: Replace balancer with upstream grpc solution
...
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
2021-02-23 10:03:15 +01:00
a836a8045b
Get rid of legacy client/v3/naming API.
...
Update grpcproxy to use the new abstractions.
2021-02-09 11:56:28 +01:00
8feb55f65c
*: implement Endpoint Watch and new Resolver
2021-02-08 20:05:45 +08:00
571ed502d4
endpoints: implement Update method for EndpointManager.
...
- Add integration test for endpoints and resolver.
2021-02-04 23:30:22 +08:00
d6d03beaea
Merge pull request #12538 from lzhfromustc/12_9_GoroutineLeak
...
test: change channel operations to avoid potential goroutine leaks
2021-02-01 21:16:43 +01:00
0d7a671c75
Tests: Fix vet warnings about Fatal in sub-goroutines.
...
% (cd tests && go vet ./...)
stderr: # go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency_test
stderr: integration/clientv3/concurrency/election_test.go:74:6: call to (*T).Fatal from a non-test goroutine
stderr: integration/clientv3/concurrency/mutex_test.go:57:4: call to (*T).Fatal from a non-test goroutine
2021-01-31 00:00:26 +01:00
5d7c1db3a9
Introduce grpc-1.30+ compatible client/v3/naming API.
...
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652
We propose here 3 packages:
- clientv3/naming/endpoints ->
That is abstraction layer over etcd that allows to write, read &
watch Endpoints information. It's independent from GRPC API. It hides
the storage details.
- clientv3/naming/endpoints/internal ->
That contains the grpc's compatible Update class to preserve the
internal JSON mashalling format.
- clientv3/naming/resolver ->
That implements the GRPC resolver API, such that etcd can be
used for connection.Dial in grpc.
Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.
2021-01-30 12:32:19 +01:00
351bdb33c5
Split intengration/clientv3 tests into multiple packages
...
They used to take >10min with coverage, so were causing interrupted
Travis runs.
Know thay fit in 100-150s (together), thanks also to parallel
execution.
2021-01-23 11:12:44 +01:00
5dcd459ae9
Merge pull request #12564 from dhermes/patch-1
...
Adding `clientv3` import alias to match usage in `register_test.go`.
2021-01-19 23:00:19 +01:00
8ccd4e1146
Fix flaky tests reported due to data race on grpc logging registration.
...
Example:
```
==================
WARNING: DATA RACE
Write at 0x000002178320 by goroutine 575:
google.golang.org/grpc/grpclog.SetLoggerV2()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/grpclog/loggerv2.go:70 +0x444
go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1.1()
/home/ptab/corp/etcd/server/embed/config_logging.go:119 +0x345
sync.(*Once).doSlow()
/usr/lib/google-golang/src/sync/once.go:66 +0x109
sync.(*Once).Do()
/usr/lib/google-golang/src/sync/once.go:57 +0x68
go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1()
/home/ptab/corp/etcd/server/embed/config_logging.go:109 +0x3b1
go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging()
/home/ptab/corp/etcd/server/embed/config_logging.go:174 +0x6af
go.etcd.io/etcd/server/v3/embed.(*Config).Validate()
/home/ptab/corp/etcd/server/embed/config.go:553 +0x55
go.etcd.io/etcd/server/v3/embed.StartEtcd()
/home/ptab/corp/etcd/server/embed/etcd.go:93 +0x84
go.etcd.io/etcd/tests/v3/integration.TestKVWithEmptyValue()
/home/ptab/corp/etcd/tests/integration/v3_kv_test.go:33 +0x18c
testing.tRunner()
/usr/lib/google-golang/src/testing/testing.go:1123 +0x202
Previous read at 0x000002178320 by goroutine 956:
[failed to restore the stack]
Goroutine 575 (running) created at:
testing.(*T).Run()
/usr/lib/google-golang/src/testing/testing.go:1168 +0x5bb
testing.runTests.func1()
/usr/lib/google-golang/src/testing/testing.go:1441 +0xa6
testing.tRunner()
/usr/lib/google-golang/src/testing/testing.go:1123 +0x202
testing.runTests()
/usr/lib/google-golang/src/testing/testing.go:1439 +0x612
testing.(*M).Run()
/usr/lib/google-golang/src/testing/testing.go:1347 +0x3c4
go.etcd.io/etcd/pkg/v3/testutil.MustTestMainWithLeakDetection()
/home/ptab/corp/etcd/pkg/testutil/leak.go:150 +0x38
go.etcd.io/etcd/tests/v3/integration.TestMain()
/home/ptab/corp/etcd/tests/integration/main_test.go:14 +0x272
main.main()
_testmain.go:349 +0x269
Goroutine 956 (finished) created at:
google.golang.org/grpc/internal/transport.newHTTP2Server()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/internal/transport/http2_server.go:288 +0x18a4
google.golang.org/grpc/internal/transport.NewServerTransport()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/internal/transport/transport.go:534 +0x2f5
google.golang.org/grpc.(*Server).newHTTP2Transport()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/server.go:726 +0x2ca
google.golang.org/grpc.(*Server).handleRawConn()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/server.go:693 +0x60f
google.golang.org/grpc.(*Server).Serve.func3()
/home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29 .1/server.go:663 +0x4c
==================
...
{"level":"info","ts":"2021-01-09T22:21:04.550+0100","caller":"embed/etcd.go:330","msg":"closed etcd server","name":"default","data-dir":"/tmp/etcd-017337431","advertise-peer-urls":["http://localhost:2380 "],"advertise-client-urls":["http://localhost:2379 "]}
--- FAIL: TestKVWithEmptyValue (1.08s)
v3_kv_test.go:62: my-namespace/foobar = data
v3_kv_test.go:62: my-namespace/foobar1 = data
v3_kv_test.go:62: namespace/foobar1 = data
v3_kv_test.go:72: foobar = data
v3_kv_test.go:72: foobar1 = data
v3_kv_test.go:87: delete keys:2
testing.go:1038: race detected during execution of test
```
2021-01-11 10:06:31 +01:00
c899316fe8
Adding clientv3 import alias to match usage in register_test.go.
2020-12-15 11:54:03 -06:00
f2a912a4e6
test: change channel operations to avoid potential goroutine leaks
...
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
2020-12-09 22:23:21 -05:00
aaf423e962
server: Update imports.
...
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
45b007b8b4
contrib,clientv3: Move contrib/recipies to clientv3/experimental/recipies/...
...
Recipies is set of patterns / primitives implementation on top of clientv3.
It's used by integration tests. It shouldn't be considered "server" code.
2020-10-22 11:10:07 +02:00
41557d9330
Merge pull request #12404 from ptabor/20201020-etcdctl-module
...
Modularization: etcdctl as a module
2020-10-21 21:48:38 +08:00
09679d29ad
etcdctl: Rename of imports after making etcdctl a module.
...
```
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/etcdctl|etcd/etcdctl/v3|g'
```
2020-10-21 11:15:35 +02:00
0a830c871e
clientv3test: fix network partition flaky test
2020-10-20 23:13:23 +08:00
e33c6dd9df
client/v3: Rename of imports
2020-10-20 10:13:06 +02:00
45e4306508
client: Move client to client/v2 as a module.
...
We make v2 client code a module go.etcd.io/etcd/client/v2.
Pretty mechanical change that can be summarized as:
mkdir client/v2
cd client/v2 && git mod init go.etcd.io/etcd/client/v2
git mv client/*.go client/v2/
find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'
+ fixing changelog, bom, go.mod etc.
2020-10-15 14:39:43 +02:00
de55bb6331
pkg: Rename imports after making 'pkg' a module
...
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
30811a06aa
etcdctl, clientv3/snapshot: snapshot (Restore,Status) code out of client
...
"snapshot" Restore/Status code was the only remaining dependency of client on 'server'
code. The code is solelly used by etcdctl. Long-term the snapshot code
should be migrated to 'etcdadm' style of tool such that we can
distinguish tool solelly depending on networking API vs. tools that
operation on etcd files directly.
We left snapshot.Save() code in clientv3.snapshot package, such that
clients can benefits from automated download&safe to file snapshot
functionality over the wire.
2020-10-10 14:14:57 +02:00
485030052a
tests/integration: Move misplaces integration tests.
...
There was undetected 'conflict' between
11ba1a61092c66612e0e
2020-10-09 09:39:55 +02:00
f67956cb7a
clientv3: Expose clientv3/examples close to the code.
...
Many of the tests had missing '// Output:' comment, so were not
runnable. They required fining.
2020-10-08 14:27:32 +02:00
dd45d04b2d
clientv3/concurrency: Expose examples close to the source-code.
2020-10-07 15:46:56 +02:00
