Compare commits

...

1684 Commits

Author SHA1 Message Date
8139dd3e55 version: 3.5.0-rc.1
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-06-09 17:57:52 -04:00
5d52faec47 Merge pull request #13095 from hexfusion/cp-13049
[release-3.5]: [Fix] --log-outputs relative path are not supported when --log-rotate-config-json is defined
2021-06-09 15:01:09 -04:00
98bbc0107f TestLogRotation add log output relative path
TestLogRotation add test log output relative path test
2021-06-09 14:12:07 -04:00
fed3bc3084 [Fix]滚动日志路径解析不正确
例如  rotate:test.log 路径解析不正确
2021-06-09 14:12:07 -04:00
b327edf521 Merge pull request #13092 from lilic/bump-client-golang-3.5
[release-3.5]: Update client_golang dependency to v1.11.0
2021-06-07 20:52:09 -04:00
9fee8bf95e Update client_golang dependency to v1.11.0 2021-06-07 15:54:57 +02:00
e721e4eb10 Merge pull request #13088 from gyuho/c
[backport 3.5] etcdserver: don't attempt to grant nil permission to a role
2021-06-04 16:22:03 -07:00
e27effa250 etcdserver: don't attempt to grant nil permission to a role
Prevent etcd from crashing when given a bad grant payload, e.g.:

$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/add
{"header":{"cluster_id":"14841639068965178418", ...
$ curl -d '{"name": "foo"}' http://localhost:2379/v3/auth/role/grant
curl: (52) Empty reply from server

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-06-04 15:34:22 -07:00
abe57c1aed Merge pull request #13085 from gyuho/c
[backport 3.5] etcdserver: don't activate alarm w/missing AlarmType
2021-06-04 15:12:28 -07:00
ae194c1470 etcdserver: don't activate alarm w/missing AlarmType
Narrowly prevent etcd from crashing when given a bad ACTIVATE payload, e.g.:

$ curl -d "{\"action\":\"ACTIVATE\"}" ${ETCD}/v3/maintenance/alarm
curl: (52) Empty reply from server
2021-06-04 14:21:04 -07:00
ab20aa29a0 version: 3.5.0-rc.0
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-06-03 21:39:37 -04:00
a6b37e2642 Merge pull request #13081 from wilsonwang371/20210603-backport13075
backport-3.5: server: skip unnecessary sprintf which executes proto.Size()
2021-06-03 13:53:49 -07:00
4563cebaa4 server: skip unnecessary sprintf which executes proto.Size()
(cherry picked from commit 11edc76b15)
2021-06-03 13:21:23 -07:00
3a82198773 Merge pull request #13079 from ptabor/20210604-deps-3.5
backport-3.5: Update of dependencies:zap, bbolt, grpc, ...
2021-06-03 12:18:27 -04:00
a5523be415 Update of dependencies:
- zap-1.17.0
  - bbolt-1.3.6
  - grpc -1.38
  - github.com/coreos/go-systemd/v22 v22.3.2
  - github.com/google/btree v1.0.1
  - github.com/json-iterator/go v1.1.11
2021-06-03 15:19:33 +02:00
fea5ace182 Merge pull request #13064 from serathius/v3.5-limit
Backport #13060 to release-3.5
2021-06-01 17:18:47 +02:00
9a18742bd7 etcdserver: Fix invalid count returned on Range with Limit 2021-06-01 16:12:53 +02:00
6cd7c02873 Merge pull request #13055 from ptabor/20210528-beckport-integration-fixes 2021-05-30 16:35:15 -04:00
3f13d3a2d5 integration.BeforeTest can be run without leak-detection. 2021-05-28 10:01:36 +02:00
7b244ae558 Merge pull request #13043 from ptabor/cherry-picks-3.5
backport-3.5: Expose clientv3.CreateDefaultZapLoggerConfig
2021-05-26 10:06:36 -07:00
4af7fc393c Expose clientv3.CreateDefaultZapLoggerConfig
Need raised in
f3f4259d3f.

BTW: Alligned the implementation of grpclogs with what we use in embed server, so reduced the client code dependencies.
2021-05-26 12:52:46 +02:00
b3f16d6691 version: 3.5.0-beta.4
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-25 20:48:48 -04:00
16214c3443 Merge pull request #13037 from ptabor/cherry-picks-3.5
Backport-3.5: seq reset + bucket as object
2021-05-25 01:27:48 -07:00
e6baf6d751 Represent bucket as object instead of []byte name.
Thanks to this change:
  - all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
  - buckets are strongly typed in backend/mvcc API.
2021-05-25 09:22:25 +02:00
8bddbdc1d6 Rename seq to bucket2seq. 2021-05-25 09:21:07 +02:00
d3b3228c1f Minor formatting fix on top of https://github.com/etcd-io/etcd/pull/12568 2021-05-25 09:21:07 +02:00
261f8b1daf backend: set seq flag for each bucket buffer 2021-05-25 09:21:06 +02:00
dd22bd747e Merge pull request #13036 from gyuho/cherry-pick
[release-3.5] server: set multiple concurrentReadTx instances share one txReadBuffer.
2021-05-24 18:00:09 -07:00
7a7d6f94a7 server: set multiple concurrentReadTx instances share one txReadBuffer.
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-24 16:59:42 -07:00
c0d1450b45 Merge pull request #13003 from gyuho/cherry-pick
[backport release-3.5] applyV2 should reapply on backend only once
2021-05-18 23:50:10 -07:00
c4ebac0c57 applyV2 should reapply on backend only once
During review of:  https://github.com/etcd-io/etcd/pull/12988 spotted
that PUT is actially writing to v3-backend.
If we are replaying WAL log, it might happened that backend's
applied_index is > than the WAL's log entry. In such situation we should
skip applying on backend V3.
I think both the methods (setVersion, setMembersAttributes) are in
practice idempotent so its not that 'serious' problem, but for
formal correctness adding the proper checks.
2021-05-18 23:16:59 -07:00
379c361bc6 version: 3.5.0-beta.3
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-18 09:44:35 -04:00
163d9fd6f4 Merge pull request #12998 from hexfusion/cp-12997
Manual cherry pick of #12997
2021-05-18 08:20:41 -04:00
d06c97209c scripts: add missing etcdutl to release pipeline
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-18 07:57:27 -04:00
1a8a063a31 version: 3.5.0-beta.2
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-18 06:32:19 -04:00
e46bcb5ad9 version: 3.5.0-beta.1
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-17 21:50:03 -04:00
535a9447ce Merge pull request #12990 from gyuho/backport
release-3.5: backport set version panic fix, ARM64 tests
2021-05-17 17:12:47 -04:00
84d9187dbf github/workflow: run arm64 tests for all branches startin from 3.5
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-17 13:52:11 -07:00
2042d2abc4 use v2 api to update cluster version 2021-05-17 13:50:38 -07:00
cf54fab42f version: v3.5.0-beta.0
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-17 13:29:57 -04:00
c49444d5d0 *: Remove Changelog
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-17 13:15:29 -04:00
932d42b027 Merge pull request #12971 from ptabor/20210514-split-etcdctl
Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files)
2021-05-17 14:07:20 +02:00
3f7a038656 Fixed documentation and scripts. 2021-05-17 11:54:03 +02:00
949c1c224b Updated changelog. 2021-05-17 11:54:03 +02:00
d99d0df5a5 Adding etcdutl test coverage. 2021-05-17 11:54:03 +02:00
b6a8ae8372 add etcdutl version. make it cov-tests compatible. 2021-05-17 11:54:03 +02:00
c09aca1ba4 Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files)
Motivation is as follows:

  - etcdctl we only depend on clientv3 APIs, no dependencies of bolt, backend, mvcc, file-layout
  - etcdctl can be officially supported across wide range of versions, while etcdutl is pretty specific to file format at particular version.
it's step towards desired modules layout, documented in: https://etcd.io/docs/next/dev-internal/modules/
2021-05-17 11:54:03 +02:00
1675101293 Merge pull request #12981 from ptabor/20210516-fix-release-script
Fix release script: add `go mod tidy` after go.mod changes
2021-05-17 11:35:21 +02:00
2712c787ab Fix release script: add go mod tidy
Seems that additinal refresh of go.sum files is needed during the
release process after editing the versions of modules.

Discovered the problem while I was testing whether etcdutl is
properly included in the target binaries using:

```
DRY_RUN=true REPOSITORY=\`pwd\` BRANCH='branch-with-etcdutl' ./scripts/release 3.5.0-foobar.2
```
2021-05-16 20:43:12 +02:00
f82b5cb776 Bring back original NewZapCoreLoggerBuilder api. (#12973)
The funcion signature has been changed in:
eafbc8c57e .
Instead we should have added new method `NewZapLoggerBuilder()`.
2021-05-16 10:26:10 -07:00
9501e8e874 Merge pull request #12980 from aaronjheng/typo
Fix typo in CHANGELOG
2021-05-16 16:18:46 +02:00
549a0f656f docs: Fix typo in CHANGELOG 2021-05-16 21:44:57 +08:00
adcf580137 Merge pull request #12976 from ptabor/20210516-fix-coverage
Fix coverage tests
2021-05-16 15:27:38 +02:00
099fd65821 Fix coverage test failure: e2e TestIssue6361.
Tested with:
```
(cd tests && COVERDIR='../../c' 'env' 'go' 'test' '-tags=cov' '-timeout' '30m' 'go.etcd.io/etcd/tests/v3/e2e' -run TestIssue6361 -v 2>&1 | tee log.log)
```
2021-05-16 10:58:41 +02:00
c7a76470d5 Fix path to the coverage folder for e2e tests. 2021-05-16 09:49:50 +02:00
13ef6fc343 Fix coverage tests
2 problems:
  - spawnCmdWithLogger was not implemented (when built with 'cov' tag)
  - the logic was depending on relative paths. We change it to absolute
to be able to run in the test-specific temporary directories.
2021-05-16 09:49:50 +02:00
bca0717f28 Rename master to main in README and other *.md files (#12977) 2021-05-15 23:46:05 -07:00
3ed0cb1dfc Update modules to more stable versions. (#12975) 2021-05-15 23:32:27 -07:00
a6be38306a Merge pull request #12974 from ptabor/20210515-fix-to-testing-env
Fixes to CI configs
2021-05-15 17:24:35 +02:00
27c74398cd Fixes to CI configs.
- Moves grpcproxy & functional tests to a separate workflows such thay can get restarted independently.
- Configures travis to run 1.16.3 go version (as travis cannot resolve 1.16 to the default).
2021-05-15 16:50:28 +02:00
85341e08f2 Merge pull request #12968 from serathius/logger-simplify
server: Simplify passing logger setup by passing only logger
2021-05-15 15:58:00 +02:00
ddc4f473c9 Merge pull request #12965 from ptabor/20210513-flake-in-e2e-debugging
Fix flakes due to 'shared dir' in grpcproxy e2e tests
2021-05-15 13:21:12 +02:00
d8550deb7f Fix pkg/proxy tests such that they don't leek goroutines and do close transports. 2021-05-14 22:45:59 +02:00
8981afb6f5 Fix unit tests logging config. 2021-05-14 22:45:57 +02:00
c18010cf42 etcdproxy e2e tests should run in dedicated directories.
So far all proxies were sharing the same (current) directory,
leading to tests flakes, e.g. due to certificates being overriden
in autoTLS mode.
2021-05-14 22:42:31 +02:00
582d02e7f5 E2E tests should log commandlines used to spawn etcd or etcd proxy binaries. 2021-05-14 22:42:31 +02:00
f5c26814ab Loggers to catch the e2e flake. 2021-05-14 22:42:31 +02:00
f2bc5eee91 Merge pull request #12969 from ptabor/20210514-etcdctl-step-by-step
Refactor common code from etcdctl (v2,v3) to pkg/corbautl.
2021-05-14 12:14:37 -07:00
00c6090110 Refactor common code from etcdctl (v2,v3) to pkg/corbautl.
Preparation for etcdutl split.
2021-05-14 14:16:53 +02:00
41ed74824e server: Simplify passing logger setup by passing only logger 2021-05-14 13:14:48 +02:00
eae7a845a7 Merge pull request #12964 from ptabor/main
Persists Term in the (bbolt) Backend
2021-05-14 06:35:21 +02:00
ab586cd463 Persists Term in the (bbolt) Backend.
Additional layer of protection, that allows to validate whether we
start replaying log not only from the proper 'index', but also of the
right 'term'.
2021-05-13 21:29:01 +02:00
e44fb40be5 Merge pull request #12962 from ptabor/20210513-write-conf-state
Save raftpb.ConfState in the backend.
2021-05-13 19:22:28 +02:00
6ce7f7409e Update changelog for storage format changes. 2021-05-13 19:21:25 +02:00
e2d67f2e3b Merge pull request #12956 from gyuho/rename-to-main
*: rename "master" branch references to "main" in source code
2021-05-13 08:26:33 -07:00
865df75714 Save raftpb.ConfState in the backend.
This makes (bbolt) backend a full feature snapshot in term of WAL/raft,
i.e. carries:
  - commit : (applied_index)
  - confState

Benefits:
  - Backend will be a sufficient point in time definition sufficient to
start replaying WAL. We have applied_index & confState in consistent
state.
  - In case of emergency a backend state can be used for recovery
2021-05-13 14:29:36 +02:00
3cb1ba4b2b Merge pull request #12954 from serathius/logger-new-ctx-client
client: Add logger argument to NewCtxClient
2021-05-13 09:03:38 +02:00
178fb876af Merge pull request #12943 from ptabor/20210430-v2-deprecation-flags
--v2-deprecation flag:  opt-in mode to validate that store-v2 has no user-content
2021-05-13 09:03:12 +02:00
fbbf284493 Merge pull request #12957 from spzala/writeoutformat
doc: fix output flag format
2021-05-12 11:50:53 -07:00
5f76adfc3e doc: fix output flag format
Add missing dash in the flag.
2021-05-12 14:13:08 -04:00
cd6d730dc5 raft: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:41:39 -07:00
b6562a24b6 .github: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:41:24 -07:00
5a0a1a4b45 markdowns: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:41:02 -07:00
3d4629c1fb hack,scripts: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:40:28 -07:00
6bab63e96b tools: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:39:53 -07:00
a8e1cf0433 client: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:38:50 -07:00
19fcefae70 tests: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:38:12 -07:00
77c8033739 server: rename "master" branch references
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:37:35 -07:00
79e3d7bd3e Add e2e tests for --v2-deprecation flag. 2021-05-12 19:20:49 +02:00
57d9666974 Merge pull request #12950 from gyuho/badge
README: add badges for all github action workflows
2021-05-12 09:18:28 -07:00
6fdd82f50e README: add badges for all github action workflows
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 09:16:31 -07:00
ead81df948 Disallow -v2-deprecation>'not-yet' combined with --enable-v2 2021-05-12 18:09:34 +02:00
7c508741b3 Adding --v2-deprecation flag. 2021-05-12 18:09:34 +02:00
f3b4a3e578 Detecting whether v2store is "empty" (metadata only). 2021-05-12 18:09:34 +02:00
1189ee3f3d client: Add logger argument to NewCtxClient 2021-05-12 16:40:55 +02:00
d982fe7176 Merge pull request #12952 from serathius/go1.16.4
Swtich CI to use newest available 1.16 Go version
2021-05-12 16:11:50 +02:00
c868bb97a9 github actions: Switch CI to use newest available 1.16 version 2021-05-12 15:25:16 +02:00
7ba53c71a9 Merge pull request #12953 from serathius/e2e-split
github actions: Move e2e to separate workflow and remove semaphoreci
2021-05-12 14:55:28 +02:00
f908e423bc github actions: Move e2e to separate workflow and remove semaphoreci 2021-05-12 14:17:17 +02:00
1929aa0a36 Merge pull request #12948 from serathius/logger
client: Allow setting zap logger in config
2021-05-12 13:49:48 +02:00
06afe87b34 client: Allow setting zap logger in config 2021-05-12 12:15:20 +02:00
e0a8484c8f Merge pull request #12941 from serathius/defrag
etcdserver: Implement running defrag if freeable space will exceed provided threshold (on boot)
2021-05-12 09:26:56 +02:00
980d70d050 Merge pull request #12947 from serathius/e2e
github actions: Run e2e tests
2021-05-12 09:23:06 +02:00
3b24496c2a Merge pull request #12925 from yangxuanjia/yxjetcd_check_resuming
check resuming len to avoid panic
2021-05-12 09:18:40 +02:00
270b60e536 CHANGELOG: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-11 19:32:31 -07:00
68b1e9f728 check resuming len to avoid panic 2021-05-12 09:51:59 +08:00
70f7e0d70f Merge pull request #12929 from gyuho/github-worker-arm64-support 2021-05-11 10:29:53 -07:00
450fd9a0b1 Merge pull request #12946 from serathius/badge
readme: Switch Travis CI test results badge to GitHub actions
2021-05-11 14:24:47 +02:00
f48609ae52 github actions: Run e2e tests 2021-05-11 14:20:46 +02:00
efc8505739 etcdserver: Implement running defrag if freeable space will exceed privided threshold 2021-05-11 14:00:29 +02:00
4bc1aa93e7 readme: Switch Travis CI test results badge to GitHub actions 2021-05-11 13:22:49 +02:00
7ae55cf981 Merge pull request #12942 from ptabor/20210510-expect_debug_true-for-e2e
Run e2e tests with: EXPECT_DEBUG='true'
2021-05-10 19:47:57 +02:00
04c9e8b531 Run e2e tests with: EXPECT_DEBUG='true'
The motivation is need to diagnose flakes like:
https://semaphoreci.com/etcd-io/etcd/branches/pull-request-12940/builds/2.

Now - when the logs are streamed through testing.T logger, the verbosity
is not that significant concern.
2021-05-10 17:46:31 +02:00
74d7d33b7c Merge pull request #12938 from serathius/test-go1.16.3
github actions: Make github actions go1.16.3 tests blocking and remove them from travis
2021-05-10 17:44:15 +02:00
e892d6de46 Merge pull request #12940 from ptabor/20210510-v2-warnings
Deprecate V2 API: --enable-v2 and --experimental-enable-v2v3 issue a warning
2021-05-10 16:43:46 +02:00
269f22c837 Deprecate V2 API: --enable-v2 and v2v3
Flags `--experimental-enable-v2v3` and '-enable-v2' will raise a warning in 3.5,
in 3.6 they are schedule for decomissioning, such that v2store can stop be written in 3.7.

Deprecation plan in: https://github.com/etcd-io/etcd/issues/12913
2021-05-10 16:19:52 +02:00
5bad818c70 Merge pull request #12919 from lilic/add-opentel-tracing 2021-05-10 07:09:33 -07:00
da33b4034f github actions: Make github actions go1.16.3 tests blocking and remove them from travis
* Leave Go tip tests and coverage in travis as one is not yet supported
2021-05-10 15:19:09 +02:00
6bc125c8d9 Merge pull request #12936 from serathius/pull_request
Trigger unit test workflow on pull_requests
2021-05-10 12:05:24 +02:00
3cdd24221b CHANGELOG-3.5.md: Add Tracing entry 2021-05-10 10:44:40 +02:00
1a718a958e Add initial Tracing with OpenTelemetry 2021-05-10 10:44:40 +02:00
f1082eec86 Trigger unit test workflow on pull_requests 2021-05-10 09:46:42 +02:00
b9c000fb8f server/etcdmain: officially supports "arm64"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-10 00:34:10 -07:00
a8f38ebd6f Merge pull request #12928 from gyuho/github-worker-arm64
github action: add ARM64 test
2021-05-10 00:32:59 -07:00
3a04b2b61f github action: add ARM64 test with README
README for self-hosted-linux-arm64-graviton2-tests.yml

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-10 00:28:41 -07:00
5f6f69c3cc Merge pull request #12932 from vvdvortsova/12821-fix-req-maybe-nil
embed: Added a check for a nil value of the request in ServeHTTP method
2021-05-10 08:35:14 +02:00
f5717016cf Merge pull request #12797 from lzhfromustc/3_23
server/etcdmain and tests: Fix goroutine leaks
2021-05-10 08:29:17 +02:00
c3303d94a4 embed: Added a check for an nil value of the request in ServeHTTP method. Fixed #12821 2021-05-09 20:45:43 +03:00
aeb9b5fc73 Merge pull request #12855 from ptabor/20210409-backend-hooks
(no)StoreV2 (Part 4): Backend hooks:  precommit updates consistency_index
2021-05-08 09:34:31 +02:00
6decbe15db Merge pull request #12774 from hexfusion/add-log-rotate
server: add support for log rotation
2021-05-07 12:18:10 -04:00
f178677765 CHANGELOG: server: add support for log rotation #12774
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-07 08:39:51 -04:00
82b2d5c67d server: add support for log rotation
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-05-07 08:39:51 -04:00
4f87333230 Merge pull request #12922 from serathius/actions
Migrate PR testing from travis to GitHub actions
2021-05-07 09:31:23 +02:00
e818f21790 Migrate PR testing from travis to GitHub actions 2021-05-06 17:06:57 +02:00
344c9f3930 Merge pull request #12896 from wilsonwang371/profiling-txn2
server: make applier use ReadTx() in Txn() instead of ConcurrentReadTx()
2021-05-06 01:59:14 -07:00
8af8f6af27 Merge pull request #12918 from iflamed/fix-doc-demo-bug
fix doc demo bug
2021-05-05 17:01:43 -04:00
a8a3eea56d fix doc demo bug
the service parameter not used.
2021-05-05 13:57:10 +08:00
7296a0708c Set ETCD_UNSUPPORTED_ARCH to ARM64 (#12557)
Signed-off-by: odidev <odidev@puresoftware.com>
2021-05-04 13:29:47 -07:00
b32bc914ff learner support snapshot RPC (#12890)
* learner support snapshot RPC

* CHANGELOG: update for 12890
2021-05-04 13:26:33 -07:00
98083ea914 server: add experimental flag for using shared buffer in transacton write 2021-05-04 11:59:08 -07:00
f1123d696c fixup! Unify shared code (and constants) with cindex package. 2021-05-04 18:21:24 +02:00
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
2fb6f0a74b Simplify lease management after cindex update is moved to 'hooks'. 2021-05-04 18:21:23 +02:00
2dbecea5b2 Simplify KVStore interaction with cindex thanks to hooks. 2021-05-04 18:21:23 +02:00
fe3254aee3 Remove explicit authStore->ConsistencyIndex updates, as they are taken care by hook. 2021-05-04 15:38:23 +02:00
50051675f9 Integrate backend::hooks with consistent_index.
Every transaction committed to backend is writing most recent consistent_index.
Makes sure that even automatically trigger commits of batch-transactions
stays "really" consistent a.d. the most recent WAL log index applied.
2021-05-04 15:38:23 +02:00
d53d2db1e2 Tests: Backend hooks support. 2021-05-04 15:38:22 +02:00
9f11b16b2d backend: Hooks interface & implementation. 2021-05-04 15:38:22 +02:00
a3fc535a5a Merge pull request #12914 from ptabor/20210430-read-membership
No-storeV2: Read membership information from the backend (Part5)
2021-05-04 10:15:01 +02:00
56154216b7 update variable declaration location 2021-05-03 10:30:15 -07:00
eb128d2d8c Merge pull request #12916 from wpedrak/pass-mlock-during-defrag
server: reapply `Mlock` flag after defrag
2021-05-03 09:01:50 -07:00
6623c008ee server: reapply Mlock flag after defrag 2021-05-03 11:01:02 +02:00
c46e96d519 Merge pull request #12910 from ptabor/20210430-zap-polish
Finish migration zap.NewExample -> zaptest.NewLogger(t) in integration tests
2021-04-29 23:12:56 +02:00
3acf6e35b2 Finish migration zap.NewExample -> zaptest.NewLogger(t) in integration tests 2021-04-29 22:32:26 +02:00
451f65d661 Merge pull request #12908 from ptabor/20210429-client-retry-logging
Clientv3 (retry interceptor) logs should use the configured logger
2021-04-29 19:25:04 +02:00
cedbea6c81 Merge pull request #12904 from wpedrak/limit_mlocked_memory
server: replace mlockall with `Mlock` in `--experimental-memory-mlock`
2021-04-29 18:21:24 +02:00
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
835643e6e2 Merge pull request #12906 from ptabor/20210429-etcdctl-v2-backup-cindex-fix
20210429 etcdctl v2 backup cindex fix
2021-04-29 18:01:45 +02:00
e908ef3da9 Merge pull request #12903 from lilic/exit-on-signal-int
etcdctl/ctlv3: Cleanup keys on signal interrupt.
2021-04-29 16:22:41 +02:00
205a1a442a Read membership information from the backend. 2021-04-29 13:45:45 +02:00
bc8d3f6639 Clientv3 (retry) logs should use the configured logger.
clientv3 logs (especially tests) were poluted with unattributed to testing.T log lines:

```
{"level":"warn","ts":"2021-04-29T12:42:11.055+0200","logger":"etcd-client","caller":"v3/retry_interceptor.go:64","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000fafc0/#initially=[unix://localhost:m10]","attempt":0,"error":"rpc error: code = ResourceExhausted desc = etcdserver: mvcc: database space exceeded"}
```

The reasons were 2 fold:
  - Interceptors were copying logger before "WithLogger" could modify it.
  - We were not propagating the loggers in a few testing contexts.
2021-04-29 12:57:09 +02:00
586b2383f4 etcdctl/ctlv3: Cleanup keys on signal interrupt of check perf
If ctc+c is pressed during the run of etcdctl check perf,
this deletes all the previously created keys that perf check created.
2021-04-29 12:55:36 +02:00
1145c57601 server: replace mlockall with Mlock in --experimental-memory-mlock
Implementation of `--experimental-memory-mlock` backed by `mlockall` syscall is replaced by `Mlock` flag (backed by mlock syscall) of bboltDB.
2021-04-29 12:08:52 +02:00
927b3a3152 server: replace mlockall with Mlock in --experimental-memory-mlock
Implementation of `--experimental-memory-mlock` backed by `mlockall` syscall is replaced by `Mlock` flag (backed by mlock syscall) of bboltDB.
2021-04-29 12:08:20 +02:00
e90504fe62 Unify shared code (and constants) with cindex package. 2021-04-29 11:51:25 +02:00
f53b70facb Embed: In case KVStoreHash verification fails, close the backend.
In case of failed verification, the server used to keep opened backend
(so the file was locked on OS level).
2021-04-29 11:51:25 +02:00
2ad893b110 Integrate verification into e2e tests. 2021-04-29 11:51:24 +02:00
911204cd76 Fix ETCDCTL_API=2 etcdctl backup --with-v3 consistent index consistency
Prior to this CL, `ETCDCTL_API=2 etcdctl backup --with-v3` was readacting WAL log
(by removal of some entries), but was NOT updating consistent_index in the backend.
Also the WAL editing logic was buggy, as it didn't took in consideration the fact
that when TERM changes, there can be entries with duplicated indexes in
the log. So its NOT sufficient to subtract number of removed entries to
get accurate log indexes.

The PR replaces removing and shifting of WAL entries with replacing them with an no-op entries.
Thanks to this consistent-index references are staying up to date.

The PR also:
  - updates 'verification' logic to check whether consistent_index does not lag befor last snapshot
  - env-gated execution of verification framework in `etcdctl backup`.

Tested with:
```
(./build.sh && cd tests && EXPECT_DEBUG=TRUE 'env' 'go' 'test' '-timeout=300m' 'go.etcd.io/etcd/tests/v3/e2e' -run=TestCtlV2Backup --count=1000 2>&1 | tee TestCtlV2BackupV3.log)
```
2021-04-29 11:51:24 +02:00
adc365e14f etcdctl2: backup command logging cleanup (zap) 2021-04-28 23:26:17 +02:00
8d8d0377a2 server: applier uses ReadTx instead of ConcurrentTx 2021-04-28 11:06:24 -07:00
ed4a87d541 Merge pull request #12901 from ptabor/20210427-verification
Verification of persisted data
2021-04-28 08:45:15 +02:00
7107cb9f86 fixup! Create 'datadir' package responsible for paths. 2021-04-28 08:44:06 +02:00
c4b13a5c83 Integrate verification framework
Verification framework is integrated with:
  - integration tests (by default)
  - `ETCD_VERIFY=all etcdctl snapshot restore` command
  - etcd shutdown when running with `ETCD_VERIFY=all` env.
2021-04-28 07:56:16 +02:00
47b28b600a Verification package: Verified given data-dir.
For now verifies whete Backend.cindex is consistent with WAL log,
but should get expanded to cover memberships & revisions.
2021-04-28 07:56:15 +02:00
6f8f506cf4 Create 'datadir' package responsible for paths. 2021-04-28 07:56:13 +02:00
d2722ff955 Merge pull request #12820 from ptabor/20210326-membership-from-be
(no)StoreV2 (Part 2): Prepare to read membership information from backend
2021-04-28 07:53:42 +02:00
067521981e v2 etcdctl backup: producing consistent state of membership 2021-04-27 19:34:34 +02:00
a70386a1a4 Simplify membership interface: Does not pass the 'unused' token. 2021-04-27 17:17:31 +02:00
4725567d5e e2e tests: More logging and expect adopted to 3.4. 2021-04-27 17:17:31 +02:00
7ae3d25f91 Membership: Add additional methods to trim/manage membership data in backend. 2021-04-27 17:17:31 +02:00
aa6597384b etcd-dump-logs: Print full confState as json for debugging purposes. 2021-04-27 17:17:31 +02:00
768da490ed sever: v2store deprecation: Fix etcdctl snapshot restore to restore
correct 'backend' (bbolt) context in aspect of membership.

Prior to this change the 'restored' backend used to still contain:
  - old memberid (mvcc deletion used, why the membership is in bolt
bucket, but not mvcc part):
    ```
	mvs := mvcc.NewStore(s.lg, be, lessor, ci, mvcc.StoreConfig{CompactionBatchLimit: math.MaxInt32})
	defer mvs.Close()
	txn := mvs.Write(traceutil.TODO())
	btx := be.BatchTx()
	del := func(k, v []byte) error {
		txn.DeleteRange(k, nil)
		return nil
	}

	// delete stored members from old cluster since using new members
	btx.UnsafeForEach([]byte("members"), del)
    ```
  - didn't get new members added.
2021-04-27 17:17:30 +02:00
06d6f09a8a Merge pull request #12894 from MakDon/patch-1
etcdserver/mvcc: update tw.trace.Step condition
2021-04-27 00:19:10 -07:00
c4f7d578d8 Merge pull request #12898 from tangcong/add-disk-io-failure-case
functional: add disk io failure case
2021-04-26 14:37:53 -07:00
4fb22093a6 functional: add SHORT_TTL_LEASE_EXPIRE checker 2021-04-26 20:00:45 +08:00
16e38e49a9 functional: add FAILPOINTS_WITH_DISK_IO_LATENCY case 2021-04-26 12:07:05 +08:00
370f9cf3b9 fix: failed to get failpoints from member 2021-04-26 11:44:53 +08:00
ddcb463822 etcdserver/mvcc: update trace.Step condition 2021-04-25 23:07:45 +08:00
9a3aff6d8f Merge pull request #12889 from ptabor/20210423-deflake-TestFirstCommitNotification
Deflake: TestFirstCommitNotification
2021-04-23 14:55:44 +02:00
9f559775b8 Deflake: TestFirstCommitNotification
Infrequently the test flaked. Reproducable with:

```
  go test go.etcd.io/etcd/tests/v3/integration --run TestFirstCommitNotification --count=500
```

The moveLeader finishes when configchange is commited by quorum.
It doesn't guarantee that the 'empty' record was committed by the new leader.
From time to time happened that appliedLeaderIndex was returning 9
(without empty entry) and the test flaked. In healthy case the
appliedIndex returned 10.

Fixed by putting kv pair after leader change. The pair is guaranteed
to be stored on index when put finishes (so the empty entry as well).
2021-04-23 13:22:15 +02:00
bd4f8e2b6c Merge pull request #12885 from ptabor/20210422-error-codes-context
Errors: `context cancelled` or `context deadline exceeded` are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown'
2021-04-23 00:45:59 +02:00
9a4b2bdccc Errors: context cancelled or context deadline exceeded are exposed as codes.Canceled, codes.DeadlineExceeded instead of 'codes.Unknown' 2021-04-22 14:35:24 +02:00
cc52d994b7 Merge pull request #12883 from ptabor/20210421-backend-refactor-testing
mvcc/backend tests: Refactor: Do not mix testing&prod code.
2021-04-21 12:29:01 +02:00
d7d110b5a8 mvcc/backend tests: Refactor: Do not mix testing&prod code. 2021-04-21 09:43:13 +02:00
ea287dd9f8 Merge pull request #12854 from ptabor/20210410-shouldApplyV3
(no)StoreV2 (Part 3): Applying consistency fix: ClusterVersionSet (and co) might get not applied on v2store
2021-04-21 09:31:38 +02:00
ce3dae6f83 Merge pull request #12873 from ptabor/20210417-test-docker-gcloud
Makefile: Use `gcloud auth configure-docker` instead of `gcloud docker ...` for test-images
2021-04-21 09:30:02 +02:00
0f2c940f64 Merge pull request #12880 from chaochn47/exclude_alarms_from_health_check
etcdhttp/metrics.go: exclude alarms from health check conditionally with `?exclude=NOSPACE`
2021-04-20 21:18:15 -04:00
b321c48b2d Update CHANGELOG-3.5.md 2021-04-20 13:17:26 -07:00
140ea4fa29 etcdhttp/metrics.go: exclude alarms from health check conditionally with ?exclude=NOSPACE 2021-04-20 13:17:09 -07:00
8162d9cbdf Merge pull request #12876 from qsyqian/branch_management_link
doc: fix branch management link
2021-04-19 23:08:42 +02:00
334e696f21 Merge pull request #12878 from lilic/fix-docker-release
Makefile, build.sh: Fix build process
2021-04-19 10:12:42 -07:00
51c28fc475 build.sh: Adjust building etcdctl to be same as etcd binary
This fixes so that the ENV vars are taken in the same way as for etcd
binary.
2021-04-19 17:51:46 +02:00
81652d16ef Makefile: Fix build-docker-release-master
Since the Dockerfile files are now per arch, this adjusts to detect ARCH
and builds docker release from the Dockerfile.<ARCH> file.
2021-04-19 17:47:03 +02:00
11249fdee9 Merge pull request #12874 from ptabor/20210417-go1.16.3
Update go for 3.5: 1.15.x -> 1.16.3
2021-04-19 16:51:49 +02:00
3423a949c0 Update go for 3.5: 1.15 -> 1.16.(3).
https://github.com/etcd-io/etcd/issues/12732
2021-04-19 16:50:54 +02:00
2cbd86b102 doc: fix branch management link
Fixes #12875
2021-04-19 15:25:19 +08:00
2f77a1ac67 Merge pull request #12864 from ssbostan/master
client: fix check datascale command for https endpoints
2021-04-18 17:44:07 +02:00
06ba0fc5a2 Merge pull request #12846 from pyiyun/fix-snaptmpfile-bug
etcdserver: remove temp files in snap dir when etcdserver starting
2021-04-17 12:58:46 +02:00
5ad8880d77 Makefile: Use gcloud auth configure-docker instead of gcloud docker. 2021-04-17 10:11:09 +00:00
97a8affdd3 fix util.go file 2021-04-17 14:24:56 +04:30
80586c5b47 etcdserver/util.go: reduce memory when logging range requests (#12871)
* etcdserver/util.go: reduce memory when logging range requests

Fixes #12835

* Update CHANGELOG-3.5.md
2021-04-16 16:39:34 -07:00
5744cdf199 Merge pull request #12870 from ptabor/20210416-fix-flake-TestSnapshotV3RestoreMultiMemberAdd-master
Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
2021-04-16 21:49:46 +02:00
17b982382e Fix TestSnapshotV3RestoreMultiMemberAdd flakes (leaks)
- most important: unix's socket transport should not keep idle
connections. For top-level Transport we close them using:
    f3c518025e/server/etcdserver/api/rafthttp/transport.go (L226)
    but currently we don't have access to close them witing the nest (unix) transport. Short idle deadline is good enough.

  - Use dialContext (instead of dial) to make sure context is passed down the stack
  - Make sure Context is cancelled as soon as the operation is done in pipeline
  - nit: use dedicated method to yeld goroutines.

Tested with:
```
d=$(date +"%Y%m%d_%H%M")
(cd tests && go test --timeout=60m ./integration/snapshot -run TestSnapshotV3RestoreMultiMemberAdd -v --count=180 2>&1 | tee log_${d}.log)
```

There were transports & cmux leaked:

```
   leak.go:118: Test appears to have leaked a Transport:
        internal/poll.runtime_pollWait(0x7f6c5c3784c8, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc003296298, 0x72, 0x0, 0x18, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc003296280, 0xc0031f60a8, 0x18, 0x18, 0x18, 0xc0009056e2, 0x203000)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc000010258, 0xc0031f60a8, 0x18, 0x18, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        github.com/soheilhy/cmux.(*bufferedReader).Read(0xc0003d24e0, 0xc0031f60a8, 0x18, 0x18, 0xc0003d24d0, 0xc0009056e2, 0xc000278400)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/buffer.go:53 +0x12d
        github.com/soheilhy/cmux.hasHTTP2Preface(0x1367e20, 0xc0003d24e0, 0x7f6c5c699f40)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/matchers.go:195 +0x8a
        github.com/soheilhy/cmux.matchersToMatchWriters.func1(0x7f6c5c699f40, 0xc000010258, 0x1367e20, 0xc0003d24e0, 0xc000010258)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:128 +0x39
        github.com/soheilhy/cmux.(*cMux).serve(0xc003228690, 0x138c410, 0xc000010258, 0xc00327f740, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:192 +0x1e7
        created by github.com/soheilhy/cmux.(*cMux).Serve
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:179 +0x191

        internal/poll.runtime_pollWait(0x7f6c5c60f3f0, 0x72, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/runtime/netpoll.go:222 +0x55
        internal/poll.(*pollDesc).wait(0xc000d53018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:87 +0x45
        internal/poll.(*pollDesc).waitRead(...)
        	/usr/lib/google-golang/src/internal/poll/fd_poll_runtime.go:92
        internal/poll.(*FD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/internal/poll/fd_unix.go:166 +0x1d5
        net.(*netFD).Read(0xc000d53000, 0xc000cfd000, 0x1000, 0x1000, 0x3, 0x3, 0x1000000000001)
        	/usr/lib/google-golang/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc00031a570, 0xc000cfd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        	/usr/lib/google-golang/src/net/net.go:183 +0x91
        net/http.(*persistConn).Read(0xc00093b320, 0xc000cfd000, 0x1000, 0x1000, 0x577750, 0x60, 0x0)
        	/usr/lib/google-golang/src/net/http/transport.go:1933 +0x77
        bufio.(*Reader).fill(0xc005702fc0)
        	/usr/lib/google-golang/src/bufio/bufio.go:101 +0x108
        bufio.(*Reader).Peek(0xc005702fc0, 0x1, 0xc00077c660, 0xc003b082a0, 0xc000d08de0, 0x5ae586, 0x11dd6c0)
        	/usr/lib/google-golang/src/bufio/bufio.go:139 +0x4f
        net/http.(*persistConn).readLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2094 +0x1a8
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1754 +0xdaa

        net/http.(*persistConn).writeLoop(0xc00093b320)
        	/usr/lib/google-golang/src/net/http/transport.go:2393 +0xf7
        created by net/http.(*Transport).dialConn
        	/usr/lib/google-golang/src/net/http/transport.go:1755 +0xdcf

        sync.runtime_Semacquire(0xc0059ba868)
        	/usr/lib/google-golang/src/runtime/sema.go:56 +0x45
        sync.(*WaitGroup).Wait(0xc0059ba860)
        	/usr/lib/google-golang/src/sync/waitgroup.go:130 +0x65
        github.com/soheilhy/cmux.(*cMux).Serve.func1(0xc003228690, 0xc0059ba860)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:158 +0x56
        github.com/soheilhy/cmux.(*cMux).Serve(0xc003228690, 0x13698c0, 0xc00377a0f0)
        	/home/ptab/private/golang/pkg/mod/github.com/soheilhy/cmux@v0.1.5/cmux.go:173 +0x115
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func1(0xc0007cc360, 0x122b75f)
        	/home/ptab/corp/etcd/server/embed/etcd.go:518 +0x2b9
        go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers.func3(0xc00036d080, 0xc0059330a0)
        	/home/ptab/corp/etcd/server/embed/etcd.go:549 +0x182
        created by go.etcd.io/etcd/server/v3/embed.(*Etcd).servePeers
        	/home/ptab/corp/etcd/server/embed/etcd.go:543 +0x73a
--- FAIL: TestSnapshotV3RestoreMultiMemberAdd (17.74s)
```
2021-04-16 20:17:28 +02:00
28a490b09c etcdserver: remove temp files in snap dir when etcdServer starting
When etcd exits abnormally, tmp files will remain in snap dir, so clean up tmp files in snap dir when etcdserver starting.

Fixes #12837
2021-04-16 20:30:04 +08:00
a9c4301c1e fix check datascale command for https endpoints 2021-04-16 03:51:04 +04:30
f3c518025e Merge pull request #12861 from ptabor/20210413-test-logging-fixes
Embedded server should not mess global loggers (by default)
2021-04-16 00:07:35 +02:00
b47c5fcc12 Address review comments a.d. logging. 2021-04-15 17:54:37 +02:00
c73da740fa Merge pull request #12862 from lilic/enable-race
.travis.yaml: Enables race in the tests
2021-04-15 17:37:03 +02:00
2843fded06 .travis.yaml: Enables race in the tests
As the TestV3WatchRestoreSnapshotUnsync flakiness was fixed, this
removes the TODO and enables the race again.
2021-04-15 14:32:52 +02:00
fad6391745 Configure proper logging for grpc integration tests. 2021-04-14 12:47:38 +02:00
d72f7ef5cc Give control to Embedded servers whether they override global loggers
So far each instance of embed server was overriding the grpc loggers and zap.global loggers.
It's counter intutitive that last created Embedded server was 'wining' and more-over it was breaking grpc expectation to change it "only" before the grpc stack is being used.

This PR introduces explicit call: `embed.Config::SetupGlobalLoggers()`, that changes the loggers where requested. The call is used by etcd main binary.

The immediate benefit from this change is reduction of  test flakiness, as there were flakes due to not a proper logger being used across tests.
2021-04-14 12:47:38 +02:00
eafbc8c57e Update zap logging dependency.
In particular bring up zapgrpc V2 code:
89e382035d
https://pkg.go.dev/google.golang.org/grpc/grpclog#LoggerV2
2021-04-14 12:15:48 +02:00
57a092b45d Merge pull request #12859 from tomwilkie/fix-mixin
Fix the mixin.
2021-04-13 23:05:52 +02:00
d69e46ea47 Make ShouldApplyV3 an enum - not bool 2021-04-13 23:01:03 +02:00
562d645ac9 Fix the mixin.
Signed-off-by: Tom Wilkie <tom@grafana.com>
2021-04-13 19:38:55 +01:00
4388bfc925 Merge pull request #12858 from hnlq715/patch-1
client: fix doc typo
2021-04-13 17:33:43 +02:00
5db0070150 Update doc.go 2021-04-13 22:22:42 +08:00
d7e971e8d8 client: fix doc typo 2021-04-13 16:33:24 +08:00
b1c04ce043 Applying consistency fix: ClusterVersionSet (and co) might get no applied on v2store
ClusterVersionSet, ClusterMemberAttrSet, DowngradeInfoSet functions are
writing both to V2store and backend. Prior this CL there were
in a branch not executed if shouldApplyV3 was false,
e.g. during restore when Backend is up-to-date (has high
consistency-index) while v2store requires replay from WAL log.

The most serious consequence of this bug was that v2store after restore
could have different index (revision) than the same exact store before restore,
so potentially different content between replicas.

Also this change is supressing double-applying of Membership
(ClusterConfig) changes on Backend (store v3) - that lackilly are not
part of MVCC/KeyValue store, so they didn't caused Revisions to be
bumped.

Inspired by jingyih@ comment:
https://github.com/etcd-io/etcd/pull/12820#issuecomment-815299406
2021-04-12 09:43:48 +02:00
7f97dfd45a Merge pull request #12795 from wpedrak/resend-read-index-on-first-commit-in-term
etcdserver: resend ReadIndex request on empty apply request
2021-04-09 16:41:54 +02:00
08ea9cb756 etcdserver: integration test covering first commit in current term notification 2021-04-09 16:05:02 +02:00
3991a8c9fa etcdserver: replace forceVersionC with FirstCommitInTermNotify 2021-04-09 11:30:42 +02:00
3d485faac5 etcdserver: resend ReadIndex request on empty apply request
Empty apply indicates first commit in current term. It is first time when follower is sure, that it's ReadIndex request can be processed.
2021-04-09 11:30:42 +02:00
bad0b4d513 Merge pull request #12823 from mtulio/chore/dash-var-refresh
chore/dash-var-refresh: change default refresh to 2(time range)
2021-04-08 15:14:53 +02:00
aeeecc06cf fix/dash-var-refresh: add const and description 2021-04-08 10:12:41 -03:00
63c25bf378 Merge pull request #12804 from ptabor/20210326-v3-publish
server: v2store deprecation: Prepare to use publishV3 instead of publish V2.
2021-04-08 09:22:22 +02:00
e776efbb2a Merge pull request #12828 from ptabor/20210404-embed-etcd
embed: etcd.Close() is closing Errc() channel as well.
2021-04-08 01:20:07 +02:00
5da9cac193 embed: etcd.Close() is closing Errc() channel as well.
Inspired by https://github.com/etcd-io/etcd/pull/9612 by purpleidea@.
2021-04-08 01:19:13 +02:00
cc0f812f51 server_test.go: Use context.Background() instead of TODO in tests.
Suggected in: https://golang.org/pkg/context/#Background
2021-04-08 01:15:16 +02:00
d4a8093ea5 Switch release-test (upgrade test) to use etcd 3.4 (instead of 3.3) as upgrade-base. 2021-04-08 01:15:16 +02:00
931af493cf Merge pull request #12830 from ptabor/20210405-split-pkg
Split client/pkg as dedicated low-dependencies module for client
2021-04-08 01:12:17 +02:00
816d332d81 Merge pull request #12830 from ptabor/20210405-split-pkg
Split client/pkg as dedicated low-dependencies module for client
2021-04-08 00:48:41 +02:00
24d2a9da70 test.sh: Turn off enforcement for minimal go.sum.
It seems that generated go.sum differs between different go versions.
This was introducing noise in the tests.
2021-04-07 20:26:10 +02:00
e24e72c5c8 Merge pull request #12836 from chalin/chalin-contrib-lock-2021-04-06
Contrib lock example
2021-04-07 20:23:08 +02:00
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
7168409e6b Merge pull request #12829 from garenchan/dev/etcdctl-lock
etcdctl: lock return exit code of exec-command
2021-04-06 23:46:20 +02:00
2ba69de281 Contrib lock example 2021-04-06 15:21:01 -04:00
c047ed593c etcdctl: lock return exit code of exec-command
Sometimes we expect to get the exit code of the command being
executed.
2021-04-06 14:34:31 +08:00
0b81fdf418 Move pkg/* used by client to client/pkg. 2021-04-05 23:47:27 +02:00
dfb03aba0b Merge pull request #12826 from allenporter/reporting-bugs
Fix "reporting bugs" link in documentation, which currently 404s
2021-04-02 11:04:38 +02:00
1906cfdc48 etcdserver: Fix "reporting bugs" link in documentation, which currently 404s
Update the "reporting bugs" link to be the current correct url from the documentation, replacing a dash with an
underscore in the URL.
2021-04-01 19:24:15 -07:00
d2bc5343fb chore/dash-var-refresh: change default refresh to 2(time range) 2021-04-01 00:06:57 -03:00
d18ac20080 Merge pull request #12766 from mrajashree/master
Update cert generation Makefile to include configurable cert file names
2021-03-30 12:59:41 +02:00
a49c7ec4fd Merge pull request #12817 from cwedgwood/3.4-changelog
CHANGELOG-3.4 updates
2021-03-30 12:53:02 +02:00
5d05be31aa CHANGELOG-3.4 updates 2021-03-29 11:38:25 -07:00
7899f76dce Merge pull request #12812 from hexfusion/bump-cl-3.2
CHANGELOG: release 3.2
2021-03-29 14:30:21 -04:00
b04477a580 CHANGELOG: release 3.2
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-28 22:19:47 -04:00
a40a6e9ad8 Merge pull request #12805 from ptabor/20210326-minor-test-fixes
tests: logging & temp-dir fixes
2021-03-28 19:14:13 -04:00
52b81bb975 Merge pull request #12807 from ptabor/20210327-update-cmux-stable
Update dependencies -> in particular cmux to stable 1.15.0
2021-03-28 19:12:43 -04:00
578ffd5783 Merge pull request #12790 from nvanbenschoten/reduce-raft-proto-size
raft: disable XXX_NoUnkeyedLiteral, XXX_unrecognized, and XXX_sizecache fields in protos
2021-03-27 21:08:50 +01:00
f290ab2e60 Update dependecies:
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
2021-03-27 20:48:33 +01:00
c49807f59e Update cmux to 1.5.0.
Executed using:
```
./scripts/update_dep.sh github.com/soheilhy/cmux v0.1.5
```
2021-03-27 11:18:13 +01:00
7665bd764e scripts/update_dep.sh: Script to update selected versions of dependencies. 2021-03-27 11:17:56 +01:00
55ccbe62a2 membership/cluster_test: Use zaptest logger. 2021-03-26 13:54:59 +01:00
03f55eeb2c Make NewTmpBackend use testing tmp location (so cleanup). 2021-03-26 13:54:55 +01:00
8ee1dd9e23 Merge pull request #12735 from ptabor/20210227-reduce-store-v2-clean
no-store_v2:  Store ConfState as part of WAL log snapshot
2021-03-25 08:23:00 +01:00
4d4c84e014 server: Written Snapshot's to WAL contains populated ConfState.
This will (among others) allow etcd-3.6 to not depend on store_v2 .snap files at all,
as WAL + db file will be self-sufficient.
2021-03-25 00:31:44 +01:00
fce0c192eb Regenerate protos. 2021-03-25 00:31:44 +01:00
9180e8e56e wal: Update WAL Snapshot format to contain ConfState. 2021-03-25 00:31:44 +01:00
65d6a1f4bf Merge pull request #12801 from ptabor/20210323-try-upgrade-proto
Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1, grpc: 1.32->1.36
2021-03-24 23:57:56 +01:00
d0962f10ae Adopt grpc_testing to grpc-1.36. 2021-03-24 23:31:03 +01:00
a60676686b Update dep: grpc: 1.32.0 -> 1.36.0. 2021-03-24 22:27:55 +01:00
45fb7b41d6 Update dep: github.com/golang/protobuf v1.3.5 ->  v1.5.1
Thanks to https://go-review.googlesource.com/c/protobuf/+/300869/ its
feasible now.
2021-03-24 22:27:12 +01:00
7d7a9c6f23 codec.go: should use google runtime
golang-proto runtime can deal with both: gogo & golang generated
protobufs. It does not work vice-versa with protobuf-1.5.1.
2021-03-24 22:06:47 +01:00
62d3eded87 test.sh: Temporarilly disable the "eeshew you" fmt check. 2021-03-24 20:40:13 +01:00
1b300cd9e2 Merge pull request #12796 from joelsmith/master
*: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh
2021-03-24 10:46:39 +01:00
21188b5e0d Merge pull request #12798 from gyuho/security
security: add private distributor list, and its application template
2021-03-23 11:34:50 -07:00
f18f609bcd security: add private distributor list, and its application template
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-03-23 11:34:15 -07:00
19f7c6ef3e *: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
2021-03-23 11:48:06 -06:00
3de6b382df server&tests: Fix goroutine leaks 2021-03-23 12:52:59 -04:00
4f34f14830 Merge pull request #12762 from wpedrak/postpone-read-index
raft: postpone MsgReadIndex until first commit in the term
2021-03-23 15:57:31 +01:00
30ce6067da Merge pull request #12780 from wpedrak/read_index_retry
Read index retry
2021-03-23 13:47:41 +01:00
758ff0163c raft: postpone MsgReadIndex until first commit in the term
Fixes #12680
2021-03-23 12:28:42 +01:00
456e129422 Merge pull request #12793 from gliptak/patch-1
Correct ineffassign
2021-03-23 09:19:26 +01:00
49d19efd82 Correct ineffassign 2021-03-22 14:11:30 -04:00
e51c697ec6 raft: disable XXX_NoUnkeyedLiteral, XXX_unrecognized, and XXX_sizecache fields in protos
This commit removes the `XXX_NoUnkeyedLiteral`, `XXX_unrecognized`, and
`XXX_sizecache` auto-generated fields from generated protobuf structs in
the raft package. This was done for all of the same reasons CockroachDB
removed the generation of these fields in https://github.com/cockroachdb/cockroach/pull/38404.
They come with very limited advantages but moderate disadvantages.

`XXX_NoUnkeyedLiteral` and `XXX_sizecache` were only enabled recently in
cc7b4fa, and this appears to have been unintentional. Meanwhile,
`XXX_unrecognized` has been around for longer and has arguably more
reason to stay because it can assist with forwards compatibility.
However, any real mixed-version upgrade story for this package is mostly
untold at this point, and keeping this field seems just as likely to
cause unexpected bugs (e.g. a field was propagated but not updated
correctly when passed through an old version) as it seems to fix real
issues, so it also doesn't warrant its cost.

This reduces the in-memory representation size of all Raft protos.
Notably, it reduces the memory size of an `Entry` proto from *80 bytes*
to *48 bytes* and the memory size of a `Message` proto from *392 bytes*
to *264 bytes*. Both of these structs are used frequently, and often in
slices, where this wasted space really starts to add up.

This was motivated by a regression in microbenchmarks in CockroachDB due
to cc7b4fa, which was caught in https://github.com/cockroachdb/cockroach/issues/62212.
2021-03-20 03:24:18 -04:00
ebc01743df raft: Add TestProtoMemorySizes test
This test checks the in-memory size of each proto struct to detect
unexpected changes to their memory representation.
2021-03-20 03:23:57 -04:00
8469108548 Merge pull request #12782 from ptabor/20210316-fixes
Integration: Test flakiness fixes
2021-03-19 20:10:11 +01:00
18321a0c85 Merge pull request #12786 from ptabor/20210318-debug-env
client: Bring back ETCD_CLIENT_DEBUG variable interpretation.
2021-03-18 21:55:51 +01:00
e2b5835e64 Fix go-tip: (1.17) fmt error:
```
gofmt' started at Thu Mar 18 18:06:00 UTC 2021
transport/sockopt_unix.go
diff -u transport/sockopt_unix.go.orig transport/sockopt_unix.go
--- transport/sockopt_unix.go.orig	2021-03-18 18:06:01.667483834 +0000
+++ transport/sockopt_unix.go	2021-03-18 18:06:01.667483834 +0000
@@ -1,3 +1,4 @@
+//go:build !windows
 // +build !windows

 package transport
transport/sockopt_windows.go
diff -u transport/sockopt_windows.go.orig transport/sockopt_windows.go
--- transport/sockopt_windows.go.orig	2021-03-18 18:06:01.667483834 +0000
+++ transport/sockopt_windows.go	2021-03-18 18:06:01.667483834 +0000
@@ -1,3 +1,4 @@
+//go:build windows
 // +build windows
```
2021-03-18 19:44:28 +01:00
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
0a02a107ea Merge pull request #12787 from wpedrak/staticcheck_fixes
*: over 20 staticcheck fixes
2021-03-18 09:52:21 -07:00
dac6e37ea1 *: over 20 staticcheck fixes 2021-03-18 15:06:17 +01:00
2932969b91 Merge pull request #12781 from ptabor/20210315-flaki-balancer
Integration tests: Use testing.T logger through zap for grpc
2021-03-18 08:30:54 +01:00
63c51709b4 Merge pull request #12784 from ptabor/20210317-readme-go-version
README: Update required go version.
2021-03-17 15:41:21 +01:00
7d53a06957 README: Update required go version. 2021-03-17 15:37:07 +01:00
725a8c5e02 Enable configuring delegated zap-logging for embed server. 2021-03-17 08:17:36 +01:00
6657d5907c Make sure all integration tests have BeforeTest.
The CL disallows to create NewCluster in tests without BeforeTest.
2021-03-16 23:50:01 +01:00
809e7629ed Add integration.BeforeTest to all missing tests. 2021-03-16 23:50:01 +01:00
a84bd093b0 Integration with grpc-settable logger. 2021-03-16 22:50:41 +01:00
18382aa234 Fix 2 sources of leaked memory: embed server HTTP & v3_snapshot.leasser. 2021-03-16 22:20:00 +01:00
e9779231ec server: add 500ms retries to ReadIndex requests for l-reads
It is second approach (with first being #12762) to solve #12680
2021-03-16 16:34:15 +01:00
a57e967d84 Integration test flakes fixes. 2021-03-16 16:08:18 +01:00
4b21e38381 refactored l-read loop in v3_server.go 2021-03-16 11:03:45 +01:00
e599f4a482 Merge pull request #12775 from ptabor/20210314-zip
etcd-raft-zap logger fixes.
2021-03-14 11:37:11 -07:00
1e7c1805d8 Unify logic of building raft-loggers for etcd.
1. We had the same code copied 3 times.
2. For no good reason the code was not reusing existing logger if this one is given.
2021-03-14 16:02:50 +01:00
44bd22307e Merge get_logger() & Logger() method. 2021-03-14 14:05:17 +01:00
527c765ece Merge pull request #12773 from ptabor/20210310-test-fixes
Minor test fixes
2021-03-14 13:36:19 +01:00
de67806175 mend 2021-03-14 13:35:47 +01:00
67491a00ea e2e/expect: In case of sut process failure, print last 40lines of logs. 2021-03-13 23:41:29 +01:00
a47c18d30a Fix 2 remaining 'defer AfterTest' calls. 2021-03-13 23:41:29 +01:00
0c1e6d05e7 Merge pull request #12772 from ptabor/20210312-3.5-todos
Fix/remove broken: TestMetricDbSizeDefragDebugging
2021-03-12 23:11:25 +01:00
b406647dd7 Fix/remove broken: TestMetricDbSizeDefragDebugging 2021-03-12 23:05:53 +01:00
efce58d1ec Merge pull request #12770 from ptabor/20210312-3.5-todos
TODO's 3.5: Decommission metrics, PreVote=true.
2021-03-12 08:51:21 -08:00
948e32ae15 Delete etcd_debug metrics scheduled for deletion in 3.5. 2021-03-12 16:30:47 +01:00
54189f2f60 Enable --pre-vote=true by default in 3.5. 2021-03-12 16:23:23 +01:00
4eba403ccc Merge pull request #12765 from ptabor/20210312-move-config
Move config (ServerConfig) out of etcdserver package.
2021-03-11 15:05:29 -08:00
8a971eb5cd Update cert generation Makefile to include configurable IP addresses 2021-03-11 11:59:03 -08:00
fd7fed1511 Move config (ServerConfig) out of etcdserver package.
Motivation:
  - ServerConfig is part of 'embed' public API, while etcdserver is more 'internal'
  - EtcdServer is already too big and config is pretty wide-spread leaf
if we were to split etcdserver (e.g. into pre & post-apply part).
2021-03-11 20:56:22 +01:00
6dcd0de075 Merge pull request #12764 from ptabor/20210311-update-gogo
Fix gogo to 1.3.2 in api/go.mod.
2021-03-11 20:21:29 +01:00
783e26fcdf Fix gogo to 1.3.2 in api/go.mod. 2021-03-11 19:34:34 +01:00
b9226d03f4 Merge pull request #12763 from hexfusion/bump-proto
vendor: bump gogo/proto to v1.3.2
2021-03-11 17:59:07 +01:00
8ff0ff836a *: regen proto
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-11 11:27:52 -05:00
d3aa3fb486 vendor: bump gogo/proto to v1.3.2
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-11 11:27:25 -05:00
3ead91ca3e Merge pull request #12739 from LeoYang90/optimization_watch_prevkv
create event do not need prevkv range
2021-03-10 09:48:42 -08:00
633e9273db Merge pull request #12759 from wpedrak/staticcheck_partial_fixes
*: partial staticcheck fix
2021-03-10 09:47:52 -08:00
2c2456bf3d *: partial staticcheck fix 2021-03-10 14:13:38 +00:00
cb0e58942f Merge pull request #12753 from ptabor/20210306-integration-zap
Integration tests: Multiple improvements
2021-03-09 19:52:48 -08:00
f46c924f10 Merge pull request #12756 from hexfusion/bump-cl-03-09-3.5
CHANGELOG: add socket option flags #12702
2021-03-09 19:52:05 -08:00
7ac1367783 CHANGELOG: add socket option flags #12702
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-09 12:47:10 -05:00
c8243a9927 Tests: Functional - in case of failure, log the exception. 2021-03-09 18:19:52 +01:00
b6c2e87a74 Testing: Integration tests does not check whether t==nil 2021-03-09 18:19:52 +01:00
5ddabfdb24 tests: Make tests operate in /tmp director instead of src.
Thanks to this, unix sockets should be not longer
created by integration tests in the the source code directory,
so potentially trigger IDE reloads and unnecessery load (and mess).
2021-03-09 18:19:52 +01:00
bfe02c0526 tests: Cluster creation that failed shouldn't leak goroutines. 2021-03-09 18:19:52 +01:00
9ba1287334 travis script. Turning off verbose for grpcproxy.
+ Upgrade to '*.sh' variant of the scripts.
2021-03-09 18:19:51 +01:00
41f6cc7234 Tests: Better isolation between store_v2v3 integration tests. 2021-03-09 18:19:51 +01:00
fb1d48e98e Integration tests: Use BeforeTest(t) instead of defer AfterTest().
Thanks to this change, a single method BeforeTest(t) can handle
before-test logic as well as registration of cleanup code
(t.Cleanup(func)).
2021-03-09 18:19:51 +01:00
87258efd90 Integration tests: Use zaptest.Logger based testing.TB
Thanks to this the logs:
  - are automatically printed if the test fails.
  - are in pretty consistent format.
  - are annotated by 'member' information of the cluster emitting them.

Side changes:
  - Set propert default got DefaultWarningApplyDuration (used to be '0')
  - Name the members based on their 'place' on the list (as opposed to
'random')
2021-03-09 18:19:51 +01:00
efb584cc9b leak.go: Make the per-test AfterTest strictly wait for none of the unwanted rountines. 2021-03-09 18:19:51 +01:00
94a371acd7 Merge pull request #12750 from ptabor/20210306-mlock
--experimental-memory-mlock support
2021-03-09 09:13:40 -08:00
6fd85af641 Merge pull request #12702 from hexfusion/add-so
*: add support for socket options
2021-03-09 09:02:24 -08:00
5b49fb41c8 fixup: add ListenerOptions
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-08 11:27:03 -05:00
a46a358577 --experimental-memory-mlock support
The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).
2021-03-07 12:32:57 +01:00
792b7f57d3 Merge pull request #12747 from wilsonwang371/master
pkg/wait: change list from single element to an array.
2021-03-07 11:53:30 +01:00
7556b9a011 Merge pull request #12752 from cwedgwood/master-nofsyncdowrite
[RFC (against master branch)] etcdserver: when using --unsafe-no-fsync write data
2021-03-07 11:51:57 +01:00
b63d31e89b etcdserver: when using --unsafe-no-fsync write data
There are situations where we don't wish to fsync but we do want to
write the data.

Typically this occurs in clusters where fsync latency (often the
result of firmware) transiently spikes.  For Kubernetes clusters this
causes (many) elections which have knock-on effects such that the API
server will transiently fail causing other components fail in turn.

By writing the data (buffered and asynchronously flushed, so in most
situations the write is fast) and avoiding the fsync we no longer
trigger this situation and opportunistically write out the data.

Anecdotally:
  Because the fsync is missing there is the argument that certain
  types of failure events will cause data corruption or loss, in
  testing this wasn't seen.  If this was to occur the expectation is
  the member can be readded to a cluster or worst-case restored from a
  robust persisted snapshot.

  The etcd members are deployed across isolated racks with different
  power feeds.  An instantaneous failure of all of them simultaneously
  is unlikely.

  Testing was usually of the form:
   * create (Kubernetes) etcd write-churn by creating replicasets of
     some 1000s of pods
   * break/fail the leader

  Failure testing included:
   * hard node power-off events
   * disk removal
   * orderly reboots/shutdown

  In all cases when the node recovered it was able to rejoin the
  cluster and synchronize.
2021-03-05 10:58:04 -08:00
f4001630d9 Merge pull request #12748 from ptabor/20210305-deflake
Test flakes: 1 fix + 1 diagnostic
2021-03-04 23:38:16 +01:00
66cef61444 Detect leaked go-routines bases on pre-normalization syntax. 2021-03-04 22:28:44 +01:00
339f8fa4bd test.sh: Run integration tests with -v and shorter deadline.
The purpose of this change is to learn more about flake cases like:
  https://travis-ci.com/github/etcd-io/etcd/jobs/488324449

```
% (cd tests && 'env' 'go' 'test' '-timeout=30m' '--race=false' '--cpu=2' './integration/...')
stderr: go: downloading github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
ok  	go.etcd.io/etcd/tests/v3/integration	197.295s
ok  	go.etcd.io/etcd/tests/v3/integration/client	0.089s
ok  	go.etcd.io/etcd/tests/v3/integration/client/examples	0.038s
ok  	go.etcd.io/etcd/tests/v3/integration/clientv3	70.365s
ok  	go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency	3.169s
ok  	go.etcd.io/etcd/tests/v3/integration/clientv3/connectivity	100.535s
ok  	go.etcd.io/etcd/tests/v3/integration/clientv3/examples	1.341s
ok  	go.etcd.io/etcd/tests/v3/integration/clientv3/experimental/recipes	3.277s

No output has been received in the last 10m0s,
```
2021-03-04 22:23:05 +01:00
432fde88a9 pkg/wait: change list from single element to an array.
We found wait lock contention when a large amount of write operations. Converting wait from single element to an array helps to improve the performance.

Fixes #12731

Signed-off-by: Wilson Wang <wilsonny371@gmail.com>
2021-03-04 12:16:41 -08:00
aefbd226b8 Merge pull request #12745 from ptabor/20210304-fix-wtfpl
Update version of certifi/gocertifi to get rid of WTF Public license
2021-03-04 12:47:06 +01:00
ba1eebe2ea Merge pull request #12744 from emilyselwood/patch-1
Metrics example 404s - fix url
2021-03-04 09:59:56 +01:00
f7a2389992 Update version of certifi/gocertifi to get rid of WTF Public license
Seems old versions of https://github.com/certifi/gocertifi where
categorized as "Do What The F*ck You Want To Public License".

Update to newer version that is explicit `Mozilla Public License` 2.0 (MPL 2.0).
2021-03-04 09:48:34 +01:00
9ebbf5f38b Metrics example 404s - fix url
The metrics example link points to a file that appears to have moved. This change points it to what I think is the right place.
2021-03-04 08:17:48 +00:00
61fef348f8 Merge pull request #12742 from wilsonwang371/master
debugutil: Remove extra space in trace handler route
2021-03-04 08:55:44 +01:00
7fc447fb5c debugutil: Remove extra space in trace handler route
debugutil: Remove extra space in trace handler route. To use trace, user needed to escape the extra space and the extra space needs to be removed.
2021-03-03 16:01:04 -08:00
f02525c75d Merge pull request #12741 from hexfusion/bump-cl-03-03
CHANGELOG: update to include experimental-apply-warning-duration
2021-03-03 13:38:22 -05:00
e9947bc018 CHANGELOG: update to include experimental-apply-warning-duration
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-03-03 12:08:58 -05:00
d70f35f8d1 create event do not need prevkv range 2021-03-02 17:43:24 +08:00
102c198444 Merge pull request #12705 from astromechza/bm_etcd_peer_server_cert
etcdmain: added peer-client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
2021-03-02 09:03:35 +01:00
102096ade2 Merge pull request #12737 from davidlanouette/12718
client v2: check for empty request from the context
2021-03-01 23:57:58 -08:00
6ace85b624 Merge pull request #12736 from ptabor/20210301-fix-flakes
tests: Fixes a few recently spotted test-flakes
2021-03-01 14:22:29 -08:00
6998c5641c client v2: rename error var for revive
The revive tool complained durring the build.  Error variable has been
renamed.

Fixes #12718

Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 17:09:16 -05:00
7d02ce2073 client v2: check for empty request from the context
If the simpleHTTPClient.Do is called and the context has a nil request, return an error early.

Fixes #12718

Signed-off-by: David Lanouette <David.Lanouette@GMail.com>
2021-03-01 16:46:30 -05:00
5e10d12996 tests: Fixes a few recently spotted test-flakes
```
Unexpected goroutines running after all test(s).
1 instances of:
syscall.Syscall(...)
	/usr/local/go/src/syscall/asm_linux_386.s:19 +0x5
syscall.Close(...)
	/usr/local/go/src/syscall/zsyscall_linux_386.go:285 +0x3d
internal/poll.(*FD).destroy(...)
	/usr/local/go/src/internal/poll/fd_unix.go:77 +0x30
internal/poll.(*FD).decref(...)
	/usr/local/go/src/internal/poll/fd_mutex.go:213 +0x38
internal/poll.(*FD).Close(...)
	/usr/local/go/src/internal/poll/fd_unix.go:99 +0x43
net.(*netFD).Close(...)
	/usr/local/go/src/net/fd_posix.go:37 +0x49
FAIL	go.etcd.io/etcd/tests/v3/integration/client	0.039s
```

```
--- FAIL: TestServer_TCP_Secure_DelayTx (0.20s)
    server_test.go:110: took 128.026085ms with no latency
    server_test.go:125: took 62.980988ms with latency 50ms��5ms
    server_test.go:133: expected took1 128.026085ms < took2 62.980988ms (with latency)
```

https://github.com/etcd-io/etcd/issues/12372
2021-03-01 18:07:38 +01:00
3d44f5bf80 *: added client-{client,key}-file parameters for supporting separate client and server certs when communicating between peers
In some environments, the CA is not able to sign certificates with both
'client auth' and 'server auth' extended usage parameters and so an operator
needs to be able to set a seperate client certificate to use when making
requests which is different to the certificate used for accepting requests.
This applies to both proxy and etcd member mode and is available as both a CLI
 flag and config file field for peer TLS.

Signed-off-by: Ben Meier <ben.meier@oracle.com>
2021-02-28 14:37:56 +00:00
d06d93d5b1 CHANGELOG: add go 1.16 for etcd 3.5, add release links
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-02-26 23:10:25 +00:00
d4d303d908 CHANGELOG: add v3.4.15
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-02-26 23:04:36 +00:00
cb040801f9 Merge pull request #12730 from ptabor/20200208-client-no-logger
clientv3: Cleaning up dead logger code
2021-02-26 10:44:42 -08:00
a7f340216d Reformat code according to 'gotip' rules.
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
54b87505a3 Remove dead legacy logger code. 2021-02-26 09:13:09 +01:00
a769916ea2 Merge pull request #12729 from ptabor/20210225-raftexample
raftExample: Allow closing raftexample node when snapshotting.
2021-02-26 08:58:30 +01:00
3976d68ed3 raftExample: Allow closing raftexample node when snapshotting.
Fix race that made the raftExample test fail.
2021-02-26 08:56:12 +01:00
9563698f64 Merge pull request #12727 from ptabor/20210225-fix-resolver-ordering
ClientV3: Ordering: Fix TestEndpointSwitchResolvesViolation test
2021-02-26 08:48:20 +01:00
39726116c5 Merge pull request #12728 from nate-double-u/12700-update-links
Updating links in .md files after removing Documentation.
2021-02-25 19:43:19 -05:00
45b1e6b470 ClientV3: Ordering: Fix the ordering test such it does not fail.
The test depended on very subtle timing semantic and on properties of
'copied' clients.

https://travis-ci.com/github/etcd-io/etcd/jobs/486191449

Examplar failure:
```
{"level":"warn","ts":"2021-02-25T12:34:47.894Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0000d6fc0/#initially=[unix://localhost:86269902489114839060]","attempt":1,"error":"rpc error: code = Unavailable desc = etcdserver: rpc not supported for learner"}
{"level":"warn","ts":"2021-02-25T12:34:48.163Z","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00035a000/#initially=[unix://localhost:78285857058450835940]","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: not leader"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:48.255Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2021-02-25T12:34:50.255Z","caller":"v3/maintenance.go:219","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2021-02-25T12:34:51.717Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:52.017Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = Canceled desc = context canceled"}
{"level":"info","ts":"2021-02-25T12:34:52.018Z","caller":"v3/maintenance.go:211","msg":"opened snapshot stream; downloading"}
{"level":"warn","ts":"2021-02-25T12:34:53.018Z","caller":"v3/maintenance.go:221","msg":"failed to receive from snapshot stream; closing","error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
--- FAIL: TestEndpointSwitchResolvesViolation (10.12s)
    ordering_util_test.go:81: failed to resolve order violation etcdclient: no cluster members have a revision higher than the previously received revision
```
2021-02-25 22:15:13 +01:00
d41e18817a Updating links in .md files after removing Documentation.
Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
2021-02-25 12:59:57 -08:00
fa82d11a95 Merge pull request #12725 from ptabor/20210225-release-scripts-fix-2
Improve release scripts: Lessons learned from 3.5.0-alpha.0
2021-02-25 11:46:12 -08:00
60a669762f Improve release scripts:
- Fix script that creates manifest-list based multi-arch-images.
    The images need to be pushed first.
  - Use docker instead of gcloud docker helper
  - Make sure docker pushes are properly 'dry run'
  - Added preparation instruction to the release script.
2021-02-25 12:28:52 +00:00
ae36379800 Merge pull request #12722 from ptabor/20210225-reporting-bugs
Github: Shorten the reporting-bugs link.
2021-02-25 09:06:04 +01:00
4af3bb3b01 Github: Shorten the reporting-bugs link. 2021-02-25 09:00:00 +01:00
926663f8d8 Merge pull request #12720 from FogDong/master
Docs: fix the report bug link in issue template
2021-02-25 08:53:49 +01:00
b0949cb49f Docs: fix the report bug link in issue template 2021-02-25 10:58:11 +08:00
60d5159091 version: bump up to 3.5.0-alpha.0 2021-02-24 19:55:45 +00:00
d4d2b80608 Merge pull request #12719 from ptabor/20210224-release-scripts-fix
Release scripts: Minor fixes discovered during attempt for release 3.5.0-alpha.0
2021-02-24 20:19:19 +01:00
c640957e2d Release scripts: Minor fixes discovered during attempt for 3.5.0-alpha.0 2021-02-24 19:16:27 +00:00
fe277f48ad Merge pull request #12716 from iAziz786/dial-journal-typo
systemd: Fix typo in DialJournal documentation
2021-02-24 12:24:30 -05:00
252dcc9bdb Fix typo in DialJournal 2021-02-24 22:23:27 +05:30
db8c8c4246 Merge pull request #12711 from spzala/updatesampleconfig
Update debugging flag in the sample config file
2021-02-24 16:44:57 +01:00
6b6579c767 Merge pull request #12714 from mrkm4ntr/example-apply-done-c
raftexample: Return an appropriate applyDoneC
2021-02-24 16:43:45 +01:00
5ae3f879c9 raftexample: Return an appropriate applyDoneC 2021-02-24 21:28:18 +09:00
8d9daec3ca Update debugging flag in the sample config file
Replace the deprecated debug flag.
2021-02-24 00:04:56 -05:00
fa14a0af89 Merge pull request #12709 from ptabor/20210222-grpc1.32
Update grpc dependency to 1.32.
2021-02-23 12:19:28 +01:00
1a9c81abda Update grpc dependency to 1.32.
Simplify grpc testing infrastructure to align with upstream changes.
2021-02-23 11:31:50 +01:00
57dcb037c0 Merge pull request #12706 from ptabor/20210218
clientv3: PS: Replace balancer with upstream grpc solution
2021-02-23 10:56:20 +01:00
4a1c24556c clientv3: PS: Replace balancer with upstream grpc solution
Addresses comments from: https://github.com/etcd-io/etcd/pull/12671#pullrequestreview-593942302
2021-02-23 10:03:15 +01:00
1cf66ab7df Merge pull request #12696 from mrkm4ntr/example-fix-join
raftexample: New joined node have to start with RestartNode
2021-02-22 19:58:10 +01:00
e8da3c3e74 Merge pull request #12707 from viviyww/new
mvcc: optimize watch logic of watchableStore
2021-02-22 19:45:55 +01:00
7ef6ebd5eb mvcc: optimize watch logic of watchableStore
Optimize watchableStore.watch func
2021-02-22 14:10:25 +08:00
cb0d256a18 raftexample: Add test for adding new node to existing cluster 2021-02-22 13:44:33 +09:00
1b1be43d65 raftexample: New joined node have to start with RestartNode 2021-02-22 09:45:44 +09:00
8d1a8ce044 Merge pull request #12701 from mrkm4ntr/example-notify-all-aplied
raftexample: Explicitly notify all committed entries are applied
2021-02-20 22:34:42 +01:00
3d7aac948b Merge pull request #12196 from ironcladlou/metrics-watch-error-fix
etcdserver: fix incorrect metrics generated when clients cancel watches
2021-02-19 12:46:49 -08:00
49078c683b *: add support for socket options
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2021-02-19 13:31:23 -05:00
cc2b039817 raftexample: Explicitly notify all committed entries are applied 2021-02-19 19:26:36 +09:00
70ac39fd98 Merge pull request #12699 from spzala/removemixinfromdoc
Remove mixin from doc
2021-02-19 00:04:14 -08:00
7fca120587 Remove mixin from doc
The kube-prometheus team has made necessary changes and not
using the mixin from doc any more.
2021-02-18 18:44:19 -05:00
b67ed4e4aa Merge pull request #12671 from ptabor/20210207-grpc-del-balancer
clientv3: Replace balancer with upstream grpc solution
2021-02-17 22:45:13 +01:00
cc99c3631f Merge pull request #12695 from mrkm4ntr/example-report
raftexample: Implement ReportUnreachable and ReportSnapshot
2021-02-17 09:16:36 +01:00
2d25f7f3da raftexample: Implement ReportUnreachable and ReportSnapshot 2021-02-17 11:59:32 +09:00
73c50b869a Merge pull request #12637 from BusyJay/check-outgoingvoters-when-restoring
raft: check `VotersOutgoing` for snapshot
2021-02-16 09:43:08 +01:00
47b2506fcb Merge pull request #12670 from postgrespro/customizable_raft_connection_timeouts
raft: makes 'ConnReadTimeout/ConnWriteTimeout' customizable
2021-02-16 09:07:41 +01:00
24d02586c1 Merge pull request #12691 from mrkm4ntr/example-save-snap-before-wal
raftexample: Save snapshot file before writing to wal
2021-02-16 08:55:33 +01:00
1302e1edb2 raftexample: Save snapshot file before writing to wal 2021-02-16 13:30:15 +09:00
719f6ce06f Merge pull request #11964 from hicqu/fast-log-rejection
raft: implement fast log rejection
2021-02-15 13:47:03 +01:00
c1e8d3a63f Clarify documentation of probing
- Add a large detailed comment about the use and necessity of
  both the follower and leader probing optimization
- fix the log message in stepLeader that previously mixed up the
  log term for the rejection and the index of the append
- improve the test via subtests
- add some verbiage in findConflictByTerm around first index
2021-02-15 09:47:18 +01:00
3ef8a2d30a Merge pull request #12688 from etcd-io/revert-12687-removemixindoc
Revert "Remove mixin from Doc"
2021-02-11 20:12:45 +01:00
9fed697435 Revert "Remove mixin from Doc" 2021-02-11 13:59:26 -05:00
080453d416 Merge pull request #12687 from spzala/removemixindoc
Remove mixin from Doc
2021-02-11 15:52:35 +01:00
6ff00fd202 Remove mixin from Doc
Mixin is moved under contrib and kube-prometheus team is now using
it from new location.
2021-02-11 09:09:38 -05:00
63cb3c97b7 Merge pull request #12686 from ptabor/20210211-prometheous-mixin
Migrate back mixin to contrib/
2021-02-11 13:07:34 +01:00
77e6df28cf Merge pull request #12675 from ptabor/20210209-grpc-remove-legacy-resolver
Cleanup grpc client/v3/naming API
2021-02-11 09:31:48 +01:00
1395a1a795 Migrate back mixin to contrib/
The mixin was moved out together with documentation.
This broke kube-prometheous: https://github.com/etcd-io/etcd/issues/12685#issuecomment-777264143
2021-02-11 09:28:30 +01:00
023aec9151 Merge pull request #12683 from nate-double-u/97-restore-etcd-mixin
Restoring etcd-mixin folder
2021-02-10 21:01:30 -05:00
433e7a0e95 Restoring etcd-mixin
Restoring Documentation/etcd-mixin folder for kube-prometheus use as per conversations in issue https://github.com/etcd-io/website/issues/97

Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
2021-02-10 15:38:49 -08:00
d33a1c91f0 Merge pull request #12660 from nate-double-u/97-remove-docs-from-etcd
Migrate documentation: Remove docs from etcd-io/etcd
2021-02-10 16:39:18 -05:00
6828517965 raft: implement fast log rejection
Signed-off-by: qupeng <qupeng@pingcap.com>
2021-02-10 15:48:32 -05:00
b757e1bc87 raft: create new probe_and_replicate.txt interactive test
This commit creates a new probe_and_replicate.txt interactive test. The
test creates a complete Raft log configuration and demonstrates how a
leader probes and replicates to each of its followers. The log
configuration constructed is identical to the one present in Figure 7 of
the raft paper (https://raft.github.io/raft.pdf), which looks like:

```
     1  2  3  4  5  6  7  8  9  10 11 12
n1: [1][1][1][4][4][5][5][6][6][6]
n2: [1][1][1][4][4][5][5][6][6]
n3: [1][1][1][4]
n4: [1][1][1][4][4][5][5][6][6][6][6]
n5: [1][1][1][4][4][5][5][6][7][7][7][7]
n6: [1][1][1][4][4][4][4]
n7: [1][1][1][2][2][2][3][3][3][3][3]
```

Once in this state, we then elect node 1 as the leader and stabilize the
entire raft group. This demonstrates how a newly elected leader probes
for matching indexes, overwrites conflicting entries, and catches up all
followers.

This will be useful to demonstrate the impact of more efficient probing
behavior.
2021-02-10 15:02:36 -05:00
2afa5ff45f removing Documentation from etcd-io/etcd
Leaving docs, files, etc that are required for generated documentation

fixes https://github.com/etcd-io/website/issues/97

Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
2021-02-10 11:14:40 -08:00
e8ba375032 Merge pull request #11889 from mrkm4ntr/example-recover-from-snap
raftexample: Fix recovery from snapshot
2021-02-10 12:18:30 +01:00
7e38cfcc8d raft: makes 'ConnReadTimeout/ConnWriteTimeout' customizable 2021-02-10 10:36:50 +07:00
a836a8045b Get rid of legacy client/v3/naming API.
Update grpcproxy to use the new abstractions.
2021-02-09 11:56:28 +01:00
44c889a8ce Merge pull request #12677 from chaochn47/reduce_verbosity_of_etcd_server_side_success_health_check_logs
etcdserver/api/etcdhttp: log successful etcd server side health check in debug level
2021-02-08 21:15:25 -08:00
028d351a46 CHANGELOG-3.5.md: update 2021-02-08 19:46:43 -08:00
2ae3e82f07 etcdserver/api/etcdhttp: log successful etcd server side health check in debug level
When we have an external component that checks /health periodically, the
etcd server logs can be quite verbose (e.g., DDOS-ing against insure
etcd health check can lead to disk space full due to large log files).

This change was introduced in #11704.

While we keep the warning logs for etcd health check failures, the
success (or OK) log level should be set to DEBUG.

Fixes #12676
2021-02-08 17:15:43 -08:00
96a9860b60 Merge pull request #12674 from ptabor/20210208-skip-386-tls-test
Fix pkg/tlsutil (test) to not fail on 386.
2021-02-08 16:46:47 -08:00
60e44286fa Fix pkg/tlsutil (test) to not fail on 386.
In fact this commit rewrites the functionality to use upstream list of
ciphers instead of checking whether the lists are in sync using ast
analysis.
2021-02-08 23:03:43 +01:00
0b75fede64 Replace client/v3/balancer with standard components: resolver + round_robin LB
This commit significantly reduces volume of custom code
in etcd client v3, while preserving full existing functionality.
2021-02-08 18:50:31 +01:00
3fddea9669 Merge pull request #12669 from lemonlinger/implement-endpoint-manager-and-resolver
Implement endpoint watch and resolver
2021-02-08 14:31:48 +01:00
8feb55f65c *: implement Endpoint Watch and new Resolver 2021-02-08 20:05:45 +08:00
be2167ebab Wait until all committed entries are applied
To take a snapshot
2021-02-05 19:05:41 +09:00
cb14cdd774 raftexample: Fix recovery from snapshot
* If there is a snapshot, HTTP server won't start.
* Resotring form snapshot occurs after replaying WAL.
* When taking a snapshot, the last change is not applied to the state machine yet.
2021-02-05 09:34:34 +09:00
dae29bb719 Merge pull request #12667 from lemonlinger/implement-endpoint-manager-and-resolver
Implement Update method for EndpointManager
2021-02-04 17:25:51 +01:00
571ed502d4 endpoints: implement Update method for EndpointManager.
- Add integration test for endpoints and resolver.
2021-02-04 23:30:22 +08:00
4d9f1a96da Merge pull request #12664 from spzala/rpctypeserrors
Doc: fix inconsistency and broken path
2021-02-03 15:06:54 -05:00
66777cf1e8 Doc: fix inconsistency and broken path
1) Fix the path for rpc errors
2) The README says 2 types of client errors vs doc mentioning 3
types of client errors. grpc erros should be considered single time
as covered in the README.
2021-02-03 12:02:49 -05:00
4accc34c98 Merge pull request #11776 from brandond/fix_srv_11321
Fix cluster peer HTTP SRV discovery when no HTTPS records exist
2021-02-03 13:17:55 +01:00
603d975599 Fix cluster peer HTTP SRV discovery
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
2021-02-03 03:08:13 -08:00
6881ea828e Merge pull request #12658 from dhermes/pr-12636-balancer
Prepare `balancer` interfaces for `>=google.golang.org/grpc@1.30.0` upgrade.
2021-02-02 23:03:56 +01:00
ea34f8dbc6 Prepare balancer interfaces for >=google.golang.org/grpc@1.30.0 upgrade. 2021-02-02 15:54:33 -06:00
50ca440c49 Merge pull request #12410 from Yanhao/master
etcdserver: export method EtcdServer.leaderChangedNotify (#12378)
2021-02-02 12:23:11 +01:00
6d82778a4e etcdserver: export method EtcdServer.leaderChangedNotify (#12378) 2021-02-02 18:13:32 +08:00
e897daaebc Merge pull request #12659 from spzala/genprotodoc
Clarify the root directory comment
2021-02-01 17:32:03 -05:00
6bbef10721 Clarify the root directory comment
A contributor had an issue when ran script changing the directory
name after clone. This was discussed on the etcd slack channel.
2021-02-01 15:41:36 -05:00
d6d03beaea Merge pull request #12538 from lzhfromustc/12_9_GoroutineLeak
test: change channel operations to avoid potential goroutine leaks
2021-02-01 21:16:43 +01:00
b5d11723d1 Merge pull request #12393 from viviyww/contrib-doc
contrib: del systemd/etcd2-backup-coreos in docs
2021-02-01 15:33:44 +01:00
855eeb75c5 Merge pull request #12654 from mengjiao-liu/debian-1.4.0
update etcd base image to debian-base 1.4.0 which is now a sane multi-architecture image
2021-02-01 08:08:03 +01:00
b5d4ea13ff update etcd base image to debian-base 1.4.0 2021-02-01 13:53:58 +08:00
4af159a30a Merge pull request #12259 from alvistack/master-aio_graceful_reboot
`etcd.service`: Define explicit dependencies of systemd etcd service
2021-01-31 23:58:12 +01:00
a18cf56d8d Merge pull request #12507 from etcd-io/add-codeql
Add CodeQL Security Scanning
2021-01-31 23:53:05 +01:00
4d0a35a0fb Merge pull request #11518 from nlamirault/patch-1
Add tag "etcd-mixin" into the Grafana dashboard
2021-01-31 23:27:11 +01:00
958f6f9878 Merge pull request #12481 from kalexmills/fix-defer-log
fix: pass argument url in defer to avoid loopclosure
2021-01-31 23:20:32 +01:00
2c8de24afc Merge pull request #12495 from VertexC/fix/testErrorMsg
raft/raft_test: minor fix on inconsistent format of test case's error message
2021-01-31 22:25:23 +01:00
6bb26ef008 Merge pull request #12533 from lucab/ups/contrib-sysusers
contrib/systemd: add a sysusers entry
2021-01-31 11:40:25 +01:00
6db49d403c Merge pull request #12600 from ravitri/remove_v2_alerts
Documentation: Removing etcd v2 specific alerts
2021-01-31 11:19:22 +01:00
2fd6787506 Merge pull request #12653 from etcd-io/20201111-fix-flakes-go15
Tests:Use temp-directory that is covered by framework level cleanup
2021-01-31 11:05:13 +01:00
0d7a671c75 Tests: Fix vet warnings about Fatal in sub-goroutines.
% (cd tests && go vet ./...)
stderr: # go.etcd.io/etcd/tests/v3/integration/clientv3/concurrency_test
stderr: integration/clientv3/concurrency/election_test.go:74:6: call to (*T).Fatal from a non-test goroutine
stderr: integration/clientv3/concurrency/mutex_test.go:57:4: call to (*T).Fatal from a non-test goroutine
2021-01-31 00:00:26 +01:00
73ef0cde01 Update travis to use 1.15.7 go version. 2021-01-30 23:47:54 +01:00
f0ecad00e3 Use temp-directory that is covered by framework level cleanup
Prior to this PR, the e2e tests where creating dirs like:
```
/tmp/testname1.etcd030299846
/tmp/testname0.etcd039445123
/tmp/testname0.etcd206372065
```
and not cleaning them, that led to disk-space-exceeded flakes.

After the PR, the testing.TB tempdir mechanism is used and the names are
being cleaned and are more miningful:

```
../../bin/etcd --name test-TestCtlV3EndpointHashKV-2 --listen-client-urls http://localhost:20010 --advertise-client-urls http://localhost:20010 --listen-peer-urls https://localhost:20011 --initial-advertise-peer-urls https://localhost:20011 --initial-cluster-token new --data-dir /tmp/TestCtlV3EndpointHashKV429176179/003 --snapshot-count 100000 --experimental-initial-corrupt-check --peer-auto-tls --initial-cluster test-TestCtlV3EndpointHashKV-0=https://localhost:20001,test-TestCtlV3EndpointHashKV-1=https://localhost:20006,test-TestCtlV3EndpointHashKV-2=https://localhost:20011
```
2021-01-30 13:25:55 +01:00
cf2153d44b Merge pull request #12614 from ptabor/prepare-to-remove-grpc-naming-package
Replacement API for client/v3/naming package to be compatible with new GRPC1.30+ resolver API.
2021-01-30 12:51:03 +01:00
5d7c1db3a9 Introduce grpc-1.30+ compatible client/v3/naming API.
This is not yet implementation, just API and tests to be filled
with implementation in next CLs,
tracked by: https://github.com/etcd-io/etcd/issues/12652

We propose here 3 packages:
 - clientv3/naming/endpoints ->
    That is abstraction layer over etcd that allows to write, read &
    watch Endpoints information. It's independent from GRPC API. It hides
    the storage details.

 - clientv3/naming/endpoints/internal ->
    That contains the grpc's compatible Update class to preserve the
    internal JSON mashalling format.

 - clientv3/naming/resolver ->
   That implements the GRPC resolver API, such that etcd can be
   used for connection.Dial in grpc.

Please see the grpc_naming.md document changes & grpcproxy/cluster.go
new integration, to see how the new abstractions work.
2021-01-30 12:32:19 +01:00
90d1b838ad Fix shellcheck warnings: Escaping.
```
build:3:65: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
test:3:64: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:46: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:30:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:38:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:53:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:73:23: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:73:33: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:74:69: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:74:78: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
scripts/genproto.sh:75:40: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:75:52: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:91:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:115:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
./test.sh:71:29: warning: Quote to prevent word splitting, or split robustly with mapfile or read -a. [SC2206]
./test.sh:400:28: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:406:33: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:415:36: note: Backslash is literal in "\*". Prefer explicit escaping: "\\*". [SC1117]
./test.sh:415:176: note: Backslash is literal in "\'". Prefer explicit escaping: "\\'". [SC1117]
./test.sh:493:56: note: Backslash is literal in "\.". Prefer explicit escaping: "\\.". [SC1117]
./test.sh:601:57: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:64: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:67: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:74: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:77: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:84: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:91: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
./test.sh:601:94: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
./test.sh:601:103: note: Backslash is literal in "\3". Prefer explicit escaping: "\\3". [SC1117]
```
2021-01-29 22:32:43 +00:00
9600bbd5fe Update root image for docker-test-image to ubuntu:20.10 2021-01-29 22:16:51 +00:00
73bfa8cfb1 wqMakefile: Docker generation should not override the src file
Also fix /tmp directory during: 'make docker-test' to make it executable.
2021-01-29 22:08:58 +00:00
a8c935eab0 test.sh: Stop requiring/defaulting CPU flag to 4 2021-01-28 19:15:04 +00:00
a4fac14353 Merge pull request #12648 from pacoxu/debian-1.3.0
update etcd base image to debian-base 1.3.0 for CVEs
2021-01-27 09:15:12 +01:00
b9358dd18b Dockerfile: update etcd base image to debian-base 1.3.0 for CVEs
Signed-off-by: pacoxu <paco.xu@daocloud.io>
2021-01-26 15:42:53 +08:00
92458228e1 Merge pull request #12423 from Murphylu1993/master
client: replace dial with dialContext
2021-01-25 20:28:40 +01:00
e268b92a44 Merge pull request #12633 from ptabor/20210119-cov2
Fix codecov collection (Part 2/2)
2021-01-26 00:45:43 +08:00
351bdb33c5 Split intengration/clientv3 tests into multiple packages
They used to take >10min with coverage, so were causing interrupted
Travis runs.

Know thay fit in 100-150s (together), thanks also to parallel
execution.
2021-01-23 11:12:44 +01:00
88ee005a0f Reduce verbosity of coverage collection. 2021-01-23 08:59:19 +01:00
330b157dbb Merge pull request #12641 from retroflexer/update-CHANGELOG-3.2
CHANGELOG: Update for cherry picks for CVE
2021-01-21 16:18:58 -05:00
3d2bb0797b CHANGELOG: Update for cherry picks for CVE 2021-01-21 15:26:40 -05:00
7d444ee327 Merge pull request #12629 from ptabor/20210118-release-attempt
Modernize release script
2021-01-21 20:33:30 +01:00
598ca6caab Modernize release script:
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
  - better interaction with ./test_lib.sh script.
  - more consistent logging
  - bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
2021-01-21 17:47:36 +01:00
f61ccb7d66 fix doc
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2021-01-21 16:12:50 +08:00
f947c815d0 raft: check VotersOutgoing for snapshot
Close #12631.

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2021-01-21 16:09:37 +08:00
f85b08c735 add test case
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2021-01-21 16:02:46 +08:00
5dcd459ae9 Merge pull request #12564 from dhermes/patch-1
Adding `clientv3` import alias to match usage in `register_test.go`.
2021-01-19 23:00:19 +01:00
1d73d2e85e Merge pull request #12632 from ptabor/20210119-fix-coverage-report
Fix code coverage measurement
2021-01-19 10:19:39 -08:00
50352affd4 Merge pull request #12630 from anguslees/manifestlist
Generate and push multi-arch docker manifest list
2021-01-19 10:18:48 -08:00
5472b3336b Merge pull request #12525 from sakateka/remove_raft.peers
raft tests: Remove Config.peers and Config.learners
2021-01-19 16:00:54 +01:00
486687843b Fix code coverage measurement. 2021-01-19 15:41:36 +01:00
2529fee958 Use /build instead of deprecated /build.sh 2021-01-19 12:46:56 +11:00
7f79ffb409 Generate and push multi-arch docker manifest list
Change amd64 docker image to use a `-amd64` suffix, consistent with
other architectures.  Introduce a multiarch docker manifest list with
the original (no-arch) docker tag.

Fixes #10380
2021-01-19 12:46:56 +11:00
bb71f706a6 Merge pull request #12580 from dlorenc/update
Switch from dgrijalva/jwt-go to form3tech-oss/jwt-go.
2021-01-16 22:15:09 +01:00
1a890a4659 Merge branch 'master' into update 2021-01-16 09:35:05 +01:00
ccfd00f687 raft: specify voters and learners via snapshot 2021-01-16 13:03:47 +07:00
c0ab7cd6a1 Merge pull request #12624 from gyuho/maintainers
MAINTAINERS: add wenjiaswe@ and ptabor@
2021-01-15 12:34:05 -08:00
84f270f88d Merge pull request #12588 from ptabor/20201230-raft-docs
Raft: Expand raft documentation, in particular point on the godocs
2021-01-15 22:49:40 +08:00
daa5bd11c0 Merge pull request #12497 from ptabor/20201127-release
Modularized release script integration
2021-01-15 22:46:41 +08:00
bf6f173d5e Document Raft.send method.
The change makes it explicit that sending messages does not happen
immidietely and is subject to proper persist & then send protocol
on the application side. See:

https://github.com/etcd-io/etcd/issues/12589#issuecomment-752867024

for more context.
2021-01-15 12:35:58 +01:00
58f78df1de Raft: Expand raft documentation, in particular point on godocs. 2021-01-15 12:34:02 +01:00
577c898fee scripts: Integrate ./scripts/release with new code for tagging modules.
Changes:
  - signing tags.
  - allows to override BRANCH and REPOSITORY using env variables.

Tested by a release in my private fork:
  BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
2021-01-15 12:31:44 +01:00
c4c7d5d848 MAINTAINERS: add wenjiaswe@ and ptabor@
Both have shown great initiatives and dedication to the project's long-term health and community engagement.

I strongly support for their recognition and continued work on the project.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-01-14 15:16:38 -08:00
69e99e80fa Merge pull request #12465 from spacewander/fdoc
chore: update the documentation link in the comment
2021-01-14 00:39:25 -05:00
e9e0cb1077 Merge pull request #12545 from sid597/dev_guide-api_concurrency-path-update
Documentation (dev_guide--api_concurrency_reference_v3) : Path to v3lock and v3election updated
2021-01-14 00:07:24 -05:00
2c11bd5e19 Merge pull request #12586 from viviyww/clientconfigerror
transport: catch new cert error
2021-01-13 23:47:36 -05:00
c95d9434f1 Merge pull request #12602 from ptabor/20210107-scripts-on-osx
Make ./script/fix.sh & test.sh OSX/BSD compatible
2021-01-13 00:30:58 +08:00
84b5b87fb2 Make test.sh scripts OSX/BSD compatible:
- build & test scripts deprecated. Call *.sh variants.
  This will avoid delete the symlinks and get rid of
  subtle dependency on 'sed --follow-symlinks' on OsX/BSD sed.

- Fix parameters to mktemp
2021-01-12 16:29:00 +01:00
bfc6e2ff30 Merge pull request #12611 from ptabor/20210111-fix-flakes
e2e tests flakes & leaks fixes: In particular TestIssue6361
2021-01-12 21:26:54 +08:00
b760bb60e5 Merge pull request #12604 from ptabor/20210107-proto-post-modules
Refresh proto generation script after moving modules files.
2021-01-12 21:08:51 +08:00
0d9cfc11c8 Fix usage of reflect.SliceHeader: reported by vet on tip golang
Example:
https://travis-ci.com/github/etcd-io/etcd/jobs/470404938

```
% (cd server && go vet ./...)
stderr: # go.etcd.io/etcd/server/v3/etcdserver/api/v2store
stderr: etcdserver/api/v2store/node_extern_test.go:107:9: possible misuse of reflect.SliceHeader
stderr: etcdserver/api/v2store/node_extern_test.go:107:16: possible misuse of reflect.SliceHeader
```
2021-01-12 00:14:51 +01:00
70b5ef1d3a Fix tests flakiness: in particular TestIssue6361.
The root reason of flakes, was that server was considered as ready to
early.
In particular:
```
../../bin/etcd-2456648: {"level":"info","ts":"2021-01-11T09:56:44.474+0100","caller":"rafthttp/stream.go:274","msg":"established TCP streaming connection with remote peer","stream-writer-type":"stream Message","local-member-id":"ed5f620d34a8e61b","remote-peer-id":"ca50e9357181d758"}
../../bin/etcd-2456648: {"level":"warn","ts":"2021-01-11T09:56:49.040+0100","caller":"etcdserver/server.go:1942","msg":"failed to publish local member to cluster through raft","local-member-id":"ed5f620d34a8e61b","local-member-attributes":"{Name:infra2 ClientURLs:[http://localhost:20030]}","request-path":"/0/members/ed5f620d34a8e61b/attributes","publish-timeout":"7s","error":"etcdserver: request timed out, possibly due to connection lost"}
../../bin/etcd-2456648: {"level":"info","ts":"2021-01-11T09:56:49.049+0100","caller":"etcdserver/server.go:1921","msg":"published local member to cluster through raft","local-member-id":"ed5f620d34a8e61b","local-member-attributes":"{Name:infra2 ClientURLs:[http://localhost:20030]}","request-path":"/0/members/ed5f620d34a8e61b/attributes","cluster-id":"34f27e83b3bc2ff","publish-timeout":"7s"}
```
was taking 5s.   If this was happening concurrently with etcdctl, the
etcdctl could timeout.

The fix, requires servers to report 'ready to serve client requests' to consider them up.

Fixed also some whitelisted 'goroutines'.
2021-01-12 00:14:51 +01:00
74274f4417 e2e: Adding better diagnostic and location for temporary files to Snapshot tests. 2021-01-12 00:14:51 +01:00
26f9b4be8f e2e tests were leaking 'defunc' etcdctl processes.
The commit ensures that spawned etcdctl processes are "closed",
so they perform proper os wait processing.
This might have contributed to file-descriptor/open-files limit being
exceeded.
2021-01-11 11:55:30 +01:00
e2a65bee6e Avoid 'interactive prompt' for root password in etcd tests.
Before:
```
{"level":"info","ts":1610273495.3791487,"caller":"agent/handler.go:668","msg":"cleaning up page cache"}
{"level":"info","ts":1610273495.3793094,"caller":"agent/handler.go:94","msg":"created etcd log file","path":"/tmp/etcd-functional-2/etcd.log"}
{"level":"info","ts":1610273495.379328,"caller":"agent/handler.go:668","msg":"cleaning up page cache"}
[sudo] password for ptab:
pam_glogin: invalid password
Sorry, try again.
[sudo] password for ptab:
```

Now the caches are dropped if the current users is in sudoers, bot not
in the other cases.
To be honest I don't see the purpose for dropping the caches at all in
the test.
2021-01-11 10:06:31 +01:00
8ccd4e1146 Fix flaky tests reported due to data race on grpc logging registration.
Example:
```
    ==================
    WARNING: DATA RACE
    Write at 0x000002178320 by goroutine 575:
      google.golang.org/grpc/grpclog.SetLoggerV2()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/grpclog/loggerv2.go:70 +0x444
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1.1()
          /home/ptab/corp/etcd/server/embed/config_logging.go:119 +0x345
      sync.(*Once).doSlow()
          /usr/lib/google-golang/src/sync/once.go:66 +0x109
      sync.(*Once).Do()
          /usr/lib/google-golang/src/sync/once.go:57 +0x68
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging.func1()
          /home/ptab/corp/etcd/server/embed/config_logging.go:109 +0x3b1
      go.etcd.io/etcd/server/v3/embed.(*Config).setupLogging()
          /home/ptab/corp/etcd/server/embed/config_logging.go:174 +0x6af
      go.etcd.io/etcd/server/v3/embed.(*Config).Validate()
          /home/ptab/corp/etcd/server/embed/config.go:553 +0x55
      go.etcd.io/etcd/server/v3/embed.StartEtcd()
          /home/ptab/corp/etcd/server/embed/etcd.go:93 +0x84
      go.etcd.io/etcd/tests/v3/integration.TestKVWithEmptyValue()
          /home/ptab/corp/etcd/tests/integration/v3_kv_test.go:33 +0x18c
      testing.tRunner()
          /usr/lib/google-golang/src/testing/testing.go:1123 +0x202

    Previous read at 0x000002178320 by goroutine 956:
      [failed to restore the stack]

    Goroutine 575 (running) created at:
      testing.(*T).Run()
          /usr/lib/google-golang/src/testing/testing.go:1168 +0x5bb
      testing.runTests.func1()
          /usr/lib/google-golang/src/testing/testing.go:1441 +0xa6
      testing.tRunner()
          /usr/lib/google-golang/src/testing/testing.go:1123 +0x202
      testing.runTests()
          /usr/lib/google-golang/src/testing/testing.go:1439 +0x612
      testing.(*M).Run()
          /usr/lib/google-golang/src/testing/testing.go:1347 +0x3c4
      go.etcd.io/etcd/pkg/v3/testutil.MustTestMainWithLeakDetection()
          /home/ptab/corp/etcd/pkg/testutil/leak.go:150 +0x38
      go.etcd.io/etcd/tests/v3/integration.TestMain()
          /home/ptab/corp/etcd/tests/integration/main_test.go:14 +0x272
      main.main()
          _testmain.go:349 +0x269

    Goroutine 956 (finished) created at:
      google.golang.org/grpc/internal/transport.newHTTP2Server()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/internal/transport/http2_server.go:288 +0x18a4
      google.golang.org/grpc/internal/transport.NewServerTransport()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/internal/transport/transport.go:534 +0x2f5
      google.golang.org/grpc.(*Server).newHTTP2Transport()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:726 +0x2ca
      google.golang.org/grpc.(*Server).handleRawConn()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:693 +0x60f
      google.golang.org/grpc.(*Server).Serve.func3()
          /home/ptab/private/golang/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:663 +0x4c
    ==================
    ...
    {"level":"info","ts":"2021-01-09T22:21:04.550+0100","caller":"embed/etcd.go:330","msg":"closed etcd server","name":"default","data-dir":"/tmp/etcd-017337431","advertise-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://localhost:2379"]}
    --- FAIL: TestKVWithEmptyValue (1.08s)
        v3_kv_test.go:62: my-namespace/foobar = data
        v3_kv_test.go:62: my-namespace/foobar1 = data
        v3_kv_test.go:62: namespace/foobar1 = data
        v3_kv_test.go:72: foobar = data
        v3_kv_test.go:72: foobar1 = data
        v3_kv_test.go:87: delete keys:2
        testing.go:1038: race detected during execution of test
```
2021-01-11 10:06:31 +01:00
5b90402082 Switch from dgrijalva/jwt-go to form3tech-oss/jwt-go.
dgrijalva/jwt-go has been abandoned and contains several serious
security issues. Most projects are now switching to the form3tech fork.

See https://snyk.io/vuln/SNYK-GOLANG-GITHUBCOMDGRIJALVAJWTGO-596515 for
info on the issues.

Signed-off-by: Dan Lorenc <dlorenc@google.com>
2021-01-10 08:04:20 -06:00
23340bb62a Refresh proto generation script after moving modules files.
With modulatiozation server protos get moved into ./server directory,
but it was not reflected in scripts/genproto.sh.
2021-01-08 16:33:12 +01:00
c632042bb9 Merge pull request #12489 from zhijianli88/cleanup-tmpfiles
Cleanup tmpfiles
2021-01-07 12:26:04 -05:00
91bdc7eb49 Putting etcd/v3 as a FORBIDDEN_DEPENDENCY for etcdctl. 2021-01-07 08:45:24 -06:00
0abc7515b9 Documentation: Removing etcd v2 specific etcd_http* alerts 2021-01-05 22:28:30 +05:30
a4570a60e7 Merge pull request #12549 from bbiao/bugfix/refresh_token
client/v3: clear auth token when encounter ErrInvalidAuthToken
2020-12-29 22:08:14 -05:00
826573586f transport: catch new cert error
in pkg/transport, we should catch NewCert error.
2020-12-30 09:33:42 +08:00
af4ef4ec04 client/v3: clear auth token when encounter ErrInvalidAuthToken
Old etcdserver which have not apply pr of #12165 will check auth token
even if the request is an Authenticate request.
If the client has a invalid auth token, it will not able to update it's
token, since the Authenticate has a invalid auth token.
This fix clear the auth token when encounter an ErrInvalidAuthToken to
talk with old version etcd servers.

Fix #12385 with #12165 and #12264
2020-12-27 11:08:22 +08:00
a1ff0d5373 Merge pull request #12328 from viviyww/tests-e2e-panic-case
tests: fix test case panic error
2020-12-24 14:54:33 -05:00
a88877ec7b Merge pull request #12579 from ianwoolf/fix-clientv3-readme
docs: update client v3's godoc link in readme
2020-12-23 14:08:30 -08:00
ad5b30297a Merge pull request #12575 from nate-double-u/81-docs-toc-compilation
Documentation Content: TOC — Compilation PR (weights & descriptions)
2020-12-23 15:27:49 -05:00
6c08437ec3 Adding weights and descriptions to Documentation files' frontmatter.
Updating for new TOC as per issue https://github.com/etcd-io/website/issues/81

Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
2020-12-23 11:13:04 -08:00
fc5b5fa31e fix godoc link in clientv3 readme 2020-12-23 10:59:13 +08:00
4726911ea5 Merge pull request #12570 from juzhiyuan/patch-1
docs: update client v3's README
2020-12-21 12:03:50 -05:00
bd97953de3 Update README.md 2020-12-21 17:40:40 +08:00
0539bcb81f Merge pull request #12509 from nate-double-u/81-docs-toc-sections
Documentation Content: TOC — Sections Order
2020-12-19 18:17:35 -05:00
549bff5980 docs: update client v3's README 2020-12-20 00:06:10 +08:00
47dbeeb65a Removing errant commented out replace line 2020-12-15 12:05:26 -06:00
c899316fe8 Adding clientv3 import alias to match usage in register_test.go. 2020-12-15 11:54:03 -06:00
a3174d0f8e Merge pull request #12544 from sid597/learning-authDesign-link-update
[3.4.0] Documentation (learning-design_auth_v3) : Path to client.go updated
2020-12-13 20:12:46 -05:00
cb6951c9f0 Merge pull request #12540 from sid597/learning-why-links-update
[3.4.0] Documentation (Learning-why) : Broken links updated
2020-12-12 21:49:30 -05:00
67da5e9722 Documentation (dev_guide--api_concurrency_reference_v3) : Path to v3lock
and v3election updated

Current path for v3lock and v3election are set to :
etcdserver/api/v3lock/v3lockpb... and etcdserver/api/v3election/v3electionpb... respectively
changed them to server/etcdserver...
2020-12-11 18:21:17 +05:30
07fbd24f3d [3.4.0] Documentation (learning-design_auth_v3) : Path to client.go
outdated
2020-12-11 18:11:26 +05:30
fe4833730d recipes link updated 2020-12-10 23:20:14 +05:30
2e873e6a9f Documentation (Learning-why) : Broken links updated 2020-12-10 22:42:38 +05:30
f2a912a4e6 test: change channel operations to avoid potential goroutine leaks
In these unit tests, goroutines may leak if certain branches are chosen. This commit edits channel operations and buffer sizes, so no matter what branch is chosen, the test will end correctly. This commit doesn't change the semantics of unit tests.
2020-12-09 22:23:21 -05:00
b0e2c70c71 contrib/systemd: add a sysusers entry
This adds a sysusers.d file, in order to create a system user/group
which matches the one used by the service unit.

Ref: https://www.freedesktop.org/software/systemd/man/sysusers.d.html
2020-12-09 13:59:46 +00:00
bedad020eb Adding weights to main section files
Updating for new TOC as per issue https://github.com/etcd-io/website/issues/81

Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
2020-12-08 16:22:29 -08:00
ca866c0242 Merge pull request #12511 from randomizedcoder/failures.md
failures.md explicit cluster members note
2020-12-03 20:02:09 -05:00
0c6357469d failures.md explicit cluster members note 2020-12-03 15:47:07 -08:00
bfc97c1123 Merge pull request #12499 from ptabor/20201127-modules-docs
Documentation: Developer's docs about etcd-3.5.x modules
2020-12-02 17:20:44 -05:00
c0d44ec4b7 Add CodeQL Security Scanning
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
2020-12-02 10:37:09 -08:00
4d173231d1 Documentation: Developer's docs about etcd-3.5.x modules 2020-12-02 10:00:09 +01:00
082b9a815d minor fix on inconsistent format of test case's error message 2020-11-30 12:18:12 +08:00
28d1af294e CHANGELOG-3.4: update release dates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 12:21:32 -08:00
64c409e124 Merge pull request #12491 from gyuho/better-exec
*: validate exec command args, remove unused "iptables" wrapper
2020-11-25 11:26:58 -08:00
b40b46f473 CHANGELOG: updates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:40:42 -08:00
85b61c0f4d pkg/netutil: remove unused "iptables" wrapper
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:38:42 -08:00
3eea37cf31 tools/etcd-dump-metrics: validate exec cmd args
To prevent arbitrary command invocations.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:37:52 -08:00
fb3f91c141 test: cleanup temporary file for TestConfigFromFile
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
2020-11-25 17:40:43 +08:00
63c284211d test: cleanup temporary files for TestZeroToEnd
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
2020-11-25 17:33:43 +08:00
3f6e0ec94b fix: pass argument url in defer to avoid loopclosure
Because of the well-known range loop closure issue, the value of u may
have changed by the time the anonymous function mentioned in the defer
is run. To address this, the simplest fix is to pass the url used in the
loop as an argument to the function run in defer.
2020-11-19 15:29:26 -06:00
b5cefb5b3d Merge pull request #12392 from ironcladlou/fixture-mutations
tests: prevent cross-test contamination via shared state
2020-11-19 10:05:42 -08:00
9571325fe8 etcdserver: fix incorrect metrics generated when clients cancel watches
Before this patch, a client which cancels the context for a watch results in the
server generating a `rpctypes.ErrGRPCNoLeader` error that leads the recording of
a gRPC `Unavailable` metric in association with the client watch cancellation.
The metric looks like this:

    grpc_server_handled_total{grpc_code="Unavailable",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}

So, the watch server has misidentified the error as a server error and then
propagates the mistake to metrics, leading to a false indicator that the leader
has been lost. This false signal then leads to false alerting.

The commit 9c103dd0de introduced an interceptor which wraps
watch streams requiring a leader, causing those streams to be actively canceled
when leader loss is detected.

However, the error handling code assumes all stream context cancellations are
from the interceptor. This assumption is broken when the context was canceled
because of a client stream cancelation.

The core challenge is lack of information conveyed via `context.Context` which
is shared by both the send and receive sides of the stream handling and is
subject to cancellation by all paths (including the gRPC library itself). If any
piece of the system cancels the shared context, there's no way for a context
consumer to understand who cancelled the context or why.

To solve the ambiguity of the stream interceptor code specifically, this patch
introduces a custom context struct which the interceptor uses to expose a custom
error through the context when the interceptor decides to actively cancel a
stream. Now the consuming side can more safely assume a generic context
cancellation can be propagated as a cancellation, and the server generated
leader error is preserved and propagated normally without any special inference.

When a client cancels the stream, there remains a race in the error handling
code between the send and receive goroutines whereby the underlying gRPC error
is lost in the case where the send path returns and is handled first, but this
issue can be taken separately as no matter which paths wins, we can detect a
generic cancellation.

This is a replacement of https://github.com/etcd-io/etcd/pull/11375.

Fixes #10289, #9725, #9576, #9166
2020-11-18 17:02:09 -05:00
c11ddc65ce Merge pull request #12448 from agargi/introduce_config_parameter
server: Added config parameter experimental-apply-warning-duration
2020-11-19 02:29:08 +08:00
c1c681adc3 server: Added config parameter experimental-warning-apply-duration 2020-11-17 17:33:19 -05:00
06e48f0486 Merge pull request #12476 from hexfusion/mixin-typo
Documentation/etcd-mixin: fix typo
2020-11-15 19:19:35 -05:00
07c15890ab Documentation/etcd-mixin: fix typo
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-11-15 12:26:18 -05:00
1b8d2b1a47 Merge pull request #12452 from ptabor/20201104-release-mod-scripts
Release scripts for modules
2020-11-14 03:42:42 -08:00
dc586a5ad2 Merge pull request #12459 from jingyih/proper_request_cancellation
server: proper cancellation for range request
2020-11-13 12:20:41 -08:00
67f040f921 Update other Documentation/v2 links 2020-11-11 09:57:01 +08:00
f2eb15a81b chore: update the documentation link in the comment
Close #12462.
2020-11-11 09:53:18 +08:00
01844fd285 Merge pull request #12455 from mborsz/metrics
Add etcd_server_apply_duration_seconds
2020-11-10 00:47:11 +08:00
718e1a7d89 Merge pull request #12451 from jingyih/update_metrics_doc
Documentation: add generated metrics docs
2020-11-09 23:23:23 +08:00
0bea7df7c1 Add metric tracking apply method duration:
* etcd_server_apply_duration_seconds

It can be used to understand which operations are slow,
in addition to the warning log message.
2020-11-06 11:11:16 +01:00
0558e379c3 server: proper request cancellation for range 2020-11-05 21:30:02 -08:00
eeafcef0d2 Use "v3.5.0-pre" to reference within-etcd modules
instead of v3.0.0-000101010000000-00000000000,
that might be misleading as we don't develop etcd v3.0.0 any longer.

This version is a virtual version and is not supposed to be tagged
within the repository. We should tag real versions like: 3.5.0-alpha.0.

Please notice that go.etcd.io/etcd/client/v2 will be versioned as `v2.305.0-pre`.
The reason is that client v2 must have v2 version. I propose a
convention to envode the major version as 100x in minor version to make
the association to the underlying repository clear, staying within v2
version family.

The change was generated using:
```
DRY_RUN=false TARGET_VERSION="v3.5.0-pre" ./scripts/release_mod.sh update_versions
```
2020-11-04 18:28:43 +01:00
b33c6c088e Documentation: add metrics docs 2020-11-04 22:20:37 +08:00
fd2f34fd13 Release: Scripts to change versions in all go.mod files and push tags upstream.
Examplar invocations:

Edit go.mod files such that all etcd modules are pointing on given version:

```
% DRY_RUN=false TARGET_VERSION="v3.5.13" ./scripts/release_mod.sh update_versions
```

Tag latest commit with current version number for all the modules and push upstream:
```
% DRY_RUN=true REMOTE_REPO="origin" ./scripts/release_mod.sh push_mod_tags
```
2020-11-04 15:16:36 +01:00
6e800b9b01 20201103 no commit title check (#12447)
* Turn off checking of format of commit message.

* scripts/fix.sh: Fix fixing whitespaces in *.sh scripts

Aparently there is a difference between:
  find ./ -print0 -name *.sh and
  find ./ -name *.sh -print0

* etcdserver unit tests: Do not call .Fatalf(...) from not test's goroutine.

Fixes following test failures:
https://travis-ci.com/github/etcd-io/etcd/jobs/425920416
```
% (cd server && go vet ./...)
stderr: # go.etcd.io/etcd/server/v3/etcdserver
stderr: etcdserver/server_test.go:1002:4: call to (*T).Fatalf from a non-test goroutine
stderr: etcdserver/server_test.go:1166:4: call to (*T).Fatalf from a non-test goroutine
FAIL: (code:2):
  % (cd server && go vet ./...)
FAIL: 'run go vet ./...' checking failed (!=0 return code)
FAIL: 'govet' failed at Tue Nov  3 04:07:47 UTC 2020
```
2020-11-03 07:59:42 -08:00
64e048bea9 Merge pull request #12444 from kolyshkin/fix-lock
pkg/fileutil: fix F_OFD_ constants
2020-11-03 23:31:22 +08:00
8c3c398676 Merge pull request #12437 from cfc4n/down_gobin_noexist
scripts: install github.com/myitcv/gobin while gobin doesn't exist.
2020-11-03 23:23:14 +08:00
cd09726ae0 Merge pull request #12430 from meadlai/master
Fix go get cmd
2020-11-03 22:52:24 +08:00
f224fa4e42 Merge pull request #12425 from viviyww/cluster-set-version
etcdserver: updated cluster version
2020-11-03 22:41:24 +08:00
4eb4250e6d pkg/fileutil: fix F_OFD_ constants
Use golang.org/x/sys/unix for F_OFD_* constants.

This fixes the issue that F_OFD_GETLK was defined incorrectly,
resulting in bugs such as https://github.com/moby/moby/issues/31182

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-11-02 19:37:25 -08:00
86185ba20f Merge pull request #12443 from cfc4n/makefile_xargs_r
Makefile: -r is only necessary on GNU xargs.
2020-11-02 10:51:09 -08:00
72ebd50d8a Makefile: -r is only necessary on GNU xargs. 2020-11-02 16:48:48 +08:00
170af891d6 Merge pull request #12429 from tangcong/fix-cert-exp
*: add self-signed-cert-validity flag to fix cert expire issue
2020-10-30 11:41:23 -07:00
2e55875cc7 scripts: install github.com/myitcv/gobin while gobin doesn't exist. 2020-10-30 20:58:00 +08:00
0b4b5d84c6 CHANGELOG: update for 12429 2020-10-30 10:10:30 +08:00
8fd24f51c3 documentation: add certificates expired note 2020-10-30 10:10:30 +08:00
a960d6b1c7 *: add self-signed-cert-validity flag 2020-10-30 10:10:26 +08:00
ec37e15caf Update README.md 2020-10-28 14:34:03 +08:00
aa1024a16e etcdserver: updated cluster version
during cluster version update in etcd cluster, the log should info from
XX to XX.
2020-10-27 16:32:40 +08:00
ed81d2e2db client: replace dial with dialContext 2020-10-27 12:43:14 +08:00
7da5182f1d Merge pull request #12422 from tangcong/fix-realpath
scripts: fix realpath command not found in mac os
2020-10-26 10:42:26 -07:00
ae7862e8bc Merge pull request #12417 from ptabor/20201020-server-module
Modularization: Make ./etcd server a module
2020-10-27 01:27:08 +08:00
8277395e1b scripts: use manual scripts to replace realpath 2020-10-27 00:36:28 +08:00
aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
6c1efd6ba5 server: Update go.mod 2020-10-26 13:02:32 +01:00
4a5e9d1261 server: Move server files to 'server' directory.
26  git mv mvcc wal auth etcdserver etcdmain proxy embed/ lease/ server
   36  git mv go.mod go.sum server
2020-10-26 12:57:19 +01:00
eee8dec0c3 Merge pull request #12421 from ptabor/20201026-fix-ws-shell
Unify tabs vs. spaces in the shell scripts
2020-10-26 04:42:59 -07:00
bc3a77d298 Merge pull request #12099 from YoyinZyc/downgrade-httphandler
[Etcd downgrade] Add http handler to enable downgrade info communication between each member
2020-10-26 04:42:24 -07:00
0ba16d8ee1 *: Convert tabulators to whitespaces in bash scripts.
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t|  |g'
```
2020-10-26 10:59:40 +01:00
c035df5317 test: Detect indention done using tab (\t) in *.sh 2020-10-26 10:59:40 +01:00
8fc5ef4a03 Merge pull request #12418 from ptabor/20201023-fix-flaky
./pkg/testutil: wait for: (*watchGrpcStream).sendCloseSubstream(...) goroutines.
2020-10-24 11:53:10 -07:00
8866d55b9b command: Enhance health command to check if there are any active alarms (#12150) 2020-10-22 15:55:15 -07:00
f2ee15a1e1 ./pkg/testutil: wait for: (*watchGrpcStream).sendCloseSubstream(...) goroutines.
Should solve the problem of flakes documented here:
https://github.com/etcd-io/etcd/issues/12372#issuecomment-706337969

```
% (cd tests && env go test -short -timeout=3m -cpu=4 --race=true ./... -p=2)

Unexpected goroutines running after all test(s).
1 instances of:
go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).sendCloseSubstream(...)
	/go/src/go.etcd.io/etcd/clientv3/watch.go:464 +0x204
created by go.etcd.io/etcd/v3/clientv3.(*watchGrpcStream).closeSubstream
	/go/src/go.etcd.io/etcd/clientv3/watch.go:480 +0x21f
FAIL	go.etcd.io/etcd/tests/v3/integration/clientv3/examples	2.111s
```

The goroutine finishes automatically with timeout of 250ms.The change
makes the test wait for it - if it still exists.

Examples:
  https://travis-ci.com/github/etcd-io/etcd/jobs/397449189
  https://travis-ci.com/github/etcd-io/etcd/jobs/397532784
  https://travis-ci.com/github/etcd-io/etcd/jobs/397696506
  https://travis-ci.com/github/etcd-io/etcd/jobs/403603526
2020-10-22 14:23:08 +02:00
97354af44b Merge pull request #12411 from ptabor/20201021-move-contrib-recipies
Modularization: Move contrib/recipies to clientv3/experimental/recipies/...
2020-10-22 17:40:05 +08:00
45b007b8b4 contrib,clientv3: Move contrib/recipies to clientv3/experimental/recipies/...
Recipies is set of patterns / primitives implementation on top of clientv3.
It's used by integration tests. It shouldn't be considered "server" code.
2020-10-22 11:10:07 +02:00
41557d9330 Merge pull request #12404 from ptabor/20201020-etcdctl-module
Modularization: etcdctl as a module
2020-10-21 21:48:38 +08:00
09679d29ad etcdctl: Rename of imports after making etcdctl a module.
```
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/etcdctl|etcd/etcdctl/v3|g'
```
2020-10-21 11:15:35 +02:00
ae962f7cda Merge pull request #12406 from tangcong/silence-stream-log
embed: only log stream error with debug level
2020-10-20 11:03:37 -07:00
d5363e3560 Merge pull request #12407 from tangcong/fix-networkpartition-flaky-test
clientv3test: fix network partition flaky test
2020-10-20 10:18:56 -07:00
0a830c871e clientv3test: fix network partition flaky test 2020-10-20 23:13:23 +08:00
27d308e173 embed: only log stream error with debug level 2020-10-20 21:56:06 +08:00
df48e499ea etcdctl: Make etcdctl a module (go.mod, LICENSE) 2020-10-20 12:07:27 +02:00
7c6c2c142b Merge pull request #12400 from ptabor/20201015-clientv2
Modularization: Migrate clientv3 to go.etcd.io/etcd/client/v3
2020-10-20 18:03:04 +08:00
e33c6dd9df client/v3: Rename of imports 2020-10-20 10:13:06 +02:00
cf795c0ae6 client/v3: Create and update go.mod 2020-10-20 10:09:12 +02:00
8a7f15511c clientv3: Move to client/v3 (just file move)
Mechanical:
% git mv clientv3 client/v3
% git mv client/mock client/v3/
2020-10-20 10:08:35 +02:00
5d930e5dc1 Merge pull request #12395 from ptabor/20201007-raft
Modularization: Make /raft a module
2020-10-20 16:00:08 +08:00
15f507f6b5 etcdserver: Fix 64 KB websocket notification message limit (#12403)
This fixes etcd being unable to send any message longer than 64 KB as
a notification over the websocket. This was because the older version
of grpc-websocket-proxy was used and WithMaxRespBodyBufferSize option
wasn't set.

Co-authored-by: Vitaliy Filippov <vitalif@yourcmc.ru>
2020-10-19 14:36:06 -07:00
e62417297d *: Rename of imports of raft (as its now a module)
% find -name '*.go' -o -name '*.md' -o -name '*.sh' | xargs sed -i --follow-symlinks 's|etcd/v3/raft|etcd/raft/v3|g'
2020-10-16 13:58:18 +02:00
bea5f379bc raft: Make raft a module
Establishing a module: Creating a go.mod files
2020-10-16 09:15:48 +02:00
b7f0f52a16 *: Refresh of dependencies (#12399)
This PR focuses on dependencies that have new stable versions.
2020-10-15 15:32:00 -07:00
e42127658c Merge pull request #12391 from ptabor/20201013-module-client
Modularization: Move /client/ -> /client/v2 and make it a module
2020-10-15 10:26:44 -07:00
45e4306508 client: Move client to client/v2 as a module.
We make v2 client code a module go.etcd.io/etcd/client/v2.

Pretty mechanical change that can be summarized as:

  mkdir client/v2
  cd client/v2 && git mod init go.etcd.io/etcd/client/v2

  git mv client/*.go client/v2/
  find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'

  + fixing changelog, bom, go.mod etc.
2020-10-15 14:39:43 +02:00
37c95c9fd4 tests: prevent cross-test contamination via shared state
The e2e tests can be flaky due to various tests mutating shared mutable
fixtures, causing non-deterministic behavior depending on the test set, order,
etc.

For example, `configTLS` is mutated in at least two tests in such a way that the
config is potentially invalidated for any subsequent test running in the same
process (e.g. by setting the `enableV2` field). This particular example caused
a substantial amount of confusion diagnosing the new test introduced for
https://github.com/etcd-io/etcd/pull/12370.

Independent tests should not share mutable state unless deliberately. This patch
refactors the e2e test config fixtures to safeguard against these problems by
replacing the package variables (which cannot easily be made immutable) with
functions that return new instances.
2020-10-14 13:58:00 -04:00
d0d927f0db Merge pull request #12397 from ptabor/20201015-update-protos2
tools: Update tools & go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6)
2020-10-14 10:31:00 -07:00
371ddf0b69 tests: Update diagnostic update in tests after change of proto version. 2020-10-14 18:46:38 +02:00
cc7b4fab9f *: Execution of ./scripts/genproto.sh after versions change 2020-10-14 18:46:38 +02:00
da5ca2c0d5 tools: Update of go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6)
This brings consistency between proto-generation code and actual versions of libraries being used in runtime:

github.com/gogo/protobuf                        v1.2.1,v1.0.0 -> v1.3.1
github.com/golang/protobuf                             v1.3.2 -> v1.3.5
github.com/grpc-ecosystem/grpc-gateway  v1.9.5,v1.4.1,v1.15.2 -> v1.14.6
google.golang.org/grpc                                v1.26.0 -> v1.29.1

Moved as far as possible, without bumping on grpc 1.30.0 "naming" decomissioning.
Please also notice that gogo/protobuf is likely to reach EOL: https://github.com/gogo/protobuf/issues/691
2020-10-14 18:46:38 +02:00
b08a46d37d Merge pull request #12396 from ptabor/20201014-deps
test: Adding dependencies consistency check
2020-10-14 20:44:29 +08:00
f7f17dd6a1 test: Adding dependencies consistency check
Examplar output for artificially inconsistent dependencies:

```
% PASSES="dep" ./test
% 'env' 'GO111MODULE=off' 'go' 'get' 'github.com/myitcv/gobin'
% 'env' 'GO111MODULE=off' 'go' 'get' 'github.com/myitcv/gobin'
% 'gofail' 'disable' 'etcdserver/' 'mvcc/backend/'
Running with -cpu=4 --race

'dep' started at Wed 14 Oct 2020 09:56:34 AM CEST
% (cd api && 'go' 'list' '-m')
% (cd api && 'go' 'list' '-f' '{{if not .Indirect}}{{if .Version}}{{.Path}},{{.Version}},go.etcd.io/etcd/api/v3{{end}}{{end}}' '-m' 'all')
% (cd pkg && 'go' 'list' '-m')
% (cd pkg && 'go' 'list' '-f' '{{if not .Indirect}}{{if .Version}}{{.Path}},{{.Version}},go.etcd.io/etcd/pkg/v3{{end}}{{end}}' '-m' 'all')
% 'go' 'list' '-m'
% 'go' 'list' '-f' '{{if not .Indirect}}{{if .Version}}{{.Path}},{{.Version}},go.etcd.io/etcd/v3{{end}}{{end}}' '-m' 'all'
% (cd tests && 'go' 'list' '-m')
% (cd tests && 'go' 'list' '-f' '{{if not .Indirect}}{{if .Version}}{{.Path}},{{.Version}},go.etcd.io/etcd/tests/v3{{end}}{{end}}' '-m' 'all')
FAIL: inconsistent versions for depencency: github.com/coreos/go-semver
github.com/coreos/go-semver:
  - v0.2.0 from: go.etcd.io/etcd/api/v3
  - v0.3.0 from: go.etcd.io/etcd/v3
FAIL: inconsistent dependencies
FAIL: 'dep' failed at Wed 14 Oct 2020 09:56:35 AM CEST
```
2020-10-14 09:57:27 +02:00
1b4f20f3df tests: fix test case panic error
Fix test case in tests/e2e/etcd_config_test.go:311, here should
check if it is nil. As the following errors:

--- FAIL: TestGrpcproxyAndCommonName (0.00s)
    etcd_config_test.go:304: Unexpected error: fork/exec ../../bin/etcd: no such file or directory
    etcd_config_test.go:309: Unexpected error: fork/exec ../../bin/etcd: no such file or directory
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x56bd96]
2020-10-14 11:30:23 +08:00
5d3609e3cf contrib: del systemd/etcd2-backup-coreos in docs
del systemd/etcd2-back-coreos in docs
2020-10-14 09:49:56 +08:00
afc5a717c9 Merge pull request #12390 from ptabor/20201013-scripts-fixes
tools: Migrate remaining tools to gobin
2020-10-13 13:58:51 -04:00
bc9e433ca2 tools: Migrate remaining tools to gobin
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.

Side changes:
  - Expose /scripts/fix.sh that fixes formatting and bom across modules
  - Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
  - Make stderr output of commands explicit and make commands use
different color than callouts.
2020-10-13 19:33:01 +02:00
061d8e8ead Merge pull request #12354 from ptabor/202009027-module-pkgs
Modularization: Make pkg a module (and distribute not pkg code)
2020-10-13 12:56:21 -04:00
667ed96957 wal: remove redundant war tmp dirs if create wal failed (#12388)
If we create 000000000-0000000.war failed in war tmp dir(XXX.tmp) and we should
remove the tmp dirs. If create war successfully in tmp dir and it will
be renamed to common dir and remove tmp dir(XXX.tmp) will also be ok.

Co-authored-by: yangweiwei <yangweiwei@cmss.chinamobile.com>
2020-10-13 08:11:52 -07:00
f408939aa1 bom: Update bill-of-materials generation logic (and refresh bom). 2020-10-13 07:50:08 +02:00
04f63b6571 go.mod,tests: Update go.mod for tests and . to reflect new dependencies. 2020-10-13 00:09:27 +02:00
de55bb6331 pkg: Rename imports after making 'pkg' a module
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
bfa91d48af pkg: Estabilish pkg as a module 2020-10-12 23:58:09 +02:00
1983c89e27 pkg/logutil, etcdserver: Move zap-raft logger to etcdserver.
This package depends on raft and is solelly used by etcdserver/raft.go.
So it does not fullfills conditions of:
```
pkg/ is a collection of utility packages used by etcd without being specific to etcd itself. A package belongs here
only if it could possibly be moved out into its own repository in the future.
```
2020-10-12 23:58:09 +02:00
5ba46f625f pkg/mock: Update imports to point on new files locations.
find ./ -name '*.go' | xargs sed -i 's|go.etcd.io/etcd/v3/pkg/mock/mockserver|go.etcd.io/etcd/v3/client/mock/mockserver|g'
find ./ -name '*.go' | xargs sed -i 's|go.etcd.io/etcd/v3/pkg/mock|go.etcd.io/etcd/v3/server/mock|g'
go fmt ./...
2020-10-12 23:58:09 +02:00
28036db6f0 Mechanical: Move mock packages out of ./pkg
Mechanical execution of:

```
git mv ./pkg/mock/mockserver ./client/pkg/mock
git mv ./pkg/mock/{mockstorage,mockstore,mockwait} ./server/pkg/mock
```

The packages depend on etcd API / protos - so they are NOT etcd-dependencies.
In such situation thay should be placed in 'pkg' folder.
2020-10-12 23:58:09 +02:00
0b95e8cef1 Merge pull request #12374 from ptabor/20201009-new-genproto
script/genproto.sh: Refactor to be explicit about versions.
2020-10-12 14:25:43 -07:00
fa83da446b Merge pull request #12369 from ptabor/20201007-snapshot-out
Modularization: clientv3/snapshot -> etcdctl/snapshot
2020-10-12 14:24:45 -07:00
891c69a5cf Merge pull request #12386 from 2hamed/patch-1
README: Fixed the link to `functional` test dir
2020-10-12 14:55:48 -04:00
f67b25102c e2e: increase the dialTimeout to fix the flaky test TestIssue6361 2020-10-12 11:13:46 -07:00
3e8ffc7cda etcdserver: add getDowngradeEnabled http handler; attach downgrade monitor to server to monitor downgrade status. 2020-10-12 10:50:35 -07:00
386bf316a4 README: Fixed the link to test dir 2020-10-11 20:40:34 +02:00
30811a06aa etcdctl, clientv3/snapshot: snapshot (Restore,Status) code out of client
"snapshot" Restore/Status code was the only remaining dependency of client on 'server'
code. The code is solelly used by etcdctl. Long-term the snapshot code
should be migrated to 'etcdadm' style of tool such that we can
distinguish tool solelly depending on networking API vs. tools that
operation on etcd files directly.

We left snapshot.Save() code in clientv3.snapshot package, such that
clients can benefits from automated download&safe to file snapshot
functionality over the wire.
2020-10-10 14:14:57 +02:00
00e49d0c10 build: Fix updating 'go.etcd.io/etcd/api/v3/version.GitSHA' + test (#12382)
During move of code to 'api' (0aab02e7b5),
I overlooked that the ./build script is setting the version on the 'symbol'.

I added a code to ./build script that checks whether the symbol's are in sync.
2020-10-09 17:32:15 -07:00
e1bf097928 Merge pull request #12266 from chaitanyaenr/etcd_fsync_alert
Add alerts for etcd fsync duration
2020-10-09 14:42:12 -04:00
bc0d8dad1e Merge pull request #12380 from ptabor/20201009-move-single-integration
[important] tests/integration: Move misplaced integration test integration/v3_kv_test.go
2020-10-09 13:14:15 -04:00
485030052a tests/integration: Move misplaces integration tests.
There was undetected 'conflict' between
  11ba1a6109
and 2c66612e0e.

Moving the file to proper location.
2020-10-09 09:39:55 +02:00
24724af7f1 etcdserver: prevent crash using expvars when server is not running (#12376)
The raft.status expvar is added at init time.
This change ensures that evaluating that expvar variable
doesn't panic during evaluation, even when there is
no server running.
2020-10-08 17:37:25 -07:00
11ba1a6109 namespace: check IsWithFromKey if keyLen equal 0. (#12307)
* namespace: check IsWithFromKey if keyLen equal 0.

Rename function isWithFromKey/isWithPrefix to IsOptsWithFromKey/IsOptsWithPrefix.

fixes: #12282

* integration: add test while WithFromKey/WithPrefix called in opts.
2020-10-08 17:34:09 -07:00
dfdda47bd8 script/genproto.sh: Refactor to be explicit about versions.
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
  - https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
  - uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
  - caches tools between executions
  - guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)

The change is no-op for the generated code.

The commit reveals a few 'worring things':
  1  We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
  2. And also     : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
  3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
2020-10-08 19:52:27 +02:00
2c66612e0e Merge pull request #12339 from ptabor/202009027-module-tests
Modularization: Establish 'testing' module (integration, functional, e2e)
2020-10-08 09:25:57 -07:00
f67956cb7a clientv3: Expose clientv3/examples close to the code.
Many of the tests had missing '// Output:' comment, so were not
runnable. They required fining.
2020-10-08 14:27:32 +02:00
dd45d04b2d clientv3/concurrency: Expose examples close to the source-code. 2020-10-07 15:46:56 +02:00
c5ccebf792 tests/integration: Simplify the testMain for examples.
We introduce a LazyCluster abstraction (instead of copy-pasted logic)
that makes clusters to be created only if there are runnable tests
in need for the infrastructure.
2020-10-07 15:42:35 +02:00
73b92fe688 tests: Make examples (for not client) to be both: documentation and integration-runnable
This CL tries to connect 2 objectives:
  - Examples should be close (the same package) to the original code,
    such that they can participate in documentation.
  - Examples should be runnable - such that they are not getting out of
    sync with underlying API/implementation.

In case of etcd-client, the examples are assuming running 'integration'
style, i.e. thay do connect to fully functional etcd-server.
That would lead to a cyclic dependencies between modules:
  - server depends on client (as client need to be lightweight)
  - client (for test purposes) depend on server.
Go modules does not allow to distingush testing dependency from
prod-code dependency.

Thus to meet the objective:
  - The examples are getting executed within testing/integration packages against real etcd
  - The examples are symlinked to 'unit' tests, such that they included in documentation.
  - Long-term the unit examples should get rewritten to use 'mocks' instead of real integration tests.
2020-10-07 15:38:40 +02:00
313087f8c5 ./test,./scripts: Update go.sum and bill-of-materials generation logic (and refresh bom). 2020-10-07 15:38:39 +02:00
9866c7e8ff .words: Expanded and resorted .words dictionary file.
Should fail following goword complains:
```
clientv3/config.go.48: // ("--max-request-bytes" flag to etcd or "embed.Config.MaxRequestBytes"). (spell: MaxRequestBytes -> ?)
clientv3/config.go.55: // ("--max-request-bytes" flag to etcd or "embed.Config.MaxRequestBytes"). (spell: MaxRequestBytes -> ?)
clientv3/leasing/doc.go.15: // Package leasing serves linearizable reads from a local cache by acquiring (spell: linearizable -> infeasible?)
clientv3/op.go.413: // it's linearizable. Serializable requests are better for lower latency (spell: linearizable -> infeasible?)
clientv3/retry.go.49: // an obvious server-side error (e.g. rpctypes.ErrRequestTooLarge). (spell: ErrRequestTooLarge -> Erectile?)
```
2020-10-07 15:38:39 +02:00
7886ec8574 test: Modules: PASSES="integration" ./test understand new file locations 2020-10-07 15:38:39 +02:00
be4e8b7013 integration/fixtures: Move the 'tests/integration/fixtures' directory up and update references.
I moved the files up as they are shared between e2e & integrational tests.
2020-10-07 15:38:28 +02:00
bb9703820c tests/integration: Move of naming/snapshot clientv3 tests (and yaml reference)
path change in clientv3/yaml/config_test

git mv clientv3/naming/grpc_test.go         tests/integration/clientv3/grpc_test.go
git mv clientv3/snapshot/*_test.go          tests/integration/snapshot
git mv clientv3/{main_test.go,example_*.go} tests/integration/clientv3/examples/
2020-10-07 15:37:24 +02:00
ffd4d5173c tests/mod.go: Update of go.mod after move of integration tests. 2020-10-07 15:37:24 +02:00
b27b17b44c tests/functional: Mechanical rename of imports etcd/v3/integration/ -> etcd/tests/v3/integration/... 2020-10-07 15:37:21 +02:00
3153038ffb tests/integration: Moving integration tests to tests/integration directory
git mv integration ./tests

git mv client/integration/* ./tests/integration/client
git mv clientv3/integration/* ./tests/integration/clientv3
git mv client/example_keys_test.go client/main_test.go tests/integration/client/examples
git mv clientv3/concurrency/*_test.go tests/integration/clientv3/concurrency

git mv etcdserver/api/v2store/store_v2v3_test.go etcdserver/api/v2v3/*_test.go tests/integration/v2store
git mv tests/integration/v2store/store_v2v3_test.go tests/integration/v2store/store_v2v3.go
git mv tests/integration/v2store/store_test.go tests/integration/v2store/store_v2v3_test.go
git mv etcdserver/api/v2store/store_test.go tests/integration/v2store
git mv etcdserver/api/v2store/store_v2_test.go tests/integration/v2store

git mv proxy/grpcproxy/*_test.go tests/integration/proxy/grpcproxy

git mv ./clientv3/snapshot/testdata ./clientv3/snapshot/*_test.go ./tests/integration/snapshot
2020-10-07 15:36:21 +02:00
b382429d01 tests: Move functional tests to 'tests' module.
Tested with:

PASSES="fmt functional unit" ./test
make build-docker-functional
2020-10-07 15:09:26 +02:00
8907b146d4 tests/functional: Mechanical rename of imports etcd/v3/functional/ -> etcd/tests/v3/functional/...
% find ./ -name "*.go" | xargs sed -i "s|go.etcd.io/etcd/v3/functional/agent|go.etcd.io/etcd/tests/v3/functional/agent|g"
% find ./ -name "*.go" | xargs sed -i "s|go.etcd.io/etcd/v3/functional/runner|go.etcd.io/etcd/tests/v3/functional/runner|g"
% find ./ -name "*.go" | xargs sed -i "s|go.etcd.io/etcd/v3/functional/tester|go.etcd.io/etcd/tests/v3/functional/tester|g"
% find ./ -name "*.go" | xargs sed -i "s|go.etcd.io/etcd/v3/functional/rpcpb|go.etcd.io/etcd/tests/v3/functional/rpcpb|g"
2020-10-07 15:09:12 +02:00
69254d8cf8 functional,tests: git mv functional tests/
Modularization: Mechanical move of "functional" directory into tests module using:

% git mv functional tests/
2020-10-07 15:03:51 +02:00
4fb48b7c14 tests: Estabilishing module for tests
For now it contains only e2e tests, but integration & functional will follow.
2020-10-07 15:03:51 +02:00
0aab02e7b5 Merge pull request #12367 from ptabor/20201005-api2client
Modularization: Move dependencies of client (protos, version) to api/ module
2020-10-07 05:56:11 -07:00
997961ebfd bom: Update bill of materials to reflect the new module. 2020-10-06 12:28:40 +02:00
ec3026fdc9 *: Run ./scripts/genproto.sh (protoc 3.12.3) after proto file moves.
The changed blobs are consequences of proto-descriptors changing as a
result of file moves.
2020-10-06 11:57:19 +02:00
28f2b07623 *: Update references to code moved to the api/ dir.
Follow up to file-moves done in the previous commit.

The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):

  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'

  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'

  I also modified manually paths in scripts/genproto.sh.

  % go fmt ./...
2020-10-06 11:56:16 +02:00
2edb08642c api: Make api/ a module that will contain proto-definitions.
The module is supposed to contain minimal set of files that establish
public etcd server API. In particular client libraries for etcd built in
different languages might want to depend on this file.
2020-10-06 11:54:50 +02:00
389642dd16 client: Move client specific code (protos, version) to api/
client: Move client specific code (protos, version) to the api/
directory. Thanks to this change /client directory will not need to depend on
the server code. In next commits we make "/api" a module on its own.

Mechanical consequences of execution:

% git mv version/version.go api/version
% git mv etcdserver/api/v3rpc/rpctypes api/v3rpc
% git mv mvcc/mvccpb api/
% git mv etcdserver/etcdserverpb api/
% git mv auth/authpb api/
% git mv etcdserver/api/membership/membershippb api/
2020-10-06 11:53:36 +02:00
7bd956fa2b Merge pull request #12366 from guusvw/fix-yaml-indention-doc
the example alert file had a wrong indentation
2020-10-05 14:08:12 -04:00
985d4cffc4 Documentation: the example alert file had a wrong indentation
Signed-off-by: Guus van Weelden <guus.vanweelden@moia.io>
2020-10-05 18:11:21 +02:00
0693e2b4df Merge pull request #12355 from cfc4n/changelog_gettoken
CHANGELOG: update for #12165 , #12264 .
2020-10-05 11:14:09 -04:00
fdb3f89730 Merge pull request #12362 from ptabor/20201001-deflake-unit-race
Fix "race" - auth unit tests leaking goroutines
2020-10-04 20:47:52 -07:00
97820f1c6e integration: Fix flakes of TestV3WatchRestoreSnapshotUnsync
```
```

The flakes manifested as:
```
--- FAIL: TestV3WatchRestoreSnapshotUnsync (3.59s)
    v3_watch_restore_test.go:82: inflight snapshot sends expected 0 or 1, got ""
FAIL
coverage: 55.2% of statements
FAIL	go.etcd.io/etcd/v3/integration	3.646s
FAIL
```

The root reason is that all the SnapMsg processing happends on both ends
(leader, follower) assynchronously in goroutines, e.g. on Fifo schedule
within EtcdServer.run, so when we observe through metrics, we don't
know whether it finised (or even got started).

Idally we should have EtcdServer.Drain() call that exits when the server
processed or internal 'queues' and is idle.
2020-10-03 19:39:08 +02:00
98b123f034 mvcc: Fix races between metrics gathering and mvcc.Restore
The races was manifesting as following flakes:

```
```
See:
  https://github.com/etcd-io/etcd/issues/12336

I'm taking the locks for short-duration of time (instead of the whole
duriation of Restore) to allow metrics being gather when the server
restoration is in progress.

```
{"level":"warn","ts":"2020-09-26T13:33:13.010Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-c9c21e47-2013-4776-8e83-e331b2caa9ae/localhost:14422410081761184170","attempt":0,"error":"rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial unix localhost:14422410081761184170: connect: no such file or directory\""}
{"level":"warn","ts":"2020-09-26T13:33:13.011Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-c9c21e47-2013-4776-8e83-e331b2caa9ae/localhost:14422410081761184170","attempt":0,"error":"rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial unix localhost:14422410081761184170: connect: no such file or directory\""}
{"level":"warn","ts":"2020-09-26T13:33:16.285Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-b504e954-e000-42a4-aa4f-70ded8dbef39/localhost:55672762955698614610","attempt":0,"error":"rpc error: code = NotFound desc = etcdserver: requested lease not found"}
{"level":"warn","ts":"2020-09-26T13:33:21.434Z","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-7945004b-f67e-42aa-af11-a7b40fbbe6fc/localhost:49623072144007561240","attempt":0,"error":"rpc error: code = Canceled desc = context canceled"}
==================
WARNING: DATA RACE
Write at 0x00c000905f78 by goroutine 764:
  go.etcd.io/etcd/v3/mvcc.(*store).restore()
      /go/src/go.etcd.io/etcd/mvcc/kvstore.go:397 +0x773
  go.etcd.io/etcd/v3/mvcc.(*store).Restore()
      /go/src/go.etcd.io/etcd/mvcc/kvstore.go:343 +0x5f1
  go.etcd.io/etcd/v3/mvcc.(*watchableStore).Restore()
      /go/src/go.etcd.io/etcd/mvcc/watchable_store.go:199 +0xe2
  go.etcd.io/etcd/v3/etcdserver.(*EtcdServer).applySnapshot()
      /go/src/go.etcd.io/etcd/etcdserver/server.go:1107 +0xa49
  go.etcd.io/etcd/v3/etcdserver.(*EtcdServer).applyAll()
      /go/src/go.etcd.io/etcd/etcdserver/server.go:1031 +0x6d
  go.etcd.io/etcd/v3/etcdserver.(*EtcdServer).run.func8()
      /go/src/go.etcd.io/etcd/etcdserver/server.go:986 +0x53
  go.etcd.io/etcd/v3/pkg/schedule.(*fifo).run()
      /go/src/go.etcd.io/etcd/pkg/schedule/schedule.go:157 +0x11e
Previous read at 0x00c000905f78 by goroutine 180:
  [failed to restore the stack]
Goroutine 764 (running) created at:
  go.etcd.io/etcd/v3/pkg/schedule.NewFIFOScheduler()
      /go/src/go.etcd.io/etcd/pkg/schedule/schedule.go:70 +0x2b1
  go.etcd.io/etcd/v3/etcdserver.(*EtcdServer).run()
      /go/src/go.etcd.io/etcd/etcdserver/server.go:871 +0x32c
Goroutine 180 (running) created at:
  net/http.(*Server).Serve()
      /usr/local/go/src/net/http/server.go:2933 +0x5b6
  net/http/httptest.(*Server).goServe.func1()
      /usr/local/go/src/net/http/httptest/server.go:308 +0xd3
==================
--- FAIL: TestV3WatchRestoreSnapshotUnsync (6.74s)
    testing.go:906: race detected during execution of test
FAIL
coverage: 83.5% of statements
FAIL	go.etcd.io/etcd/v3/integration	231.272s
FAIL
Command 'go test -timeout=30m -cpu=1 --race --cover=true go.etcd.io/etcd/v3/integration' failed.
```
2020-10-03 19:39:08 +02:00
220f711a2a clientv3/integration: Fix leaked goroutine in case of skipped test. 2020-10-03 19:38:54 +02:00
528f5315d6 auth: Fix "race" - auth unit tests leaking goroutines
- We were leaking goroutines in auth-test
  - The go-routines were depending / modifying global test environment
variables (simpleTokenTTLDefault) leading to races

Removed the leaked go-routines, and expanded 'auth' package to
be covered we leaked go-routines detection.
2020-10-03 19:38:30 +02:00
0e5d81704f .travis.yml, scripts: Fix minor bugs in the test script.
1. setting environment variable cannot be in quote
2. "--race" testing for unit tests is supposed to be part of linux-amd64-unit-4-cpu-race config.
3. 'run' function in test script should log_error in case of failed
command (wrong operator for ints comparison in bash).
2020-10-01 14:34:58 +02:00
b0e8ec951c CHANGELOG: update for #12165 , #12264 . 2020-09-30 16:57:15 +08:00
ab4cc3caef raft/log: remove redundant code logic (#12346)
Remove redundant code logic

Co-authored-by: yangweiwei <yangweiwei@cmss.chinamobile.com>
2020-09-29 19:48:32 -07:00
a6b0375b7b Travis: Reduce footprint of unit tests - so hopefully flakiness (#12350)
* ./tests: Remove legacy coverage collection code

The legacy tests/cover.test.bash script was not ./test script
compatible for a long time.

The following method of coverage collection works (also across
packages) and does not make all the test execution slower.

```
COVERDIR=coverage PASSES="build build_cov cov" ./test
go tool cover -html ./coverage/cover.out
```

* CI: Reduce duplicated coverage between different variants on Travis

We used to execute unit tests in 3 different jobs,
every time with --race detection and every time in 3 variants:1,2,4
CPUS.

The proposed change makes each of the jobs use different variant of
CPUS, and only 4-cpu variant is running with --race detection
(as the more-parallel variant is more likely to experience races),
2020-09-29 19:47:05 -07:00
b47cd2f470 Merge pull request #12322 from ptabor/20200920-test-script
./test: Refactoring of test script for modularization
2020-09-29 11:09:24 -04:00
3022bd73ce Documentation/etcd-mixin/mixin.libsonnet: Add alerts for etcd fsync duration
This commit adds support to check the 99th percentile of the etcd
members fsync duration and fires a critical alert when it is greater
than 1 sec. The recommended fsync for etcd is 20 ms but there might
be scenarios where a user might be using bad disks for reasons. This
will make sure to let the user/admin know that it is critical for
etcd performance.
2020-09-28 08:08:20 -04:00
b3bbe10465 go.sum: Update & make sure PASSES="mod_tidy" ./test detects such problems.
Commit inspired by this failure:
  https://travis-ci.com/github/etcd-io/etcd/jobs/391164537

This is not happanning locally - but can be forced by removal of go.sum
file. Let's watch how frequently we will need to refresh go.sum.
2020-09-28 12:00:25 +02:00
f1d4593241 ./test: Refactoring of test script for modularization
This refactoring offers following benefits:

  - Unified way how go test commands are being called (in terms of flags intepretation)
  - Uses standard go mechanisms (like go lists) to find files/packages that are subject for test. The mechanism are module aware.
  - Added instruction how to install tools needed for the tests/checkers.
  - Added colors to the output to make it easier to spot any failure.

Confirmed to work using:
- COVERDIR="./coverage" CPU="4" RACE=false COVER=false PASSES="build build_cov cov" ./test
- CPU="4" RACE=false COVER=false PASSES="e2e functional integration" ./test
- COVERDIR="./coverage" COVER="false" CPU="4" RACE="false" PASSES="fmt build unit build_cov integration e2e integration_e2e grpcproxy cov" ./test
- PASSES=unit PKG=./wal TIMEOUT=1m ./test
- PASSES=integration PKG=./clientv3 TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE=TestNew TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE="\bTestNew\b" TIMEOUT=1m ./test
- PASSES=integration PKG=./client/integration TESTCASE="\bTestV2NoRetryEOF\b" TIMEOUT=1m ./test
- COVERDIR=coverage PASSES="build_cov cov" ./test
2020-09-28 11:07:50 +02:00
cc2b4cd05e etcdserver: add more detailed traces on linearized reading (#12335)
To improve debuggability of `agreement among raft nodes before
linearized reading`, we added some tracing inside
`linearizableReadLoop`.

This will allow us to know the timing of `s.r.ReadIndex` vs
`s.applyWait.Wait(rs.Index)`.
2020-09-26 19:08:36 -07:00
31426b0041 clientv3/ordering: Split mocked part of the test from integration-level test. 2020-09-26 08:44:58 +02:00
d65b5d6791 Makefile: Improve the 'make clean' to remove all the tmp artifacts 2020-09-25 22:20:52 +02:00
16eeedffaa pkg/testutil: Fixing flakes due to >>leak" text/template/parse goroutines.
Examplar flake: https://travis-ci.com/github/etcd-io/etcd/jobs/388806782
```
go test -timeout=5m -cpu=1 --run=Example ./client/...

ok  	go.etcd.io/etcd/v3/client	0.085s
testing: warning: no tests to run
PASS
Unexpected goroutines running after all test(s).
1 instances of:
text/template/parse.(*lexer).emit(...)
	/usr/local/go/src/text/template/parse/lex.go:157
text/template/parse.lexText(...)
	/usr/local/go/src/text/template/parse/lex.go:269 +0x4f0
text/template/parse.(*lexer).run(...)
	/usr/local/go/src/text/template/parse/lex.go:230 +0x37
created by text/template/parse.lex
	/usr/local/go/src/text/template/parse/lex.go:223 +0x190
FAIL	go.etcd.io/etcd/v3/client/integration	0.013s
```
2020-09-25 22:10:43 +02:00
73e5714bc5 integration: 'go test -tags cluster_proxy -v ./integration/... ./clientv3/...' passes now. (#12319)
The grpc-proxy test logic was assuming that the context associated to client is closed,
while in practice all tests called client.Close() without explicit context close.

The current testing strategy is complicated 2 fold:
  - grpc proxy works like man-in-the middle of each Connection issues
from integration tests and its lifetime is bound to the connection.
  - both connections (client -> proxy, and proxy -> etcd-server) are
represented by the same ClientV3 object instance (with substituted
implementations of KV or watcher).

The fix splits context representing proxy from context representing proxy -> etcd-server connection,
thus allowing cancelation of the proxy context.
2020-09-25 12:18:58 -07:00
3f36143790 pkg/traceutil: skip subTraceStart/subTraceEnd steps when logging steps (#12262)
SubTraceStart and SubTraceEnd steps are only placeholders, not really
steps, we should skip them when logging the long duration steps,
otherwise these steps will lead to incorrect start time and duration
 of subsequent steps.
2020-09-25 11:46:06 -07:00
7bf75824bf CHANGELOG: Update changelog about modules instead of ./vendor (#12313)
Follow up to PR: https://github.com/etcd-io/etcd/pull/12279
2020-09-25 11:29:51 -07:00
ed82418799 Documentation: Add etcd database quota alerts (#12249)
This commit:
- Fires a critical alert when the etcd database quota is 95% full
  at any given point of time to alert the user to defrag or increase
  the quota in order to avoid the alarm getting triggered which blocks
  all the writes to etcd meaning there can't be any new objects created.
  This is needed to make sure the cluster supports running large number
  of nodes and objects.
- Fires a warning when there is a sudden surge in etcd writes leading to
  increase in the etcd database quota size at an alarming rate as it
  is disruptive. It might be because of a rougue process and it's
  important to alert the admin.
2020-09-25 11:03:04 -07:00
8050881aaf clientv3:get AuthToken gracefully without extra connection. (#12165)
* etcdserver: check authinfo if it is not InternalAuthenticateRequest.

* credentials: let GetRequestMetadata() return nil when authToken isn't initialized.

* clientv3: get AuthToken gracefully without extra connection.
2020-09-25 11:01:54 -07:00
74fea11ddc Documentation/etcd-mixin: Adhere to monitoring mixins annotation guidelines (#12224)
* replaced `message` annotation field with `description`
* added simple `summary` field

Signed-off-by: paulfantom <pawel@krupa.net.pl>
2020-09-25 10:56:52 -07:00
add86bbd1a pkg/fileutil: use fcntl syscall wrappers from golang.org/x/sys/unix (#12316)
Direct syscalls using syscall.Syscall(SYS_*, ...) should no longer be
used on darwin, see [1]. Instead, use the fcntl libSystem wrappers
provided by the golang.org/x/sys/unix package which implement the same
functionality.

[1] https://golang.org/doc/go1.12#darwin
2020-09-24 23:02:32 -07:00
4136df7933 lease: fix lease expiry bases on wall clock (#12292)
fix https://github.com/etcd-io/etcd/issues/12291
2020-09-24 22:59:25 -07:00
8c192d99df clientv3: get AuthToken automatically when clientConn is ready. (#12264)
fixes: #11954
2020-09-24 22:43:21 -07:00
205a656cc5 Merge pull request #11853 from viviyww/dev1
tools: fix test case errors in etcd-dump-logs
2020-09-21 09:13:31 -07:00
353fcf0924 Merge pull request #12314 from viviyww/incorrect-log
etcdserver: fix log info error
2020-09-21 03:17:59 -07:00
de1550d7c8 Merge pull request #12318 from ptabor/20200920-leak-detection-flake-fix
pkg/testutil: Ignore flakes due to "leaked" testing.runTests goroutine
2020-09-21 01:22:06 -07:00
ff516e3a36 etcdserver: fix log info error
etcdserver#util.go#warnOfExpensiveReadOnlyTxnRequest logs with wrong prefix "read-only range ".
It has to be "read-only txn ".

fixes #12295
2020-09-21 16:00:01 +08:00
c88d1497ee tools: fix test case errors in etcd-dump-logs
Fix test case errors in etcd-dump-logs and the error is the time zone.
When the GOST time zone is CST or PST and the test case will be failed.
So we should set UTC as the standard time zone.
2020-09-21 15:54:47 +08:00
14566556f3 Merge pull request #12283 from teddylear/feature/FixPortOnTest
embed: TestStartEtcdWrongToken now uses dynamic ports instead of default
2020-09-20 08:30:48 -07:00
132098b028 Merge pull request #12311 from ptabor/20200917-proxy-watcher-progress-panic
integration,proxy: Skip WatchRequestProgress test in grpc-proxy mode.
2020-09-20 08:07:20 -07:00
3b92b9f884 embed: TestStartEtcdWrongToken now uses dynamic ports instead of default
To avoid issues with a test failing due to port conflict when etcd is
already running, this test now uses dynamic ports instead.

Fixes #11956
2020-09-19 11:29:22 -04:00
5f9a1394db integration,proxy: Skip WatchRequestProgress test in grpc-proxy mode.
Fixes:
  go test -tags cluster_proxy ./clientv3/integration -v -run TestWatchRequestProgress

Does not fail the grpc-server (completely) by a not implemented RPC.
Failing whole server by remote request is anti-pattern and security
risk.

Prior to the fix, the command line above was failing with:

```
=== RUN   TestWatchRequestProgress/0-watcher
panic: not implemented

goroutine 602 [running]:
go.etcd.io/etcd/v3/proxy/grpcproxy.(*watchProxyStream).recvLoop(0xc0004779d0, 0x0, 0x0)
	/home/ptab/corp/etcd/proxy/grpcproxy/watch.go:275 +0xac5
go.etcd.io/etcd/v3/proxy/grpcproxy.(*watchProxy).Watch.func1(0xc0034f94a0, 0xc0004779d0)
	/home/ptab/corp/etcd/proxy/grpcproxy/watch.go:129 +0x53
created by go.etcd.io/etcd/v3/proxy/grpcproxy.(*watchProxy).Watch
	/home/ptab/corp/etcd/proxy/grpcproxy/watch.go:127 +0x3c8
FAIL	go.etcd.io/etcd/v3/clientv3/integration	0.215s
FAIL
```
2020-09-19 17:27:07 +02:00
04b91945f4 pkg/testutil: Ignore flakes due to "leaked" testing.runTests goroutine.
The flake happened e.g. in:
https://travis-ci.com/github/etcd-io/etcd/jobs/386607570

```
--- PASS: TestWatchClose (0.37s)
PASS
Unexpected goroutines running after all test(s).
1 instances of:
testing.runTests.func1.1(...)
	/usr/local/go/src/testing/testing.go:1289 +0x60
created by testing.runTests.func1
	/usr/local/go/src/testing/testing.go:1289 +0xdb
FAIL	go.etcd.io/etcd/v3/clientv3/integration	344.389s
FAIL
```

This is implementation detail of Go testing.lib and we should not worry.
2020-09-19 17:16:21 +02:00
6d5b77b91b Merge pull request #12315 from BinacsLee/binacs-mvcc-fix-typo
mvcc: fix typo
2020-09-18 09:17:59 -07:00
6968c45f58 mvcc: fix typo 2020-09-18 06:50:56 +00:00
588c021ddb Merge pull request #12308 from zcchew1202/readme-fix
Doc: Add that grpc-proxy is optional in readme
2020-09-17 16:29:18 -04:00
be348f0ea6 Doc: Add that grpc-proxy is optional in readme
The script allows optionally enabling `grpc-proxy` so reflecting it in the doc…
2020-09-17 12:13:18 -04:00
528b01c327 Merge pull request #12303 from ptabor/20200915-v3compactor-metric-reporting
etcdserver: v3compactor should use proper clock for latency (took) reporting
2020-09-17 02:26:56 -07:00
2fee6bd5f3 Merge pull request #12279 from ptabor/20200908-no-vendor
vendor: Get rid of ./vendor
2020-09-17 02:21:56 -07:00
17ceed9b47 etcd.service: Support Graceful Reboot for AIO Node
Currently our sample systemd service file `contrib/systemd/etcd.service`
have startup/shutdown dependency as below:

    [Unit]
    After=network.target

For some rare condition, e.g. bare matel deployment with slow network
startup, IP could not be assigned e arly enough before etcd default
`ETCD_HEARTBEAT_INTERVAL="100"` and `ETCD_ELECTION_TIMEOUT="1000"` get
timeouted, after graceful system reboot.

This cause etcd false negative classify itself use unhealthy, therefore
stop rejoining the remaining online cluster members.

This PR introduce:

  - `etcd.service`: Ensure startup after `network-online.target` and
    `time-sync.target`, so effective network connectivity and synced
    time is available.

The logic is concept proof by
<https://github.com/alvistack/ansible-role-etcd/tree/develop>; also
works as expected with Ceph + Kubernetes deployment by
<https://github.com/alvistack/ansible-collection-kubernetes/tree/develop>.
No more deadlock happened during graceful system reboot, both AIO
single/multiple node with loopback mount.

Also see:

  - <https://github.com/ceph/ceph/pull/36776>
  - <https://github.com/etcd-io/etcd/pull/12259>
  - <https://github.com/cri-o/cri-o/pull/4128>
  - <https://github.com/kubernetes/release/pull/1504>

Signed-off-by: Wong Hoi Sing Edison <hswong3i@gmail.com>
2020-09-17 16:59:12 +08:00
3e1a64913a Merge pull request #12286 from ptabor/20200911-short-test-separated
*: 'go test --short ./...' runs all UNIT tests in <25s
2020-09-16 05:16:18 -07:00
3f449a8548 *: 'go test --short ./...' runs all UNIT tests.
Marked all 'integrational, e2e' as skipped in the --short mode.

Thanks to this we will be able to significantly simplify ./test script.

The run currently takes ~23s.
With (follow up) move of ~clientv3/snapshot to integration tests (as
part of modularization), we can expect this to fall to 5-10s.

```
% time go test --short ./... --count=1
ok  	go.etcd.io/etcd/v3	0.098s
?   	go.etcd.io/etcd/v3/Documentation/learning/lock/client	[no test files]
?   	go.etcd.io/etcd/v3/Documentation/learning/lock/storage	[no test files]
ok  	go.etcd.io/etcd/v3/auth	0.724s
?   	go.etcd.io/etcd/v3/auth/authpb	[no test files]
ok  	go.etcd.io/etcd/v3/client	0.166s
ok  	go.etcd.io/etcd/v3/client/integration	0.166s
ok  	go.etcd.io/etcd/v3/clientv3	3.219s
ok  	go.etcd.io/etcd/v3/clientv3/balancer	1.102s
?   	go.etcd.io/etcd/v3/clientv3/balancer/connectivity	[no test files]
?   	go.etcd.io/etcd/v3/clientv3/balancer/picker	[no test files]
?   	go.etcd.io/etcd/v3/clientv3/balancer/resolver/endpoint	[no test files]
ok  	go.etcd.io/etcd/v3/clientv3/clientv3util	0.096s [no tests to run]
ok  	go.etcd.io/etcd/v3/clientv3/concurrency	3.323s
?   	go.etcd.io/etcd/v3/clientv3/credentials	[no test files]
ok  	go.etcd.io/etcd/v3/clientv3/integration	0.131s
?   	go.etcd.io/etcd/v3/clientv3/leasing	[no test files]
?   	go.etcd.io/etcd/v3/clientv3/mirror	[no test files]
ok  	go.etcd.io/etcd/v3/clientv3/namespace	0.041s
ok  	go.etcd.io/etcd/v3/clientv3/naming	0.115s
ok  	go.etcd.io/etcd/v3/clientv3/ordering	0.121s
ok  	go.etcd.io/etcd/v3/clientv3/snapshot	19.325s
ok  	go.etcd.io/etcd/v3/clientv3/yaml	0.090s
ok  	go.etcd.io/etcd/v3/contrib/raftexample	7.572s
?   	go.etcd.io/etcd/v3/contrib/recipes	[no test files]
ok  	go.etcd.io/etcd/v3/embed	0.282s
ok  	go.etcd.io/etcd/v3/etcdctl	0.054s
?   	go.etcd.io/etcd/v3/etcdctl/ctlv2	[no test files]
ok  	go.etcd.io/etcd/v3/etcdctl/ctlv2/command	0.117s
?   	go.etcd.io/etcd/v3/etcdctl/ctlv3	[no test files]
ok  	go.etcd.io/etcd/v3/etcdctl/ctlv3/command	0.070s
ok  	go.etcd.io/etcd/v3/etcdmain	0.172s
ok  	go.etcd.io/etcd/v3/etcdserver	1.698s
?   	go.etcd.io/etcd/v3/etcdserver/api	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/etcdhttp	0.075s
ok  	go.etcd.io/etcd/v3/etcdserver/api/membership	0.104s
?   	go.etcd.io/etcd/v3/etcdserver/api/membership/membershippb	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/rafthttp	0.181s
ok  	go.etcd.io/etcd/v3/etcdserver/api/snap	0.078s
?   	go.etcd.io/etcd/v3/etcdserver/api/snap/snappb	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2auth	0.142s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2discovery	0.035s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2error	0.043s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2http	0.070s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2http/httptypes	0.031s
?   	go.etcd.io/etcd/v3/etcdserver/api/v2stats	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2store	0.645s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v2v3	0.218s
?   	go.etcd.io/etcd/v3/etcdserver/api/v3alarm	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3client	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/v3compactor	1.765s
?   	go.etcd.io/etcd/v3/etcdserver/api/v3election	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3election/v3electionpb	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3election/v3electionpb/gw	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3lock	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3lock/v3lockpb	[no test files]
?   	go.etcd.io/etcd/v3/etcdserver/api/v3lock/v3lockpb/gw	[no test files]
ok  	go.etcd.io/etcd/v3/etcdserver/api/v3rpc	0.091s
ok  	go.etcd.io/etcd/v3/etcdserver/api/v3rpc/rpctypes	0.012s
ok  	go.etcd.io/etcd/v3/etcdserver/cindex	0.054s
ok  	go.etcd.io/etcd/v3/etcdserver/etcdserverpb	0.039s
?   	go.etcd.io/etcd/v3/etcdserver/etcdserverpb/gw	[no test files]
ok  	go.etcd.io/etcd/v3/functional/agent	0.094s
?   	go.etcd.io/etcd/v3/functional/cmd/etcd-agent	[no test files]
?   	go.etcd.io/etcd/v3/functional/cmd/etcd-proxy	[no test files]
?   	go.etcd.io/etcd/v3/functional/cmd/etcd-runner	[no test files]
?   	go.etcd.io/etcd/v3/functional/cmd/etcd-tester	[no test files]
ok  	go.etcd.io/etcd/v3/functional/rpcpb	0.060s
?   	go.etcd.io/etcd/v3/functional/runner	[no test files]
ok  	go.etcd.io/etcd/v3/functional/tester	0.079s
ok  	go.etcd.io/etcd/v3/integration	0.684s
ok  	go.etcd.io/etcd/v3/integration/embed	0.101s
ok  	go.etcd.io/etcd/v3/lease	3.455s
ok  	go.etcd.io/etcd/v3/lease/leasehttp	2.185s
?   	go.etcd.io/etcd/v3/lease/leasepb	[no test files]
ok  	go.etcd.io/etcd/v3/mvcc	7.246s
ok  	go.etcd.io/etcd/v3/mvcc/backend	0.354s
?   	go.etcd.io/etcd/v3/mvcc/mvccpb	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/adt	0.025s
?   	go.etcd.io/etcd/v3/pkg/contention	[no test files]
?   	go.etcd.io/etcd/v3/pkg/cpuutil	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/crc	0.008s
?   	go.etcd.io/etcd/v3/pkg/debugutil	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/expect	0.015s
ok  	go.etcd.io/etcd/v3/pkg/fileutil	0.268s
ok  	go.etcd.io/etcd/v3/pkg/flags	0.021s
ok  	go.etcd.io/etcd/v3/pkg/httputil	0.020s
ok  	go.etcd.io/etcd/v3/pkg/idutil	0.008s
ok  	go.etcd.io/etcd/v3/pkg/ioutil	0.025s
ok  	go.etcd.io/etcd/v3/pkg/logutil	0.047s
?   	go.etcd.io/etcd/v3/pkg/mock/mockserver	[no test files]
?   	go.etcd.io/etcd/v3/pkg/mock/mockstorage	[no test files]
?   	go.etcd.io/etcd/v3/pkg/mock/mockstore	[no test files]
?   	go.etcd.io/etcd/v3/pkg/mock/mockwait	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/netutil	1.024s
ok  	go.etcd.io/etcd/v3/pkg/osutil	0.021s
ok  	go.etcd.io/etcd/v3/pkg/pathutil	0.008s
ok  	go.etcd.io/etcd/v3/pkg/pbutil	0.008s
ok  	go.etcd.io/etcd/v3/pkg/proxy	4.081s
ok  	go.etcd.io/etcd/v3/pkg/report	0.008s
?   	go.etcd.io/etcd/v3/pkg/runtime	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/schedule	0.009s
ok  	go.etcd.io/etcd/v3/pkg/srv	0.019s
ok  	go.etcd.io/etcd/v3/pkg/stringutil	0.008s
?   	go.etcd.io/etcd/v3/pkg/systemd	[no test files]
ok  	go.etcd.io/etcd/v3/pkg/testutil	0.023s
ok  	go.etcd.io/etcd/v3/pkg/tlsutil	3.965s
ok  	go.etcd.io/etcd/v3/pkg/traceutil	0.034s
ok  	go.etcd.io/etcd/v3/pkg/transport	0.532s
ok  	go.etcd.io/etcd/v3/pkg/types	0.028s
ok  	go.etcd.io/etcd/v3/pkg/wait	0.023s
ok  	go.etcd.io/etcd/v3/proxy/grpcproxy	0.101s
?   	go.etcd.io/etcd/v3/proxy/grpcproxy/adapter	[no test files]
?   	go.etcd.io/etcd/v3/proxy/grpcproxy/cache	[no test files]
ok  	go.etcd.io/etcd/v3/proxy/httpproxy	0.044s
ok  	go.etcd.io/etcd/v3/proxy/tcpproxy	0.047s
ok  	go.etcd.io/etcd/v3/raft	0.312s
ok  	go.etcd.io/etcd/v3/raft/confchange	0.183s
ok  	go.etcd.io/etcd/v3/raft/quorum	0.316s
ok  	go.etcd.io/etcd/v3/raft/raftpb	0.024s
ok  	go.etcd.io/etcd/v3/raft/rafttest	0.640s
ok  	go.etcd.io/etcd/v3/raft/tracker	0.026s
ok  	go.etcd.io/etcd/v3/tests/e2e	0.077s
?   	go.etcd.io/etcd/v3/tools/benchmark	[no test files]
?   	go.etcd.io/etcd/v3/tools/benchmark/cmd	[no test files]
?   	go.etcd.io/etcd/v3/tools/etcd-dump-db	[no test files]
ok  	go.etcd.io/etcd/v3/tools/etcd-dump-logs	0.088s
?   	go.etcd.io/etcd/v3/tools/etcd-dump-metrics	[no test files]
?   	go.etcd.io/etcd/v3/tools/local-tester/bridge	[no test files]
?   	go.etcd.io/etcd/v3/version	[no test files]
ok  	go.etcd.io/etcd/v3/wal	1.517s
?   	go.etcd.io/etcd/v3/wal/walpb	[no test files]
go test --short ./... --count=1  76.12s user 12.57s system 375% cpu 23.635 total
```
2020-09-16 13:21:46 +02:00
138a312643 scripts: Removed updatedep.sh.
As we get rid of ./vendor there is no need to update the dependencies.

If anyone needs up-to-date vendor directory locally, getting it is as
simple as:

```go mod vendor```
2020-09-16 13:04:58 +02:00
2aaf4f3d7a Merge pull request #12278 from ptabor/20200907-fix-cov-integration
integration: Fix 'go test --tags cluster_proxy --timeout=30m -v ./integration/...'
2020-09-16 01:55:09 -07:00
6d07caee50 Merge pull request #12304 from jingyih/update_changelog_12288
CHANGELOG: update from #12288 and #12299
2020-09-15 22:31:01 -04:00
37fe4b9afa CHANGELOG: update from 12288 and 12299 2020-09-15 13:25:25 -07:00
872e5a78a3 clientv3/integration: Fix TestLeasingTxtOwnerGet test getting stuck in 'cluster_proxy' mode:
Fixes:
  go test --tags cluster_proxy --timeout=30m -run TestLeasingTxnOwnerGet -v ./clientv3/integration/...

The explicit code to close client is needed due to:
  76e769ce95/clientv3/watch.go (L72)
as just ctx close by LeasingKeyValue store does not interrupts opened Watches.
The only way to interrupt open Watch is to close the 'whole' Watcher / Client.
2020-09-15 18:06:58 +02:00
3752333162 Merge pull request #12288 from galal-hussein/fix_learner_panic_master
etcdserver: add ConfChangeAddLearnerNode to the list of config changes
2020-09-15 08:59:36 -07:00
2048c80760 etcdserver: v3compactor should use proper clock for latency (took) reporting.
The code used to:
  - report time since previous compaction success as a 'duration' of compaction process itself.
  - mix real clock with 'injected' clock. This led to strange log-lines
in tests in order of 234543543h of duration.
2020-09-15 16:43:05 +02:00
aa10b1ee7f ./test: Add PASSES="mod_tidy" check
Added check that ensures that go.mod & go.sum files are up-to-date.
The check verifies whether 'go mod tidy' does not generate any mutations
in these files.
The check can be run on its own:
  PASSES="mod_tidy" ./test
Or as part of "fmt" pass:
  PASSES="fmt" ./test

Examplar outputs:

```
% PASSES="fmt" ./test

Running with TEST_CPUS: 1,2,4
Starting 'fmt' pass at Fri 11 Sep 2020 11:07:54 PM CEST
'shellcheck' started at Fri 11 Sep 2020 11:07:54 PM CEST
'shellcheck' completed at Fri 11 Sep 2020 11:07:54 PM CEST
'markdown_you' started at Fri 11 Sep 2020 11:07:54 PM CEST
'markdown_you' completed at Fri 11 Sep 2020 11:07:54 PM CEST
'goword' started at Fri 11 Sep 2020 11:07:54 PM CEST
'goword' completed at Fri 11 Sep 2020 11:07:54 PM CEST
'gofmt' started at Fri 11 Sep 2020 11:07:54 PM CEST
'gofmt' completed at Fri 11 Sep 2020 11:07:55 PM CEST
'govet' started at Fri 11 Sep 2020 11:07:55 PM CEST
'govet' completed at Fri 11 Sep 2020 11:07:57 PM CEST
'revive' started at Fri 11 Sep 2020 11:07:57 PM CEST
Skipping revive...
'revive' completed at Fri 11 Sep 2020 11:07:57 PM CEST
'license_header' started at Fri 11 Sep 2020 11:07:57 PM CEST
'license_header' completed at Fri 11 Sep 2020 11:07:58 PM CEST
'receiver_name' started at Fri 11 Sep 2020 11:07:58 PM CEST
'receiver_name' completed at Fri 11 Sep 2020 11:07:58 PM CEST
'commit_title' started at Fri 11 Sep 2020 11:07:58 PM CEST
'commit_title' completed at Fri 11 Sep 2020 11:07:58 PM CEST
'mod_tidy' started at Fri 11 Sep 2020 11:07:58 PM CEST
*** /tmp/fileiALKRA_go.mod	2020-09-11 23:07:58.838010716 +0200
--- ./go.mod	2020-09-11 23:07:58.974010922 +0200
***************
*** 29,39 ****
  	github.com/mattn/go-runewidth v0.0.2 // indirect
  	github.com/modern-go/reflect2 v1.0.1
  	github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
  	github.com/prometheus/client_golang v1.0.0
  	github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
- 	github.com/prometheus/common v0.4.1
  	github.com/sirupsen/logrus v1.4.2 // indirect
  	github.com/soheilhy/cmux v0.1.4
  	github.com/spf13/cobra v0.0.3
  	github.com/spf13/pflag v1.0.1
  	github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
--- 29,38 ----
./go.mod is not in sync with 'go mod tidy'
```

```
% PASSES="mod_tidy" ./test

Running with TEST_CPUS: 1,2,4
Starting 'mod_tidy' pass at Fri 11 Sep 2020 11:09:21 PM CEST
*** /tmp/file9gy4so_go.mod	2020-09-11 23:09:21.166133290 +0200
--- ./go.mod	2020-09-11 23:09:21.286133466 +0200
***************
*** 29,39 ****
  	github.com/mattn/go-runewidth v0.0.2 // indirect
  	github.com/modern-go/reflect2 v1.0.1
  	github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
  	github.com/prometheus/client_golang v1.0.0
  	github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
- 	github.com/prometheus/common v0.4.1
  	github.com/sirupsen/logrus v1.4.2 // indirect
  	github.com/soheilhy/cmux v0.1.4
  	github.com/spf13/cobra v0.0.3
  	github.com/spf13/pflag v1.0.1
  	github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
--- 29,38 ----
./go.mod is not in sync with 'go mod tidy'
```
2020-09-15 14:38:54 +02:00
0f195baca1 ./test: CI (./test) should never edit go.mod nor go.sum files
Tested and received as expected:
```
...
go: updates to go.mod needed, disabled by -mod=readonly
```
2020-09-15 14:38:54 +02:00
d19758b501 go.sum: Regenerate go.sum to keep it fully in sync. 2020-09-15 14:38:54 +02:00
fb2e96e651 vendor: Get rid of ./vendor cleanup
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.

Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.

1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:

```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build  8.22s user 2.31s system 117% cpu 8.961 total
```

Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.

./build  34.97s user 4.15s system 236% cpu 16.555 total
```

6. Rebuild without changes:

```
% time ./build
./build  1.43s user 0.83s system 168% cpu 1.336 total
```

7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):

```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  0.51s user 0.44s system 110% cpu 0.861 total
```

8. Fresh instantation of vendor:

```
% rm -rf vendor
% [sudo] rm -rf ~/go

% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  3.62s user 1.30s system 127% cpu 3.854 total
```

9. Size of the repository - before: 39M, after: 18M

Before:

```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git  4.66s user 1.02s system 93% cpu 6.068 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
52K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
21M	./vendor
39M
```

After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor  5.56s user 1.05s system 105% cpu 6.260 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
56K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
19M	.
```
2020-09-15 14:38:54 +02:00
2d76f9d17d vendor: Remove /vendor and add to .gitignore 2020-09-15 14:38:47 +02:00
0a106754d3 etcdserver: Speed-up v3compactor unit tests.
The tests were taking ~15s because of 5s wait time in Recorder !?
2020-09-15 12:00:46 +02:00
7af8241764 etcdserver: add ConfChangeAddLearnerNode to the list of config changes
To fix a panic that happens when trying to get ids of etcd members in
force new cluster mode, the issue happen if the cluster previously had
etcd learner nodes added to the cluster

Fixes #12285
2020-09-14 17:49:57 +02:00
01bd0d468d Merge pull request #12297 from spzala/issue12093
config: grpc gateway
2020-09-14 08:19:45 -07:00
c7458015d3 config: grpc gateway
Currently, the grpc gateway is enabled by default when run the etcd server
using command line configuration. However, when provide config through
config file, the grpc gateway is disabled by default. We should either use
the same approach or at least document this.

related #https://github.com/etcd-io/etcd/issues/12093
2020-09-14 09:49:53 -04:00
2d43a3157e integration: Fix 'go test --tags cluster_proxy --timeout=30m -v ./integration/...'
grpc proxy opens additional 2 watching channels. The metric is shared
between etcd-server & grpc_proxy, so all assertions on number of open
watch channels need to take in consideration the additional "2"
channels.
2020-09-10 14:54:09 +02:00
32bad8e9a8 integration/cluster_proxy.go: Fix broken build with tags=cov.
Prior to the fix:
```
go test -mod=mod -timeout 30m -tags cluster_proxy "./integration/..."
```

Was failing with:
```
integration/cluster_proxy.go:58:37: not enough arguments in call to grpcproxy.NewWatchProxy
   have (*clientv3.Client)
   want (*zap.Logger, *clientv3.Client)
FAIL    go.etcd.io/etcd/v3/integration [build failed]
```
2020-09-10 14:54:09 +02:00
76e769ce95 Merge pull request #12273 from ptabor/2020-09-07-fix-grpc-proxy-tests
testing/e2e,grpcproxy: Fix: go test --tags "cluster_proxy" -v ./tests/e2e/...
2020-09-09 12:03:09 -07:00
e81cae77a3 Merge pull request #12274 from ptabor/20200907-fix-cov-e2e-tests
tests/e2e,etcdctl,etcdmain: Fix go test --tags cov -v ./tests/e2e
2020-09-09 11:14:18 -07:00
9d5a840942 etcdmain/grpc_proxy: Remove superflous logging line. 2020-09-09 20:04:25 +02:00
7c880e5263 etcdctl: Rename Start / StartWithErrors to MustStart 2020-09-09 19:32:50 +02:00
10fa9614e1 Merge pull request #12271 from jingyih/add_watch_notify_interval_flag_in_testing
integration: add WatchProgressNotifyInterval in integration test
2020-09-09 08:56:48 -07:00
c32180d772 tests/e2e,etcdctl,etcdmain: Fix go test --tags cov -v ./tests/e2e
This CL fixes:
  COVERDIR=./coverage PASSES="build_cov" && go test --tags cov -v ./tests/e2e
and is part of the effort to make:
  COVERDIR=coverage PASSES="build_cov cov" ./test
fully pass.

The args passed to ./bin/etcd_test and ./bin/etcdctl_test binaries were
mismatched. The protocol of passing the arguments using
environment variables has been replaces with proper passing of flags.

How the measurement of coverage by e2e tests works:
  1. COVERDIR=./coverage PASSES="build_cov" are generating
./bin/etcd_test and ./bin/etcdctl_test binaries.

  2. These binaries are tests (as coverage can be computed only for
tests) [see ./main_test.go ./etcdctl/main_test.go], but this tests are
running the main logic of the server and uppon termination (or SIGTERM
signal) are writting proper .coverprofile files in the $COVERDIR folder.
The binaries used to take arguments using env variables, but its not
needed any longer. The binaries can consume any command line arguments
that either test (so --test.fooo) or the original binary can consume.

 3.  The tests/e2e (when compiled with the --tags cov) are starting the
_test binaries instead of the original binaries, such that the coverage
is being collected.
2020-09-09 12:56:15 +02:00
73817b53fd integration: add flag WatchProgressNotifyInterval in integration test 2020-09-07 08:32:54 -07:00
093282f5ea tests/e2e: cluster_proxy tests use CN-less cert for etcd-server auth.
Change tests/e2e to use proper (client-nocn.crt) certificate when
running in tags="cluster_proxy" mode.

Thanks to this (and previous in this PR) changes, the following test run
finally succeeds:
  ./build && go test --tags "cluster_proxy" -v ./tests/e2e/...
2020-09-07 12:55:01 +02:00
2d0ce9de3d etcdmain: grpc-proxy should only require CN-less certificates for --cert flags.
We have following communication schema:
client --- 1 ---> grpc-proxy --- 2 --- > etcd-server

There are 2 sets of flags/certs in grpc proxy [ https://github.com/etcd-io/etcd/blob/master/etcdmain/grpc_proxy.go#L140 ]:
 A. (cert-file, key-file, trusted-ca-file, auto-tls) this are controlling [1] so client to proxy connection and in particular they are describing proxy public identity.
 B. (cert,key, cacert ) - these are controlling [2] so what's the identity that proxy uses to make connections to the etcd-server.

If 2 (B.) contains certificate with CN and etcd-server is running with --client-cert-auth=true, the CN can be used as identity of 'client' from service perspective. This is permission escalation, that we should forbid.

If 1 (A.) contains certificate with CN - it should be considered perfectly valid. The server can (should) have full identity.

So only --cert flag (and not --cert-file flag) should be validated for empty CN.
2020-09-07 11:59:28 +02:00
2c93127c7b integration: Regenerate certificates, add client-nocn.crt
Executed:
(cd ./integration/fixtures && ./gencerts.sh)

This in particular cereated a new client-nocn.crt (and key) that can be
used for testing grpc-proxy -> etcd-server connections.
2020-09-07 11:48:38 +02:00
966e8cecf0 integration: gencerts.sh cleanup and supports no-CN certs
integration/fixtures/gencerts.sh:
  - refactored common logic to a helper function
  - added definition for client-nocn certificate
    (used for grpc-proxy -> etcd-server) communication.
2020-09-07 11:47:24 +02:00
c20cc05fc5 mvcc: Export a "Last DB compaction" timestamp metric (#12176)
This is to aid with debugging the effectiveness of systems that
manually take care of cluster compaction, and have greater visibity
into recent compactions.

It can be handy to alert on the exactly how long it was since a
compaction (and also to put on dashboards) had happened.

---

Tested using a test cluster, the final result looks like this:

```
	root@etcd-1:~# ETCDCTL_API=3 /tmp/test-etcd/etcdctl --endpoints=192.168.232.10:2379 compact 1012
	compacted revision 1012

	root@etcd-1:~# curl -s 192.168.232.10:2379/metrics | grep last
	# HELP etcd_debugging_mvcc_db_compaction_last The unix time since the last db compaction.  Resets to 0 on start.
	# TYPE etcd_debugging_mvcc_db_compaction_last gauge
	etcd_debugging_mvcc_db_compaction_last 1.595873939e+09
```
2020-08-26 16:27:10 -07:00
facd0c9460 Merge pull request #12252 from spzala/changelogfileperm34and33
CHANGELOG: file perm updates in 3.4 and 3.3
2020-08-24 12:30:21 -07:00
1c0d73d248 CHANGELOG: file perm updates in 3.4 and 3.3
Updates for https://github.com/etcd-io/etcd/pull/12250 and
https://github.com/etcd-io/etcd/pull/12251
2020-08-24 11:58:19 -04:00
ae66916226 pkg: file stat warning (#12242)
Provide warning and doc instead of enforcing file permission.
2020-08-23 17:20:16 -07:00
c199d3d8c3 server: use buffered channel to avoid goroutine leak (#11941)
Signed-off-by: Gaurav Singh <gaurav1086@gmail.com>
2020-08-21 18:46:28 -07:00
261aa31dc5 Merge pull request #12243 from hexfusion/bump-x/text
vendor: bump golang.org/x/text
2020-08-21 10:03:01 -04:00
100c443a10 vendor: bump golang.org/x/text
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-08-21 09:14:40 -04:00
76993f1fc6 Merge pull request #12240 from avorima/fdusage_readdirnames
etcdserver: Use Readdirnames to count fds for FDUsage
2020-08-20 21:58:18 -07:00
ab9c14f477 Merge pull request #12241 from philips/add-asset-transparency-github-action
github: workflows: add asset-transparency release action
2020-08-20 14:03:19 -07:00
142358c13d github: workflows: add asset-transparency release action
From etcd-dev discussion:
https://groups.google.com/u/2/g/etcd-dev/c/oMGSBqs_7sc

I have been working on this system called Asset Transparency[1] which
helps users verify they have received the correct contents from a URL.
If you are familiar with the "download a file, download a SHA256SUM
file, run `sha256sum -c`, etc" process? This tool helps to automate
that for users into something like this[2]:

$ tl get https://github.com/etcd-io/etcd/releases/download/v3.4.12/etcd-v3.4.12-darwin-amd64.zip

And a best practice for this Asset Transparency system is that URLs
are registered with the log as soon as possible. Why? Well, the sooner
a URL is entered the longer it can protect people consuming a URL from
unexpected content modification from say a GitHub credential
compromise.

To that end I have written a GitHub Action[3] that will automatically
do that on every release. It is easy to activate and should be hands
free after installation. So, before I enable it I want to see if there
are any concerns from maintainers. The only change to our repo will be
a new file in .github/workflows.

[1] https://www.transparencylog.com
[2] https://github.com/transparencylog/tl
[3] https://github.com/transparencylog/publish-releases-asset-transparency-action
2020-08-20 11:32:36 -07:00
be70400fb5 etcdserver: Use Readdirnames to count fds for FDUsage
Readdir already calls Readdirnames, but continues to allocate
os.FileInfo with Lstat for each result.
2020-08-20 16:51:29 +02:00
4b6a0eea49 CHANGELOG: update with server panic fix
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-19 20:02:52 -07:00
44dea5df03 Merge pull request #12238 from liggitt/slow-v2-panic
etcdserver: Avoid panics logging slow v2 requests in integration tests
2020-08-19 09:20:47 -07:00
ad57fea4c5 etcdserver: Avoid panics logging slow v2 requests in integration tests 2020-08-19 11:30:15 -04:00
cfdc296a3c CHANGELOG: update release-3.2 dates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-18 09:43:45 -07:00
edaac6e2a9 CHANGELOG: add v3.3.24 release dates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-18 09:30:27 -07:00
e37b28bd28 CHANGELOG: add v3.4.11
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-18 09:23:24 -07:00
OG
0526f461e1 Doc: Update curl command to fix 400 Bad Request (#11911) 2020-08-16 16:12:39 -07:00
0e4ba37b6c Merge pull request #12193 from mitake/integration_test
test: avoid non existing package for integration test
2020-08-15 23:26:34 -07:00
d35933c351 Merge pull request #12221 from wenjiaswe/changelog_12215
CHANGELOG: update from 12215
2020-08-14 13:29:40 -07:00
32982ef469 CHANGELOG: update from 12215
Change-Id: I17e076554a56c95fabd95af111eccd8d7409966b
2020-08-14 13:18:02 -07:00
92f9e6eba2 Merge pull request #12216 from jingyih/experimental_flag_for_watch_notify_interval
*: add experimental flag for watch notify interval
2020-08-14 12:47:43 -07:00
799b16c2d1 CHANGELOG: update for PR12216 2020-08-14 12:06:38 -07:00
9a698476bf *: add experimental flag for watch notify interval 2020-08-14 12:01:00 -07:00
06f89cc4f8 Merge pull request #12212 from gyuho/logger
*: upgrade zap logger to 1.15, replace global logger
2020-08-13 09:46:44 -07:00
93cf449205 Merge pull request #12214 from gyuho/fd
*: optimize runtime.FDUsage + add OS level FD metrics
2020-08-12 18:37:05 -07:00
5678779665 CHANGELOG: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 10:32:27 -07:00
421df2ecbb etcdserver: add OS level FD metrics
Similar counts are exposed via Prometheus.
This adds the one that are perceived by etcd server.

e.g.

os_fd_limit 120000
os_fd_used 14
process_cpu_seconds_total 0.31
process_max_fds 120000
process_open_fds 17

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 10:32:27 -07:00
53fdcdc5a2 pkg/runtime: optimize FDUsage by removing sort
No need sort when we just want the counts.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 10:32:24 -07:00
d8ed233791 CHANGELOG: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 09:50:00 -07:00
7eac6bd497 *: upgrade zap logger to 1.15, replace global logger
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-08-12 09:50:00 -07:00
ed27d9d2de Merge pull request #12198 from ptabor/20200803-int-to-string-test-fix
etcdserver, wal: Fix tests unintended CASTing of int->String.
2020-08-11 21:35:20 -07:00
8c44d25f2a Merge pull request #12211 from tangcong/ignore-errcompacted
etcdserver: ignore ErrCompacted error
2020-08-11 21:34:38 -07:00
fe36be2251 Merge pull request #12195 from tangcong/optimize-healthcheck
*: check health by using v3 range request and its corresponding timeout
2020-08-11 21:32:44 -07:00
8a4c7751d8 CHANGELOG: update for 12195 2020-08-12 08:10:13 +08:00
18adf55c92 Merge pull request #12199 from ptabor/20200803-expect-replace-fix
tests/e2e: Update github.com/creack/pty v1.1.7 -> v1.1.11
2020-08-11 11:59:12 -07:00
844091dda3 Merge pull request #12206 from ptabor/20200807-setLoggingDataRace
integration: Fix flakes due to .setupLogging race.
2020-08-11 11:58:47 -07:00
afa0e8196c etcdserver: ignore mvcc.ErrCompacted error 2020-08-11 23:45:20 +08:00
cd25d6c06e Merge pull request #12130 from ptabor/master
functional/tester: Update cluster_test.go to reflect functional.yaml
2020-08-09 02:14:31 +08:00
9d182c2a70 clientv3/integration: Fix flaky TestGetTokenWithoutAuth (#12200)
The test is vary flaky on Travis.

Seems that since (https://github.com/etcd-io/etcd/issues/7724) the
client is expected to simply ignore whether server is in AuthDisabled
mode even if the user supplies credentials.

The tests used to:
  * use very large cluster (10 nodes)
  * set very low timeout (1 sec)

Such setup led to frequent deadlineExceed errors or following failures:

    === RUN   TestGetTokenWithoutAuth
    {"level":"warn","ts":"2020-08-04T16:50:48.686+0200","caller":"clientv3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"endpoint://client-35573307-1ee5-441b-acc7-d073f0bd7de5/localhost:69820396562031027440","attempt":0,"error":"rpc error: code = Unavailable desc = etcdserver: leader changed"}
        user_test.go:151: other errors:etcdserver: leader changed
    --- FAIL: TestGetTokenWithoutAuth (10.91s)
2020-08-07 13:32:32 -07:00
830618e44d ./integration: Fix flakes due to .setupLogging race.
The source of problem was the fact that multiple tests were creating
their clusters (and some of them were setting global grpclog).
If the test was running after some other test that created HttpServer
(so accessed grpclog), this was reported as race.

Tested with:
  go test ./clientv3/. -v "--run=(Example).*" --count=2
  go test ./clientv3/. -v "--run=(Test).*" --count=2
  go test ./integration/embed/. -v "--run=(Test).*" --count=2
2020-08-07 13:54:41 +02:00
f395f82a75 Merge pull request #12202 from spzala/auditchangelog
CHANGELOG: update with added audit report
2020-08-05 08:56:16 -07:00
eafd374309 CHANGELOG: update with added audit report
Update the changelog for recently added audit report. Also mention the
report in the security readme.
2020-08-04 22:44:48 -04:00
d29af0f22b Merge pull request #12201 from spzala/audit
Add audit report
2020-08-04 20:25:24 -04:00
4edd2679f0 doc: add audit report
Adding audit report.
2020-08-04 19:07:18 -04:00
00de56a4a4 etcdserver, wal: Fix tests that were performing unintended casting of int to String.
Fixes following problems during "./etcd# go test ./..."

> go.etcd.io/etcd/v3/etcdserver/api/v2store_test
etcdserver/api/v2store/store_test.go:847:24: conversion from int to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)

> go.etcd.io/etcd/v3/wal
wal/wal_test.go:242:68: conversion from int to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
2020-08-04 20:28:41 +02:00
b151a47d1b tests/e2e: Update github.com/creack/pty v1.1.7 -> v1.1.11
The fix is needed to mitigate consequences of
https://github.com/golang/go/issues/29458 "golang breaking change" that
causes following test failures on etcd end:

--- FAIL: TestCtlV2Set (0.00s)
    ctl_v2_test.go:552: could not start etcd process cluster (fork/exec ../../bin/etcd: Setctty set but Ctty not valid in child)
--- FAIL: TestCtlV2SetQuorum (0.00s)
    ctl_v2_test.go:552: could not start etcd process cluster (fork/exec ../../bin/etcd: Setctty set but Ctty not valid in child)
--- FAIL: TestCtlV2SetClientTLS (0.00s)
    ctl_v2_test.go:552: could not start etcd process cluster (fork/exec ../../bin/etcd: Setctty set but Ctty not valid in child)
2020-08-04 16:12:12 +02:00
36f07cb591 functional/tester: Update cluster_test.go to reflect current content of functional.yaml.
Before the change:

.../etcd/functional% go test -v ./...
...
?   	go.etcd.io/etcd/v3/functional/runner	[no test files]
=== RUN   Test_read
{"level":"info","ts":1596018672.2852418,"caller":"tester/cluster_read_config.go:36","msg":"opened configuration file","path":"../../functional.yaml"}
    Test_read: cluster_test.go:269: expected &{lg:<nil> agentConns:[] agentClients:[] agentStreams:[] agentRequests:[] testerHTTPServer:<nil> Members:[EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:19027" FailpointHTTPAddr:"http://127.0.0.1:7381" BaseDir:"/tmp/etcd-functional-1" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:1379" Etcd:<Name:"s1" DataDir:"/tmp/etcd-functional-1/etcd.data" WALDir:"/tmp/etcd-functional-1/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:1379" AdvertiseClientURLs:"https://127.0.0.1:1379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:1380" AdvertisePeerURLs:"https://127.0.0.1:1381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:10000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-1/etcd.log" > SnapshotPath:"/tmp/etcd-functional-1.snapshot.db"  EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:29027" FailpointHTTPAddr:"http://127.0.0.1:7382" BaseDir:"/tmp/etcd-functional-2" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:2379" Etcd:<Name:"s2" DataDir:"/tmp/etcd-functional-2/etcd.data" WALDir:"/tmp/etcd-functional-2/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:2379" AdvertiseClientURLs:"https://127.0.0.1:2379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:2380" AdvertisePeerURLs:"https://127.0.0.1:2381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:10000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-2/etcd.log" > SnapshotPath:"/tmp/etcd-functional-2.snapshot.db"  EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:39027" FailpointHTTPAddr:"http://127.0.0.1:7383" BaseDir:"/tmp/etcd-functional-3" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:3379" Etcd:<Name:"s3" DataDir:"/tmp/etcd-functional-3/etcd.data" WALDir:"/tmp/etcd-functional-3/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:3379" AdvertiseClientURLs:"https://127.0.0.1:3379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:3380" AdvertisePeerURLs:"https://127.0.0.1:3381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:10000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-3/etcd.log" > SnapshotPath:"/tmp/etcd-functional-3.snapshot.db" ] Tester:DataDir:"/tmp/etcd-tester-data" Network:"tcp" Addr:"127.0.0.1:9028" DelayLatencyMs:5000 DelayLatencyMsRv:500 UpdatedDelayLatencyMs:5000 RoundLimit:1 ExitOnCaseFail:true EnablePprof:true CaseDelayMs:7000 CaseShuffle:true Cases:"SIGTERM_ONE_FOLLOWER" Cases:"SIGTERM_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT" Cases:"SIGTERM_LEADER" Cases:"SIGTERM_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"SIGTERM_QUORUM" Cases:"SIGTERM_ALL" Cases:"SIGQUIT_AND_REMOVE_ONE_FOLLOWER" Cases:"SIGQUIT_AND_REMOVE_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT" Cases:"BLACKHOLE_PEER_PORT_TX_RX_LEADER" Cases:"BLACKHOLE_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"BLACKHOLE_PEER_PORT_TX_RX_QUORUM" Cases:"BLACKHOLE_PEER_PORT_TX_RX_ALL" Cases:"DELAY_PEER_PORT_TX_RX_LEADER" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_LEADER" Cases:"DELAY_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"DELAY_PEER_PORT_TX_RX_QUORUM" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_QUORUM" Cases:"DELAY_PEER_PORT_TX_RX_ALL" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_ALL" Cases:"NO_FAIL_WITH_STRESS" Cases:"NO_FAIL_WITH_NO_STRESS_FOR_LIVENESS" FailpointCommands:"panic(\"etcd-tester\")" RunnerExecPath:"./bin/etcd-runner" Stressers:<Type:"KV_WRITE_SMALL" Weight:0.35 > Stressers:<Type:"KV_WRITE_LARGE" Weight:0.002 > Stressers:<Type:"KV_READ_ONE_KEY" Weight:0.07 > Stressers:<Type:"KV_READ_RANGE" Weight:0.07 > Stressers:<Type:"KV_DELETE_ONE_KEY" Weight:0.07 > Stressers:<Type:"KV_DELETE_RANGE" Weight:0.07 > Stressers:<Type:"KV_TXN_WRITE_DELETE" Weight:0.35 > Stressers:<Type:"LEASE" > Checkers:"KV_HASH" Checkers:"LEASE_EXPIRE" StressKeySize:100 StressKeySizeLarge:32769 StressKeySuffixRange:250000 StressKeySuffixRangeTxn:100 StressKeyTxnOps:10 StressClients:100 StressQPS:2000  cases:[] rateLimiter:<nil> stresser:<nil> checkers:[] currentRevision:0 rd:0 cs:0}, got &{lg:<nil> agentConns:[] agentClients:[] agentStreams:[] agentRequests:[] testerHTTPServer:<nil> Members:[EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:19027" FailpointHTTPAddr:"http://127.0.0.1:7381" BaseDir:"/tmp/etcd-functional-1" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:1379" Etcd:<Name:"s1" DataDir:"/tmp/etcd-functional-1/etcd.data" WALDir:"/tmp/etcd-functional-1/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:1379" AdvertiseClientURLs:"https://127.0.0.1:1379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:1380" AdvertisePeerURLs:"https://127.0.0.1:1381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:2000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-1/etcd.log" LogLevel:"info" > SnapshotPath:"/tmp/etcd-functional-1.snapshot.db"  EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:29027" FailpointHTTPAddr:"http://127.0.0.1:7382" BaseDir:"/tmp/etcd-functional-2" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:2379" Etcd:<Name:"s2" DataDir:"/tmp/etcd-functional-2/etcd.data" WALDir:"/tmp/etcd-functional-2/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:2379" AdvertiseClientURLs:"https://127.0.0.1:2379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:2380" AdvertisePeerURLs:"https://127.0.0.1:2381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:2000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-2/etcd.log" LogLevel:"info" > SnapshotPath:"/tmp/etcd-functional-2.snapshot.db"  EtcdExec:"./bin/etcd" AgentAddr:"127.0.0.1:39027" FailpointHTTPAddr:"http://127.0.0.1:7383" BaseDir:"/tmp/etcd-functional-3" EtcdPeerProxy:true EtcdClientEndpoint:"127.0.0.1:3379" Etcd:<Name:"s3" DataDir:"/tmp/etcd-functional-3/etcd.data" WALDir:"/tmp/etcd-functional-3/etcd.data/member/wal" HeartbeatIntervalMs:100 ElectionTimeoutMs:1000 ListenClientURLs:"https://127.0.0.1:3379" AdvertiseClientURLs:"https://127.0.0.1:3379" ClientAutoTLS:true ListenPeerURLs:"https://127.0.0.1:3380" AdvertisePeerURLs:"https://127.0.0.1:3381" PeerAutoTLS:true InitialCluster:"s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381" InitialClusterState:"new" InitialClusterToken:"tkn" SnapshotCount:2000 QuotaBackendBytes:10740000000 PreVote:true InitialCorruptCheck:true Logger:"zap" LogOutputs:"/tmp/etcd-functional-3/etcd.log" LogLevel:"info" > SnapshotPath:"/tmp/etcd-functional-3.snapshot.db" ] Tester:DataDir:"/tmp/etcd-tester-data" Network:"tcp" Addr:"127.0.0.1:9028" DelayLatencyMs:5000 DelayLatencyMsRv:500 UpdatedDelayLatencyMs:5000 RoundLimit:1 ExitOnCaseFail:true EnablePprof:true CaseDelayMs:7000 CaseShuffle:true Cases:"SIGTERM_ONE_FOLLOWER" Cases:"SIGTERM_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT" Cases:"SIGTERM_LEADER" Cases:"SIGTERM_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"SIGTERM_QUORUM" Cases:"SIGTERM_ALL" Cases:"SIGQUIT_AND_REMOVE_ONE_FOLLOWER" Cases:"SIGQUIT_AND_REMOVE_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT" Cases:"BLACKHOLE_PEER_PORT_TX_RX_LEADER" Cases:"BLACKHOLE_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"BLACKHOLE_PEER_PORT_TX_RX_QUORUM" Cases:"BLACKHOLE_PEER_PORT_TX_RX_ALL" Cases:"DELAY_PEER_PORT_TX_RX_LEADER" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_LEADER" Cases:"DELAY_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT" Cases:"DELAY_PEER_PORT_TX_RX_QUORUM" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_QUORUM" Cases:"DELAY_PEER_PORT_TX_RX_ALL" Cases:"RANDOM_DELAY_PEER_PORT_TX_RX_ALL" Cases:"NO_FAIL_WITH_STRESS" Cases:"NO_FAIL_WITH_NO_STRESS_FOR_LIVENESS" FailpointCommands:"panic(\"etcd-tester\")" RunnerExecPath:"./bin/etcd-runner" Stressers:<Type:"KV_WRITE_SMALL" Weight:0.35 > Stressers:<Type:"KV_WRITE_LARGE" Weight:0.002 > Stressers:<Type:"KV_READ_ONE_KEY" Weight:0.07 > Stressers:<Type:"KV_READ_RANGE" Weight:0.07 > Stressers:<Type:"KV_DELETE_ONE_KEY" Weight:0.07 > Stressers:<Type:"KV_DELETE_RANGE" Weight:0.07 > Stressers:<Type:"KV_TXN_WRITE_DELETE" Weight:0.35 > Stressers:<Type:"LEASE" > Checkers:"KV_HASH" Checkers:"LEASE_EXPIRE" StressKeySize:100 StressKeySizeLarge:32769 StressKeySuffixRange:250000 StressKeySuffixRangeTxn:100 StressKeyTxnOps:10 StressClients:100 StressQPS:2000  cases:[] rateLimiter:<nil> stresser:<nil> checkers:[] currentRevision:0 rd:0 cs:0}
--- FAIL: Test_read (0.00s)
FAIL
FAIL	go.etcd.io/etcd/v3/functional/tester	0.050s
...

After:
go test -v ../...

...
=== RUN   Test_read
{"level":"info","ts":1596018561.408186,"caller":"tester/cluster_read_config.go:36","msg":"opened configuration file","path":"../../functional.yaml"}
{"level":"info","ts":1596018561.408739,"caller":"tester/cluster_shuffle.go:35","msg":"shuffled test failure cases","total":22}
{"level":"info","ts":1596018561.4087567,"caller":"tester/cluster_shuffle.go:35","msg":"shuffled test failure cases","total":22}
--- PASS: Test_read (0.00s)
PASS

...
2020-08-04 15:47:41 +02:00
25220a0287 *: check health by using v3 range request and its corresponding timeout 2020-08-04 00:34:18 +08:00
76539cee57 test: avoid non existing package for integration test 2020-08-03 00:13:47 +09:00
1af6d61a1c Merge pull request #12177 from ironcladlou/etcdmembersdown-tweak
Documentation: Further improve etcdMembersDown alert
2020-07-31 15:06:52 -04:00
cd3df73944 Documentation: Further improve etcdMembersDown alert
Before this change, the default window for the etcdMembersDown network failure
rate function was recently changed to 1 minute. While this helps detect a etcd
recovery more quickly, it depends on scrape intervals of <= 15s to collect
sufficient data points for the rate function. In practice, an interval of >= 30s
is more typical, which causes the rate function to be less accurate.

This patch increases the window to 2m, which is a compromise between the
original value of 3m and the 1m change introuced with 2aa5684, and should
accomodate more typical scrape intervals.

To offset the window change and to further improve the chance that the alert
will only fire when etcd is truly dead, this patch changes the `for` clause from
3m to 10m. The rationale is as follows:

1. There can be significant variance in durations following a reboot before etcd
is scraped and detected as available.

2. A conservative trigger like 10m seems less likely to produce a false alarm in
the face of such variance.

3. In this alerting situation, if the outage is real, it seems unlikely that an
additional 7 minutes of delay before (for example) paging somebody will make a
significant impact on the overall response.
2020-07-31 09:26:46 -04:00
cc564110bd clientv3: remove excessive watch cancel logging (#12187) 2020-07-29 14:58:53 -07:00
6c81b20ec8 rafthttp: fix streamHandle outgoingConn peerID (#12179) 2020-07-28 14:41:10 -07:00
bc67babee8 package adt: rename the filename to be consistent with the package name (#12170) 2020-07-28 14:40:34 -07:00
9006d8d4f9 Documentation/learning/lock/client: Add defer Unlock (#11802) 2020-07-26 11:22:19 -07:00
51de68ddac 12126: snapshot: corrupted in Embedded server (#12129) 2020-07-26 11:14:46 -07:00
Jay
26b89fd418 raft: don't campaign with pending snapshot (#12163)
Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2020-07-26 00:04:46 -07:00
c9a5889915 Documentation/etcd-mixin: Reformulate alerting rules to use without rather than by (#12122)
* etcd-mixin: Reformulate alerting rules to use `without` rather than `by`

With aggregations using `by`, all additional target labels that a user
might have configured, are aggregated away. However, those target
labels are useful for e.g. alert routing. With this commit, nothing
should change for vanilla job/instance target labels, but whoever has
more target labels can now still make use of them.

Signed-off-by: beorn7 <beorn@grafana.com>

* etcd-mixin: Parametrize instance labels to aggregate away

Signed-off-by: beorn7 <beorn@grafana.com>
2020-07-23 16:02:26 -07:00
Jay
d0e4fe56a5 raft: check pending conf change before campaign (#12134)
* raft: check conf change before campaign

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* raft: extract hup function

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>

* raft: check pending conf change for transferleader

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2020-07-22 17:04:48 -07:00
772dfbfe35 wal: Fix format error avoid using reflect.DeepEqual with errors (#12118) 2020-07-20 19:49:57 -07:00
e9d16c2b62 etcdserverpb: add TestInvalidGoTypeIntPanic test (#12116)
Tested conditions that cause
panic: invalid Go type int for field k8s_io.kubernetes.vendor.go_etcd_io.etcd.etcdserver.etcdserverpb.loggablePutRequest.value_size

Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
2020-07-20 19:18:52 -07:00
7f27697df9 v3client: implement clientv3.Auth interface (#12140) 2020-07-20 16:53:25 -07:00
Jay
cc656718fa raft: correct pendingConfIndex check for AutoLeave (#12137)
Close #12136

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
2020-07-20 16:49:22 -07:00
93637b1779 raft: bug fix (#12123)
we need to test the case when configuration set is changed, but there is typo.

None

Signed-off-by: accelsao <bebe8277@gmail.com>
2020-07-20 16:30:17 -07:00
ef866a6d8b Merge pull request #11943 from mitake/bcrypt-in-api
auth, etcdserver: hash password in the API layer
2020-07-20 10:52:24 -04:00
89da79188f raft: simplify the code in progress.go (#12119) 2020-07-20 00:13:05 -07:00
da78fb5544 Merge pull request #12145 from tangcong/add_breaking_changes
CHANGELOG: add breaking changes doc for v3.4.10/v3.3.23
2020-07-17 10:00:09 -04:00
1a548c355a CHANGELOG: add breaking changes doc 2020-07-17 16:04:20 +08:00
26db9e1498 CHANGELOG: add release dates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-07-16 15:14:39 -07:00
69f7e163c6 Merge pull request #12141 from jingyih/update_changelog_12000
CHANGELOG: update from 12000
2020-07-15 13:06:07 -04:00
08c0a9d4b3 CHANGELOG: update from 12000 2020-07-15 09:59:59 -07:00
92774e83c7 Merge pull request #12121 from ironcladlou/improve-etcdmembersdown-accuracy
Documentation: Tweak etcdMembersDown to reduce false negatives
2020-07-15 08:14:59 -04:00
4c6881ffe4 Merge pull request #12131 from dissoupov/client
12125: panic: zap.Logger is nil in Embed client
2020-07-13 15:08:19 -07:00
701f02f90a 12125: panic: zap.Logger is nil in Embed client 2020-07-13 09:08:20 -07:00
2c41d9960b Documentation: describe the change of WAL entries related to auth 2020-07-14 00:15:19 +09:00
5a3da48cdf auth, etcdserver: hash password in the API layer 2020-07-14 00:15:19 +09:00
2aa5684ada Documentation: Tweak etcdMembersDown to reduce false negatives
Before this change, during a reboot in which etcd recovers quickly (e.g. 1 min),
the etcdMembersDown alert tends to fire even when etcd is fully healthy because
the averaging function can take more than 3 minutes to average back down below
the 0.01 threshold.

This change tries to reduce the possibility of a false negative by considering a
shorter (1 min) failure rate window which tends to average down below the
threshold far more quickly (within 1 min). The `for` clause of the alert should
ensure that the alert still fires if the poor conditions are sustained for an
unreasonable overall time (3 min).
2020-07-13 08:58:21 -04:00
07461ecc8c Merge pull request #12114 from tangcong/grpcproxy-healthcheck
*: add health handler for grpcproxy self
2020-07-10 10:44:59 -07:00
eb4f8dcb56 CHANGELOG: update for 12107,12114 2020-07-10 20:17:49 +08:00
0898c5b978 *: add health check for grpcproxy self 2020-07-10 20:17:44 +08:00
2f25f536f7 Merge pull request #12117 from wking/drop-alerting-syntax-rules
Documentation/op-guide: Drop old alert_rules
2020-07-08 16:56:03 -04:00
4160b8396d Documentation/op-guide: Drop old alert_rules
Frederic says [1]:

> Side note, we can probably remove the old alerting syntax rules,
> Prometheus has removed this syntax >2.5 years ago.

[1]: https://github.com/etcd-io/etcd/pull/12080#issuecomment-649982787
2020-07-08 09:37:34 -07:00
429826b467 Merge pull request #12080 from wking/raise-etcd-leader-changes-to-four
Documentation/etcd-mixin: Raise etcdHighNumberOfLeaderChanges threshold to 4
2020-07-08 08:37:50 -04:00
58bb8ae09f Merge pull request #12018 from spzala/umaskconsideration
pkg: consider umask when use MkdirAll
2020-07-06 17:33:33 -07:00
27ddb84268 Merge pull request #12107 from tangcong/register-grpcproxy-metrics
*: register metrics handler for grpcproxy self
2020-07-06 13:00:35 -07:00
fff5d3cc03 *: register metrics handler for grpcproxy self 2020-07-02 22:55:18 +08:00
15884e9085 Merge pull request #12086 from tangcong/print-expensive-request-v3rpc
v3rpc: Log expensive request in UnaryInterceptor
2020-07-01 23:53:50 +08:00
5c6d7b4d2c CHANGELOG: update for 12086 2020-07-01 00:54:31 +08:00
644d09edb5 v3rpc: print expensive request info in UnaryInterceptor 2020-07-01 00:54:23 +08:00
7f726db202 Merge pull request #12092 from mitake/warn
etcdmain: let grpc proxy warn about insecure-skip-tls-verify
2020-06-29 11:59:53 -04:00
01f1109f6d Merge pull request #12091 from mitake/password-strength-2
Documentation: refine the description about password strength
2020-06-29 11:53:15 -04:00
e582d7dc80 Documentation: refine the description about password strength 2020-06-29 23:40:44 +09:00
db2165dfb1 etcdmain: let grpc proxy warn about insecure-skip-tls-verify 2020-06-29 23:37:04 +09:00
e94dc39edc Merge pull request #12079 from cfc4n/changelog_3.3-3.4
CHANGELOG-3.3-3.4: update for #12069
2020-06-26 11:28:40 -07:00
0c5cffc60b Documentation/etcd-mixin: Raise etcdHighNumberOfLeaderChanges threshold to 4
A cluster with three members could see three leader changes during a
healthy rolling reboot, and we don't want to alert on that.  Growing
to 4 reduces false-alarms for clusters with three or fewer members,
and that's probably most clusters.  It will also slightly increase the
risk of false-negatives, but if the cluster is struggling with high
latency, it seems likely that it would quickly pass the new threshold
too.

The hard-coded threshold means that we are still likely to get
false-positives during rolling reboots of clusters with four or more
members.  Ideally we'd scale this with the cluster size, or something,
but I'm not sure how to do that.  Three members is the minimum size
for high availability, so reducing false positives for that case seems
worth addressing even if we leave larger clusters largely unchanges.

Also manually catch etcd3_alert.rules up to speed, since it seems to
have been passed over by 16fc8a2b4b (Documentation/op-guide:
Re-generate alert rules and dashboard from mixin, 2020-04-07, #11768).
2020-06-25 15:38:15 -07:00
2b79442d8e backend: Create base type for readTx and concurrentReadTx (#11976)
backend: Create base type for readTx and concurrentReadTx

backend: Implemented review comments to rename rTx to baseReadTx and remove TODO

backend: Resolved comments around baseReadTx

backend: Implemented a review comment
2020-06-25 10:26:17 -07:00
3ced62a6c7 CHANGELOG-3.3-3.4: update for https://github.com/etcd-io/etcd/pull/12069 ,and other cherry pick PRs. 2020-06-25 20:54:46 +08:00
94bd71adac Merge pull request #12066 from tangcong/fix-fmt-shadow-err
etcdmain: fix shadow error
2020-06-24 20:40:24 -07:00
ae59425235 Merge pull request #11919 from YoyinZyc/downgrade-policy
[Etcd downgrade] Apply downgrade policy to server
2020-06-24 11:33:32 -07:00
2541b0bba9 etcdserver: apply downgrade policy to server. 2020-06-24 11:06:29 -07:00
a08c52945f etcdmain: fix shadow error 2020-06-24 17:47:18 +08:00
beb5614aad doc: add TLS related warnings (#12060) 2020-06-23 21:07:36 -07:00
8f19fecb82 Merge pull request #12030 from tangcong/fix-grpc-proxy-hang
proxy/grpcproxy: fix grpc proxy hang when broadcast failed to cancel a watcher
2020-06-22 10:53:13 -07:00
9380327e70 Merge pull request #12047 from jingyih/update_bbolt_1p3p5
vendor: update bbolt to v1.3.5
2020-06-22 10:52:43 -07:00
238ea9bf93 vendor: update bbolt to v1.3.5 2020-06-22 04:29:06 -07:00
854c327179 CHANGELOG: update for 12030 2020-06-22 14:57:02 +08:00
34e3dbe3d4 proxy/grpcproxy: add zap logger 2020-06-22 14:56:58 +08:00
ebdccedbde Merge pull request #11980 from cfc4n/simple-token-ttl
auth: Customize the settings of simpleTokenTTL.
2020-06-21 23:23:45 -07:00
6ec5d5f60f Merge pull request #12001 from cfc4n/master
CHANGELOG: update for PR #11980 ,  #11986 ,  # 11987 ,
2020-06-21 23:20:50 -07:00
a71fa242d8 Merge branch 'master' into master 2020-06-21 23:20:34 -07:00
f1057001c3 Merge pull request #12033 from jingyih/update_changelog_PR11946
CHANGELOG: update for #11946
2020-06-21 19:18:11 -07:00
fac83fbecd CHANGELOG: update for PR11946
Also update helper message to include the new flag.
2020-06-20 12:26:36 -07:00
f5a85e9987 pkg: consider umask when use MkdirAll
os.MkdirAll creates directory before umask so make sure that a desired
permission is set after creating a directory with MkdirAll. Use the
existing TouchDirAll function which checks for permission if dir is already
exist and when create a new dir.
2020-06-20 13:10:11 -04:00
10cdabe721 CHANGELOG: update for https://github.com/etcd-io/etcd/pull/11980 , https://github.com/etcd-io/etcd/pull/11986 , https://github.com/etcd-io/etcd/pull/11987 . 2020-06-21 00:00:41 +08:00
b5a07728d0 proxy/grpcproxy: make sure watchproxy do not block forever when failed to cancel a watcher 2020-06-20 11:20:31 +08:00
d8c8f903ee Merge pull request #11983 from tangcong/add_reason_field_for_health
etcdserver/api/etcdhttp: add reason field for /health response
2020-06-15 23:17:00 -07:00
8720f980ac Merge pull request #11986 from cfc4n/fdusage-limit
etcdserver:set FDUsage ticker to 10 minute from 5 seconds.
2020-06-15 22:12:58 -07:00
144beba3f8 etcdserver:FDUsage set ticker to 10 minute from 5 seconds.
This ticker will check File Descriptor Requirements ,and count all fds in used.
And recorded some logs when in used >= limit/5*4. Just recorded message.
If fds was more than 10K,It's low performance due to FDUsage() works.
So need to increase it.

see https://github.com/etcd-io/etcd/issues/11969 for more detail.
2020-06-16 12:44:33 +08:00
af7128ccd0 Merge pull request #11997 from tangcong/optimize-file-stats-error
pkg/fileutil: optimize file stats error
2020-06-15 11:19:58 -04:00
2f6ceb7aca Merge pull request #12012 from mitake/doc-disk-data
Documentation: note on data encryption
2020-06-15 23:25:23 +09:00
4f16d0698f Merge pull request #12010 from WuShaoQiang/master
raft: fix typo
2020-06-14 16:35:58 -07:00
c13415c581 Documentation: note on data encryption 2020-06-15 00:29:32 +09:00
8f72368070 raft: fix typo 2020-06-14 13:26:37 +08:00
c9dd080afb Merge pull request #11998 from hwdef/master
Documentation: fix broken links
2020-06-13 00:30:55 -04:00
ed04d0d98a etcdserver: change protobuf field type from int to int64 (#12000) 2020-06-12 16:10:56 -07:00
b6d1987ccd auth: return incorrect result 'ErrUserNotFound' when client request without username or username was empty. (#12005)
Fiexs https://github.com/etcd-io/etcd/issues/12004 .
2020-06-12 16:10:19 -07:00
e3014072ba Documentation: fix broken links 2020-06-12 09:51:33 +08:00
d507ab4aad auth: Customize simpleTokenTTL settings.
see https://github.com/etcd-io/etcd/issues/11978 for more detail.
2020-06-11 17:15:42 +08:00
9b444c6355 CHANGELOG: add data dir permission breaking changes doc 2020-06-11 10:10:32 +08:00
b0d2edfc68 Documentation: Added Recover cluster from minority failure (#11988) 2020-06-10 14:36:44 -07:00
26f6aee395 mvcc: chanLen 1024 is to biger,and it used more memory. (#11987)
128 seems to be enough.
Sometimes the consumption speed is more than the production speed.

See https://github.com/etcd-io/etcd/issues/11906 for more detail.
2020-06-10 14:35:48 -07:00
50d7a10a3f pkg/fileutil: print desired file permission in error log 2020-06-10 22:45:36 +08:00
330424142c tests/e2e: fix failed /health test case 2020-06-10 11:16:25 +08:00
f77b21ce05 CHANGELOG: update for 11983 2020-06-10 11:15:00 +08:00
4acaa5a2a0 etcdserver/api/etcdhttp: add reason field for /health response 2020-06-10 11:14:56 +08:00
3082a7d521 etcdserver/api/v2store: fix race in TestIssue2746WithThree (#11982) 2020-06-09 19:11:08 -07:00
133b0c2b04 Use http constant instead of a number (#11975)
* embed: Use http constant instead of a number

* embed: Use http constant instead of a number

* embed: Replaced a number with Go http constant
2020-06-09 19:08:38 -07:00
84a5ff2d97 Merge pull request #11962 from spzala/incorrectflag
logging: create logger to prevent nil pointer
2020-06-09 14:34:22 -04:00
1647889253 Merge pull request #11936 from wswcfan/update-3.2-changelog-for-11691
CHANGELOG: update 3.2 changelog and 3.3 upgrade document for #11691
2020-06-09 11:38:13 -04:00
1c1029ecba Merge pull request #11990 from tangcong/add-limit-for-revisions
mvcc: push down RangeOptions.limit argv into index tree to reduce memory overhead
2020-06-08 21:31:51 -07:00
f3deba09b4 CHANGELOG: update 3.2 changelog and 3.3 upgrade document for #11691 2020-06-09 11:39:46 +08:00
1677fd80a5 CHANGELOG: update for 11990 2020-06-09 01:11:11 +08:00
26c930f27d mvcc: push down RangeOptions.limit argv into index tree 2020-06-09 01:11:06 +08:00
6e6f22db43 vendor: add missing failpoint
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-06-07 22:25:55 -07:00
49f91d629a Merge pull request #11945 from polyrabbit/fix-permission-on-Windows
pkg: Fix dir permission check on Windows
2020-06-04 18:16:48 -04:00
570e55db65 functional/agent: fmt
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-06-03 17:41:34 -07:00
237c8b528b Merge pull request #11913 from jpbetz/gofail-startup
Persist failpoints across member restart
2020-06-03 17:41:17 -07:00
62f1fefe4a Merge pull request #11959 from tedyu/store-clone-rd-lock
etcdserver: take read lock when cloning store
2020-06-03 14:13:53 -07:00
a09533d63d Merge pull request #11960 from tedyu/internal-create-err-chk
etcdserver: check error return for store#internalCreate
2020-06-03 14:13:19 -07:00
be0639090d Merge pull request #11939 from tedyu/chk-ret-from-write
etcdserver: check the error return from Write()
2020-06-03 14:12:50 -07:00
659fb01a6c logging: create logger to prevent nil pointer
Create a new logger if one not found.
2020-06-02 09:53:35 -04:00
5c95114190 etcdserver: check the error return from Write()
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-31 09:27:01 -07:00
17483944fc etcdserver: check error return for store#internalCreate
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-31 09:20:05 -07:00
59ec251812 etcdserver: take read lock when cloning store
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-31 09:08:26 -07:00
09fcf55ca4 Merge pull request #11845 from xiang90/tcp_proxy
etcdmain: best effort detection of self pointing in tcp proxy
2020-05-31 00:57:49 -07:00
b2d228641a Merge pull request #11905 from tedyu/node-tick-read
raft: avoid data race by not reading raft.lead
2020-05-31 00:57:01 -07:00
8af52dc0e7 Merge pull request #11935 from tangcong/save-cindex-for-lease
lease: ensure grant/revoke won't be applied repeatedly after restarting etcd
2020-05-31 00:56:12 -07:00
29a0c5f221 Merge pull request #11957 from gyuho/build-scripts
*: remove old gitignore and build functions
2020-05-31 00:53:40 -07:00
ab65285418 vendor: regenerate
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-30 14:36:18 -07:00
72a26d0797 vendor/modules.txt: regenerate
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-30 14:36:09 -07:00
a48706ecfb *: remove old gitignore and build functions
Not needed since we are now using go module.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-30 14:35:34 -07:00
9b6c3e3378 Merge pull request #11946 from crawshaw/crawshaw/nofsync
etcdserver, et al: add --unsafe-no-fsync flag
2020-05-27 09:00:31 -07:00
e06006ea28 pkg: Fix dir permission check on Windows 2020-05-26 11:15:25 +08:00
66cb045d47 etcdserver, et al: add --unsafe-no-fsync flag
This makes it possible to run an etcd node for testing and development
without placing lots of load on the file system.

Fixes #11930.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
2020-05-26 09:21:49 +10:00
747ff75c96 Merge pull request #11942 from tedyu/client-3-cancel
clientv3: cancel the client if there is no endpoint
2020-05-24 15:55:53 -04:00
dafd474677 clientv3: cancel the client if there is no endpoint
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-23 13:35:17 -07:00
e007d4f58d Merge pull request #11938 from tedyu/mk-v2-evt-nil
Check events against nil
2020-05-22 13:38:53 -07:00
df57a68b47 Check events against nil
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-22 13:35:25 -07:00
4c9571d909 Merge pull request #11933 from needkane/PR
etcdserver: Missing the cfg.Logger causes panic
2020-05-22 09:23:27 -07:00
426dba8628 CHANGELOG: update for 11935 2020-05-22 21:47:37 +08:00
e9ae8eb5a1 *: ensure grant/revoke won't be applied repeatedly after restarting etcd 2020-05-22 21:47:24 +08:00
c8ae694dea etcdserver: Missing the cfg.Logger causes panic 2020-05-22 03:42:44 -04:00
a4ada8cb1f Merge pull request #11922 from tedyu/non-recursive-watch
clientv3: non-recursive Watch()
2020-05-20 20:36:09 -07:00
f976138186 clientv3: non-recursive watch
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-20 15:44:41 -07:00
f8ee908330 Merge pull request #11921 from tedyu/client-watch-lock
clientv3: use Mutex for watcher lock
2020-05-20 14:56:23 -07:00
5f9bb45e39 CHANGELOG: highlight WAL fix with github issue, add 3.3/3.4 patch release
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-20 11:49:14 -07:00
cac53867ac rafthttp: log snapshot download
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-20 10:55:43 -07:00
188c66ada8 Merge pull request #11924 from tangcong/fix-crc-mismatch
wal: fix crc mismatch crash bug
2020-05-20 10:30:40 -07:00
02b9ad3939 wal: add TestValidSnapshotEntriesAfterPurgeWal testcase 2020-05-21 00:51:53 +08:00
8450b16e1c CHANGELOG: update for 11924 2020-05-21 00:51:49 +08:00
1836cbb15e wal: fix crc mismatch crash bug 2020-05-20 13:21:07 +08:00
35d026246c clientv3: use Mutex for watcher lock
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-19 19:20:44 -07:00
4869b6c9aa Persist failpoints across member restart 2020-05-18 14:14:02 -07:00
732df43cf8 CHANGELOG: update dates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-18 10:57:53 -07:00
1fd1318409 etcdserver/api/rafthttp: improve snapshot send logging
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-18 02:29:20 -07:00
5e2815e9cd Merge pull request #11801 from YoyinZyc/downgrade-server
[Etcd downgrade] Implement downgrade validate, enable and cancel
2020-05-17 22:47:16 -07:00
52edb7d38c Merge pull request #11900 from tedyu/omit-orphaned-files
etcdserver: snapNames() should exclude orphaned defragmentation file
2020-05-17 15:37:10 -07:00
f35015abc5 etcdserver: snapNames() should exclude orphaned defragmentation file
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-17 14:19:33 -07:00
8b50f4e11d Merge pull request #11899 from tedyu/rel-snap-db
etcdserver: continue releasing snap db in case of error
2020-05-17 14:01:33 -07:00
9d20ba4822 raft: avoid data race by not reading raft.lead
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-16 20:40:25 -07:00
b84d257b92 etcdserver: continue releasing snap db in case of error
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-16 04:43:55 -07:00
52456659bd Merge pull request #11903 from jchauncey/fix-dockerfile-release
fix(Dockerfile-release): from image was missing image name
2020-05-15 20:19:05 -07:00
71c062e3d8 fix(Dockerfile-release): from image was missing image name 2020-05-15 22:48:11 -04:00
31b6e5a087 Merge pull request #11896 from gyuho/snapshot
*: make sure snapshot save downloads SHA256 checksum
2020-05-15 19:41:50 -07:00
d230e6ba8c etcdserver: handle and apply downgrade validate, enable and cancel requests 2020-05-15 14:45:00 -07:00
37e598a20b etcdserver: define error string and link with grpc error code. 2020-05-15 11:51:39 -07:00
d4ae7baea1 words: add SHA
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 22:57:52 -07:00
39c43cfb3c clientv3: make sure snapshot has integrity check hash
I've seen some cases SHA blobs are missing (still investigating).
Adding a check to make sure snapshot save always downloads
hash digests for integrity checks.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 22:57:52 -07:00
4ddcc36057 etcdserver/api/v3rpc: document, clean up snapshot sends
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 22:57:52 -07:00
f1179fd70d etcdserver/api/snap: update format string in "ReleaseSnapDBs"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 22:57:20 -07:00
3d3ed29d64 CHANGELOG: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 20:12:44 -07:00
e821ac8b35 functional: bump up Go versions to 1.14.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 20:01:27 -07:00
93de72537e Makefile: bump up default Go version to 1.14.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 20:01:12 -07:00
2c7670f9c6 travis: bump up to Go 1.14.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-14 20:00:57 -07:00
3488e25a75 Merge pull request #11888 from jpbetz/out-of-range-fix
Fix state.commit is out of range on restart
2020-05-14 17:55:20 -07:00
743e6e92cb etcdserver/*, wal/*: Remove orphaned .snap.db files during Release 2020-05-14 15:16:51 -07:00
dd7055be51 Merge pull request #11874 from jackkleeman/patch-1
Update CHANGELOG for 11850
2020-05-14 17:38:54 -04:00
b68eea236e etcdserver/*, wal/*:Add comments, clean up error messages and tests 2020-05-14 13:34:55 -07:00
3b6d222627 CHANGELOG: update for 11850
https://github.com/etcd-io/etcd/pull/11850
2020-05-14 19:30:13 +01:00
bd16071846 etcdserver/*, wal/*: find valid snapshots by cross checking snap files and wal snap entries 2020-05-14 09:22:45 -07:00
50517039ae etcdserver/*, wal/*: add Sync method 2020-05-13 08:47:47 -07:00
91efa67cb1 etcdserver/*: rollback default settings 2020-05-13 08:47:47 -07:00
5435e7686a etcdserver/*: fix tests 2020-05-13 08:47:41 -07:00
3d2b565f98 etcdserver/*: changes to snapshots and wal logic to fix #10219 2020-05-13 08:46:15 -07:00
9162cd613d etcdserver/*, wal/*: changes to snapshots and wal logic to fix #10219 2020-05-13 08:42:03 -07:00
7cc2f8a411 raft: break out of nested loop when id is found (#11870)
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-12 16:59:22 -07:00
6fb3b9e591 chore(Dockerfile-release): Update base image to use new debian-base:v2.1.0 (#11885)
Updates the base image for all arch releases to the new location (us.gcr.io/k8s-artifacts-prod/build-image) as well as to the latest version (v2.1.0). This should fix a number of vulnerabilities that are in the 1.0.0 base image release.
2020-05-12 16:58:38 -07:00
ec13797407 mvcc: avoid negative watcher count metrics (#11882)
The watch count metrics are not robust to duplicate cancellations. These
cause the count to be decremented twice, leading eventually to negative
counts. We are seeing this in production. The duplicate cancellations
themselves are not themselves a big problem (except performance), but
they are caused by the new proactive cancellation logic (#11850) which
cancels proactively even immediately before initiating a Close, thus
nearly guaranteeing a Close-cancel race, as discussed in
watchable_store.go. We can avoid this in most cases by not sending a
cancellation when we are going to Close.
2020-05-12 16:50:53 -07:00
ab494956bf Merge pull request #11862 from gyuho/const
etcdserver: compute max request bytes humanized string once
2020-05-11 19:36:37 -07:00
5f07d46f70 Merge pull request #11863 from gyuho/gofmt
*: run "gofmt -s" on genproto complete
2020-05-11 18:08:22 -07:00
2ecf268367 Merge pull request #11864 from gyuho/go1.14
*: use Go 1.14 for CI
2020-05-11 18:08:07 -07:00
11569821db CHANGELOG-3.5: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-11 18:07:39 -07:00
60ce7bc2fa functional/scripts: update default Go version
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-11 18:05:03 -07:00
c667c14d8d fix memberList inconsistent output (#11812)
* etcdctl/ctlv3: support to print memberlist in hex format json

* e2e: add memberListWithHexTest

* CHANGELOG: update for #11812
2020-05-11 00:11:40 -07:00
0a0c27a4df functional: add "-mod=mod" flags
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 18:36:38 -07:00
b79abb6f76 travis: set "-mod=mod"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 18:36:38 -07:00
6874b9a76b test: add "mod" to build tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 18:36:38 -07:00
b34c0c55a5 pkg/tlsutil: add "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256" to support cipher suite
pkg/tlsutil: support TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 18:36:28 -07:00
e94201c9fd test: set mod flags for bom test
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 13:37:18 -07:00
027c26a0fd test: test '-mod=mod'
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 12:39:11 -07:00
b6ea32ee0f test: ignore vendor directory in govet test
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-10 12:35:15 -07:00
628bd294e1 *: use Go 1.14 for CI
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-09 23:34:20 -07:00
d93c36bbb9 *: run "gofmt -s" on genproto complete
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-09 23:14:32 -07:00
89a3a24d5a etcdserver: compute max request bytes humanized string once
No need to compute the request size limit everytime in runtime

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-05-09 23:11:45 -07:00
b95f135e10 Merge pull request #11861 from tedyu/node-run-found
raft: break out of nested loop when raft id is found
2020-05-09 23:05:26 -07:00
485e1409ac Merge pull request #11860 from tedyu/range-tx-lock
mvcc: Obtain tx Lock once in readTx#UnsafeRange
2020-05-09 22:23:02 -07:00
43794efb1f raft: break out of nested loop when raft id is found
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-09 17:56:44 -07:00
2dbbe6ce85 mvcc: Obtain txLock once in readTx#UnsafeRange
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-09 16:28:47 -07:00
c9377195d5 Merge pull request #11857 from tangcong/update-3.4-3.3-changelog-for-11817
CHANGELOG: update 3.4/3.3 changelog for 11817
2020-05-07 19:59:59 -07:00
e495c9220d CHANGELOG: update 3.4/3.3 changelog for 11817 2020-05-08 10:18:46 +08:00
d472f67c77 Merge pull request #11817 from tangcong/fix-mvcc-deadlock-bug
mvcc: fix a deadlock bug in mvcc
2020-05-07 16:03:45 -07:00
6d067997e9 Merge pull request #11850 from jackkleeman/cancel-watch
clientv3: cancel watches proactively on client context cancellation
2020-05-07 11:06:49 -07:00
87aa5a9e60 clientv3: cancel watches proactively on client context cancellation
Currently, watch cancel requests are only sent to the server after a
message comes through on a watch where the client has cancelled. This
means that cancelled watches that don't receive any new messages are
never cancelled; they persist for the lifetime of the client stream.
This has negative connotations for locking applications where a watch
may observe a key which might never change again after cancellation,
leading to many accumulating watches on the server.

By cancelling proactively, in most cases we simply move the cancel
request to happen earlier, and additionally we solve the case where the
cancel request would never be sent.

Fixes #9416
Heavy inspiration drawn from the solutions proposed there.
2020-05-07 16:50:26 +01:00
feb56298dd etcdserver: don't let InternalAuthenticateRequest have password (#11818) 2020-05-06 16:46:41 -07:00
035e1db0a2 Merge pull request #11830 from mitake/insecure-default
Documentation: note on the policy of insecure by default
2020-05-06 23:54:22 +09:00
356f647866 Documentation: note on the policy of insecure by default 2020-05-05 22:44:24 +09:00
d9d850c885 rafthttp: add snapshot limit const (#11846) 2020-05-04 23:26:22 -07:00
17ed0c6638 clientv3: Check the error return from Write in v3Manager#Status (#11837)
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-04 17:43:08 -07:00
df1d2ca78e clientv3: v3Manager#saveDB should always close db file (#11838)
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-05-04 15:46:36 -07:00
16810cd211 etcdmain: best effort detection of self pointing in tcp proxy 2020-05-04 15:42:01 -07:00
4a64198a9f Merge pull request #11844 from spzala/securityrelprocess
doc: remove Product Security Committee individual emails
2020-05-04 13:37:55 -04:00
658fd15b61 doc: remove Product Security Committee individual emails
emails here can be an unnecessary maintainance. The provided
GitHub IDs have the latest emails and also the process has
security@etcd.io mentioned in this doc as a contact email.
2020-05-04 12:40:34 -04:00
71170dd1d8 Merge pull request #11841 from mitake/warn-insecure-skip-tls-verify
etcdctl, etcdmain: warn about --insecure-skip-tls-verify options
2020-05-03 23:32:25 +09:00
c92691dc47 etcdctl, etcdmain: warn about --insecure-skip-tls-verify options 2020-05-03 17:17:30 +09:00
dfb0a40509 Merge pull request #11832 from krukkrz/issue_template_broken_link
.github/ISSUE_TEMPLATE: fix of a broken link
2020-04-30 11:08:33 -04:00
fc64b56058 .github/ISSUE_TEMPLATE: fix of a broken link
to make redirection possible. Changed undercore to dash.
2020-04-30 13:59:17 +02:00
1044a8b07c Merge pull request #11768 from brancz/uid
Use UID instead of ID in Grafana dashboard
2020-04-29 05:35:06 -07:00
fb77f9b1d5 Merge pull request #11814 from tedyu/dir-closer
wal: close the directory when Fsync() encounters error
2020-04-28 23:27:08 -07:00
3b79ed8506 Merge pull request #11821 from tedyu/wal-close-all
closeAll should attempt to close all ReadCloser's
2020-04-28 20:54:13 -07:00
62755f484e wal: closeAll should attempt to close all ReadCloser's
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-04-28 20:33:40 -07:00
d8bbf7f58d Merge pull request #11816 from gaurav1086/fix_race_condition_kv_test
mvcc: fix race in TestKVTxnBlockWriteOperations
2020-04-28 16:11:26 -07:00
b28a6272b7 Merge pull request #11823 from philips/v3-mod-attempt2
go.mod: name packages with go.etcd.io/etcd/v3
2020-04-28 15:24:29 -07:00
eddca8dce2 Merge pull request #11819 from gaurav1086/fix_race_cond_client_test
clientv3: fix DialTimeout race condition
2020-04-28 15:12:50 -07:00
d88d765ba4 Documentation, CHANGELOG: use new go.etcd.io/etcd/v3 pkg
Use the new package path in the docs and announce it in the CHANGELOG
2020-04-28 22:02:19 +00:00
86a97533bb clientv3: fix DialTimeout race condition 2020-04-28 10:56:28 -04:00
e5ddc91228 mvcc: fix race in TestKVTxnBlockWriteOperations 2020-04-28 10:53:25 -04:00
c9cdefaeac *: run gofmt and make fmt make test pass
gofmt tests were failing, fix.
2020-04-28 00:57:52 +00:00
03248a9045 bill-of-materials: updates for /etcd/v3 packaging
bill-of-materials:
- update to new package names

test/updatebom.sh scripts:
- Update to the right package names
- Don't add bom tool to go.mod
2020-04-28 00:57:52 +00:00
dd21e710e1 testutil: leak: fix path for go routine filter
add go.etcd.io/etcd/v3 to existing filters
2020-04-28 00:57:52 +00:00
c066230bad go.sum: cleanup after all the changes 2020-04-28 00:57:52 +00:00
2ea4d88299 *: use go.etcd.io/etcd/v3 where needed
Found potential locations via

```
git grep 'go.etcd.io/etcd$'
```
2020-04-28 00:57:52 +00:00
7a5fefedcd *: fix imports missed by gomove 2020-04-28 00:57:52 +00:00
96cce208c2 go.mod: use go.etcd.io/etcd/v3 versioning
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.

Used this tool to update package imports:
  https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
a7423f2b7b CHANGELOG: update for 11817 2020-04-27 06:42:10 +08:00
8fa7783875 mvcc: fix deadlock bug 2020-04-27 06:42:06 +08:00
99eb5cdfdb wal: close the directory when Fsync() encounters error
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-04-25 11:08:39 -07:00
262c939805 Merge pull request #11810 from tedyu/dir-after-rename
Use known dir-path in log message
2020-04-25 12:54:23 -04:00
9d0ecc3ba3 wal: Use known dir-path in log message
Signed-off-by: Ted Yu <yuzhihong@gmail.com>
2020-04-25 09:14:30 -07:00
69b70e8c43 Merge pull request #11807 from spzala/walread
wal: fix panic when decoder not set
2020-04-23 14:55:11 -07:00
0d5c692eb6 wal: fix panic when decoder not set
Handle the related panic and clarify doc.
2020-04-23 15:53:05 -04:00
7a69d72a68 Merge pull request #11806 from philips/brandon-leaving-red-hat
MAINTAINERS: update @philips email
2020-04-23 11:02:27 -07:00
e2e8fe63df MAINTAINERS: update @philips email
Brandon is no longer at Red Hat. Use a different email.
2020-04-23 10:47:02 -07:00
f4b650b51d Merge pull request #11793 from gyuho/fix
wal: check out of range slice in "ReadAll", entry limits in "decodeRecord"
2020-04-23 11:24:42 -04:00
c716d764d0 wal: check out of range slice in "ReadAll", "decoder"
wal: add slice bound checks in decoder

CHANGELOG-3.5: add wal slice bound check
CHANGELOG-3.5: add "decodeRecord"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-22 19:07:40 -07:00
2369cb3678 Documentation: note on password strength (#11796) 2020-04-22 15:50:29 -07:00
6aea6edc34 Merge pull request #11791 from linux-on-ibm-z/s390x-changelog
CHANGELOG-3.5: update from #11548 and #11358
2020-04-22 11:38:58 -04:00
43e8ffafd8 Merge pull request #11795 from mitake/cn-auth-log
auth: a new error code for the case of password auth against no passw…
2020-04-22 11:26:07 -04:00
b6032eb7d2 auth: a new error code for the case of password auth against no password user 2020-04-22 23:25:24 +09:00
b40f18b506 pkg: check file stats (#11798)
modify file util.
2020-04-21 19:58:48 -07:00
c1e7f73a02 CHANGELOG-3.4: remove invalid gRPC-go dependency bump
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-04-20 13:53:59 -04:00
8d52be5c85 Merge pull request #11790 from mlmhl/changelog-3.5
CHANGELOG: update from #11711
2020-04-20 11:31:17 -04:00
16570bbf98 CHANGELOG: update from #11711 2020-04-20 16:27:20 +08:00
0461b3fa51 grpcproxy: make grpc keep alive related options configurable (#11711)
Currently grpc-proxy doesn't config keep alive related options, so it
will use the default values provided by the underlay gprc library. If
clients uses a keep alive ping interval smaller than server's default
minTime, connections between server and clients will be closed and
reopened frequently.
2020-04-17 21:20:03 -07:00
0908a8bd10 Merge pull request #11725 from YoyinZyc/downgrade-api
[Etcd Downgrade] Store downgrade info to backend
2020-04-17 15:29:10 -07:00
b368be577b Merge pull request #11779 from tangcong/optimize-etcd-startup-time
etcdserver: significantly reduces start-up time
2020-04-16 19:34:51 -07:00
6c41190bc5 etcdserver: make recoverSnapshotBackend more robust 2020-04-17 09:30:22 +08:00
3e6426198f CHANGELOG: update for 11779 2020-04-17 06:43:24 +08:00
3c85b67f7b etcdserver: remove redundant storage restore operation 2020-04-17 06:43:24 +08:00
f2c5dd4aa2 membership: add downgradeInfo to cluster; add getter and setter of downgradeInfo 2020-04-16 12:01:33 -07:00
1166b1f195 embed: fix compaction runtime err (#11787)
Handle negative value input which currently gives a runtime error.
2020-04-15 16:13:19 -07:00
3faa06e2e7 Merge pull request #11782 from andreimatei/confchange.comment
raft: minor comment fix in confchange
2020-04-15 18:16:36 -04:00
38ef3dad1f CHANGELOG-3.5: update from #11548 and #11358
Signed-off-by: Nirman Narang <narang@us.ibm.com>
2020-04-13 14:53:18 +00:00
0db04a38a5 raft: minor comment fix in confchange 2020-04-12 22:44:37 -04:00
59f5fb25a5 Merge pull request #11771 from tangcong/optimize-range-count-only-performance
mvcc: reduce count-only range overhead
2020-04-10 10:14:15 -07:00
b62cec9e40 Merge branch 'master' into optimize-range-count-only-performance 2020-04-10 23:38:36 +08:00
c8d3e99db0 Merge pull request #11750 from tangcong/changelog
CHANGELOG: update for #11699,#11670,#11652,#11735,#11659
2020-04-10 23:24:46 +08:00
2ecbcb8f9d CHANGELOG: update for 11771 2020-04-10 19:30:32 +08:00
3594ab94cf e2e: add getCountOnlyTest testcase 2020-04-10 19:24:54 +08:00
730f3f1d78 mvcc: reduce count-only range overhead 2020-04-10 19:23:38 +08:00
16fc8a2b4b Documentation/op-guide: Re-generate alert rules and dashboard from mixin 2020-04-07 18:15:02 +02:00
2c4877064e Documentation/etcd-mixin: Use etcd_mvcc_db_total_size_in_bytes metric 2020-04-07 18:14:23 +02:00
68c5f6066f Documentation/etcd-mixin: Set unique UID for Grafana dashboard 2020-04-07 18:13:41 +02:00
7eae024ead raft: only redirect msg produced by own node (#11466)
Signed-off-by: Fullstop000 <fullstop1005@gmail.com>
2020-04-06 20:27:46 -07:00
5404b44b15 github: add stale issue close time (#11763)
Adding the numbers of days in the info.
2020-04-06 13:56:45 -07:00
b41711276e clientv3/concurrency: do not swallow error (#11458) 2020-04-06 12:07:59 -07:00
be103ea558 github: stale.yml initial commit (#11760)
There are open issues and pull requests that are years old on the etcd
repo and hundreds of issues that remain open.

Unfortunately it is just far outside the ability of the maintainer team
to take care of all of these issues manually.

So, follow the lead of Kubernetes and introduce a bot that closes PRs
and issues if they are stale.

See more information here: https://github.com/probot/stale#probot-stale
2020-04-06 10:49:49 -07:00
a18cbc8a22 Documentation: add note for #11689 (#11759) 2020-04-06 10:16:47 -07:00
22e1baddcb Merge pull request #11754 from polyrabbit/unpermitted-watch
etcdserver: watch stream got closed if one watch request is not permitted
2020-04-06 21:56:05 +08:00
03fec33fa0 Merge pull request #11756 from mlmhl/changelog-11743
CHANGELOG: update from #11743
2020-04-06 22:46:14 +09:00
5eacb4b374 Merge pull request #11743 from mlmhl/kubectl
etcdctl: support query count only of specified prefix
2020-04-06 22:45:55 +09:00
12030cfda3 CHANGELOG: update from #11743 2020-04-06 16:55:55 +08:00
73b936b50a etcdserver: watch stream got closed once one request is not permitted (#11708) 2020-04-06 13:43:37 +08:00
b2ebf6ee35 CHANGELOG: update for #11699,#11670,#11652,#11735,#11659 2020-04-06 11:17:12 +08:00
c623f798cb etcdserver: add trace for txn request (#11491)
* etcdserver: add trace for txn request

* pkg/traceutil: added StopSubTrace as a sign of the end of subtrace. Added test case for logging out subtrace.
2020-04-04 14:46:03 -07:00
2092b5b1a9 pkg, clientv3: etcdctl snaprestore when data-dir empty (#11648) 2020-04-04 14:41:19 -07:00
dbcf540c88 raft/rafttest: Prune Unused Functions (#11625)
* raft/rafttest: prune unused network interface

* raft/rafttest: prune unused raftNetwork.heal()

* raft/rafttest: prune unused logLevels.strToLev()

* raft/rafttest: prune unused InteractionEnv.writeErr()
2020-04-04 14:38:14 -07:00
51bdeb39e6 Merge pull request #11742 from zhangbitao/master
etcdmain: fix typo
2020-04-02 21:42:48 +08:00
aa7b056a77 etcdctl: support query count only of specified prefix 2020-04-02 11:04:26 +08:00
209102655b etcdmain: fix typo 2020-04-02 10:04:35 +08:00
0941cfc11d CHANGELOG: add compaction updates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-01 12:05:00 -07:00
0f1694c96f CHANGELOG*: add new patch release
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-04-01 10:55:48 -07:00
5ceb7019f8 Merge pull request #11738 from gyuho/wal-metrics
wal: add "etcd_wal_writes_bytes_total"
2020-04-01 09:08:10 -07:00
054b0e5d93 Merge pull request #11735 from wswcfan/optimize-auth-lock-performance
auth: dramatically improve checkPassword performance
2020-04-01 23:24:35 +08:00
381e77c667 CHANGELOG-3.5: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-31 20:34:16 -07:00
af261f1a1b wal: add "etcd_wal_writes_bytes_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-31 20:31:32 -07:00
4051e3e36a pkg/ioutil: add "FlushN"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-31 20:31:24 -07:00
9cf3162d11 auth: optimize lock scope for CheckPassword
to improve authentication performance in concurrent scenarios when enable auth and using authentication based password
2020-04-01 02:21:26 +08:00
dd816f0735 CHANGELOG-3.4: update release date
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-29 12:44:05 -07:00
587e08822d CHANGELOG: add lease fix
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-29 12:38:04 -07:00
2ffbde8376 Merge pull request #11731 from tangcong/fix-lease-memory-leak
lease: fix memory leak in LeaseGrant
2020-03-29 12:30:49 -07:00
b1da829844 lease: fix memory leak in LeaseGrant when node is follower 2020-03-29 10:13:56 +08:00
a04b74d4b4 Merge pull request #11639 from jingyih/linearizable_memberlist
*: serve member list API with linearizable guarantee
2020-03-27 11:13:58 -04:00
0344b70906 *: make MemberList linearizable
- Add linearizable field to etcdserverpb.MemberListRequest.
- Change behavior of clienv3 MemberList API. Now it is served with
linearizable guarantee.
2020-03-25 20:16:20 -07:00
3ac7a11515 Merge pull request #11699 from tangcong/refactor-consistentindex
*: refactor consistentindex
2020-03-25 18:07:44 -07:00
3cea310b61 Merge pull request #11637 from eddycjy/update-changelog
CHANGELOG-3.5: update from #11564
2020-03-25 15:19:02 -07:00
6325754d96 Merge pull request #11710 from tangcong/revert-rolegrantpermission
auth: ensure RoleGrantPermission is compatible with older versions
2020-03-25 23:17:34 +08:00
ef57a615ad Merge branch 'master' into update-changelog 2020-03-25 15:02:24 +08:00
71a241e876 Merge pull request #11715 from YoyinZyc/downgrade-api
[Etcd downgrade] Add downgrade related api in server side
2020-03-24 21:31:51 -07:00
631c23d198 etcdserver/cindex: add unit test 2020-03-25 10:59:19 +08:00
804070ce87 clientv3/snapshot: clean up initIndex 2020-03-25 10:59:19 +08:00
7b2018683a *: refactor consistent index 2020-03-25 10:59:15 +08:00
d70600feca auth: ensure RoleGrantPermission is compatible with older versions 2020-03-25 09:50:07 +08:00
bbb0fcfae9 Merge pull request #11719 from fengpf/master
*: fix goroutines typos
2020-03-24 13:50:56 -07:00
86e607a2cd Merge pull request #11723 from gyuho/init
etcdmain: log "SdNotify" error/success
2020-03-24 13:50:34 -07:00
2329acf05b etcdmain: log "SdNotify" error/success
I am seeing some silent failures around systemd service.
This will help find out bugs in "go-systemd"

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-24 12:20:20 -07:00
1f70c0b075 *: fix goroutines typos 2020-03-24 15:33:47 +08:00
073bc22d35 etcdserver: add downgrade api in maintenance server. 2020-03-22 22:35:08 -07:00
d8b9b54348 etcdserver: add downgrade rpc proto api. 2020-03-20 17:37:26 -07:00
0eee733220 mvcc: no need to hold tx lock when converting kvs to watch events (#11701)
Method `kvsToEvents` can take a long time if a large number of events need to be synchronized. However, this method does not access any shared resources, so it no need to hold the tx lock when executing. So we can move it outside to reduce lock holding time.
2020-03-19 21:01:36 -07:00
72526cef00 Merge pull request #11709 from gyuho/bbolt
vendor: update bbolt v1.3.4, zap v1.14.1
2020-03-19 20:28:00 -07:00
3ffb3aed3a vendor: update bbolt v1.3.4, zap v1.14.1
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-19 13:36:43 -07:00
45a45d3c0a Merge pull request #11706 from gyuho/fix-race
clientv3: fix racy writes to context key
2020-03-19 10:34:08 -04:00
b6d98719f6 words: whitelist "racey"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:32:42 -07:00
eeb371b7c1 clientv3: fix racy writes to context key
=== RUN   TestWatchOverlapContextCancel

==================

WARNING: DATA RACE

Write at 0x00c42110dd40 by goroutine 99:

  runtime.mapassign()

      /usr/local/go/src/runtime/hashmap.go:485 +0x0

  github.com/coreos/etcd/clientv3.metadataSet()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c

  github.com/coreos/etcd/clientv3.withVersion()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137

  github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81

  google.golang.org/grpc.NewClientStream()

      /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e

  github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3

  github.com/coreos/etcd/clientv3.(*watchGrpcStream).run()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b

Previous read at 0x00c42110dd40 by goroutine 130:

  reflect.maplen()

      /usr/local/go/src/runtime/hashmap.go:1165 +0x0

  reflect.Value.MapKeys()

      /usr/local/go/src/reflect/value.go:1090 +0x43b

  fmt.(*pp).printValue()

      /usr/local/go/src/fmt/print.go:741 +0x1885

  fmt.(*pp).printArg()

      /usr/local/go/src/fmt/print.go:682 +0x1b1

  fmt.(*pp).doPrintf()

      /usr/local/go/src/fmt/print.go:998 +0x1cad

  fmt.Sprintf()

      /usr/local/go/src/fmt/print.go:196 +0x77

  github.com/coreos/etcd/clientv3.streamKeyFromCtx()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 99 (running) created at:

  github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d

  github.com/coreos/etcd/clientv3.(*watcher).Watch()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e

Goroutine 130 (running) created at:

  github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d

  github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel()

      /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44

  testing.tRunner()

      /usr/local/go/src/testing/testing.go:657 +0x107

==================

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 17:05:05 -07:00
bf883bd15b CHANGELOG*: add new patch releases
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 15:28:05 -07:00
b50c92cb73 Merge pull request #11704 from gyuho/log-health
*: log server-side /health checks
2020-03-18 12:33:13 -07:00
92f180c574 *: log server-side /health checks
To make it easier to root-cause when /health check fails.
For example, we are using load balancer to health check
each etcd instance, and when one etcd node gets terminated,
it's hard to tell whether etcd "server" was really failing
or client (or load balancer") failed to reach the etcd cluster
which is also failure in load balancer health check.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-18 11:14:05 -07:00
1c16c242db Merge pull request #11687 from gyuho/embed-client-version
clientv3: fix "hasleader" metadata key, embed client version
2020-03-18 09:16:34 -07:00
33907477dd *: add "etcd_server_client_requests_total", tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
58ba322bb4 clientv3: embed API version
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
5cb1e0b342 clientv3: fix metadata overwrites in "WithRequireLeader"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-03-17 12:16:11 -07:00
07a74d61cb Merge pull request #11675 from GSokol/master
etcdserver/api/etcdhttp: checkHealth refactoring
2020-03-17 22:43:27 +08:00
a33e1b5fae etcdserver/api/etcdhttp: checkHealth refactoring
Small refactoring of
`go.etcd.io/etcd/etcdserver/api/etcdhttp/metrics.go.checkHealth`
function just to avoid anoying repeatings of `if h.Health == "true" {`
2020-03-17 15:37:11 +02:00
c4d8bdc881 Merge pull request #11698 from hexfusion/cl-bump-11694
CHANGELOG: bump to include #11694
2020-03-17 14:07:06 +08:00
31e7b34f71 Merge pull request #11700 from jingyih/correct_changelog_3p4
CHANGELOG-3.4: correction
2020-03-17 04:03:02 +08:00
33fb62098b CHANGELOG: correction 2020-03-16 12:18:28 -07:00
193e60ebd9 Merge pull request #11670 from tangcong/optimize-auth-store-log
*: optimize auth/etcdserver logs to facilitate troubleshooting data inconsistency
2020-03-16 00:15:14 +08:00
8e35c0b867 CHANGELOG: bump to include #11694
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-03-15 07:41:41 -04:00
7444d3ad53 proxy/grpcproxy: add return on error for metrics handler
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2020-03-14 16:34:16 -04:00
23c2380105 Merge pull request #11666 from jingyih/update_changelog_from_11638
CHANGELOG: update from #11638
2020-03-11 19:23:59 -04:00
5a17367923 etcdserver: print warn log when failed to apply request 2020-03-10 23:13:53 +08:00
0d084d3a08 auth: cleanup saveConsistentIndex in NewAuthStore 2020-03-10 10:04:58 +08:00
6d1982efe8 Merge pull request #11659 from wswcfan/add-auth-revision-status
etcdserver: add auth revision to AuthStatus to improve observability and testability
2020-03-09 23:08:55 +09:00
221f0cc107 Merge pull request #11564 from eddycjy/balancer-error
clientv3: Fix grpc-go(v1.27.0) incompatible changes to balancer/resolver.
2020-03-06 10:35:22 -08:00
e0ff5ca318 RFC Documentation: enhance description of lock and lease (#11490)
* Documentation: enhance description of lock and lease

* Documentation: an executable implementation of fencing

* docs: api guarantees

cleanup lease grammar slightly

* docs: learning/lock/README.md improve grammar

Co-Authored-By: Steven E. Harris <seh@panix.com>

* docs: learning: improve locks and leases grammar

Co-authored-by: Brandon Philips <brandon@ifup.org>
Co-authored-by: Steven E. Harris <seh@panix.com>
2020-03-05 10:31:47 -08:00
15eeb2c4ae etcdserver: add auth revision to AuthStatus to improve observability and testability 2020-03-04 22:37:24 +08:00
c6fce8c320 Documentation: generate *.swagger.json using latest protoc-gen-swagger 2020-03-04 22:36:13 +08:00
6f850a65a1 raft: cleanup read index code (#11528)
Signed-off-by: qupeng <qupeng@pingcap.com>
2020-03-03 09:20:25 -08:00
ebe256d61c auth: print warning log when error is ErrAuthOldRevision 2020-03-03 23:21:25 +08:00
3f4f7c5a6c Merge pull request #11669 from jingyih/update_changelog_11621
CHANGELOG: update from #11621
2020-03-02 15:53:56 -05:00
cec9807a85 changelog: update from 11621
Also moving an previous update in changelog 3.4 to the right spot.
2020-03-02 10:11:13 -08:00
cb633418a2 Merge pull request #11652 from tangcong/fix-auth-store-corruption-bug
auth/store: save consistentIndex to fix a data corruption bug
2020-03-01 18:40:11 +08:00
e2ff769244 CHANGELOG: update from 11638 2020-02-29 07:33:55 -08:00
d774916f6d Merge pull request #11638 from jingyih/fix_etcdctl_member_list
etcdctl: fix member add (again...)
2020-02-29 23:08:33 +08:00
f14d2a087f auth: add new metric 'etcd_debugging_auth_revision' 2020-02-29 13:31:44 +08:00
08a8b80e32 tools/etcd-dump-db: add auth decoder, optimize print format 2020-02-29 13:31:44 +08:00
06ad53321e *: fix auth revision corruption bug 2020-02-29 13:31:37 +08:00
52fba431b6 Merge pull request #11046 from tbg/joint-auto-leave
raft: fix auto-transitioning out of joint config
2020-02-25 13:18:29 +01:00
0544f33248 raft: clarify ApplyConfChange contract for rejected conf changes
Apps typically maintain the raft configuration as part of the state
machine. As a result, they want to be able to reject configuration change
entries at apply time based on the state on which the entry is supposed
to be applied. When this happens, the app should not call
ApplyConfChange, but the comments did not make this clear.

As a result, it was tempting to pass an empty pb.ConfChange or it's V2
version instead of not calling ApplyConfChange.

However, an empty V1 or V2 proto aren't noops when the configuration is
joint: an empty V1 change is treated internally as a single
configuration change for NodeID zero and will cause a panic when applied
in a joint state. An empty V2 proto is treated as a signal to leave a
joint state, which means that the app's config and raft's would diverge.

The comments updated in this commit now ask users to not call
ApplyConfState when they reject a conf change. Apps that never use joint
consensus can keep their old behavior since the distinction only matters
when in a joint state, but we don't want to encourage that.
2020-02-25 12:45:45 +01:00
37c7e4d1d8 raft: fix auto-transitioning out of joint config
The code doing so was undertested and buggy: it would launch multiple
attempts to transition out when the conf change was not the last element
in the log.

This commit fixes the problem and adds a regression test. It also
reworks the code to handle a former untested edge case, in which the
auto-transition append is refused. This can't happen any more with the
current version of the code because this proposal has size zero and is
special cased in increaseUncommittedSize. Last but not least, the
auto-leave proposal now also bumps pendingConfIndex, which was not done
previously due to an oversight.
2020-02-25 12:35:51 +01:00
2332705f10 raft: remove bogus tail end of membership change interaction test
The test was supposed to end earlier, but some old copy pasta
survived.
2020-02-25 12:35:51 +01:00
5d7d8c09af CHANGELOG-3.4: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-02-24 13:42:21 -08:00
085b19b5ea Merge pull request #11645 from jingyih/update_changelog_11640
CHANGELOG: update from #11640 and #11644
2020-02-23 14:42:50 +08:00
d911a2b4d1 CHANGELOG: update from 11640 and 11644 2020-02-21 23:24:11 -08:00
8c1a820a10 Merge pull request #11640 from ereslibre/fix-learner-promotion
Fix quorum calculation when promoting a learner member
2020-02-21 12:32:32 -05:00
8756286fe8 Merge pull request #11568 from lzhfromustc/GL_test_7_bugs
transport: simple fix in test functions to prevent goroutine leak
2020-02-20 16:54:42 +01:00
6991f619f2 etcdserver: fix quorum calculation when promoting a learner member
When promoting a learner member we should not count already a voting
member, but take only into account the number of existing voting
members and their current status (started, unstarted) when taking the
decision whether a learner member can be promoted.

Before this change, it was impossible to grow from a quorum N to a N+1
through promoting a learning member.

Fixes: #11633
2020-02-20 12:06:35 +01:00
d94f4cacf2 CHANGELOG-3.5: update from 11564 2020-02-19 17:19:17 +08:00
1cb959698a Merge branch 'master' into update-changelog 2020-02-19 16:31:55 +08:00
b3b75c8f56 CHANGELOG-3.5: update from 11564 2020-02-19 16:29:59 +08:00
8f5ce68adb etcdctl: fix member add command
Use members information from member add response, which is
guaranteed to be up to date.
2020-02-19 00:12:15 -08:00
d6a3c995cf Merge pull request #11621 from jingyih/corruption_check_tls
etcdserver: make corruption check work under peer mTLS
2020-02-18 18:55:13 +08:00
4258cdd2ef clientv3: fix grpc-go(v1.27.0) incompatible changes to balancer/resolver. 2020-02-18 18:27:53 +08:00
f0faa5501d Merge pull request #11630 from jingyih/check_nil_before_use_boltOpenOptions
mvcc/backend: check for nil boltOpenOptions
2020-02-15 15:25:44 +08:00
b6ee807e93 mvcc/backend: check for nil boltOpenOptions
Check if boltOpenOptions is nil before use it.
2020-02-14 21:26:52 -08:00
74cfe52809 Merge pull request #11628 from tedyu/rm-db-tmp
mvcc/backend: remove db.tmp without checking logger presence
2020-02-15 04:42:32 +08:00
7e0e6bf497 mvcc/backend: remove db.tmp regardless of logger presence 2020-02-14 12:10:24 -08:00
c031b27491 etcdserver: corruption check via http
During corruption check, get peer's hashKV via http call.
2020-02-14 06:03:15 -08:00
84ace97ad7 Merge pull request #11572 from lzhfromustc/API_Fatal_4
integration/clientv3: fix 4 API misusage in test functions
2020-02-14 00:27:41 -08:00
bdf69dff90 Merge pull request #11626 from jpbetz/changelog-11613-backport
changelog: Add #11613 backport to 3.2, 3.3 and 3.4 changelogs
2020-02-13 14:53:30 -08:00
605f30f328 changelog: Add 11613 backport to 3.2, 3.3 and 3.4 changelogs 2020-02-13 13:03:46 -08:00
8bf7b2b1b4 Merge pull request #11613 from jpbetz/fix-defrag-orphan-file
mvcc/backend: Fix corruption bug in defrag
2020-02-13 12:02:05 -08:00
f00394e384 integration/clientv3: fix 4 API misusage in test functions 2020-02-13 13:13:18 -05:00
213f7f7877 mvcc/backend: Delete orphaned db.tmp files before defrag 2020-02-12 22:40:28 -08:00
b340dfdcbb transport: simple fix in test functions to prevent goroutine leak 2020-02-12 18:32:03 -05:00
ca7a292790 Merge pull request #11617 from jingyih/remove_capnslog_etcd
vendor: remove capnslog
2020-02-12 13:06:57 -08:00
09304a4d82 vendor: remove capnslog
Clean up capnslog from etcd code base. Remove its dependency.
2020-02-12 12:33:16 -08:00
ae983523db pkg: remove capnslog (#11616)
* pkg: remove capnslog

* CHANGELOG: function signature change
2020-02-12 11:15:41 -08:00
bb29615359 proxy: remove capnslog (#11614)
* proxy: remove capnslog

* CHANGELOG: function signature change
2020-02-12 10:30:09 -08:00
71e3220d36 Merge pull request #11605 from spzala/indexcasting
etcdctl: use appropriate type conversion
2020-02-12 10:08:37 -05:00
61f279454e etcdserver/api: remove capnslog (#11606)
* etcdserver/api/rafthttp: remove capnslog

* etcdserver/api/membership: remove capnslog

* etcdserver/api/v2auth: remove capnslog

* etcdserver/api/v2discovery: remove capnslog

* etdserver/api/v2stats: remove capnslog

* etcdserver/api/v2http: remove capnslog

* etcdserver/api/v3rpc: remove capnslog

* etcdserver/api: remove capnslog

Remove capnslog from etcdserver/api. Note that capnslog was
already removed in some packages under etcdserver/api in
previous commits.
2020-02-11 13:51:25 -08:00
0a05f1a131 etcdmain: remove capnslog (#11612) 2020-02-11 13:45:44 -08:00
c94782cd55 etcdserver: remove capnslog (#11611)
remove capnslog from etcdserver pkg, except etcdserver/api.
2020-02-11 08:54:14 -08:00
a0bb739c4e Merge pull request #11610 from mitake/authstatus
etcdserver: mark AuthStatus as no side effect request
2020-02-11 11:07:29 -05:00
fda8d38bd4 etcdserver: mark AuthStatus as no side effect request 2020-02-11 23:26:50 +09:00
e5c90ebf90 embed: don't compare uint to any negative number (#11603) 2020-02-10 14:16:57 -08:00
4c25efc1f8 Discovery: do not allow passing negative cluster size (#11608)
When an etcd instance attempts to perform service discovery, if a
cluster size with negative value  is provided, the etcd instance
will panic without recovery because of
2020-02-10 10:43:41 -08:00
5852755250 Merge pull request #11604 from spzala/strconvuint
auth: parse to uint instead of int
2020-02-10 23:02:11 +09:00
384c0f1432 Merge pull request #11607 from eladb/patch-1
contributing: invalid link to "reporting-bugs"
2020-02-09 08:52:17 -05:00
819f5cf32f contributing: invalid link to "reporting-bugs"
Fix link to the reporting bugs document
2020-02-09 14:37:25 +02:00
bc4adb8b5c etcdserver: populate ResponseHeader in Alarm method (#11600)
When no Alarms are found, the response has no header. The header should always
be populated. Some components, like fields printer used by etcdctl, break when
the header is not populated.

Fixes #11581

Signed-off-by: Daniel Lipovetsky <dlipovetsky@d2iq.com>
2020-02-08 23:07:58 -08:00
acb33a5f3f etcdctl: use appropriate type conversion
index variable is being passed as uint64 so we shouldn't use Int.
Also the if loop is redundant so remove it.
2020-02-08 22:01:20 -05:00
cc5999db3a auth: parse to uint instead of int
The strconv.Atoi is equivalent to ParseInt. The index is later used as
uint so we should use strconv.ParseUint.
2020-02-08 20:28:44 -05:00
4e5314e9b5 doc: remove out-date introduction video link. (#11601)
It's easy to find etcd introduction video, and the introduction video
from the rfc doc is outdated, so removing this link.

Fixes 11591.
2020-02-07 20:49:05 -08:00
2f2354bca2 Merge pull request #11595 from jingyih/remove_capnslog_in_mvcc
mvcc: remove capnslog
2020-02-07 13:01:04 -05:00
6b389bf23c mvcc: remove capnslog 2020-02-07 07:44:44 -08:00
f6c7d5c46a Makefile: cleanup all data dirs. (#11583) 2020-02-06 13:55:02 -08:00
84fe23d530 auth: remove capnslog (#11596) 2020-02-06 12:28:14 -08:00
aea3354854 Merge pull request #11548 from linux-on-ibm-z/s390x-docker
scripts: Added s390x support for docker image release
2020-02-06 11:13:16 -08:00
6b17141ece Merge pull request #11598 from jingyih/remove_capnslog_in_wal
wal: remove capnslog
2020-02-06 13:16:48 -05:00
2fee0b153e wal: remove capnslog 2020-02-06 06:11:59 -08:00
071e70cdc4 *: add a new API and command for checking auth status (#11536)
This changes have started at etcdctl under auth.go, and make changes to stub out everything down into the internal raft.  Made changes to the .proto files and regenerated them so that the local version would build successfully.
2020-02-05 19:27:42 -08:00
74d5ba5777 embed: remove capnslog (#11592) 2020-02-05 10:22:32 -08:00
a924600700 Merge pull request #11590 from alrs/fix-dropped-test-error
etcdserver/api/v2v3: fix dropped test error
2020-02-05 08:41:07 -08:00
f2c3bcd086 etcdserver/api/v2v3: fix dropped test error
etcdserver/api/v2v3: use testing.T instead of log in tests
2020-02-05 07:20:12 -08:00
7395ed8e5d Merge pull request #11578 from jingyih/set_zap_as_default_logger
*: set zap as default logger, remove capnslog
2020-02-04 22:58:45 -08:00
5a807dedbc CHANGELOG: update from 11586 (#11589) 2020-02-04 08:12:43 -08:00
3ad2b679f0 Merge pull request #11582 from YoyinZyc/changelog-11574
CHANGELOG-3.5: add for #11574
2020-02-04 09:06:24 -05:00
725e09023a *: set zap as default logger, remove capnslog
Set zap as default logger. Remove capnslog and deprecated logging
flags.
2020-02-04 04:57:49 -08:00
1e726077c5 Merge pull request #11586 from jingyih/use_debug_level_for_token
auth: use correct logging level
2020-02-04 20:51:37 +09:00
081d5f5797 auth: correct logging level 2020-02-03 22:54:39 -08:00
05a0e361aa CHANGELOG-3.5: add for #11574 2020-02-03 13:21:35 -08:00
a698ad65f5 Merge pull request #11574 from YoyinZyc/simpligy-grpc
clientv3: simplify grpc dialer usage.
2020-02-03 12:41:41 -08:00
b9d00aae7c Documentation: add section headings to integrations doc (#11573)
Signed-off-by: lucperkins <lucperkins@gmail.com>
2020-01-31 17:02:08 -08:00
b90d801beb clientV3: simplify grpc dialer usage. Remove workaround #11184 after bumping grpc to 1.26.0. 2020-01-30 14:51:24 -08:00
467e08c32a in multiple packages: fixed goroutine leak bugs in tests (cont.d) (#11570) 2020-01-30 10:55:59 -08:00
cad92706cf in multiple packages: fixed goroutine leak bugs in tests (#11569) 2020-01-30 10:45:59 -08:00
53f15caf73 raft: Fixed missing package name in README.md (#11566) 2020-01-29 10:15:00 -08:00
3898452b54 Merge pull request #11412 from lucperkins/lperkins/docs-restructuring-v2
Restructure documentation source files
2020-01-27 18:15:56 -05:00
342c2464ae Documentation: specify starting revision (#11559) 2020-01-27 10:18:27 -08:00
6db3c8a370 vendor: add losing grpc files (#11557) 2020-01-27 10:18:12 -08:00
23810ea285 Documentation: unify the explanation of isolation level and consistency (#11474) 2020-01-27 10:17:38 -08:00
2f25061a16 Merge remote-tracking branch 'upstream/master' into lperkins/docs-restructuring-v2 2020-01-25 11:45:54 -08:00
798c073b89 proxy/tcpproxy: fix dup unweighted remotes (#11552) 2020-01-23 14:27:34 -08:00
f806c3aee4 Merge remote-tracking branch 'upstream/master' into lperkins/docs-restructuring-v2 2020-01-22 13:08:46 -08:00
c3497c84ed mvcc: use correct error variable in defragdb (#11527)
Co-authored-by: yutedz <54038175+yutedz@users.noreply.github.com>
2020-01-22 11:03:23 -08:00
1be2f4b8e2 Documentation: Restructure directory to accommodate new site generation system
Signed-off-by: lucperkins <lucperkins@gmail.com>
2020-01-21 14:29:54 -08:00
d17a607da2 Merge pull request #11411 from spzala/missingchangelog
CHANGELOG: add missing changelogs
2020-01-20 11:25:56 -08:00
17a220eaee scripts: Added s390x support for docker image release 2020-01-20 07:09:59 -08:00
c00e929d8f Merge pull request #11535 from spzala/goversion
doc: update required go version for master
2020-01-20 08:14:30 -05:00
f2ed13af05 Merge pull request #11546 from gyuho/fix-typo
clientv3/integration: fix a typo in "TestLeasingTxnRangeCmp"
2020-01-20 02:05:40 -08:00
993e82a85b clientv3/integration: fix a typo in "TestLeasingTxnRangeCmp"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-19 23:56:22 -08:00
4fc6fa4ce5 Merge pull request #11545 from gyuho/fix
clientv3/integration: fix typo in "TestLeasingRevGet"
2020-01-19 15:11:19 -05:00
2669b83c68 clientv3/integration: fix typo in "TestLeasingRevGet"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-19 10:20:23 -08:00
45156cf3ca Merge pull request #11540 from gyuho/tests
clientv3/integration: fix tests
2020-01-17 15:40:55 -08:00
a592b808c5 Merge pull request #11538 from andyxning/fix_typo
mvcc: fix error log typo
2020-01-17 13:10:11 -08:00
091b84f154 clientv3/integration: fix "TestLeaseKeepAliveNotFound"
with "default" select, the failure case will never be selected

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-17 12:04:10 -08:00
255944bf51 clientv3/integration: expect "ErrCompacted" in "TestKVCompact"
Making tests more strict

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-01-17 12:03:13 -08:00
f260df559d mvcc: fix error log typo 2020-01-17 17:27:04 +08:00
8aa7a3dc09 mvcc: log error if tmpdb is not removed (#11523)
Co-authored-by: yutedz <54038175+yutedz@users.noreply.github.com>
2020-01-16 17:34:45 -08:00
0cfadaaaeb doc: update required go version for master
changelog and readme are already updated.
2020-01-16 16:05:46 -05:00
e5a35c827c mvcc: incomplete fix in #11502 (#11533) 2020-01-16 12:05:10 -08:00
2e3b55eb7b Changelog-3.5: update changelog for grpc upgrade (#11526) 2020-01-15 17:45:52 -08:00
250b0acac4 mvcc: check the error return in defragdb (#11524)
Co-authored-by: yutedz <54038175+yutedz@users.noreply.github.com>
2020-01-15 16:57:12 -08:00
67bf39b2ec vendor: update gRPC Go to v1.26.0 (#11522) 2020-01-15 16:56:21 -08:00
574ee42ad7 mvcc: rollback tmptx in case of error (#11525)
Co-authored-by: yutedz <54038175+yutedz@users.noreply.github.com>
2020-01-15 13:54:52 -08:00
af131b58eb CHANGELOG: update recommended versions (#11520) 2020-01-15 10:05:54 -08:00
a96bc48268 mvcc: fix snapshot sendRateBytes unit (#11502) 2020-01-14 11:29:05 -08:00
908f45ebe4 Merge pull request #11462 from jingyih/etcdctl_support_watch_progress_notify
etcdctl: support progress notify option
2020-01-14 10:07:37 -05:00
e90c56020d CHANGELOG-3.5: update 2020-01-14 01:20:36 -08:00
21f68298a5 Add tag "etcd-mixin" into the Grafana dashboard 2020-01-14 09:12:19 +01:00
b95a465235 Auth: declare auth variable inside the loop (#11476)
Co-authored-by: yutedz <54038175+yutedz@users.noreply.github.com>
2020-01-13 19:23:00 -08:00
0c787e26bc Update README.md 2020-01-09 10:06:27 -08:00
6ffd9f659e tools/benchmark: exit when txn-ops is larger than key-space-size to avoid key duplicated error. (#11499) 2020-01-06 16:36:51 -08:00
e6980b1f9f Merge pull request #11482 from cuiyuan/fix-rbcolor-string
pkg/adt: rbcolor string fix
2019-12-30 11:26:07 -08:00
98f62d27be pkg/adt: rbcolor string fix 2019-12-30 19:52:34 +08:00
5adad5e224 Merge pull request #11452 from tedyu/watch-grpc-send-err
clientv3: log warning in case of error sending request
2019-12-20 15:31:02 -08:00
cb8bf089f8 clientv3: log warning in case of error sending request 2019-12-20 15:08:33 -08:00
eaa0612e02 raft: abort leader transferring if the target is demoted (#11417)
Signed-off-by: qupeng <qupeng@pingcap.com>
2019-12-20 12:07:52 +08:00
c0de070527 etcdctl: support progress notify option
Add support for progress notify option to etcdctl watch command.
2019-12-17 14:16:42 -08:00
5770a6d286 Merge pull request #11453 from mindw/missing_metrics_doc
Documentation: added v3.4 metrics docs
2019-12-17 11:27:42 -08:00
fb7703ab43 clientv3: remove ineffective nil check (#11451) 2019-12-15 21:38:25 -08:00
8223006a97 Documentation: added v3.4.x metrics docs 2019-12-15 14:13:36 +02:00
322c38e169 Documentation/etcd-mixin: Fix etcdHighNumberOfLeaderChanges (#11448)
The `etcdHighNumberOfLeaderChanges` alert had a copy and paste
error when it was converted from docs to mixin in 10244 - we moved
from "increase over 15m > 3" to "rate over 15m > 3" which is not
the same (rate is measured per second, so it should have been
"rate over 15m > (3 / 60 / 15)").  As part of fixing that, we
need to capture when prometheus starts or when new etcd clusters
are captured with a high leader change - i.e. if you start a new
etcd cluster and at the moment prometheus first scrapes you are
already at 5 leader changes, we should fire on that transition.

This alert is also now more responsive, so if you get a quick
burst of 3 leader changes we'll alert within 5m rather than 15m.
2019-12-13 16:00:11 -08:00
7f3dd59d22 mvcc: Log warning when compaction encounters error (#11447) 2019-12-13 15:59:49 -08:00
378b05b8dd Merge pull request #11443 from YoyinZyc/changelog-11418
CHANGELOG: Add #11418 to changelog-3.4, changelog-3.5
2019-12-11 18:16:30 -08:00
769d2f868e CHANGELOG: update for #11418 2019-12-11 16:09:20 -08:00
d70251835d integration: gracefully shut down gRPC server (#11437) 2019-12-11 13:07:16 -08:00
b7d9f45a43 Merge pull request #11432 from YoyinZyc/changelog-11427
CHANGELOG-3.5: update for #11427
2019-12-11 11:45:17 -08:00
69841bdb0b CHANGELOG-3.5: update for #11427 2019-12-11 11:43:04 -08:00
4f35794458 Merge pull request #11418 from YoyinZyc/fix-add-user-grpc
auth: fix NoPassWord check when add user
2019-12-10 12:47:35 -08:00
5127cfb45f e2e: test curl auth on onoption user 2019-12-10 11:25:16 -08:00
aea34c14ca auth: fix NoPassWord check when add user 2019-12-09 14:10:54 -08:00
9f81002a11 Merge pull request #11427 from YoyinZyc/migration-cluster-attr
Migrate cluster attributes to use v3 backend
2019-12-09 13:30:15 -08:00
7784ca8997 etcdserver: remove v2 version set; e2e: fix tests. 2019-12-09 13:08:00 -08:00
ed5a01a48d etcdserver: recover cluster version from backend 2019-12-05 16:25:13 -08:00
5cd2502ab1 etcdserver: use v3 to publish member attr 2019-12-05 16:25:13 -08:00
0c3401fa76 etcdserver: use V3 to update cluster version 2019-12-05 16:25:13 -08:00
dcd622b2c7 etcdserver: add v3 request type for cluster attr
Added ClusterVersionSetRequest for setting cluster version via v3 apply.

Added ClusterMemberAttrSetRequest for setting clsuter member attributes
via v3 apply.
2019-12-05 16:25:13 -08:00
1f8764be3b integration: prevent goroutines leaks in test (#11318)
Some goroutines in test functions will be leaked in certain cases.
This patch stops these leaks no matter what happens in test, by
putting the blocking channel in select together with a new stopc
channel, or adding 1 buffer to the blocking channel.
2019-12-05 15:40:10 -08:00
fd2dddb39f vendor: bump go-systemd to v22.0.0 to fix mod tidy error (#11387) 2019-12-04 18:47:14 -08:00
sun
c5887cdc49 mvcc: go style err check (#11419) 2019-12-04 18:46:17 -08:00
d4d7ad3908 mirror: make etcdctl make-mirror subcommand accept --dest-user and --dest-password (#11384) 2019-12-04 14:06:49 -08:00
e5356cca80 Merge pull request #11408 from xiang90/lock
concurrency: make lock more reliable
2019-12-04 12:10:47 -08:00
659f5a4086 Update CHANGELOG-3.3.md 2019-12-03 16:00:17 -08:00
18e9e45bda Merge pull request #11358 from linux-on-ibm-z/s390x-binary
scripts/build-binary: Added s390x support for binary release
2019-12-02 14:40:28 -08:00
ae2bc166db Merge pull request #11413 from YoyinZyc/fix-user-options-nil-pointer
auth: fix user.Options nil pointer
2019-12-02 14:39:38 -08:00
a45b99725f CHANGELOG: add missing changelogs
related #11184 #11211 #11354
2019-12-02 17:28:42 -05:00
c8ffd921d4 auth: fix user.Options nil pointer 2019-12-02 14:09:07 -08:00
b14c7cd448 concurrency: make lock more reliable 2019-12-02 10:54:30 -08:00
d1139d3a66 Merge pull request #11404 from jingyih/update_changelog
CHANGELOG: update from PR 11403
2019-11-27 13:28:41 -08:00
6be629cc14 CHANGELOG: update from PR 11403 2019-11-27 13:19:29 -08:00
ccc52bef1f Merge pull request #11400 from WIZARD-CXY/compactfix
mvcc/kvstore:fixcompactbug
2019-11-27 13:03:32 -08:00
3275e5f064 Merge pull request #11399 from johncming/remove-dup-code
etcdmain: remove duplicate codes.
2019-11-27 12:56:53 -08:00
655df65e93 mvcc/kvstore:fixcompactbug 2019-11-27 17:55:42 +08:00
6c0e2d31fb etcdmain: remove duplicate codes. 2019-11-27 17:04:43 +08:00
419ffb77ab Merge pull request #11287 from YoyinZyc/version-test-rolling-start
e2e: add cluster version test with rolling start servers
2019-11-26 15:12:54 -08:00
9c08316794 Merge pull request #11395 from jingyih/update_changelog
CHANGELOG: update from PR #11374
2019-11-26 15:03:22 -08:00
49c3f74314 CHANGELOG: update from PR 11374 2019-11-26 14:16:37 -08:00
44fc9e3d92 Merge pull request #11266 from YoyinZyc/upgrade-downgrade-test
e2e: only expect cluster's major version is > 3 in release upgrade test
2019-11-25 14:03:57 -08:00
cab4cace5e tests: change MANUAL_VER to 3.4.3 2019-11-25 10:57:47 -08:00
5a1102341e e2e: only expect cluster's major version is > 3 in release upgrade tests. 2019-11-25 10:57:27 -08:00
ec5221774c Merge pull request #11363 from yeasy/patch-1
docs: Update the raft usage by adding Hyperledger project
2019-11-20 16:18:32 -08:00
908ef19d86 Merge pull request #11374 from YoyinZyc/put_throughput_metrics
mvcc: add "etcd_mvcc_put_size_in_bytes" metrics
2019-11-19 23:38:30 -08:00
a5e747b2d4 mvcc: update to "etcd_debugging_mvcc_total_put_size_in_bytes" 2019-11-19 15:41:58 -08:00
28f40d7794 mvcc: add "etcd_mvcc_put_size_in_bytes" to monitor the throughput of put request. 2019-11-19 13:55:37 -08:00
b373cb9051 Merge pull request #11373 from gtamas/master
Documentation: changed ETCD manager URL
2019-11-19 13:28:29 -08:00
a0571166bc feat: changed ETCD manager URL
It now points to our domain instead of the Github page.
2019-11-19 22:17:29 +01:00
4b755e8935 docs: Update the raft usage by adding Hyperledger project 2019-11-19 09:54:16 -08:00
cabf5942b2 scripts/build-binary: Added s390x support for binary release 2019-11-13 22:57:23 -08:00
63dd73c186 Merge pull request #11354 from spzala/issue11353
CHANGELOG: Add v3.4.3
2019-11-13 16:21:43 -05:00
8adebf4a28 CHANGELOG: Add v3.4.3
Add v3.4.3 and fix v3.4.2 release date.

Fixes # 11353
2019-11-13 14:27:23 -05:00
d05459ed5b Merge pull request #11322 from jcalvert/etcdctl_healthoutput
etcdctl: Print healthy results to stdout, not stderr
2019-11-10 21:30:23 -05:00
7fe2719d6d Merge pull request #11341 from hexfusion/3.2.28_rl
CHANGELOG: update for #11308
2019-11-10 17:30:29 -08:00
5fa686f912 Merge pull request #11342 from hexfusion/bump_3.2.28
CHANGELOG: bump release 3.2.28
2019-11-10 17:29:53 -08:00
e9c30c570f CHANGELOG: update for #11308
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-11-10 19:37:02 -05:00
350626ba90 CHANGELOG: bump release 3.2.28
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-11-10 19:33:30 -05:00
b852e37895 Merge pull request #11333 from el10savio/Procfiles_Comments
Procfiles: Added Comments To Procfiles #11329
2019-11-06 11:14:39 -08:00
65a497dded Procfiles: Added Comments To Procfiles
To indicate to change the path of bin/etcd binary as it failes at times to
run using Goreman because of etcd located in another directory.

Fixes #11329
2019-11-06 08:44:04 +05:30
cbc1340af5 Merge pull request #11330 from shenjiangc/master
mvcc/kvstore: Optimize compaction, slove conflict for #11150
2019-11-04 20:56:01 -08:00
72cc5dcf39 mvcc/kvstore:when the number key-value is greater than one million, compact take too long and blocks other requests 2019-11-05 11:48:32 +08:00
edd011c6d1 Merge pull request #11325 from lijianwh/#11320_fix_bug
etcdserver: fix a bug which append object to a new allocated sized slice
2019-11-02 10:18:53 -07:00
cdc28507ef etcdserver: fix append object to a new allocated sized slice 2019-11-01 10:19:45 +08:00
fa972cf296 Merge pull request #11300 from lzhfromustc/MU_mvcc2
mvcc: Add Unlock before panic to prevent double lock
2019-10-31 10:14:34 -07:00
cb0ba4af59 Merge pull request #11301 from lzhfromustc/MU_benchmarkL
lease:Add Unlock before break in loop
2019-10-31 10:13:27 -07:00
9177098779 Merge pull request #11313 from gyuho/retry-log
clientv3: fix retry/streamer error message
2019-10-31 10:05:13 -07:00
e48ad568b9 etcdctl: Print healthy results to stdout, not stderr
Printing to stderr when there is no actual error is not intuitive
and makes tool integration more difficult
2019-10-31 10:01:03 -05:00
5dc98c50d7 clientv3: fix retry/streamer error message
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-10-30 21:29:44 -07:00
bbe1e78e62 Merge pull request #11308 from jingyih/wait_purgefile_loop
etcdserver: wait purge file loop to finish during shutdown
2019-10-30 15:54:49 -07:00
c447955d93 etcdserver: wait purge file loop during shutdown
To prevent the purge file loop from accidentally acquiring the file lock
and remove the files during server shutdowm.
2019-10-30 14:21:08 -07:00
ce61cd4324 lease:Add Unlock before break in loop 2019-10-28 22:52:24 -04:00
235d7c2e51 mvcc: Add Unlock before panic to prevent double lock 2019-10-28 22:38:34 -04:00
84e2788c2e Merge pull request #10468 from jingyih/remove_auth_loop
etcdserver: remove infinite loop for auth in raftRequest
2019-10-29 00:11:40 +09:00
41539df490 integration: disable TestV3AuthOldRevConcurrent
Disable TestV3AuthOldRevConcurrent for now. See
https://github.com/etcd-io/etcd/pull/10468#issuecomment-463253361
2019-10-25 18:39:07 -07:00
669cd6b5c6 Merge pull request #11288 from ZYunH/format-errors
fileutil, src: format errors
2019-10-25 14:43:33 -04:00
279fee668b fileutil, src: format errors 2019-10-25 22:31:49 +08:00
f17107eb23 Merge pull request #11289 from gyuho/rrr
scripts/release: list GPG key only when tagging is needed
2019-10-23 11:12:35 -07:00
fd5a25fcec scripts/release: list GPG key only when tagging is needed
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-10-23 10:33:39 -07:00
fac434b952 *: use Go 1.13.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-10-23 10:32:39 -07:00
4243ebd3ef e2e: add cluster version test for rolling start servers 2019-10-22 12:47:56 -07:00
b5afbdd8d0 Merge pull request #11285 from WIZARD-CXY/changelog
Update CHANGELOG-3.5
2019-10-22 11:52:51 -07:00
503ce2fdfd CHANGELOG: update Changelog-3.5 2019-10-22 17:39:21 +08:00
92f313811e Merge pull request #11257 from lzhfromustc/grpcproxy_mis_unlock
grpcproxy: Add an Unlock before continue to prevent double lock
2019-10-21 17:47:43 -07:00
342b58ba31 Merge pull request #11279 from YoyinZyc/stream-version-test
rafthttp: add test stream support for current version.
2019-10-21 17:33:43 -07:00
c57f8b3af8 nit: fix type s/custer/cluster 2019-10-21 13:00:19 -07:00
bd0a67a2d8 Merge pull request #11282 from YoyinZyc/cluster-version-test
e2e: add cluster version test
2019-10-21 11:54:41 -07:00
321e3ce1a6 e2e: add cluster version test 2019-10-21 11:14:18 -07:00
80a177292e rafthttp: add test stream support for current version. 2019-10-21 09:45:00 -07:00
5dc12f2725 Merge pull request #11274 from YoyinZyc/fix-upgrade-failure
rafthttp: add 3.4.0,3.5.0 stream type
2019-10-20 19:20:06 -07:00
9c48dfabff Merge pull request #11278 from spzala/pathref
doc: update file ref path
2019-10-17 20:58:30 -04:00
d185a54cb4 doc: update file ref path
Update the adopters file path.
2019-10-17 20:34:24 -04:00
de9cc79efe Merge pull request #11276 from spzala/adopters
doc: move production users to a standard ADOPTERS file
2019-10-17 17:12:39 -07:00
d73e04efd9 doc: move production users to a standard ADOPTERS file
The details of production users fits better in the standard
ADOPTERS file as used by many other CNCF projects like
CoreDNS, containerd etc.
2019-10-17 18:36:28 -04:00
a0e528e4b1 rafthttp: add 3.4.0,3.5.0 stream type 2019-10-17 14:25:56 -07:00
51b1677d99 Merge pull request #11272 from etcd-io/wenjiaswe-changelog11261-1
Update CHANGELOG-3.3 for 11261
2019-10-17 14:18:12 -07:00
7534e5c14b Merge pull request #11273 from etcd-io/wenjiaswe-patch-1
Update CHANGELOG-3.2 for #11271
2019-10-17 14:17:56 -07:00
8dc181dc5e Update CHANGELOG-3.2 for #11271
Add changelog for etcd_cluster_version metrics from ##11271
2019-10-17 13:10:46 -07:00
8bedf3dde8 Update CHANGELOG-3.3 for 11261
Add log for etcd_cluster_version metrics addition in #11261
2019-10-17 13:06:06 -07:00
a65e4d3357 Merge pull request #11262 from WIZARD-CXY/updateflag
*: promote the boltdb-freelistType from experimental to official
2019-10-17 10:56:45 -07:00
f62ea1ceca *: promote the boltdb-freelistType from experimental to official and set default type to hashmap 2019-10-17 15:40:38 +08:00
adb126afc4 Merge pull request #11267 from k-ye/fix
backend: fix comment for bucketBuffer.merge()
2019-10-16 20:57:28 -07:00
49f4a418f9 backend: fix comment for bucketBuffer.merge() 2019-10-17 08:48:12 +09:00
debf86c129 Merge pull request #11265 from jingyih/fix_cluster_version_metrics
etcdserver: strip patch version in metrics
2019-10-16 16:25:15 -07:00
57632d27ee Merge pull request #11260 from jingyih/update_changelog
CHANGELOG: update from PR 11254
2019-10-16 12:43:00 -07:00
444bfdff59 etcdserver: strip patch version in metrics
Strip patch version in cluster version metrics during node restart.
2019-10-16 12:39:17 -07:00
9ac0d65af8 CHANGELOG: update from PR 11254 2019-10-15 12:27:42 -07:00
e8a60d735f grpcproxy: Add an Unlock before continue to prevent double lock 2019-10-14 21:44:07 -04:00
bcc147127d Merge pull request #11254 from jingyih/strip_patch_version_in_cluster_version_metrics
etcdserver: strip patch version in cluster version metrics
2019-10-14 17:22:33 -07:00
1333abc606 etcdserver: strip patch version in cluster version
Strip patch version in cluster version metrics.
2019-10-14 16:59:09 -07:00
1b041a5783 Merge pull request #11249 from spzala/issue9726
doc: add lease time
2019-10-14 19:47:08 -04:00
3ef2ad8e11 Merge pull request #11247 from jpbetz/version-tag-branch-checks
Add version, tag and branch checks to release script
2019-10-13 17:49:27 -07:00
9002c1951f doc: add lease time
The current lease time is short and as such can lead to a timeout
error as explained in the related issue which can be confusing.

Fixes #9726
2019-10-13 16:38:28 -04:00
a4e9d81df7 Merge pull request #11225 from spzala/issue11210
pkg: handle version env variable
2019-10-13 11:30:33 -07:00
27535517b1 pkg: handle version env variable
Version flag is meant for displaying version and it's a boolean.
It's for an internal use, and we may not want users to set
it in the environment.

Fixes # 11210
2019-10-13 13:23:59 -04:00
f20daa5977 Merge pull request #11233 from jingyih/unset_old_cluster_version_metrics
etcdserver: unset old cluster version in metrics
2019-10-12 12:07:45 -07:00
9c4194f6ef etcdserver: unset old cluster version in metrics 2019-10-11 22:25:03 -07:00
36696fe2cc Add version, tag and branch checks to release script 2019-10-11 15:47:15 -07:00
0824b08cab Merge pull request #11237 from YoyinZyc/prevent-darwin-build
scripts: avoid release builds on darwin machine.
2019-10-11 10:39:14 -07:00
99731be14a Merge pull request #11242 from jpbetz/changelog-3316-bad
CHANGELOG: Note that 3.3.16 is a bad release and add 3.3.17 release notes
2019-10-11 10:38:24 -07:00
53718bbf33 CHANGELOG: Note that 3.3.16 is a bad release and add 3.3.17 release notes 2019-10-11 10:20:43 -07:00
b9358a0bc8 scripts: avoid release builds on darwin machine. 2019-10-11 09:56:48 -07:00
b7bf33bf5d Merge pull request #11234 from YoyinZyc/release-script
Scripts: fix read failure prompt in release
2019-10-10 16:01:22 -07:00
55efdfe364 scripts: fix read failure prompt in release; use https for git clone. 2019-10-10 13:23:36 -07:00
c60618234b Merge pull request #11228 from heroku/ffz/BumpLogrus
go.mod: Bump Logrus and x/crypto
2019-10-10 11:08:02 -07:00
efc154b095 Merge pull request #11231 from swapnilgm/patch-1
Documenration: Add gardener/etcd-backup-restore to the tools list.
2019-10-10 10:36:00 -07:00
e5aecf8678 Documentation: Add gardener/etcd-backup-restore to the tools list
Etcd-backup-restore is collection of components to backup and restore the etcd. It features the periodic full and incremental backups, automated restore, Validation of etcd data directory with multi cloud provider support.
2019-10-10 21:18:41 +05:30
5c883280df Merge pull request #4 from etcd-io/master
Rebase master
2019-10-09 16:22:54 -07:00
9efa0448e2 go.mod: bump x/crypto because of [CVE-2019-11840]
Related to https://github.com/sirupsen/logrus/issues/1035
2019-10-09 16:03:31 -07:00
9830872d31 go.mod: bump the version of logrus
Related to https://github.com/sirupsen/logrus/issues/1035
2019-10-09 16:03:23 -07:00
a6a1fdd457 Merge pull request #11224 from YoyinZyc/update-changelog
CHANGELOG: update #11179 in changelog-3.4
2019-10-09 13:27:28 -07:00
fe2ada4b45 CHANGELOG: update #11179 in changelog-3.4 2019-10-09 13:06:11 -07:00
7f20ab8369 Merge pull request #3 from etcd-io/master
Rebase
2019-10-09 12:50:27 -07:00
f197ab4578 Merge pull request #11219 from andyliuliming/changelog_for_cherry_pick
changelog: for the skip client san verification option.
2019-10-09 09:41:49 -07:00
1e1ce6b651 changelog for the skip client san verification option. 2019-10-09 13:28:23 +08:00
56bfdab2f0 Merge pull request #11212 from YoyinZyc/tls-san-testing
Add tests for certs with dns names.
2019-10-08 16:41:30 -07:00
6aae90325c Merge pull request #11211 from jpbetz/ipv6-endpoints
Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints
2019-10-08 15:15:16 -07:00
594354b886 Replace endpoint.ParseHostPort with net.SplitHostPort to fix IPv6 client endpoints 2019-10-08 14:53:49 -07:00
eb4b800fda tests: updated ubuntu version in Dockerfile 2019-10-08 14:30:15 -07:00
79bfc90f2b tests/docker-dns: adding tests for certs with dns names. 2019-10-08 14:23:10 -07:00
340f0ac797 Merge pull request #11179 from YoyinZyc/trace
Add tracing to range request in etcd server.
2019-10-08 13:23:53 -07:00
2263930462 Merge pull request #2 from etcd-io/master
Rebase master
2019-10-08 12:52:05 -07:00
57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction()
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
2019-10-07 09:55:27 -07:00
555eb1951f Merge pull request #11192 from i-sevostyanov/master
Removed duplicate check in confchange.Simple
2019-10-04 08:25:35 -04:00
a0d24279ee Merge pull request #11202 from jingyih/update_changelog
CHANGELOG: update from #11194
2019-10-03 16:03:47 -07:00
2bf8e03c73 CHANGELOG: update from #11194 2019-10-03 14:51:16 -07:00
c2f23094c8 Merge pull request #11190 from gyuho/grpc
vendor: upgrade gRPC Go to v1.24.0
2019-10-03 10:26:48 -07:00
6e74099b6f Merge pull request #11184 from jpbetz/correct-authority-checks
clientv3: Set authority used in cert checks to host of endpoint
2019-10-03 10:21:55 -07:00
81a6b4ec9a Merge pull request #11194 from jingyih/fix_etcdctl_memberAdd
etcdctl: fix member add command
2019-10-03 09:04:06 -07:00
a5abf91771 etcdctl: fix member add command 2019-10-02 17:43:32 -07:00
3a3eb24c69 etcdserver: trace raft requests. 2019-10-01 15:38:52 -07:00
401df4bb8e etcdserver: add put request steps.
mvcc: add put request steps; add trace to KV.Write() as input parameter.
2019-10-01 14:08:06 -07:00
d487b16de1 confchange: removed duplicate check in confchange.Simple. 2019-10-01 11:32:12 +03:00
236ac2a905 Merge pull request #11175 from spzala/governance
*: create project governance
2019-09-30 13:41:07 -07:00
1d6ef8370e pkg: use zap logger to format the structure log output. 2019-09-30 13:11:21 -07:00
3830b3ef11 pkg: add field to record additional detail of trace; add stepThreshold
to reduce log volume.
2019-09-30 13:11:21 -07:00
f4e7fc56a7 pkg: create package traceutil for tracing. mvcc: add tracing
steps:range from the in-memory index tree; range from boltdb.
etcdserver: add tracing steps: agreement among raft nodes before
linerized reading; authentication; filter and sort kv pairs; assemble
the response.
2019-09-30 13:06:02 -07:00
036cb96fd1 *: create project governance
Create project governance.
2019-09-30 15:24:14 -04:00
426463c433 vendor: upgrade gRPC Go to v1.24.0
Picking up some performance improvements and bug fixes.

https://github.com/grpc/grpc-go/releases/tag/v1.24.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-30 10:43:02 -07:00
3712a5d045 CHANGELOG-3.4: update
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-27 23:38:49 -07:00
fce0e23d1b *: use Go 1.13.1
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-27 23:37:11 -07:00
46f3e8349b scripts/build-binary: fix darwin tar commands
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-27 23:35:34 -07:00
97388ce454 clientv3: Set authority used in cert checks to host of endpoint 2019-09-25 15:14:23 -07:00
efd1fc634b Merge pull request #11181 from jingyih/fix_doc
doc: clarify metrics flag
2019-09-25 10:47:12 -07:00
20acacdea5 doc: clarify metrics flag 2019-09-24 15:27:46 -07:00
06ec6c4b66 *: create project governance
Create project governance.
2019-09-24 12:50:21 -04:00
6075b9def1 Merge pull request #11174 from anakaiti/patch-1
README: fix formatting on hangouts link
2019-09-23 08:58:12 -07:00
7cb2bb67bf README: fix formatting on hangouts link
Added link and removed wrongly copied text
2019-09-23 22:19:37 +07:00
4f4777182b Merge pull request #11172 from spzala/slack
*: add slack contact
2019-09-22 23:05:36 -07:00
4681061b1f *: add slack contact
Add slack chat contact.
2019-09-22 16:12:31 -04:00
129138760c Update README.md with 9/19/2019 meeting recording 2019-09-19 11:55:43 -07:00
cbbaf2b148 Merge pull request #11167 from lsytj0413/fix-unknown-field
test(functional): remove unknown field Etcd.Debug
2019-09-18 19:06:30 -07:00
2c95b49b63 test(functional): remove unknown field Etcd.Debug 2019-09-19 09:51:51 +08:00
e84029cc04 Merge pull request #11162 from yaojingguo/remove-cancel
clientv3: remove the redundant CancelFunc invocation
2019-09-18 12:39:49 -07:00
9088d07062 Merge pull request #11164 from spzala/codeofconduct
*: update project code of conduct
2019-09-18 14:39:54 -04:00
5370570fec *: update project code of conduct
Update the code of conduct. Remove notice file.
2019-09-18 14:09:18 -04:00
e24564224a clientv3: remove the redundant CancelFunc invocation 2019-09-18 23:06:49 +08:00
501bb07dd3 Merge pull request #11161 from yaojingguo/remove-slash
clientv3/concurrency: remove the unneeded slash
2019-09-18 18:54:01 +08:00
e53298afb1 clientv3/concurrency: remove the unneeded slash
Since NewMutex will append a slash to pfx, there is no need to append a
slash beforehand.
2019-09-18 16:37:31 +08:00
838315283d scripts/release: fix SHA256SUMS command
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-17 14:12:50 -07:00
c327120ba0 scripts/release: fix docker push command
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-17 13:52:22 -07:00
2530c900fd CHANGELOG: update with patch release
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-17 13:29:30 -07:00
a546864790 Merge pull request #11153 from beautytiger/dev-190916
integration:  fix bug in for loop, make it break properly
2019-09-17 10:53:36 -07:00
39b4d147e1 Merge pull request #11151 from vimalk78/integration-fixes
travis: re-enable bom tests
2019-09-17 08:47:12 -07:00
6287052bd0 integration: fix bug in for loop, make it break properly 2019-09-17 11:27:31 +08:00
9bb9a8819c Merge pull request #11152 from spzala/cherrypick
hack: fix cherrypick instruction
2019-09-16 12:12:26 -07:00
78fb1e34f8 hack: fix cherrypick instruction
Remotes is not a valid git command. Also, set the environmental variable
correctly.
2019-09-15 17:48:04 -04:00
e4cb346c3a travis: re-enable bom tests
bill-of-materials was fixed for module aware 'go list' as part of https://github.com/coreos/license-bill-of-materials/pull/17
So can re enable bom tests

fixes #11132
2019-09-14 13:57:55 -07:00
589ab747f7 Merge pull request #11014 from dbavatar/peervalidation
etcdserver: Fix PeerURL validation
2019-09-13 17:42:39 -07:00
0dd10cf6b8 etcdserver: Fix PeerURL validation
In case of URLs that are synonyms, the current lexicographic sorting
and compare of the URLs fails with frustrating errors. Make sure to do
a full comparison between every set of PeerURLs before failing.

Fixes #11013
2019-09-13 17:53:40 -04:00
4de594a3fe Merge pull request #11149 from spzala/embedzaplogger
CHANGELOG: update 3.4.1 and 3.5
2019-09-13 14:07:53 -07:00
b4be60515b CHANGELOG: update 3.4.1 and 3.5
Update from #11147 and #11148
2019-09-13 15:51:46 -04:00
bf3d1fb484 Merge pull request #11147 from ChrisRx/revert-embed-logger
embed: expose ZapLoggerBuilder
2019-09-13 09:31:05 -07:00
e8660c0cec embed: expose ZapLoggerBuilder
This exposes the ZapLoggerBuilder in the embed.Config to allow for
custom loggers to be defined and used by embedded etcd.

Fixes #11144
2019-09-13 07:46:01 -04:00
1e7e66727e Merge pull request #11146 from vimalk78/clientv3-fixes
CHANGELOG : Update CHANGELOG-3.5.md for Mutex.TryLock
2019-09-12 21:13:01 -07:00
82e3579250 *: Update CHANGELOG-3.5.md for Mutex.TryLock
Added https://github.com/etcd-io/etcd/pull/11104 to CHANGELOG for 3.5
2019-09-12 14:34:26 -07:00
79b15a9c46 Merge pull request #11104 from vimalk78/clientv3-fixes
clientv3/concurrency: Added Mutex.TryLock()
2019-09-12 12:23:29 -07:00
2822da8659 CHANGELOG: update gRPC Dependency changes
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-11 14:56:18 -07:00
cee9a84c82 Merge pull request #11140 from gyuho/grpc
vendor: upgrade to gRPC v1.23.1
2019-09-11 14:16:07 -07:00
2ed05bfc33 vendor: upgrade to gRPC v1.23.1
https://github.com/grpc/grpc-go/releases/tag/v1.23.1

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-11 12:25:49 -07:00
62822c52c8 Merge pull request #11138 from spzala/lock10840
ETCDCTL_README: clarify the usage of ETCDCTL_* variables
2019-09-10 15:31:26 -04:00
c9d00ce130 README: require 1.13
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-10 10:43:06 -07:00
04ddfa8b8d clientv3/concurrency: Added Mutex.TryLock()
TryLock locks the mutex if not already locked by another session.
If lock is held by another session, return immediately after attempting necessary cleanup

Added integration test

Fixes #10493
2019-09-09 20:16:07 -07:00
55b37261bc ETCDCTL_README: clarify the usage of ETCDCTL_* variables
fixes # 10840
2019-09-09 16:31:32 -04:00
594005d7de Merge pull request #11136 from jingyih/update_changelog
CHANGELOG: update to add new metrics
2019-09-06 23:06:25 -07:00
93d97bedde CHANGELOG: update to add new metrics 2019-09-06 17:27:58 -07:00
dc56dfc68f Merge pull request #11109 from vimalk78/tools-fix
tools/etcd-dump-logs: Fixed default values for -entry-type flag
2019-09-06 13:48:33 -07:00
00171cc17d CHANGELOG: update 3.4 patch release, 3.5
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 13:25:31 -07:00
d2c2130d72 Merge pull request #11110 from gyuho/go
*: update test Go version / release version
2019-09-06 13:16:44 -07:00
ef0eec1b04 CHANGELOG: update Go version, 3.4 patch release
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 13:06:42 -07:00
92317650ec travis: skipping bom tests for now
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 12:58:43 -07:00
d4d57c5422 test: skip govet shadow tests for now
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 12:51:10 -07:00
88d998be4c Merge pull request #10975 from lzhfromustc/currentRev
Add critical section to protect s.currentRev
2019-09-06 12:27:53 -07:00
e98479ddfe tools/etcd-dump-logs: Fixed test cases
Fixed failing test cases due to changes in previous commit
2019-09-06 20:11:41 +01:00
4e551d06ed tools/etcd-dump-logs: Fixed default values for -entry-type flag
The tool takes default values but it was not visible which default
values were taken. Added default values in proper place, and added a
newline at the end of output.
2019-09-06 20:11:41 +01:00
077dd74827 mvcc: add a TODO in (*store).Hash, to warn against potential data race of currentRev and suggest feasible fix 2019-09-06 15:05:43 -04:00
2cab6edf94 go.mod: use Go 1.13
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 11:39:16 -07:00
1dff148f69 scripts/updatedep: use Go 1.13
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 11:39:03 -07:00
a3f7202c5f *: use TLS.Config.MaxVersion to TLS 1.2
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-06 11:38:03 -07:00
42ad719155 *: update test Go version / release version
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-05 21:46:56 -07:00
64b3e59766 Merge pull request #11129 from vimalk78/integration-fixes
integration : fix TestTLSClientCipherSuitesMismatch in go1.13
2019-09-05 21:46:43 -07:00
130cf96c98 Merge pull request #11128 from jingyih/update_changelog
CHANGELOG-3.5: update from #11126
2019-09-05 20:15:42 -04:00
2648a36f8c integration : fix TestTLSClientCipherSuitesMismatch in go1.13
In go1.13, the TLS13 is enablled by default, and as per go1.13 release notes :
TLS 1.3 cipher suites are not configurable. All supported cipher suites are safe,
and if PreferServerCipherSuites is set in Config the preference order is based
on the available hardware.

Fixing the test case for go1.13 by limiting the TLS version to TLS12
2019-09-06 01:12:32 +01:00
c40fc778ce CHANGELOG: update changelog 3.5
Add etcd_debugging_mvcc_current_revision and
etcd_debugging_mvcc_compact_revision.
2019-09-05 16:45:11 -07:00
fc8f29bc40 Merge pull request #11126 from jingyih/add_metric_store_revision
mvcc: add store revision metrics
2019-09-05 16:41:59 -07:00
0f8c46a0f3 mvcc: add store revision metrics
Add experimental metrics etcd_debugging_mvcc_current_revision and
etcd_debugging_mvcc_compact_revision.
2019-09-05 15:40:06 -07:00
e2035ba49a Merge pull request #11113 from vimalk78/learner-fixes
. : added Procfile.learner
2019-09-05 12:52:36 -07:00
53fd227f5c . : added Procfile.learner
Added Procfile.learner to easily start cluster with learner
Modified README.md to highlight its usage
Update review comments
2019-09-05 17:18:48 +01:00
cbfaecdb85 Merge pull request #11117 from vimalk78/wal-fixes
wal : wal.Verify defer close the opened WAL files
2019-09-04 23:25:29 -07:00
01a79d4a9d Merge pull request #11118 from zhangjianweibj/re-fix-percent-bug
etcdserver: remove dup percentage sign in log
2019-09-04 22:02:56 -07:00
81a34ab6d5 etcdserver: remove dup percentage sign in log 2019-09-05 11:24:39 +08:00
adbae79565 wal : wal.Verify defer close the opened WAL files
wal.Verify() : The opened WAL files are not closed in error cases. Fixed by adding a
defer.
2019-09-04 23:38:14 +01:00
fb41ebea52 Merge pull request #11108 from gyuho/fix-zap
embed: fix secure server logging message
2019-09-03 09:42:54 -07:00
f3d67acf3d Merge pull request #11103 from mrueg/shellcheck
scripts/release: Apply shellcheck findings
2019-09-03 09:42:13 -07:00
ea0f08de20 embed: fix secure server logging message
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-09-03 08:56:32 -07:00
d981fe6b6b scripts/release: Apply shellcheck findings
I run https://github.com/koalaman/shellcheck/ over scripts/* and fixed
the findings it returned.

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2019-09-02 17:50:04 +02:00
14624b7d64 Merge pull request #11097 from philips/sha256sums
scripts/release: rename SHA256SUM to SHA256SUMS
2019-08-30 13:34:50 -07:00
f108a32393 scripts/release: rename SHA256SUM to SHA256SUMS
These files are commonly called SHA256SUMS and with this change rget
works for v3.4.0 as well.
2019-08-30 13:31:17 -07:00
93ae5d2f5b Merge pull request #11095 from KeepCaim/master
Documentation:fix clerical error
2019-08-30 09:54:38 -04:00
6795d20ff0 Documentation:fix clerical error 2019-08-30 13:20:22 +08:00
83414f3db0 CHANGELOG-3.4: update "golang.org/x/sys" vendor
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-29 14:46:26 -07:00
876df8d123 Merge pull request #10834 from carlosedp/riscv64
vendor: update x/sys and x/net modules to support Risc-V
2019-08-29 14:03:32 -07:00
7c6ef608bd vendor: x/sys and x/net to support building on Risc-V
Signed-off-by: Carlos de Paula <me@carlosedp.com>
2019-08-29 17:08:41 -03:00
4d210173ae CHANGELOG-3.4: add v3.4.0 for tomorrow
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-29 12:07:24 -07:00
9214f4e4ba CHANGELOG: v3.4.0-rc.4
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-29 09:40:05 -07:00
837fe89d0b scripts/release: fix sha256sum
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-29 09:39:18 -07:00
5b991ab51c Merge pull request #11093 from hexfusion/fx_cl_4.3
CHANGELOG-3.4: add sha256sum
2019-08-29 08:39:08 -07:00
431756ab59 CHANGELOG-3.4: add sha256sum
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-08-29 13:00:13 +00:00
af840f0666 Merge pull request #11087 from hexfusion/add_sum
scripts/release: add sha256sum summary of release assets
2019-08-28 23:11:02 -07:00
2386b945c6 scripts/release: add sha256sum summary of release assets
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
2019-08-28 15:56:39 +00:00
7b4ae58c41 Merge pull request #11076 from vimalk78/docs-fixes
Documentation: Add section headers to learning/design-learner.md
2019-08-27 10:18:07 -07:00
9164d27aaa CHANGELOG-3.4: add v3.4.0-rc.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-27 10:11:29 -07:00
9da9221d3f Merge pull request #11081 from gyuho/zap
*: fix zap logger --log-outputs without "stderr"
2019-08-27 09:49:42 -07:00
f22c7c80c1 embed: fix "--log-outputs" setup without "stderr"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-26 22:45:48 -07:00
6ee0c5c5dd pkg/logutil: change to "MergeOutputPaths"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-26 22:45:36 -07:00
7e9a26eb6d Merge pull request #11073 from Wine93/typo
raft: fixed some typos
2019-08-26 11:54:48 -07:00
021adfea21 Merge pull request #11077 from wesraph/fixMissingOptions
etcdserver: add check for nil options
2019-08-26 10:47:30 -07:00
ade5337b92 etcdserver: add check for nil options 2019-08-26 17:43:05 +07:00
f09b09519b Documentation: Add section headers to etcd Learner
In the Background section, the document describes various challenges for cluster membership change.
Added section header for each case described for better readability.
2019-08-25 09:42:33 +01:00
b0534c1b44 raft/log_test: fixed wrong index 2019-08-25 04:47:11 +00:00
5f42161750 raft: fixed some typos and simplify minor logic 2019-08-25 04:46:29 +00:00
6f0ae78b39 Merge pull request #11074 from vimalk78/docs-fixes
Documentation: snapshot can be requested from one etcd node only
2019-08-24 15:27:53 -04:00
751071e867 Documentation: snapshot can be requested from one etcd node only
Updated Snapshot section of demo.md to reflect that snapsot can be requested only from one etcd node at a time.

Fixes : #10855
2019-08-24 18:16:58 +01:00
dd2b88d0ed Merge pull request #11066 from vimalk78/clientv3-fixes
clientv3: add nil checks in Client.Close()
2019-08-23 23:22:42 -07:00
928abd5647 CHANGELOG-3.4: update v3.4.0-rc.2
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-23 01:15:45 -07:00
f4dfd1976d Merge pull request #10971 from nilsocket/codeReformat
raft : write compact if statements
2019-08-23 09:10:26 +08:00
702c69c906 raft : Write compact if statements 2019-08-23 04:31:40 +05:30
c7982425d9 Update 8/22 community meeting video
Update 8/22 community meeting video
2019-08-22 12:53:12 -07:00
522ff40705 Update community meeting info in README.md
Update community meeting info in README.md
2019-08-22 10:46:34 -07:00
56f483ae44 Merge pull request #11069 from jingyih/fix_TestKVPutError
integration: fix TestKVPutError
2019-08-22 10:09:11 -04:00
2680c2afe7 integration: fix TestKVPutError
Give backend quota enough overhead.
2019-08-21 19:56:38 -07:00
7f47de8414 clientv3: add nil checks in Close()
Added nil checks in Close() for Watcher and Lease fields
Added test case
2019-08-21 23:04:42 +01:00
8037e6e087 Merge pull request #11060 from etcd-io/wenjiaswe-patch-1
functional test: fix typo in agent log
2019-08-20 15:22:08 -07:00
341380d84f functional test: fix typo in agent log
Fix typo in functional test agent log to avoid debugging confusion.
2019-08-20 15:04:30 -07:00
b7abc9bf0a Merge pull request #11059 from jingyih/update_changelog_from_PR11056
CHANGELOG-3.3: update from #11056
2019-08-20 11:27:30 -07:00
867d93636a CHANGELOG-3.3: update_from_PR11056 2019-08-20 11:18:14 -07:00
c89777dd70 Merge pull request #11047 from ethan-daocloud/patch-4
lease: cleanup typo words in lessor.go
2019-08-19 19:09:22 -07:00
0a6173c9b6 lease: cleanup typo words in lessor.go
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-20 09:35:19 +08:00
62d565c419 CHANGELOG-3.3: release v3.3.15
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-19 12:08:28 -07:00
43fe8c3ee0 CHANGELOG-3.4: fix casing
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-16 16:50:46 -07:00
8b653bad4a CHANGELOG: release v3.3.14
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-16 16:28:57 -07:00
f4bb43887f Documentation/upgrades: special upgrade guides for >= 3.3.14
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-16 16:18:53 -07:00
fd1181d8b6 Merge pull request #11045 from jingyih/update_etcdmain_helper
etcdmain: update help message
2019-08-16 13:09:35 -07:00
4a6bf3552e CHANGELOG-3.4: log raft prober change
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-16 09:02:47 -07:00
4a2b4c8f7e Merge pull request #11037 from tbg/interactive
raft: proactively probe newly added followers
2019-08-16 10:24:42 +02:00
47ae53d25d rafttest: print Ready before processing it
It was confusing to see the effects of the Ready (i.e. log messages)
printed before the Ready itself.
2019-08-16 09:41:35 +02:00
99f8046fd1 raft: fix a test file name 2019-08-16 09:38:44 +02:00
8d1946d16a raft: document problem with leader self-removal
When a leader removes itself, it will retain its leadership but not
accept new proposals, making the range effectively stuck until manual
intervention triggers a campaign event.

This commit documents the behavior. It does not correct it yet.
2019-08-16 09:38:44 +02:00
306e75a96f raft: add a batch of interaction-driven conf change tests
Verifiy the behavior in various v1 and v2 conf change operations.
This also includes various fixups, notably it adds protection
against transitioning in and out of new configs when this is not
permissible.

There are more threads to pull, but those are left for future commits.
2019-08-16 09:38:44 +02:00
871c2d716f etcdmain: update help message
Add experimental-peer-skip-client-san-verification flag description to
help message. Add default values.
2019-08-15 21:22:17 -07:00
9b29151d30 Merge pull request #11044 from gtamas/master
Documentation: added ETCD Manager tool
2019-08-15 16:25:22 -07:00
9d3468d7be CHANGELOG-3.3: highlight v3.3.14-rc.0 release
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 15:07:03 -07:00
c337cd9a57 CHANGELOG: update Go version
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 14:48:54 -07:00
f6e3f94f3e Documentation: added ETCD Manager tool
New tool: ETCD Manager

ETCD Manager is a multi-platform ETCD v3 client. Currently, builds are available form Mac, Wiindows and Linux, but iOS / Android builds will also be added in the future.
It aims to be a modern, efficient and easy to use GUI with full coverage of ETCD APIs / functionality. The first public (beta) release is already available.
2019-08-15 23:42:24 +02:00
966321bdcd *: use Go 1.12.9
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 14:28:41 -07:00
577d5305f8 CHANGELOG: update new compaction limit change
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 14:03:21 -07:00
c55410ccea Merge pull request #11034 from jpbetz/force-commit-compact
mvcc: Optimize compaction for short commit pauses
2019-08-15 13:28:32 -07:00
f279bfb775 mvcc: keep 64-bit alignment in "store" struct
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 13:01:32 -07:00
b5aa46486b clientv3/integration: fix "mvcc.NewStore" call
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-15 11:48:53 -07:00
9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
d57bc6e724 mvcc: Optimize compaction for short commit pauses 2019-08-15 11:47:23 -07:00
3c31fbbce3 Merge pull request #11040 from ethan-daocloud/patch-3
cleanup: fix typos in doc/comment
2019-08-15 09:33:48 -07:00
110808ebdd Merge pull request #11024 from xiang90/maintainers
MAINTAINERS: add Tobias Grieger as a maintainer to raft
2019-08-15 09:25:48 -07:00
9365660ca8 cleanup: fix typos in doc/comment
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-15 21:29:32 +08:00
c7b106896f CHANGELOG: highlight breaking changes in clientv3 resolver
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 17:00:53 -07:00
508203db15 Documentation/upgrades: fix 3.4 guides
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 17:00:46 -07:00
bbb379fa4a Merge pull request #10974 from lzhfromustc/watchers
proxy: Add critical section to protect wps.watchers and wps.nextWatcherID
2019-08-14 16:14:27 -07:00
02ed271527 CHANGELOG-3.3: add v3.3.14-beta.0
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 12:33:23 -07:00
4e19150676 raft: proactively probe newly added followers
When the leader applied a new configuration that added voters, it would
not immediately probe these voters, delaying when they would be caught
up.

I noticed this while writing an interaction-driven test, which has now
been cleaned up and completed.
2019-08-14 20:53:34 +02:00
3d6721f751 rafttest: add _breakpoint directive
It is a helper case to attach a debugger to when a problem needs
to be investigated in a longer test file. In such a case, add the
following stanza immediately before the interesting behavior starts:

_breakpoint:
----
ok

and set a breakpoint on the _breakpoint case.
2019-08-14 20:53:34 +02:00
fdaed88f14 raft: initialize new Progress at LastIndex, not LastIndex+1
Initializing at LastIndex+1 meant that new peers would not be probed
immediately when they appeared in the leader's config, which delays
their getting caught up.
2019-08-14 20:53:34 +02:00
c2d9514370 raft/rafttest: fix stabilize handler
It was bailing out too early.
2019-08-14 17:24:14 +02:00
d047fe84ee CHANGELOG-3.3: highlight 3.3 backports
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 01:44:01 -07:00
916e6b5400 CHANGELOG-3.3: add more changes
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 01:35:02 -07:00
9aa27c303e CHANGELOG: update backports
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-14 01:10:49 -07:00
fd0d43dad9 scripts/release: clean up minor tag docker commands
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 22:00:07 -07:00
08f2685330 CHANGELOG: update 3.3 + 3.4 changes
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 21:46:09 -07:00
5535092ce3 Makefile: explicit about GOOS in docker-test builds
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 16:56:52 -07:00
1ea2acd9b3 Merge pull request #11032 from jingyih/changelog_add_mvcc_db_open_read_transactions
CHANGELOG-3.4: add etcd_mvcc_db_open_read_transactions
2019-08-13 15:50:08 -07:00
a6c792dfce CHANGELOG: add etcd_mvcc_db_open_read_transactions 2019-08-13 15:34:35 -07:00
96ce27cf5d Merge pull request #11031 from gyuho/ccc
*: highlight "--enable-v2=false"
2019-08-13 15:32:21 -07:00
092ee6d321 Documentation/upgrades: highlight "--enable-v2=false"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 15:23:58 -07:00
fb7981b6e5 CHANGELOG: --enable-v2=false by default
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 15:14:16 -07:00
170861c5c9 Merge pull request #11029 from gyuho/grpc
vendor: upgrade grpc-go to 1.23.0
2019-08-13 14:44:07 -07:00
daa104093f Merge pull request #11030 from jingyih/update_changelog_from_pr10523
CHANGELOG-3.4: update from 10523
2019-08-13 14:28:19 -07:00
578ff0d416 CHANGELOG-3.4: update from 10523 2019-08-13 14:18:00 -07:00
4e3c2e0a3a vendor: update "net/http2" to latest
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 13:35:50 -07:00
02b2779814 vendor: upgrade grpc-go to 1.23.0
https://github.com/grpc/grpc-go/releases/tag/v1.23.0

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 12:57:14 -07:00
c0161f7e7d *: use Go 1.12.8
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-13 12:45:12 -07:00
fdbaf3ea9c Merge pull request #11025 from gyuho/ccc
clientv3: use Endpoints(), fix context creation
2019-08-13 11:17:43 -07:00
97fd40c587 clientv3: use Endpoints(), fix context creation
If overwritten, the previous context should be canceled first.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 21:39:50 -07:00
c6e4076353 MAINTAINERS: add tbg 2019-08-12 17:16:36 -07:00
8d823e70dd Merge pull request #11018 from gyuho/yq
scripts/release: update "yq" command
2019-08-12 11:03:37 -07:00
dd0ed3f05e CHANGELOG-3.4: add "code changes" from 3.3
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:23:33 -07:00
c72fa51b2a scripts: remove ".aci" commands
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:21 -07:00
af38185f92 scripts/release: fix version check commands
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:13 -07:00
ac613c481f scripts: fix build docker commands, add more logging
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 10:20:05 -07:00
7299a6a106 *: remove "acbuild"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 09:31:22 -07:00
1f4f8ae731 scripts/release: update "yq" command
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-12 09:26:04 -07:00
65e226ad92 Merge pull request #11017 from ethan-daocloud/patch-2
cleanup: correct summary message in put.go
2019-08-12 10:20:51 -04:00
b5ee1de076 cleanup: correct summary message in put.go 2019-08-12 21:13:58 +08:00
029401ab81 Merge pull request #11005 from tbg/interactiontest
raft/rafttest: introduce datadriven testing
2019-08-12 11:52:52 +02:00
e8090e57a2 raft/rafttest: introduce datadriven testing
It has often been tedious to test the interactions between multi-member
Raft groups, especially when many steps were required to reach a certain
scenario. Often, this boilerplate was as boring as it is hard to write
and hard to maintain, making it attractive to resort to shortcuts
whenever possible, which in turn tended to undercut how meaningful and
maintainable the tests ended up being - that is, if the tests were even
written, which sometimes they weren't.

This change introduces a datadriven framework specifically for testing
deterministically the interaction between multiple members of a raft group
with the goal of reducing the friction for writing these tests to near
zero.

In the near term, this will be used to add thorough testing for joint
consensus (which is already available today, but wildly undertested),
but just converting an existing test into this framework has shown that
the concise representation and built-in inspection of log messages
highlights unexpected behavior much more readily than the previous unit
tests did (the test in question is `snapshot_succeed_via_app_resp`; the
reader is invited to compare the old and new version of it).

The main building block is `InteractionEnv`, which holds on to the state
of the whole system and exposes various relevant methods for
manipulating it, including but not limited to adding nodes, delivering
and dropping messages, and proposing configuration changes. All of this
is extensible so that in the future I hope to use it to explore the
phenomena discussed in

https://github.com/etcd-io/etcd/issues/7625#issuecomment-488798263

which requires injecting appropriate "crash points" in the Ready
handling loop. Discussions of the "what if X happened in state Y"
can quickly be made concrete by "scripting up an interaction test".

Additionally, this framework is intentionally not kept internal to the
raft package.. Though this is in its infancy, a goal is that it should
be possible for a suite of interaction tests to allow applications to
validate that their Storage implementation behaves accordingly, simply
by running a raft-provided interaction suite against their Storage.
2019-08-12 11:13:51 +02:00
56ad881b1b Merge pull request #11015 from gyuho/typo
raft: fix typo
2019-08-10 17:26:29 +02:00
8e2225b4f1 Merge pull request #11016 from ethan-daocloud/patch-1
etcd-dump-logs: correct logging message word
2019-08-10 04:56:50 -07:00
867b31e01a etcd-dump-logs: correct logging message word
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-10 17:01:57 +08:00
6c87b21821 raft: fix typo
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-09 21:26:48 -07:00
f57c16c271 vendor: bump datadriven
Picks up some fixes for papercuts.
2019-08-10 00:02:59 +02:00
4a4629fd9f Merge pull request #10957 from Hanaasagi/fix-metric-name-typo
test: fix metric name typo
2019-08-09 13:23:26 -07:00
5e90267d1b CHANGELOG: update 3.3 + 3.4 with raft changes
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-09 11:32:10 -07:00
4cec8dddc6 Merge pull request #11003 from tbg/interaction/restore
raft: fix restoring joint configurations
2019-08-09 20:13:04 +02:00
84c69cca76 Merge pull request #10970 from nilsocket/minorFix1
raft : remove unnecessary, if check
2019-08-09 11:03:01 -07:00
37ab5bdd21 raft: fix restoring joint configurations
While writing interaction tests for joint configuration changes, I
realized that this wasn't working yet - restoring had no notion of
the joint configuration and was simply dropping it on the floor.

This commit introduces a helper `confchange.Restore` which takes
a `ConfState` and initializes a `Tracker` from it.

This is then used both in `(*raft).restore` as well as in `newRaft`.
2019-08-09 19:28:43 +02:00
a5f785a232 confchange: clean up unnecessary block 2019-08-09 19:28:43 +02:00
7948f39790 Merge pull request #11004 from tbg/interaction/unused-type
raft/tracker: visit Progress in stable order
2019-08-09 12:32:04 +02:00
5ce1856cce Merge pull request #11010 from etcd-io/wenjiaswe-patch-1
functional: Update functional test README.md
2019-08-08 20:57:26 -07:00
ab9e3d9829 functional: Update functional test README.md 2019-08-08 18:40:15 -07:00
0b2b25e1c1 CHANGELOG: update metrics
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 15:08:10 -07:00
046c705f97 Merge pull request #11009 from gyuho/snapshot
*: add inflight snapshot metrics
2019-08-08 13:56:14 -07:00
06b82c200f etcdserver: add "etcd_server_snapshot_apply_inflights_total"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:52 -07:00
a4badc33a3 integration: test snapshot inflights metrics
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:52 -07:00
46bddacacb etcdserver/api: add "etcd_network_snapshot_send_inflights_total", "etcd_network_snapshot_receive_inflights_total"
Useful for deciding when to terminate the unhealthy follower.
If the follower is receiving a leader snapshot, operator may wait.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 13:33:48 -07:00
43ce2eefaa Merge pull request #10995 from yuzeming/patch-3
agent: fix data race and deadlock
2019-08-08 12:22:20 -07:00
c762a3d7f7 agent: fix a data race and deadlock
add 1-size buffer for `errc`  to avoid deadlock of child goroutine
add a local variable to a void data race in `err`
when `case <-stream.Context().Done():` is taken
2019-08-08 11:05:30 -07:00
e745649cce Merge pull request #10960 from spzala/readmesec
README: update security reference
2019-08-08 13:31:33 -04:00
c4b8ec5369 CHANGELOG: update links, raft updates
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 09:20:16 -07:00
bfcd590f05 Merge pull request #11000 from retroflexer/doc-fix-broken-links
doc: Fix broken links referring to readthedocs.io
2019-08-08 09:12:39 -07:00
1c65af7acf Merge pull request #11006 from gyuho/functional
functiona: fix flaky tests
2019-08-08 09:08:23 -07:00
72e00cea3a functional/agent: copy file, instead of renaming
To retain failure logs in CI testing.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 08:12:05 -07:00
d1c7be24b0 functional/rpcpb: make client log less verbose
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 08:06:48 -07:00
0926a434b7 functional.yaml: try lower snapshot count for flaky tests, error threshold
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-08 08:06:48 -07:00
a41bd303ec Merge pull request #10998 from tbg/learners-vote
raft: let learners vote
2019-08-08 11:26:17 +02:00
1b3e0821a7 raft/tracker: visit Progress in stable order
This is helpful for upcoming testing work which allows datadriven
testing of the interaction of multiple nodes. This testing requires
determinism to work correctly.
2019-08-08 09:37:33 +02:00
a0b2c6ad4b proxy/grpcproxy: Add critical section to protect wps.watchers and wps.nextWatcherID 2019-08-07 17:58:49 -07:00
03bd10076f Merge pull request #10955 from lzhfromustc/master
Avoid potential double lock of tsafeSet
2019-08-07 15:50:05 -07:00
0e7173b447 pkg/types: Avoid potential double lock of tsafeSet.
(tsafeSet).Sub and (tsafeSet).Equals can cause double lock bug if ts and other is pointing the same variable

gofmt the code and add some comments
2019-08-07 15:08:00 -07:00
158354755a test: output etcd server logs when functional tests fail
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-07 10:16:15 -07:00
9553994cd7 raft/auorum: remove unused type 2019-08-07 18:53:01 +02:00
742f928c6a Broken link in runtime-configuration.md
See the issue created here:
https://github.com/etcd-io/etcd/issues/10989#issuecomment-518726038

doc: fix broken links referring to etcd.redhatdocs.io

Adding links to internal Documentation within github.com.

Update runtime-configuration.md

Update runtime-configuration.md

Update CHANGELOG-3.3.md

Remove extra space

Keep the formatting similar to original
2019-08-07 10:50:21 -04:00
c30c2e345b raft: let learners vote
It turns out that that learners must be allowed to cast votes.

This seems counter- intuitive but is necessary in the situation in which
a learner has been promoted (i.e. is now a voter) but has not learned
about this yet.

For example, consider a group in which id=1 is a learner and id=2 and
id=3 are voters. A configuration change promoting 1 can be committed on
the quorum `{2,3}` without the config change being appended to the
learner's log. If the leader (say 2) fails, there are de facto two
voters remaining. Only 3 can win an election (due to its log containing
all committed entries), but to do so it will need 1 to vote. But 1
considers itself a learner and will continue to do so until 3 has
stepped up as leader, replicates the conf change to 1, and 1 applies it.

Ultimately, by receiving a request to vote, the learner realizes that
the candidate believes it to be a voter, and that it should act
accordingly. The candidate's config may be stale, too; but in that case
it won't win the election, at least in the absence of the bug discussed
in:
https://github.com/etcd-io/etcd/issues/7625#issuecomment-488798263.
2019-08-07 12:03:18 +02:00
0d85aa1b41 Merge pull request #10993 from yuzeming/patch-1
integration: fix a data race about `err`
2019-08-06 15:58:21 -07:00
88f4b83ba9 mvcc: fix typo in test
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-06 15:09:05 -07:00
a996a8a912 Merge pull request #10990 from gyuho/grpc
vendor: update gRPC to latest
2019-08-06 15:08:11 -07:00
877aa2497e Merge pull request #10994 from yuzeming/patch-2
v3rpc: fix a typo `err`
2019-08-06 15:06:27 -07:00
181419256d integration: fix a data race about err
don't share `err` between goroutines
2019-08-06 14:58:15 -07:00
3edb569ad3 v3rpc: fix a typo err
don't read return value in child goroutine which causes data race.
2019-08-06 14:04:58 -07:00
017b6c424e stream: Prevent panic when newAttemptLocked fails to get a transport for the new attempt
Testing https://github.com/grpc/grpc-go/pull/2958

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-06 13:08:47 -07:00
f5f400b14a vendor: update gRPC to latest
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-06 10:20:02 -07:00
44a00a33ef Merge pull request #10987 from wenjiaswe/functional-test-fix
functional: update go.etcd.io/etcd link and go image registry for func…
2019-08-05 22:45:12 -07:00
f7397d0628 functional:update go.etcd.io/etcd link and go image registry for functional test 2019-08-05 22:19:45 -07:00
bcaaeebc82 Merge pull request #10985 from etcd-io/wenjiaswe-update-functional-readme
functional test: Update functional README.md
2019-08-05 21:45:09 -07:00
0a0b2be8fc functional test: Update functional README.md 2019-08-05 20:51:24 -07:00
a494e0658d Merge pull request #10981 from gyuho/deprecate-grpc-clientconnclosing
Deprecate "grpc.ErrClientConnClosing"
2019-08-05 14:37:44 -07:00
a0cabb57b5 Documentation/upgrades: highlight "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 13:29:45 -07:00
3b71672f84 CHANGELOG-3.4: deprecate "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 13:29:03 -07:00
9b2f18c6fb proxy/grpcproxy: deprecate "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 13:27:16 -07:00
0bd27ea963 functional: deprecate "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 13:27:16 -07:00
9b385737f5 clientv3: deprecate "grpc.ErrClientConnClosing"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 13:27:16 -07:00
4bf584893c functional: use Go 1.12.7 as default
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 12:40:31 -07:00
ac87ebdb02 pkg/adt: remove TODO
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-05 00:24:39 -07:00
4b0af5b4ac clientv3: document "WithBlock" dial option
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:52:17 -07:00
2847367b60 travis: do not allow CPU 4 test failures
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:35:19 -07:00
f7992237ce scripts/release: remove acbuild commands
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:20:41 -07:00
af19d014e1 CHANGELOG-3.4: highlight rbtree fix
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:09:07 -07:00
003362ef8e pkg/adt: fix interval tree black-height property based on rbtree
Author: xkey <xk33430@ly.com>
ref. https://github.com/etcd-io/etcd/pull/10978

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-04 23:08:45 -07:00
9a2af7378a tests/e2e: skip release tests until release candidate
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-03 00:09:44 -07:00
b842cf7770 tests/e2e: fix upgrade, metrics tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:57:45 -07:00
189071dcd4 functional: remove "embed" support in tests
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:56:52 -07:00
7fbbb9c8bf *: add 3.5 capability for 3.5 dev tree
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:27:54 -07:00
3658571e3a etcdserver/api: enable 3.4 capability
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-08-02 15:24:18 -07:00
0d99469cdb raft : newRaft() does check for validity of Config 2019-08-02 03:09:51 +05:30
3ef1683bab test: fix metric name typo 2019-07-31 12:43:46 +09:00
7b26cd37e9 README: update security reference
Now that we have security disclosure and release process in place,
update the related reference.
2019-07-30 22:23:55 -04:00
7aa6358510 etcdserver: remove auth validation loop
Remove auth validation loop in v3_server.raftRequest(). Re-validation
when error ErrAuthOldRevision occurs should be handled on client side.
2019-02-12 15:40:39 -08:00
2518 changed files with 61389 additions and 502701 deletions

View File

@ -1,2 +1,2 @@
Please read https://github.com/etcd-io/etcd/blob/master/Documentation/reporting_bugs.md.
Please read https://etcd.io/docs/latest/reporting_bugs/

View File

@ -0,0 +1,28 @@
---
name: Distributors Application
title: Distributors Application for <YOUR DISTRIBUTION HERE>
about: Apply for membership of security@etcd.io
---
<!--
Please answer the following questions and provide supporting evidence for
meeting the membership criteria.
-->
**Actively monitored security email alias for our project:**
**1. Have a user base not limited to your own organization.**
**2. Have a publicly verifiable track record up to present day of fixing security issues.**
**3. Not be a downstream or rebuild of another distribution.**
**4. Be a participant and active contributor in the community.**
**5. Accept the Embargo Policy.**
<!-- https://github.com/etcd-io/etcd/blob/main/security/security-release-process.md#disclosures -->
**6. Be willing to contribute back.**
<!-- Per https://github.com/etcd-io/etcd/blob/main/security/security-release-process.md#patch-release-and-public-communication -->
**7. Have someone already on the list vouch for the person requesting membership on behalf of your distribution.**

View File

@ -1,2 +1,2 @@
Please read https://github.com/etcd-io/etcd/blob/master/CONTRIBUTING.md#contribution-flow.
Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

2
.github/SECURITY.md vendored
View File

@ -1,2 +1,2 @@
Please read https://github.com/etcd-io/etcd/blob/master/security/README.md.
Please read https://github.com/etcd-io/etcd/blob/main/security/README.md.

60
.github/stale.yml vendored Normal file
View File

@ -0,0 +1,60 @@
# Configuration for probot-stale - https://github.com/probot/stale
# Number of days of inactivity before an Issue or Pull Request becomes stale
daysUntilStale: 90
# Number of days of inactivity before an Issue or Pull Request with the stale label is closed.
# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale.
daysUntilClose: 21
# Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled)
onlyLabels: []
# Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable
exemptLabels:
- "area/security"
- "Investigating"
# Set to true to ignore issues in a project (defaults to false)
exemptProjects: false
# Set to true to ignore issues in a milestone (defaults to false)
exemptMilestones: false
# Set to true to ignore issues with an assignee (defaults to false)
exemptAssignees: false
# Label to use when marking as stale
staleLabel: stale
# Comment to post when marking as stale. Set to `false` to disable
markComment: >
This issue has been automatically marked as stale because it has not had
recent activity. It will be closed after 21 days if no further activity
occurs. Thank you for your contributions.
# Comment to post when removing the stale label.
# unmarkComment: >
# Your comment here.
# Comment to post when closing a stale Issue or Pull Request.
# closeComment: >
# Your comment here.
# Limit the number of actions per hour, from 1-30. Default is 30
limitPerRun: 30
# Limit to only `issues` or `pulls`
# only: issues
# Optionally, specify configuration settings that are specific to just 'issues' or 'pulls':
# pulls:
# daysUntilStale: 30
# markComment: >
# This pull request has been automatically marked as stale because it has not had
# recent activity. It will be closed if no further activity occurs. Thank you
# for your contributions.
# issues:
# exemptLabels:
# - confirmed

View File

@ -0,0 +1,18 @@
name: Publish Release Assets to Asset Transparency Log
on:
release:
types: [published, created, edited, released]
jobs:
github_release_asset_transparency_log_publish_job:
runs-on: ubuntu-latest
name: Publish GitHub release asset digests to https://beta-asset.transparencylog.net
steps:
- name: Gather URLs from GitHub release and publish
id: asset-transparency
uses: transparencylog/github-releases-asset-transparency-verify-action@v11
- name: List verified and published URLs
run: echo "Verified URLs ${{ steps.asset-transparency.outputs.verified }}"
- name: List failed URLs
run: echo "Failed URLs ${{ steps.asset-transparency.outputs.failed }}"

67
.github/workflows/codeql-analysis.yml vendored Normal file
View File

@ -0,0 +1,67 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
on:
push:
branches: [ main, release-0.4, release-2.0, release-2.1, release-2.2, release-2.3, release-3.0, release-3.1 ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
schedule:
- cron: '20 14 * * 5'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
# Learn more:
# https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
steps:
- name: Checkout repository
uses: actions/checkout@v2
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
# Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language
#- run: |
# make bootstrap
# make release
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1

35
.github/workflows/e2e.yaml vendored Normal file
View File

@ -0,0 +1,35 @@
name: E2E
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
target:
- linux-amd64-e2e
- linux-386-e2e
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: "^1.16"
- run: date
- env:
TARGET: ${{ matrix.target }}
run: |
echo "${TARGET}"
case "${TARGET}" in
linux-amd64-e2e)
PASSES='build release e2e' MANUAL_VER=v3.4.7 CPU='4' EXPECT_DEBUG='true' COVER='false' RACE='true' ./test.sh 2>&1 | tee test.log
! egrep "(--- FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test.log
;;
linux-386-e2e)
GOARCH=386 PASSES='build e2e' CPU='4' EXPECT_DEBUG='true' COVER='false' RACE='true' ./test.sh 2>&1 | tee test.log
! egrep "(--- FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test.log
;;
*)
echo "Failed to find target"
exit 1
;;
esac

29
.github/workflows/functional.yaml vendored Normal file
View File

@ -0,0 +1,29 @@
name: functional-tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
target:
- linux-amd64-functional
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: "^1.16"
- run: date
- env:
TARGET: ${{ matrix.target }}
run: |
echo "${TARGET}"
case "${TARGET}" in
linux-amd64-functional)
GO_BUILD_FLAGS='-v -mod=readonly' ./build && GOARCH=amd64 PASSES='functional' ./test
;;
*)
echo "Failed to find target"
exit 1
;;
esac

30
.github/workflows/grpcproxy.yaml vendored Normal file
View File

@ -0,0 +1,30 @@
name: grpcProxy-tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
target:
- linux-amd64-grpcproxy
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: "^1.16"
- run: date
- env:
TARGET: ${{ matrix.target }}
run: |
echo "${TARGET}"
case "${TARGET}" in
linux-amd64-grpcproxy)
PASSES='build grpcproxy' CPU='4' COVER='false' RACE='true' ./test.sh 2>&1 | tee test.log
! egrep "(--- FAIL:|DATA RACE|panic: test timed out|appears to have leaked)" -B50 -A10 test.log
;;
*)
echo "Failed to find target"
exit 1
;;
esac

View File

@ -0,0 +1,19 @@
name: Linux ARM64 Graviton2
on: [push, pull_request]
jobs:
test:
runs-on: [self-hosted, linux, ARM64, graviton2]
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: |
sudo amazon-linux-extras install epel
sudo yum install -y git gcc ShellCheck
- uses: actions/setup-go@v2
with:
go-version: "^1.16"
- run: go version
- run: date
- name: Run tests
run: TEST_OPTS="PASSES='fmt bom dep build unit integration_e2e'" make test

View File

@ -0,0 +1,142 @@
## Graviton-based self-hosted github action worker
### Step 1. Create an EC2 instance with Graviton
Create an AWS Graviton-based EC2 instance. For example,
```
# or download from https://github.com/aws/aws-k8s-tester/releases
cd ${HOME}/go/src/github.com/aws/aws-k8s-tester
go install -v ./cmd/ec2-utils
# create arm64 AL2 instance
AWS_K8S_TESTER_EC2_ON_FAILURE_DELETE=true \
AWS_K8S_TESTER_EC2_LOG_COLOR=true \
AWS_K8S_TESTER_EC2_REGION=us-west-2 \
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE=true \
AWS_K8S_TESTER_EC2_S3_BUCKET_CREATE_KEEP=true \
AWS_K8S_TESTER_EC2_REMOTE_ACCESS_KEY_CREATE=true \
AWS_K8S_TESTER_EC2_ASGS_FETCH_LOGS=false \
AWS_K8S_TESTER_EC2_ASGS='{"GetRef.Name-arm64-al2-cpu":{"name":"GetRef.Name-arm64-al2-cpu","remote-access-user-name":"ec2-user","ami-type":"AL2_arm_64","image-id-ssm-parameter":"/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-arm64-gp2","instance-types":["m6g.xlarge"],"volume-size":40,"asg-min-size":1,"asg-max-size":1,"asg-desired-capacity":1}}' \
AWS_K8S_TESTER_EC2_ROLE_CREATE=true \
AWS_K8S_TESTER_EC2_VPC_CREATE=true \
ec2-utils create instances --enable-prompt=true --auto-path
```
### Step 2. Install github action on the host
SSH into the instance, and install the github action self-hosted runner (see [install scripts](https://github.com/etcd-io/etcd/settings/actions/runners/new?arch=arm64&os=linux)).
### Step 3. Configure github action on the host
SSH into the instance, and configure the github action self-hosted runner.
First, we need disable ICU install (see [actions/runner issue on ARM64](https://github.com/actions/runner/issues/629)):
```
sudo yum install -y patch
```
And write this bash script:
```
#!/bin/bash -e
patch -p1 <<ICU_PATCH
diff -Naur a/bin/Runner.Listener.runtimeconfig.json b/bin/Runner.Listener.runtimeconfig.json
--- a/bin/Runner.Listener.runtimeconfig.json 2020-07-01 02:21:09.000000000 +0000
+++ b/bin/Runner.Listener.runtimeconfig.json 2020-07-28 00:02:38.748868613 +0000
@@ -8,7 +8,8 @@
}
],
"configProperties": {
- "System.Runtime.TieredCompilation.QuickJit": true
+ "System.Runtime.TieredCompilation.QuickJit": true,
+ "System.Globalization.Invariant": true
}
}
-}
\ No newline at end of file
+}
diff -Naur a/bin/Runner.PluginHost.runtimeconfig.json b/bin/Runner.PluginHost.runtimeconfig.json
--- a/bin/Runner.PluginHost.runtimeconfig.json 2020-07-01 02:21:22.000000000 +0000
+++ b/bin/Runner.PluginHost.runtimeconfig.json 2020-07-28 00:02:59.358680003 +0000
@@ -8,7 +8,8 @@
}
],
"configProperties": {
- "System.Runtime.TieredCompilation.QuickJit": true
+ "System.Runtime.TieredCompilation.QuickJit": true,
+ "System.Globalization.Invariant": true
}
}
-}
\ No newline at end of file
+}
diff -Naur a/bin/Runner.Worker.runtimeconfig.json b/bin/Runner.Worker.runtimeconfig.json
--- a/bin/Runner.Worker.runtimeconfig.json 2020-07-01 02:21:16.000000000 +0000
+++ b/bin/Runner.Worker.runtimeconfig.json 2020-07-28 00:02:19.159028531 +0000
@@ -8,7 +8,8 @@
}
],
"configProperties": {
- "System.Runtime.TieredCompilation.QuickJit": true
+ "System.Runtime.TieredCompilation.QuickJit": true,
+ "System.Globalization.Invariant": true
}
}
-}
\ No newline at end of file
+}
ICU_PATCH
```
And now patch the github action runner:
```
cd ${HOME}/actions-runner
bash patch.sh
```
```
patching file bin/Runner.Listener.runtimeconfig.json
patching file bin/Runner.PluginHost.runtimeconfig.json
patching file bin/Runner.Worker.runtimeconfig.json
```
And now configure:
```
sudo yum install -y wget
INSTANCE_ID=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
echo ${INSTANCE_ID}
# get token from https://github.com/etcd-io/etcd/settings/actions/runners/new?arch=arm64&os=linux
cd ${HOME}/actions-runner
./config.sh \
--work "_work" \
--name ${INSTANCE_ID} \
--labels self-hosted,linux,ARM64,graviton2 \
--url https://github.com/etcd-io/etcd \
--token ...
```
And run:
```
# run this as a process in the terminal
cd ${HOME}/actions-runner
./run.sh
# or run this as a systemd service
cd ${HOME}/actions-runner
sudo ./svc.sh install
sudo ./svc.sh start
sudo ./svc.sh status
```
### Step 4. Create github action configuration
See https://github.com/etcd-io/etcd/pull/12928.

60
.github/workflows/tests.yaml vendored Normal file
View File

@ -0,0 +1,60 @@
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
target:
- linux-amd64-fmt
- linux-amd64-integration-1-cpu
- linux-amd64-integration-2-cpu
- linux-amd64-integration-4-cpu
- linux-amd64-unit-4-cpu-race
- all-build
- linux-386-unit-1-cpu
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: "^1.16"
- run: date
- env:
TARGET: ${{ matrix.target }}
run: |
echo "${TARGET}"
case "${TARGET}" in
linux-amd64-fmt)
GOARCH=amd64 PASSES='fmt bom dep' ./test.sh
;;
linux-amd64-integration-1-cpu)
GOARCH=amd64 CPU=1 PASSES='integration' RACE='false' ./test.sh
;;
linux-amd64-integration-2-cpu)
GOARCH=amd64 CPU=2 PASSES='integration' RACE='false' ./test.sh
;;
linux-amd64-integration-4-cpu)
GOARCH=amd64 CPU=4 PASSES='integration' RACE='false' ./test.sh
;;
linux-amd64-unit-4-cpu-race)
GOARCH=amd64 PASSES='unit' RACE='true' CPU='4' ./test.sh -p=2
;;
all-build)
GOARCH=amd64 PASSES='build' ./test.sh
GOARCH=386 PASSES='build' ./test.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOOS=darwin GOARCH=amd64 ./build.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOOS=windows GOARCH=amd64 ./build.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOARCH=arm ./build.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOARCH=arm64 ./build.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOARCH=ppc64le ./build.sh
GO_BUILD_FLAGS='-v -mod=readonly' GOARCH=s390x ./build.sh
;;
linux-386-unit-1-cpu)
GOARCH=386 PASSES='unit' RACE='false' CPU='1' ./test -p=4
;;
*)
echo "Failed to find target"
exit 1
;;
esac

23
.gitignore vendored
View File

@ -7,6 +7,7 @@
/bin
*.etcd
*.log
*.swp
/etcd
/hack/insta-discovery/.env
*.coverprofile
@ -15,22 +16,8 @@ hack/tls-setup/certs
.idea
/contrib/raftexample/raftexample
/contrib/raftexample/raftexample-*
# TODO: use dep prune
# https://github.com/golang/dep/issues/120#issuecomment-306518546
vendor/**/*
!vendor/**/
!vendor/**/*.go
!vendor/**/*.c
!vendor/**/*.cpp
!vendor/**/*.s
!vendor/**/COPYING*
!vendor/**/PATENTS*
!vendor/**/NOTICE*
!vendor/**/Licence*
!vendor/**/License*
!vendor/**/LICENCE*
!vendor/**/LICENSE*
vendor/**/*_test.go
/vendor
/tests/e2e/default.proxy
*.tmp
*.bak
.gobincache/

View File

@ -1,13 +1,13 @@
language: go
go_import_path: go.etcd.io/etcd
go_import_path: go.etcd.io/etcd/v3
sudo: required
services: docker
go:
- 1.12.7
- tip
- "1.16.3"
- tip
notifications:
on_success: never
@ -15,118 +15,37 @@ notifications:
env:
matrix:
- TARGET=linux-amd64-fmt
- TARGET=linux-amd64-integration-1-cpu
- TARGET=linux-amd64-integration-2-cpu
- TARGET=linux-amd64-integration-4-cpu
- TARGET=linux-amd64-functional
- TARGET=linux-amd64-unit
- TARGET=all-build
- TARGET=linux-amd64-grpcproxy
- TARGET=linux-amd64-coverage
- TARGET=linux-amd64-fmt-unit-go-tip
- TARGET=linux-386-unit
- TARGET=linux-amd64-coverage
- TARGET=linux-amd64-fmt-unit-go-tip-2-cpu
matrix:
fast_finish: true
allow_failures:
- go: 1.12.7
env: TARGET=linux-amd64-integration-4-cpu
- go: 1.12.7
env: TARGET=linux-amd64-grpcproxy
- go: 1.12.7
env: TARGET=linux-amd64-coverage
- go: tip
env: TARGET=linux-amd64-fmt-unit-go-tip
- go: 1.12.7
env: TARGET=linux-386-unit
- go: "1.16.3"
env: TARGET=linux-amd64-coverage
- go: tip
env: TARGET=linux-amd64-fmt-unit-go-tip-2-cpu
exclude:
- go: tip
env: TARGET=linux-amd64-fmt
- go: tip
env: TARGET=linux-amd64-integration-1-cpu
- go: tip
env: TARGET=linux-amd64-integration-2-cpu
- go: tip
env: TARGET=linux-amd64-integration-4-cpu
- go: tip
env: TARGET=linux-amd64-functional
- go: tip
env: TARGET=linux-amd64-unit
- go: tip
env: TARGET=all-build
- go: tip
env: TARGET=linux-amd64-grpcproxy
- go: tip
env: TARGET=linux-amd64-coverage
- go: 1.12.7
env: TARGET=linux-amd64-fmt-unit-go-tip
- go: tip
env: TARGET=linux-386-unit
- go: tip
env: TARGET=linux-amd64-coverage
- go: "1.16.3"
env: TARGET=linux-amd64-fmt-unit-go-tip-2-cpu
before_install:
- if [[ $TRAVIS_GO_VERSION == 1.* ]]; then docker pull gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION}; fi
- if [[ $TRAVIS_GO_VERSION == 1.* ]]; then docker pull gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION}; fi
install:
- go get -t -v -d ./...
- date
script:
- echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}"
- >
- date
- echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}"
- >
case "${TARGET}" in
linux-amd64-fmt)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 PASSES='fmt bom dep' ./test"
;;
linux-amd64-integration-1-cpu)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 CPU=1 PASSES='integration' ./test"
;;
linux-amd64-integration-2-cpu)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 CPU=2 PASSES='integration' ./test"
;;
linux-amd64-integration-4-cpu)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 CPU=4 PASSES='integration' ./test"
;;
linux-amd64-functional)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "./build && GOARCH=amd64 PASSES='functional' ./test"
;;
linux-amd64-unit)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 PASSES='unit' ./test"
;;
all-build)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=amd64 PASSES='build' ./test \
&& GOARCH=386 PASSES='build' ./test \
&& GO_BUILD_FLAGS='-v' GOOS=darwin GOARCH=amd64 ./build \
&& GO_BUILD_FLAGS='-v' GOOS=windows GOARCH=amd64 ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=arm ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=arm64 ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=ppc64le ./build"
;;
linux-amd64-grpcproxy)
sudo HOST_TMP_DIR=/tmp TEST_OPTS="PASSES='build grpcproxy'" make docker-test
;;
linux-amd64-coverage)
sudo HOST_TMP_DIR=/tmp make docker-test-coverage
sudo HOST_TMP_DIR=/tmp TEST_OPTS="VERBOSE='1'" make docker-test-coverage
;;
linux-amd64-fmt-unit-go-tip)
GOARCH=amd64 PASSES='fmt unit' ./test
;;
linux-386-unit)
docker run --rm \
--volume=`pwd`:/go/src/go.etcd.io/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "GOARCH=386 PASSES='unit' ./test"
linux-amd64-fmt-unit-go-tip-2-cpu)
GOARCH=amd64 PASSES='fmt unit' CPU='2' RACE='false' ./test.sh -p=2
;;
esac

143
.words
View File

@ -1,109 +1,116 @@
DefaultMaxRequestBytes
ErrCodeEnhanceYourCalm
ErrTimeout
GoAway
KeepAlive
Keepalive
MiB
ResourceExhausted
RPC
RPCs
parsedTarget
SRV
WithRequireLeader
InfoLevel
accessors
addrConns
args
atomics
backoff
BackoffFunc
BackoffLinearWithJitter
Balancer
BidiStreams
blackhole
blackholed
CallOptions
cancelable
cancelation
ccBalancerWrapper
clientURLs
clusterName
cluster_proxy
consistentIndex
ConsistentIndexGetter
DefaultMaxRequestBytes
defragment
defragmenting
deleter
dev
/dev/null
dev/null
DNS
errClientDisconnected
ErrCodeEnhanceYourCalm
ErrConnClosing
ErrRequestTooLarge
ErrTimeout
etcd
gRPC
FIXME
github
GoAway
goroutine
goroutines
gRPC
grpcAddr
hasleader
healthcheck
hostname
iff
inflight
InfoLevel
jitter
jitter
jitter
keepalive
Keepalive
KeepAlive
keepalives
keyspace
lexically
lexicographically
linearizable
linearization
liveness
linearized
liveness
localhost
__lostleader
MaxRequestBytes
MiB
middleware
mutators
mutex
nils
nondeterministically
nop
OutputWALDir
parsedTarget
passthrough
PermitWithoutStream
prefetching
protobuf
prometheus
protobuf
racey
rafthttp
rebalanced
reconnection
repin
ResourceExhausted
retriable
retriable
rpc
RPC
RPCs
saveWALAndSnap
serializable
ServerStreams
SHA
SRV
statusError
subConn
subconns
SubConns
teardown
TestBalancerDoNotBlockOnClose
todo
too_many_pings
transactional
uncontended
unprefixed
unlisting
nondeterministically
atomics
transferee
Balancer
lexicographically
lexically
accessors
unbuffered
nils
reconnection
mutators
ConsistentIndexGetter
OutputWALDir
WAL
consistentIndex
todo
saveWALAndSnap
subconns
nop
SubConns
DNS
passthrough
ccBalancerWrapper
rebalanced
addrConns
subConn
TestBalancerDoNotBlockOnClose
middleware
clusterName
jitter
FIXME
retriable
github
retriable
jitter
WithBackoff
BackoffLinearWithJitter
jitter
WithMax
ServerStreams
BidiStreams
transientFailure
BackoffFunc
CallOptions
PermitWithoutStream
__lostleader
ErrConnClosing
unbuffered
uncontended
unfreed
grpcAddr
clientURLs
unlisting
unprefixed
WatchProgressNotifyInterval
WAL
WithBackoff
WithDialer
WithMax
WithRequireLeader

View File

@ -1,16 +0,0 @@
<hr>
## [v2.3.8](https://github.com/etcd-io/etcd/releases/tag/v2.3.8) (2017-02-17)
See [code changes](https://github.com/etcd-io/etcd/compare/v2.3.7...v2.3.8).
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>

View File

@ -1,291 +0,0 @@
<hr>
## [v3.0.16](https://github.com/etcd-io/etcd/releases/tag/v3.0.16) (2016-11-13)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.15...v3.0.16) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.4*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.15](https://github.com/etcd-io/etcd/releases/tag/v3.0.15) (2016-11-11)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.14...v3.0.15) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Fixed
- Fix cancel watch request with wrong range end.
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.14](https://github.com/etcd-io/etcd/releases/tag/v3.0.14) (2016-11-04)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.13...v3.0.14) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Added
- v3 `etcdctl migrate` command now supports `--no-ttl` flag to discard keys on transform.
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.13](https://github.com/etcd-io/etcd/releases/tag/v3.0.13) (2016-10-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.12...v3.0.13) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.12](https://github.com/etcd-io/etcd/releases/tag/v3.0.12) (2016-10-07)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.11...v3.0.12) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.11](https://github.com/etcd-io/etcd/releases/tag/v3.0.11) (2016-10-07)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.10...v3.0.11) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Added
- Server returns previous key-value (optional)
- `clientv3.WithPrevKV` option
- v3 etcdctl `put,watch,del --prev-kv` flag
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.10](https://github.com/etcd-io/etcd/releases/tag/v3.0.10) (2016-09-23)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.9...v3.0.10) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.9](https://github.com/etcd-io/etcd/releases/tag/v3.0.9) (2016-09-15)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.8...v3.0.9) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Added
- Warn on domain names on listen URLs (v3.2 will reject domain names).
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.8](https://github.com/etcd-io/etcd/releases/tag/v3.0.8) (2016-09-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.7...v3.0.8) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Other
- Allow only IP addresses in listen URLs (domain names are rejected).
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.7](https://github.com/etcd-io/etcd/releases/tag/v3.0.7) (2016-08-31)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.6...v3.0.7) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Other
- SRV records only allow A records (RFC 2052).
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.6](https://github.com/etcd-io/etcd/releases/tag/v3.0.6) (2016-08-19)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.5...v3.0.6) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.5](https://github.com/etcd-io/etcd/releases/tag/v3.0.5) (2016-08-19)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.4...v3.0.5) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Other
- SRV records (e.g., infra1.example.com) must match the discovery domain (i.e., example.com) if no custom certificate authority is given.
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.4](https://github.com/etcd-io/etcd/releases/tag/v3.0.4) (2016-07-27)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.3...v3.0.4) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Added
- v2 `etcdctl ls` command now supports `--output=json`.
- Add /var/lib/etcd directory to etcd official Docker image.
### Other
- v2 auth can now use common name from TLS certificate when `--client-cert-auth` is enabled.
### Go
- Compile with [*Go 1.6.3*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.3](https://github.com/etcd-io/etcd/releases/tag/v3.0.3) (2016-07-15)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.2...v3.0.3) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Other
- Revert Dockerfile to use `CMD`, instead of `ENTRYPOINT`, to support `etcdctl` run.
- Docker commands for v3.0.2 won't work without specifying executable binary paths.
- v3 etcdctl default endpoints are now `127.0.0.1:2379`.
### Go
- Compile with [*Go 1.6.2*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.2](https://github.com/etcd-io/etcd/releases/tag/v3.0.2) (2016-07-08)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.1...v3.0.2) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Other
- Dockerfile uses `ENTRYPOINT`, instead of `CMD`, to run etcd without binary path specified.
### Go
- Compile with [*Go 1.6.2*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.1](https://github.com/etcd-io/etcd/releases/tag/v3.0.1) (2016-07-01)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.0...v3.0.1) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.2*](https://golang.org/doc/devel/release.html#go1.6).
<hr>
## [v3.0.0](https://github.com/etcd-io/etcd/releases/tag/v3.0.0) (2016-06-30)
See [code changes](https://github.com/etcd-io/etcd/compare/v2.3.0...v3.0.0) and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md).**
### Go
- Compile with [*Go 1.6.2*](https://golang.org/doc/devel/release.html#go1.6).
<hr>

View File

@ -1,574 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.0](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.0.md).
The minimum recommended etcd versions to run in **production** are 3.1.11+, 3.2.26+, and 3.3.11+.
<hr>
## [v3.1.21](https://github.com/etcd-io/etcd/releases/tag/v3.1.21) (2019-TBD)
### etcdctl
- [Strip out insecure endpoints from DNS SRV records when using discovery](https://github.com/etcd-io/etcd/pull/10443) with etcdctl v2
- Add [`etcdctl endpoint health --write-out` support](https://github.com/etcd-io/etcd/pull/9540).
- Previously, [`etcdctl endpoint health --write-out json` did not work](https://github.com/etcd-io/etcd/issues/9532).
- The command output is changed. Previously, if endpoint is unreachable, the command output is
"\<endpoint\> is unhealthy: failed to connect: \<error message\>". This change unified the error message, all error types
now have the same output "\<endpoint\> is unhealthy: failed to commit proposal: \<error message\>".
### Metrics, Monitoring
- Fix bug where [db_compaction_total_duration_milliseconds metric incorrectly measured duration as 0](https://github.com/etcd-io/etcd/pull/10646).
<hr>
## [v3.1.20](https://github.com/etcd-io/etcd/releases/tag/v3.1.20) (2018-10-10)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.19...v3.1.20) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Improved
- Improve ["became inactive" warning log](https://github.com/etcd-io/etcd/pull/10024), which indicates message send to a peer failed.
- Improve [read index wait timeout warning log](https://github.com/etcd-io/etcd/pull/10026), which indicates that local node might have slow network.
- Add [gRPC interceptor for debugging logs](https://github.com/etcd-io/etcd/pull/9990); enable `etcd --debug` flag to see per-request debug information.
- Add [consistency check in snapshot status](https://github.com/etcd-io/etcd/pull/10109). If consistency check on snapshot file fails, `snapshot status` returns `"snapshot file integrity check failed..."` error.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.1.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Improve [`etcd_network_peer_round_trip_time_seconds`](https://github.com/etcd-io/etcd/pull/10155) Prometheus metric to track leader heartbeats.
- Previously, it only samples the TCP connection for snapshot messages.
- Display all registered [gRPC metrics at start](https://github.com/etcd-io/etcd/pull/10034).
- Add [`etcd_snap_db_fsync_duration_seconds_count`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_snap_db_save_total_duration_seconds_bucket`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_server_id`](https://github.com/etcd-io/etcd/pull/9998) Prometheus metric.
- Add [`etcd_server_health_success`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_health_failures`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_read_indexes_failed_total`](https://github.com/etcd-io/etcd/pull/10094) Prometheus metric.
### client v3
- Fix logic on [release lock key if cancelled](https://github.com/etcd-io/etcd/pull/10153) in `clientv3/concurrency` package.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.19](https://github.com/etcd-io/etcd/releases/tag/v3.1.19) (2018-07-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.18...v3.1.19) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Improved
- Improve [Raft Read Index timeout warning messages](https://github.com/etcd-io/etcd/pull/9897).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.1.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_go_version`](https://github.com/etcd-io/etcd/pull/9957) Prometheus metric.
- Add [`etcd_server_slow_read_indexes_total`](https://github.com/etcd-io/etcd/pull/9897) Prometheus metric.
- Add [`etcd_server_quota_backend_bytes`](https://github.com/etcd-io/etcd/pull/9820) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
- Add [`etcd_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819) Prometheus metric.
- In addition to [`etcd_debugging_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819).
- Add [`etcd_mvcc_db_total_size_in_use_in_bytes`](https://github.com/etcd-io/etcd/pull/9256) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
### client v3
- Fix [lease keepalive interval updates when response queue is full](https://github.com/etcd-io/etcd/pull/9952).
- If `<-chan *clientv3LeaseKeepAliveResponse` from `clientv3.Lease.KeepAlive` was never consumed or channel is full, client was [sending keepalive request every 500ms](https://github.com/etcd-io/etcd/issues/9911) instead of expected rate of every "TTL / 3" duration.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.18](https://github.com/etcd-io/etcd/releases/tag/v3.1.18) (2018-06-15)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.17...v3.1.18) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.1.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_version`](https://github.com/etcd-io/etcd/pull/8960) Prometheus metric.
- To replace [Kubernetes `etcd-version-monitor`](https://github.com/etcd-io/etcd/issues/8948).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.17](https://github.com/etcd-io/etcd/releases/tag/v3.1.17) (2018-06-06)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.16...v3.1.17) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Fix [v3 snapshot recovery](https://github.com/etcd-io/etcd/issues/7628).
- A follower receives a leader snapshot to be persisted as a `[SNAPSHOT-INDEX].snap.db` file on disk.
- Now, server [ensures that the incoming snapshot be persisted on disk before loading it](https://github.com/etcd-io/etcd/pull/7876).
- Otherwise, index mismatch happens and triggers server-side panic (e.g. newer WAL entry with outdated snapshot index).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.16](https://github.com/etcd-io/etcd/releases/tag/v3.1.16) (2018-05-31)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.15...v3.1.16) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Fix [`mvcc` server panic from restore operation](https://github.com/etcd-io/etcd/pull/9775).
- Let's assume that a watcher had been requested with a future revision X and sent to node A that became network-partitioned thereafter. Meanwhile, cluster makes progress. Then when the partition gets removed, the leader sends a snapshot to node A. Previously if the snapshot's latest revision is still lower than the watch revision X, **etcd server panicked** during snapshot restore operation.
- Now, this server-side panic has been fixed.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.15](https://github.com/etcd-io/etcd/releases/tag/v3.1.15) (2018-05-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.14...v3.1.15) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Purge old [`*.snap.db` snapshot files](https://github.com/etcd-io/etcd/pull/7967).
- Previously, etcd did not respect `--max-snapshots` flag to purge old `*.snap.db` files.
- Now, etcd purges old `*.snap.db` files to keep maximum `--max-snapshots` number of files on disk.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.14](https://github.com/etcd-io/etcd/releases/tag/v3.1.14) (2018-04-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.13...v3.1.14) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.1.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_is_leader`](https://github.com/etcd-io/etcd/pull/9587) Prometheus metric.
### etcd server
- Add [`--initial-election-tick-advance`](https://github.com/etcd-io/etcd/pull/9591) flag to configure initial election tick fast-forward.
- By default, `--initial-election-tick-advance=true`, then local member fast-forwards election ticks to speed up "initial" leader election trigger.
- This benefits the case of larger election ticks. For instance, cross datacenter deployment may require longer election timeout of 10-second. If true, local node does not need wait up to 10-second. Instead, forwards its election ticks to 8-second, and have only 2-second left before leader election.
- Major assumptions are that: cluster has no active leader thus advancing ticks enables faster leader election. Or cluster already has an established leader, and rejoining follower is likely to receive heartbeats from the leader after tick advance and before election timeout.
- However, when network from leader to rejoining follower is congested, and the follower does not receive leader heartbeat within left election ticks, disruptive election has to happen thus affecting cluster availabilities.
- Now, this can be disabled by setting `--initial-election-tick-advance=false`.
- Disabling this would slow down initial bootstrap process for cross datacenter deployments. Make tradeoffs by configuring `--initial-election-tick-advance` at the cost of slow initial bootstrap.
- If single-node, it advances ticks regardless.
- Address [disruptive rejoining follower node](https://github.com/etcd-io/etcd/issues/9333).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.13](https://github.com/etcd-io/etcd/releases/tag/v3.1.13) (2018-03-29)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.12...v3.1.13) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Improved
- Adjust [election timeout on server restart](https://github.com/etcd-io/etcd/pull/9415) to reduce [disruptive rejoining servers](https://github.com/etcd-io/etcd/issues/9333).
- Previously, etcd fast-forwards election ticks on server start, with only one tick left for leader election. This is to speed up start phase, without having to wait until all election ticks elapse. Advancing election ticks is useful for cross datacenter deployments with larger election timeouts. However, it was affecting cluster availability if the last tick elapses before leader contacts the restarted node.
- Now, when etcd restarts, it adjusts election ticks with more than one tick left, thus more time for leader to prevent disruptive restart.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.1.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add missing [`etcd_network_peer_sent_failures_total` count](https://github.com/etcd-io/etcd/pull/9437).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.12](https://github.com/etcd-io/etcd/releases/tag/v3.1.12) (2018-03-08)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.11...v3.1.12) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Fix [`mvcc` "unsynced" watcher restore operation](https://github.com/etcd-io/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/etcd-io/etcd/issues/9086).
- A node gets network partitioned with a watcher on a future revision, and falls behind receiving a leader snapshot after partition gets removed. When applying this snapshot, etcd watch storage moves current synced watchers to unsynced since sync watchers might have become stale during network partition. And reset synced watcher group to restart watcher routines. Previously, there was a bug when moving from synced watcher group to unsynced, thus client would miss events when the watcher was requested to the network-partitioned node.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.11](https://github.com/etcd-io/etcd/releases/tag/v3.1.11) (2017-11-28)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.10...v3.1.11) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- [#8411](https://github.com/etcd-io/etcd/issues/8411),[#8806](https://github.com/etcd-io/etcd/pull/8806) backport "mvcc: sending events after restore"
- [#8009](https://github.com/etcd-io/etcd/issues/8009),[#8902](https://github.com/etcd-io/etcd/pull/8902) backport coreos/bbolt v1.3.1-coreos.5
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.1.10](https://github.com/etcd-io/etcd/releases/tag/v3.1.10) (2017-07-14)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.9...v3.1.10) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Added
- Tag docker images with minor versions.
- e.g. `docker pull quay.io/coreos/etcd:v3.1` to fetch latest v3.1 versions.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
- Fix panic on `net/http.CloseNotify`
<hr>
## [v3.1.9](https://github.com/etcd-io/etcd/releases/tag/v3.1.9) (2017-06-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.8...v3.1.9) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Allow v2 snapshot over 512MB.
### Go
- Compile with [*Go 1.7.6*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.8](https://github.com/etcd-io/etcd/releases/tag/v3.1.8) (2017-05-19)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.7...v3.1.8) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.7](https://github.com/etcd-io/etcd/releases/tag/v3.1.7) (2017-04-28)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.6...v3.1.7) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.6](https://github.com/etcd-io/etcd/releases/tag/v3.1.6) (2017-04-19)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.5...v3.1.6) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Fill in Auth API response header.
- Remove auth check in Status API.
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.5](https://github.com/etcd-io/etcd/releases/tag/v3.1.5) (2017-03-27)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.4...v3.1.5) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd server
- Fix raft memory leak issue.
- Fix Windows file path issues.
### Other
- Add `/etc/nsswitch.conf` file to alpine-based Docker image.
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.4](https://github.com/etcd-io/etcd/releases/tag/v3.1.4) (2017-03-22)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.3...v3.1.4) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.3](https://github.com/etcd-io/etcd/releases/tag/v3.1.3) (2017-03-10)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.2...v3.1.3) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd gateway
- Fix `etcd gateway` schema handling in DNS discovery.
- Fix sd_notify behaviors in `gateway`, `grpc-proxy`.
### gRPC Proxy
- Fix sd_notify behaviors in `gateway`, `grpc-proxy`.
### Other
- Use machine default host when advertise URLs are default values(`localhost:2379,2380`) AND if listen URL is `0.0.0.0`.
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.2](https://github.com/etcd-io/etcd/releases/tag/v3.1.2) (2017-02-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.1...v3.1.2) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### etcd gateway
- Fix `etcd gateway` with multiple endpoints.
### Other
- Use IPv4 default host, by default (when IPv4 and IPv6 are available).
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.1](https://github.com/etcd-io/etcd/releases/tag/v3.1.1) (2017-02-17)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.0...v3.1.1) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Go
- Compile with [*Go 1.7.5*](https://golang.org/doc/devel/release.html#go1.7).
<hr>
## [v3.1.0](https://github.com/etcd-io/etcd/releases/tag/v3.1.0) (2017-01-20)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.0.0...v3.1.0) and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.1 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md).**
### Improved
- Faster linearizable reads (implements Raft [read-index](https://github.com/etcd-io/etcd/pull/6212)).
- v3 authentication API is now stable.
### Breaking Changes
- Deprecated following gRPC metrics in favor of [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
- `etcd_grpc_requests_total`
- `etcd_grpc_requests_failed_total`
- `etcd_grpc_active_streams`
- `etcd_grpc_unary_requests_duration_seconds`
### Dependency
- Upgrade [`github.com/ugorji/go/codec`](https://github.com/ugorji/go) to [**`ugorji/go@9c7f9b7`**](https://github.com/ugorji/go/commit/9c7f9b7a2bc3a520f7c7b30b34b7f85f47fe27b6), and [regenerate v2 `client`](https://github.com/etcd-io/etcd/pull/6945).
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- SRV records (e.g., infra1.example.com) must match the discovery domain (i.e., example.com) if no custom certificate authority is given.
- `TLSConfig.ServerName` is ignored with user-provided certificates for backwards compatibility; to be deprecated.
- For example, `etcd --discovery-srv=example.com` will only authenticate peers/clients when the provided certs have root domain `example.com` as an entry in Subject Alternative Name (SAN) field.
### etcd server
- Automatic leadership transfer when leader steps down.
- etcd flags
- `--strict-reconfig-check` flag is set by default.
- Add `--log-output` flag.
- Add `--metrics` flag.
- etcd uses default route IP if advertise URL is not given.
- Cluster rejects removing members if quorum will be lost.
- Discovery now has upper limit for waiting on retries.
- Warn on binding listeners through domain names; to be deprecated.
- v3.0 and v3.1 with `--auto-compaction-retention=10` run periodic compaction on v3 key-value store for every 10-hour.
- Compactor only supports periodic compaction.
- Compactor records latest revisions every 5-minute, until it reaches the first compaction period (e.g. 10-hour).
- In order to retain key-value history of last compaction period, it uses the last revision that was fetched before compaction period, from the revision records that were collected every 5-minute.
- When `--auto-compaction-retention=10`, compactor uses revision 100 for compact revision where revision 100 is the latest revision fetched from 10 hours ago.
- If compaction succeeds or requested revision has already been compacted, it resets period timer and starts over with new historical revision records (e.g. restart revision collect and compact for the next 10-hour period).
- If compaction fails, it retries in 5 minutes.
### client v3
- Add `SetEndpoints` method; update endpoints at runtime.
- Add `Sync` method; auto-update endpoints at runtime.
- Add `Lease TimeToLive` API; fetch lease information.
- replace Config.Logger field with global logger.
- Get API responses are sorted in ascending order by default.
### etcdctl v3
- Add `lease timetolive` command.
- Add `--print-value-only` flag to get command.
- Add `--dest-prefix` flag to make-mirror command.
- `get` command responses are sorted in ascending order by default.
### gRPC Proxy
- Experimental gRPC proxy feature.
### Other
- `recipes` now conform to sessions defined in `clientv3/concurrency`.
- ACI has symlinks to `/usr/local/bin/etcd*`.
### Go
- Compile with [*Go 1.7.4*](https://golang.org/doc/devel/release.html#go1.7).
<hr>

View File

@ -1,884 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.1](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.1.md).
The minimum recommended etcd versions to run in **production** are 3.1.11+, 3.2.26+, and 3.3.11+.
<hr>
## [v3.2.27](https://github.com/etcd-io/etcd/releases/tag/v3.2.27) (2019-TBD)
### etcdctl
- [Strip out insecure endpoints from DNS SRV records when using discovery](https://github.com/etcd-io/etcd/pull/10443) with etcdctl v2
- Add [`etcdctl endpoint health --write-out` support](https://github.com/etcd-io/etcd/pull/9540).
- Previously, [`etcdctl endpoint health --write-out json` did not work](https://github.com/etcd-io/etcd/issues/9532).
- The command output is changed. Previously, if endpoint is unreachable, the command output is
"\<endpoint\> is unhealthy: failed to connect: \<error message\>". This change unified the error message, all error types
now have the same output "\<endpoint\> is unhealthy: failed to commit proposal: \<error message\>".
### Metrics, Monitoring
- Fix bug where [db_compaction_total_duration_milliseconds metric incorrectly measured duration as 0](https://github.com/etcd-io/etcd/pull/10646).
<hr>
## [v3.2.26](https://github.com/etcd-io/etcd/releases/tag/v3.2.26) (2019-01-11)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.25...v3.2.26) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### gRPC Proxy
- Fix [memory leak in cache layer](https://github.com/etcd-io/etcd/pull/10327).
### Security, Authentication
- Disable [CommonName authentication for gRPC-gateway](https://github.com/etcd-io/etcd/pull/10366) gRPC-gateway proxy requests to etcd server use the etcd client server TLS certificate. If that certificate contains CommonName we do not want to use that for authentication as it could lead to permission escalation.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.25](https://github.com/etcd-io/etcd/releases/tag/v3.2.25) (2018-10-10)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.24...v3.2.25) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Improve ["became inactive" warning log](https://github.com/etcd-io/etcd/pull/10024), which indicates message send to a peer failed.
- Improve [read index wait timeout warning log](https://github.com/etcd-io/etcd/pull/10026), which indicates that local node might have slow network.
- Add [gRPC interceptor for debugging logs](https://github.com/etcd-io/etcd/pull/9990); enable `etcd --debug` flag to see per-request debug information.
- Add [consistency check in snapshot status](https://github.com/etcd-io/etcd/pull/10109). If consistency check on snapshot file fails, `snapshot status` returns `"snapshot file integrity check failed..."` error.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.17/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Improve [`etcd_network_peer_round_trip_time_seconds`](https://github.com/etcd-io/etcd/pull/10155) Prometheus metric to track leader heartbeats.
- Previously, it only samples the TCP connection for snapshot messages.
- Display all registered [gRPC metrics at start](https://github.com/etcd-io/etcd/pull/10032).
- Add [`etcd_snap_db_fsync_duration_seconds_count`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_snap_db_save_total_duration_seconds_bucket`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_server_id`](https://github.com/etcd-io/etcd/pull/9998) Prometheus metric.
- Add [`etcd_server_health_success`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_health_failures`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_read_indexes_failed_total`](https://github.com/etcd-io/etcd/pull/10094) Prometheus metric.
### client v3
- Fix logic on [release lock key if cancelled](https://github.com/etcd-io/etcd/pull/10153) in `clientv3/concurrency` package.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.24](https://github.com/etcd-io/etcd/releases/tag/v3.2.24) (2018-07-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.23...v3.2.24) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Improve [Raft Read Index timeout warning messages](https://github.com/etcd-io/etcd/pull/9897).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_go_version`](https://github.com/etcd-io/etcd/pull/9957) Prometheus metric.
- Add [`etcd_server_heartbeat_send_failures_total`](https://github.com/etcd-io/etcd/pull/9942) Prometheus metric.
- Add [`etcd_server_slow_apply_total`](https://github.com/etcd-io/etcd/pull/9942) Prometheus metric.
- Add [`etcd_disk_backend_defrag_duration_seconds`](https://github.com/etcd-io/etcd/pull/9942) Prometheus metric.
- Add [`etcd_mvcc_hash_duration_seconds`](https://github.com/etcd-io/etcd/pull/9942) Prometheus metric.
- Add [`etcd_server_slow_read_indexes_total`](https://github.com/etcd-io/etcd/pull/9897) Prometheus metric.
- Add [`etcd_server_quota_backend_bytes`](https://github.com/etcd-io/etcd/pull/9820) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
- Add [`etcd_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819) Prometheus metric.
- In addition to [`etcd_debugging_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819).
- Add [`etcd_mvcc_db_total_size_in_use_in_bytes`](https://github.com/etcd-io/etcd/pull/9256) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_server_quota_backend_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
### gRPC Proxy
- Add [flags for specifying TLS for connecting to proxy](https://github.com/etcd-io/etcd/pull/9894):
- Add `grpc-proxy start --cert-file`, `grpc-proxy start --key-file` and `grpc-proxy start --trusted-ca-file` flags.
- Add [`grpc-proxy start --metrics-addr` flag for specifying a separate metrics listen address](https://github.com/etcd-io/etcd/pull/9894).
### client v3
- Fix [lease keepalive interval updates when response queue is full](https://github.com/etcd-io/etcd/pull/9952).
- If `<-chan *clientv3LeaseKeepAliveResponse` from `clientv3.Lease.KeepAlive` was never consumed or channel is full, client was [sending keepalive request every 500ms](https://github.com/etcd-io/etcd/issues/9911) instead of expected rate of every "TTL / 3" duration.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.23](https://github.com/etcd-io/etcd/releases/tag/v3.2.23) (2018-06-15)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.22...v3.2.23) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Improve [slow request apply warning log](https://github.com/etcd-io/etcd/pull/9288).
- e.g. `read-only range request "key:\"/a\" range_end:\"/b\" " with result "range_response_count:3 size:96" took too long (97.966µs) to execute`.
- Redact [request value field](https://github.com/etcd-io/etcd/pull/9822).
- Provide [response size](https://github.com/etcd-io/etcd/pull/9826).
- Add [backoff on watch retries on transient errors](https://github.com/etcd-io/etcd/pull/9840).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_version`](https://github.com/etcd-io/etcd/pull/8960) Prometheus metric.
- To replace [Kubernetes `etcd-version-monitor`](https://github.com/etcd-io/etcd/issues/8948).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.22](https://github.com/etcd-io/etcd/releases/tag/v3.2.22) (2018-06-06)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.21...v3.2.22) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Security, Authentication
- Support TLS cipher suite whitelisting.
- To block [weak cipher suites](https://github.com/etcd-io/etcd/issues/8320).
- TLS handshake fails when client hello is requested with invalid cipher suites.
- Add [`etcd --cipher-suites`](https://github.com/etcd-io/etcd/pull/9801) flag.
- If empty, Go auto-populates the list.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.21](https://github.com/etcd-io/etcd/releases/tag/v3.2.21) (2018-05-31)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.20...v3.2.21) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Fix [auth storage panic when simple token provider is disabled](https://github.com/etcd-io/etcd/pull/8695).
- Fix [`mvcc` server panic from restore operation](https://github.com/etcd-io/etcd/pull/9775).
- Let's assume that a watcher had been requested with a future revision X and sent to node A that became network-partitioned thereafter. Meanwhile, cluster makes progress. Then when the partition gets removed, the leader sends a snapshot to node A. Previously if the snapshot's latest revision is still lower than the watch revision X, **etcd server panicked** during snapshot restore operation.
- Now, this server-side panic has been fixed.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.20](https://github.com/etcd-io/etcd/releases/tag/v3.2.20) (2018-05-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.19...v3.2.20) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Purge old [`*.snap.db` snapshot files](https://github.com/etcd-io/etcd/pull/7967).
- Previously, etcd did not respect `--max-snapshots` flag to purge old `*.snap.db` files.
- Now, etcd purges old `*.snap.db` files to keep maximum `--max-snapshots` number of files on disk.
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.19](https://github.com/etcd-io/etcd/releases/tag/v3.2.19) (2018-04-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.18...v3.2.19) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Fix [`etcd_debugging_server_lease_expired_total`](https://github.com/etcd-io/etcd/pull/9557) Prometheus metric.
- Fix [race conditions in v2 server stat collecting](https://github.com/etcd-io/etcd/pull/9562).
- Add [`etcd_server_is_leader`](https://github.com/etcd-io/etcd/pull/9587) Prometheus metric.
### Security, Authentication
- Fix [TLS reload](https://github.com/etcd-io/etcd/pull/9570) when [certificate SAN field only includes IP addresses but no domain names](https://github.com/etcd-io/etcd/issues/9541).
- In Go, server calls `(*tls.Config).GetCertificate` for TLS reload if and only if server's `(*tls.Config).Certificates` field is not empty, or `(*tls.ClientHelloInfo).ServerName` is not empty with a valid SNI from the client. Previously, etcd always populates `(*tls.Config).Certificates` on the initial client TLS handshake, as non-empty. Thus, client was always expected to supply a matching SNI in order to pass the TLS verification and to trigger `(*tls.Config).GetCertificate` to reload TLS assets.
- However, a certificate whose SAN field does [not include any domain names but only IP addresses](https://github.com/etcd-io/etcd/issues/9541) would request `*tls.ClientHelloInfo` with an empty `ServerName` field, thus failing to trigger the TLS reload on initial TLS handshake; this becomes a problem when expired certificates need to be replaced online.
- Now, `(*tls.Config).Certificates` is created empty on initial TLS client handshake, first to trigger `(*tls.Config).GetCertificate`, and then to populate rest of the certificates on every new TLS connection, even when client SNI is empty (e.g. cert only includes IPs).
### etcd server
- Add [`etcd --initial-election-tick-advance`](https://github.com/etcd-io/etcd/pull/9591) flag to configure initial election tick fast-forward.
- By default, `etcd --initial-election-tick-advance=true`, then local member fast-forwards election ticks to speed up "initial" leader election trigger.
- This benefits the case of larger election ticks. For instance, cross datacenter deployment may require longer election timeout of 10-second. If true, local node does not need wait up to 10-second. Instead, forwards its election ticks to 8-second, and have only 2-second left before leader election.
- Major assumptions are that: cluster has no active leader thus advancing ticks enables faster leader election. Or cluster already has an established leader, and rejoining follower is likely to receive heartbeats from the leader after tick advance and before election timeout.
- However, when network from leader to rejoining follower is congested, and the follower does not receive leader heartbeat within left election ticks, disruptive election has to happen thus affecting cluster availabilities.
- Now, this can be disabled by setting `--initial-election-tick-advance=false`.
- Disabling this would slow down initial bootstrap process for cross datacenter deployments. Make tradeoffs by configuring `--initial-election-tick-advance` at the cost of slow initial bootstrap.
- If single-node, it advances ticks regardless.
- Address [disruptive rejoining follower node](https://github.com/etcd-io/etcd/issues/9333).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.18](https://github.com/etcd-io/etcd/releases/tag/v3.2.18) (2018-03-29)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.17...v3.2.18) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Adjust [election timeout on server restart](https://github.com/etcd-io/etcd/pull/9415) to reduce [disruptive rejoining servers](https://github.com/etcd-io/etcd/issues/9333).
- Previously, etcd fast-forwards election ticks on server start, with only one tick left for leader election. This is to speed up start phase, without having to wait until all election ticks elapse. Advancing election ticks is useful for cross datacenter deployments with larger election timeouts. However, it was affecting cluster availability if the last tick elapses before leader contacts the restarted node.
- Now, when etcd restarts, it adjusts election ticks with more than one tick left, thus more time for leader to prevent disruptive restart.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add missing [`etcd_network_peer_sent_failures_total` count](https://github.com/etcd-io/etcd/pull/9437).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.17](https://github.com/etcd-io/etcd/releases/tag/v3.2.17) (2018-03-08)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.16...v3.2.17) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Fix [server panic on invalid Election Proclaim/Resign HTTP(S) requests](https://github.com/etcd-io/etcd/pull/9379).
- Previously, wrong-formatted HTTP requests to Election API could trigger panic in etcd server.
- e.g. `curl -L http://localhost:2379/v3/election/proclaim -X POST -d '{"value":""}'`, `curl -L http://localhost:2379/v3/election/resign -X POST -d '{"value":""}'`.
- Prevent [overflow by large `TTL` values for `Lease` `Grant`](https://github.com/etcd-io/etcd/pull/9399).
- `TTL` parameter to `Grant` request is unit of second.
- Leases with too large `TTL` values exceeding `math.MaxInt64` [expire in unexpected ways](https://github.com/etcd-io/etcd/issues/9374).
- Server now returns `rpctypes.ErrLeaseTTLTooLarge` to client, when the requested `TTL` is larger than *9,000,000,000 seconds* (which is >285 years).
- Again, etcd `Lease` is meant for short-periodic keepalives or sessions, in the range of seconds or minutes. Not for hours or days!
- Enable etcd server [`raft.Config.CheckQuorum` when starting with `ForceNewCluster`](https://github.com/etcd-io/etcd/pull/9347).
### Proxy v2
- Fix [v2 proxy leaky HTTP requests](https://github.com/etcd-io/etcd/pull/9336).
### Go
- Compile with [*Go 1.8.7*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.16](https://github.com/etcd-io/etcd/releases/tag/v3.2.16) (2018-02-12)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.15...v3.2.16) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Fix [`mvcc` "unsynced" watcher restore operation](https://github.com/etcd-io/etcd/pull/9297).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/etcd-io/etcd/issues/9086).
- A node gets network partitioned with a watcher on a future revision, and falls behind receiving a leader snapshot after partition gets removed. When applying this snapshot, etcd watch storage moves current synced watchers to unsynced since sync watchers might have become stale during network partition. And reset synced watcher group to restart watcher routines. Previously, there was a bug when moving from synced watcher group to unsynced, thus client would miss events when the watcher was requested to the network-partitioned node.
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.15](https://github.com/etcd-io/etcd/releases/tag/v3.2.15) (2018-01-22)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.14...v3.2.15) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Prevent [server panic from member update/add](https://github.com/etcd-io/etcd/pull/9174) with [wrong scheme URLs](https://github.com/etcd-io/etcd/issues/9173).
- Log [user context cancel errors on stream APIs in debug level with TLS](https://github.com/etcd-io/etcd/pull/9178).
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.14](https://github.com/etcd-io/etcd/releases/tag/v3.2.14) (2018-01-11)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.13...v3.2.14) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Log [user context cancel errors on stream APIs in debug level](https://github.com/etcd-io/etcd/pull/9105).
### etcd server
- Fix [`mvcc/backend.defragdb` nil-pointer dereference on create bucket failure](https://github.com/etcd-io/etcd/pull/9119).
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.13](https://github.com/etcd-io/etcd/releases/tag/v3.2.13) (2018-01-02)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.12...v3.2.13) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Remove [verbose error messages on stream cancel and gRPC info-level logs](https://github.com/etcd-io/etcd/pull/9080) in server-side.
- Fix [gRPC server panic on `GracefulStop` TLS-enabled server](https://github.com/etcd-io/etcd/pull/8987).
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.12](https://github.com/etcd-io/etcd/releases/tag/v3.2.12) (2017-12-20)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.11...v3.2.12) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Dependency
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases/tag) from [**`v1.7.4`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.4) to [**`v1.7.5`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5).
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) from [**`v1.3`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3) to [**`v1.3.0`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3.0).
### etcd server
- Fix [error message of `Revision` compactor](https://github.com/etcd-io/etcd/pull/8999) in server-side.
### client v3
- Add [`MaxCallSendMsgSize` and `MaxCallRecvMsgSize`](https://github.com/etcd-io/etcd/pull/9047) fields to [`clientv3.Config`](https://godoc.org/github.com/etcd-io/etcd/clientv3#Config).
- Fix [exceeded response size limit error in client-side](https://github.com/etcd-io/etcd/issues/9043).
- Address [kubernetes#51099](https://github.com/kubernetes/kubernetes/issues/51099).
- In previous versions(v3.2.10, v3.2.11), client response size was limited to only 4 MiB.
- `MaxCallSendMsgSize` default value is 2 MiB, if not configured.
- `MaxCallRecvMsgSize` default value is `math.MaxInt32`, if not configured.
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.11](https://github.com/etcd-io/etcd/releases/tag/v3.2.11) (2017-12-05)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.10...v3.2.11) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Dependency
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases/tag) from [**`v1.7.3`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.3) to [**`v1.7.4`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.4).
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- Log [more details on TLS handshake failures](https://github.com/etcd-io/etcd/pull/8952/files).
### client v3
- Fix racey grpc-go's server handler transport `WriteStatus` call to prevent [TLS-enabled etcd server crash](https://github.com/etcd-io/etcd/issues/8904).
- Add [gRPC RPC failure warnings](https://github.com/etcd-io/etcd/pull/8939) to help debug such issues in the future.
### Documentation
- Remove `--listen-metrics-urls` flag in monitoring document (non-released in `v3.2.x`, planned for `v3.3.x`).
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.10](https://github.com/etcd-io/etcd/releases/tag/v3.2.10) (2017-11-16)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.9...v3.2.10) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Dependency
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases/tag) from [**`v1.2.1`**](https://github.com/grpc/grpc-go/releases/tag/v1.2.1) to [**`v1.7.3`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.3).
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) from [**`v1.2.0`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.2.0) to [**`v1.3`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3).
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- Revert [discovery SRV auth `ServerName` with `*.{ROOT_DOMAIN}`](https://github.com/etcd-io/etcd/pull/8651) to support non-wildcard subject alternative names in the certs (see [issue #8445](https://github.com/etcd-io/etcd/issues/8445) for more contexts).
- For instance, `etcd --discovery-srv=etcd.local` will only authenticate peers/clients when the provided certs have root domain `etcd.local` (**not `*.etcd.local`**) as an entry in Subject Alternative Name (SAN) field.
### etcd server
- Replace backend key-value database `boltdb/bolt` with [`coreos/bbolt`](https://github.com/coreos/bbolt/releases) to address [backend database size issue](https://github.com/etcd-io/etcd/issues/8009).
### client v3
- Rewrite balancer to handle [network partitions](https://github.com/etcd-io/etcd/issues/8711).
### Go
- Compile with [*Go 1.8.5*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.9](https://github.com/etcd-io/etcd/releases/tag/v3.2.9) (2017-10-06)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.8...v3.2.9) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- Update `golang.org/x/crypto/bcrypt` (see [golang/crypto@6c586e1](https://github.com/golang/crypto/commit/6c586e17d90a7d08bbbc4069984180dce3b04117)).
- Fix discovery SRV bootstrapping to [authenticate `ServerName` with `*.{ROOT_DOMAIN}`](https://github.com/etcd-io/etcd/pull/8651), in order to support sub-domain wildcard matching (see [issue #8445](https://github.com/etcd-io/etcd/issues/8445) for more contexts).
- For instance, `etcd --discovery-srv=etcd.local` will only authenticate peers/clients when the provided certs have root domain `*.etcd.local` as an entry in Subject Alternative Name (SAN) field.
### Go
- Compile with [*Go 1.8.4*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.8](https://github.com/etcd-io/etcd/releases/tag/v3.2.8) (2017-09-29)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.7...v3.2.8) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### client v2
- Fix v2 client failover to next endpoint on mutable operation.
### gRPC Proxy
- Handle [`KeysOnly` flag](https://github.com/etcd-io/etcd/pull/8552).
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.7](https://github.com/etcd-io/etcd/releases/tag/v3.2.7) (2017-09-01)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.6...v3.2.7) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Security, Authentication
- Fix [server-side auth so concurrent auth operations do not return old revision error](https://github.com/etcd-io/etcd/pull/8306).
### client v3
- Fix [`concurrency/stm` Put with serializable snapshot](https://github.com/etcd-io/etcd/pull/8439).
- Use store revision from first fetch to resolve write conflicts instead of modified revision.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.6](https://github.com/etcd-io/etcd/releases/tag/v3.2.6) (2017-08-21)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.5...v3.2.6) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Fix watch restore from snapshot.
- Fix multiple URLs for `--listen-peer-urls` flag.
- Add `--enable-pprof` flag to etcd configuration file format.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Fix `etcd_debugging_mvcc_keys_total` inconsistency.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.5](https://github.com/etcd-io/etcd/releases/tag/v3.2.5) (2017-08-04)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.4...v3.2.5) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcdctl v3
- Return non-zero exit code on unhealthy `endpoint health`.
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- [Server supports reverse-lookup on wildcard DNS `SAN`](https://github.com/etcd-io/etcd/pull/8281). For instance, if peer cert contains only DNS names (no IP addresses) in Subject Alternative Name (SAN) field, server first reverse-lookups the remote IP address to get a list of names mapping to that address (e.g. `nslookup IPADDR`). Then accepts the connection if those names have a matching name with peer cert's DNS names (either by exact or wildcard match). If none is matched, server forward-lookups each DNS entry in peer cert (e.g. look up `example.default.svc` when the entry is `*.example.default.svc`), and accepts connection only when the host's resolved addresses have the matching IP address with the peer's remote IP address. For example, peer B's CSR (with `cfssl`) SAN field is `["*.example.default.svc", "*.example.default.svc.cluster.local"]` when peer B's remote IP address is `10.138.0.2`. When peer B tries to join the cluster, peer A reverse-lookup the IP `10.138.0.2` to get the list of host names. And either exact or wildcard match the host names with peer B's cert DNS names in Subject Alternative Name (SAN) field. If none of reverse/forward lookups worked, it returns an error `"tls: "10.138.0.2" does not match any of DNSNames ["*.example.default.svc","*.example.default.svc.cluster.local"]`. See [issue#8268](https://github.com/etcd-io/etcd/issues/8268) for more detail.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Fix unreachable `/metrics` endpoint when `--enable-v2=false`.
### gRPC Proxy
- Handle [`PrevKv` flag](https://github.com/etcd-io/etcd/pull/8366).
### Other
- Add container registry `gcr.io/etcd-development/etcd`.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.4](https://github.com/etcd-io/etcd/releases/tag/v3.2.4) (2017-07-19)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.3...v3.2.4) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Do not block on active client stream when stopping server
### gRPC proxy
- Fix gRPC proxy Snapshot RPC error handling
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.3](https://github.com/etcd-io/etcd/releases/tag/v3.2.3) (2017-07-14)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.2...v3.2.3) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### client v3
- Let clients establish unlimited streams
### Other
- Tag docker images with minor versions
- e.g. `docker pull quay.io/coreos/etcd:v3.2` to fetch latest v3.2 versions
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.2](https://github.com/etcd-io/etcd/releases/tag/v3.2.2) (2017-07-07)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.1...v3.2.2) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Rate-limit lease revoke on expiration.
- Extend leases on promote to avoid queueing effect on lease expiration.
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- [Server accepts connections if IP matches, without checking DNS entries](https://github.com/etcd-io/etcd/pull/8223). For instance, if peer cert contains IP addresses and DNS names in Subject Alternative Name (SAN) field, and the remote IP address matches one of those IP addresses, server just accepts connection without further checking the DNS names. For example, peer B's CSR (with `cfssl`) SAN field is `["invalid.domain", "10.138.0.2"]` when peer B's remote IP address is `10.138.0.2` and `invalid.domain` is a invalid host. When peer B tries to join the cluster, peer A successfully authenticates B, since Subject Alternative Name (SAN) field has a valid matching IP address. See [issue#8206](https://github.com/etcd-io/etcd/issues/8206) for more detail.
### etcd server
- Accept connection with matched IP SAN but no DNS match.
- Don't check DNS entries in certs if there's a matching IP.
### gRPC gateway
- Use user-provided listen address to connect to gRPC gateway.
- `net.Listener` rewrites IPv4 0.0.0.0 to IPv6 [::], breaking IPv6 disabled hosts.
- Only v3.2.0, v3.2.1 are affected.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.1](https://github.com/etcd-io/etcd/releases/tag/v3.2.1) (2017-06-23)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.0...v3.2.1) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### etcd server
- Fix backend database in-memory index corruption issue on restore (only 3.2.0 is affected).
### gRPC gateway
- Fix Txn marshaling.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Fix backend database size debugging metrics.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>
## [v3.2.0](https://github.com/etcd-io/etcd/releases/tag/v3.2.0) (2017-06-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.1.0...v3.2.0) and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.2 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md).**
### Improved
- Improve backend read concurrency.
### Breaking Changes
- Increased [`--snapshot-count` default value from 10,000 to 100,000](https://github.com/etcd-io/etcd/pull/7160).
- Higher snapshot count means it holds Raft entries in memory for longer before discarding old entries.
- It is a trade-off between less frequent snapshotting and [higher memory usage](https://github.com/kubernetes/kubernetes/issues/60589#issuecomment-371977156).
- User lower `--snapshot-count` value for lower memory usage.
- User higher `--snapshot-count` value for better availabilities of slow followers (less frequent snapshots from leader).
- `clientv3.Lease.TimeToLive` returns `LeaseTimeToLiveResponse.TTL == -1` on lease not found.
- `clientv3.NewFromConfigFile` is moved to `clientv3/yaml.NewConfig`.
- `embed.Etcd.Peers` field is now `[]*peerListener`.
- Rejects domains names for `--listen-peer-urls` and `--listen-client-urls` (3.1 only prints out warnings), since [domain name is invalid for network interface binding](https://github.com/etcd-io/etcd/issues/6336).
### Dependency
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) from [**`v1.0.4`**](https://github.com/grpc/grpc-go/releases/tag/v1.0.4) to [**`v1.2.1`**](https://github.com/grpc/grpc-go/releases/tag/v1.2.1).
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) to [**`v1.2.0`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.2.0).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.2.27/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_disk_backend_snapshot_duration_seconds`](https://github.com/etcd-io/etcd/pull/7892)
- Add `etcd_debugging_server_lease_expired_total` metrics.
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- [TLS certificates get reloaded on every client connection](https://github.com/etcd-io/etcd/pull/7829). This is useful when replacing expiry certs without stopping etcd servers; it can be done by overwriting old certs with new ones. Refreshing certs for every connection should not have too much overhead, but can be improved in the future, with caching layer. Example tests can be found [here](https://github.com/etcd-io/etcd/blob/b041ce5d514a4b4aaeefbffb008f0c7570a18986/integration/v3_grpc_test.go#L1601-L1757).
- [Server denies incoming peer certs with wrong IP `SAN`](https://github.com/etcd-io/etcd/pull/7687). For instance, if peer cert contains any IP addresses in Subject Alternative Name (SAN) field, server authenticates a peer only when the remote IP address matches one of those IP addresses. This is to prevent unauthorized endpoints from joining the cluster. For example, peer B's CSR (with `cfssl`) SAN field is `["*.example.default.svc", "*.example.default.svc.cluster.local", "10.138.0.27"]` when peer B's actual IP address is `10.138.0.2`, not `10.138.0.27`. When peer B tries to join the cluster, peer A will reject B with the error `x509: certificate is valid for 10.138.0.27, not 10.138.0.2`, because B's remote IP address does not match the one in Subject Alternative Name (SAN) field.
- [Server resolves TLS `DNSNames` when checking `SAN`](https://github.com/etcd-io/etcd/pull/7767). For instance, if peer cert contains only DNS names (no IP addresses) in Subject Alternative Name (SAN) field, server authenticates a peer only when forward-lookups (`dig b.com`) on those DNS names have matching IP with the remote IP address. For example, peer B's CSR (with `cfssl`) SAN field is `["b.com"]` when peer B's remote IP address is `10.138.0.2`. When peer B tries to join the cluster, peer A looks up the incoming host `b.com` to get the list of IP addresses (e.g. `dig b.com`). And rejects B if the list does not contain the IP `10.138.0.2`, with the error `tls: 10.138.0.2 does not match any of DNSNames ["b.com"]`.
- Auth support JWT token.
### etcd server
- RPCs
- Add Election, Lock service.
- Native client `etcdserver/api/v3client`
- client "embedded" in the server.
- Logging, monitoring
- Server warns large snapshot operations.
- Add `etcd --enable-v2` flag to enable v2 API server.
- `etcd --enable-v2=true` by default.
- Add `etcd --auth-token` flag.
- v3.2 compactor runs [every hour](https://github.com/etcd-io/etcd/pull/7875).
- Compactor only supports periodic compaction.
- Compactor continues to record latest revisions every 5-minute.
- For every hour, it uses the last revision that was fetched before compaction period, from the revision records that were collected every 5-minute.
- That is, for every hour, compactor discards historical data created before compaction period.
- The retention window of compaction period moves to next hour.
- For instance, when hourly writes are 100 and `--auto-compaction-retention=10`, v3.1 compacts revision 1000, 2000, and 3000 for every 10-hour, while v3.2 compacts revision 1000, 1100, and 1200 for every 1-hour.
- If compaction succeeds or requested revision has already been compacted, it resets period timer and removes used compacted revision from historical revision records (e.g. start next revision collect and compaction from previously collected revisions).
- If compaction fails, it retries in 5 minutes.
- Allow snapshot over 512MB.
### client v3
- STM prefetching.
- Add namespace feature.
- Add `ErrOldCluster` with server version checking.
- Translate `WithPrefix()` into `WithFromKey()` for empty key.
### etcdctl v3
- Add `check perf` command.
- Add `etcdctl --from-key` flag to role grant-permission command.
- `lock` command takes an optional command to execute.
### gRPC Proxy
- Proxy endpoint discovery.
- Namespaces.
- Coalesce lease requests.
### etcd gateway
- Support [DNS SRV priority](https://github.com/etcd-io/etcd/pull/7882) for [smart proxy routing](https://github.com/etcd-io/etcd/issues/4378).
### Other
- v3 client
- concurrency package's elections updated to match RPC interfaces.
- let client dial endpoints not in the balancer.
- Release
- Annotate acbuild with supports-systemd-notify.
- Add `nsswitch.conf` to Docker container image.
- Add ppc64le, arm64(experimental) builds.
### Go
- Compile with [*Go 1.8.3*](https://golang.org/doc/devel/release.html#go1.8).
<hr>

View File

@ -1,734 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.2](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.2.md).
The minimum recommended etcd versions to run in **production** are 3.1.11+, 3.2.26+, and 3.3.11+.
<hr>
## [v3.3.14](https://github.com/etcd-io/etcd/releases/tag/v3.3.14) (2019-TBD)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.13...v3.3.14) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### etcd server
- Fix [race condition in `rafthttp` transport pause/resume](https://github.com/etcd-io/etcd/pull/10826).
### client v3
- Fix [gRPC panic "send on closed channel](https://github.com/etcd-io/etcd/issues/9956) by upgrading [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) from [**`v1.7.5`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5) to [**`v1.23.0`**](https://github.com/grpc/grpc-go/releases/tag/v1.23.0).
- Rewrite [client balancer](https://github.com/etcd-io/etcd/pull/9860) with [new gRPC balancer interface](https://github.com/etcd-io/etcd/issues/9106).
- Upgrade [gRPC to v1.23.0](https://github.com/etcd-io/etcd/pull/10911).
- Improve [client balancer failover against secure endpoints](https://github.com/etcd-io/etcd/pull/10911).
- Fix ["kube-apiserver 1.13.x refuses to work when first etcd-server is not available" (kubernetes#72102)](https://github.com/kubernetes/kubernetes/issues/72102).
### etcdctl v3
- Add [`etcdctl endpoint health --write-out` support](https://github.com/etcd-io/etcd/pull/9540).
- Previously, [`etcdctl endpoint health --write-out json` did not work](https://github.com/etcd-io/etcd/issues/9532).
- The command output is changed. Previously, if endpoint is unreachable, the command output is
"\<endpoint\> is unhealthy: failed to connect: \<error message\>". This change unified the error message, all error types
now have the same output "\<endpoint\> is unhealthy: failed to commit proposal: \<error message\>".
- Add [missing newline in `etcdctl endpoint health`](https://github.com/etcd-io/etcd/pull/10793).
<hr>
## [v3.3.13](https://github.com/etcd-io/etcd/releases/tag/v3.3.13) (2019-05-02)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.12...v3.3.13) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Improve [heartbeat send failure logging](https://github.com/etcd-io/etcd/pull/10663).
- Add [`Verify` function to perform corruption check on WAL contents](https://github.com/etcd-io/etcd/pull/10603).
### Metrics, Monitoring
See [List of metrics](https://etcd.readthedocs.io/en/latest/operate.html#v3-3) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Fix bug where [db_compaction_total_duration_milliseconds metric incorrectly measured duration as 0](https://github.com/etcd-io/etcd/pull/10646).
### client v3
- Fix [`(*Client).Endpoints()` method race condition](https://github.com/etcd-io/etcd/pull/10595).
### Package `wal`
- Add [`Verify` function to perform corruption check on WAL contents](https://github.com/etcd-io/etcd/pull/10603).
### Dependency
- Migrate [`github.com/ugorji/go/codec`](https://github.com/ugorji/go/releases) to [**`github.com/json-iterator/go`**](https://github.com/json-iterator/go) (See [#10667](https://github.com/etcd-io/etcd/pull/10667) for more).
- Migrate [`github.com/ghodss/yaml`](https://github.com/ghodss/yaml/releases) to [**`sigs.k8s.io/yaml`**](https://github.com/kubernetes-sigs/yaml) (See [#10718](https://github.com/etcd-io/etcd/pull/10718) for more).
### Go
- Compile with [*Go 1.10.8*](https://golang.org/doc/devel/release.html#go1.10).
<hr>
## [v3.3.12](https://github.com/etcd-io/etcd/releases/tag/v3.3.12) (2019-02-07)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.11...v3.3.12) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### etcdctl
- [Strip out insecure endpoints from DNS SRV records when using discovery](https://github.com/etcd-io/etcd/pull/10443) with etcdctl v2
### Go
- Compile with [*Go 1.10.8*](https://golang.org/doc/devel/release.html#go1.10).
<hr>
## [v3.3.11](https://github.com/etcd-io/etcd/releases/tag/v3.3.11) (2019-01-11)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.10...v3.3.11) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### gRPC Proxy
- Fix [memory leak in cache layer](https://github.com/etcd-io/etcd/pull/10327).
### Security, Authentication
- Disable [CommonName authentication for gRPC-gateway](https://github.com/etcd-io/etcd/pull/10366) gRPC-gateway proxy requests to etcd server use the etcd client server TLS certificate. If that certificate contains CommonName we do not want to use that for authentication as it could lead to permission escalation.
### Go
- Compile with [*Go 1.10.7*](https://golang.org/doc/devel/release.html#go1.10).
<hr>
## [v3.3.10](https://github.com/etcd-io/etcd/releases/tag/v3.3.10) (2018-10-10)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.9...v3.3.10) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Improve ["became inactive" warning log](https://github.com/etcd-io/etcd/pull/10024), which indicates message send to a peer failed.
- Improve [read index wait timeout warning log](https://github.com/etcd-io/etcd/pull/10026), which indicates that local node might have slow network.
- Add [gRPC interceptor for debugging logs](https://github.com/etcd-io/etcd/pull/9990); enable `etcd --debug` flag to see per-request debug information.
- Add [consistency check in snapshot status](https://github.com/etcd-io/etcd/pull/10109). If consistency check on snapshot file fails, `snapshot status` returns `"snapshot file integrity check failed..."` error.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.3.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Improve [`etcd_network_peer_round_trip_time_seconds`](https://github.com/etcd-io/etcd/pull/10155) Prometheus metric to track leader heartbeats.
- Previously, it only samples the TCP connection for snapshot messages.
- Add [`etcd_snap_db_fsync_duration_seconds_count`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_snap_db_save_total_duration_seconds_bucket`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_server_id`](https://github.com/etcd-io/etcd/pull/9998) Prometheus metric.
- Add [`etcd_server_health_success`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_health_failures`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_read_indexes_failed_total`](https://github.com/etcd-io/etcd/pull/10094) Prometheus metric.
### client v3
- Fix logic on [release lock key if cancelled](https://github.com/etcd-io/etcd/pull/10153) in `clientv3/concurrency` package.
### Go
- Compile with [*Go 1.10.4*](https://golang.org/doc/devel/release.html#go1.10).
<hr>
## [v3.3.9](https://github.com/etcd-io/etcd/releases/tag/v3.3.9) (2018-07-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.8...v3.3.9) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Improve [Raft Read Index timeout warning messages](https://github.com/etcd-io/etcd/pull/9897).
### Security, Authentication
- Compile with [*Go 1.10.3*](https://golang.org/doc/devel/release.html#go1.10) to support [crypto/x509 "Name Constraints"](https://github.com/etcd-io/etcd/issues/9912).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.3.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_go_version`](https://github.com/etcd-io/etcd/pull/9957) Prometheus metric.
- Add [`etcd_server_heartbeat_send_failures_total`](https://github.com/etcd-io/etcd/pull/9940) Prometheus metric.
- Add [`etcd_server_slow_apply_total`](https://github.com/etcd-io/etcd/pull/9940) Prometheus metric.
- Add [`etcd_disk_backend_defrag_duration_seconds`](https://github.com/etcd-io/etcd/pull/9940) Prometheus metric.
- Add [`etcd_mvcc_hash_duration_seconds`](https://github.com/etcd-io/etcd/pull/9940) Prometheus metric.
- Add [`etcd_mvcc_hash_rev_duration_seconds`](https://github.com/etcd-io/etcd/pull/9940) Prometheus metric.
- Add [`etcd_server_slow_read_indexes_total`](https://github.com/etcd-io/etcd/pull/9897) Prometheus metric.
- Add [`etcd_server_quota_backend_bytes`](https://github.com/etcd-io/etcd/pull/9820) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
- Add [`etcd_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819) Prometheus metric.
- In addition to [`etcd_debugging_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819).
- Add [`etcd_mvcc_db_total_size_in_use_in_bytes`](https://github.com/etcd-io/etcd/pull/9256) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
### client v3
- Fix [lease keepalive interval updates when response queue is full](https://github.com/etcd-io/etcd/pull/9952).
- If `<-chan *clientv3LeaseKeepAliveResponse` from `clientv3.Lease.KeepAlive` was never consumed or channel is full, client was [sending keepalive request every 500ms](https://github.com/etcd-io/etcd/issues/9911) instead of expected rate of every "TTL / 3" duration.
### Go
- Compile with [*Go 1.10.3*](https://golang.org/doc/devel/release.html#go1.10).
<hr>
## [v3.3.8](https://github.com/etcd-io/etcd/releases/tag/v3.3.8) (2018-06-15)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.7...v3.3.8) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Improve [slow request apply warning log](https://github.com/etcd-io/etcd/pull/9288).
- e.g. `read-only range request "key:\"/a\" range_end:\"/b\" " with result "range_response_count:3 size:96" took too long (97.966µs) to execute`.
- Redact [request value field](https://github.com/etcd-io/etcd/pull/9822).
- Provide [response size](https://github.com/etcd-io/etcd/pull/9826).
- Add [backoff on watch retries on transient errors](https://github.com/etcd-io/etcd/pull/9840).
### Go
- Compile with [*Go 1.9.7*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.7](https://github.com/etcd-io/etcd/releases/tag/v3.3.7) (2018-06-06)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.6...v3.3.7) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Security, Authentication
- Support TLS cipher suite whitelisting.
- To block [weak cipher suites](https://github.com/etcd-io/etcd/issues/8320).
- TLS handshake fails when client hello is requested with invalid cipher suites.
- Add [`etcd --cipher-suites`](https://github.com/etcd-io/etcd/pull/9801) flag.
- If empty, Go auto-populates the list.
### etcdctl v3
- Fix [`etcdctl move-leader` command for TLS-enabled endpoints](https://github.com/etcd-io/etcd/pull/9807).
### Go
- Compile with [*Go 1.9.6*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.6](https://github.com/etcd-io/etcd/releases/tag/v3.3.6) (2018-05-31)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.5...v3.3.6) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### etcd server
- Allow [empty auth token](https://github.com/etcd-io/etcd/pull/9369).
- Previously, when auth token is an empty string, it returns [`failed to initialize the etcd server: auth: invalid auth options` error](https://github.com/etcd-io/etcd/issues/9349).
- Fix [auth storage panic on server lease revoke routine with JWT token](https://github.com/etcd-io/etcd/issues/9695).
- Fix [`mvcc` server panic from restore operation](https://github.com/etcd-io/etcd/pull/9775).
- Let's assume that a watcher had been requested with a future revision X and sent to node A that became network-partitioned thereafter. Meanwhile, cluster makes progress. Then when the partition gets removed, the leader sends a snapshot to node A. Previously if the snapshot's latest revision is still lower than the watch revision X, **etcd server panicked** during snapshot restore operation.
- Now, this server-side panic has been fixed.
### Go
- Compile with [*Go 1.9.6*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.5](https://github.com/etcd-io/etcd/releases/tag/v3.3.5) (2018-05-09)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.4...v3.3.5) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### etcdctl v3
- Fix [`etcdctl watch [key] [range_end] -- [exec-command…]`](https://github.com/etcd-io/etcd/pull/9688) parsing.
- Previously, `ETCDCTL_API=3 ./bin/etcdctl watch foo -- echo watch event received` panicked.
### Go
- Compile with [*Go 1.9.6*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.4](https://github.com/etcd-io/etcd/releases/tag/v3.3.4) (2018-04-24)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.3...v3.3.4) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.3.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_server_is_leader`](https://github.com/etcd-io/etcd/pull/9587) Prometheus metric.
- Fix [`etcd_debugging_server_lease_expired_total`](https://github.com/etcd-io/etcd/pull/9557) Prometheus metric.
- Fix [race conditions in v2 server stat collecting](https://github.com/etcd-io/etcd/pull/9562).
### Security, Authentication
- Fix [TLS reload](https://github.com/etcd-io/etcd/pull/9570) when [certificate SAN field only includes IP addresses but no domain names](https://github.com/etcd-io/etcd/issues/9541).
- In Go, server calls `(*tls.Config).GetCertificate` for TLS reload if and only if server's `(*tls.Config).Certificates` field is not empty, or `(*tls.ClientHelloInfo).ServerName` is not empty with a valid SNI from the client. Previously, etcd always populates `(*tls.Config).Certificates` on the initial client TLS handshake, as non-empty. Thus, client was always expected to supply a matching SNI in order to pass the TLS verification and to trigger `(*tls.Config).GetCertificate` to reload TLS assets.
- However, a certificate whose SAN field does [not include any domain names but only IP addresses](https://github.com/etcd-io/etcd/issues/9541) would request `*tls.ClientHelloInfo` with an empty `ServerName` field, thus failing to trigger the TLS reload on initial TLS handshake; this becomes a problem when expired certificates need to be replaced online.
- Now, `(*tls.Config).Certificates` is created empty on initial TLS client handshake, first to trigger `(*tls.Config).GetCertificate`, and then to populate rest of the certificates on every new TLS connection, even when client SNI is empty (e.g. cert only includes IPs).
### etcd server
- Add [`etcd --initial-election-tick-advance`](https://github.com/etcd-io/etcd/pull/9591) flag to configure initial election tick fast-forward.
- By default, `etcd --initial-election-tick-advance=true`, then local member fast-forwards election ticks to speed up "initial" leader election trigger.
- This benefits the case of larger election ticks. For instance, cross datacenter deployment may require longer election timeout of 10-second. If true, local node does not need wait up to 10-second. Instead, forwards its election ticks to 8-second, and have only 2-second left before leader election.
- Major assumptions are that: cluster has no active leader thus advancing ticks enables faster leader election. Or cluster already has an established leader, and rejoining follower is likely to receive heartbeats from the leader after tick advance and before election timeout.
- However, when network from leader to rejoining follower is congested, and the follower does not receive leader heartbeat within left election ticks, disruptive election has to happen thus affecting cluster availabilities.
- Now, this can be disabled by setting `--initial-election-tick-advance=false`.
- Disabling this would slow down initial bootstrap process for cross datacenter deployments. Make tradeoffs by configuring `etcd --initial-election-tick-advance` at the cost of slow initial bootstrap.
- If single-node, it advances ticks regardless.
- Address [disruptive rejoining follower node](https://github.com/etcd-io/etcd/issues/9333).
### Package `embed`
- Add [`embed.Config.InitialElectionTickAdvance`](https://github.com/etcd-io/etcd/pull/9591) to enable/disable initial election tick fast-forward.
- `embed.NewConfig()` would return `*embed.Config` with `InitialElectionTickAdvance` as true by default.
### Go
- Compile with [*Go 1.9.5*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.3](https://github.com/etcd-io/etcd/releases/tag/v3.3.3) (2018-03-29)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.2...v3.3.3) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Adjust [election timeout on server restart](https://github.com/etcd-io/etcd/pull/9415) to reduce [disruptive rejoining servers](https://github.com/etcd-io/etcd/issues/9333).
- Previously, etcd fast-forwards election ticks on server start, with only one tick left for leader election. This is to speed up start phase, without having to wait until all election ticks elapse. Advancing election ticks is useful for cross datacenter deployments with larger election timeouts. However, it was affecting cluster availability if the last tick elapses before leader contacts the restarted node.
- Now, when etcd restarts, it adjusts election ticks with more than one tick left, thus more time for leader to prevent disruptive restart.
- Adjust [periodic compaction retention window](https://github.com/etcd-io/etcd/pull/9485).
- e.g. `etcd --auto-compaction-mode=revision --auto-compaction-retention=1000` automatically `Compact` on `"latest revision" - 1000` every 5-minute (when latest revision is 30000, compact on revision 29000).
- e.g. Previously, `etcd --auto-compaction-mode=periodic --auto-compaction-retention=72h` automatically `Compact` with 72-hour retention windown for every 7.2-hour. **Now, `Compact` happens, for every 1-hour but still with 72-hour retention window.**
- e.g. Previously, `etcd --auto-compaction-mode=periodic --auto-compaction-retention=30m` automatically `Compact` with 30-minute retention windown for every 3-minute. **Now, `Compact` happens, for every 30-minute but still with 30-minute retention window.**
- Periodic compactor keeps recording latest revisions for every compaction period when given period is less than 1-hour, or for every 1-hour when given compaction period is greater than 1-hour (e.g. 1-hour when `etcd --auto-compaction-mode=periodic --auto-compaction-retention=24h`).
- For every compaction period or 1-hour, compactor uses the last revision that was fetched before compaction period, to discard historical data.
- The retention window of compaction period moves for every given compaction period or hour.
- For instance, when hourly writes are 100 and `etcd --auto-compaction-mode=periodic --auto-compaction-retention=24h`, `v3.2.x`, `v3.3.0`, `v3.3.1`, and `v3.3.2` compact revision 2400, 2640, and 2880 for every 2.4-hour, while `v3.3.3` *or later* compacts revision 2400, 2500, 2600 for every 1-hour.
- Futhermore, when `etcd --auto-compaction-mode=periodic --auto-compaction-retention=30m` and writes per minute are about 1000, `v3.3.0`, `v3.3.1`, and `v3.3.2` compact revision 30000, 33000, and 36000, for every 3-minute, while `v3.3.3` *or later* compacts revision 30000, 60000, and 90000, for every 30-minute.
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.3.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add missing [`etcd_network_peer_sent_failures_total` count](https://github.com/etcd-io/etcd/pull/9437).
### Go
- Compile with [*Go 1.9.5*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.2](https://github.com/etcd-io/etcd/releases/tag/v3.3.2) (2018-03-08)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.1...v3.3.2) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### etcd server
- Fix [server panic on invalid Election Proclaim/Resign HTTP(S) requests](https://github.com/etcd-io/etcd/pull/9379).
- Previously, wrong-formatted HTTP requests to Election API could trigger panic in etcd server.
- e.g. `curl -L http://localhost:2379/v3/election/proclaim -X POST -d '{"value":""}'`, `curl -L http://localhost:2379/v3/election/resign -X POST -d '{"value":""}'`.
- Fix [revision-based compaction retention parsing](https://github.com/etcd-io/etcd/pull/9339).
- Previously, `etcd --auto-compaction-mode revision --auto-compaction-retention 1` was [translated to revision retention 3600000000000](https://github.com/etcd-io/etcd/issues/9337).
- Now, `etcd --auto-compaction-mode revision --auto-compaction-retention 1` is correctly parsed as revision retention 1.
- Prevent [overflow by large `TTL` values for `Lease` `Grant`](https://github.com/etcd-io/etcd/pull/9399).
- `TTL` parameter to `Grant` request is unit of second.
- Leases with too large `TTL` values exceeding `math.MaxInt64` [expire in unexpected ways](https://github.com/etcd-io/etcd/issues/9374).
- Server now returns `rpctypes.ErrLeaseTTLTooLarge` to client, when the requested `TTL` is larger than *9,000,000,000 seconds* (which is >285 years).
- Again, etcd `Lease` is meant for short-periodic keepalives or sessions, in the range of seconds or minutes. Not for hours or days!
- Enable etcd server [`raft.Config.CheckQuorum` when starting with `ForceNewCluster`](https://github.com/etcd-io/etcd/pull/9347).
### Proxy v2
- Fix [v2 proxy leaky HTTP requests](https://github.com/etcd-io/etcd/pull/9336).
### Go
- Compile with [*Go 1.9.4*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.1](https://github.com/etcd-io/etcd/releases/tag/v3.3.1) (2018-02-12)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0...v3.3.1) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
### Improved
- Add [warnings on requests taking too long](https://github.com/etcd-io/etcd/pull/9288).
- e.g. `etcdserver: read-only range request "key:\"\\000\" range_end:\"\\000\" " took too long [3.389041388s] to execute`
### etcd server
- Fix [`mvcc` "unsynced" watcher restore operation](https://github.com/etcd-io/etcd/pull/9281).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/etcd-io/etcd/issues/9086).
- A node gets network partitioned with a watcher on a future revision, and falls behind receiving a leader snapshot after partition gets removed. When applying this snapshot, etcd watch storage moves current synced watchers to unsynced since sync watchers might have become stale during network partition. And reset synced watcher group to restart watcher routines. Previously, there was a bug when moving from synced watcher group to unsynced, thus client would miss events when the watcher was requested to the network-partitioned node.
### Go
- Compile with [*Go 1.9.4*](https://golang.org/doc/devel/release.html#go1.9).
<hr>
## [v3.3.0](https://github.com/etcd-io/etcd/releases/tag/v3.3.0) (2018-02-01)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.2.0...v3.3.0) and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.3 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md).**
- [v3.3.0-rc.4](https://github.com/etcd-io/etcd/releases/tag/v3.3.0-rc.4) (2018-01-22), see [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0-rc.3...v3.3.0-rc.4).
- [v3.3.0-rc.3](https://github.com/etcd-io/etcd/releases/tag/v3.3.0-rc.3) (2018-01-17), see [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0-rc.2...v3.3.0-rc.3).
- [v3.3.0-rc.2](https://github.com/etcd-io/etcd/releases/tag/v3.3.0-rc.2) (2018-01-11), see [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0-rc.1...v3.3.0-rc.2).
- [v3.3.0-rc.1](https://github.com/etcd-io/etcd/releases/tag/v3.3.0-rc.1) (2018-01-02), see [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0-rc.0...v3.3.0-rc.1).
- [v3.3.0-rc.0](https://github.com/etcd-io/etcd/releases/tag/v3.3.0-rc.0) (2017-12-20), see [code changes](https://github.com/etcd-io/etcd/compare/v3.2.0...v3.3.0-rc.0).
### Improved
- Use [`coreos/bbolt`](https://github.com/coreos/bbolt/releases) to replace [`boltdb/bolt`](https://github.com/boltdb/bolt#project-status).
- Fix [etcd database size grows until `mvcc: database space exceeded`](https://github.com/etcd-io/etcd/issues/8009).
- [Support database size larger than 8GiB](https://github.com/etcd-io/etcd/pull/7525) (8GiB is now a suggested maximum size for normal environments)
- [Reduce memory allocation](https://github.com/etcd-io/etcd/pull/8428) on [Range operations](https://github.com/etcd-io/etcd/pull/8475).
- [Rate limit](https://github.com/etcd-io/etcd/pull/8099) and [randomize](https://github.com/etcd-io/etcd/pull/8101) lease revoke on restart or leader elections.
- Prevent [spikes in Raft proposal rate](https://github.com/etcd-io/etcd/issues/8096).
- Support `clientv3` balancer failover under [network faults/partitions](https://github.com/etcd-io/etcd/issues/8711).
- Better warning on [mismatched `etcd --initial-cluster`](https://github.com/etcd-io/etcd/pull/8083) flag.
- etcd compares `etcd --initial-advertise-peer-urls` against corresponding `etcd --initial-cluster` URLs with forward-lookup.
- If resolved IP addresses of `etcd --initial-advertise-peer-urls` and `etcd --initial-cluster` do not match (e.g. [due to DNS error](https://github.com/etcd-io/etcd/pull/9210)), etcd will exit with errors.
- v3.2 error: `etcd --initial-cluster must include s1=https://s1.test:2380 given --initial-advertise-peer-urls=https://s1.test:2380`.
- v3.3 error: `failed to resolve https://s1.test:2380 to match --initial-cluster=s1=https://s1.test:2380 (failed to resolve "https://s1.test:2380" (error ...))`.
### Breaking Changes
- Require [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) [**`v1.7.4`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.4) or [**`v1.7.5`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5).
- Deprecate [`metadata.Incoming/OutgoingContext`](https://github.com/etcd-io/etcd/pull/7896).
- Deprecate `grpclog.Logger`, upgrade to [`grpclog.LoggerV2`](https://github.com/etcd-io/etcd/pull/8533).
- Deprecate [`grpc.ErrClientConnTimeout`](https://github.com/etcd-io/etcd/pull/8505) errors in `clientv3`.
- Use [`MaxRecvMsgSize` and `MaxSendMsgSize`](https://github.com/etcd-io/etcd/pull/8437) to limit message size, in etcd server.
- Translate [gRPC status error in v3 client `Snapshot` API](https://github.com/etcd-io/etcd/pull/9038).
- v3 `etcdctl` [`lease timetolive LEASE_ID`](https://github.com/etcd-io/etcd/issues/9028) on expired lease now prints [`"lease LEASE_ID already expired"`](https://github.com/etcd-io/etcd/pull/9047).
- <=3.2 prints `"lease LEASE_ID granted with TTL(0s), remaining(-1s)"`.
- Replace [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) endpoint `/v3alpha` with [`/v3beta`](https://github.com/etcd-io/etcd/pull/8880).
- To deprecate [`/v3alpha`](https://github.com/etcd-io/etcd/issues/8125) in v3.4.
- In v3.3, `curl -L http://localhost:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` still works as a fallback to `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'`, but `curl -L http://localhost:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` won't work in v3.4. Use `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
- Change `etcd --auto-compaction-retention` flag to [accept string values](https://github.com/etcd-io/etcd/pull/8563) with [finer granularity](https://github.com/etcd-io/etcd/issues/8503).
- Now that `etcd --auto-compaction-retention` accepts string values, etcd configuration YAML file `auto-compaction-retention` field must be changed to `string` type.
- Previously, `--config-file etcd.config.yaml` can have `auto-compaction-retention: 24` field, now must be `auto-compaction-retention: "24"` or `auto-compaction-retention: "24h"`.
- If configured as `etcd --auto-compaction-mode periodic --auto-compaction-retention "24h"`, the time duration value for `etcd --auto-compaction-retention` flag must be valid for [`time.ParseDuration`](https://golang.org/pkg/time/#ParseDuration) function in Go.
### Dependency
- Upgrade [`boltdb/bolt`](https://github.com/boltdb/bolt#project-status) from [**`v1.3.0`**](https://github.com/boltdb/bolt/releases/tag/v1.3.0) to [`coreos/bbolt`](https://github.com/coreos/bbolt/releases) [**`v1.3.1-coreos.6`**](https://github.com/coreos/bbolt/releases/tag/v1.3.1-coreos.6).
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) from [**`v1.2.1`**](https://github.com/grpc/grpc-go/releases/tag/v1.2.1) to [**`v1.7.5`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5).
- Upgrade [`github.com/ugorji/go/codec`](https://github.com/ugorji/go) to [**`v1.1`**](https://github.com/ugorji/go/releases/tag/v1.1), and [regenerate v2 `client`](https://github.com/etcd-io/etcd/pull/8721).
- Upgrade [`github.com/ugorji/go/codec`](https://github.com/ugorji/go) to [**`ugorji/go@54210f4e0`**](https://github.com/ugorji/go/commit/54210f4e076c57f351166f0ed60e67d3fca57a36), and [regenerate v2 `client`](https://github.com/etcd-io/etcd/pull/8574).
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) from [**`v1.2.2`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.2.2) to [**`v1.3.0`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3.0).
- Upgrade [`golang.org/x/crypto/bcrypt`](https://github.com/golang/crypto) to [**`golang/crypto@6c586e17d`**](https://github.com/golang/crypto/commit/6c586e17d90a7d08bbbc4069984180dce3b04117).
### Metrics, Monitoring
See [List of metrics](https://etcd.io/docs/v3.3.12/metrics/) for all metrics per release.
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd --listen-metrics-urls`](https://github.com/etcd-io/etcd/pull/8242) flag for additional `/metrics` and `/health` endpoints.
- Useful for [bypassing critical APIs when monitoring etcd](https://github.com/etcd-io/etcd/issues/8060).
- Add [`etcd_server_version`](https://github.com/etcd-io/etcd/pull/8960) Prometheus metric.
- To replace [Kubernetes `etcd-version-monitor`](https://github.com/etcd-io/etcd/issues/8948).
- Add [`etcd_debugging_mvcc_db_compaction_keys_total`](https://github.com/etcd-io/etcd/pull/8280) Prometheus metric.
- Add [`etcd_debugging_server_lease_expired_total`](https://github.com/etcd-io/etcd/pull/8064) Prometheus metric.
- To improve [lease revoke monitoring](https://github.com/etcd-io/etcd/issues/8050).
- Document [Prometheus 2.0 rules](https://github.com/etcd-io/etcd/pull/8879).
- Initialize gRPC server [metrics with zero values](https://github.com/etcd-io/etcd/pull/8878).
- Fix [range/put/delete operation metrics](https://github.com/etcd-io/etcd/pull/8054) with transaction.
- `etcd_debugging_mvcc_range_total`
- `etcd_debugging_mvcc_put_total`
- `etcd_debugging_mvcc_delete_total`
- `etcd_debugging_mvcc_txn_total`
- Fix [`etcd_debugging_mvcc_keys_total`](https://github.com/etcd-io/etcd/pull/8390) on restore.
- Fix [`etcd_debugging_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/8120) on restore.
- Also change to [`prometheus.NewGaugeFunc`](https://github.com/etcd-io/etcd/pull/8150).
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- Add [CRL based connection rejection](https://github.com/etcd-io/etcd/pull/8124) to manage [revoked certs](https://github.com/etcd-io/etcd/issues/4034).
- Document [TLS authentication changes](https://github.com/etcd-io/etcd/pull/8895).
- [Server accepts connections if IP matches, without checking DNS entries](https://github.com/etcd-io/etcd/pull/8223). For instance, if peer cert contains IP addresses and DNS names in Subject Alternative Name (SAN) field, and the remote IP address matches one of those IP addresses, server just accepts connection without further checking the DNS names.
- [Server supports reverse-lookup on wildcard DNS `SAN`](https://github.com/etcd-io/etcd/pull/8281). For instance, if peer cert contains only DNS names (no IP addresses) in Subject Alternative Name (SAN) field, server first reverse-lookups the remote IP address to get a list of names mapping to that address (e.g. `nslookup IPADDR`). Then accepts the connection if those names have a matching name with peer cert's DNS names (either by exact or wildcard match). If none is matched, server forward-lookups each DNS entry in peer cert (e.g. look up `example.default.svc` when the entry is `*.example.default.svc`), and accepts connection only when the host's resolved addresses have the matching IP address with the peer's remote IP address.
- Add [`etcd --peer-cert-allowed-cn`](https://github.com/etcd-io/etcd/pull/8616) flag.
- To support [CommonName(CN) based auth](https://github.com/etcd-io/etcd/issues/8262) for inter peer connection.
- [Swap priority](https://github.com/etcd-io/etcd/pull/8594) of cert CommonName(CN) and username + password.
- To address ["username and password specified in the request should take priority over CN in the cert"](https://github.com/etcd-io/etcd/issues/8584).
- Protect [lease revoke with auth](https://github.com/etcd-io/etcd/pull/8031).
- Provide user's role on [auth permission error](https://github.com/etcd-io/etcd/pull/8164).
- Fix [auth store panic with disabled token](https://github.com/etcd-io/etcd/pull/8695).
### etcd server
- Add [`etcd --experimental-initial-corrupt-check`](https://github.com/etcd-io/etcd/pull/8554) flag to [check cluster database hashes before serving client/peer traffic](https://github.com/etcd-io/etcd/issues/8313).
- `etcd --experimental-initial-corrupt-check=false` by default.
- v3.4 will enable `--initial-corrupt-check=true` by default.
- Add [`etcd --experimental-corrupt-check-time`](https://github.com/etcd-io/etcd/pull/8420) flag to [raise corrupt alarm monitoring](https://github.com/etcd-io/etcd/issues/7125).
- `etcd --experimental-corrupt-check-time=0s` disabled by default.
- Add [`etcd --experimental-enable-v2v3`](https://github.com/etcd-io/etcd/pull/8407) flag to [emulate v2 API with v3](https://github.com/etcd-io/etcd/issues/6925).
- `etcd --experimental-enable-v2v3=false` by default.
- Add [`etcd --max-txn-ops`](https://github.com/etcd-io/etcd/pull/7976) flag to [configure maximum number operations in transaction](https://github.com/etcd-io/etcd/issues/7826).
- Add [`etcd --max-request-bytes`](https://github.com/etcd-io/etcd/pull/7968) flag to [configure maximum client request size](https://github.com/etcd-io/etcd/issues/7923).
- If not configured, it defaults to 1.5 MiB.
- Add [`etcd --client-crl-file`, `--peer-crl-file`](https://github.com/etcd-io/etcd/pull/8124) flags for [Certificate revocation list](https://github.com/etcd-io/etcd/issues/4034).
- Add [`etcd --peer-cert-allowed-cn`](https://github.com/etcd-io/etcd/pull/8616) flag to support [CN-based auth for inter-peer connection](https://github.com/etcd-io/etcd/issues/8262).
- Add [`etcd --listen-metrics-urls`](https://github.com/etcd-io/etcd/pull/8242) flag for additional `/metrics` and `/health` endpoints.
- Support [additional (non) TLS `/metrics` endpoints for a TLS-enabled cluster](https://github.com/etcd-io/etcd/pull/8282).
- e.g. `etcd --listen-metrics-urls=https://localhost:2378,http://localhost:9379` to serve `/metrics` and `/health` on secure port 2378 and insecure port 9379.
- Useful for [bypassing critical APIs when monitoring etcd](https://github.com/etcd-io/etcd/issues/8060).
- Add [`etcd --auto-compaction-mode`](https://github.com/etcd-io/etcd/pull/8123) flag to [support revision-based compaction](https://github.com/etcd-io/etcd/issues/8098).
- Change `etcd --auto-compaction-retention` flag to [accept string values](https://github.com/etcd-io/etcd/pull/8563) with [finer granularity](https://github.com/etcd-io/etcd/issues/8503).
- Now that `etcd --auto-compaction-retention` accepts string values, etcd configuration YAML file `auto-compaction-retention` field must be changed to `string` type.
- Previously, `etcd --config-file etcd.config.yaml` can have `auto-compaction-retention: 24` field, now must be `auto-compaction-retention: "24"` or `auto-compaction-retention: "24h"`.
- If configured as `--auto-compaction-mode periodic --auto-compaction-retention "24h"`, the time duration value for `etcd --auto-compaction-retention` flag must be valid for [`time.ParseDuration`](https://golang.org/pkg/time/#ParseDuration) function in Go.
- e.g. `etcd --auto-compaction-mode=revision --auto-compaction-retention=1000` automatically `Compact` on `"latest revision" - 1000` every 5-minute (when latest revision is 30000, compact on revision 29000).
- e.g. `etcd --auto-compaction-mode=periodic --auto-compaction-retention=72h` automatically `Compact` with 72-hour retention windown, for every 7.2-hour.
- e.g. `etcd --auto-compaction-mode=periodic --auto-compaction-retention=30m` automatically `Compact` with 30-minute retention windown, for every 3-minute.
- Periodic compactor continues to record latest revisions for every 1/10 of given compaction period (e.g. 1-hour when `etcd --auto-compaction-mode=periodic --auto-compaction-retention=10h`).
- For every 1/10 of given compaction period, compactor uses the last revision that was fetched before compaction period, to discard historical data.
- The retention window of compaction period moves for every 1/10 of given compaction period.
- For instance, when hourly writes are 100 and `--auto-compaction-retention=10`, v3.1 compacts revision 1000, 2000, and 3000 for every 10-hour, while v3.2.x, v3.3.0, v3.3.1, and v3.3.2 compact revision 1000, 1100, and 1200 for every 1-hour. Futhermore, when writes per minute are 1000, v3.3.0, v3.3.1, and v3.3.2 with `--auto-compaction-mode=periodic --auto-compaction-retention=30m` compact revision 30000, 33000, and 36000, for every 3-minute with more finer granularity.
- Whether compaction succeeds or not, this process repeats for every 1/10 of given compaction period. If compaction succeeds, it just removes compacted revision from historical revision records.
- Add [`etcd --grpc-keepalive-min-time`, `etcd --grpc-keepalive-interval`, `etcd --grpc-keepalive-timeout`](https://github.com/etcd-io/etcd/pull/8535) flags to configure server-side keepalive policies.
- Serve [`/health` endpoint as unhealthy](https://github.com/etcd-io/etcd/pull/8272) when [alarm (e.g. `NOSPACE`) is raised or there's no leader](https://github.com/etcd-io/etcd/issues/8207).
- Define [`etcdhttp.Health`](https://godoc.org/github.com/coreos/etcd/etcdserver/api/etcdhttp#Health) struct with JSON encoder.
- Note that `"health"` field is [`string` type, not `bool`](https://github.com/etcd-io/etcd/pull/9143).
- e.g. `{"health":"false"}`, `{"health":"true"}`
- [Remove `"errors"` field](https://github.com/etcd-io/etcd/pull/9162) since `v3.3.0-rc.3` (did exist only in `v3.3.0-rc.0`, `v3.3.0-rc.1`, `v3.3.0-rc.2`).
- Move [logging setup to embed package](https://github.com/etcd-io/etcd/pull/8810)
- Disable gRPC server info-level logs by default (can be enabled with `etcd --debug` flag).
- Use [monotonic time in Go 1.9](https://github.com/etcd-io/etcd/pull/8507) for `lease` package.
- Warn on [empty hosts in advertise URLs](https://github.com/etcd-io/etcd/pull/8384).
- Address [advertise client URLs accepts empty hosts](https://github.com/etcd-io/etcd/issues/8379).
- etcd v3.4 will exit on this error.
- e.g. `etcd --advertise-client-urls=http://:2379`.
- Warn on [shadowed environment variables](https://github.com/etcd-io/etcd/pull/8385).
- Address [error on shadowed environment variables](https://github.com/etcd-io/etcd/issues/8380).
- etcd v3.4 will exit on this error.
### API
- Support [ranges in transaction comparisons](https://github.com/etcd-io/etcd/pull/8025) for [disconnected linearized reads](https://github.com/etcd-io/etcd/issues/7924).
- Add [nested transactions](https://github.com/etcd-io/etcd/pull/8102) to extend [proxy use cases](https://github.com/etcd-io/etcd/issues/7857).
- Add [lease comparison target in transaction](https://github.com/etcd-io/etcd/pull/8324).
- Add [lease list](https://github.com/etcd-io/etcd/pull/8358).
- Add [hash by revision](https://github.com/etcd-io/etcd/pull/8263) for [better corruption checking against boltdb](https://github.com/etcd-io/etcd/issues/8016).
### client v3
- Add [health balancer](https://github.com/etcd-io/etcd/pull/8545) to fix [watch API hangs](https://github.com/etcd-io/etcd/issues/7247), improve [endpoint switch under network faults](https://github.com/etcd-io/etcd/issues/7941).
- [Refactor balancer](https://github.com/etcd-io/etcd/pull/8840) and add [client-side keepalive pings](https://github.com/etcd-io/etcd/pull/8199) to handle [network partitions](https://github.com/etcd-io/etcd/issues/8711).
- Add [`MaxCallSendMsgSize` and `MaxCallRecvMsgSize`](https://github.com/etcd-io/etcd/pull/9047) fields to [`clientv3.Config`](https://godoc.org/github.com/coreos/etcd/clientv3#Config).
- Fix [exceeded response size limit error in client-side](https://github.com/etcd-io/etcd/issues/9043).
- Address [kubernetes#51099](https://github.com/kubernetes/kubernetes/issues/51099).
- In previous versions(v3.2.10, v3.2.11), client response size was limited to only 4 MiB.
- `MaxCallSendMsgSize` default value is 2 MiB, if not configured.
- `MaxCallRecvMsgSize` default value is `math.MaxInt32`, if not configured.
- Accept [`Compare_LEASE`](https://github.com/etcd-io/etcd/pull/8324) in [`clientv3.Compare`](https://godoc.org/github.com/coreos/etcd/clientv3#Compare).
- Add [`LeaseValue` helper](https://github.com/etcd-io/etcd/pull/8488) to `Cmp` `LeaseID` values in `Txn`.
- Add [`MoveLeader`](https://github.com/etcd-io/etcd/pull/8153) to `Maintenance`.
- Add [`HashKV`](https://github.com/etcd-io/etcd/pull/8351) to `Maintenance`.
- Add [`Leases`](https://github.com/etcd-io/etcd/pull/8358) to `Lease`.
- Add [`clientv3/ordering`](https://github.com/etcd-io/etcd/pull/8092) for enforce [ordering in serialized requests](https://github.com/etcd-io/etcd/issues/7623).
- Fix ["put at-most-once" violation](https://github.com/etcd-io/etcd/pull/8335).
- Fix [`WatchResponse.Canceled`](https://github.com/etcd-io/etcd/pull/8283) on [compacted watch request](https://github.com/etcd-io/etcd/issues/8231).
- Fix [`concurrency/stm` `Put` with serializable snapshot](https://github.com/etcd-io/etcd/pull/8439).
- Use store revision from first fetch to resolve write conflicts instead of modified revision.
### etcdctl v3
- Add [`etcdctl --discovery-srv`](https://github.com/etcd-io/etcd/pull/8462) flag.
- Add [`etcdctl --keepalive-time`, `--keepalive-timeout`](https://github.com/etcd-io/etcd/pull/8663) flags.
- Add [`etcdctl lease list`](https://github.com/etcd-io/etcd/pull/8358) command.
- Add [`etcdctl lease keep-alive --once`](https://github.com/etcd-io/etcd/pull/8775) flag.
- Make [`lease timetolive LEASE_ID`](https://github.com/etcd-io/etcd/issues/9028) on expired lease print [`lease LEASE_ID already expired`](https://github.com/etcd-io/etcd/pull/9047).
- <=3.2 prints `lease LEASE_ID granted with TTL(0s), remaining(-1s)`.
- Add [`etcdctl snapshot restore --wal-dir`](https://github.com/etcd-io/etcd/pull/9124) flag.
- Add [`etcdctl defrag --data-dir`](https://github.com/etcd-io/etcd/pull/8367) flag.
- Add [`etcdctl move-leader`](https://github.com/etcd-io/etcd/pull/8153) command.
- Add [`etcdctl endpoint hashkv`](https://github.com/etcd-io/etcd/pull/8351) command.
- Add [`etcdctl endpoint --cluster`](https://github.com/etcd-io/etcd/pull/8143) flag, equivalent to [v2 `etcdctl cluster-health`](https://github.com/etcd-io/etcd/issues/8117).
- Make `etcdctl endpoint health` command terminate with [non-zero exit code on unhealthy status](https://github.com/etcd-io/etcd/pull/8342).
- Add [`etcdctl lock --ttl`](https://github.com/etcd-io/etcd/pull/8370) flag.
- Support [`etcdctl watch [key] [range_end] -- [exec-command…]`](https://github.com/etcd-io/etcd/pull/8919), equivalent to [v2 `etcdctl exec-watch`](https://github.com/etcd-io/etcd/issues/8814).
- Make `etcdctl watch -- [exec-command]` set environmental variables [`ETCD_WATCH_REVISION`, `ETCD_WATCH_EVENT_TYPE`, `ETCD_WATCH_KEY`, `ETCD_WATCH_VALUE`](https://github.com/etcd-io/etcd/pull/9142) for each event.
- Support [`etcdctl watch` with environmental variables `ETCDCTL_WATCH_KEY` and `ETCDCTL_WATCH_RANGE_END`](https://github.com/etcd-io/etcd/pull/9142).
- Enable [`clientv3.WithRequireLeader(context.Context)` for `watch`](https://github.com/etcd-io/etcd/pull/8672) command.
- Print [`"del"` instead of `"delete"`](https://github.com/etcd-io/etcd/pull/8297) in `txn` interactive mode.
- Print [`ETCD_INITIAL_ADVERTISE_PEER_URLS` in `member add`](https://github.com/etcd-io/etcd/pull/8332).
### etcdctl v3
- Handle [empty key permission](https://github.com/etcd-io/etcd/pull/8514) in `etcdctl`.
### etcdctl v2
- Add [`etcdctl backup --with-v3`](https://github.com/etcd-io/etcd/pull/8479) flag.
### gRPC Proxy
- Add [`grpc-proxy start --experimental-leasing-prefix`](https://github.com/etcd-io/etcd/pull/8341) flag.
- For disconnected linearized reads.
- Based on [V system leasing](https://github.com/etcd-io/etcd/issues/6065).
- See ["Disconnected consistent reads with etcd" blog post](https://coreos.com/blog/coreos-labs-disconnected-consistent-reads-with-etcd).
- Add [`grpc-proxy start --experimental-serializable-ordering`](https://github.com/etcd-io/etcd/pull/8315) flag.
- To ensure serializable reads have monotonically increasing store revisions across endpoints.
- Add [`grpc-proxy start --metrics-addr`](https://github.com/etcd-io/etcd/pull/8242) flag for an additional `/metrics` endpoint.
- Set `--metrics-addr=http://[HOST]:9379` to serve `/metrics` in insecure port 9379.
- Serve [`/health` endpoint in grpc-proxy](https://github.com/etcd-io/etcd/pull/8322).
- Add [`grpc-proxy start --debug`](https://github.com/etcd-io/etcd/pull/8994) flag.
- Add [`grpc-proxy start --max-send-bytes`](https://github.com/etcd-io/etcd/pull/9250) flag to [configure maximum client request size](https://github.com/etcd-io/etcd/issues/7923).
- Add [`grpc-proxy start --max-recv-bytes`](https://github.com/etcd-io/etcd/pull/9250) flag to [configure maximum client request size](https://github.com/etcd-io/etcd/issues/7923).
- Fix [Snapshot API error handling](https://github.com/etcd-io/etcd/commit/dbd16d52fbf81e5fd806d21ff5e9148d5bf203ab).
- Fix [KV API `PrevKv` flag handling](https://github.com/etcd-io/etcd/pull/8366).
- Fix [KV API `KeysOnly` flag handling](https://github.com/etcd-io/etcd/pull/8552).
### gRPC gateway
- Replace [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) endpoint `/v3alpha` with [`/v3beta`](https://github.com/etcd-io/etcd/pull/8880).
- To deprecate [`/v3alpha`](https://github.com/etcd-io/etcd/issues/8125) in v3.4.
- In v3.3, `curl -L http://localhost:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` still works as a fallback to `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'`, but `curl -L http://localhost:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` won't work in v3.4. Use `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
- Support ["authorization" token](https://github.com/etcd-io/etcd/pull/7999).
- Support [websocket for bi-directional streams](https://github.com/etcd-io/etcd/pull/8257).
- Fix [`Watch` API with gRPC gateway](https://github.com/etcd-io/etcd/issues/8237).
- Upgrade gRPC gateway to [v1.3.0](https://github.com/etcd-io/etcd/issues/8838).
### etcd server
- Fix [backend database in-memory index corruption](https://github.com/etcd-io/etcd/pull/8127) issue on restore (only 3.2.0 is affected).
- Fix [watch restore from snapshot](https://github.com/etcd-io/etcd/pull/8427).
- Fix [`mvcc/backend.defragdb` nil-pointer dereference on create bucket failure](https://github.com/etcd-io/etcd/pull/9119).
- Fix [server crash](https://github.com/etcd-io/etcd/pull/8010) on [invalid transaction request from gRPC gateway](https://github.com/etcd-io/etcd/issues/7889).
- Prevent [server panic from member update/add](https://github.com/etcd-io/etcd/pull/9174) with [wrong scheme URLs](https://github.com/etcd-io/etcd/issues/9173).
- Make [peer dial timeout longer](https://github.com/etcd-io/etcd/pull/8599).
- See [coreos/etcd-operator#1300](https://github.com/etcd-io/etcd-operator/issues/1300) for more detail.
- Make server [wait up to request time-out](https://github.com/etcd-io/etcd/pull/8267) with [pending RPCs](https://github.com/etcd-io/etcd/issues/8224).
- Fix [`grpc.Server` panic on `GracefulStop`](https://github.com/etcd-io/etcd/pull/8987) with [TLS-enabled server](https://github.com/etcd-io/etcd/issues/8916).
- Fix ["multiple peer URLs cannot start" issue](https://github.com/etcd-io/etcd/issues/8383).
- Fix server-side auth so [concurrent auth operations do not return old revision error](https://github.com/etcd-io/etcd/pull/8442).
- Handle [WAL renaming failure on Windows](https://github.com/etcd-io/etcd/pull/8286).
- Upgrade [`coreos/go-systemd`](https://github.com/coreos/go-systemd/releases) to `v15` (see https://github.com/coreos/go-systemd/releases/tag/v15).
- [Put back `/v2/machines`](https://github.com/etcd-io/etcd/pull/8062) endpoint for python-etcd wrapper.
### client v2
- [Fail-over v2 client](https://github.com/etcd-io/etcd/pull/8519) to next endpoint on [oneshot failure](https://github.com/etcd-io/etcd/issues/8515).
### Package `raft`
- Add [non-voting member](https://github.com/etcd-io/etcd/pull/8751).
- To implement [Raft thesis 4.2.1 Catching up new servers](https://github.com/etcd-io/etcd/issues/8568).
- `Learner` node does not vote or promote itself.
### Other
- Support previous two minor versions (see our [new release policy](https://github.com/etcd-io/etcd/pull/8805)).
- `v3.3.x` is the last release cycle that supports `ACI`.
- [AppC was officially suspended](https://github.com/appc/spec#-disclaimer-), as of late 2016.
- [`acbuild`](https://github.com/containers/build#this-project-is-currently-unmaintained) is not maintained anymore.
- `*.aci` files won't be available from etcd v3.4 release.
- Add container registry [`gcr.io/etcd-development/etcd`](https://gcr.io/etcd-development/etcd).
- [quay.io/coreos/etcd](https://quay.io/coreos/etcd) is still supported as secondary.
### Go
- Require [*Go 1.9+*](https://github.com/etcd-io/etcd/issues/6174).
- Compile with [*Go 1.9.3*](https://golang.org/doc/devel/release.html#go1.9).
- Deprecate [`golang.org/x/net/context`](https://github.com/etcd-io/etcd/pull/8511).
<hr>

View File

@ -1,557 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.3](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md).
<hr>
## v3.4.0 (2019-08-30)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.3.0...v3.4.0) and [v3.4 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_4.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.4 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_4.md).**
### Documentation
- etcd now has a new website! Please visit https://etcd.io.
### Improved
- Add Raft learner: [etcd#10725](https://github.com/etcd-io/etcd/pull/10725), [etcd#10727](https://github.com/etcd-io/etcd/pull/10727), [etcd#10730](https://github.com/etcd-io/etcd/pull/10730).
- User guide: [runtime-configuration document](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/runtime-configuration.md#add-a-new-member-as-learner).
- API change: [API reference document](https://github.com/etcd-io/etcd/blob/master/Documentation/dev-guide/api_reference_v3.md).
- More details on implementation: [learner design document](https://github.com/etcd-io/etcd/blob/master/Documentation/learning/design-learner.md) and [implementation task list](https://github.com/etcd-io/etcd/issues/10537).
- Rewrite [client balancer](https://github.com/etcd-io/etcd/pull/9860) with [new gRPC balancer interface](https://github.com/etcd-io/etcd/issues/9106).
- Upgrade [gRPC to v1.23.0](https://github.com/etcd-io/etcd/pull/10911).
- Improve [client balancer failover against secure endpoints](https://github.com/etcd-io/etcd/pull/10911).
- Fix ["kube-apiserver 1.13.x refuses to work when first etcd-server is not available" (kubernetes#72102)](https://github.com/kubernetes/kubernetes/issues/72102).
- Fix [gRPC panic "send on closed channel](https://github.com/etcd-io/etcd/issues/9956).
- Add [backoff on watch retries on transient errors](https://github.com/etcd-io/etcd/pull/9840).
- Add [jitter to watch progress notify](https://github.com/etcd-io/etcd/pull/9278) to prevent [spikes in `etcd_network_client_grpc_sent_bytes_total`](https://github.com/etcd-io/etcd/issues/9246).
- Improve [read index wait timeout warning log](https://github.com/etcd-io/etcd/pull/10026), which indicates that local node might have slow network.
- Improve [slow request apply warning log](https://github.com/etcd-io/etcd/pull/9288).
- e.g. `read-only range request "key:\"/a\" range_end:\"/b\" " with result "range_response_count:3 size:96" took too long (97.966µs) to execute`.
- Redact [request value field](https://github.com/etcd-io/etcd/pull/9822).
- Provide [response size](https://github.com/etcd-io/etcd/pull/9826).
- Improve ["became inactive" warning log](https://github.com/etcd-io/etcd/pull/10024), which indicates message send to a peer failed.
- Improve [TLS setup error logging](https://github.com/etcd-io/etcd/pull/9518) to help debug [TLS-enabled cluster configuring issues](https://github.com/etcd-io/etcd/issues/9400).
- Improve [long-running concurrent read transactions under light write workloads](https://github.com/etcd-io/etcd/pull/9296).
- Previously, periodic commit on pending writes blocks incoming read transactions, even if there is no pending write.
- Now, periodic commit operation does not block concurrent read transactions, thus improves long-running read transaction performance.
- Rebased [etcd#10523](https://github.com/etcd-io/etcd/pull/10523).
- Improve [Raft Read Index timeout warning messages](https://github.com/etcd-io/etcd/pull/9897).
- Adjust [election timeout on server restart](https://github.com/etcd-io/etcd/pull/9415) to reduce [disruptive rejoining servers](https://github.com/etcd-io/etcd/issues/9333).
- Previously, etcd fast-forwards election ticks on server start, with only one tick left for leader election. This is to speed up start phase, without having to wait until all election ticks elapse. Advancing election ticks is useful for cross datacenter deployments with larger election timeouts. However, it was affecting cluster availability if the last tick elapses before leader contacts the restarted node.
- Now, when etcd restarts, it adjusts election ticks with more than one tick left, thus more time for leader to prevent disruptive restart.
- Add [Raft Pre-Vote feature](https://github.com/etcd-io/etcd/pull/9352) to reduce [disruptive rejoining servers](https://github.com/etcd-io/etcd/issues/9333).
- For instance, a flaky(or rejoining) member may drop in and out, and start campaign. This member will end up with a higher term, and ignore all incoming messages with lower term. In this case, a new leader eventually need to get elected, thus disruptive to cluster availability. Raft implements Pre-Vote phase to prevent this kind of disruptions. If enabled, Raft runs an additional phase of election to check if pre-candidate can get enough votes to win an election.
- Adjust [periodic compaction retention window](https://github.com/etcd-io/etcd/pull/9485).
- e.g. `etcd --auto-compaction-mode=revision --auto-compaction-retention=1000` automatically `Compact` on `"latest revision" - 1000` every 5-minute (when latest revision is 30000, compact on revision 29000).
- e.g. Previously, `etcd --auto-compaction-mode=periodic --auto-compaction-retention=24h` automatically `Compact` with 24-hour retention windown for every 2.4-hour. Now, `Compact` happens for every 1-hour.
- e.g. Previously, `etcd --auto-compaction-mode=periodic --auto-compaction-retention=30m` automatically `Compact` with 30-minute retention windown for every 3-minute. Now, `Compact` happens for every 30-minute.
- Periodic compactor keeps recording latest revisions for every compaction period when given period is less than 1-hour, or for every 1-hour when given compaction period is greater than 1-hour (e.g. 1-hour when `etcd --auto-compaction-mode=periodic --auto-compaction-retention=24h`).
- For every compaction period or 1-hour, compactor uses the last revision that was fetched before compaction period, to discard historical data.
- The retention window of compaction period moves for every given compaction period or hour.
- For instance, when hourly writes are 100 and `etcd --auto-compaction-mode=periodic --auto-compaction-retention=24h`, `v3.2.x`, `v3.3.0`, `v3.3.1`, and `v3.3.2` compact revision 2400, 2640, and 2880 for every 2.4-hour, while `v3.3.3` *or later* compacts revision 2400, 2500, 2600 for every 1-hour.
- Futhermore, when `etcd --auto-compaction-mode=periodic --auto-compaction-retention=30m` and writes per minute are about 1000, `v3.3.0`, `v3.3.1`, and `v3.3.2` compact revision 30000, 33000, and 36000, for every 3-minute, while `v3.3.3` *or later* compacts revision 30000, 60000, and 90000, for every 30-minute.
- Improve [lease expire/revoke operation performance](https://github.com/etcd-io/etcd/pull/9418), address [lease scalability issue](https://github.com/etcd-io/etcd/issues/9496).
- Make [Lease `Lookup` non-blocking with concurrent `Grant`/`Revoke`](https://github.com/etcd-io/etcd/pull/9229).
- Make etcd server return `raft.ErrProposalDropped` on internal Raft proposal drop in [v3 applier](https://github.com/etcd-io/etcd/pull/9549) and [v2 applier](https://github.com/etcd-io/etcd/pull/9558).
- e.g. a node is removed from cluster, or [`raftpb.MsgProp` arrives at current leader while there is an ongoing leadership transfer](https://github.com/etcd-io/etcd/issues/8975).
- Add [`snapshot`](https://github.com/etcd-io/etcd/pull/9118) package for easier snapshot workflow (see [`godoc.org/github.com/etcd/clientv3/snapshot`](https://godoc.org/github.com/etcd-io/etcd/clientv3/snapshot) for more).
- Improve [functional tester](https://github.com/etcd-io/etcd/tree/master/functional) coverage: [proxy layer to run network fault tests in CI](https://github.com/etcd-io/etcd/pull/9081), [TLS is enabled both for server and client](https://github.com/etcd-io/etcd/pull/9534), [liveness mode](https://github.com/etcd-io/etcd/issues/9230), [shuffle test sequence](https://github.com/etcd-io/etcd/issues/9381), [membership reconfiguration failure cases](https://github.com/etcd-io/etcd/pull/9564), [disastrous quorum loss and snapshot recover from a seed member](https://github.com/etcd-io/etcd/pull/9565), [embedded etcd](https://github.com/etcd-io/etcd/pull/9572).
- Improve [index compaction blocking](https://github.com/etcd-io/etcd/pull/9511) by using a copy on write clone to avoid holding the lock for the traversal of the entire index.
- Update [JWT methods](https://github.com/etcd-io/etcd/pull/9883) to allow for use of any supported signature method/algorithm.
- Add [Lease checkpointing](https://github.com/etcd-io/etcd/pull/9924) to persist remaining TTLs to the consensus log periodically so that long lived leases progress toward expiry in the presence of leader elections and server restarts.
- Enabled by experimental flag "--experimental-enable-lease-checkpoint".
- Add [gRPC interceptor for debugging logs](https://github.com/etcd-io/etcd/pull/9990); enable `etcd --debug` flag to see per-request debug information.
- Add [consistency check in snapshot status](https://github.com/etcd-io/etcd/pull/10109). If consistency check on snapshot file fails, `snapshot status` returns `"snapshot file integrity check failed..."` error.
- Add [`Verify` function to perform corruption check on WAL contents](https://github.com/etcd-io/etcd/pull/10603).
- Improve [heartbeat send failure logging](https://github.com/etcd-io/etcd/pull/10663).
- Support [users with no password](https://github.com/etcd-io/etcd/pull/9817) for reducing security risk introduced by leaked password. The users can only be authenticated with CommonName based auth.
- Add flag `--experimental-peer-skip-client-san-verification` to [skip verification of peer client address](https://github.com/etcd-io/etcd/pull/10524)
### Breaking Changes
- Require [*Go 1.12+*](https://github.com/etcd-io/etcd/pull/10045).
- Use [Go module](https://github.com/etcd-io/etcd/pull/10063) for dependency management.
- Move [`"github.com/coreos/etcd"`](https://github.com/etcd-io/etcd/issues/9965) to [`"github.com/etcd-io/etcd"`](https://github.com/etcd-io/etcd/issues/9965).
- Change import path to `"go.etcd.io/etcd"`.
- e.g. `import "go.etcd.io/etcd/raft"`.
- Make [`ETCDCTL_API=3 etcdctl` default](https://github.com/etcd-io/etcd/issues/9600).
- Now, `etcdctl set foo bar` must be `ETCDCTL_API=2 etcdctl set foo bar`.
- Now, `ETCDCTL_API=3 etcdctl put foo bar` could be just `etcdctl put foo bar`.
- **Remove `etcd --ca-file` flag**, instead [use `etcd --trusted-ca-file`](https://github.com/etcd-io/etcd/pull/9470) (`etcd --ca-file` flag has been marked deprecated since v2.1).
- **Remove `etcd --peer-ca-file` flag**, instead [use `etcd --peer-trusted-ca-file`](https://github.com/etcd-io/etcd/pull/9470) (`etcd --peer-ca-file` flag has been marked deprecated since v2.1).
- **Remove `pkg/transport.TLSInfo.CAFile` field**, instead [use `pkg/transport.TLSInfo.TrustedCAFile`](https://github.com/etcd-io/etcd/pull/9470) (`CAFile` field has been marked deprecated since v2.1).
- Deprecated `latest` [release container](https://console.cloud.google.com/gcr/images/etcd-development/GLOBAL/etcd) tag.
- **`docker pull gcr.io/etcd-development/etcd:latest` would not be up-to-date**.
- Deprecated [minor](https://semver.org/) version [release container](https://console.cloud.google.com/gcr/images/etcd-development/GLOBAL/etcd) tags.
- `docker pull gcr.io/etcd-development/etcd:v3.3` would still work.
- **`docker pull gcr.io/etcd-development/etcd:v3.4` would not work**.
- Use **`docker pull gcr.io/etcd-development/etcd:v3.4.x`** instead, with the exact patch version.
- Drop [ACIs from official release](https://github.com/etcd-io/etcd/pull/9059).
- [AppC was officially suspended](https://github.com/appc/spec#-disclaimer-), as of late 2016.
- [`acbuild`](https://github.com/containers/build#this-project-is-currently-unmaintained) is not maintained anymore.
- `*.aci` files are not available from `v3.4` release.
- Exit on [empty hosts in advertise URLs](https://github.com/etcd-io/etcd/pull/8786).
- Address [advertise client URLs accepts empty hosts](https://github.com/etcd-io/etcd/issues/8379).
- e.g. exit with error on `--advertise-client-urls=http://:2379`.
- e.g. exit with error on `--initial-advertise-peer-urls=http://:2380`.
- Exit on [shadowed environment variables](https://github.com/etcd-io/etcd/pull/9382).
- Address [error on shadowed environment variables](https://github.com/etcd-io/etcd/issues/8380).
- e.g. exit with error on `ETCD_NAME=abc etcd --name=def`.
- e.g. exit with error on `ETCD_INITIAL_CLUSTER_TOKEN=abc etcd --initial-cluster-token=def`.
- e.g. exit with error on `ETCDCTL_ENDPOINTS=abc.com ETCDCTL_API=3 etcdctl endpoint health --endpoints=def.com`.
- Change [`etcdserverpb.AuthRoleRevokePermissionRequest/key,range_end` fields type from `string` to `bytes`](https://github.com/etcd-io/etcd/pull/9433).
- Deprecating `etcd_debugging_mvcc_db_total_size_in_bytes` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819) instead.
- Deprecating `etcd_debugging_mvcc_put_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_put_total`](https://github.com/etcd-io/etcd/pull/10962) instead.
- Deprecating `etcd_debugging_mvcc_delete_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_delete_total`](https://github.com/etcd-io/etcd/pull/10962) instead.
- Deprecating `etcd_debugging_mvcc_range_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_range_total`](https://github.com/etcd-io/etcd/pull/10968) instead.
- Deprecating `etcd_debugging_mvcc_txn_total`Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_txn_total`](https://github.com/etcd-io/etcd/pull/10968) instead.
- Rename `etcdserver.ServerConfig.SnapCount` field to `etcdserver.ServerConfig.SnapshotCount`, to be consistent with the flag name `etcd --snapshot-count`.
- Rename `embed.Config.SnapCount` field to [`embed.Config.SnapshotCount`](https://github.com/etcd-io/etcd/pull/9745), to be consistent with the flag name `etcd --snapshot-count`.
- Change [`embed.Config.CorsInfo` in `*cors.CORSInfo` type to `embed.Config.CORS` in `map[string]struct{}` type](https://github.com/etcd-io/etcd/pull/9490).
- Remove [`embed.Config.SetupLogging`](https://github.com/etcd-io/etcd/pull/9572).
- Now logger is set up automatically based on [`embed.Config.Logger`, `embed.Config.LogOutputs`, `embed.Config.Debug` fields](https://github.com/etcd-io/etcd/pull/9572).
- Rename [`etcd --log-output` to `etcd --log-outputs`](https://github.com/etcd-io/etcd/pull/9624) to support multiple log outputs.
- **`etcd --log-output`** will be deprecated in v3.5.
- Rename [**`embed.Config.LogOutput`** to **`embed.Config.LogOutputs`**](https://github.com/etcd-io/etcd/pull/9624) to support multiple log outputs.
- Change [**`embed.Config.LogOutputs`** type from `string` to `[]string`](https://github.com/etcd-io/etcd/pull/9579) to support multiple log outputs.
- Now that `etcd --log-outputs` accepts multiple writers, etcd configuration YAML file `log-outputs` field must be changed to `[]string` type.
- Previously, `etcd --config-file etcd.config.yaml` can have `log-outputs: default` field, now must be `log-outputs: [default]`.
- Deprecating [`etcd --debug`](https://github.com/etcd-io/etcd/pull/10947) flag. Use `etcd --log-level=debug` flag instead.
- v3.5 will deprecate `etcd --debug` flag in favor of `etcd --log-level=debug`.
- Change v3 `etcdctl snapshot` exit codes with [`snapshot` package](https://github.com/etcd-io/etcd/pull/9118/commits/df689f4280e1cce4b9d61300be13ca604d41670a).
- Exit on error with exit code 1 (no more exit code 5 or 6 on `snapshot save/restore` commands).
- Migrate dependency management tool from `glide` to [`golang/dep`](https://github.com/etcd-io/etcd/pull/9155).
- <= 3.3 puts `vendor` directory under `cmd/vendor` directory to [prevent conflicting transitive dependencies](https://github.com/etcd-io/etcd/issues/4913).
- 3.4 moves `cmd/vendor` directory to `vendor` at repository root.
- Remove recursive symlinks in `cmd` directory.
- Now `go get/install/build` on `etcd` packages (e.g. `clientv3`, `tools/benchmark`) enforce builds with etcd `vendor` directory.
- Replace [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) endpoint `/v3beta` with [`/v3`](https://github.com/etcd-io/etcd/pull/9298).
- Deprecated [`/v3alpha`](https://github.com/etcd-io/etcd/pull/9298).
- To deprecate [`/v3beta`](https://github.com/etcd-io/etcd/issues/9189) in v3.5.
- In v3.4, `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` still works as a fallback to `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'`, but `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` won't work in v3.5. Use `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
- Change [`wal` package function signatures](https://github.com/etcd-io/etcd/pull/9572) to support [structured logger and logging to file](https://github.com/etcd-io/etcd/issues/9438) in server-side.
- Previously, `Open(dirpath string, snap walpb.Snapshot) (*WAL, error)`, now `Open(lg *zap.Logger, dirpath string, snap walpb.Snapshot) (*WAL, error)`.
- Previously, `OpenForRead(dirpath string, snap walpb.Snapshot) (*WAL, error)`, now `OpenForRead(lg *zap.Logger, dirpath string, snap walpb.Snapshot) (*WAL, error)`.
- Previously, `Repair(dirpath string) bool`, now `Repair(lg *zap.Logger, dirpath string) bool`.
- Previously, `Create(dirpath string, metadata []byte) (*WAL, error)`, now `Create(lg *zap.Logger, dirpath string, metadata []byte) (*WAL, error)`.
- Remove [`pkg/cors` package](https://github.com/etcd-io/etcd/pull/9490).
- Move internal packages to `etcdserver`.
- `"github.com/coreos/etcd/alarm"` to `"go.etcd.io/etcd/etcdserver/api/v3alarm"`.
- `"github.com/coreos/etcd/compactor"` to `"go.etcd.io/etcd/etcdserver/api/v3compactor"`.
- `"github.com/coreos/etcd/discovery"` to `"go.etcd.io/etcd/etcdserver/api/v2discovery"`.
- `"github.com/coreos/etcd/etcdserver/auth"` to `"go.etcd.io/etcd/etcdserver/api/v2auth"`.
- `"github.com/coreos/etcd/etcdserver/membership"` to `"go.etcd.io/etcd/etcdserver/api/membership"`.
- `"github.com/coreos/etcd/etcdserver/stats"` to `"go.etcd.io/etcd/etcdserver/api/v2stats"`.
- `"github.com/coreos/etcd/error"` to `"go.etcd.io/etcd/etcdserver/api/v2error"`.
- `"github.com/coreos/etcd/rafthttp"` to `"go.etcd.io/etcd/etcdserver/api/rafthttp"`.
- `"github.com/coreos/etcd/snap"` to `"go.etcd.io/etcd/etcdserver/api/snap"`.
- `"github.com/coreos/etcd/store"` to `"go.etcd.io/etcd/etcdserver/api/v2store"`.
- Change [snapshot file permissions](https://github.com/etcd-io/etcd/pull/9977): On Linux, the snapshot file changes from readable by all (mode 0644) to readable by the user only (mode 0600).
- Change [`pkg/adt.IntervalTree` from `struct` to `interface`](https://github.com/etcd-io/etcd/pull/10959).
- See [`pkg/adt` README](https://github.com/etcd-io/etcd/tree/master/pkg/adt) and [`pkg/adt` godoc](https://godoc.org/go.etcd.io/etcd/pkg/adt).
- Release branch `/version` defines version `3.4.x-pre`, instead of `3.4.y+git`.
- Use `3.4.5-pre`, instead of `3.4.4+git`.
### Dependency
- Upgrade [`github.com/coreos/bbolt`](https://github.com/etcd-io/bbolt/releases) from [**`v1.3.1-coreos.6`**](https://github.com/etcd-io/bbolt/releases/tag/v1.3.1-coreos.6) to [`go.etcd.io/bbolt`](https://github.com/etcd-io/bbolt/releases) [**`v1.3.3`**](https://github.com/etcd-io/bbolt/releases/tag/v1.3.3).
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) from [**`v1.7.5`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5) to [**`v1.23.0`**](https://github.com/grpc/grpc-go/releases/tag/v1.23.0).
- Migrate [`github.com/ugorji/go/codec`](https://github.com/ugorji/go/releases) to [**`github.com/json-iterator/go`**](https://github.com/json-iterator/go), to [regenerate v2 `client`](https://github.com/etcd-io/etcd/pull/9494) (See [#10667](https://github.com/etcd-io/etcd/pull/10667) for more).
- Migrate [`github.com/ghodss/yaml`](https://github.com/ghodss/yaml/releases) to [**`sigs.k8s.io/yaml`**](https://github.com/kubernetes-sigs/yaml) (See [#10687](https://github.com/etcd-io/etcd/pull/10687) for more).
- Upgrade [`golang.org/x/crypto`](https://github.com/golang/crypto) from [**`crypto@9419663f5`**](https://github.com/golang/crypto/commit/9419663f5a44be8b34ca85f08abc5fe1be11f8a3) to [**`crypto@0709b304e793`**](https://github.com/golang/crypto/commit/0709b304e793a5edb4a2c0145f281ecdc20838a4).
- Upgrade [`golang.org/x/net`](https://github.com/golang/net) from [**`net@66aacef3d`**](https://github.com/golang/net/commit/66aacef3dd8a676686c7ae3716979581e8b03c47) to [**`net@adae6a3d119a`**](https://github.com/golang/net/commit/adae6a3d119ae4890b46832a2e88a95adc62b8e7).
- Upgrade [`golang.org/x/sys`](https://github.com/golang/sys) from [**`sys@ebfc5b463`**](https://github.com/golang/sys/commit/ebfc5b4631820b793c9010c87fd8fef0f39eb082) to [**`sys@56ede360e`**](https://github.com/golang/sys/commit/56ede360ec1c541828fb88741b3f1049406d28f5).
- Upgrade [`golang.org/x/text`](https://github.com/golang/text) from [**`text@b19bf474d`**](https://github.com/golang/text/commit/b19bf474d317b857955b12035d2c5acb57ce8b01) to [**`v0.3.0`**](https://github.com/golang/text/releases/tag/v0.3.0).
- Upgrade [`golang.org/x/time`](https://github.com/golang/time) from [**`time@c06e80d93`**](https://github.com/golang/time/commit/c06e80d9300e4443158a03817b8a8cb37d230320) to [**`time@fbb02b229`**](https://github.com/golang/time/commit/fbb02b2291d28baffd63558aa44b4b56f178d650).
- Upgrade [`github.com/golang/protobuf`](https://github.com/golang/protobuf/releases) from [**`golang/protobuf@1e59b77b5`**](https://github.com/golang/protobuf/commit/1e59b77b52bf8e4b449a57e6f79f21226d571845) to [**`v1.3.2`**](https://github.com/golang/protobuf/releases/tag/v1.3.2).
- Upgrade [`gopkg.in/yaml.v2`](https://github.com/go-yaml/yaml/releases) from [**`yaml@cd8b52f82`**](https://github.com/go-yaml/yaml/commit/cd8b52f8269e0feb286dfeef29f8fe4d5b397e0b) to [**`yaml@5420a8b67`**](https://github.com/go-yaml/yaml/commit/5420a8b6744d3b0345ab293f6fcba19c978f1183).
- Upgrade [`github.com/dgrijalva/jwt-go`](https://github.com/dgrijalva/jwt-go/releases) from [**`v3.0.0`**](https://github.com/dgrijalva/jwt-go/releases/tag/v3.0.0) to [**`v3.2.0`**](https://github.com/dgrijalva/jwt-go/releases/tag/v3.2.0).
- Upgrade [`github.com/soheilhy/cmux`](https://github.com/soheilhy/cmux/releases) from [**`v0.1.3`**](https://github.com/soheilhy/cmux/releases/tag/v0.1.3) to [**`v0.1.4`**](https://github.com/soheilhy/cmux/releases/tag/v0.1.4).
- Upgrade [`github.com/google/btree`](https://github.com/google/btree/releases) from [**`google/btree@925471ac9`**](https://github.com/google/btree/commit/925471ac9e2131377a91e1595defec898166fe49) to [**`v1.0.0`**](https://github.com/google/btree/releases/tag/v1.0.0).
- Upgrade [`github.com/spf13/cobra`](https://github.com/spf13/cobra/releases) from [**`spf13/cobra@1c44ec8d3`**](https://github.com/spf13/cobra/commit/1c44ec8d3f1552cac48999f9306da23c4d8a288b) to [**`v0.0.3`**](https://github.com/spf13/cobra/releases/tag/v0.0.3).
- Upgrade [`github.com/spf13/pflag`](https://github.com/spf13/pflag/releases) from [**`v1.0.0`**](https://github.com/spf13/pflag/releases/tag/v1.0.0) to [**`spf13/pflag@1ce0cc6db`**](https://github.com/spf13/pflag/commit/1ce0cc6db4029d97571db82f85092fccedb572ce).
- Upgrade [`github.com/coreos/go-systemd`](https://github.com/coreos/go-systemd/releases) from [**`v15`**](https://github.com/coreos/go-systemd/releases/tag/v15) to [**`v17`**](https://github.com/coreos/go-systemd/releases/tag/v17).
- Upgrade [`github.com/prometheus/client_golang`](https://github.com/prometheus/client_golang/releases) from [**``prometheus/client_golang@5cec1d042``**](https://github.com/prometheus/client_golang/commit/5cec1d0429b02e4323e042eb04dafdb079ddf568) to [**`v1.0.0`**](https://github.com/prometheus/client_golang/releases/tag/v1.0.0).
- Upgrade [`github.com/grpc-ecosystem/go-grpc-prometheus`](https://github.com/grpc-ecosystem/go-grpc-prometheus/releases) from [**``grpc-ecosystem/go-grpc-prometheus@0dafe0d49``**](https://github.com/grpc-ecosystem/go-grpc-prometheus/commit/0dafe0d496ea71181bf2dd039e7e3f44b6bd11a7) to [**`v1.2.0`**](https://github.com/grpc-ecosystem/go-grpc-prometheus/releases/tag/v1.2.0).
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) from [**`v1.3.1`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3.1) to [**`v1.4.1`**](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.4.1).
- Migrate [`github.com/kr/pty`](https://github.com/kr/pty/releases) to [**`github.com/creack/pty`**](https://github.com/creack/pty/releases/tag/v1.1.7), as the later has replaced the original module.
- Upgrade [`github.com/gogo/protobuf`](https://github.com/gogo/protobuf/releases) from [**`v1.0.0`**](https://github.com/gogo/protobuf/releases/tag/v1.0.0) to [**`v1.2.1`**](https://github.com/gogo/protobuf/releases/tag/v1.2.1).
### Metrics, Monitoring
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Add [`etcd_snap_db_fsync_duration_seconds_count`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_snap_db_save_total_duration_seconds_bucket`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_send_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_success`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_failures`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_snapshot_receive_total_duration_seconds`](https://github.com/etcd-io/etcd/pull/9997) Prometheus metric.
- Add [`etcd_network_active_peers`](https://github.com/etcd-io/etcd/pull/9762) Prometheus metric.
- Let's say `"7339c4e5e833c029"` server `/metrics` returns `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="729934363faa4a24"} 1` and `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="b548c2511513015"} 1`. This indicates that the local node `"7339c4e5e833c029"` currently has two active remote peers `"729934363faa4a24"` and `"b548c2511513015"` in a 3-node cluster. If the node `"b548c2511513015"` is down, the local node `"7339c4e5e833c029"` will show `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="729934363faa4a24"} 1` and `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="b548c2511513015"} 0`.
- Add [`etcd_network_disconnected_peers_total`](https://github.com/etcd-io/etcd/pull/9762) Prometheus metric.
- If a remote peer `"b548c2511513015"` is down, the local node `"7339c4e5e833c029"` server `/metrics` would return `etcd_network_disconnected_peers_total{Local="7339c4e5e833c029",Remote="b548c2511513015"} 1`, while active peer metrics will show `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="729934363faa4a24"} 1` and `etcd_network_active_peers{Local="7339c4e5e833c029",Remote="b548c2511513015"} 0`.
- Add [`etcd_network_server_stream_failures_total`](https://github.com/etcd-io/etcd/pull/9760) Prometheus metric.
- e.g. `etcd_network_server_stream_failures_total{API="lease-keepalive",Type="receive"} 1`
- e.g. `etcd_network_server_stream_failures_total{API="watch",Type="receive"} 1`
- Improve [`etcd_network_peer_round_trip_time_seconds`](https://github.com/etcd-io/etcd/pull/10155) Prometheus metric to track leader heartbeats.
- Previously, it only samples the TCP connection for snapshot messages.
- Increase [`etcd_network_peer_round_trip_time_seconds`](https://github.com/etcd-io/etcd/pull/9762) Prometheus metric histogram upper-bound.
- Previously, highest bucket only collects requests taking 0.8192 seconds or more.
- Now, highest buckets collect 0.8192 seconds, 1.6384 seconds, and 3.2768 seconds or more.
- Add [`etcd_server_is_leader`](https://github.com/etcd-io/etcd/pull/9587) Prometheus metric.
- Add [`etcd_server_id`](https://github.com/etcd-io/etcd/pull/9998) Prometheus metric.
- Add [`etcd_cluster_version`](https://github.com/etcd-io/etcd/pull/10257) Prometheus metric.
- Add [`etcd_server_version`](https://github.com/etcd-io/etcd/pull/8960) Prometheus metric.
- To replace [Kubernetes `etcd-version-monitor`](https://github.com/etcd-io/etcd/issues/8948).
- Add [`etcd_server_go_version`](https://github.com/etcd-io/etcd/pull/9957) Prometheus metric.
- Add [`etcd_server_health_success`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_health_failures`](https://github.com/etcd-io/etcd/pull/10156) Prometheus metric.
- Add [`etcd_server_read_indexes_failed_total`](https://github.com/etcd-io/etcd/pull/10094) Prometheus metric.
- Add [`etcd_server_heartbeat_send_failures_total`](https://github.com/etcd-io/etcd/pull/9761) Prometheus metric.
- Add [`etcd_server_slow_apply_total`](https://github.com/etcd-io/etcd/pull/9761) Prometheus metric.
- Add [`etcd_server_slow_read_indexes_total`](https://github.com/etcd-io/etcd/pull/9897) Prometheus metric.
- Add [`etcd_server_quota_backend_bytes`](https://github.com/etcd-io/etcd/pull/9820) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
- Add [`etcd_mvcc_db_total_size_in_use_in_bytes`](https://github.com/etcd-io/etcd/pull/9256) Prometheus metric.
- Use it with `etcd_mvcc_db_total_size_in_bytes` and `etcd_mvcc_db_total_size_in_use_in_bytes`.
- `etcd_server_quota_backend_bytes 2.147483648e+09` means current quota size is 2 GB.
- `etcd_mvcc_db_total_size_in_bytes 20480` means current physically allocated DB size is 20 KB.
- `etcd_mvcc_db_total_size_in_use_in_bytes 16384` means future DB size if defragment operation is complete.
- `etcd_mvcc_db_total_size_in_bytes - etcd_mvcc_db_total_size_in_use_in_bytes` is the number of bytes that can be saved on disk with defragment operation.
- Add [`etcd_snap_fsync_duration_seconds`](https://github.com/etcd-io/etcd/pull/9762) Prometheus metric.
- Add [`etcd_disk_backend_defrag_duration_seconds`](https://github.com/etcd-io/etcd/pull/9761) Prometheus metric.
- Add [`etcd_mvcc_hash_duration_seconds`](https://github.com/etcd-io/etcd/pull/9761) Prometheus metric.
- Add [`etcd_mvcc_hash_rev_duration_seconds`](https://github.com/etcd-io/etcd/pull/9761) Prometheus metric.
- Add [`etcd_debugging_disk_backend_commit_rebalance_duration_seconds`](https://github.com/etcd-io/etcd/pull/9834) Prometheus metric.
- Add [`etcd_debugging_disk_backend_commit_spill_duration_seconds`](https://github.com/etcd-io/etcd/pull/9834) Prometheus metric.
- Add [`etcd_debugging_disk_backend_commit_write_duration_seconds`](https://github.com/etcd-io/etcd/pull/9834) Prometheus metric.
- Add [`etcd_debugging_lease_granted_total`](https://github.com/etcd-io/etcd/pull/9778) Prometheus metric.
- Add [`etcd_debugging_lease_revoked_total`](https://github.com/etcd-io/etcd/pull/9778) Prometheus metric.
- Add [`etcd_debugging_lease_renewed_total`](https://github.com/etcd-io/etcd/pull/9778) Prometheus metric.
- Add [`etcd_debugging_lease_ttl_total`](https://github.com/etcd-io/etcd/pull/9778) Prometheus metric.
- Add [`etcd_server_is_learner`](https://github.com/etcd-io/etcd/pull/10731) Prometheus metric.
- Add [`etcd_server_learner_promote_failures`](https://github.com/etcd-io/etcd/pull/10731) Prometheus metric.
- Add [`etcd_server_learner_promote_successes`](https://github.com/etcd-io/etcd/pull/10731) Prometheus metric.
- Increase [`etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds`](https://github.com/etcd-io/etcd/pull/9762) Prometheus metric histogram upper-bound.
- Previously, highest bucket only collects requests taking 1.024 seconds or more.
- Now, highest buckets collect 1.024 seconds, 2.048 seconds, and 4.096 seconds or more.
- Fix missing [`etcd_network_peer_sent_failures_total`](https://github.com/etcd-io/etcd/pull/9437) Prometheus metric count.
- Fix [`etcd_debugging_server_lease_expired_total`](https://github.com/etcd-io/etcd/pull/9557) Prometheus metric.
- Fix [race conditions in v2 server stat collecting](https://github.com/etcd-io/etcd/pull/9562).
- Change [gRPC proxy to expose etcd server endpoint /metrics](https://github.com/etcd-io/etcd/pull/10618).
- The metrics that were exposed via the proxy were not etcd server members but instead the proxy itself.
- Fix bug where [db_compaction_total_duration_milliseconds metric incorrectly measured duration as 0](https://github.com/etcd-io/etcd/pull/10646).
- Deprecating `etcd_debugging_mvcc_db_total_size_in_bytes` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_db_total_size_in_bytes`](https://github.com/etcd-io/etcd/pull/9819) instead.
- Deprecating `etcd_debugging_mvcc_put_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_put_total`](https://github.com/etcd-io/etcd/pull/10962) instead.
- Deprecating `etcd_debugging_mvcc_delete_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_delete_total`](https://github.com/etcd-io/etcd/pull/10962) instead.
- Deprecating `etcd_debugging_mvcc_range_total` Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_range_total`](https://github.com/etcd-io/etcd/pull/10968) instead.
- Deprecating `etcd_debugging_mvcc_txn_total`Prometheus metric (to be removed in v3.5). Use [`etcd_mvcc_txn_total`](https://github.com/etcd-io/etcd/pull/10968) instead.
### Security, Authentication
See [security doc](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/security.md) for more details.
- Support TLS cipher suite whitelisting.
- To block [weak cipher suites](https://github.com/etcd-io/etcd/issues/8320).
- TLS handshake fails when client hello is requested with invalid cipher suites.
- Add [`etcd --cipher-suites`](https://github.com/etcd-io/etcd/pull/9801) flag.
- If empty, Go auto-populates the list.
- Add [`etcd --host-whitelist`](https://github.com/etcd-io/etcd/pull/9372) flag, [`etcdserver.Config.HostWhitelist`](https://github.com/etcd-io/etcd/pull/9372), and [`embed.Config.HostWhitelist`](https://github.com/etcd-io/etcd/pull/9372), to prevent ["DNS Rebinding"](https://en.wikipedia.org/wiki/DNS_rebinding) attack.
- Any website can simply create an authorized DNS name, and direct DNS to `"localhost"` (or any other address). Then, all HTTP endpoints of etcd server listening on `"localhost"` becomes accessible, thus vulnerable to [DNS rebinding attacks (CVE-2018-5702)](https://bugs.chromium.org/p/project-zero/issues/detail?id=1447#c2).
- Client origin enforce policy works as follow:
- If client connection is secure via HTTPS, allow any hostnames..
- If client connection is not secure and `"HostWhitelist"` is not empty, only allow HTTP requests whose Host field is listed in whitelist.
- By default, `"HostWhitelist"` is `"*"`, which means insecure server allows all client HTTP requests.
- Note that the client origin policy is enforced whether authentication is enabled or not, for tighter controls.
- When specifying hostnames, loopback addresses are not added automatically. To allow loopback interfaces, add them to whitelist manually (e.g. `"localhost"`, `"127.0.0.1"`, etc.).
- e.g. `etcd --host-whitelist example.com`, then the server will reject all HTTP requests whose Host field is not `example.com` (also rejects requests to `"localhost"`).
- Support [`etcd --cors`](https://github.com/etcd-io/etcd/pull/9490) in v3 HTTP requests (gRPC gateway).
- Support [`ttl` field for `etcd` Authentication JWT token](https://github.com/etcd-io/etcd/pull/8302).
- e.g. `etcd --auth-token jwt,pub-key=<pub key path>,priv-key=<priv key path>,sign-method=<sign method>,ttl=5m`.
- Allow empty token provider in [`etcdserver.ServerConfig.AuthToken`](https://github.com/etcd-io/etcd/pull/9369).
- Fix [TLS reload](https://github.com/etcd-io/etcd/pull/9570) when [certificate SAN field only includes IP addresses but no domain names](https://github.com/etcd-io/etcd/issues/9541).
- In Go, server calls `(*tls.Config).GetCertificate` for TLS reload if and only if server's `(*tls.Config).Certificates` field is not empty, or `(*tls.ClientHelloInfo).ServerName` is not empty with a valid SNI from the client. Previously, etcd always populates `(*tls.Config).Certificates` on the initial client TLS handshake, as non-empty. Thus, client was always expected to supply a matching SNI in order to pass the TLS verification and to trigger `(*tls.Config).GetCertificate` to reload TLS assets.
- However, a certificate whose SAN field does [not include any domain names but only IP addresses](https://github.com/etcd-io/etcd/issues/9541) would request `*tls.ClientHelloInfo` with an empty `ServerName` field, thus failing to trigger the TLS reload on initial TLS handshake; this becomes a problem when expired certificates need to be replaced online.
- Now, `(*tls.Config).Certificates` is created empty on initial TLS client handshake, first to trigger `(*tls.Config).GetCertificate`, and then to populate rest of the certificates on every new TLS connection, even when client SNI is empty (e.g. cert only includes IPs).
### etcd server
- Add [`rpctypes.ErrLeaderChanged`](https://github.com/etcd-io/etcd/pull/10094).
- Now linearizable requests with read index would fail fast when there is a leadership change, instead of waiting until context timeout.
- Add [`etcd --initial-election-tick-advance`](https://github.com/etcd-io/etcd/pull/9591) flag to configure initial election tick fast-forward.
- By default, `etcd --initial-election-tick-advance=true`, then local member fast-forwards election ticks to speed up "initial" leader election trigger.
- This benefits the case of larger election ticks. For instance, cross datacenter deployment may require longer election timeout of 10-second. If true, local node does not need wait up to 10-second. Instead, forwards its election ticks to 8-second, and have only 2-second left before leader election.
- Major assumptions are that: cluster has no active leader thus advancing ticks enables faster leader election. Or cluster already has an established leader, and rejoining follower is likely to receive heartbeats from the leader after tick advance and before election timeout.
- However, when network from leader to rejoining follower is congested, and the follower does not receive leader heartbeat within left election ticks, disruptive election has to happen thus affecting cluster availabilities.
- Now, this can be disabled by setting `etcd --initial-election-tick-advance=false`.
- Disabling this would slow down initial bootstrap process for cross datacenter deployments. Make tradeoffs by configuring `etcd --initial-election-tick-advance` at the cost of slow initial bootstrap.
- If single-node, it advances ticks regardless.
- Address [disruptive rejoining follower node](https://github.com/etcd-io/etcd/issues/9333).
- Add [`etcd --pre-vote`](https://github.com/etcd-io/etcd/pull/9352) flag to enable to run an additional Raft election phase.
- For instance, a flaky(or rejoining) member may drop in and out, and start campaign. This member will end up with a higher term, and ignore all incoming messages with lower term. In this case, a new leader eventually need to get elected, thus disruptive to cluster availability. Raft implements Pre-Vote phase to prevent this kind of disruptions. If enabled, Raft runs an additional phase of election to check if pre-candidate can get enough votes to win an election.
- `etcd --pre-vote=false` by default.
- v3.5 will enable `etcd --pre-vote=true` by default.
- Add [`etcd --discovery-srv-name`](https://github.com/etcd-io/etcd/pull/8690) flag to support custom DNS SRV name with discovery.
- If not given, etcd queries `_etcd-server-ssl._tcp.[YOUR_HOST]` and `_etcd-server._tcp.[YOUR_HOST]`.
- If `etcd --discovery-srv-name="foo"`, then query `_etcd-server-ssl-foo._tcp.[YOUR_HOST]` and `_etcd-server-foo._tcp.[YOUR_HOST]`.
- Useful for operating multiple etcd clusters under the same domain.
- Support TLS cipher suite whitelisting.
- To block [weak cipher suites](https://github.com/etcd-io/etcd/issues/8320).
- TLS handshake fails when client hello is requested with invalid cipher suites.
- Add [`etcd --cipher-suites`](https://github.com/etcd-io/etcd/pull/9801) flag.
- If empty, Go auto-populates the list.
- Support [`etcd --cors`](https://github.com/etcd-io/etcd/pull/9490) in v3 HTTP requests (gRPC gateway).
- Rename [`etcd --log-output` to `etcd --log-outputs`](https://github.com/etcd-io/etcd/pull/9624) to support multiple log outputs.
- **`etcd --log-output` will be deprecated in v3.5**.
- Add [`etcd --logger`](https://github.com/etcd-io/etcd/pull/9572) flag to support [structured logger and multiple log outputs](https://github.com/etcd-io/etcd/issues/9438) in server-side.
- **`etcd --logger=capnslog` will be deprecated in v3.5**.
- Main motivation is to promote automated etcd monitoring, rather than looking back server logs when it starts breaking. Future development will make etcd log as few as possible, and make etcd easier to monitor with metrics and alerts.
- `etcd --logger=capnslog --log-outputs=default` is the default setting and same as previous etcd server logging format.
- `etcd --logger=zap --log-outputs=default` is not supported when `etcd --logger=zap`.
- Use `etcd --logger=zap --log-outputs=stderr` instead.
- Or, use `etcd --logger=zap --log-outputs=systemd/journal` to send logs to the local systemd journal.
- Previously, if etcd parent process ID (PPID) is 1 (e.g. run with systemd), `etcd --logger=capnslog --log-outputs=default` redirects server logs to local systemd journal. And if write to journald fails, it writes to `os.Stderr` as a fallback.
- However, even with PPID 1, it can fail to dial systemd journal (e.g. run embedded etcd with Docker container). Then, [every single log write will fail](https://github.com/etcd-io/etcd/pull/9729) and fall back to `os.Stderr`, which is inefficient.
- To avoid this problem, systemd journal logging must be configured manually.
- `etcd --logger=zap --log-outputs=stderr` will log server operations in [JSON-encoded format](https://godoc.org/go.uber.org/zap#NewProductionEncoderConfig) and writes logs to `os.Stderr`. Use this to override journald log redirects.
- `etcd --logger=zap --log-outputs=stdout` will log server operations in [JSON-encoded format](https://godoc.org/go.uber.org/zap#NewProductionEncoderConfig) and writes logs to `os.Stdout` Use this to override journald log redirects.
- `etcd --logger=zap --log-outputs=a.log` will log server operations in [JSON-encoded format](https://godoc.org/go.uber.org/zap#NewProductionEncoderConfig) and writes logs to the specified file `a.log`.
- `etcd --logger=zap --log-outputs=a.log,b.log,c.log,stdout` [writes server logs to multiple files `a.log`, `b.log` and `c.log` at the same time](https://github.com/etcd-io/etcd/pull/9579) and outputs to `os.Stderr`, in [JSON-encoded format](https://godoc.org/go.uber.org/zap#NewProductionEncoderConfig).
- `etcd --logger=zap --log-outputs=/dev/null` will discard all server logs.
- Add [`etcd --log-level`](https://github.com/etcd-io/etcd/pull/10947) flag to support log level.
- v3.5 will deprecate `etcd --debug` flag in favor of `etcd --log-level=debug`.
- Add [`etcd --backend-batch-limit`](https://github.com/etcd-io/etcd/pull/10283) flag.
- Add [`etcd --backend-batch-interval`](https://github.com/etcd-io/etcd/pull/10283) flag.
- Fix [`mvcc` "unsynced" watcher restore operation](https://github.com/etcd-io/etcd/pull/9281).
- "unsynced" watcher is watcher that needs to be in sync with events that have happened.
- That is, "unsynced" watcher is the slow watcher that was requested on old revision.
- "unsynced" watcher restore operation was not correctly populating its underlying watcher group.
- Which possibly causes [missing events from "unsynced" watchers](https://github.com/etcd-io/etcd/issues/9086).
- A node gets network partitioned with a watcher on a future revision, and falls behind receiving a leader snapshot after partition gets removed. When applying this snapshot, etcd watch storage moves current synced watchers to unsynced since sync watchers might have become stale during network partition. And reset synced watcher group to restart watcher routines. Previously, there was a bug when moving from synced watcher group to unsynced, thus client would miss events when the watcher was requested to the network-partitioned node.
- Fix [`mvcc` server panic from restore operation](https://github.com/etcd-io/etcd/pull/9775).
- Let's assume that a watcher had been requested with a future revision X and sent to node A that became network-partitioned thereafter. Meanwhile, cluster makes progress. Then when the partition gets removed, the leader sends a snapshot to node A. Previously if the snapshot's latest revision is still lower than the watch revision X, **etcd server panicked** during snapshot restore operation.
- Now, this server-side panic has been fixed.
- Fix [server panic on invalid Election Proclaim/Resign HTTP(S) requests](https://github.com/etcd-io/etcd/pull/9379).
- Previously, wrong-formatted HTTP requests to Election API could trigger panic in etcd server.
- e.g. `curl -L http://localhost:2379/v3/election/proclaim -X POST -d '{"value":""}'`, `curl -L http://localhost:2379/v3/election/resign -X POST -d '{"value":""}'`.
- Fix [revision-based compaction retention parsing](https://github.com/etcd-io/etcd/pull/9339).
- Previously, `etcd --auto-compaction-mode revision --auto-compaction-retention 1` was [translated to revision retention 3600000000000](https://github.com/etcd-io/etcd/issues/9337).
- Now, `etcd --auto-compaction-mode revision --auto-compaction-retention 1` is correctly parsed as revision retention 1.
- Prevent [overflow by large `TTL` values for `Lease` `Grant`](https://github.com/etcd-io/etcd/pull/9399).
- `TTL` parameter to `Grant` request is unit of second.
- Leases with too large `TTL` values exceeding `math.MaxInt64` [expire in unexpected ways](https://github.com/etcd-io/etcd/issues/9374).
- Server now returns `rpctypes.ErrLeaseTTLTooLarge` to client, when the requested `TTL` is larger than *9,000,000,000 seconds* (which is >285 years).
- Again, etcd `Lease` is meant for short-periodic keepalives or sessions, in the range of seconds or minutes. Not for hours or days!
- Fix [expired lease revoke](https://github.com/etcd-io/etcd/pull/10693).
- Fix ["the key is not deleted when the bound lease expires"](https://github.com/etcd-io/etcd/issues/10686).
- Enable etcd server [`raft.Config.CheckQuorum` when starting with `ForceNewCluster`](https://github.com/etcd-io/etcd/pull/9347).
- Allow [non-WAL files in `etcd --wal-dir` directory](https://github.com/etcd-io/etcd/pull/9743).
- Previously, existing files such as [`lost+found`](https://github.com/etcd-io/etcd/issues/7287) in WAL directory prevent etcd server boot.
- Now, WAL directory that contains only `lost+found` or a file that's not suffixed with `.wal` is considered non-initialized.
- Fix [`ETCD_CONFIG_FILE` env variable parsing in `etcd`](https://github.com/etcd-io/etcd/pull/10762).
- Fix [race condition in `rafthttp` transport pause/resume](https://github.com/etcd-io/etcd/pull/10826).
- Fix [server crash from creating an empty role](https://github.com/etcd-io/etcd/pull/10907).
- Previously, creating a role with an empty name crashed etcd server with an error code `Unavailable`.
- Now, creating a role with an empty name is not allowed with an error code `InvalidArgument`.
### API
- Add `isLearner` field to `etcdserverpb.Member`, `etcdserverpb.MemberAddRequest` and `etcdserverpb.StatusResponse` as part of [raft learner implementation](https://github.com/etcd-io/etcd/pull/10725).
- Add `MemberPromote` rpc to `etcdserverpb.Cluster` interface and the corresponding `MemberPromoteRequest` and `MemberPromoteResponse` as part of [raft learner implementation](https://github.com/etcd-io/etcd/pull/10725).
- Add [`snapshot`](https://github.com/etcd-io/etcd/pull/9118) package for snapshot restore/save operations (see [`godoc.org/github.com/etcd/clientv3/snapshot`](https://godoc.org/github.com/coreos/etcd/clientv3/snapshot) for more).
- Add [`watch_id` field to `etcdserverpb.WatchCreateRequest`](https://github.com/etcd-io/etcd/pull/9065) to allow user-provided watch ID to `mvcc`.
- Corresponding `watch_id` is returned via `etcdserverpb.WatchResponse`, if any.
- Add [`fragment` field to `etcdserverpb.WatchCreateRequest`](https://github.com/etcd-io/etcd/pull/9291) to request etcd server to [split watch events](https://github.com/etcd-io/etcd/issues/9294) when the total size of events exceeds `etcd --max-request-bytes` flag value plus gRPC-overhead 512 bytes.
- The default server-side request bytes limit is `embed.DefaultMaxRequestBytes` which is 1.5 MiB plus gRPC-overhead 512 bytes.
- If watch response events exceed this server-side request limit and watch request is created with `fragment` field `true`, the server will split watch events into a set of chunks, each of which is a subset of watch events below server-side request limit.
- Useful when client-side has limited bandwidths.
- For example, watch response contains 10 events, where each event is 1 MiB. And server `etcd --max-request-bytes` flag value is 1 MiB. Then, server will send 10 separate fragmented events to the client.
- For example, watch response contains 5 events, where each event is 2 MiB. And server `etcd --max-request-bytes` flag value is 1 MiB and `clientv3.Config.MaxCallRecvMsgSize` is 1 MiB. Then, server will try to send 5 separate fragmented events to the client, and the client will error with `"code = ResourceExhausted desc = grpc: received message larger than max (...)"`.
- Client must implement fragmented watch event merge (which `clientv3` does in etcd v3.4).
- Add [`raftAppliedIndex` field to `etcdserverpb.StatusResponse`](https://github.com/etcd-io/etcd/pull/9176) for current Raft applied index.
- Add [`errors` field to `etcdserverpb.StatusResponse`](https://github.com/etcd-io/etcd/pull/9206) for server-side error.
- e.g. `"etcdserver: no leader", "NOSPACE", "CORRUPT"`
- Add [`dbSizeInUse` field to `etcdserverpb.StatusResponse`](https://github.com/etcd-io/etcd/pull/9256) for actual DB size after compaction.
- Add [`WatchRequest.WatchProgressRequest`](https://github.com/etcd-io/etcd/pull/9869).
- To manually trigger broadcasting watch progress event (empty watch response with latest header) to all associated watch streams.
- Think of it as `WithProgressNotify` that can be triggered manually.
Note: **v3.5 will deprecate `etcd --log-package-levels` flag for `capnslog`**; `etcd --logger=zap --log-outputs=stderr` will the default. **v3.5 will deprecate `[CLIENT-URL]/config/local/log` endpoint.**
### Package `embed`
- Add [`embed.Config.CipherSuites`](https://github.com/etcd-io/etcd/pull/9801) to specify a list of supported cipher suites for TLS handshake between client/server and peers.
- If empty, Go auto-populates the list.
- Both `embed.Config.ClientTLSInfo.CipherSuites` and `embed.Config.CipherSuites` cannot be non-empty at the same time.
- If not empty, specify either `embed.Config.ClientTLSInfo.CipherSuites` or `embed.Config.CipherSuites`.
- Add [`embed.Config.InitialElectionTickAdvance`](https://github.com/etcd-io/etcd/pull/9591) to enable/disable initial election tick fast-forward.
- `embed.NewConfig()` would return `*embed.Config` with `InitialElectionTickAdvance` as true by default.
- Define [`embed.CompactorModePeriodic`](https://godoc.org/github.com/etcd-io/etcd/embed#pkg-variables) for `compactor.ModePeriodic`.
- Define [`embed.CompactorModeRevision`](https://godoc.org/github.com/etcd-io/etcd/embed#pkg-variables) for `compactor.ModeRevision`.
- Change [`embed.Config.CorsInfo` in `*cors.CORSInfo` type to `embed.Config.CORS` in `map[string]struct{}` type](https://github.com/etcd-io/etcd/pull/9490).
- Remove [`embed.Config.SetupLogging`](https://github.com/etcd-io/etcd/pull/9572).
- Now logger is set up automatically based on [`embed.Config.Logger`, `embed.Config.LogOutputs`, `embed.Config.Debug` fields](https://github.com/etcd-io/etcd/pull/9572).
- Add [`embed.Config.Logger`](https://github.com/etcd-io/etcd/pull/9518) to support [structured logger `zap`](https://github.com/uber-go/zap) in server-side.
- Add [`embed.Config.LogLevel`](https://github.com/etcd-io/etcd/pull/10947).
- Rename `embed.Config.SnapCount` field to [`embed.Config.SnapshotCount`](https://github.com/etcd-io/etcd/pull/9745), to be consistent with the flag name `etcd --snapshot-count`.
- Rename [**`embed.Config.LogOutput`** to **`embed.Config.LogOutputs`**](https://github.com/etcd-io/etcd/pull/9624) to support multiple log outputs.
- Change [**`embed.Config.LogOutputs`** type from `string` to `[]string`](https://github.com/etcd-io/etcd/pull/9579) to support multiple log outputs.
- Add [`embed.Config.BackendBatchLimit`](https://github.com/etcd-io/etcd/pull/10283) field.
- Add [`embed.Config.BackendBatchInterval`](https://github.com/etcd-io/etcd/pull/10283) field.
### Package `integration`
- Add [`CLUSTER_DEBUG` to enable test cluster logging](https://github.com/etcd-io/etcd/pull/9678).
- Deprecated `capnslog` in integration tests.
### client v3
- Add [`MemberAddAsLearner`](https://github.com/etcd-io/etcd/pull/10725) to `Clientv3.Cluster` interface. This API is used to add a learner member to etcd cluster.
- Add [`MemberPromote`](https://github.com/etcd-io/etcd/pull/10727) to `Clientv3.Cluster` interface. This API is used to promote a learner member in etcd cluster.
- Client may receive [`rpctypes.ErrLeaderChanged`](https://github.com/etcd-io/etcd/pull/10094) from server.
- Now linearizable requests with read index would fail fast when there is a leadership change, instead of waiting until context timeout.
- Add [`WithFragment` `OpOption`](https://github.com/etcd-io/etcd/pull/9291) to support [watch events fragmentation](https://github.com/etcd-io/etcd/issues/9294) when the total size of events exceeds `etcd --max-request-bytes` flag value plus gRPC-overhead 512 bytes.
- Watch fragmentation is disabled by default.
- The default server-side request bytes limit is `embed.DefaultMaxRequestBytes` which is 1.5 MiB plus gRPC-overhead 512 bytes.
- If watch response events exceed this server-side request limit and watch request is created with `fragment` field `true`, the server will split watch events into a set of chunks, each of which is a subset of watch events below server-side request limit.
- Useful when client-side has limited bandwidths.
- For example, watch response contains 10 events, where each event is 1 MiB. And server `etcd --max-request-bytes` flag value is 1 MiB. Then, server will send 10 separate fragmented events to the client.
- For example, watch response contains 5 events, where each event is 2 MiB. And server `etcd --max-request-bytes` flag value is 1 MiB and `clientv3.Config.MaxCallRecvMsgSize` is 1 MiB. Then, server will try to send 5 separate fragmented events to the client, and the client will error with `"code = ResourceExhausted desc = grpc: received message larger than max (...)"`.
- Add [`Watcher.RequestProgress` method](https://github.com/etcd-io/etcd/pull/9869).
- To manually trigger broadcasting watch progress event (empty watch response with latest header) to all associated watch streams.
- Think of it as `WithProgressNotify` that can be triggered manually.
- Fix [lease keepalive interval updates when response queue is full](https://github.com/etcd-io/etcd/pull/9952).
- If `<-chan *clientv3LeaseKeepAliveResponse` from `clientv3.Lease.KeepAlive` was never consumed or channel is full, client was [sending keepalive request every 500ms](https://github.com/etcd-io/etcd/issues/9911) instead of expected rate of every "TTL / 3" duration.
- Change [snapshot file permissions](https://github.com/etcd-io/etcd/pull/9977): On Linux, the snapshot file changes from readable by all (mode 0644) to readable by the user only (mode 0600).
- Client may choose to send keepalive pings to server using [`PermitWithoutStream`](https://github.com/etcd-io/etcd/pull/10146).
- By setting `PermitWithoutStream` to true, client can send keepalive pings to server without any active streams(RPCs). In other words, it allows sending keepalive pings with unary or simple RPC calls.
- `PermitWithoutStream` is set to false by default.
- Fix logic on [release lock key if cancelled](https://github.com/etcd-io/etcd/pull/10153) in `clientv3/concurrency` package.
- Fix [`(*Client).Endpoints()` method race condition](https://github.com/etcd-io/etcd/pull/10595).
### etcdctl v3
- Make [`ETCDCTL_API=3 etcdctl` default](https://github.com/etcd-io/etcd/issues/9600).
- Now, `etcdctl set foo bar` must be `ETCDCTL_API=2 etcdctl set foo bar`.
- Now, `ETCDCTL_API=3 etcdctl put foo bar` could be just `etcdctl put foo bar`.
- Add [`etcdctl member add --learner` and `etcdctl member promote`](https://github.com/etcd-io/etcd/pull/10725) to add and promote raft learner member in etcd cluster.
- Add [`etcdctl --password`](https://github.com/etcd-io/etcd/pull/9730) flag.
- To support [`:` character in user name](https://github.com/etcd-io/etcd/issues/9691).
- e.g. `etcdctl --user user --password password get foo`
- Add [`etcdctl user add --new-user-password`](https://github.com/etcd-io/etcd/pull/9730) flag.
- Add [`etcdctl check datascale`](https://github.com/etcd-io/etcd/pull/9185) command.
- Add [`etcdctl check datascale --auto-compact, --auto-defrag`](https://github.com/etcd-io/etcd/pull/9351) flags.
- Add [`etcdctl check perf --auto-compact, --auto-defrag`](https://github.com/etcd-io/etcd/pull/9330) flags.
- Add [`etcdctl defrag --cluster`](https://github.com/etcd-io/etcd/pull/9390) flag.
- Add ["raft applied index" field to `endpoint status`](https://github.com/etcd-io/etcd/pull/9176).
- Add ["errors" field to `endpoint status`](https://github.com/etcd-io/etcd/pull/9206).
- Add [`etcdctl endpoint health --write-out` support](https://github.com/etcd-io/etcd/pull/9540).
- Previously, [`etcdctl endpoint health --write-out json` did not work](https://github.com/etcd-io/etcd/issues/9532).
- Add [missing newline in `etcdctl endpoint health`](https://github.com/etcd-io/etcd/pull/10793).
- Fix [`etcdctl watch [key] [range_end] -- [exec-command…]`](https://github.com/etcd-io/etcd/pull/9688) parsing.
- Previously, `ETCDCTL_API=3 etcdctl watch foo -- echo watch event received` panicked.
- Fix [`etcdctl move-leader` command for TLS-enabled endpoints](https://github.com/etcd-io/etcd/pull/9807).
- Add [`progress` command to `etcdctl watch --interactive`](https://github.com/etcd-io/etcd/pull/9869).
- To manually trigger broadcasting watch progress event (empty watch response with latest header) to all associated watch streams.
- Think of it as `WithProgressNotify` that can be triggered manually.
- Add [timeout](https://github.com/etcd-io/etcd/pull/10301) to `etcdctl snapshot
save`.
- User can specify timeout of `etcdctl snapshot save` command using flag `--command-timeout`.
- Fix etcdctl to [strip out insecure endpoints from DNS SRV records when using discovery](https://github.com/etcd-io/etcd/pull/10443)
### gRPC proxy
- Fix [etcd server panic from restore operation](https://github.com/etcd-io/etcd/pull/9775).
- Let's assume that a watcher had been requested with a future revision X and sent to node A that became network-partitioned thereafter. Meanwhile, cluster makes progress. Then when the partition gets removed, the leader sends a snapshot to node A. Previously if the snapshot's latest revision is still lower than the watch revision X, **etcd server panicked** during snapshot restore operation.
- Especially, gRPC proxy was affected, since it detects a leader loss with a key `"proxy-namespace__lostleader"` and a watch revision `"int64(math.MaxInt64 - 2)"`.
- Now, this server-side panic has been fixed.
- Fix [memory leak in cache layer](https://github.com/etcd-io/etcd/pull/10327).
- Change [gRPC proxy to expose etcd server endpoint /metrics](https://github.com/etcd-io/etcd/pull/10618).
- The metrics that were exposed via the proxy were not etcd server members but instead the proxy itself.
### gRPC gateway
- Replace [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) endpoint `/v3beta` with [`/v3`](https://github.com/etcd-io/etcd/pull/9298).
- Deprecated [`/v3alpha`](https://github.com/etcd-io/etcd/pull/9298).
- To deprecate [`/v3beta`](https://github.com/etcd-io/etcd/issues/9189) in v3.5.
- In v3.4, `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` still works as a fallback to `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'`, but `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` won't work in v3.5. Use `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
- Add API endpoints [`/{v3beta,v3}/lease/leases, /{v3beta,v3}/lease/revoke, /{v3beta,v3}/lease/timetolive`](https://github.com/etcd-io/etcd/pull/9450).
- To deprecate [`/{v3beta,v3}/kv/lease/leases, /{v3beta,v3}/kv/lease/revoke, /{v3beta,v3}/kv/lease/timetolive`](https://github.com/etcd-io/etcd/issues/9430) in v3.5.
- Support [`etcd --cors`](https://github.com/etcd-io/etcd/pull/9490) in v3 HTTP requests (gRPC gateway).
### Package `raft`
- Fix [deadlock during PreVote migration process](https://github.com/etcd-io/etcd/pull/8525).
- Add [`raft.ErrProposalDropped`](https://github.com/etcd-io/etcd/pull/9067).
- Now [`(r *raft) Step` returns `raft.ErrProposalDropped`](https://github.com/etcd-io/etcd/pull/9137) if a proposal has been ignored.
- e.g. a node is removed from cluster, or [`raftpb.MsgProp` arrives at current leader while there is an ongoing leadership transfer](https://github.com/etcd-io/etcd/issues/8975).
- Improve [Raft `becomeLeader` and `stepLeader`](https://github.com/etcd-io/etcd/pull/9073) by keeping track of latest `pb.EntryConfChange` index.
- Previously record `pendingConf` boolean field scanning the entire tail of the log, which can delay hearbeat send.
- Fix [missing learner nodes on `(n *node) ApplyConfChange`](https://github.com/etcd-io/etcd/pull/9116).
- Add [`raft.Config.MaxUncommittedEntriesSize`](https://github.com/etcd-io/etcd/pull/10167) to limit the total size of the uncommitted entries in bytes.
- Once exceeded, raft returns `raft.ErrProposalDropped` error.
- Prevent [unbounded Raft log growth](https://github.com/cockroachdb/cockroach/issues/27772).
- There was a bug in [PR#10167](https://github.com/etcd-io/etcd/pull/10167) but fixed via [PR#10199](https://github.com/etcd-io/etcd/pull/10199).
- Add [`raft.Ready.CommittedEntries` pagination using `raft.Config.MaxSizePerMsg`](https://github.com/etcd-io/etcd/pull/9982).
- This prevents out-of-memory errors if the raft log has become very large and commits all at once.
- Fix [correctness bug in CommittedEntries pagination](https://github.com/etcd-io/etcd/pull/10063).
- Optimize [message send flow control](https://github.com/etcd-io/etcd/pull/9985).
- Leader now sends more append entries if it has more non-empty entries to send after updating flow control information.
- Now, Raft allows multiple in-flight append messages.
- Optimize [memory allocation when boxing slice in `maybeCommit`](https://github.com/etcd-io/etcd/pull/10679).
- By boxing a heap-allocated slice header instead of the slice header on the stack, we can avoid an allocation when passing through the sort.Interface interface.
- Avoid [memory allocation in Raft entry `String` method](https://github.com/etcd-io/etcd/pull/10680).
- Avoid [multiple memory allocations when merging stable and unstable log](https://github.com/etcd-io/etcd/pull/10684).
- Extract [progress tracking into own component](https://github.com/etcd-io/etcd/pull/10683).
- Add [package `raft/tracker`](https://github.com/etcd-io/etcd/pull/10807).
- Optimize [string representation of `Progress`](https://github.com/etcd-io/etcd/pull/10882).
- Make [relationship between `node` and `RawNode` explicit](https://github.com/etcd-io/etcd/pull/10803).
- Prevent [learners from becoming leader](https://github.com/etcd-io/etcd/pull/10822).
- Add [package `raft/quorum` to reason about committed indexes as well as vote outcomes for both majority and joint quorums](https://github.com/etcd-io/etcd/pull/10779).
- Bundle [Voters and Learner into `raft/tracker.Config` struct](https://github.com/etcd-io/etcd/pull/10865).
- Use [membership sets in progress tracking](https://github.com/etcd-io/etcd/pull/10779).
- Implement [joint quorum computation](https://github.com/etcd-io/etcd/pull/10779).
- Refactor [`raft/node.go` to centralize configuration change application](https://github.com/etcd-io/etcd/pull/10865).
- Allow [voter to become learner through snapshot](https://github.com/etcd-io/etcd/pull/10864).
- Add [package `raft/confchange` to internally support joint consensus](https://github.com/etcd-io/etcd/pull/10779).
- Use [`RawNode` for node's event loop](https://github.com/etcd-io/etcd/pull/10892).
- Add [`RawNode.Bootstrap` method](https://github.com/etcd-io/etcd/pull/10892).
- Add [`raftpb.ConfChangeV2` to use joint quorums](https://github.com/etcd-io/etcd/pull/10914).
- `raftpb.ConfChange` continues to work as today: it allows carrying out a single configuration change. A `pb.ConfChange` proposal gets added to the Raft log as such and is thus also observed by the app during Ready handling, and fed back to ApplyConfChange.
- `raftpb.ConfChangeV2` allows joint configuration changes but will continue to carry out configuration changes in "one phase" (i.e. without ever entering a joint config) when this is possible.
- `raftpb.ConfChangeV2` messages initiate configuration changes. They support both the simple "one at a time" membership change protocol and full Joint Consensus allowing for arbitrary changes in membership.
- Change [`raftpb.ConfState.Nodes` to `raftpb.ConfState.Voters`](https://github.com/etcd-io/etcd/pull/10914).
### Package `wal`
- Add [`Verify` function to perform corruption check on WAL contents](https://github.com/etcd-io/etcd/pull/10603).
- Fix [`wal` directory cleanup on creation failures](https://github.com/etcd-io/etcd/pull/10689).
### Tooling
- Add [`etcd-dump-logs --entry-type`](https://github.com/etcd-io/etcd/pull/9628) flag to support WAL log filtering by entry type.
- Add [`etcd-dump-logs --stream-decoder`](https://github.com/etcd-io/etcd/pull/9790) flag to support custom decoder.
### Go
- Require [*Go 1.12+*](https://github.com/etcd-io/etcd/pull/10045).
- Compile with [*Go 1.12*](https://golang.org/doc/devel/release.html#go1.12).
### Dockerfile
- [Rebase etcd image from Alpine to Debian](https://github.com/etcd-io/etcd/pull/10805) to improve security and maintenance effort for etcd release.
<hr>

View File

@ -1,80 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.4](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.4.md).
<hr>
## v3.5.0 (TBD)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.4.0...v3.5.0) and [v3.5 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_5.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v3.5 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_3_5.md).**
### Breaking Changes
- [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) only supports [`/v3`](TODO) endpoint.
- Deprecated [`/v3beta`](https://github.com/etcd-io/etcd/pull/9298).
- `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` does work in v3.5. Use `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
- **`etcd --experimental-enable-v2v3` flag has been deprecated.** Use **`etcd --enable-v2v3`** instead.
- Change [`etcd --experimental-enable-v2v3`](TODO) flag to `etcd --enable-v2v3`; v2 storage emulation is now stable.
- **`etcd --debug` flag has been deprecated.** Use **`etcd --log-level=debug`** instead.
- Remove [`embed.Config.Debug`](https://github.com/etcd-io/etcd/pull/10947).
- **`etcd --log-output` flag has been deprecated.** Use **`etcd --log-outputs`** instead.
- **`etcd --logger=zap --log-outputs=stderr`** is now the default.
- **`etcd --logger=capnslog` flag value has been deprecated.**
- **`etcd --logger=zap --log-outputs=default` flag value is not supported.**.
- Use `etcd --logger=zap --log-outputs=stderr`.
- Or, use `etcd --logger=zap --log-outputs=systemd/journal` to send logs to the local systemd journal.
- Previously, if etcd parent process ID (PPID) is 1 (e.g. run with systemd), `etcd --logger=capnslog --log-outputs=default` redirects server logs to local systemd journal. And if write to journald fails, it writes to `os.Stderr` as a fallback.
- However, even with PPID 1, it can fail to dial systemd journal (e.g. run embedded etcd with Docker container). Then, [every single log write will fail](https://github.com/etcd-io/etcd/pull/9729) and fall back to `os.Stderr`, which is inefficient.
- To avoid this problem, systemd journal logging must be configured manually.
- **`etcd --log-outputs=stderr`** is now the default.
- **`etcd --log-package-levels` flag for `capnslog` has been deprecated.** Now, **`etcd --logger=zap --log-outputs=stderr`** is the default.
- **`[CLIENT-URL]/config/local/log` endpoint has been deprecated, as is `etcd --log-package-levels` flag.**
- `curl http://127.0.0.1:2379/config/local/log -XPUT -d '{"Level":"DEBUG"}'` won't work.
- Please use `etcd --logger=zap --log-outputs=stderr` instead.
- Deprecated `etcd_debugging_mvcc_db_total_size_in_bytes` Prometheus metric. Use `etcd_mvcc_db_total_size_in_bytes` instead.
- Deprecated `etcd_debugging_mvcc_put_total` Prometheus metric. Use `etcd_mvcc_put_total` instead.
- Deprecated `etcd_debugging_mvcc_delete_total` Prometheus metric. Use `etcd_mvcc_delete_total` instead.
- Deprecated `etcd_debugging_mvcc_txn_total` Prometheus metric. Use `etcd_mvcc_txn_total` instead.
- Deprecated `etcd_debugging_mvcc_range_total` Prometheus metric. Use `etcd_mvcc_range_total` instead.
- Master branch `/version` outputs `3.5.0-pre`, instead of `3.4.0+git`.
### Metrics, Monitoring
Note that any `etcd_debugging_*` metrics are experimental and subject to change.
- Deprecated `etcd_debugging_mvcc_db_total_size_in_bytes` Prometheus metric. Use `etcd_mvcc_db_total_size_in_bytes` instead.
- Deprecated `etcd_debugging_mvcc_put_total` Prometheus metric. Use `etcd_mvcc_put_total` instead.
- Deprecated `etcd_debugging_mvcc_delete_total` Prometheus metric. Use `etcd_mvcc_delete_total` instead.
- Deprecated `etcd_debugging_mvcc_txn_total` Prometheus metric. Use `etcd_mvcc_txn_total` instead.
- Deprecated `etcd_debugging_mvcc_range_total` Prometheus metric. Use `etcd_mvcc_range_total` instead.
### etcd server
- [`etcd --enable-v2v3`](TODO) flag is now stable.
- `etcd --experimental-enable-v2v3` has been deprecated.
- Added [more v2v3 integration tests](https://github.com/etcd-io/etcd/pull/9634).
- `etcd --enable-v2=true --enable-v2v3=''` by default, to enable v2 API server that is backed by **v2 store**.
- `etcd --enable-v2=true --enable-v2v3=/aaa` to enable v2 API server that is backed by **v3 storage**.
- `etcd --enable-v2=false --enable-v2v3=''` to disable v2 API server.
- `etcd --enable-v2=false --enable-v2v3=/aaa` to disable v2 API server. TODO: error?
- Automatically [create parent directory if it does not exist](https://github.com/etcd-io/etcd/pull/9626) (fix [issue#9609](https://github.com/etcd-io/etcd/issues/9609)).
- v4.0 will configure `etcd --enable-v2=true --enable-v2v3=/aaa` to enable v2 API server that is backed by **v3 storage**.
### Package `embed`
- Remove [`embed.Config.Debug`](https://github.com/etcd-io/etcd/pull/10947).
- Use `embed.Config.LogLevel` instead.
### gRPC gateway
- [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) only supports [`/v3`](TODO) endpoint.
- Deprecated [`/v3beta`](https://github.com/etcd-io/etcd/pull/9298).
- `curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` does work in v3.5. Use `curl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'` instead.
<hr>

View File

@ -1,48 +0,0 @@
Previous change logs can be found at [CHANGELOG-3.x](https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.x.md).
<hr>
## v4.0.0 (TBD)
See [code changes](https://github.com/etcd-io/etcd/compare/v3.5.0...v4.0.0) and [v4.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_4_0.md) for any breaking changes.
**Again, before running upgrades from any previous release, please make sure to read change logs below and [v4.0 upgrade guide](https://github.com/etcd-io/etcd/blob/master/Documentation/upgrades/upgrade_4_0.md).**
### Breaking Changes
- [Secure etcd by default](https://github.com/etcd-io/etcd/issues/9475)?
- Change `/health` endpoint output.
- Previously, `{"health":"true"}`.
- Now, `{"health":true}`.
- Breaks [Kubernetes `kubectl get componentstatuses` command](https://github.com/kubernetes/kubernetes/issues/58240).
- Deprecate [`etcd --proxy*`](TODO) flags; **no more v2 proxy**.
- Deprecate [v2 storage backend](https://github.com/etcd-io/etcd/issues/9232); **no more v2 store**.
- v2 API is still supported via [v2 emulation](TODO).
- Deprecate [`etcdctl backup`](TODO) command.
- `clientv3.Client.KeepAlive(ctx context.Context, id LeaseID) (<-chan *LeaseKeepAliveResponse, error)` is now [`clientv4.Client.KeepAlive(ctx context.Context, id LeaseID) <-chan *LeaseKeepAliveResponse`](TODO).
- Similar to `Watch`, [`KeepAlive` does not return errors](https://github.com/etcd-io/etcd/issues/7488).
- If there's an unknown server error, kill all open channels and create a new stream on the next `KeepAlive` call.
- Rename `github.com/coreos/client` to `github.com/coreos/clientv2`.
- [`etcd --experimental-initial-corrupt-check`](TODO) has been deprecated.
- Use [`etcd --initial-corrupt-check`](TODO) instead.
- [`etcd --experimental-corrupt-check-time`](TODO) has been deprecated.
- Use [`etcd --corrupt-check-time`](TODO) instead.
### etcd server
- [`etcd --initial-corrupt-check`](TODO) flag is now stable (`etcd --experimental-initial-corrupt-check` has been deprecated).
- `etcd --initial-corrupt-check=true` by default, to check cluster database hashes before serving client/peer traffic.
- [`etcd --corrupt-check-time`](TODO) flag is now stable (`etcd --experimental-corrupt-check-time` has been deprecated).
- `etcd --corrupt-check-time=12h` by default, to check cluster database hashes for every 12-hour.
### Go
- Require [*Go 2*](https://blog.golang.org/go2draft)?
<hr>

View File

@ -6,6 +6,7 @@ etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
- Email: [etcd-dev](https://groups.google.com/forum/?hl=en#!forum/etcd-dev)
- IRC: #[etcd](irc://irc.freenode.org:6667/#etcd) IRC channel on freenode.org
- Slack: [#etcd](https://kubernetes.slack.com/messages/C3HD8ARJ5/details/)
## Getting started
@ -14,13 +15,13 @@ etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
## Reporting bugs and creating issues
Reporting bugs is one of the best ways to contribute. However, a good bug report has some very specific qualities, so please read over our short document on [reporting bugs](https://github.com/etcd-io/etcd/blob/master/Documentation/reporting_bugs.md) before submitting a bug report. This document might contain links to known issues, another good reason to take a look there before reporting a bug.
Reporting bugs is one of the best ways to contribute. However, a good bug report has some very specific qualities, so please read over our short document on [reporting bugs](https://etcd.io/docs/latest/reporting_bugs) before submitting a bug report. This document might contain links to known issues, another good reason to take a look there before reporting a bug.
## Contribution flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where to base the contribution. This is usually master.
- Create a topic branch from where to base the contribution. This is usually main.
- Make commits of logical units.
- Make sure commit messages are in the proper format (see below).
- Push changes in a topic branch to a personal fork of the repository.

View File

@ -1,7 +1,8 @@
FROM k8s.gcr.io/debian-base:v1.0.0
FROM k8s.gcr.io/build-image/debian-base:buster-v1.4.0
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD etcdutl /usr/local/bin/
RUN mkdir -p /var/etcd/
RUN mkdir -p /var/lib/etcd/

View File

@ -1,9 +1,11 @@
FROM k8s.gcr.io/debian-base-arm64:v1.0.0
FROM k8s.gcr.io/build-image/debian-base-arm64:buster-v1.4.0
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD etcdutl /usr/local/bin/
ADD var/etcd /var/etcd
ADD var/lib/etcd /var/lib/etcd
ENV ETCD_UNSUPPORTED_ARCH=arm64
EXPOSE 2379 2380

View File

@ -1,7 +1,8 @@
FROM k8s.gcr.io/debian-base-ppc64le:v1.0.0
FROM k8s.gcr.io/build-image/debian-base-ppc64le:buster-v1.4.0
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD etcdutl /usr/local/bin/
ADD var/etcd /var/etcd
ADD var/lib/etcd /var/lib/etcd

12
Dockerfile-release.s390x Normal file
View File

@ -0,0 +1,12 @@
FROM k8s.gcr.io/build-image/debian-base-s390x:buster-v1.4.0
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD etcdutl /usr/local/bin/
ADD var/etcd /var/etcd
ADD var/lib/etcd /var/lib/etcd
EXPOSE 2379 2380
# Define default command.
CMD ["/usr/local/bin/etcd"]

View File

@ -1 +0,0 @@
docs.md

6
Documentation/README.md Normal file
View File

@ -0,0 +1,6 @@
# The etcd documentation
etcd is a distributed key-value store designed to reliably and quickly preserve and provide access to critical data. It enables reliable distributed coordination through distributed locking, leader elections, and write barriers. An etcd cluster is intended for high availability and permanent data storage and retrieval.
These documents have moved to the [etcd-io/website repo](https://github.com/etcd-io/website/), and can be viewed live at [https://etcd.io/docs/latest/](https://etcd.io/docs/latest/).

View File

@ -1,18 +0,0 @@
# Benchmarks
etcd benchmarks will be published regularly and tracked for each release below:
- [etcd v2.1.0-alpha][2.1]
- [etcd v2.2.0-rc][2.2]
- [etcd v3 demo][3.0]
# Memory Usage Benchmarks
It records expected memory usage in different scenarios.
- [etcd v2.2.0-rc][2.2-mem]
[2.1]: etcd-2-1-0-alpha-benchmarks.md
[2.2]: etcd-2-2-0-rc-benchmarks.md
[2.2-mem]: etcd-2-2-0-rc-memory-benchmarks.md
[3.0]: etcd-3-demo-benchmarks.md

View File

@ -1,56 +0,0 @@
---
title: Benchmarking etcd v2.1.0
---
## Physical machines
GCE n1-highcpu-2 machine type
- 1x dedicated local SSD mounted under /var/lib/etcd
- 1x dedicated slow disk for the OS
- 1.8 GB memory
- 2x CPUs
- etcd version 2.1.0 alpha
## etcd Cluster
3 etcd members, each runs on a single machine
## Testing
Bootstrap another machine and use the [hey HTTP benchmark tool][hey] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.
## Performance
### reading one single key
| key size in bytes | number of clients | target etcd server | read QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|--------------------|----------|---------------|
| 64 | 1 | leader only | 1534 | 0.7 |
| 64 | 64 | leader only | 10125 | 9.1 |
| 64 | 256 | leader only | 13892 | 27.1 |
| 256 | 1 | leader only | 1530 | 0.8 |
| 256 | 64 | leader only | 10106 | 10.1 |
| 256 | 256 | leader only | 14667 | 27.0 |
| 64 | 64 | all servers | 24200 | 3.9 |
| 64 | 256 | all servers | 33300 | 11.8 |
| 256 | 64 | all servers | 24800 | 3.9 |
| 256 | 256 | all servers | 33000 | 11.5 |
### writing one single key
| key size in bytes | number of clients | target etcd server | write QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|--------------------|-----------|---------------|
| 64 | 1 | leader only | 60 | 21.4 |
| 64 | 64 | leader only | 1742 | 46.8 |
| 64 | 256 | leader only | 3982 | 90.5 |
| 256 | 1 | leader only | 58 | 20.3 |
| 256 | 64 | leader only | 1770 | 47.8 |
| 256 | 256 | leader only | 4157 | 105.3 |
| 64 | 64 | all servers | 1028 | 123.4 |
| 64 | 256 | all servers | 3260 | 123.8 |
| 256 | 64 | all servers | 1033 | 121.5 |
| 256 | 256 | all servers | 3061 | 119.3 |
[hey]: https://github.com/rakyll/hey
[hack-benchmark]: https://github.com/coreos/etcd/tree/master/hack/benchmark

View File

@ -1,71 +0,0 @@
---
title: Benchmarking etcd v2.2.0
---
## Physical Machines
GCE n1-highcpu-2 machine type
- 1x dedicated local SSD mounted as etcd data directory
- 1x dedicated slow disk for the OS
- 1.8 GB memory
- 2x CPUs
## etcd Cluster
3 etcd 2.2.0 members, each runs on a single machine.
Detailed versions:
```
etcd Version: 2.2.0
Git SHA: e4561dd
Go Version: go1.5
Go OS/Arch: linux/amd64
```
## Testing
Bootstrap another machine, outside of the etcd cluster, and run the [`hey` HTTP benchmark tool](https://github.com/rakyll/hey) with a connection reuse patch to send requests to each etcd cluster member. See the [benchmark instructions](../../hack/benchmark/) for the patch and the steps to reproduce our procedures.
The performance is calculated through results of 100 benchmark rounds.
## Performance
### Single Key Read Performance
| key size in bytes | number of clients | target etcd server | average read QPS | read QPS stddev | average 90th Percentile Latency (ms) | latency stddev |
|-------------------|-------------------|--------------------|------------------|-----------------|--------------------------------------|----------------|
| 64 | 1 | leader only | 2303 | 200 | 0.49 | 0.06 |
| 64 | 64 | leader only | 15048 | 685 | 7.60 | 0.46 |
| 64 | 256 | leader only | 14508 | 434 | 29.76 | 1.05 |
| 256 | 1 | leader only | 2162 | 214 | 0.52 | 0.06 |
| 256 | 64 | leader only | 14789 | 792 | 7.69| 0.48 |
| 256 | 256 | leader only | 14424 | 512 | 29.92 | 1.42 |
| 64 | 64 | all servers | 45752 | 2048 | 2.47 | 0.14 |
| 64 | 256 | all servers | 46592 | 1273 | 10.14 | 0.59 |
| 256 | 64 | all servers | 45332 | 1847 | 2.48| 0.12 |
| 256 | 256 | all servers | 46485 | 1340 | 10.18 | 0.74 |
### Single Key Write Performance
| key size in bytes | number of clients | target etcd server | average write QPS | write QPS stddev | average 90th Percentile Latency (ms) | latency stddev |
|-------------------|-------------------|--------------------|------------------|-----------------|--------------------------------------|----------------|
| 64 | 1 | leader only | 55 | 4 | 24.51 | 13.26 |
| 64 | 64 | leader only | 2139 | 125 | 35.23 | 3.40 |
| 64 | 256 | leader only | 4581 | 581 | 70.53 | 10.22 |
| 256 | 1 | leader only | 56 | 4 | 22.37| 4.33 |
| 256 | 64 | leader only | 2052 | 151 | 36.83 | 4.20 |
| 256 | 256 | leader only | 4442 | 560 | 71.59 | 10.03 |
| 64 | 64 | all servers | 1625 | 85 | 58.51 | 5.14 |
| 64 | 256 | all servers | 4461 | 298 | 89.47 | 36.48 |
| 256 | 64 | all servers | 1599 | 94 | 60.11| 6.43 |
| 256 | 256 | all servers | 4315 | 193 | 88.98 | 7.01 |
## Performance Changes
- Because etcd now records metrics for each API call, read QPS performance seems to see a minor decrease in most scenarios. This minimal performance impact was judged a reasonable investment for the breadth of monitoring and debugging information returned.
- Write QPS to cluster leaders seems to be increased by a small margin. This is because the main loop and entry apply loops were decoupled in the etcd raft logic, eliminating several blocks between them.
- Write QPS to all members seems to be increased by a significant margin, because followers now receive the latest commit index sooner, and commit proposals more quickly.

View File

@ -1,76 +0,0 @@
---
title: Benchmarking etcd v2.2.0-rc
---
## Physical machine
GCE n1-highcpu-2 machine type
- 1x dedicated local SSD mounted under /var/lib/etcd
- 1x dedicated slow disk for the OS
- 1.8 GB memory
- 2x CPUs
## etcd Cluster
3 etcd 2.2.0-rc members, each runs on a single machine.
Detailed versions:
```
etcd Version: 2.2.0-alpha.1+git
Git SHA: 59a5a7e
Go Version: go1.4.2
Go OS/Arch: linux/amd64
```
Also, we use 3 etcd 2.1.0 alpha-stage members to form cluster to get base performance. etcd's commit head is at [c7146bd5][c7146bd5], which is the same as the one that we use in [etcd 2.1 benchmark][etcd-2.1-benchmark].
## Testing
Bootstrap another machine and use the [hey HTTP benchmark tool][hey] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions.
## Performance
### reading one single key
| key size in bytes | number of clients | target etcd server | read QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|--------------------|----------|---------------|
| 64 | 1 | leader only | 2804 (-5%) | 0.4 (+0%) |
| 64 | 64 | leader only | 17816 (+0%) | 5.7 (-6%) |
| 64 | 256 | leader only | 18667 (-6%) | 20.4 (+2%) |
| 256 | 1 | leader only | 2181 (-15%) | 0.5 (+25%) |
| 256 | 64 | leader only | 17435 (-7%) | 6.0 (+9%) |
| 256 | 256 | leader only | 18180 (-8%) | 21.3 (+3%) |
| 64 | 64 | all servers | 46965 (-4%) | 2.1 (+0%) |
| 64 | 256 | all servers | 55286 (-6%) | 7.4 (+6%) |
| 256 | 64 | all servers | 46603 (-6%) | 2.1 (+5%) |
| 256 | 256 | all servers | 55291 (-6%) | 7.3 (+4%) |
### writing one single key
| key size in bytes | number of clients | target etcd server | write QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|--------------------|-----------|---------------|
| 64 | 1 | leader only | 76 (+22%) | 19.4 (-15%) |
| 64 | 64 | leader only | 2461 (+45%) | 31.8 (-32%) |
| 64 | 256 | leader only | 4275 (+1%) | 69.6 (-10%) |
| 256 | 1 | leader only | 64 (+20%) | 16.7 (-30%) |
| 256 | 64 | leader only | 2385 (+30%) | 31.5 (-19%) |
| 256 | 256 | leader only | 4353 (-3%) | 74.0 (+9%) |
| 64 | 64 | all servers | 2005 (+81%) | 49.8 (-55%) |
| 64 | 256 | all servers | 4868 (+35%) | 81.5 (-40%) |
| 256 | 64 | all servers | 1925 (+72%) | 47.7 (-59%) |
| 256 | 256 | all servers | 4975 (+36%) | 70.3 (-36%) |
### performance changes explanation
- read QPS in most scenarios is decreased by 5~8%. The reason is that etcd records store metrics for each store operation. The metrics is important for monitoring and debugging, so this is acceptable.
- write QPS to leader is increased by 20~30%. This is because we decouple raft main loop and entry apply loop, which avoids them blocking each other.
- write QPS to all servers is increased by 30~80% because follower could receive latest commit index earlier and commit proposals faster.
[hey]: https://github.com/rakyll/hey
[c7146bd5]: https://github.com/coreos/etcd/commits/c7146bd5f2c73716091262edc638401bb8229144
[etcd-2.1-benchmark]: etcd-2-1-0-alpha-benchmarks.md
[hack-benchmark]: ../../hack/benchmark/

View File

@ -1,51 +0,0 @@
---
title: Benchmarking etcd v2.2.0-rc-memory
---
## Physical machine
GCE n1-standard-2 machine type
- 1x dedicated local SSD mounted under /var/lib/etcd
- 1x dedicated slow disk for the OS
- 7.5 GB memory
- 2x CPUs
## etcd
```
etcd Version: 2.2.0-rc.0+git
Git SHA: 103cb5c
Go Version: go1.5
Go OS/Arch: linux/amd64
```
## Testing
Start 3-member etcd cluster, each of which uses 2 cores.
The length of key name is always 64 bytes, which is a reasonable length of average key bytes.
## Memory Maximal Usage
- etcd may use maximal memory if one follower is dead and the leader keeps sending snapshots.
- `max RSS` is the maximal memory usage recorded in 3 runs.
| value bytes | key number | data size(MB) | max RSS(MB) | max RSS/data rate on leader |
|-------------|-------------|---------------|-------------|-----------------------------|
| 128 | 50000 | 6 | 433 | 72x |
| 128 | 100000 | 12 | 659 | 54x |
| 128 | 200000 | 24 | 1466 | 61x |
| 1024 | 50000 | 48 | 1253 | 26x |
| 1024 | 100000 | 96 | 2344 | 24x |
| 1024 | 200000 | 192 | 4361 | 22x |
## Data Size Threshold
- When etcd reaches data size threshold, it may trigger leader election easily and drop part of proposals.
- For most cases, the etcd cluster should work smoothly if it doesn't hit the threshold. If it doesn't work well due to insufficient resources, decrease its data size.
| value bytes | key number limitation | suggested data size threshold(MB) | consumed RSS(MB) |
|-------------|-----------------------|-----------------------------------|------------------|
| 128 | 400K | 48 | 2400 |
| 1024 | 300K | 292 | 6500 |

View File

@ -1,46 +0,0 @@
---
title: Benchmarking etcd v3
---
## Physical machines
GCE n1-highcpu-2 machine type
- 1x dedicated local SSD mounted under /var/lib/etcd
- 1x dedicated slow disk for the OS
- 1.8 GB memory
- 2x CPUs
- etcd version 2.2.0
## etcd Cluster
1 etcd member running in v3 demo mode
## Testing
Use [etcd v3 benchmark tool][etcd-v3-benchmark].
## Performance
### reading one single key
| key size in bytes | number of clients | read QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|----------|---------------|
| 256 | 1 | 2716 | 0.4 |
| 256 | 64 | 16623 | 6.1 |
| 256 | 256 | 16622 | 21.7 |
The performance is nearly the same as the one with empty server handler.
### reading one single key after putting
| key size in bytes | number of clients | read QPS | 90th Percentile Latency (ms) |
|-------------------|-------------------|----------|---------------|
| 256 | 1 | 2269 | 0.5 |
| 256 | 64 | 13582 | 8.6 |
| 256 | 256 | 13262 | 47.5 |
The performance with empty server handler is not affected by one put. So the
performance downgrade should be caused by storage package.
[etcd-v3-benchmark]: ../../tools/benchmark/

View File

@ -1,79 +0,0 @@
---
title: Watch Memory Usage Benchmark
---
*NOTE*: The watch features are under active development, and their memory usage may change as that development progresses. We do not expect it to significantly increase beyond the figures stated below.
A primary goal of etcd is supporting a very large number of watchers doing a massively large amount of watching. etcd aims to support O(10k) clients, O(100K) watch streams (O(10) streams per client) and O(10M) total watchings (O(100) watching per stream). The memory consumed by each individual watching accounts for the largest portion of etcd's overall usage, and is therefore the focus of current and future optimizations.
Three related components of etcd watch consume physical memory: each `grpc.Conn`, each watch stream, and each instance of the watching activity. `grpc.Conn` maintains the actual TCP connection and other gRPC connection state. Each `grpc.Conn` consumes O(10kb) of memory, and might have multiple watch streams attached.
Each watch stream is an independent HTTP2 connection which consumes another O(10kb) of memory.
Multiple watchings might share one watch stream.
Watching is the actual struct that tracks the changes on the key-value store. Each watching should only consume < O(1kb).
```
+-------+
| watch |
+---------> | foo |
| +-------+
+------+-----+
| stream |
+--------------> | |
| +------+-----+ +-------+
| | | watch |
| +---------> | bar |
+-----+------+ +-------+
| | +------------+
| conn +-------> | stream |
| | | |
+-----+------+ +------------+
|
|
|
| +------------+
+--------------> | stream |
| |
+------------+
```
The theoretical memory consumption of watch can be approximated with the formula:
`memory = c1 * number_of_conn + c2 * avg_number_of_stream_per_conn + c3 * avg_number_of_watch_stream`
## Testing Environment
etcd version
- git head https://github.com/coreos/etcd/commit/185097ffaa627b909007e772c175e8fefac17af3
GCE n1-standard-2 machine type
- 7.5 GB memory
- 2x CPUs
## Overall memory usage
The overall memory usage captures how much [RSS][rss] etcd consumes with the client watchers. While the result may vary by as much as 10%, it is still meaningful, since the goal is to learn about the rough memory usage and the pattern of allocations.
With the benchmark result, we can calculate roughly that `c1 = 17kb`, `c2 = 18kb` and `c3 = 350bytes`. So each additional client connection consumes 17kb of memory and each additional stream consumes 18kb of memory, and each additional watching only cause 350bytes. A single etcd server can maintain millions of watchings with a few GB of memory in normal case.
| clients | streams per client | watchings per stream | total watching | memory usage |
|---------|---------|-----------|----------------|--------------|
| 1k | 1 | 1 | 1k | 50MB |
| 2k | 1 | 1 | 2k | 90MB |
| 5k | 1 | 1 | 5k | 200MB |
| 1k | 10 | 1 | 10k | 217MB |
| 2k | 10 | 1 | 20k | 417MB |
| 5k | 10 | 1 | 50k | 980MB |
| 1k | 50 | 1 | 50k | 1001MB |
| 2k | 50 | 1 | 100k | 1960MB |
| 5k | 50 | 1 | 250k | 4700MB |
| 1k | 50 | 10 | 500k | 1171MB |
| 2k | 50 | 10 | 1M | 2371MB |
| 5k | 50 | 10 | 2.5M | 5710MB |
| 1k | 50 | 100 | 5M | 2380MB |
| 2k | 50 | 100 | 10M | 4672MB |
| 5k | 50 | 100 | 25M | *OOM* |
[rss]: https://en.wikipedia.org/wiki/Resident_set_size

View File

@ -1,100 +0,0 @@
---
title: Storage Memory Usage Benchmark
---
<!---todo: link storage to storage design doc-->
Two components of etcd storage consume physical memory. The etcd process allocates an *in-memory index* to speed key lookup. The process's *page cache*, managed by the operating system, stores recently-accessed data from disk for quick re-use.
The in-memory index holds all the keys in a [B-tree][btree] data structure, along with pointers to the on-disk data (the values). Each key in the B-tree may contain multiple pointers, pointing to different versions of its values. The theoretical memory consumption of the in-memory index can hence be approximated with the formula:
`N * (c1 + avg_key_size) + N * (avg_versions_of_key) * (c2 + size_of_pointer)`
where `c1` is the key metadata overhead and `c2` is the version metadata overhead.
The graph shows the detailed structure of the in-memory index B-tree.
```
In mem index
+------------+
| key || ... |
+--------------+ | || |
| | +------------+
| | | v1 || ... |
| disk <----------------| || | Tree Node
| | +------------+
| | | v2 || ... |
| <----------------+ || |
| | +------------+
+--------------+ +-----+ | | |
| | | | |
| +------------+
|
|
^
------+
| ... |
| |
+-----+
| ... | Tree Node
| |
+-----+
| ... |
| |
------+
```
[Page cache memory][pagecache] is managed by the operating system and is not covered in detail in this document.
## Testing Environment
etcd version
- git head https://github.com/coreos/etcd/commit/776e9fb7be7eee5e6b58ab977c8887b4fe4d48db
GCE n1-standard-2 machine type
- 7.5 GB memory
- 2x CPUs
## In-memory index memory usage
In this test, we only benchmark the memory usage of the in-memory index. The goal is to find `c1` and `c2` mentioned above and to understand the hard limit of memory consumption of the storage.
We calculate the memory usage consumption via the Go runtime.ReadMemStats. We calculate the total allocated bytes difference before creating the index and after creating the index. It cannot perfectly reflect the memory usage of the in-memory index itself but can show the rough consumption pattern.
| N | versions | key size | memory usage |
|------|----------|----------|--------------|
| 100K | 1 | 64bytes | 22MB |
| 100K | 5 | 64bytes | 39MB |
| 1M | 1 | 64bytes | 218MB |
| 1M | 5 | 64bytes | 432MB |
| 100K | 1 | 256bytes | 41MB |
| 100K | 5 | 256bytes | 65MB |
| 1M | 1 | 256bytes | 409MB |
| 1M | 5 | 256bytes | 506MB |
Based on the result, we can calculate `c1=120bytes`, `c2=30bytes`. We only need two sets of data to calculate `c1` and `c2`, since they are the only unknown variable in the formula. The `c1=120bytes` and `c2=30bytes` are the average value of the 4 sets of `c1` and `c2` we calculated. The key metadata overhead is still relatively nontrivial (50%) for small key-value pairs. However, this is a significant improvement over the old store, which had at least 1000% overhead.
## Overall memory usage
The overall memory usage captures how much RSS etcd consumes with the storage. The value size should have very little impact on the overall memory usage of etcd, since we keep values on disk and only retain hot values in memory, managed by the OS page cache.
| N | versions | key size | value size | memory usage |
|------|----------|----------|------------|--------------|
| 100K | 1 | 64bytes | 256bytes | 40MB |
| 100K | 5 | 64bytes | 256bytes | 89MB |
| 1M | 1 | 64bytes | 256bytes | 470MB |
| 1M | 5 | 64bytes | 256bytes | 880MB |
| 100K | 1 | 64bytes | 1KB | 102MB |
| 100K | 5 | 64bytes | 1KB | 164MB |
| 1M | 1 | 64bytes | 1KB | 587MB |
| 1M | 5 | 64bytes | 1KB | 836MB |
Based on the result, we know the value size does not significantly impact the memory consumption. There is some minor increase due to more data held in the OS page cache.
[btree]: https://en.wikipedia.org/wiki/B-tree
[pagecache]: https://en.wikipedia.org/wiki/Page_cache

View File

@ -1,28 +0,0 @@
---
title: Branch management
---
## Guide
* New development occurs on the [master branch][master].
* Master branch should always have a green build!
* Backwards-compatible bug fixes should target the master branch and subsequently be ported to stable branches.
* Once the master branch is ready for release, it will be tagged and become the new stable branch.
The etcd team has adopted a *rolling release model* and supports two stable versions of etcd.
### Master branch
The `master` branch is our development branch. All new features land here first.
To try new and experimental features, pull `master` and play with it. Note that `master` may not be stable because new features may introduce bugs.
Before the release of the next stable version, feature PRs will be frozen. A [release manager](./dev-internal/release.md#release-management) will be assigned to major/minor version and will lead the etcd community in test, bug-fix and documentation of the release for one to two weeks.
### Stable branches
All branches with prefix `release-` are considered _stable_ branches.
After every minor release (http://semver.org/), we will have a new stable branch for that release, managed by a [patch release manager](./dev-internal/release.md#release-management). We will keep fixing the backwards-compatible bugs for the latest two stable releases. A _patch_ release to each supported release branch, incorporating any bug fixes, will be once every two weeks, given any patches.
[master]: https://github.com/coreos/etcd/tree/master

View File

@ -1,456 +0,0 @@
---
title: Demo
---
This series of examples shows the basic procedures for working with an etcd cluster.
## Set up a cluster
<img src="https://storage.googleapis.com/etcd/demo/01_etcd_clustering_2016051001.gif" alt="01_etcd_clustering_2016050601"/>
On each etcd node, specify the cluster members:
```
TOKEN=token-01
CLUSTER_STATE=new
NAME_1=machine-1
NAME_2=machine-2
NAME_3=machine-3
HOST_1=10.240.0.17
HOST_2=10.240.0.18
HOST_3=10.240.0.19
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
```
Run this on each machine:
```
# For machine 1
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
# For machine 2
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
# For machine 3
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
```
Or use our public discovery service:
```
curl https://discovery.etcd.io/new?size=3
https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92
# grab this token
TOKEN=token-01
CLUSTER_STATE=new
NAME_1=machine-1
NAME_2=machine-2
NAME_3=machine-3
HOST_1=10.240.0.17
HOST_2=10.240.0.18
HOST_3=10.240.0.19
DISCOVERY=https://discovery.etcd.io/a81b5818e67a6ea83e9d4daea5ecbc92
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--discovery ${DISCOVERY} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--discovery ${DISCOVERY} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://${THIS_IP}:2379 \
--discovery ${DISCOVERY} \
--initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}
```
Now etcd is ready! To connect to etcd with etcdctl:
```
export ETCDCTL_API=3
HOST_1=10.240.0.17
HOST_2=10.240.0.18
HOST_3=10.240.0.19
ENDPOINTS=$HOST_1:2379,$HOST_2:2379,$HOST_3:2379
etcdctl --endpoints=$ENDPOINTS member list
```
## Access etcd
<img src="https://storage.googleapis.com/etcd/demo/02_etcdctl_access_etcd_2016051001.gif" alt="02_etcdctl_access_etcd_2016051001"/>
`put` command to write:
```
etcdctl --endpoints=$ENDPOINTS put foo "Hello World!"
```
`get` to read from etcd:
```
etcdctl --endpoints=$ENDPOINTS get foo
etcdctl --endpoints=$ENDPOINTS --write-out="json" get foo
```
## Get by prefix
<img src="https://storage.googleapis.com/etcd/demo/03_etcdctl_get_by_prefix_2016050501.gif" alt="03_etcdctl_get_by_prefix_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS put web1 value1
etcdctl --endpoints=$ENDPOINTS put web2 value2
etcdctl --endpoints=$ENDPOINTS put web3 value3
etcdctl --endpoints=$ENDPOINTS get web --prefix
```
## Delete
<img src="https://storage.googleapis.com/etcd/demo/04_etcdctl_delete_2016050601.gif" alt="04_etcdctl_delete_2016050601"/>
```
etcdctl --endpoints=$ENDPOINTS put key myvalue
etcdctl --endpoints=$ENDPOINTS del key
etcdctl --endpoints=$ENDPOINTS put k1 value1
etcdctl --endpoints=$ENDPOINTS put k2 value2
etcdctl --endpoints=$ENDPOINTS del k --prefix
```
## Transactional write
`txn` to wrap multiple requests into one transaction:
<img src="https://storage.googleapis.com/etcd/demo/05_etcdctl_transaction_2016050501.gif" alt="05_etcdctl_transaction_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS put user1 bad
etcdctl --endpoints=$ENDPOINTS txn --interactive
compares:
value("user1") = "bad"
success requests (get, put, delete):
del user1
failure requests (get, put, delete):
put user1 good
```
## Watch
`watch` to get notified of future changes:
<img src="https://storage.googleapis.com/etcd/demo/06_etcdctl_watch_2016050501.gif" alt="06_etcdctl_watch_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS watch stock1
etcdctl --endpoints=$ENDPOINTS put stock1 1000
etcdctl --endpoints=$ENDPOINTS watch stock --prefix
etcdctl --endpoints=$ENDPOINTS put stock1 10
etcdctl --endpoints=$ENDPOINTS put stock2 20
```
## Lease
`lease` to write with TTL:
<img src="https://storage.googleapis.com/etcd/demo/07_etcdctl_lease_2016050501.gif" alt="07_etcdctl_lease_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS lease grant 300
# lease 2be7547fbc6a5afa granted with TTL(300s)
etcdctl --endpoints=$ENDPOINTS put sample value --lease=2be7547fbc6a5afa
etcdctl --endpoints=$ENDPOINTS get sample
etcdctl --endpoints=$ENDPOINTS lease keep-alive 2be7547fbc6a5afa
etcdctl --endpoints=$ENDPOINTS lease revoke 2be7547fbc6a5afa
# or after 300 seconds
etcdctl --endpoints=$ENDPOINTS get sample
```
## Distributed locks
`lock` for distributed lock:
<img src="https://storage.googleapis.com/etcd/demo/08_etcdctl_lock_2016050501.gif" alt="08_etcdctl_lock_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS lock mutex1
# another client with the same name blocks
etcdctl --endpoints=$ENDPOINTS lock mutex1
```
## Elections
`elect` for leader election:
<img src="https://storage.googleapis.com/etcd/demo/09_etcdctl_elect_2016050501.gif" alt="09_etcdctl_elect_2016050501"/>
```
etcdctl --endpoints=$ENDPOINTS elect one p1
# another client with the same name blocks
etcdctl --endpoints=$ENDPOINTS elect one p2
```
## Cluster status
Specify the initial cluster configuration for each machine:
<img src="https://storage.googleapis.com/etcd/demo/10_etcdctl_endpoint_2016050501.gif" alt="10_etcdctl_endpoint_2016050501"/>
```
etcdctl --write-out=table --endpoints=$ENDPOINTS endpoint status
+------------------+------------------+---------+---------+-----------+-----------+------------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
+------------------+------------------+---------+---------+-----------+-----------+------------+
| 10.240.0.17:2379 | 4917a7ab173fabe7 | 3.0.0 | 45 kB | true | 4 | 16726 |
| 10.240.0.18:2379 | 59796ba9cd1bcd72 | 3.0.0 | 45 kB | false | 4 | 16726 |
| 10.240.0.19:2379 | 94df724b66343e6c | 3.0.0 | 45 kB | false | 4 | 16726 |
+------------------+------------------+---------+---------+-----------+-----------+------------+
```
```
etcdctl --endpoints=$ENDPOINTS endpoint health
10.240.0.17:2379 is healthy: successfully committed proposal: took = 3.345431ms
10.240.0.19:2379 is healthy: successfully committed proposal: took = 3.767967ms
10.240.0.18:2379 is healthy: successfully committed proposal: took = 4.025451ms
```
## Snapshot
`snapshot` to save point-in-time snapshot of etcd database:
<img src="https://storage.googleapis.com/etcd/demo/11_etcdctl_snapshot_2016051001.gif" alt="11_etcdctl_snapshot_2016051001"/>
```
etcdctl --endpoints=$ENDPOINTS snapshot save my.db
Snapshot saved at my.db
```
```
etcdctl --write-out=table --endpoints=$ENDPOINTS snapshot status my.db
+---------+----------+------------+------------+
| HASH | REVISION | TOTAL KEYS | TOTAL SIZE |
+---------+----------+------------+------------+
| c55e8b8 | 9 | 13 | 25 kB |
+---------+----------+------------+------------+
```
## Migrate
`migrate` to transform etcd v2 to v3 data:
<img src="https://storage.googleapis.com/etcd/demo/12_etcdctl_migrate_2016061602.gif" alt="12_etcdctl_migrate_2016061602"/>
```
# write key in etcd version 2 store
export ETCDCTL_API=2
etcdctl --endpoints=http://$ENDPOINT set foo bar
# read key in etcd v2
etcdctl --endpoints=$ENDPOINTS --output="json" get foo
# stop etcd node to migrate, one by one
# migrate v2 data
export ETCDCTL_API=3
etcdctl --endpoints=$ENDPOINT migrate --data-dir="default.etcd" --wal-dir="default.etcd/member/wal"
# restart etcd node after migrate, one by one
# confirm that the key got migrated
etcdctl --endpoints=$ENDPOINTS get /foo
```
## Member
`member` to add,remove,update membership:
<img src="https://storage.googleapis.com/etcd/demo/13_etcdctl_member_2016062301.gif" alt="13_etcdctl_member_2016062301"/>
```
# For each machine
TOKEN=my-etcd-token-1
CLUSTER_STATE=new
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_3=etcd-node-3
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_3=10.240.0.15
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
# For node 1
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 \
--listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 \
--listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} \
--initial-cluster-token ${TOKEN}
# For node 2
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 \
--listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 \
--listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} \
--initial-cluster-token ${TOKEN}
# For node 3
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 \
--listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 \
--listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} \
--initial-cluster-token ${TOKEN}
```
Then replace a member with `member remove` and `member add` commands:
```
# get member ID
export ETCDCTL_API=3
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_3=10.240.0.15
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379,${HOST_3}:2379 member list
# remove the member
MEMBER_ID=278c654c9a6dfd3b
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379,${HOST_3}:2379 \
member remove ${MEMBER_ID}
# add a new member (node 4)
export ETCDCTL_API=3
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_4=etcd-node-4
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_4=10.240.0.16 # new member
etcdctl --endpoints=${HOST_1}:2379,${HOST_2}:2379 \
member add ${NAME_4} \
--peer-urls=http://${HOST_4}:2380
```
Next, start the new member with `--initial-cluster-state existing` flag:
```
# [WARNING] If the new member starts from the same disk space,
# make sure to remove the data directory of the old member
#
# restart with 'existing' flag
TOKEN=my-etcd-token-1
CLUSTER_STATE=existing
NAME_1=etcd-node-1
NAME_2=etcd-node-2
NAME_4=etcd-node-4
HOST_1=10.240.0.13
HOST_2=10.240.0.14
HOST_4=10.240.0.16 # new member
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_4}=http://${HOST_4}:2380
THIS_NAME=${NAME_4}
THIS_IP=${HOST_4}
etcd --data-dir=data.etcd --name ${THIS_NAME} \
--initial-advertise-peer-urls http://${THIS_IP}:2380 \
--listen-peer-urls http://${THIS_IP}:2380 \
--advertise-client-urls http://${THIS_IP}:2379 \
--listen-client-urls http://${THIS_IP}:2379 \
--initial-cluster ${CLUSTER} \
--initial-cluster-state ${CLUSTER_STATE} \
--initial-cluster-token ${TOKEN}
```
## Auth
`auth`,`user`,`role` for authentication:
<img src="https://storage.googleapis.com/etcd/demo/14_etcdctl_auth_2016062301.gif" alt="14_etcdctl_auth_2016062301"/>
```
export ETCDCTL_API=3
ENDPOINTS=localhost:2379
etcdctl --endpoints=${ENDPOINTS} role add root
etcdctl --endpoints=${ENDPOINTS} role grant-permission root readwrite foo
etcdctl --endpoints=${ENDPOINTS} role get root
etcdctl --endpoints=${ENDPOINTS} user add root
etcdctl --endpoints=${ENDPOINTS} user grant-role root root
etcdctl --endpoints=${ENDPOINTS} user get root
etcdctl --endpoints=${ENDPOINTS} auth enable
# now all client requests go through auth
etcdctl --endpoints=${ENDPOINTS} --user=root:123 put foo bar
etcdctl --endpoints=${ENDPOINTS} get foo
etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo
etcdctl --endpoints=${ENDPOINTS} --user=root:123 get foo1
```

View File

@ -1,168 +0,0 @@
### etcd concurrency API Reference
This is a generated documentation. Please read the proto files for more.
##### service `Lock` (etcdserver/api/v3lock/v3lockpb/v3lock.proto)
The lock service exposes client-side locking facilities as a gRPC interface.
| Method | Request Type | Response Type | Description |
| ------ | ------------ | ------------- | ----------- |
| Lock | LockRequest | LockResponse | Lock acquires a distributed shared lock on a given named lock. On success, it will return a unique key that exists so long as the lock is held by the caller. This key can be used in conjunction with transactions to safely ensure updates to etcd only occur while holding lock ownership. The lock is held until Unlock is called on the key or the lease associate with the owner expires. |
| Unlock | UnlockRequest | UnlockResponse | Unlock takes a key returned by Lock and releases the hold on lock. The next Lock caller waiting for the lock will then be woken up and given ownership of the lock. |
##### message `LockRequest` (etcdserver/api/v3lock/v3lockpb/v3lock.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| name | name is the identifier for the distributed shared lock to be acquired. | bytes |
| lease | lease is the ID of the lease that will be attached to ownership of the lock. If the lease expires or is revoked and currently holds the lock, the lock is automatically released. Calls to Lock with the same lease will be treated as a single acquisition; locking twice with the same lease is a no-op. | int64 |
##### message `LockResponse` (etcdserver/api/v3lock/v3lockpb/v3lock.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
| key | key is a key that will exist on etcd for the duration that the Lock caller owns the lock. Users should not modify this key or the lock may exhibit undefined behavior. | bytes |
##### message `UnlockRequest` (etcdserver/api/v3lock/v3lockpb/v3lock.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| key | key is the lock ownership key granted by Lock. | bytes |
##### message `UnlockResponse` (etcdserver/api/v3lock/v3lockpb/v3lock.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
##### service `Election` (etcdserver/api/v3election/v3electionpb/v3election.proto)
The election service exposes client-side election facilities as a gRPC interface.
| Method | Request Type | Response Type | Description |
| ------ | ------------ | ------------- | ----------- |
| Campaign | CampaignRequest | CampaignResponse | Campaign waits to acquire leadership in an election, returning a LeaderKey representing the leadership if successful. The LeaderKey can then be used to issue new values on the election, transactionally guard API requests on leadership still being held, and resign from the election. |
| Proclaim | ProclaimRequest | ProclaimResponse | Proclaim updates the leader's posted value with a new value. |
| Leader | LeaderRequest | LeaderResponse | Leader returns the current election proclamation, if any. |
| Observe | LeaderRequest | LeaderResponse | Observe streams election proclamations in-order as made by the election's elected leaders. |
| Resign | ResignRequest | ResignResponse | Resign releases election leadership so other campaigners may acquire leadership on the election. |
##### message `CampaignRequest` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| name | name is the election's identifier for the campaign. | bytes |
| lease | lease is the ID of the lease attached to leadership of the election. If the lease expires or is revoked before resigning leadership, then the leadership is transferred to the next campaigner, if any. | int64 |
| value | value is the initial proclaimed value set when the campaigner wins the election. | bytes |
##### message `CampaignResponse` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
| leader | leader describes the resources used for holding leadereship of the election. | LeaderKey |
##### message `LeaderKey` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| name | name is the election identifier that correponds to the leadership key. | bytes |
| key | key is an opaque key representing the ownership of the election. If the key is deleted, then leadership is lost. | bytes |
| rev | rev is the creation revision of the key. It can be used to test for ownership of an election during transactions by testing the key's creation revision matches rev. | int64 |
| lease | lease is the lease ID of the election leader. | int64 |
##### message `LeaderRequest` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| name | name is the election identifier for the leadership information. | bytes |
##### message `LeaderResponse` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
| kv | kv is the key-value pair representing the latest leader update. | mvccpb.KeyValue |
##### message `ProclaimRequest` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| leader | leader is the leadership hold on the election. | LeaderKey |
| value | value is an update meant to overwrite the leader's current value. | bytes |
##### message `ProclaimResponse` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
##### message `ResignRequest` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| leader | leader is the leadership to relinquish by resignation. | LeaderKey |
##### message `ResignResponse` (etcdserver/api/v3election/v3electionpb/v3election.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| header | | etcdserverpb.ResponseHeader |
##### message `Event` (mvcc/mvccpb/kv.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| type | type is the kind of event. If type is a PUT, it indicates new data has been stored to the key. If type is a DELETE, it indicates the key was deleted. | EventType |
| kv | kv holds the KeyValue for the event. A PUT event contains current kv pair. A PUT event with kv.Version=1 indicates the creation of a key. A DELETE/EXPIRE event contains the deleted key with its modification revision set to the revision of deletion. | KeyValue |
| prev_kv | prev_kv holds the key-value pair before the event happens. | KeyValue |
##### message `KeyValue` (mvcc/mvccpb/kv.proto)
| Field | Description | Type |
| ----- | ----------- | ---- |
| key | key is the key in bytes. An empty key is not allowed. | bytes |
| create_revision | create_revision is the revision of last creation on this key. | int64 |
| mod_revision | mod_revision is the revision of last modification on this key. | int64 |
| version | version is the version of the key. A deletion resets the version to zero and any modification of the key increases its version. | int64 |
| value | value is the value held by the key, in bytes. | bytes |
| lease | lease is the ID of the lease that attached to key. When the attached lease expires, the key will be deleted. If lease is 0, then no lease is attached to the key. | int64 |

View File

@ -1,137 +0,0 @@
---
title: Why gRPC gateway
---
etcd v3 uses [gRPC][grpc] for its messaging protocol. The etcd project includes a gRPC-based [Go client][go-client] and a command line utility, [etcdctl][etcdctl], for communicating with an etcd cluster through gRPC. For languages with no gRPC support, etcd provides a JSON [gRPC gateway][grpc-gateway]. This gateway serves a RESTful proxy that translates HTTP/JSON requests into gRPC messages.
## Using gRPC gateway
The gateway accepts a [JSON mapping][json-mapping] for etcd's [protocol buffer][api-ref] message definitions. Note that `key` and `value` fields are defined as byte arrays and therefore must be base64 encoded in JSON. The following examples use `curl`, but any HTTP/JSON client should work all the same.
### Notes
gRPC gateway endpoint has changed since etcd v3.3:
- etcd v3.2 or before uses only `[CLIENT-URL]/v3alpha/*`.
- etcd v3.3 uses `[CLIENT-URL]/v3beta/*` while keeping `[CLIENT-URL]/v3alpha/*`.
- etcd v3.4 uses `[CLIENT-URL]/v3/*` while keeping `[CLIENT-URL]/v3beta/*`.
- **`[CLIENT-URL]/v3alpha/*` is deprecated**.
- etcd v3.5 or later uses only `[CLIENT-URL]/v3/*`.
- **`[CLIENT-URL]/v3beta/*` is deprecated**.
gRPC-gateway does not support authentication using TLS Common Name.
### Put and get keys
Use the `/v3/kv/range` and `/v3/kv/put` services to read and write keys:
```bash
<<COMMENT
https://www.base64encode.org/
foo is 'Zm9v' in Base64
bar is 'YmFy'
COMMENT
curl -L http://localhost:2379/v3/kv/put \
-X POST -d '{"key": "Zm9v", "value": "YmFy"}'
# {"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"2","raft_term":"3"}}
curl -L http://localhost:2379/v3/kv/range \
-X POST -d '{"key": "Zm9v"}'
# {"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"2","raft_term":"3"},"kvs":[{"key":"Zm9v","create_revision":"2","mod_revision":"2","version":"1","value":"YmFy"}],"count":"1"}
# get all keys prefixed with "foo"
curl -L http://localhost:2379/v3/kv/range \
-X POST -d '{"key": "Zm9v", "range_end": "Zm9w"}'
# {"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"2","raft_term":"3"},"kvs":[{"key":"Zm9v","create_revision":"2","mod_revision":"2","version":"1","value":"YmFy"}],"count":"1"}
```
### Watch keys
Use the `/v3/watch` service to watch keys:
```bash
curl -N http://localhost:2379/v3/watch \
-X POST -d '{"create_request": {"key":"Zm9v"} }' &
# {"result":{"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"1","raft_term":"2"},"created":true}}
curl -L http://localhost:2379/v3/kv/put \
-X POST -d '{"key": "Zm9v", "value": "YmFy"}' >/dev/null 2>&1
# {"result":{"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"2","raft_term":"2"},"events":[{"kv":{"key":"Zm9v","create_revision":"2","mod_revision":"2","version":"1","value":"YmFy"}}]}}
```
### Transactions
Issue a transaction with `/v3/kv/txn`:
```bash
# target CREATE
curl -L http://localhost:2379/v3/kv/txn \
-X POST \
-d '{"compare":[{"target":"CREATE","key":"Zm9v","createRevision":"2"}],"success":[{"requestPut":{"key":"Zm9v","value":"YmFy"}}]}'
# {"header":{"cluster_id":"12585971608760269493","member_id":"13847567121247652255","revision":"3","raft_term":"2"},"succeeded":true,"responses":[{"response_put":{"header":{"revision":"3"}}}]}
```
```bash
# target VERSION
curl -L http://localhost:2379/v3/kv/txn \
-X POST \
-d '{"compare":[{"version":"4","result":"EQUAL","target":"VERSION","key":"Zm9v"}],"success":[{"requestRange":{"key":"Zm9v"}}]}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"6","raft_term":"3"},"succeeded":true,"responses":[{"response_range":{"header":{"revision":"6"},"kvs":[{"key":"Zm9v","create_revision":"2","mod_revision":"6","version":"4","value":"YmF6"}],"count":"1"}}]}
```
### Authentication
Set up authentication with the `/v3/auth` service:
```bash
# create root user
curl -L http://localhost:2379/v3/auth/user/add \
-X POST -d '{"name": "root", "password": "pass"}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"1","raft_term":"2"}}
# create root role
curl -L http://localhost:2379/v3/auth/role/add \
-X POST -d '{"name": "root"}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"1","raft_term":"2"}}
# grant root role
curl -L http://localhost:2379/v3/auth/user/grant \
-X POST -d '{"user": "root", "role": "root"}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"1","raft_term":"2"}}
# enable auth
curl -L http://localhost:2379/v3/auth/enable -X POST -d '{}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"1","raft_term":"2"}}
```
Authenticate with etcd for an authentication token using `/v3/auth/authenticate`:
```bash
# get the auth token for the root user
curl -L http://localhost:2379/v3/auth/authenticate \
-X POST -d '{"name": "root", "password": "pass"}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"1","raft_term":"2"},"token":"sssvIpwfnLAcWAQH.9"}
```
Set the `Authorization` header to the authentication token to fetch a key using authentication credentials:
```bash
curl -L http://localhost:2379/v3/kv/put \
-H 'Authorization : sssvIpwfnLAcWAQH.9' \
-X POST -d '{"key": "Zm9v", "value": "YmFy"}'
# {"header":{"cluster_id":"14841639068965178418","member_id":"10276657743932975437","revision":"2","raft_term":"2"}}
```
## Swagger
Generated [Swagger][swagger] API definitions can be found at [rpc.swagger.json][swagger-doc].
[api-ref]: ./api_reference_v3.md
[go-client]: https://github.com/coreos/etcd/tree/master/clientv3
[etcdctl]: https://github.com/coreos/etcd/tree/master/etcdctl
[grpc]: https://www.grpc.io/
[grpc-gateway]: https://github.com/grpc-ecosystem/grpc-gateway
[json-mapping]: https://developers.google.com/protocol-buffers/docs/proto3#json
[swagger]: http://swagger.io/
[swagger-doc]: apispec/swagger/rpc.swagger.json

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -1,13 +1,9 @@
{
"swagger": "2.0",
"info": {
"title": "etcdserver/api/v3election/v3electionpb/v3election.proto",
"title": "server/etcdserver/api/v3election/v3electionpb/v3election.proto",
"version": "version not set"
},
"schemes": [
"http",
"https"
],
"consumes": [
"application/json"
],
@ -18,13 +14,19 @@
"/v3/election/campaign": {
"post": {
"summary": "Campaign waits to acquire leadership in an election, returning a LeaderKey\nrepresenting the leadership if successful. The LeaderKey can then be used\nto issue new values on the election, transactionally guard API requests on\nleadership still being held, and resign from the election.",
"operationId": "Campaign",
"operationId": "Election_Campaign",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3electionpbCampaignResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -45,13 +47,19 @@
"/v3/election/leader": {
"post": {
"summary": "Leader returns the current election proclamation, if any.",
"operationId": "Leader",
"operationId": "Election_Leader",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3electionpbLeaderResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -72,12 +80,27 @@
"/v3/election/observe": {
"post": {
"summary": "Observe streams election proclamations in-order as made by the election's\nelected leaders.",
"operationId": "Observe",
"operationId": "Election_Observe",
"responses": {
"200": {
"description": "A successful response.(streaming responses)",
"schema": {
"$ref": "#/x-stream-definitions/v3electionpbLeaderResponse"
"type": "object",
"properties": {
"result": {
"$ref": "#/definitions/v3electionpbLeaderResponse"
},
"error": {
"$ref": "#/definitions/runtimeStreamError"
}
},
"title": "Stream result of v3electionpbLeaderResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
@ -99,13 +122,19 @@
"/v3/election/proclaim": {
"post": {
"summary": "Proclaim updates the leader's posted value with a new value.",
"operationId": "Proclaim",
"operationId": "Election_Proclaim",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3electionpbProclaimResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -126,13 +155,19 @@
"/v3/election/resign": {
"post": {
"summary": "Resign releases election leadership so other campaigners may acquire\nleadership on the election.",
"operationId": "Resign",
"operationId": "Election_Resign",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3electionpbResignResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -224,6 +259,27 @@
}
}
},
"runtimeError": {
"type": "object",
"properties": {
"error": {
"type": "string"
},
"code": {
"type": "integer",
"format": "int32"
},
"message": {
"type": "string"
},
"details": {
"type": "array",
"items": {
"$ref": "#/definitions/protobufAny"
}
}
}
},
"runtimeStreamError": {
"type": "object",
"properties": {
@ -367,19 +423,5 @@
}
}
}
},
"x-stream-definitions": {
"v3electionpbLeaderResponse": {
"type": "object",
"properties": {
"result": {
"$ref": "#/definitions/v3electionpbLeaderResponse"
},
"error": {
"$ref": "#/definitions/runtimeStreamError"
}
},
"title": "Stream result of v3electionpbLeaderResponse"
}
}
}

View File

@ -1,13 +1,9 @@
{
"swagger": "2.0",
"info": {
"title": "etcdserver/api/v3lock/v3lockpb/v3lock.proto",
"title": "server/etcdserver/api/v3lock/v3lockpb/v3lock.proto",
"version": "version not set"
},
"schemes": [
"http",
"https"
],
"consumes": [
"application/json"
],
@ -18,13 +14,19 @@
"/v3/lock/lock": {
"post": {
"summary": "Lock acquires a distributed shared lock on a given named lock.\nOn success, it will return a unique key that exists so long as the\nlock is held by the caller. This key can be used in conjunction with\ntransactions to safely ensure updates to etcd only occur while holding\nlock ownership. The lock is held until Unlock is called on the key or the\nlease associate with the owner expires.",
"operationId": "Lock",
"operationId": "Lock_Lock",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3lockpbLockResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -45,13 +47,19 @@
"/v3/lock/unlock": {
"post": {
"summary": "Unlock takes a key returned by Lock and releases the hold on lock. The\nnext Lock caller waiting for the lock will then be woken up and given\nownership of the lock.",
"operationId": "Unlock",
"operationId": "Lock_Unlock",
"responses": {
"200": {
"description": "A successful response.",
"schema": {
"$ref": "#/definitions/v3lockpbUnlockResponse"
}
},
"default": {
"description": "An unexpected error response",
"schema": {
"$ref": "#/definitions/runtimeError"
}
}
},
"parameters": [
@ -96,6 +104,39 @@
}
}
},
"protobufAny": {
"type": "object",
"properties": {
"type_url": {
"type": "string"
},
"value": {
"type": "string",
"format": "byte"
}
}
},
"runtimeError": {
"type": "object",
"properties": {
"error": {
"type": "string"
},
"code": {
"type": "integer",
"format": "int32"
},
"message": {
"type": "string"
},
"details": {
"type": "array",
"items": {
"$ref": "#/definitions/protobufAny"
}
}
}
},
"v3lockpbLockRequest": {
"type": "object",
"properties": {

View File

@ -1,9 +0,0 @@
---
title: Experimental APIs and features
---
For the most part, the etcd project is stable, but we are still moving fast! We believe in the release fast philosophy. We want to get early feedback on features still in development and stabilizing. Thus, there are, and will be more, experimental features and APIs. We plan to improve these features based on the early feedback from the community, or abandon them if there is little interest, in the next few releases. Please do not rely on any experimental features or APIs in production environment.
## The current experimental API/features are:
- [KV ordering](https://godoc.org/github.com/etcd-io/etcd/clientv3/ordering) wrapper. When an etcd client switches endpoints, responses to serializable reads may go backward in time if the new endpoint is lagging behind the rest of the cluster. The ordering wrapper caches the current cluster revision from response headers. If a response revision is less than the cached revision, the client selects another endpoint and reissues the read. Enable in grpcproxy with `--experimental-serializable-ordering`.

View File

@ -1,67 +0,0 @@
---
title: gRPC naming and discovery
---
etcd provides a gRPC resolver to support an alternative name system that fetches endpoints from etcd for discovering gRPC services. The underlying mechanism is based on watching updates to keys prefixed with the service name.
## Using etcd discovery with go-grpc
The etcd client provides a gRPC resolver for resolving gRPC endpoints with an etcd backend. The resolver is initialized with an etcd client and given a target for resolution:
```go
import (
"go.etcd.io/etcd/clientv3"
etcdnaming "go.etcd.io/etcd/clientv3/naming"
"google.golang.org/grpc"
)
...
cli, cerr := clientv3.NewFromURL("http://localhost:2379")
r := &etcdnaming.GRPCResolver{Client: cli}
b := grpc.RoundRobin(r)
conn, gerr := grpc.Dial("my-service", grpc.WithBalancer(b), grpc.WithBlock(), ...)
```
## Managing service endpoints
The etcd resolver treats all keys under the prefix of the resolution target following a "/" (e.g., "my-service/") with JSON-encoded go-grpc `naming.Update` values as potential service endpoints. Endpoints are added to the service by creating new keys and removed from the service by deleting keys.
### Adding an endpoint
New endpoints can be added to the service through `etcdctl`:
```sh
ETCDCTL_API=3 etcdctl put my-service/1.2.3.4 '{"Addr":"1.2.3.4","Metadata":"..."}'
```
The etcd client's `GRPCResolver.Update` method can also register new endpoints with a key matching the `Addr`:
```go
r.Update(context.TODO(), "my-service", naming.Update{Op: naming.Add, Addr: "1.2.3.4", Metadata: "..."})
```
### Deleting an endpoint
Hosts can be deleted from the service through `etcdctl`:
```sh
ETCDCTL_API=3 etcdctl del my-service/1.2.3.4
```
The etcd client's `GRPCResolver.Update` method also supports deleting endpoints:
```go
r.Update(context.TODO(), "my-service", naming.Update{Op: naming.Delete, Addr: "1.2.3.4"})
```
### Registering an endpoint with a lease
Registering an endpoint with a lease ensures that if the host can't maintain a keepalive heartbeat (e.g., its machine fails), it will be removed from the service:
```sh
lease=`ETCDCTL_API=3 etcdctl lease grant 5 | cut -f2 -d' '`
ETCDCTL_API=3 etcdctl put --lease=$lease my-service/1.2.3.4 '{"Addr":"1.2.3.4","Metadata":"..."}'
ETCDCTL_API=3 etcdctl lease keep-alive $lease
```

View File

@ -1,499 +0,0 @@
---
title: Interacting with etcd
---
Users mostly interact with etcd by putting or getting the value of a key. This section describes how to do that by using etcdctl, a command line tool for interacting with etcd server. The concepts described here should apply to the gRPC APIs or client library APIs.
The API version used by etcdctl to speak to etcd may be set to version `2` or `3` via the `ETCDCTL_API` environment variable. By default, etcdctl on master (3.4) uses the v3 API and earlier versions (3.3 and earlier) default to the v2 API.
Note that any key that was created using the v2 API will not be able to be queried via the v3 API. A v3 API ```etcdctl get``` of a v2 key will exit with 0 and no key data, this is the expected behaviour.
```bash
export ETCDCTL_API=3
```
## Find versions
etcdctl version and Server API version can be useful in finding the appropriate commands to be used for performing various operations on etcd.
Here is the command to find the versions:
```bash
$ etcdctl version
etcdctl version: 3.1.0-alpha.0+git
API version: 3.1
```
## Write a key
Applications store keys into the etcd cluster by writing to keys. Every stored key is replicated to all etcd cluster members through the Raft protocol to achieve consistency and reliability.
Here is the command to set the value of key `foo` to `bar`:
```bash
$ etcdctl put foo bar
OK
```
Also a key can be set for a specified interval of time by attaching lease to it.
Here is the command to set the value of key `foo1` to `bar1` for 10s.
```bash
$ etcdctl put foo1 bar1 --lease=1234abcd
OK
```
Note: The lease id `1234abcd` in the above command refers to id returned on creating the lease of 10s. This id can then be attached to the key.
## Read keys
Applications can read values of keys from an etcd cluster. Queries may read a single key, or a range of keys.
Suppose the etcd cluster has stored the following keys:
```bash
foo = bar
foo1 = bar1
foo2 = bar2
foo3 = bar3
```
Here is the command to read the value of key `foo`:
```bash
$ etcdctl get foo
foo
bar
```
Here is the command to read the value of key `foo` in hex format:
```bash
$ etcdctl get foo --hex
\x66\x6f\x6f # Key
\x62\x61\x72 # Value
```
Here is the command to read only the value of key `foo`:
```bash
$ etcdctl get foo --print-value-only
bar
```
Here is the command to range over the keys from `foo` to `foo3`:
```bash
$ etcdctl get foo foo3
foo
bar
foo1
bar1
foo2
bar2
```
Note that `foo3` is excluded since the range is over the half-open interval `[foo, foo3)`, excluding `foo3`.
Here is the command to range over all keys prefixed with `foo`:
```bash
$ etcdctl get --prefix foo
foo
bar
foo1
bar1
foo2
bar2
foo3
bar3
```
Here is the command to range over all keys prefixed with `foo`, limiting the number of results to 2:
```bash
$ etcdctl get --prefix --limit=2 foo
foo
bar
foo1
bar1
```
## Read past version of keys
Applications may want to read superseded versions of a key. For example, an application may wish to roll back to an old configuration by accessing an earlier version of a key. Alternatively, an application may want a consistent view over multiple keys through multiple requests by accessing key history.
Since every modification to the etcd cluster key-value store increments the global revision of an etcd cluster, an application can read superseded keys by providing an older etcd revision.
Suppose an etcd cluster already has the following keys:
```bash
foo = bar # revision = 2
foo1 = bar1 # revision = 3
foo = bar_new # revision = 4
foo1 = bar1_new # revision = 5
```
Here are an example to access the past versions of keys:
```bash
$ etcdctl get --prefix foo # access the most recent versions of keys
foo
bar_new
foo1
bar1_new
$ etcdctl get --prefix --rev=4 foo # access the versions of keys at revision 4
foo
bar_new
foo1
bar1
$ etcdctl get --prefix --rev=3 foo # access the versions of keys at revision 3
foo
bar
foo1
bar1
$ etcdctl get --prefix --rev=2 foo # access the versions of keys at revision 2
foo
bar
$ etcdctl get --prefix --rev=1 foo # access the versions of keys at revision 1
```
## Read keys which are greater than or equal to the byte value of the specified key
Applications may want to read keys which are greater than or equal to the byte value of the specified key.
Suppose an etcd cluster already has the following keys:
```bash
a = 123
b = 456
z = 789
```
Here is the command to read keys which are greater than or equal to the byte value of key `b` :
```bash
$ etcdctl get --from-key b
b
456
z
789
```
## Delete keys
Applications can delete a key or a range of keys from an etcd cluster.
Suppose an etcd cluster already has the following keys:
```bash
foo = bar
foo1 = bar1
foo3 = bar3
zoo = val
zoo1 = val1
zoo2 = val2
a = 123
b = 456
z = 789
```
Here is the command to delete key `foo`:
```bash
$ etcdctl del foo
1 # one key is deleted
```
Here is the command to delete keys ranging from `foo` to `foo9`:
```bash
$ etcdctl del foo foo9
2 # two keys are deleted
```
Here is the command to delete key `zoo` with the deleted key value pair returned:
```bash
$ etcdctl del --prev-kv zoo
1 # one key is deleted
zoo # deleted key
val # the value of the deleted key
```
Here is the command to delete keys having prefix as `zoo`:
```bash
$ etcdctl del --prefix zoo
2 # two keys are deleted
```
Here is the command to delete keys which are greater than or equal to the byte value of key `b` :
```bash
$ etcdctl del --from-key b
2 # two keys are deleted
```
## Watch key changes
Applications can watch on a key or a range of keys to monitor for any updates.
Here is the command to watch on key `foo`:
```bash
$ etcdctl watch foo
# in another terminal: etcdctl put foo bar
PUT
foo
bar
```
Here is the command to watch on key `foo` in hex format:
```bash
$ etcdctl watch foo --hex
# in another terminal: etcdctl put foo bar
PUT
\x66\x6f\x6f # Key
\x62\x61\x72 # Value
```
Here is the command to watch on a range key from `foo` to `foo9`:
```bash
$ etcdctl watch foo foo9
# in another terminal: etcdctl put foo bar
PUT
foo
bar
# in another terminal: etcdctl put foo1 bar1
PUT
foo1
bar1
```
Here is the command to watch on keys having prefix `foo`:
```bash
$ etcdctl watch --prefix foo
# in another terminal: etcdctl put foo bar
PUT
foo
bar
# in another terminal: etcdctl put fooz1 barz1
PUT
fooz1
barz1
```
Here is the command to watch on multiple keys `foo` and `zoo`:
```bash
$ etcdctl watch -i
$ watch foo
$ watch zoo
# in another terminal: etcdctl put foo bar
PUT
foo
bar
# in another terminal: etcdctl put zoo val
PUT
zoo
val
```
## Watch historical changes of keys
Applications may want to watch for historical changes of keys in etcd. For example, an application may wish to receive all the modifications of a key; if the application stays connected to etcd, then `watch` is good enough. However, if the application or etcd fails, a change may happen during the failure, and the application will not receive the update in real time. To guarantee the update is delivered, the application must be able to watch for historical changes to keys. To do this, an application can specify a historical revision on a watch, just like reading past version of keys.
Suppose we finished the following sequence of operations:
```bash
$ etcdctl put foo bar # revision = 2
OK
$ etcdctl put foo1 bar1 # revision = 3
OK
$ etcdctl put foo bar_new # revision = 4
OK
$ etcdctl put foo1 bar1_new # revision = 5
OK
```
Here is an example to watch the historical changes:
```bash
# watch for changes on key `foo` since revision 2
$ etcdctl watch --rev=2 foo
PUT
foo
bar
PUT
foo
bar_new
```
```bash
# watch for changes on key `foo` since revision 3
$ etcdctl watch --rev=3 foo
PUT
foo
bar_new
```
Here is an example to watch only from the last historical change:
```bash
# watch for changes on key `foo` and return last revision value along with modified value
$ etcdctl watch --prev-kv foo
# in another terminal: etcdctl put foo bar_latest
PUT
foo # key
bar_new # last value of foo key before modification
foo # key
bar_latest # value of foo key after modification
```
## Watch progress
Applications may want to check the progress of a watch to determine how up-to-date the watch stream is. For example, if a watch is used to update a cache, it can be useful to know if the cache is stale compared to the revision from a quorum read.
Progress requests can be issued using the "progress" command in interactive watch session to ask the etcd server to send a progress notify update in the watch stream:
```bash
$ etcdctl watch -i
$ watch a
$ progress
progress notify: 1
# in another terminal: etcdctl put x 0
# in another terminal: etcdctl put y 1
$ progress
progress notify: 3
```
Note: The revision number in the progress notify response is the revision from the local etcd server node that the watch stream is connected to. If this node is partitioned and not part of quorum, this progress notify revision might be lower than
than the revision returned by a quorum read against a non-partitioned etcd server node.
## Compacted revisions
As we mentioned, etcd keeps revisions so that applications can read past versions of keys. However, to avoid accumulating an unbounded amount of history, it is important to compact past revisions. After compacting, etcd removes historical revisions, releasing resources for future use. All superseded data with revisions before the compacted revision will be unavailable.
Here is the command to compact the revisions:
```bash
$ etcdctl compact 5
compacted revision 5
# any revisions before the compacted one are not accessible
$ etcdctl get --rev=4 foo
Error: rpc error: code = 11 desc = etcdserver: mvcc: required revision has been compacted
```
Note: The current revision of etcd server can be found using get command on any key (existent or non-existent) in json format. Example is shown below for mykey which does not exist in etcd server:
```bash
$ etcdctl get mykey -w=json
{"header":{"cluster_id":14841639068965178418,"member_id":10276657743932975437,"revision":15,"raft_term":4}}
```
## Grant leases
Applications can grant leases for keys from an etcd cluster. When a key is attached to a lease, its lifetime is bound to the lease's lifetime which in turn is governed by a time-to-live (TTL). Each lease has a minimum time-to-live (TTL) value specified by the application at grant time. The lease's actual TTL value is at least the minimum TTL and is chosen by the etcd cluster. Once a lease's TTL elapses, the lease expires and all attached keys are deleted.
Here is the command to grant a lease:
```bash
# grant a lease with 10 second TTL
$ etcdctl lease grant 10
lease 32695410dcc0ca06 granted with TTL(10s)
# attach key foo to lease 32695410dcc0ca06
$ etcdctl put --lease=32695410dcc0ca06 foo bar
OK
```
## Revoke leases
Applications revoke leases by lease ID. Revoking a lease deletes all of its attached keys.
Suppose we finished the following sequence of operations:
```bash
$ etcdctl lease grant 10
lease 32695410dcc0ca06 granted with TTL(10s)
$ etcdctl put --lease=32695410dcc0ca06 foo bar
OK
```
Here is the command to revoke the same lease:
```bash
$ etcdctl lease revoke 32695410dcc0ca06
lease 32695410dcc0ca06 revoked
$ etcdctl get foo
# empty response since foo is deleted due to lease revocation
```
## Keep leases alive
Applications can keep a lease alive by refreshing its TTL so it does not expire.
Suppose we finished the following sequence of operations:
```bash
$ etcdctl lease grant 10
lease 32695410dcc0ca06 granted with TTL(10s)
```
Here is the command to keep the same lease alive:
```bash
$ etcdctl lease keep-alive 32695410dcc0ca06
lease 32695410dcc0ca06 keepalived with TTL(10)
lease 32695410dcc0ca06 keepalived with TTL(10)
lease 32695410dcc0ca06 keepalived with TTL(10)
...
```
## Get lease information
Applications may want to know about lease information, so that they can be renewed or to check if the lease still exists or it has expired. Applications may also want to know the keys to which a particular lease is attached.
Suppose we finished the following sequence of operations:
```bash
# grant a lease with 500 second TTL
$ etcdctl lease grant 500
lease 694d5765fc71500b granted with TTL(500s)
# attach key zoo1 to lease 694d5765fc71500b
$ etcdctl put zoo1 val1 --lease=694d5765fc71500b
OK
# attach key zoo2 to lease 694d5765fc71500b
$ etcdctl put zoo2 val2 --lease=694d5765fc71500b
OK
```
Here is the command to get information about the lease:
```bash
$ etcdctl lease timetolive 694d5765fc71500b
lease 694d5765fc71500b granted with TTL(500s), remaining(258s)
```
Here is the command to get information about the lease along with the keys attached with the lease:
```bash
$ etcdctl lease timetolive --keys 694d5765fc71500b
lease 694d5765fc71500b granted with TTL(500s), remaining(132s), attached keys([zoo2 zoo1])
# if the lease has expired or does not exist it will give the below response:
Error: etcdserver: requested lease not found
```

View File

@ -1,11 +0,0 @@
---
title: System limits
---
## Request size limit
etcd is designed to handle small key value pairs typical for metadata. Larger requests will work, but may increase the latency of other requests. By default, the maximum size of any request is 1.5 MiB. This limit is configurable through `--max-request-bytes` flag for etcd server.
## Storage size limit
The default storage size limit is 2GB, configurable with `--quota-backend-bytes` flag. 8GB is a suggested maximum size for normal environments and etcd warns at startup if the configured value exceeds it.

View File

@ -1,151 +0,0 @@
---
title: Set up a local cluster
---
For testing and development deployments, the quickest and easiest way is to configure a local cluster. For a production deployment, refer to the [clustering][clustering] section.
## Local standalone cluster
### Starting a cluster
Run the following to deploy an etcd cluster as a standalone cluster:
```
$ ./etcd
...
```
If the `etcd` binary is not present in the current working directory, it might be located either at `$GOPATH/bin/etcd` or at `/usr/local/bin/etcd`. Run the command appropriately.
The running etcd member listens on `localhost:2379` for client requests.
### Interacting with the cluster
Use `etcdctl` to interact with the running cluster:
1. Store an example key-value pair in the cluster:
```
$ ./etcdctl put foo bar
OK
```
If OK is printed, storing key-value pair is successful.
2. Retrieve the value of `foo`:
```
$ ./etcdctl get foo
bar
```
If `bar` is returned, interaction with the etcd cluster is working as expected.
## Local multi-member cluster
### Starting a cluster
A `Procfile` at the base of the etcd git repository is provided to easily configure a local multi-member cluster. To start a multi-member cluster, navigate to the root of the etcd source tree and perform the following:
1. Install `goreman` to control Procfile-based applications:
```
$ go get github.com/mattn/goreman
```
2. Start a cluster with `goreman` using etcd's stock Procfile:
```
$ goreman -f Procfile start
```
The members start running. They listen on `localhost:2379`, `localhost:22379`, and `localhost:32379` respectively for client requests.
### Interacting with the cluster
Use `etcdctl` to interact with the running cluster:
1. Print the list of members:
```
$ etcdctl --write-out=table --endpoints=localhost:2379 member list
```
The list of etcd members are displayed as follows:
```
+------------------+---------+--------+------------------------+------------------------+
| ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS |
+------------------+---------+--------+------------------------+------------------------+
| 8211f1d0f64f3269 | started | infra1 | http://127.0.0.1:2380 | http://127.0.0.1:2379 |
| 91bc3c398fb3c146 | started | infra2 | http://127.0.0.1:22380 | http://127.0.0.1:22379 |
| fd422379fda50e48 | started | infra3 | http://127.0.0.1:32380 | http://127.0.0.1:32379 |
+------------------+---------+--------+------------------------+------------------------+
```
2. Store an example key-value pair in the cluster:
```
$ etcdctl put foo bar
OK
```
If OK is printed, storing key-value pair is successful.
### Testing fault tolerance
To exercise etcd's fault tolerance, kill a member and attempt to retrieve the key.
1. Identify the process name of the member to be stopped.
The `Procfile` lists the properties of the multi-member cluster. For example, consider the member with the process name, `etcd2`.
2. Stop the member:
```
# kill etcd2
$ goreman run stop etcd2
```
3. Store a key:
```
$ etcdctl put key hello
OK
```
4. Retrieve the key that is stored in the previous step:
```
$ etcdctl get key
hello
```
5. Retrieve a key from the stopped member:
```
$ etcdctl --endpoints=localhost:22379 get key
```
The command should display an error caused by connection failure:
```
2017/06/18 23:07:35 grpc: Conn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp 127.0.0.1:22379: getsockopt: connection refused"; Reconnecting to "localhost:22379"
Error: grpc: timed out trying to connect
```
6. Restart the stopped member:
```
$ goreman run restart etcd2
```
7. Get the key from the restarted member:
```
$ etcdctl --endpoints=localhost:22379 get key
hello
```
Restarting the member re-establish the connection. `etcdctl` will now be able to retrieve the key successfully. To learn more about interacting with etcd, read [interacting with etcd section][interacting].
[interacting]: ./interacting_v3.md
[clustering]: ../op-guide/clustering.md

View File

@ -1,115 +0,0 @@
---
title: Discovery service protocol
---
Discovery service protocol helps new etcd member to discover all other members in cluster bootstrap phase using a shared discovery URL.
Discovery service protocol is _only_ used in cluster bootstrap phase, and cannot be used for runtime reconfiguration or cluster monitoring.
The protocol uses a new discovery token to bootstrap one _unique_ etcd cluster. Remember that one discovery token can represent only one etcd cluster. As long as discovery protocol on this token starts, even if it fails halfway, it must not be used to bootstrap another etcd cluster.
The rest of this article will walk through the discovery process with examples that correspond to a self-hosted discovery cluster. The public discovery service, discovery.etcd.io, functions the same way, but with a layer of polish to abstract away ugly URLs, generate UUIDs automatically, and provide some protections against excessive requests. At its core, the public discovery service still uses an etcd cluster as the data store as described in this document.
## Protocol workflow
The idea of discovery protocol is to use an internal etcd cluster to coordinate bootstrap of a new cluster. First, all new members interact with discovery service and help to generate the expected member list. Then each new member bootstraps its server using this list, which performs the same functionality as -initial-cluster flag.
In the following example workflow, we will list each step of protocol in curl format for ease of understanding.
By convention the etcd discovery protocol uses the key prefix `_etcd/registry`. If `http://example.com` hosts an etcd cluster for discovery service, a full URL to discovery keyspace will be `http://example.com/v2/keys/_etcd/registry`. We will use this as the URL prefix in the example.
### Creating a new discovery token
Generate a unique token that will identify the new cluster. This will be used as a unique prefix in discovery keyspace in the following steps. An easy way to do this is to use `uuidgen`:
```
UUID=$(uuidgen)
```
### Specifying the expected cluster size
The discovery token expects a cluster size that must be specified. The size is used by the discovery service to know when it has found all members that will initially form the cluster.
```
curl -X PUT http://example.com/v2/keys/_etcd/registry/${UUID}/_config/size -d value=${cluster_size}
```
Usually the cluster size is 3, 5 or 7. Check [optimal cluster size][cluster-size] for more details.
### Bringing up etcd processes
Given the discovery URL, use it as `-discovery` flag and bring up etcd processes. Every etcd process will follow this next few steps internally if given a `-discovery` flag.
### Registering itself
The first thing for etcd process is to register itself into the discovery URL as a member. This is done by creating member ID as a key in the discovery URL.
```
curl -X PUT http://example.com/v2/keys/_etcd/registry/${UUID}/${member_id}?prevExist=false -d value="${member_name}=${member_peer_url_1}&${member_name}=${member_peer_url_2}"
```
### Checking the status
It checks the expected cluster size and registration status in discovery URL, and decides what the next action is.
```
curl -X GET http://example.com/v2/keys/_etcd/registry/${UUID}/_config/size
curl -X GET http://example.com/v2/keys/_etcd/registry/${UUID}
```
If registered members are still not enough, it will wait for left members to appear.
If the number of registered members is bigger than the expected size N, it treats the first N registered members as the member list for the cluster. If the member itself is in the member list, the discovery procedure succeeds and it fetches all peers through the member list. If it is not in the member list, the discovery procedure finishes with the failure that the cluster has been full.
In etcd implementation, the member may check the cluster status even before registering itself. So it could fail quickly if the cluster has been full.
### Waiting for all members
The wait process is described in detail in the [etcd API documentation][api].
```
curl -X GET http://example.com/v2/keys/_etcd/registry/${UUID}?wait=true&waitIndex=${current_etcd_index}
```
It keeps waiting until finding all members.
## Public discovery service
CoreOS Inc. hosts a public discovery service at https://discovery.etcd.io/ , which provides some nice features for ease of use.
### Mask key prefix
Public discovery service will redirect `https://discovery.etcd.io/${UUID}` to etcd cluster behind for the key at `/v2/keys/_etcd/registry`. It masks register key prefix for short and readable discovery url.
### Get new token
```
GET /new
Sent query:
size=${cluster_size}
Possible status codes:
200 OK
400 Bad Request
200 Body:
generated discovery url
```
The generation process in the service follows the steps from [Creating a New Discovery Token][new-discovery-token] to [Specifying the Expected Cluster Size][expected-cluster-size].
### Check discovery status
```
GET /${UUID}
```
The status for this discovery token, including the machines that have been registered, can be checked by requesting the value of the UUID.
### Open-source repository
The repository is located at https://github.com/coreos/discovery.etcd.io. It could be used to build a custom discovery service.
[api]: ../v2/api.md#waiting-for-a-change
[cluster-size]: ../v2/admin_guide.md#optimal-cluster-size
[expected-cluster-size]: #specifying-the-expected-cluster-size
[new-discovery-token]: #creating-a-new-discovery-token

View File

@ -1,31 +0,0 @@
---
title: Logging conventions
---
etcd uses the [capnslog][capnslog] library for logging application output categorized into *levels*. A log message's level is determined according to these conventions:
* Error: Data has been lost, a request has failed for a bad reason, or a required resource has been lost
* Examples:
* A failure to allocate disk space for WAL
* Warning: (Hopefully) Temporary conditions that may cause errors, but may work fine. A replica disappearing (that may reconnect) is a warning.
* Examples:
* Failure to send raft message to a remote peer
* Failure to receive heartbeat message within the configured election timeout
* Notice: Normal, but important (uncommon) log information.
* Examples:
* Add a new node into the cluster
* Add a new user into auth subsystem
* Info: Normal, working log information, everything is fine, but helpful notices for auditing or common operations.
* Examples:
* Startup configuration
* Start to do snapshot
* Debug: Everything is still fine, but even common operations may be logged, and less helpful but more quantity of notices.
* Examples:
* Send a normal message to a remote peer
* Write a log entry to disk
[capnslog]: https://github.com/coreos/pkg/tree/master/capnslog

View File

@ -1,162 +0,0 @@
---
title: etcd release guide
---
The guide talks about how to release a new version of etcd.
The procedure includes some manual steps for sanity checking, but it can probably be further scripted. Please keep this document up-to-date if making changes to the release process.
## Release management
etcd community members are assigned to manage the release each etcd major/minor version as well as manage patches
and to each stable release branch. The managers are responsible for communicating the timelines and status of each
release and for ensuring the stability of the release branch.
| Releases | Manager |
| -------- | ------- |
| 3.1 patch (post 3.1.0) | Joe Betz [@jpbetz](https://github.com/jpbetz) |
| 3.2 patch (post 3.2.0) | Joe Betz [@jpbetz](https://github.com/jpbetz) |
| 3.3 patch (post 3.3.0) | Gyuho Lee [@gyuho](https://github.com/gyuho) |
## Prepare release
Set desired version as environment variable for following steps. Here is an example to release 2.3.0:
```
export VERSION=v2.3.0
export PREV_VERSION=v2.2.5
```
All releases version numbers follow the format of [semantic versioning 2.0.0](http://semver.org/).
### Major, minor version release, or its pre-release
- Ensure the relevant milestone on GitHub is complete. All referenced issues should be closed, or moved elsewhere.
- Remove this release from [roadmap](https://github.com/etcd-io/etcd/blob/master/ROADMAP.md), if necessary.
- Ensure the latest upgrade documentation is available.
- Bump [hardcoded MinClusterVerion in the repository](https://github.com/etcd-io/etcd/blob/master/version/version.go#L29), if necessary.
- Add feature capability maps for the new version, if necessary.
### Patch version release
- To request a backport, devlopers submit cherrypick PRs targeting the release branch. The commits should not include merge commits. The commits should be restricted to bug fixes and security patches.
- The cherrypick PRs should target the appropriate release branch (`base:release-<major>-<minor>`). `hack/patch/cherrypick.sh` may be used to automatically generate cherrypick PRs.
- The release patch manager reviews the cherrypick PRs. Please discuss carefully what is backported to the patch release. Each patch release should be strictly better than it's predecessor.
- The release patch manager will cherry-pick these commits starting from the oldest one into stable branch.
## Write release note
- Write introduction for the new release. For example, what major bug we fix, what new features we introduce or what performance improvement we make.
- Put `[GH XXXX]` at the head of change line to reference Pull Request that introduces the change. Moreover, add a link on it to jump to the Pull Request.
- Find PRs with `release-note` label and explain them in `NEWS` file, as a straightforward summary of changes for end-users.
## Tag version
- Bump [hardcoded Version in the repository](https://github.com/etcd-io/etcd/blob/master/version/version.go#L30) to the latest version `${VERSION}`.
- Ensure all tests on CI system are passed.
- Manually check etcd is buildable in Linux, Darwin and Windows.
- Manually check upgrade etcd cluster of previous minor version works well.
- Manually check new features work well.
- Add a signed tag through `git tag -s ${VERSION}`.
- Sanity check tag correctness through `git show tags/$VERSION`.
- Push the tag to GitHub through `git push origin tags/$VERSION`. This assumes `origin` corresponds to "https://github.com/etcd-io/etcd".
## Build release binaries and images
- Ensure `acbuild` is available.
- Ensure `docker` is available.
Run release script in root directory:
```
TAG=gcr.io/etcd-development/etcd ./scripts/release.sh ${VERSION}
```
It generates all release binaries and images under directory ./release.
## Sign binaries, images, and source code
etcd project key must be used to sign the generated binaries and images.`$SUBKEYID` is the key ID of etcd project Yubikey. Connect the key and run `gpg2 --card-status` to get the ID.
The following commands are used for public release sign:
```
cd release
for i in etcd-*{.zip,.tar.gz}; do gpg2 --default-key $SUBKEYID --armor --output ${i}.asc --detach-sign ${i}; done
for i in etcd-*{.zip,.tar.gz}; do gpg2 --verify ${i}.asc ${i}; done
# sign zipped source code files
wget https://github.com/etcd-io/etcd/archive/${VERSION}.zip
gpg2 --armor --default-key $SUBKEYID --output ${VERSION}.zip.asc --detach-sign ${VERSION}.zip
gpg2 --verify ${VERSION}.zip.asc ${VERSION}.zip
wget https://github.com/etcd-io/etcd/archive/${VERSION}.tar.gz
gpg2 --armor --default-key $SUBKEYID --output ${VERSION}.tar.gz.asc --detach-sign ${VERSION}.tar.gz
gpg2 --verify ${VERSION}.tar.gz.asc ${VERSION}.tar.gz
```
The public key for GPG signing can be found at [CoreOS Application Signing Key](https://coreos.com/security/app-signing-key)
## Publish release page in GitHub
- Set release title as the version name.
- Follow the format of previous release pages.
- Attach the generated binaries and signatures.
- Select whether it is a pre-release.
- Publish the release!
## Publish docker image in gcr.io
- Push docker image:
```
gcloud docker -- login -u _json_key -p "$(cat /etc/gcp-key-etcd.json)" https://gcr.io
for TARGET_ARCH in "-arm64" "-ppc64le" ""; do
gcloud docker -- push gcr.io/etcd-development/etcd:${VERSION}${TARGET_ARCH}
done
```
- Add `latest` tag to the new image on [gcr.io](https://console.cloud.google.com/gcr/images/etcd-development/GLOBAL/etcd?project=etcd-development&authuser=1) if this is a stable release.
## Publish docker image in Quay.io
- Build docker images with quay.io:
```
for TARGET_ARCH in "amd64" "arm64" "ppc64le"; do
TAG=quay.io/coreos/etcd GOARCH=${TARGET_ARCH} \
BINARYDIR=release/etcd-${VERSION}-linux-${TARGET_ARCH} \
BUILDDIR=release \
./scripts/build-docker ${VERSION}
done
```
- Push docker image:
```
docker login quay.io
for TARGET_ARCH in "-arm64" "-ppc64le" ""; do
docker push quay.io/coreos/etcd:${VERSION}${TARGET_ARCH}
done
```
- Add `latest` tag to the new image on [quay.io](https://quay.io/repository/coreos/etcd?tag=latest&tab=tags) if this is a stable release.
## Announce to the etcd-dev Googlegroup
- Follow the format of [previous release emails](https://groups.google.com/forum/#!forum/etcd-dev).
- Make sure to include a list of authors that contributed since the previous release - something like the following might be handy:
```
git log ...${PREV_VERSION} --pretty=format:"%an" | sort | uniq | tr '\n' ',' | sed -e 's#,#, #g' -e 's#, $##'
```
- Send email to etcd-dev@googlegroups.com
## Post release
- Create new stable branch through `git push origin ${VERSION_MAJOR}.${VERSION_MINOR}` if this is a major stable release. This assumes `origin` corresponds to "https://github.com/etcd-io/etcd".
- Bump [hardcoded Version in the repository](https://github.com/etcd-io/etcd/blob/master/version/version.go#L30) to the version `${VERSION}+git`.

View File

@ -1,69 +0,0 @@
---
title: Download and build
---
## System requirements
The etcd performance benchmarks run etcd on 8 vCPU, 16GB RAM, 50GB SSD GCE instances, but any relatively modern machine with low latency storage and a few gigabytes of memory should suffice for most use cases. Applications with large v2 data stores will require more memory than a large v3 data store since data is kept in anonymous memory instead of memory mapped from a file. For running etcd on a cloud provider, see the [Example hardware configuration][example-hardware-configurations] documentation.
## Download the pre-built binary
The easiest way to get etcd is to use one of the pre-built release binaries which are available for OSX, Linux, Windows, appc, and Docker. Instructions for using these binaries are on the [GitHub releases page][github-release].
## Build the latest version
For those wanting to try the very latest version, build etcd from the `master` branch. [Go](https://golang.org/) version 1.9+ is required to build the latest version of etcd. To ensure etcd is built against well-tested libraries, etcd vendors its dependencies for official release binaries. However, etcd's vendoring is also optional to avoid potential import conflicts when embedding the etcd server or using the etcd client.
To build `etcd` from the `master` branch without a `GOPATH` using the official `build` script:
```sh
$ git clone https://github.com/etcd-io/etcd.git
$ cd etcd
$ ./build
```
To build a vendored `etcd` from the `master` branch via `go get`:
```sh
# GOPATH should be set
$ echo $GOPATH
/Users/example/go
$ go get -v go.etcd.io/etcd
$ go get -v go.etcd.io/etcd/etcdctl
```
## Test the installation
Check the etcd binary is built correctly by starting etcd and setting a key.
### Starting etcd
If etcd is built without using `go get`, run the following:
```sh
$ ./bin/etcd
```
If etcd is built using `go get`, run the following:
```sh
$ $GOPATH/bin/etcd
```
### Setting a key
Run the following:
```sh
$ ./bin/etcdctl put foo bar
OK
```
(or `$GOPATH/bin/etcdctl put foo bar` if etcdctl was installed with `go get`)
If OK is printed, then etcd is working!
[github-release]: https://github.com/etcd-io/etcd/releases/
[go]: https://golang.org/doc/install
[build-script]: ../build
[cmd-directory]: ../cmd
[example-hardware-configurations]: op-guide/hardware.md#example-hardware-configurations

View File

@ -1,118 +0,0 @@
# Documentation
etcd is a distributed key-value store designed to reliably and quickly preserve and provide access to critical data. It enables reliable distributed coordination through distributed locking, leader elections, and write barriers. An etcd cluster is intended for high availability and permanent data storage and retrieval.
## Getting started
New etcd users and developers should get started by [downloading and building][download_build] etcd. After getting etcd, follow this [quick demo][demo] to see the basics of creating and working with an etcd cluster.
## Developing with etcd
The easiest way to get started using etcd as a distributed key-value store is to [set up a local cluster][local_cluster].
- [Setting up local clusters][local_cluster]
- [Interacting with etcd][interacting]
- gRPC [etcd core][api_ref] and [etcd concurrency][api_concurrency_ref] API references
- [HTTP JSON API through the gRPC gateway][api_grpc_gateway]
- [gRPC naming and discovery][grpc_naming]
- [Client][namespace_client] and [proxy][namespace_proxy] namespacing
- [Embedding etcd][embed_etcd]
- [Experimental features and APIs][experimental]
- [System limits][system-limit]
## Operating etcd clusters
Administrators who need a fault-tolerant etcd cluster for either development or production should begin with a [cluster on multiple machines][clustering].
### Setting up etcd
- [Configuration flags][conf]
- [Multi-member cluster][clustering]
- [gRPC proxy][grpc_proxy]
- [L4 gateway][gateway]
### System configuration
- [Supported systems][supported_platforms]
- [Hardware recommendations][hardware]
- [Performance benchmarking][performance]
- [Tuning][tuning]
### Platform guides
- [Amazon Web Services][aws_platform]
- [Container Linux, systemd][container_linux_platform]
- [FreeBSD][freebsd_platform]
- [Docker container][container_docker]
- [rkt container][container_rkt]
### Security
- [TLS][security]
- [Role-based access control][authentication]
### Maintenance and troubleshooting
- [Frequently asked questions][faq]
- [Monitoring][monitoring]
- [Maintenance][maintenance]
- [Failure modes][failures]
- [Disaster recovery][recovery]
- [Upgrading][upgrading]
## Learning
To learn more about the concepts and internals behind etcd, read the following pages:
- [Why etcd?][why]
- [Understand data model][data_model]
- [Understand APIs][understand_apis]
- [Glossary][glossary]
- Design
- [Auth subsystem][design-auth-v3]
- [Client][design-client]
- [Learner][design-learner]
[api_ref]: dev-guide/api_reference_v3.md
[api_concurrency_ref]: dev-guide/api_concurrency_reference_v3.md
[api_grpc_gateway]: dev-guide/api_grpc_gateway.md
[clustering]: op-guide/clustering.md
[conf]: op-guide/configuration.md
[system-limit]: dev-guide/limit.md
[faq]: faq.md
[why]: learning/why.md
[data_model]: learning/data_model.md
[demo]: demo.md
[download_build]: dl_build.md
[embed_etcd]: https://godoc.org/github.com/etcd-io/etcd/embed
[grpc_naming]: dev-guide/grpc_naming.md
[failures]: op-guide/failures.md
[gateway]: op-guide/gateway.md
[glossary]: learning/glossary.md
[namespace_client]: https://godoc.org/github.com/etcd-io/etcd/clientv3/namespace
[namespace_proxy]: op-guide/grpc_proxy.md#namespacing
[grpc_proxy]: op-guide/grpc_proxy.md
[hardware]: op-guide/hardware.md
[interacting]: dev-guide/interacting_v3.md
[local_cluster]: dev-guide/local_cluster.md
[performance]: op-guide/performance.md
[recovery]: op-guide/recovery.md
[maintenance]: op-guide/maintenance.md
[security]: op-guide/security.md
[monitoring]: op-guide/monitoring.md
[v2_migration]: op-guide/v2-migration.md
[container_rkt]: op-guide/container.md#rkt
[container_docker]: op-guide/container.md#docker
[understand_apis]: learning/api.md
[versioning]: op-guide/versioning.md
[supported_platforms]: op-guide/supported-platform.md
[container_linux_platform]: platforms/container-linux-systemd.md
[freebsd_platform]: platforms/freebsd.md
[aws_platform]: platforms/aws.md
[experimental]: dev-guide/experimental_apis.md
[authentication]: op-guide/authentication.md
[design-auth-v3]: learning/design-auth-v3.md
[design-client]: learning/design-client.md
[design-learner]: learning/design-learner.md
[tuning]: tuning.md
[upgrading]: upgrades/upgrading-etcd.md

View File

@ -1,85 +0,0 @@
rule_files:
- mixin.yaml
evaluation_interval: 1m
tests:
- interval: 1m
input_series:
- series: 'up{job="etcd",instance="10.10.10.0"}'
values: '1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0'
- series: 'up{job="etcd",instance="10.10.10.1"}'
values: '1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0'
- series: 'up{job="etcd",instance="10.10.10.2"}'
values: '1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0'
alert_rule_test:
- eval_time: 3m
alertname: etcdInsufficientMembers
- eval_time: 5m
alertname: etcdInsufficientMembers
- eval_time: 5m
alertname: etcdMembersDown
- eval_time: 7m
alertname: etcdMembersDown
exp_alerts:
- exp_labels:
job: etcd
severity: critical
exp_annotations:
message: 'etcd cluster "etcd": members are down (1).'
- eval_time: 7m
alertname: etcdInsufficientMembers
- eval_time: 11m
alertname: etcdInsufficientMembers
exp_alerts:
- exp_labels:
job: etcd
severity: critical
exp_annotations:
message: 'etcd cluster "etcd": insufficient members (1).'
- eval_time: 15m
alertname: etcdInsufficientMembers
exp_alerts:
- exp_labels:
job: etcd
severity: critical
exp_annotations:
message: 'etcd cluster "etcd": insufficient members (0).'
- interval: 1m
input_series:
- series: 'up{job="etcd",instance="10.10.10.0"}'
values: '1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0'
- series: 'up{job="etcd",instance="10.10.10.1"}'
values: '1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0'
- series: 'up{job="etcd",instance="10.10.10.2"}'
values: '1 1 1 1 0 0 0 0'
alert_rule_test:
- eval_time: 10m
alertname: etcdMembersDown
exp_alerts:
- exp_labels:
job: etcd
severity: critical
exp_annotations:
message: 'etcd cluster "etcd": members are down (2).'
- interval: 1m
input_series:
- series: 'up{job="etcd",instance="10.10.10.0"}'
values: '1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0'
- series: 'up{job="etcd",instance="10.10.10.1"}'
values: '1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0'
- series: 'etcd_network_peer_sent_failures_total{To="member-1",job="etcd",endpoint="test"}'
values: '0 0 1 2 3 4 5 6 7 8 9 10'
alert_rule_test:
- eval_time: 4m
alertname: etcdMembersDown
- eval_time: 6m
alertname: etcdMembersDown
exp_alerts:
- exp_labels:
job: etcd
severity: critical
exp_annotations:
message: 'etcd cluster "etcd": members are down (1).'

View File

@ -1,165 +0,0 @@
---
title: Frequently Asked Questions (FAQ)
---
## etcd, general
### Do clients have to send requests to the etcd leader?
[Raft][raft] is leader-based; the leader handles all client requests which need cluster consensus. However, the client does not need to know which node is the leader. Any request that requires consensus sent to a follower is automatically forwarded to the leader. Requests that do not require consensus (e.g., serialized reads) can be processed by any cluster member.
## Configuration
### What is the difference between listen-<client,peer>-urls, advertise-client-urls or initial-advertise-peer-urls?
`listen-client-urls` and `listen-peer-urls` specify the local addresses etcd server binds to for accepting incoming connections. To listen on a port for all interfaces, specify `0.0.0.0` as the listen IP address.
`advertise-client-urls` and `initial-advertise-peer-urls` specify the addresses etcd clients or other etcd members should use to contact the etcd server. The advertise addresses must be reachable from the remote machines. Do not advertise addresses like `localhost` or `0.0.0.0` for a production setup since these addresses are unreachable from remote machines.
### Why doesn't changing `--listen-peer-urls` or `--initial-advertise-peer-urls` update the advertised peer URLs in `etcdctl member list`?
A member's advertised peer URLs come from `--initial-advertise-peer-urls` on initial cluster boot. Changing the listen peer URLs or the initial advertise peers after booting the member won't affect the exported advertise peer URLs since changes must go through quorum to avoid membership configuration split brain. Use `etcdctl member update` to update a member's peer URLs.
## Deployment
### System requirements
Since etcd writes data to disk, its performance strongly depends on disk performance. For this reason, SSD is highly recommended. To assess whether a disk is fast enough for etcd, one possibility is using a disk benchmarking tool such as [fio][fio]. For an example on how to do that, read [here][fio-blog-post]. To prevent performance degradation or unintentionally overloading the key-value store, etcd enforces a configurable storage size quota set to 2GB by default. To avoid swapping or running out of memory, the machine should have at least as much RAM to cover the quota. 8GB is a suggested maximum size for normal environments and etcd warns at startup if the configured value exceeds it. At CoreOS, an etcd cluster is usually deployed on dedicated CoreOS Container Linux machines with dual-core processors, 2GB of RAM, and 80GB of SSD *at the very least*. **Note that performance is intrinsically workload dependent; please test before production deployment**. See [hardware][hardware-setup] for more recommendations.
Most stable production environment is Linux operating system with amd64 architecture; see [supported platform][supported-platform] for more.
### Why an odd number of cluster members?
An etcd cluster needs a majority of nodes, a quorum, to agree on updates to the cluster state. For a cluster with n members, quorum is (n/2)+1. For any odd-sized cluster, adding one node will always increase the number of nodes necessary for quorum. Although adding a node to an odd-sized cluster appears better since there are more machines, the fault tolerance is worse since exactly the same number of nodes may fail without losing quorum but there are more nodes that can fail. If the cluster is in a state where it can't tolerate any more failures, adding a node before removing nodes is dangerous because if the new node fails to register with the cluster (e.g., the address is misconfigured), quorum will be permanently lost.
### What is maximum cluster size?
Theoretically, there is no hard limit. However, an etcd cluster probably should have no more than seven nodes. [Google Chubby lock service][chubby], similar to etcd and widely deployed within Google for many years, suggests running five nodes. A 5-member etcd cluster can tolerate two member failures, which is enough in most cases. Although larger clusters provide better fault tolerance, the write performance suffers because data must be replicated across more machines.
### What is failure tolerance?
An etcd cluster operates so long as a member quorum can be established. If quorum is lost through transient network failures (e.g., partitions), etcd automatically and safely resumes once the network recovers and restores quorum; Raft enforces cluster consistency. For power loss, etcd persists the Raft log to disk; etcd replays the log to the point of failure and resumes cluster participation. For permanent hardware failure, the node may be removed from the cluster through [runtime reconfiguration][runtime reconfiguration].
It is recommended to have an odd number of members in a cluster. An odd-size cluster tolerates the same number of failures as an even-size cluster but with fewer nodes. The difference can be seen by comparing even and odd sized clusters:
| Cluster Size | Majority | Failure Tolerance |
|:-:|:-:|:-:|
| 1 | 1 | 0 |
| 2 | 2 | 0 |
| 3 | 2 | 1 |
| 4 | 3 | 1 |
| 5 | 3 | 2 |
| 6 | 4 | 2 |
| 7 | 4 | 3 |
| 8 | 5 | 3 |
| 9 | 5 | 4 |
Adding a member to bring the size of cluster up to an even number doesn't buy additional fault tolerance. Likewise, during a network partition, an odd number of members guarantees that there will always be a majority partition that can continue to operate and be the source of truth when the partition ends.
### Does etcd work in cross-region or cross data center deployments?
Deploying etcd across regions improves etcd's fault tolerance since members are in separate failure domains. The cost is higher consensus request latency from crossing data center boundaries. Since etcd relies on a member quorum for consensus, the latency from crossing data centers will be somewhat pronounced because at least a majority of cluster members must respond to consensus requests. Additionally, cluster data must be replicated across all peers, so there will be bandwidth cost as well.
With longer latencies, the default etcd configuration may cause frequent elections or heartbeat timeouts. See [tuning] for adjusting timeouts for high latency deployments.
## Operation
### How to backup a etcd cluster?
etcdctl provides a `snapshot` command to create backups. See [backup][backup] for more details.
### Should I add a member before removing an unhealthy member?
When replacing an etcd node, it's important to remove the member first and then add its replacement.
etcd employs distributed consensus based on a quorum model; (n+1)/2 members, a majority, must agree on a proposal before it can be committed to the cluster. These proposals include key-value updates and membership changes. This model totally avoids any possibility of split brain inconsistency. The downside is permanent quorum loss is catastrophic.
How this applies to membership: If a 3-member cluster has 1 downed member, it can still make forward progress because the quorum is 2 and 2 members are still live. However, adding a new member to a 3-member cluster will increase the quorum to 3 because 3 votes are required for a majority of 4 members. Since the quorum increased, this extra member buys nothing in terms of fault tolerance; the cluster is still one node failure away from being unrecoverable.
Additionally, that new member is risky because it may turn out to be misconfigured or incapable of joining the cluster. In that case, there's no way to recover quorum because the cluster has two members down and two members up, but needs three votes to change membership to undo the botched membership addition. etcd will by default reject member add attempts that could take down the cluster in this manner.
On the other hand, if the downed member is removed from cluster membership first, the number of members becomes 2 and the quorum remains at 2. Following that removal by adding a new member will also keep the quorum steady at 2. So, even if the new node can't be brought up, it's still possible to remove the new member through quorum on the remaining live members.
### Why won't etcd accept my membership changes?
etcd sets `strict-reconfig-check` in order to reject reconfiguration requests that would cause quorum loss. Abandoning quorum is really risky (especially when the cluster is already unhealthy). Although it may be tempting to disable quorum checking if there's quorum loss to add a new member, this could lead to full fledged cluster inconsistency. For many applications, this will make the problem even worse ("disk geometry corruption" being a candidate for most terrifying).
### Why does etcd lose its leader from disk latency spikes?
This is intentional; disk latency is part of leader liveness. Suppose the cluster leader takes a minute to fsync a raft log update to disk, but the etcd cluster has a one second election timeout. Even though the leader can process network messages within the election interval (e.g., send heartbeats), it's effectively unavailable because it can't commit any new proposals; it's waiting on the slow disk. If the cluster frequently loses its leader due to disk latencies, try [tuning][tuning] the disk settings or etcd time parameters.
### What does the etcd warning "request ignored (cluster ID mismatch)" mean?
Every new etcd cluster generates a new cluster ID based on the initial cluster configuration and a user-provided unique `initial-cluster-token` value. By having unique cluster ID's, etcd is protected from cross-cluster interaction which could corrupt the cluster.
Usually this warning happens after tearing down an old cluster, then reusing some of the peer addresses for the new cluster. If any etcd process from the old cluster is still running it will try to contact the new cluster. The new cluster will recognize a cluster ID mismatch, then ignore the request and emit this warning. This warning is often cleared by ensuring peer addresses among distinct clusters are disjoint.
### What does "mvcc: database space exceeded" mean and how do I fix it?
The [multi-version concurrency control][api-mvcc] data model in etcd keeps an exact history of the keyspace. Without periodically compacting this history (e.g., by setting `--auto-compaction`), etcd will eventually exhaust its storage space. If etcd runs low on storage space, it raises a space quota alarm to protect the cluster from further writes. So long as the alarm is raised, etcd responds to write requests with the error `mvcc: database space exceeded`.
To recover from the low space quota alarm:
1. [Compact][maintenance-compact] etcd's history.
2. [Defragment][maintenance-defragment] every etcd endpoint.
3. [Disarm][maintenance-disarm] the alarm.
### What does the etcd warning "etcdserver/api/v3rpc: transport: http2Server.HandleStreams failed to read frame: read tcp 127.0.0.1:2379->127.0.0.1:43020: read: connection reset by peer" mean?
This is gRPC-side warning when a server receives a TCP RST flag with client-side streams being prematurely closed. For example, a client closes its connection, while gRPC server has not yet processed all HTTP/2 frames in the TCP queue. Some data may have been lost in server side, but it is ok so long as client connection has already been closed.
Only [old versions of gRPC](https://github.com/grpc/grpc-go/issues/1362) log this. etcd [>=v3.2.13 by default log this with DEBUG level](https://github.com/etcd-io/etcd/pull/9080), thus only visible with `--debug` flag enabled.
## Performance
### How should I benchmark etcd?
Try the [benchmark] tool. Current [benchmark results][benchmark-result] are available for comparison.
### What does the etcd warning "apply entries took too long" mean?
After a majority of etcd members agree to commit a request, each etcd server applies the request to its data store and persists the result to disk. Even with a slow mechanical disk or a virtualized network disk, such as Amazons EBS or Googles PD, applying a request should normally take fewer than 50 milliseconds. If the average apply duration exceeds 100 milliseconds, etcd will warn that entries are taking too long to apply.
Usually this issue is caused by a slow disk. The disk could be experiencing contention among etcd and other applications, or the disk is too simply slow (e.g., a shared virtualized disk). To rule out a slow disk from causing this warning, monitor [backend_commit_duration_seconds][backend_commit_metrics] (p99 duration should be less than 25ms) to confirm the disk is reasonably fast. If the disk is too slow, assigning a dedicated disk to etcd or using faster disk will typically solve the problem.
The second most common cause is CPU starvation. If monitoring of the machines CPU usage shows heavy utilization, there may not be enough compute capacity for etcd. Moving etcd to dedicated machine, increasing process resource isolation cgroups, or renicing the etcd server process into a higher priority can usually solve the problem.
Expensive user requests which access too many keys (e.g., fetching the entire keyspace) can also cause long apply latencies. Accessing fewer than a several hundred keys per request, however, should always be performant.
If none of the above suggestions clear the warnings, please [open an issue][new_issue] with detailed logging, monitoring, metrics and optionally workload information.
### What does the etcd warning "failed to send out heartbeat on time" mean?
etcd uses a leader-based consensus protocol for consistent data replication and log execution. Cluster members elect a single leader, all other members become followers. The elected leader must periodically send heartbeats to its followers to maintain its leadership. Followers infer leader failure if no heartbeats are received within an election interval and trigger an election. If a leader doesnt send its heartbeats in time but is still running, the election is spurious and likely caused by insufficient resources. To catch these soft failures, if the leader skips two heartbeat intervals, etcd will warn it failed to send a heartbeat on time.
Usually this issue is caused by a slow disk. Before the leader sends heartbeats attached with metadata, it may need to persist the metadata to disk. The disk could be experiencing contention among etcd and other applications, or the disk is too simply slow (e.g., a shared virtualized disk). To rule out a slow disk from causing this warning, monitor [wal_fsync_duration_seconds][wal_fsync_duration_seconds] (p99 duration should be less than 10ms) to confirm the disk is reasonably fast. If the disk is too slow, assigning a dedicated disk to etcd or using faster disk will typically solve the problem. To tell whether a disk is fast enough for etcd, a benchmarking tool such as [fio][fio] can be used. Read [here][fio-blog-post] for an example.
The second most common cause is CPU starvation. If monitoring of the machines CPU usage shows heavy utilization, there may not be enough compute capacity for etcd. Moving etcd to dedicated machine, increasing process resource isolation with cgroups, or renicing the etcd server process into a higher priority can usually solve the problem.
A slow network can also cause this issue. If network metrics among the etcd machines shows long latencies or high drop rate, there may not be enough network capacity for etcd. Moving etcd members to a less congested network will typically solve the problem. However, if the etcd cluster is deployed across data centers, long latency between members is expected. For such deployments, tune the `heartbeat-interval` configuration to roughly match the round trip time between the machines, and the `election-timeout` configuration to be at least 5 * `heartbeat-interval`. See [tuning documentation][tuning] for detailed information.
If none of the above suggestions clear the warnings, please [open an issue][new_issue] with detailed logging, monitoring, metrics and optionally workload information.
### What does the etcd warning "snapshotting is taking more than x seconds to finish ..." mean?
etcd sends a snapshot of its complete key-value store to refresh slow followers and for [backups][backup]. Slow snapshot transfer times increase MTTR; if the cluster is ingesting data with high throughput, slow followers may livelock by needing a new snapshot before finishing receiving a snapshot. To catch slow snapshot performance, etcd warns when sending a snapshot takes more than thirty seconds and exceeds the expected transfer time for a 1Gbps connection.
[hardware-setup]: ./op-guide/hardware.md
[supported-platform]: ./op-guide/supported-platform.md
[wal_fsync_duration_seconds]: ./metrics.md#disk
[tuning]: ./tuning.md
[new_issue]: https://github.com/etcd-io/etcd/issues/new
[backend_commit_metrics]: ./metrics.md#disk
[raft]: https://raft.github.io/raft.pdf
[backup]: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/recovery.md#snapshotting-the-keyspace
[chubby]: http://static.googleusercontent.com/media/research.google.com/en//archive/chubby-osdi06.pdf
[runtime reconfiguration]: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/runtime-configuration.md
[benchmark]: https://github.com/coreos/etcd/tree/master/tools/benchmark
[benchmark-result]: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/performance.md
[api-mvcc]: learning/api.md#revisions
[maintenance-compact]: op-guide/maintenance.md#history-compaction
[maintenance-defragment]: op-guide/maintenance.md#defragmentation
[maintenance-disarm]: ../etcdctl/README.md#alarm-disarm
[fio]: https://github.com/axboe/fio
[fio-blog-post]: https://www.ibm.com/blogs/bluemix/2019/04/using-fio-to-tell-whether-your-storage-is-fast-enough-for-etcd/

View File

@ -1,179 +0,0 @@
---
title: Libraries and tools
---
**Tools**
- [etcdctl](https://github.com/etcd-io/etcd/tree/master/etcdctl) - A command line client for etcd
- [etcd-backup](https://github.com/fanhattan/etcd-backup) - A powerful command line utility for dumping/restoring etcd - Supports v2
- [etcd-dump](https://npmjs.org/package/etcd-dump) - Command line utility for dumping/restoring etcd.
- [etcd-fs](https://github.com/xetorthio/etcd-fs) - FUSE filesystem for etcd
- [etcddir](https://github.com/rekby/etcddir) - Realtime sync etcd and local directory. Work with windows and linux.
- [etcd-browser](https://github.com/henszey/etcd-browser) - A web-based key/value editor for etcd using AngularJS
- [etcd-lock](https://github.com/datawisesystems/etcd-lock) - Master election & distributed r/w lock implementation using etcd - Supports v2
- [etcd-console](https://github.com/matishsiao/etcd-console) - A web-base key/value editor for etcd using PHP
- [etcd-viewer](https://github.com/nikfoundas/etcd-viewer) - An etcd key-value store editor/viewer written in Java
- [etcdtool](https://github.com/mickep76/etcdtool) - Export/Import/Edit etcd directory as JSON/YAML/TOML and Validate directory using JSON schema
- [etcd-rest](https://github.com/mickep76/etcd-rest) - Create generic REST API in Go using etcd as a backend with validation using JSON schema
- [etcdsh](https://github.com/kamilhark/etcdsh) - A command line client with support of command history and tab completion. Supports v2
- [etcdloadtest](https://github.com/sinsharat/etcdloadtest) - A command line load test client for etcd version 3.0 and above.
- [lucas](https://github.com/ringtail/lucas) - A web-based key-value viewer for kubernetes etcd3.0+ cluster.
**Go libraries**
- [etcd/clientv3](https://github.com/etcd-io/etcd/blob/master/clientv3) - the officially maintained Go client for v3
- [etcd/client](https://github.com/etcd-io/etcd/blob/master/client) - the officially maintained Go client for v2
- [go-etcd](https://github.com/coreos/go-etcd) - the deprecated official client. May be useful for older (<2.0.0) versions of etcd.
- [encWrapper](https://github.com/lumjjb/etcd/tree/enc_wrapper/clientwrap/encwrapper) - encWrapper is an encryption wrapper for the etcd client Keys API/KV.
**Java libraries**
- [coreos/jetcd](https://github.com/etcd-io/jetcd) - Supports v3
- [boonproject/etcd](https://github.com/boonproject/boon/blob/master/etcd/README.md) - Supports v2, Async/Sync and waits
- [justinsb/jetcd](https://github.com/justinsb/jetcd)
- [diwakergupta/jetcd](https://github.com/diwakergupta/jetcd) - Supports v2
- [jurmous/etcd4j](https://github.com/jurmous/etcd4j) - Supports v2, Async/Sync, waits and SSL
- [AdoHe/etcd4j](http://github.com/AdoHe/etcd4j) - Supports v2 (enhance for real production cluster)
- [cdancy/etcd-rest](https://github.com/cdancy/etcd-rest) - Uses jclouds to provide a complete implementation of v2 API.
**Scala libraries**
- [maciej/etcd-client](https://github.com/maciej/etcd-client) - Supports v2. Akka HTTP-based fully async client
- [eiipii/etcdhttpclient](https://bitbucket.org/eiipii/etcdhttpclient) - Supports v2. Async HTTP client based on Netty and Scala Futures.
**Perl libraries**
- [hexfusion/perl-net-etcd](https://github.com/hexfusion/perl-net-etcd) - Supports v3 grpc gateway HTTP API
- [robn/p5-etcd](https://github.com/robn/p5-etcd) - Supports v2
**Python libraries**
- [kragniz/python-etcd3](https://github.com/kragniz/python-etcd3) - Client for v3
- [jplana/python-etcd](https://github.com/jplana/python-etcd) - Supports v2
- [russellhaering/txetcd](https://github.com/russellhaering/txetcd) - a Twisted Python library
- [cholcombe973/autodock](https://github.com/cholcombe973/autodock) - A docker deployment automation tool
- [lisael/aioetcd](https://github.com/lisael/aioetcd) - (Python 3.4+) Asyncio coroutines client (Supports v2)
- [txaio-etcd](https://github.com/crossbario/txaio-etcd) - Asynchronous etcd v3-only client library for Twisted (today) and asyncio (future)
- [dims/etcd3-gateway](https://github.com/dims/etcd3-gateway) - etcd v3 API library using the HTTP grpc gateway
- [aioetcd3](https://github.com/gaopeiliang/aioetcd3) - (Python 3.6+) etcd v3 API for asyncio
- [Revolution1/etcd3-py](https://github.com/Revolution1/etcd3-py) - (python2.7 and python3.5+) Python client for etcd v3, using gRPC-JSON-Gateway
**Node libraries**
- [mixer/etcd3](https://github.com/mixer/etcd3) - Supports v3
- [stianeikeland/node-etcd](https://github.com/stianeikeland/node-etcd) - Supports v2 (w Coffeescript)
- [lavagetto/nodejs-etcd](https://github.com/lavagetto/nodejs-etcd) - Supports v2
- [deedubs/node-etcd-config](https://github.com/deedubs/node-etcd-config) - Supports v2
**Ruby libraries**
- [iconara/etcd-rb](https://github.com/iconara/etcd-rb)
- [jpfuentes2/etcd-ruby](https://github.com/jpfuentes2/etcd-ruby)
- [ranjib/etcd-ruby](https://github.com/ranjib/etcd-ruby) - Supports v2
- [davissp14/etcdv3-ruby](https://github.com/davissp14/etcdv3-ruby) - Supports v3
**C libraries**
- [apache/celix/etcdlib](https://github.com/apache/celix/tree/develop/etcdlib) - Supports v2
- [jdarcy/etcd-api](https://github.com/jdarcy/etcd-api) - Supports v2
- [shafreeck/cetcd](https://github.com/shafreeck/cetcd) - Supports v2
**C++ libraries**
- [edwardcapriolo/etcdcpp](https://github.com/edwardcapriolo/etcdcpp) - Supports v2
- [suryanathan/etcdcpp](https://github.com/suryanathan/etcdcpp) - Supports v2 (with waits)
- [nokia/etcd-cpp-api](https://github.com/nokia/etcd-cpp-api) - Supports v2
- [nokia/etcd-cpp-apiv3](https://github.com/nokia/etcd-cpp-apiv3) - Supports v3
**Clojure libraries**
- [aterreno/etcd-clojure](https://github.com/aterreno/etcd-clojure)
- [dwwoelfel/cetcd](https://github.com/dwwoelfel/cetcd) - Supports v2
- [rthomas/clj-etcd](https://github.com/rthomas/clj-etcd) - Supports v2
**Erlang libraries**
- [marshall-lee/etcd.erl](https://github.com/marshall-lee/etcd.erl) - Supports v2
- [zhongwencool/eetcd](https://github.com/zhongwencool/eetcd) - Supports v3+ (GRPC only)
**.Net Libraries**
- [wangjia184/etcdnet](https://github.com/wangjia184/etcdnet) - Supports v2
- [drusellers/etcetera](https://github.com/drusellers/etcetera)
- [shubhamranjan/dotnet-etcd](https://github.com/shubhamranjan/dotnet-etcd) - Supports v3+ (GRPC only)
**PHP Libraries**
- [linkorb/etcd-php](https://github.com/linkorb/etcd-php)
- [activecollab/etcd](https://github.com/activecollab/etcd)
- [ouqiang/etcd-php](https://github.com/ouqiang/etcd-php) - Client for v3 gRPC gateway
**Haskell libraries**
- [wereHamster/etcd-hs](https://github.com/wereHamster/etcd-hs)
**R libraries**
- [ropensci/etseed](https://github.com/ropensci/etseed)
**Nim libraries**
- [etcd_client](https://github.com/FedericoCeratto/nim-etcd-client)
**Tcl libraries**
- [efrecon/etcd-tcl](https://github.com/efrecon/etcd-tcl) - Supports v2, except wait.
**Rust libraries**
- [jimmycuadra/rust-etcd](https://github.com/jimmycuadra/rust-etcd) - Supports v2
**Gradle Plugins**
- [gradle-etcd-rest-plugin](https://github.com/cdancy/gradle-etcd-rest-plugin) - Supports v2
**Chef Integration**
- [coderanger/etcd-chef](https://github.com/coderanger/etcd-chef)
**Chef Cookbook**
- [spheromak/etcd-cookbook](https://github.com/spheromak/etcd-cookbook)
**BOSH Releases**
- [cloudfoundry-community/etcd-boshrelease](https://github.com/cloudfoundry-community/etcd-boshrelease)
- [cloudfoundry/cf-release](https://github.com/cloudfoundry/cf-release/tree/master/jobs/etcd)
**Projects using etcd**
- [etcd Raft users](../raft/README.md#notable-users) - projects using etcd's raft library implementation.
- [apache/celix](https://github.com/apache/celix) - an implementation of the OSGi specification adapted to C and C++
- [binocarlos/yoda](https://github.com/binocarlos/yoda) - etcd + ZeroMQ
- [blox/blox](https://github.com/blox/blox) - a collection of open source projects for container management and orchestration with AWS ECS
- [calavera/active-proxy](https://github.com/calavera/active-proxy) - HTTP Proxy configured with etcd
- [chain/chain](https://github.com/chain/chain) - software designed to operate and connect to highly scalable permissioned blockchain networks
- [derekchiang/etcdplus](https://github.com/derekchiang/etcdplus) - A set of distributed synchronization primitives built upon etcd
- [go-discover](https://github.com/flynn/go-discover) - service discovery in Go
- [gleicon/goreman](https://github.com/gleicon/goreman/tree/etcd) - Branch of the Go Foreman clone with etcd support
- [garethr/hiera-etcd](https://github.com/garethr/hiera-etcd) - Puppet hiera backend using etcd
- [mattn/etcd-vim](https://github.com/mattn/etcd-vim) - SET and GET keys from inside vim
- [mattn/etcdenv](https://github.com/mattn/etcdenv) - "env" shebang with etcd integration
- [kelseyhightower/confd](https://github.com/kelseyhightower/confd) - Manage local app config files using templates and data from etcd
- [configdb](https://git.autistici.org/ai/configdb/tree/master) - A REST relational abstraction on top of arbitrary database backends, aimed at storing configs and inventories.
- [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) - Container cluster manager introduced by Google.
- [mailgun/vulcand](https://github.com/mailgun/vulcand) - HTTP proxy that uses etcd as a configuration backend.
- [duedil-ltd/discodns](https://github.com/duedil-ltd/discodns) - Simple DNS nameserver using etcd as a database for names and records.
- [skynetservices/skydns](https://github.com/skynetservices/skydns) - RFC compliant DNS server
- [xordataexchange/crypt](https://github.com/xordataexchange/crypt) - Securely store values in etcd using GPG encryption
- [spf13/viper](https://github.com/spf13/viper) - Go configuration library, reads values from ENV, pflags, files, and etcd with optional encryption
- [lytics/metafora](https://github.com/lytics/metafora) - Go distributed task library
- [ryandoyle/nss-etcd](https://github.com/ryandoyle/nss-etcd) - A GNU libc NSS module for resolving names from etcd.
- [Gru](https://github.com/dnaeon/gru) - Orchestration made easy with Go
- [Vitess](http://vitess.io/) - Vitess is a database clustering system for horizontal scaling of MySQL.
- [lclarkmichalek/etcdhcp](https://github.com/lclarkmichalek/etcdhcp) - DHCP server that uses etcd for persistence and coordination.
- [openstack/networking-vpp](https://github.com/openstack/networking-vpp) - A networking driver that programs the [FD.io VPP dataplane](https://wiki.fd.io/view/VPP) to provide [OpenStack](https://www.openstack.org/) cloud virtual networking
- [OpenStack](https://github.com/openstack/governance/blob/master/reference/base-services.rst) - OpenStack services can rely on etcd as a base service.
- [CoreDNS](https://github.com/coredns/coredns/tree/master/plugin/etcd) - CoreDNS is a DNS server that chains plugins, part of CNCF and Kubernetes
- [Uber M3](https://github.com/m3db/m3) - M3: Ubers Open Source, Large-scale Metrics Platform for Prometheus
- [Rook](https://github.com/rook/rook) - Storage Orchestration for Kubernetes
- [Patroni](https://github.com/zalando/patroni) - A template for PostgreSQL High Availability with ZooKeeper, etcd, or Consul
- [Trillian](https://github.com/google/trillian) - Trillian implements a Merkle tree whose contents are served from a data storage layer, to allow scalability to extremely large trees.

View File

@ -1,483 +0,0 @@
---
title: etcd3 API
---
This document is meant to give an overview of the etcd3 API's central design. It is by no means all encompassing, but intended to focus on the basic ideas needed to understand etcd without the distraction of less common API calls. All etcd3 API's are defined in [gRPC services][grpc-service], which categorize remote procedure calls (RPCs) understood by the etcd server. A full listing of all etcd RPCs are documented in markdown in the [gRPC API listing][grpc-api].
## gRPC Services
Every API request sent to an etcd server is a gRPC remote procedure call. RPCs in etcd3 are categorized based on functionality into services.
Services important for dealing with etcd's key space include:
* KV - Creates, updates, fetches, and deletes key-value pairs.
* Watch - Monitors changes to keys.
* Lease - Primitives for consuming client keep-alive messages.
Services which manage the cluster itself include:
* Auth - Role based authentication mechanism for authenticating users.
* Cluster - Provides membership information and configuration facilities.
* Maintenance - Takes recovery snapshots, defragments the store, and returns per-member status information.
### Requests and Responses
All RPCs in etcd3 follow the same format. Each RPC has a function `Name` which takes `NameRequest` as an argument and returns `NameResponse` as a response. For example, here is the `Range` RPC description:
```protobuf
service KV {
Range(RangeRequest) returns (RangeResponse)
...
}
```
### Response header
All Responses from etcd API have an attached response header which includes cluster metadata for the response:
```proto
message ResponseHeader {
uint64 cluster_id = 1;
uint64 member_id = 2;
int64 revision = 3;
uint64 raft_term = 4;
}
```
* Cluster_ID - the ID of the cluster generating the response.
* Member_ID - the ID of the member generating the response.
* Revision - the revision of the key-value store when generating the response.
* Raft_Term - the Raft term of the member when generating the response.
An application may read the `Cluster_ID` or `Member_ID` field to ensure it is communicating with the intended cluster (member).
Applications can use the `Revision` field to know the latest revision of the key-value store. This is especially useful when applications specify a historical revision to make a `time travel query` and wish to know the latest revision at the time of the request.
Applications can use `Raft_Term` to detect when the cluster completes a new leader election.
## Key-Value API
The Key-Value API manipulates key-value pairs stored inside etcd. The majority of requests made to etcd are usually key-value requests.
### System primitives
### Key-Value pair
A key-value pair is the smallest unit that the key-value API can manipulate. Each key-value pair has a number of fields, defined in [protobuf format][kv-proto]:
```protobuf
message KeyValue {
bytes key = 1;
int64 create_revision = 2;
int64 mod_revision = 3;
int64 version = 4;
bytes value = 5;
int64 lease = 6;
}
```
* Key - key in bytes. An empty key is not allowed.
* Value - value in bytes.
* Version - version is the version of the key. A deletion resets the version to zero and any modification of the key increases its version.
* Create_Revision - revision of the last creation on the key.
* Mod_Revision - revision of the last modification on the key.
* Lease - the ID of the lease attached to the key. If lease is 0, then no lease is attached to the key.
In addition to just the key and value, etcd attaches additional revision metadata as part of the key message. This revision information orders keys by time of creation and modification, which is useful for managing concurrency for distributed synchronization. The etcd client's [distributed shared locks][locks] use the creation revision to wait for lock ownership. Similarly, the modification revision is used for detecting [software transactional memory][STM] read set conflicts and waiting on [leader election][elections] updates.
#### Revisions
etcd maintains a 64-bit cluster-wide counter, the store revision, that is incremented each time the key space is modified. The revision serves as a global logical clock, sequentially ordering all updates to the store. The change represented by a new revision is incremental; the data associated with a revision is the data that changed the store. Internally, a new revision means writing the changes to the backend's B+tree, keyed by the incremented revision.
Revisions become more valuable when considering etcd3's [multi-version concurrency control][mvcc] backend. The MVCC model means that the key-value store can be viewed from past revisions since historical key revisions are retained. The retention policy for this history can be configured by cluster administrators for fine-grained storage management; usually etcd3 discards old revisions of keys on a timer. A typical etcd3 cluster retains superseded key data for hours. This also provides reliable handling for long client disconnection, not just transient network disruptions: watchers simply resume from the last observed historical revision. Similarly, to read from the store at a particular point-in-time, read requests can be tagged with a revision to return keys from a view of the key space at the point-in-time that revision was committed.
#### Key ranges
The etcd3 data model indexes all keys over a flat binary key space. This differs from other key-value store systems that use a hierarchical system of organizing keys into directories. Instead of listing keys by directory, keys are listed by key intervals `[a, b)`.
These intervals are often referred to as "ranges" in etcd3. Operations over ranges are more powerful than operations on directories. Like a hierarchical store, intervals support single key lookups via `[a, a+1)` (e.g., ['a', 'a\x00') looks up 'a') and directory lookups by encoding keys by directory depth. In addition to those operations, intervals can also encode prefixes; for example the interval `['a', 'b')` looks up all keys prefixed by the string 'a'.
By convention, ranges for a request are denoted by the fields `key` and `range_end`. The `key` field is the first key of the range and should be non-empty. The `range_end` is the key following the last key of the range. If `range_end` is not given or empty, the range is defined to contain only the key argument. If `range_end` is `key` plus one (e.g., "aa"+1 == "ab", "a\xff"+1 == "b"), then the range represents all keys prefixed with key. If both `key` and `range_end` are '\0', then range represents all keys. If `range_end` is '\0', the range is all keys greater than or equal to the key argument.
### Range
Keys are fetched from the key-value store using the `Range` API call, which takes a `RangeRequest`:
```protobuf
message RangeRequest {
enum SortOrder {
NONE = 0; // default, no sorting
ASCEND = 1; // lowest target value first
DESCEND = 2; // highest target value first
}
enum SortTarget {
KEY = 0;
VERSION = 1;
CREATE = 2;
MOD = 3;
VALUE = 4;
}
bytes key = 1;
bytes range_end = 2;
int64 limit = 3;
int64 revision = 4;
SortOrder sort_order = 5;
SortTarget sort_target = 6;
bool serializable = 7;
bool keys_only = 8;
bool count_only = 9;
int64 min_mod_revision = 10;
int64 max_mod_revision = 11;
int64 min_create_revision = 12;
int64 max_create_revision = 13;
}
```
* Key, Range_End - The key range to fetch.
* Limit - the maximum number of keys returned for the request. When limit is set to 0, it is treated as no limit.
* Revision - the point-in-time of the key-value store to use for the range. If revision is less or equal to zero, the range is over the latest key-value store. If the revision is compacted, ErrCompacted is returned as a response.
* Sort_Order - the ordering for sorted requests.
* Sort_Target - the key-value field to sort.
* Serializable - sets the range request to use serializable member-local reads. By default, Range is linearizable; it reflects the current consensus of the cluster. For better performance and availability, in exchange for possible stale reads, a serializable range request is served locally without needing to reach consensus with other nodes in the cluster.
* Keys_Only - return only the keys and not the values.
* Count_Only - return only the count of the keys in the range.
* Min_Mod_Revision - the lower bound for key mod revisions; filters out lesser mod revisions.
* Max_Mod_Revision - the upper bound for key mod revisions; filters out greater mod revisions.
* Min_Create_Revision - the lower bound for key create revisions; filters out lesser create revisions.
* Max_Create_Revision - the upper bound for key create revisions; filters out greater create revisions.
The client receives a `RangeResponse` message from the `Range` call:
```protobuf
message RangeResponse {
ResponseHeader header = 1;
repeated mvccpb.KeyValue kvs = 2;
bool more = 3;
int64 count = 4;
}
```
* Kvs - the list of key-value pairs matched by the range request. When `Count_Only` is set, `Kvs` is empty.
* More - indicates if there are more keys to return in the requested range if `limit` is set.
* Count - the total number of keys satisfying the range request.
### Put
Keys are saved into the key-value store by issuing a `Put` call, which takes a `PutRequest`:
```protobuf
message PutRequest {
bytes key = 1;
bytes value = 2;
int64 lease = 3;
bool prev_kv = 4;
bool ignore_value = 5;
bool ignore_lease = 6;
}
```
* Key - the name of the key to put into the key-value store.
* Value - the value, in bytes, to associate with the key in the key-value store.
* Lease - the lease ID to associate with the key in the key-value store. A lease value of 0 indicates no lease.
* Prev_Kv - when set, responds with the key-value pair data before the update from this `Put` request.
* Ignore_Value - when set, update the key without changing its current value. Returns an error if the key does not exist.
* Ignore_Lease - when set, update the key without changing its current lease. Returns an error if the key does not exist.
The client receives a `PutResponse` message from the `Put` call:
```protobuf
message PutResponse {
ResponseHeader header = 1;
mvccpb.KeyValue prev_kv = 2;
}
```
* Prev_Kv - the key-value pair overwritten by the `Put`, if `Prev_Kv` was set in the `PutRequest`.
### Delete Range
Ranges of keys are deleted using the `DeleteRange` call, which takes a `DeleteRangeRequest`:
```protobuf
message DeleteRangeRequest {
bytes key = 1;
bytes range_end = 2;
bool prev_kv = 3;
}
```
* Key, Range_End - The key range to delete.
* Prev_Kv - when set, return the contents of the deleted key-value pairs.
The client receives a `DeleteRangeResponse` message from the `DeleteRange` call:
```protobuf
message DeleteRangeResponse {
ResponseHeader header = 1;
int64 deleted = 2;
repeated mvccpb.KeyValue prev_kvs = 3;
}
```
* Deleted - number of keys deleted.
* Prev_Kv - a list of all key-value pairs deleted by the `DeleteRange` operation.
### Transaction
A transaction is an atomic If/Then/Else construct over the key-value store. It provides a primitive for grouping requests together in atomic blocks (i.e., then/else) whose execution is guarded (i.e., if) based on the contents of the key-value store. Transactions can be used for protecting keys from unintended concurrent updates, building compare-and-swap operations, and developing higher-level concurrency control.
A transaction can atomically process multiple requests in a single request. For modifications to the key-value store, this means the store's revision is incremented only once for the transaction and all events generated by the transaction will have the same revision. However, modifications to the same key multiple times within a single transaction are forbidden.
All transactions are guarded by a conjunction of comparisons, similar to an `If` statement. Each comparison checks a single key in the store. It may check for the absence or presence of a value, compare with a given value, or check a key's revision or version. Two different comparisons may apply to the same or different keys. All comparisons are applied atomically; if all comparisons are true, the transaction is said to succeed and etcd applies the transaction's then / `success` request block, otherwise it is said to fail and applies the else / `failure` request block.
Each comparison is encoded as a `Compare` message:
```protobuf
message Compare {
enum CompareResult {
EQUAL = 0;
GREATER = 1;
LESS = 2;
NOT_EQUAL = 3;
}
enum CompareTarget {
VERSION = 0;
CREATE = 1;
MOD = 2;
VALUE= 3;
}
CompareResult result = 1;
// target is the key-value field to inspect for the comparison.
CompareTarget target = 2;
// key is the subject key for the comparison operation.
bytes key = 3;
oneof target_union {
int64 version = 4;
int64 create_revision = 5;
int64 mod_revision = 6;
bytes value = 7;
}
}
```
* Result - the kind of logical comparison operation (e.g., equal, less than, etc).
* Target - the key-value field to be compared. Either the key's version, create revision, modification revision, or value.
* Key - the key for the comparison.
* Target_Union - the user-specified data for the comparison.
After processing the comparison block, the transaction applies a block of requests. A block is a list of `RequestOp` messages:
```protobuf
message RequestOp {
// request is a union of request types accepted by a transaction.
oneof request {
RangeRequest request_range = 1;
PutRequest request_put = 2;
DeleteRangeRequest request_delete_range = 3;
}
}
```
* Request_Range - a `RangeRequest`.
* Request_Put - a `PutRequest`. The keys must be unique. It may not share keys with any other Puts or Deletes.
* Request_Delete_Range - a `DeleteRangeRequest`. It may not share keys with any Puts or Deletes requests.
All together, a transaction is issued with a `Txn` API call, which takes a `TxnRequest`:
```protobuf
message TxnRequest {
repeated Compare compare = 1;
repeated RequestOp success = 2;
repeated RequestOp failure = 3;
}
```
* Compare - A list of predicates representing a conjunction of terms for guarding the transaction.
* Success - A list of requests to process if all compare tests evaluate to true.
* Failure - A list of requests to process if any compare test evaluates to false.
The client receives a `TxnResponse` message from the `Txn` call:
```protobuf
message TxnResponse {
ResponseHeader header = 1;
bool succeeded = 2;
repeated ResponseOp responses = 3;
}
```
* Succeeded - Whether `Compare` evaluated to true or false.
* Responses - A list of responses corresponding to the results from applying the `Success` block if succeeded is true or the `Failure` if succeeded is false.
The `Responses` list corresponds to the results from the applied `RequestOp` list, with each response encoded as a `ResponseOp`:
```protobuf
message ResponseOp {
oneof response {
RangeResponse response_range = 1;
PutResponse response_put = 2;
DeleteRangeResponse response_delete_range = 3;
}
}
```
## Watch API
The `Watch` API provides an event-based interface for asynchronously monitoring changes to keys. An etcd3 watch waits for changes to keys by continuously watching from a given revision, either current or historical, and streams key updates back to the client.
### Events
Every change to every key is represented with `Event` messages. An `Event` message provides both the update's data and the type of update:
```protobuf
message Event {
enum EventType {
PUT = 0;
DELETE = 1;
}
EventType type = 1;
KeyValue kv = 2;
KeyValue prev_kv = 3;
}
```
* Type - The kind of event. A PUT type indicates new data has been stored to the key. A DELETE indicates the key was deleted.
* KV - The KeyValue associated with the event. A PUT event contains current kv pair. A PUT event with kv.Version=1 indicates the creation of a key. A DELETE event contains the deleted key with its modification revision set to the revision of deletion.
* Prev_KV - The key-value pair for the key from the revision immediately before the event. To save bandwidth, it is only filled out if the watch has explicitly enabled it.
### Watch streams
Watches are long-running requests and use gRPC streams to stream event data. A watch stream is bi-directional; the client writes to the stream to establish watches and reads to receive watch events. A single watch stream can multiplex many distinct watches by tagging events with per-watch identifiers. This multiplexing helps reducing the memory footprint and connection overhead on the core etcd cluster.
Watches make three guarantees about events:
* Ordered - events are ordered by revision; an event will never appear on a watch if it precedes an event in time that has already been posted.
* Reliable - a sequence of events will never drop any subsequence of events; if there are events ordered in time as a < b < c, then if the watch receives events a and c, it is guaranteed to receive b.
* Atomic - a list of events is guaranteed to encompass complete revisions; updates in the same revision over multiple keys will not be split over several lists of events.
A client creates a watch by sending a `WatchCreateRequest` over a stream returned by `Watch`:
```protobuf
message WatchCreateRequest {
bytes key = 1;
bytes range_end = 2;
int64 start_revision = 3;
bool progress_notify = 4;
enum FilterType {
NOPUT = 0;
NODELETE = 1;
}
repeated FilterType filters = 5;
bool prev_kv = 6;
}
```
* Key, Range_End - The key range to watch.
* Start_Revision - An optional revision for where to inclusively begin watching. If not given, it will stream events following the revision of the watch creation response header revision. The entire available event history can be watched starting from the last compaction revision.
* Progress_Notify - When set, the watch will periodically receive a WatchResponse with no events, if there are no recent events. It is useful when clients wish to recover a disconnected watcher starting from a recent known revision. The etcd server decides how often to send notifications based on current server load.
* Filters - A list of event types to filter away at server side.
* Prev_Kv - When set, the watch receives the key-value data from before the event happens. This is useful for knowing what data has been overwritten.
In response to a `WatchCreateRequest` or if there is a new event for some established watch, the client receives a `WatchResponse`:
```protobuf
message WatchResponse {
ResponseHeader header = 1;
int64 watch_id = 2;
bool created = 3;
bool canceled = 4;
int64 compact_revision = 5;
repeated mvccpb.Event events = 11;
}
```
* Watch_ID - the ID of the watch that corresponds to the response.
* Created - set to true if the response is for a create watch request. The client should store the ID and expect to receive events for the watch on the stream. All events sent to the created watcher will have the same watch_id.
* Canceled - set to true if the response is for a cancel watch request. No further events will be sent to the canceled watcher.
* Compact_Revision - set to the minimum historical revision available to etcd if a watcher tries watching at a compacted revision. This happens when creating a watcher at a compacted revision or the watcher cannot catch up with the progress of the key-value store. The watcher will be canceled; creating new watches with the same start_revision will fail.
* Events - a list of new events in sequence corresponding to the given watch ID.
If the client wishes to stop receiving events for a watch, it issues a `WatchCancelRequest`:
```protobuf
message WatchCancelRequest {
int64 watch_id = 1;
}
```
* Watch_ID - the ID of the watch to cancel so that no more events are transmitted.
## Lease API
Leases are a mechanism for detecting client liveness. The cluster grants leases with a time-to-live. A lease expires if the etcd cluster does not receive a keepAlive within a given TTL period.
To tie leases into the key-value store, each key may be attached to at most one lease. When a lease expires or is revoked, all keys attached to that lease will be deleted. Each expired key generates a delete event in the event history.
### Obtaining leases
Leases are obtained through the `LeaseGrant` API call, which takes a `LeaseGrantRequest`:
```protobuf
message LeaseGrantRequest {
int64 TTL = 1;
int64 ID = 2;
}
```
* TTL - the advisory time-to-live, in seconds.
* ID - the requested ID for the lease. If ID is set to 0, etcd will choose an ID.
The client receives a `LeaseGrantResponse` from the `LeaseGrant` call:
```protobuf
message LeaseGrantResponse {
ResponseHeader header = 1;
int64 ID = 2;
int64 TTL = 3;
}
```
* ID - the lease ID for the granted lease.
* TTL - is the server selected time-to-live, in seconds, for the lease.
```protobuf
message LeaseRevokeRequest {
int64 ID = 1;
}
```
* ID - the lease ID to revoke. When the lease is revoked, all attached keys are deleted.
### Keep alives
Leases are refreshed using a bi-directional stream created with the `LeaseKeepAlive` API call. When the client wishes to refresh a lease, it sends a `LeaseKeepAliveRequest` over the stream:
```protobuf
message LeaseKeepAliveRequest {
int64 ID = 1;
}
```
* ID - the lease ID for the lease to keep alive.
The keep alive stream responds with a `LeaseKeepAliveResponse`:
```protobuf
message LeaseKeepAliveResponse {
ResponseHeader header = 1;
int64 ID = 2;
int64 TTL = 3;
}
```
* ID - the lease that was refreshed with a new TTL.
* TTL - the new time-to-live, in seconds, that the lease has remaining.
[elections]: https://github.com/etcd-io/etcd/blob/master/clientv3/concurrency/election.go
[kv-proto]: https://github.com/etcd-io/etcd/blob/master/mvcc/mvccpb/kv.proto
[grpc-api]: ../dev-guide/api_reference_v3.md
[grpc-service]: https://github.com/etcd-io/etcd/blob/master/etcdserver/etcdserverpb/rpc.proto
[locks]: https://github.com/etcd-io/etcd/blob/master/clientv3/concurrency/mutex.go
[mvcc]: https://en.wikipedia.org/wiki/Multiversion_concurrency_control
[stm]: https://github.com/etcd-io/etcd/blob/master/clientv3/concurrency/stm.go

View File

@ -1,66 +0,0 @@
---
title: KV API guarantees
---
etcd is a consistent and durable key value store with [mini-transaction][txn] support. The key value store is exposed through the KV APIs. etcd tries to ensure the strongest consistency and durability guarantees for a distributed system. This specification enumerates the KV API guarantees made by etcd.
### APIs to consider
* Read APIs
* range
* watch
* Write APIs
* put
* delete
* Combination (read-modify-write) APIs
* txn
### etcd specific definitions
#### Operation completed
An etcd operation is considered complete when it is committed through consensus, and therefore “executed” -- permanently stored -- by the etcd storage engine. The client knows an operation is completed when it receives a response from the etcd server. Note that the client may be uncertain about the status of an operation if it times out, or there is a network disruption between the client and the etcd member. etcd may also abort operations when there is a leader election. etcd does not send `abort` responses to clients outstanding requests in this event.
#### Revision
An etcd operation that modifies the key value store is assigned a single increasing revision. A transaction operation might modify the key value store multiple times, but only one revision is assigned. The revision attribute of a key value pair that was modified by the operation has the same value as the revision of the operation. The revision can be used as a logical clock for key value store. A key value pair that has a larger revision is modified after a key value pair with a smaller revision. Two key value pairs that have the same revision are modified by an operation "concurrently".
### Guarantees provided
#### Atomicity
All API requests are atomic; an operation either completes entirely or not at all. For watch requests, all events generated by one operation will be in one watch response. Watch never observes partial events for a single operation.
#### Consistency
All API calls ensure [sequential consistency][seq_consistency], the strongest consistency guarantee available from distributed systems. No matter which etcd member server a client makes requests to, a client reads the same events in the same order. If two members complete the same number of operations, the state of the two members is consistent.
For watch operations, etcd guarantees to return the same value for the same key across all members for the same revision. For range operations, etcd has a similar guarantee for [linearized][Linearizability] access; serialized access may be behind the quorum state, so that the later revision is not yet available.
As with all distributed systems, it is impossible for etcd to ensure [strict consistency][strict_consistency]. etcd does not guarantee that it will return to a read the “most recent” value (as measured by a wall clock when a request is completed) available on any cluster member.
#### Isolation
etcd ensures [serializable isolation][serializable_isolation], which is the highest isolation level available in distributed systems. Read operations will never observe any intermediate data.
#### Durability
Any completed operations are durable. All accessible data is also durable data. A read will never return data that has not been made durable.
#### Linearizability
Linearizability (also known as Atomic Consistency or External Consistency) is a consistency level between strict consistency and sequential consistency.
For linearizability, suppose each operation receives a timestamp from a loosely synchronized global clock. Operations are linearized if and only if they always complete as though they were executed in a sequential order and each operation appears to complete in the order specified by the program. Likewise, if an operations timestamp precedes another, that operation must also precede the other operation in the sequence.
For example, consider a client completing a write at time point 1 (*t1*). A client issuing a read at *t2* (for *t2* > *t1*) should receive a value at least as recent as the previous write, completed at *t1*. However, the read might actually complete only by *t3*. Linearizability guarantees the read returns the most current value. Without linearizability guarantee, the returned value, current at *t2* when the read began, might be "stale" by *t3* because a concurrent write might happen between *t2* and *t3*.
etcd does not ensure linearizability for watch operations. Users are expected to verify the revision of watch responses to ensure correct ordering.
etcd ensures linearizability for all other operations by default. Linearizability comes with a cost, however, because linearized requests must go through the Raft consensus process. To obtain lower latencies and higher throughput for read requests, clients can configure a requests consistency mode to `serializable`, which may access stale data with respect to quorum, but removes the performance penalty of linearized accesses' reliance on live consensus.
[seq_consistency]: https://en.wikipedia.org/wiki/Consistency_model#Sequential_consistency
[strict_consistency]: https://en.wikipedia.org/wiki/Consistency_model#Strict_consistency
[serializable_isolation]: https://en.wikipedia.org/wiki/Isolation_(database_systems)#Serializable
[Linearizability]: #Linearizability
[txn]: api.md#transactions

View File

@ -1,27 +0,0 @@
---
title: Data model
---
etcd is designed to reliably store infrequently updated data and provide reliable watch queries. etcd exposes previous versions of key-value pairs to support inexpensive snapshots and watch history events (“time travel queries”). A persistent, multi-version, concurrency-control data model is a good fit for these use cases.
etcd stores data in a multiversion [persistent][persistent-ds] key-value store. The persistent key-value store preserves the previous version of a key-value pair when its value is superseded with new data. The key-value store is effectively immutable; its operations do not update the structure in-place, but instead always generate a new updated structure. All past versions of keys are still accessible and watchable after modification. To prevent the data store from growing indefinitely over time and from maintaining old versions, the store may be compacted to shed the oldest versions of superseded data.
### Logical view
The stores logical view is a flat binary key space. The key space has a lexically sorted index on byte string keys so range queries are inexpensive.
The key space maintains multiple **revisions**. Each atomic mutative operation (e.g., a transaction operation may contain multiple operations) creates a new revision on the key space. All data held by previous revisions remains unchanged. Old versions of key can still be accessed through previous revisions. Likewise, revisions are indexed as well; ranging over revisions with watchers is efficient. If the store is compacted to save space, revisions before the compact revision will be removed. Revisions are monotonically increasing over the lifetime of a cluster.
A key's life spans a generation, from creation to deletion. Each key may have one or multiple generations. Creating a key increments the **version** of that key, starting at 1 if the key does not exist at the current revision. Deleting a key generates a key tombstone, concluding the keys current generation by resetting its version to 0. Each modification of a key increments its version; so, versions are monotonically increasing within a key's generation. Once a compaction happens, any generation ended before the compaction revision will be removed, and values set before the compaction revision except the latest one will be removed.
### Physical view
etcd stores the physical data as key-value pairs in a persistent [b+tree][b+tree]. Each revision of the stores state only contains the delta from its previous revision to be efficient. A single revision may correspond to multiple keys in the tree.
The key of key-value pair is a 3-tuple (major, sub, type). Major is the store revision holding the key. Sub differentiates among keys within the same revision. Type is an optional suffix for special value (e.g., `t` if the value contains a tombstone). The value of the key-value pair contains the modification from previous revision, thus one delta from previous revision. The b+tree is ordered by key in lexical byte-order. Ranged lookups over revision deltas are fast; this enables quickly finding modifications from one specific revision to another. Compaction removes out-of-date keys-value pairs.
etcd also keeps a secondary in-memory [btree][btree] index to speed up range queries over keys. The keys in the btree index are the keys of the store exposed to user. The value is a pointer to the modification of the persistent b+tree. Compaction removes dead pointers.
[persistent-ds]: https://en.wikipedia.org/wiki/Persistent_data_structure
[btree]: https://en.wikipedia.org/wiki/B-tree
[b+tree]: https://en.wikipedia.org/wiki/B%2B_tree

View File

@ -1,79 +0,0 @@
---
title: etcd v3 authentication design
---
## Why not reuse the v2 auth system?
The v3 protocol uses gRPC as its transport instead of a RESTful interface like v2. This new protocol provides an opportunity to iterate on and improve the v2 design. For example, v3 auth has connection based authentication, rather than v2's slower per-request authentication. Additionally, v2 auth's semantics tend to be unwieldy in practice with respect to reasoning about consistency, which will be described in the next sections. For v3, there is a well-defined description and implementation of the authentication mechanism which fixes the deficiencies in the v2 auth system.
### Functionality requirements
* Per connection authentication, not per request
* User ID + password based authentication implemented for the gRPC API
* Authentication must be refreshed after auth policy changes
* Its functionality should be as simple and useful as v2
* v3 provides a flat key space, unlike the directory structure of v2. Permission checking will be provided as interval matching.
* It should have stronger consistency guarantees than v2 auth
### Main required changes
* A client must create a dedicated connection only for authentication before sending authenticated requests
* Add permission information (user ID and authorized revision) to the Raft commands (`etcdserverpb.InternalRaftRequest`)
* Every request is permission checked in the state machine layer, rather than API layer
### Permission metadata consistency
The metadata for auth should also be stored and managed in the storage controlled by etcd's Raft protocol like other data stored in etcd. It is required for not sacrificing availability and consistency of the entire etcd cluster. If reading or writing the metadata (e.g. permission information) needs an agreement of every node (more than quorum), single node failure can stop the entire cluster. Requiring all nodes to agree at once means that checking ordinary read/write requests cannot be completed if any cluster member is down, even if the cluster has an available quorum. This unanimous scheme ultimately degrades cluster availability; quorum based consensus from raft should suffice since agreement follows from consistent ordering.
The authentication mechanism in the etcd v2 protocol has a tricky part because the metadata consistency should work as in the above, but does not: each permission check is processed by the etcd member that receives the client request (etcdserver/api/v2http/client.go), including follower members. Therefore, it's possible the check may be based on stale metadata.
This staleness means that auth configuration cannot be reflected as soon as operators execute etcdctl. Therefore there is no way to know how long the stale metadata is active. Practically, the configuration change is reflected immediately after the command execution. However, in some cases of heavy load, the inconsistent state can be prolonged and it might result in counter-intuitive situations for users and developers. It requires a workaround like this: https://github.com/etcd-io/etcd/pull/4317#issuecomment-179037582
### Inconsistent permissions are unsafe for linearized requests
Inconsistent authentication state is most serious for writes. Even if an operator disables write on a user, if the write is only ordered with respect to the key value store but not the authentication system, it's possible the write will complete successfully. Without ordering on both the auth store and the key-value store, the system will be susceptible to stale permission attacks.
Therefore, the permission checking logic should be added to the state machine of etcd. Each state machine should check the requests based on its permission information in the apply phase (so the auth information must not be stale).
## Design and implementation
### Authentication
At first, a client must create a gRPC connection only to authenticate its user ID and password. An etcd server will respond with an authentication reply. The response will be an authentication token on success or an error on failure. The client can use its authentication token to present its credentials to etcd when making API requests.
The client connection used to request the authentication token is typically thrown away; it cannot carry the new token's credentials. This is because gRPC doesn't provide a way for adding per RPC credential after creation of the connection (calling `grpc.Dial()`). Therefore, a client cannot assign a token to its connection that is obtained through the connection. The client needs a new connection for using the token.
#### Notes on the implementation of `Authenticate()` RPC
`Authenticate()` RPC generates an authentication token based on a given user name and password. etcd saves and checks a configured password and a given password using Go's `bcrypt` package. By design, `bcrypt`'s password checking mechanism is computationally expensive, taking nearly 100ms on an ordinary x64 server. Therefore, performing this check in the state machine apply phase would cause performance trouble: the entire etcd cluster can only serve almost 10 `Authenticate()` requests per second.
For good performance, the v3 auth mechanism checks passwords in etcd's API layer, where it can be parallelized outside of raft. However, this can lead to potential time-of-check/time-of-use (TOCTOU) permission lapses:
1. client A sends a request `Authenticate()`
1. the API layer processes the password checking part of `Authenticate()`
1. another client B sends a request of `ChangePassword()` and the server completes it
1. the state machine layer processes the part of getting a revision number for the `Authenticate()` from A
1. the server returns a success to A
1. now A is authenticated on an obsolete password
For avoiding such a situation, the API layer performs *version number validation* based on the revision number of the auth store. During password checking, the API layer saves the revision number of auth store. After successful password checking, the API layer compares the saved revision number and the latest revision number. If the numbers differ, it means someone else updated the auth metadata. So it retries the checking. With this mechanism, the successful password checking based on the obsolete password can be avoided.
### Resolving a token in the API layer
After authenticating with `Authenticate()`, a client can create a gRPC connection as it would without auth. In addition to the existing initialization process, the client must associate the token with the newly created connection. `grpc.WithPerRPCCredentials()` provides the functionality for this purpose.
Every authenticated request from the client has a token. The token can be obtained with `grpc.metadata.FromIncomingContext()` in the server side. The server can obtain who is issuing the request and when the user was authorized. The information will be filled by the API layer in the header (`etcdserverpb.RequestHeader.Username` and `etcdserverpb.RequestHeader.AuthRevision`) of a raft log entry (`etcdserverpb.InternalRaftRequest`).
### Checking permission in the state machine
The auth info in `etcdserverpb.RequestHeader` is checked in the apply phase of the state machine. This step checks the user is granted permission to requested keys on the latest revision of auth store.
### Two types of tokens: simple and JWT
There are two kinds of token types: simple and JWT. The simple token isn't designed for production use cases. Its tokens aren't cryptographically signed and servers must statefully track token-user correspondence; it is meant for development testing. JWT tokens should be used for production deployments since it is cryptographically signed and verified. From the implementation perspective, JWT is stateless. Its token can include metadata including username and revision, so servers don't need to remember correspondence between tokens and the metadata.
## Notes on the difference between KVS models and file system models
etcd v3 is a KVS, not a file system. So the permissions can be granted to the users in form of an exact key name or a key range like `["start key", "end key")`. It means that granting a permission of a nonexistent key is possible. Users should care about unintended permission granting. In a case of file system like system (e.g. Chubby or ZooKeeper), an inode like data structure can include the permission information. So granting permission to a nonexist key won't be possible (except the case of sticky bits).
The etcd v3 model requires multiple lookup of the metadata unlike the file system like systems. The worst case lookup cost will be sum the user's total granted keys and intervals. The cost cannot be avoided because v3's flat key space is completely different from Unix's file system model (every inode includes permission metadata). Practically the cost wont be a serious problem because the metadata is small enough to benefit from caching.

View File

@ -1,138 +0,0 @@
---
title: etcd client design
---
etcd Client Design
==================
*Gyuho Lee (github.com/gyuho, Amazon Web Services, Inc.), Joe Betz (github.com/jpbetz, Google Inc.)*
Introduction
============
etcd server has proven its robustness with years of failure injection testing. Most complex application logic is already handled by etcd server and its data stores (e.g. cluster membership is transparent to clients, with Raft-layer forwarding proposals to leader). Although server components are correct, its composition with client requires a different set of intricate protocols to guarantee its correctness and high availability under faulty conditions. Ideally, etcd server provides one logical cluster view of many physical machines, and client implements automatic failover between replicas. This documents client architectural decisions and its implementation details.
Glossary
========
*clientv3*: etcd Official Go client for etcd v3 API.
*clientv3-grpc1.0*: Official client implementation, with [`grpc-go v1.0.x`](https://github.com/grpc/grpc-go/releases/tag/v1.0.0), which is used in latest etcd v3.1.
*clientv3-grpc1.7*: Official client implementation, with [`grpc-go v1.7.x`](https://github.com/grpc/grpc-go/releases/tag/v1.7.0), which is used in latest etcd v3.2 and v3.3.
*clientv3-grpc1.23*: Official client implementation, with [`grpc-go v1.23.x`](https://github.com/grpc/grpc-go/releases/tag/v1.23.0), which is used in latest etcd v3.4.
*Balancer*: etcd client load balancer that implements retry and failover mechanism. etcd client should automatically balance loads between multiple endpoints.
*Endpoints*: A list of etcd server endpoints that clients can connect to. Typically, 3 or 5 client URLs of an etcd cluster.
*Pinned endpoint*: When configured with multiple endpoints, <= v3.3 client balancer chooses only one endpoint to establish a TCP connection, in order to conserve total open connections to etcd cluster. In v3.4, balancer round-robins pinned endpoints for every request, thus distributing loads more evenly.
*Client Connection*: TCP connection that has been established to an etcd server, via gRPC Dial.
*Sub Connection*: gRPC SubConn interface. Each sub-connection contains a list of addresses. Balancer creates a SubConn from a list of resolved addresses. gRPC ClientConn can map to multiple SubConn (e.g. example.com resolves to `10.10.10.1` and `10.10.10.2` of two sub-connections). etcd v3.4 balancer employs internal resolver to establish one sub-connection for each endpoint.
*Transient disconnect*: When gRPC server returns a status error of [`code Unavailable`](https://godoc.org/google.golang.org/grpc/codes#Code).
Client Requirements
===================
*Correctness*. Requests may fail in the presence of server faults. However, it never violates consistency guarantees: global ordering properties, never write corrupted data, at-most once semantics for mutable operations, watch never observes partial events, and so on.
*Liveness*. Servers may fail or disconnect briefly. Clients should make progress in either way. Clients should [never deadlock](https://github.com/etcd-io/etcd/issues/8980) waiting for a server to come back from offline, unless configured to do so. Ideally, clients detect unavailable servers with HTTP/2 ping and failover to other nodes with clear error messages.
*Effectiveness*. Clients should operate effectively with minimum resources: previous TCP connections should be [gracefully closed](https://github.com/etcd-io/etcd/issues/9212) after endpoint switch. Failover mechanism should effectively predict the next replica to connect, without wastefully retrying on failed nodes.
*Portability*. Official client should be clearly documented and its implementation be applicable to other language bindings. Error handling between different language bindings should be consistent. Since etcd is fully committed to gRPC, implementation should be closely aligned with gRPC long-term design goals (e.g. pluggable retry policy should be compatible with [gRPC retry](https://github.com/grpc/proposal/blob/master/A6-client-retries.md)). Upgrades between two client versions should be non-disruptive.
Client Overview
===============
etcd client implements the following components:
* balancer that establishes gRPC connections to an etcd cluster,
* API client that sends RPCs to an etcd server, and
* error handler that decides whether to retry a failed request or switch endpoints.
Languages may differ in how to establish an initial connection (e.g. configure TLS), how to encode and send Protocol Buffer messages to server, how to handle stream RPCs, and so on. However, errors returned from etcd server will be the same. So should be error handling and retry policy.
For example, etcd server may return `"rpc error: code = Unavailable desc = etcdserver: request timed out"`, which is transient error that expects retries. Or return `rpc error: code = InvalidArgument desc = etcdserver: key is not provided`, which means request was invalid and should not be retried. Go client can parse errors with `google.golang.org/grpc/status.FromError`, and Java client with `io.grpc.Status.fromThrowable`.
clientv3-grpc1.0: Balancer Overview
-----------------------------------
`clientv3-grpc1.0` maintains multiple TCP connections when configured with multiple etcd endpoints. Then pick one address and use it to send all client requests. The pinned address is maintained until the client object is closed (see *Figure 1*). When the client receives an error, it randomly picks another and retries.
![client-balancer-figure-01.png](img/client-balancer-figure-01.png)
clientv3-grpc1.0: Balancer Limitation
-------------------------------------
`clientv3-grpc1.0` opening multiple TCP connections may provide faster balancer failover but requires more resources. The balancer does not understand nodes health status or cluster membership. So, it is possible that balancer gets stuck with one failed or partitioned node.
clientv3-grpc1.7: Balancer Overview
------------------------------------
`clientv3-grpc1.7` maintains only one TCP connection to a chosen etcd server. When given multiple cluster endpoints, a client first tries to connect to them all. As soon as one connection is up, balancer pins the address, closing others (see *Figure 2*). The pinned address is to be maintained until the client object is closed. An error, from server or client network fault, is sent to client error handler (see *Figure 3*).
![client-balancer-figure-02.png](img/client-balancer-figure-02.png)
![client-balancer-figure-03.png](img/client-balancer-figure-03.png)
The client error handler takes an error from gRPC server, and decides whether to retry on the same endpoint, or to switch to other addresses, based on the error code and message (see *Figure 4* and *Figure 5*).
![client-balancer-figure-04.png](img/client-balancer-figure-04.png)
![client-balancer-figure-05.png](img/client-balancer-figure-05.png)
Stream RPCs, such as Watch and KeepAlive, are often requested with no timeouts. Instead, client can send periodic HTTP/2 pings to check the status of a pinned endpoint; if the server does not respond to the ping, balancer switches to other endpoints (see *Figure 6*).
![client-balancer-figure-06.png](img/client-balancer-figure-06.png)
clientv3-grpc1.7: Balancer Limitation
-------------------------------------
`clientv3-grpc1.7` balancer sends HTTP/2 keepalives to detect disconnects from streaming requests. It is a simple gRPC server ping mechanism and does not reason about cluster membership, thus unable to detect network partitions. Since partitioned gRPC server can still respond to client pings, balancer may get stuck with a partitioned node. Ideally, keepalive ping detects partition and triggers endpoint switch, before request time-out (see [etcd#8673](https://github.com/etcd-io/etcd/issues/8673) and *Figure 7*).
![client-balancer-figure-07.png](img/client-balancer-figure-07.png)
`clientv3-grpc1.7` balancer maintains a list of unhealthy endpoints. Disconnected addresses are added to “unhealthy” list, and considered unavailable until after wait duration, which is hard coded as dial timeout with default value 5-second. Balancer can have false positives on which endpoints are unhealthy. For instance, endpoint A may come back right after being blacklisted, but still unusable for next 5 seconds (see *Figure 8*).
`clientv3-grpc1.0` suffered the same problems above.
![client-balancer-figure-08.png](img/client-balancer-figure-08.png)
Upstream gRPC Go had already migrated to new balancer interface. For example, `clientv3-grpc1.7` underlying balancer implementation uses new gRPC balancer and tries to be consistent with old balancer behaviors. While its compatibility has been maintained reasonably well, etcd client still [suffered from subtle breaking changes](https://github.com/grpc/grpc-go/issues/1649). Furthermore, gRPC maintainer recommends to [not rely on the old balancer interface](https://github.com/grpc/grpc-go/issues/1942#issuecomment-375368665). In general, to get better support from upstream, it is best to be in sync with latest gRPC releases. And new features, such as retry policy, may not be backported to gRPC 1.7 branch. Thus, both etcd server and client must migrate to latest gRPC versions.
clientv3-grpc1.23: Balancer Overview
------------------------------------
`clientv3-grpc1.7` is so tightly coupled with old gRPC interface, that every single gRPC dependency upgrade broke client behavior. Majority of development and debugging efforts were devoted to fixing those client behavior changes. As a result, its implementation has become overly complicated with bad assumptions on server connectivities.
The primary goal of `clientv3-grpc1.23` is to simplify balancer failover logic; rather than maintaining a list of unhealthy endpoints, which may be stale, simply roundrobin to the next endpoint whenever client gets disconnected from the current endpoint. It does not assume endpoint status. Thus, no more complicated status tracking is needed (see *Figure 8* and above). Upgrading to `clientv3-grpc1.23` should be no issue; all changes were internal while keeping all the backward compatibilities.
Internally, when given multiple endpoints, `clientv3-grpc1.23` creates multiple sub-connections (one sub-connection per each endpoint), while `clientv3-grpc1.7` creates only one connection to a pinned endpoint (see *Figure 9*). For instance, in 5-node cluster, `clientv3-grpc1.23` balancer would require 5 TCP connections, while `clientv3-grpc1.7` only requires one. By preserving the pool of TCP connections, `clientv3-grpc1.23` may consume more resources but provide more flexible load balancer with better failover performance. The default balancing policy is round robin but can be easily extended to support other types of balancers (e.g. power of two, pick leader, etc.). `clientv3-grpc1.23` uses gRPC resolver group and implements balancer picker policy, in order to delegate complex balancing work to upstream gRPC. On the other hand, `clientv3-grpc1.7` manually handles each gRPC connection and balancer failover, which complicates the implementation. `clientv3-grpc1.23` implements retry in the gRPC interceptor chain that automatically handles gRPC internal errors and enables more advanced retry policies like backoff, while `clientv3-grpc1.7` manually interprets gRPC errors for retries.
![client-balancer-figure-09.png](img/client-balancer-figure-09.png)
clientv3-grpc1.23: Balancer Limitation
--------------------------------------
Improvements can be made by caching the status of each endpoint. For instance, balancer can ping each server in advance to maintain a list of healthy candidates, and use this information when doing round-robin. Or when disconnected, balancer can prioritize healthy endpoints. This may complicate the balancer implementation, thus can be addressed in later versions.
Client-side keepalive ping still does not reason about network partitions. Streaming request may get stuck with a partitioned node. Advanced health checking service need to be implemented to understand the cluster membership (see [etcd#8673](https://github.com/etcd-io/etcd/issues/8673) for more detail).
![client-balancer-figure-07.png](img/client-balancer-figure-07.png)
Currently, retry logic is handled manually as an interceptor. This may be simplified via [official gRPC retries](https://github.com/grpc/proposal/blob/master/A6-client-retries.md).

View File

@ -1,119 +0,0 @@
---
title: etcd learner design
---
etcd Learner
============
*Gyuho Lee (github.com/gyuho, Amazon Web Services, Inc.), Joe Betz (github.com/jpbetz, Google Inc.)*
Background
==========
Membership reconfiguration has been one of the biggest operational challenges. Lets review common challenges.
A newly joined etcd member starts with no data, thus demanding more updates from leader until it catches up with leaders logs. Then leaders network is more likely to be overloaded, blocking or dropping leader heartbeats to followers. In such case, a follower may election-timeout to start a new leader election. That is, a cluster with a new member is more vulnerable to leader election. Both leader election and the subsequent update propagation to the new member are prone to causing periods of cluster unavailability (see *Figure 1*).
![server-learner-figure-01](img/server-learner-figure-01.png)
What if network partition happens? It depends on leader partition. If the leader still maintains the active quorum, the cluster would continue to operate (see *Figure 2*).
![server-learner-figure-02](img/server-learner-figure-02.png)
What if the leader becomes isolated from the rest of the cluster? Leader monitors progress of each follower. When leader loses connectivity from the quorum, it reverts back to follower which will affect the cluster availability (see *Figure 3*).
![server-learner-figure-03](img/server-learner-figure-03.png)
When a new node is added to 3 node cluster, the cluster size becomes 4 and the quorum size becomes 3. What if a new node had joined the cluster, and then network partition happens? It depends on which partition the new member gets located after partition. If the new node happens to be located in the same partition as leaders, the leader still maintains the active quorum of 3. No leadership election happens, and no cluster availability gets affected (see *Figure 4*).
![server-learner-figure-04](img/server-learner-figure-04.png)
If the cluster is 2-and-2 partitioned, then neither of partition maintains the quorum of 3. In this case, leadership election happens (see *Figure 5*).
![server-learner-figure-05](img/server-learner-figure-05.png)
What if network partition happens first, and then a new member gets added? A partitioned 3-node cluster already has one disconnected follower. When a new member is added, the quorum changes from 2 to 3. Now, this cluster has only 2 active nodes out 4, thus losing quorum and starting a new leadership election (see *Figure 6*).
![server-learner-figure-06](img/server-learner-figure-06.png)
Since member add operation can change the size of quorum, it is always recommended to “member remove” first to replace an unhealthy node.
Adding a new member to a 1-node cluster changes the quorum size to 2, immediately causing a leader election when the previous leader finds out quorum is not active. This is because “member add” operation is a 2-step process where user needs to apply “member add” command first, and then starts the new node process (see *Figure 7*).
![server-learner-figure-07](img/server-learner-figure-07.png)
An even worse case is when an added member is misconfigured. Membership reconfiguration is a two-step process: “etcdctl member add” and starting an etcd server process with the given peer URL. That is, “member add” command is applied regardless of URL, even when the URL value is invalid. If the first step is applied with invalid URLs, the second step cannot even start the new etcd. Once the cluster loses quorum, there is no way to revert the membership change (see *Figure 8*).
![server-learner-figure-08](img/server-learner-figure-08.png)
Same applies to a multi-node cluster. For example, the cluster has two members down (one is failed, the other is misconfigured) and two members up, but now it requires at least 3 votes to change the cluster membership (see *Figure 9*).
![server-learner-figure-09](img/server-learner-figure-09.png)
As seen above, a simple misconfiguration can fail the whole cluster into an inoperative state. In such case, an operator need manually recreate the cluster with `etcd --force-new-cluster` flag. As etcd has become a mission-critical service for Kubernetes, even the slightest outage may have significant impact on users. What can we better to make etcd such operations easier? Among other things, leader election is most critical to cluster availability: Can we make membership reconfiguration less disruptive by not changing the size of quorum? Can a new node be idle, only requesting the minimum updates from leader, until it catches up? Can membership misconfiguration be always reversible and handled in a more secure way (wrong member add command run should never fail the cluster)? Should an user worry about network topology when adding a new member? Can member add API work regardless of the location of nodes and ongoing network partitions?
Raft Learner
============
In order to mitigate such availability gaps in the previous section, [Raft §4.2.1](https://github.com/ongardie/dissertation/blob/master/stanford.pdf) introduces a new node state “Learner”, which joins the cluster as a **non-voting member** until it catches up to leaders logs.
Features in v3.4
----------------
An operator should do the minimum amount of work possible to add a new learner node. `member add --learner` command to add a new learner, which joins cluster as a non-voting member but still receives all data from leader (see *Figure 10*).
![server-learner-figure-10](img/server-learner-figure-10.png)
When a learner has caught up with leaders progress, the learner can be promoted to a voting member using `member promote` API, which then counts towards the quorum (see *Figure 11*).
![server-learner-figure-11](img/server-learner-figure-11.png)
etcd server validates promote request to ensure its operational safety. Only after its log has caught up to leaders can learner be promoted to a voting member (see *Figure 12*).
![server-learner-figure-12](img/server-learner-figure-12.png)
Learner only serves as a standby node until promoted: Leadership cannot be transferred to learner. Learner rejects client reads and writes (client balancer should not route requests to learner). Which means learner does not need issue Read Index requests to leader. Such limitation simplifies the initial learner implementation in v3.4 release (see *Figure 13*).
![server-learner-figure-13](img/server-learner-figure-13.png)
In addition, etcd limits the total number of learners that a cluster can have, and avoids overloading the leader with log replication. Learner never promotes itself. While etcd provides learner status information and safety checks, cluster operator must make the final decision whether to promote learner or not.
Features in v3.5
----------------
*Make learner state only and default*: Defaulting a new member state to learner will greatly improve membership reconfiguration safety, because learner does not change the size of quorum. Misconfiguration will always be reversible without losing the quorum.
*Make voting-member promotion fully automatic*: Once a learner catches up to leaders logs, a cluster can automatically promote the learner. etcd requires certain thresholds to be defined by the user, and once the requirements are satisfied, learner promotes itself to a voting member. From a users perspective, “member add” command would work the same way as today but with greater safety provided by learner feature.
*Make learner standby failover node*: A learner joins as a standby node, and gets automatically promoted when the cluster availability is affected.
*Make learner read-only*: A learner can serve as a read-only node that never gets promoted. In a weak consistency mode, learner only receives data from leader and never process writes. Serving reads locally without consensus overhead would greatly decrease the workloads to leader but may serve stale data. In a strong consistency mode, learner requests read index from leader to serve latest data, but still rejects writes.
Learner vs. Mirror Maker
========================
etcd implements “mirror maker” using watch API to continuously relay key creates and updates to a separate cluster. Mirroring usually has low latency overhead once it completes initial synchronization. Learner and mirroring overlap in that both can be used to replicate existing data for read-only. However, mirroring does not guarantee linearizability. During network disconnects, previous key-values might have been discarded, and clients are expected to verify watch responses for correct ordering. Thus, there is no ordering guarantee in mirror. Use mirror for minimum latency (e.g. cross data center) at the costs of consistency. Use learner to retain all historical data and its ordering.
Appendix: Learner Implementation in v3.4
========================================
*Expose "Learner" node type to "MemberAdd" API.*
etcd client adds a flag to “MemberAdd” API for learner node. And etcd server handler applies membership change entry with `pb.ConfChangeAddLearnerNode` type. Once the command has been applied, a server joins the cluster with `etcd --initial-cluster-state=existing` flag. This learner node can neither vote nor count as quorum.
etcd server must not transfer leadership to learner, since it may still lag behind and does not count as quorum. etcd server limits the number of learners that cluster can have to one: the more learners we have, the more data the leader has to propagate. Clients may talk to learner node, but learner rejects all requests other than serializable read and member status API. This is for simplicity of initial implementation. In the future, learner can be extended as a read-only server that continuously mirrors cluster data. Client balancer must provide helper function to exclude learner node endpoint. Otherwise, request sent to learner may fail. Client sync member call should factor into learner node type. So should client endpoints update call.
`MemberList` and `MemberStatus` responses should indicate which node is learner.
*Add "MemberPromote" API.*
Internally in Raft, second `MemberAdd` call to learner node promotes it to a voting member. Leader maintains the progress of each follower and learner. If learner has not completed its snapshot message, reject promote request. Only accept promote request if and only if: The learner node is in a healthy state. The learner is in sync with leader or the delta is within the threshold (e.g. the number of entries to replicate to learner is less than 1/10 of snapshot count, which means it is less likely that even after promotion leader would not need send snapshot to the learner). All these logic are hard-coded in `etcdserver` package and not configurable.
Reference
=========
- Original github issue: [etcd#9161](https://github.com/etcd-io/etcd/issues/9161)
- Use case: [etcd#3715](https://github.com/etcd-io/etcd/issues/3715)
- Use case: [etcd#8888](https://github.com/etcd-io/etcd/issues/8888)
- Use case: [etcd#10114](https://github.com/etcd-io/etcd/issues/10114)

View File

@ -1,99 +0,0 @@
---
title: Glossary
---
This document defines the various terms used in etcd documentation, command line and source code.
## Alarm
The etcd server raises an alarm whenever the cluster needs operator intervention to remain reliable.
## Authentication
Authentication manages user access permissions for etcd resources.
## Client
A client connects to the etcd cluster to issue service requests such as fetching key-value pairs, writing data, or watching for updates.
## Cluster
Cluster consists of several members.
The node in each member follows raft consensus protocol to replicate logs. Cluster receives proposals from members, commits them and apply to local store.
## Compaction
Compaction discards all etcd event history and superseded keys prior to a given revision. It is used to reclaim storage space in the etcd backend database.
## Election
The etcd cluster holds elections among its members to choose a leader as part of the raft consensus protocol.
## Endpoint
A URL pointing to an etcd service or resource.
## Key
A user-defined identifier for storing and retrieving user-defined values in etcd.
## Key range
A set of keys containing either an individual key, a lexical interval for all x such that a < x <= b, or all keys greater than a given key.
## Keyspace
The set of all keys in an etcd cluster.
## Lease
A short-lived renewable contract that deletes keys associated with it on its expiry.
## Member
A logical etcd server that participates in serving an etcd cluster.
## Modification Revision
The first revision to hold the last write to a given key.
## Peer
Peer is another member of the same cluster.
## Proposal
A proposal is a request (for example a write request, a configuration change request) that needs to go through raft protocol.
## Quorum
The number of active members needed for consensus to modify the cluster state. etcd requires a member majority to reach quorum.
## Revision
A 64-bit cluster-wide counter that is incremented each time the keyspace is modified.
## Role
A unit of permissions over a set of key ranges which may be granted to a set of users for access control.
## Snapshot
A point-in-time backup of the etcd cluster state.
## Store
The physical storage backing the cluster keyspace.
## Transaction
An atomically executed set of operations. All modified keys in a transaction share the same modification revision.
## Key Version
The number of writes to a key since it was created, starting at 1. The version of a nonexistent or deleted key is 0.
## Watcher
A client opens a watcher to observe updates on a given key range.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 257 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 218 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 104 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 145 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 128 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 89 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 60 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

View File

@ -1,118 +0,0 @@
---
title: etcd versus other key-value stores
---
The name "etcd" originated from two ideas, the unix "/etc" folder and "d"istributed systems. The "/etc" folder is a place to store configuration data for a single system whereas etcd stores configuration information for large scale distributed systems. Hence, a "d"istributed "/etc" is "etcd".
etcd is designed as a general substrate for large scale distributed systems. These are systems that will never tolerate split-brain operation and are willing to sacrifice availability to achieve this end. etcd stores metadata in a consistent and fault-tolerant way. An etcd cluster is meant to provide key-value storage with best of class stability, reliability, scalability and performance.
Distributed systems use etcd as a consistent key-value store for configuration management, service discovery, and coordinating distributed work. Many [organizations][production-users] use etcd to implement production systems such as container schedulers, service discovery services, and distributed data storage. Common distributed patterns using etcd include [leader election][etcd-etcdctl-elect], [distributed locks][etcd-etcdctl-lock], and monitoring machine liveness.
## Use cases
- Container Linux by CoreOS: Applications running on [Container Linux][container-linux] get automatic, zero-downtime Linux kernel updates. Container Linux uses [locksmith] to coordinate updates. Locksmith implements a distributed semaphore over etcd to ensure only a subset of a cluster is rebooting at any given time.
- [Kubernetes][kubernetes] stores configuration data into etcd for service discovery and cluster management; etcd's consistency is crucial for correctly scheduling and operating services. The Kubernetes API server persists cluster state into etcd. It uses etcd's watch API to monitor the cluster and roll out critical configuration changes.
## Comparison chart
Perhaps etcd already seems like a good fit, but as with all technological decisions, proceed with caution. Please note this documentation is written by the etcd team. Although the ideal is a disinterested comparison of technology and features, the authors expertise and biases obviously favor etcd. Use only as directed.
The table below is a handy quick reference for spotting the differences among etcd and its most popular alternatives at a glance. Further commentary and details for each column are in the sections following the table.
| | etcd | ZooKeeper | Consul | NewSQL (Cloud Spanner, CockroachDB, TiDB) |
| --- | --- | --- | --- | --- |
| Concurrency Primitives | [Lock RPCs][etcd-v3lock], [Election RPCs][etcd-v3election], [command line locks][etcd-etcdctl-lock], [command line elections][etcd-etcdctl-elect], [recipes][etcd-recipe] in go | External [curator recipes][curator] in Java | [Native lock API][consul-lock] | [Rare][newsql-leader], if any |
| Linearizable Reads | [Yes][etcd-linread] | No | [Yes][consul-linread] | Sometimes |
| Multi-version Concurrency Control | [Yes][etcd-mvcc] | No | No | Sometimes |
| Transactions | [Field compares, Read, Write][etcd-txn] | [Version checks, Write][zk-txn] | [Field compare, Lock, Read, Write][consul-txn] | SQL-style |
| Change Notification | [Historical and current key intervals][etcd-watch] | [Current keys and directories][zk-watch] | [Current keys and prefixes][consul-watch] | Triggers (sometimes) |
| User permissions | [Role based][etcd-rbac] | [ACLs][zk-acl] | [ACLs][consul-acl] | Varies (per-table [GRANT][cockroach-grant], per-database [roles][spanner-roles]) |
| HTTP/JSON API | [Yes][etcd-json] | No | [Yes][consul-json] | Rarely |
| Membership Reconfiguration | [Yes][etcd-reconfig] | [>3.5.0][zk-reconfig] | [Yes][consul-reconfig] | Yes |
| Maximum reliable database size | Several gigabytes | Hundreds of megabytes (sometimes several gigabytes) | Hundreds of MBs | Terabytes+ |
| Minimum read linearization latency | Network RTT | No read linearization | RTT + fsync | Clock barriers (atomic, NTP) |
### ZooKeeper
ZooKeeper solves the same problem as etcd: distributed system coordination and metadata storage. However, etcd has the luxury of hindsight taken from engineering and operational experience with ZooKeepers design and implementation. The lessons learned from Zookeeper certainly informed etcds design, helping it support large scale systems like Kubernetes. The improvements etcd made over Zookeeper include:
* Dynamic cluster membership reconfiguration
* Stable read/write under high load
* A multi-version concurrency control data model
* Reliable key monitoring which never silently drop events
* Lease primitives decoupling connections from sessions
* APIs for safe distributed shared locks
Furthermore, etcd supports a wide range of languages and frameworks out of the box. Whereas Zookeeper has its own custom Jute RPC protocol, which is totally unique to Zookeeper and limits its [supported language bindings][zk-bindings], etcds client protocol is built from [gRPC][grpc], a popular RPC framework with language bindings for go, C++, Java, and more. Likewise, gRPC can be serialized into JSON over HTTP, so even general command line utilities like `curl` can talk to it. Since systems can select from a variety of choices, they are built on etcd with native tooling rather than around etcd with a single fixed set of technologies.
When considering features, support, and stability, new applications planning to use Zookeeper for a consistent key value store would do well to choose etcd instead.
### Consul
Consul is an end-to-end service discovery framework. It provides built-in health checking, failure detection, and DNS services. In addition, Consul exposes a key value store with RESTful HTTP APIs. [As it stands in Consul 1.0][dbtester-comparison-results], the storage system does not scale as well as other systems like etcd or Zookeeper in key-value operations; systems requiring millions of keys will suffer from high latencies and memory pressure. The key value API is missing, most notably, multi-version keys, conditional transactions, and reliable streaming watches.
etcd and Consul solve different problems. If looking for a distributed consistent key value store, etcd is a better choice over Consul. If looking for end-to-end cluster service discovery, etcd will not have enough features; choose Kubernetes, Consul, or SmartStack.
### NewSQL (Cloud Spanner, CockroachDB, TiDB)
Both etcd and NewSQL databases (e.g., [Cockroach][cockroach], [TiDB][tidb], [Google Spanner][spanner]) provide strong data consistency guarantees with high availability. However, the significantly different system design parameters lead to significantly different client APIs and performance characteristics.
NewSQL databases are meant to horizontally scale across data centers. These systems typically partition data across multiple consistent replication groups (shards), potentially distant, storing data sets on the order of terabytes and above. This sort of scaling makes them poor candidates for distributed coordination as they have long latencies from waiting on clocks and expect updates with mostly localized dependency graphs. The data is organized into tables, including SQL-style query facilities with richer semantics than etcd, but at the cost of additional complexity for processing, planning, and optimizing queries.
In short, choose etcd for storing metadata or coordinating distributed applications. If storing more than a few GB of data or if full SQL queries are needed, choose a NewSQL database.
## Using etcd for metadata
etcd replicates all data within a single consistent replication group. For storing up to a few GB of data with consistent ordering, this is the most efficient approach. Each modification of cluster state, which may change multiple keys, is assigned a global unique ID, called a revision in etcd, from a monotonically increasing counter for reasoning over ordering. Since theres only a single replication group, the modification request only needs to go through the raft protocol to commit. By limiting consensus to one replication group, etcd gets distributed consistency with a simple protocol while achieving low latency and high throughput.
The replication behind etcd cannot horizontally scale because it lacks data sharding. In contrast, NewSQL databases usually shard data across multiple consistent replication groups, storing data sets on the order of terabytes and above. However, to assign each modification a global unique and increasing ID, each request must go through an additional coordination protocol among replication groups. This extra coordination step may potentially conflict on the global ID, forcing ordered requests to retry. The result is a more complicated approach with typically worse performance than etcd for strict ordering.
If an application reasons primarily about metadata or metadata ordering, such as to coordinate processes, choose etcd. If the application needs a large data store spanning multiple data centers and does not heavily depend on strong global ordering properties, choose a NewSQL database.
## Using etcd for distributed coordination
etcd has distributed coordination primitives such as event watches, leases, elections, and distributed shared locks out of the box. These primitives are both maintained and supported by the etcd developers; leaving these primitives to external libraries shirks the responsibility of developing foundational distributed software, essentially leaving the system incomplete. NewSQL databases usually expect these distributed coordination primitives to be authored by third parties. Likewise, ZooKeeper famously has a separate and independent [library][curator] of coordination recipes. Consul, which provides a native locking API, goes so far as to apologize that its “[not a bulletproof method][consul-bulletproof]”.
In theory, its possible to build these primitives atop any storage systems providing strong consistency. However, the algorithms tend to be subtle; it is easy to develop a locking algorithm that appears to work, only to suddenly break due to thundering herd and timing skew. Furthermore, other primitives supported by etcd, such as transactional memory depend on etcds MVCC data model; simple strong consistency is not enough.
For distributed coordination, choosing etcd can help prevent operational headaches and save engineering effort.
[production-users]: ../production-users.md
[grpc]: https://www.grpc.io
[consul-bulletproof]: https://www.consul.io/docs/internals/sessions.html
[curator]: http://curator.apache.org/
[cockroach]: https://github.com/cockroachdb/cockroach
[spanner]: https://cloud.google.com/spanner/
[tidb]: https://github.com/pingcap/tidb
[etcd-v3lock]: https://godoc.org/github.com/etcd-io/etcd/etcdserver/api/v3lock/v3lockpb
[etcd-v3election]: https://godoc.org/github.com/coreos/etcd-io/etcdserver/api/v3election/v3electionpb
[etcd-etcdctl-lock]: ../../etcdctl/README.md#lock-lockname-command-arg1-arg2-
[etcd-etcdctl-elect]: ../../etcdctl/README.md#elect-options-election-name-proposal
[etcd-mvcc]: data_model.md
[etcd-recipe]: https://godoc.org/github.com/etcd-io/etcd/contrib/recipes
[consul-lock]: https://www.consul.io/docs/commands/lock.html
[newsql-leader]: http://dl.acm.org/citation.cfm?id=2960999
[etcd-reconfig]: ../op-guide/runtime-configuration.md
[zk-reconfig]: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html
[consul-reconfig]: https://www.consul.io/docs/guides/servers.html
[etcd-linread]: api_guarantees.md#linearizability
[consul-linread]: https://www.consul.io/docs/agent/http.html#consistency
[etcd-json]: ../dev-guide/api_grpc_gateway.md
[consul-json]: https://www.consul.io/docs/agent/http.html#formatted-json-output
[etcd-txn]: api.md#transaction
[zk-txn]: https://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#multi(java.lang.Iterable)
[consul-txn]: https://www.consul.io/docs/agent/http/kv.html#txn
[etcd-watch]: api.md#watch-streams
[zk-watch]: https://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#ch_zkWatches
[consul-watch]: https://www.consul.io/docs/agent/watches.html
[etcd-commonname]: ../op-guide/authentication.md#using-tls-common-name
[etcd-rbac]: ../op-guide/authentication.md#working-with-roles
[zk-acl]: https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#sc_ZooKeeperAccessControl
[consul-acl]: https://www.consul.io/docs/internals/acl.html
[cockroach-grant]: https://www.cockroachlabs.com/docs/stable/grant.html
[spanner-roles]: https://cloud.google.com/spanner/docs/iam#roles
[zk-bindings]: https://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html#ch_bindings
[container-linux]: https://coreos.com/why
[locksmith]: https://github.com/coreos/locksmith
[kubernetes]: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/
[dbtester-comparison-results]: https://github.com/coreos/dbtester/tree/master/test-results/2018Q1-02-etcd-zookeeper-consul

View File

@ -1,118 +0,0 @@
---
title: Metrics
---
etcd uses [Prometheus][prometheus] for metrics reporting. The metrics can be used for real-time monitoring and debugging. etcd does not persist its metrics; if a member restarts, the metrics will be reset.
The simplest way to see the available metrics is to cURL the metrics endpoint `/metrics`. The format is described [here](http://prometheus.io/docs/instrumenting/exposition_formats/).
Follow the [Prometheus getting started doc][prometheus-getting-started] to spin up a Prometheus server to collect etcd metrics.
The naming of metrics follows the suggested [Prometheus best practices][prometheus-naming]. A metric name has an `etcd` or `etcd_debugging` prefix as its namespace and a subsystem prefix (for example `wal` and `etcdserver`).
## etcd namespace metrics
The metrics under the `etcd` prefix are for monitoring and alerting. They are stable high level metrics. If there is any change of these metrics, it will be included in release notes.
Metrics that are etcd2 related are documented [v2 metrics guide][v2-http-metrics].
### Server
These metrics describe the status of the etcd server. In order to detect outages or problems for troubleshooting, the server metrics of every production etcd cluster should be closely monitored.
All these metrics are prefixed with `etcd_server_`
| Name | Description | Type |
|---------------------------|----------------------------------------------------------|---------|
| has_leader | Whether or not a leader exists. 1 is existence, 0 is not.| Gauge |
| leader_changes_seen_total | The number of leader changes seen. | Counter |
| proposals_committed_total | The total number of consensus proposals committed. | Gauge |
| proposals_applied_total | The total number of consensus proposals applied. | Gauge |
| proposals_pending | The current number of pending proposals. | Gauge |
| proposals_failed_total | The total number of failed proposals seen. | Counter |
`has_leader` indicates whether the member has a leader. If a member does not have a leader, it is
totally unavailable. If all the members in the cluster do not have any leader, the entire cluster
is totally unavailable.
`leader_changes_seen_total` counts the number of leader changes the member has seen since its start. Rapid leadership changes impact the performance of etcd significantly. It also signals that the leader is unstable, perhaps due to network connectivity issues or excessive load hitting the etcd cluster.
`proposals_committed_total` records the total number of consensus proposals committed. This gauge should increase over time if the cluster is healthy. Several healthy members of an etcd cluster may have different total committed proposals at once. This discrepancy may be due to recovering from peers after starting, lagging behind the leader, or being the leader and therefore having the most commits. It is important to monitor this metric across all the members in the cluster; a consistently large lag between a single member and its leader indicates that member is slow or unhealthy.
`proposals_applied_total` records the total number of consensus proposals applied. The etcd server applies every committed proposal asynchronously. The difference between `proposals_committed_total` and `proposals_applied_total` should usually be small (within a few thousands even under high load). If the difference between them continues to rise, it indicates that the etcd server is overloaded. This might happen when applying expensive queries like heavy range queries or large txn operations.
`proposals_pending` indicates how many proposals are queued to commit. Rising pending proposals suggests there is a high client load or the member cannot commit proposals.
`proposals_failed_total` are normally related to two issues: temporary failures related to a leader election or longer downtime caused by a loss of quorum in the cluster.
### Disk
These metrics describe the status of the disk operations.
All these metrics are prefixed with `etcd_disk_`.
| Name | Description | Type |
|------------------------------------|-------------------------------------------------------|-----------|
| wal_fsync_duration_seconds | The latency distributions of fsync called by wal | Histogram |
| backend_commit_duration_seconds | The latency distributions of commit called by backend.| Histogram |
A `wal_fsync` is called when etcd persists its log entries to disk before applying them.
A `backend_commit` is called when etcd commits an incremental snapshot of its most recent changes to disk.
High disk operation latencies (`wal_fsync_duration_seconds` or `backend_commit_duration_seconds`) often indicate disk issues. It may cause high request latency or make the cluster unstable.
### Network
These metrics describe the status of the network.
All these metrics are prefixed with `etcd_network_`
| Name | Description | Type |
|---------------------------|--------------------------------------------------------------------|---------------|
| peer_sent_bytes_total | The total number of bytes sent to the peer with ID `To`. | Counter(To) |
| peer_received_bytes_total | The total number of bytes received from the peer with ID `From`. | Counter(From) |
| peer_sent_failures_total | The total number of send failures from the peer with ID `To`. | Counter(To) |
| peer_received_failures_total | The total number of receive failures from the peer with ID `From`. | Counter(From) |
| peer_round_trip_time_seconds | Round-Trip-Time histogram between peers. | Histogram(To) |
| client_grpc_sent_bytes_total | The total number of bytes sent to grpc clients. | Counter |
| client_grpc_received_bytes_total| The total number of bytes received to grpc clients. | Counter |
`peer_sent_bytes_total` counts the total number of bytes sent to a specific peer. Usually the leader member sends more data than other members since it is responsible for transmitting replicated data.
`peer_received_bytes_total` counts the total number of bytes received from a specific peer. Usually follower members receive data only from the leader member.
### gRPC requests
These metrics are exposed via [go-grpc-prometheus][go-grpc-prometheus].
## etcd_debugging namespace metrics
The metrics under the `etcd_debugging` prefix are for debugging. They are very implementation dependent and volatile. They might be changed or removed without any warning in new etcd releases. Some of the metrics might be moved to the `etcd` prefix when they become more stable.
### Snapshot
| Name | Description | Type |
|--------------------------------------------|------------------------------------------------------------|-----------|
| snapshot_save_total_duration_seconds | The total latency distributions of save called by snapshot | Histogram |
Abnormally high snapshot duration (`snapshot_save_total_duration_seconds`) indicates disk issues and might cause the cluster to be unstable.
## Prometheus supplied metrics
The Prometheus client library provides a number of metrics under the `go` and `process` namespaces. There are a few that are particularly interesting.
| Name | Description | Type |
|-----------------------------------|--------------------------------------------|--------------|
| process_open_fds | Number of open file descriptors. | Gauge |
| process_max_fds | Maximum number of open file descriptors. | Gauge |
Heavy file descriptor (`process_open_fds`) usage (i.e., near the process's file descriptor limit, `process_max_fds`) indicates a potential file descriptor exhaustion issue. If the file descriptors are exhausted, etcd may panic because it cannot create new WAL files.
[glossary-proposal]: learning/glossary.md#proposal
[prometheus]: http://prometheus.io/
[prometheus-getting-started]: http://prometheus.io/docs/introduction/getting_started/
[prometheus-naming]: http://prometheus.io/docs/practices/naming/
[v2-http-metrics]: v2/metrics.md#http-requests
[go-grpc-prometheus]: https://github.com/grpc-ecosystem/go-grpc-prometheus

View File

@ -1,975 +0,0 @@
# server version: etcd_server_version{server_version="3.3.0+git"}
# name: "etcd_debugging_disk_backend_commit_rebalance_duration_seconds"
# description: "The latency distributions of commit.rebalance called by bboltdb backend."
# type: "histogram"
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.001"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.002"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.004"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.008"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.016"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.032"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.064"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.128"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.256"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="0.512"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="1.024"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="2.048"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="4.096"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="8.192"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_bucket{le="+Inf"}
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_sum
etcd_debugging_disk_backend_commit_rebalance_duration_seconds_count
# name: "etcd_debugging_disk_backend_commit_spill_duration_seconds"
# description: "The latency distributions of commit.spill called by bboltdb backend."
# type: "histogram"
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.001"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.002"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.004"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.008"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.016"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.032"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.064"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.128"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.256"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="0.512"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="1.024"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="2.048"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="4.096"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="8.192"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_bucket{le="+Inf"}
etcd_debugging_disk_backend_commit_spill_duration_seconds_sum
etcd_debugging_disk_backend_commit_spill_duration_seconds_count
# name: "etcd_debugging_disk_backend_commit_write_duration_seconds"
# description: "The latency distributions of commit.write called by bboltdb backend."
# type: "histogram"
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.001"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.002"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.004"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.008"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.016"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.032"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.064"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.128"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.256"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="0.512"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="1.024"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="2.048"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="4.096"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="8.192"}
etcd_debugging_disk_backend_commit_write_duration_seconds_bucket{le="+Inf"}
etcd_debugging_disk_backend_commit_write_duration_seconds_sum
etcd_debugging_disk_backend_commit_write_duration_seconds_count
# name: "etcd_debugging_lease_granted_total"
# description: "The total number of granted leases."
# type: "counter"
etcd_debugging_lease_granted_total
# name: "etcd_debugging_lease_renewed_total"
# description: "The number of renewed leases seen by the leader."
# type: "counter"
etcd_debugging_lease_renewed_total
# name: "etcd_debugging_lease_revoked_total"
# description: "The total number of revoked leases."
# type: "counter"
etcd_debugging_lease_revoked_total
# name: "etcd_debugging_lease_ttl_total"
# description: "Bucketed histogram of lease TTLs."
# type: "histogram"
etcd_debugging_lease_ttl_total_bucket{le="1"}
etcd_debugging_lease_ttl_total_bucket{le="2"}
etcd_debugging_lease_ttl_total_bucket{le="4"}
etcd_debugging_lease_ttl_total_bucket{le="8"}
etcd_debugging_lease_ttl_total_bucket{le="16"}
etcd_debugging_lease_ttl_total_bucket{le="32"}
etcd_debugging_lease_ttl_total_bucket{le="64"}
etcd_debugging_lease_ttl_total_bucket{le="128"}
etcd_debugging_lease_ttl_total_bucket{le="256"}
etcd_debugging_lease_ttl_total_bucket{le="512"}
etcd_debugging_lease_ttl_total_bucket{le="1024"}
etcd_debugging_lease_ttl_total_bucket{le="2048"}
etcd_debugging_lease_ttl_total_bucket{le="4096"}
etcd_debugging_lease_ttl_total_bucket{le="8192"}
etcd_debugging_lease_ttl_total_bucket{le="16384"}
etcd_debugging_lease_ttl_total_bucket{le="32768"}
etcd_debugging_lease_ttl_total_bucket{le="65536"}
etcd_debugging_lease_ttl_total_bucket{le="131072"}
etcd_debugging_lease_ttl_total_bucket{le="262144"}
etcd_debugging_lease_ttl_total_bucket{le="524288"}
etcd_debugging_lease_ttl_total_bucket{le="1.048576e+06"}
etcd_debugging_lease_ttl_total_bucket{le="2.097152e+06"}
etcd_debugging_lease_ttl_total_bucket{le="4.194304e+06"}
etcd_debugging_lease_ttl_total_bucket{le="8.388608e+06"}
etcd_debugging_lease_ttl_total_bucket{le="+Inf"}
etcd_debugging_lease_ttl_total_sum
etcd_debugging_lease_ttl_total_count
# name: "etcd_debugging_mvcc_db_compaction_keys_total"
# description: "Total number of db keys compacted."
# type: "counter"
etcd_debugging_mvcc_db_compaction_keys_total
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database physically allocated in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_server_lease_expired_total"
# description: "The total number of expired leases."
# type: "counter"
etcd_debugging_server_lease_expired_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_backend_defrag_duration_seconds"
# description: "The latency distribution of backend defragmentation."
# type: "histogram"
etcd_disk_backend_defrag_duration_seconds_bucket{le="0.1"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="0.2"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="0.4"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="0.8"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="1.6"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="3.2"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="6.4"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="12.8"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="25.6"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="51.2"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="102.4"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="204.8"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="409.6"}
etcd_disk_backend_defrag_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_defrag_duration_seconds_sum
etcd_disk_backend_defrag_duration_seconds_count
# name: "etcd_disk_backend_snapshot_duration_seconds"
# description: "The latency distribution of backend snapshots."
# type: "histogram"
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.01"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.02"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.04"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.08"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.16"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.32"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="0.64"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="1.28"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="2.56"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="5.12"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="10.24"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="20.48"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="40.96"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="81.92"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="163.84"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="327.68"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="655.36"}
etcd_disk_backend_snapshot_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_snapshot_duration_seconds_sum
etcd_disk_backend_snapshot_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by WAL."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database physically allocated in bytes."
# type: "gauge"
etcd_mvcc_db_total_size_in_bytes
# name: "etcd_mvcc_db_total_size_in_use_in_bytes"
# description: "Total size of the underlying database logically in use in bytes."
# type: "gauge"
etcd_mvcc_db_total_size_in_use_in_bytes
# name: "etcd_mvcc_hash_duration_seconds"
# description: "The latency distribution of storage hash operation."
# type: "histogram"
etcd_mvcc_hash_duration_seconds_bucket{le="0.01"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.02"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.04"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.08"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.16"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.32"}
etcd_mvcc_hash_duration_seconds_bucket{le="0.64"}
etcd_mvcc_hash_duration_seconds_bucket{le="1.28"}
etcd_mvcc_hash_duration_seconds_bucket{le="2.56"}
etcd_mvcc_hash_duration_seconds_bucket{le="5.12"}
etcd_mvcc_hash_duration_seconds_bucket{le="10.24"}
etcd_mvcc_hash_duration_seconds_bucket{le="20.48"}
etcd_mvcc_hash_duration_seconds_bucket{le="40.96"}
etcd_mvcc_hash_duration_seconds_bucket{le="81.92"}
etcd_mvcc_hash_duration_seconds_bucket{le="163.84"}
etcd_mvcc_hash_duration_seconds_bucket{le="+Inf"}
etcd_mvcc_hash_duration_seconds_sum
etcd_mvcc_hash_duration_seconds_count
# name: "etcd_mvcc_hash_rev_duration_seconds"
# description: "The latency distribution of storage hash by revision operation."
# type: "histogram"
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.01"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.02"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.04"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.08"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.16"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.32"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="0.64"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="1.28"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="2.56"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="5.12"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="10.24"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="20.48"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="40.96"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="81.92"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="163.84"}
etcd_mvcc_hash_rev_duration_seconds_bucket{le="+Inf"}
etcd_mvcc_hash_rev_duration_seconds_sum
etcd_mvcc_hash_rev_duration_seconds_count
# name: "etcd_network_active_peers"
# description: "The current number of active peer connections."
# type: "gauge"
etcd_network_active_peers{Local="LOCAL_NODE_ID",Remote="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers"
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="1.6384"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="3.2768"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_go_version"
# description: "Which Go version server is running with. 1 for 'server_go_version' label with current version."
# type: "gauge"
etcd_server_go_version{server_go_version="go1.11.1"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_health_failures"
# description: "The total number of failed health checks"
# type: "counter"
etcd_server_health_failures
# name: "etcd_server_health_success"
# description: "The total number of successful health checks"
# type: "counter"
etcd_server_health_success
# name: "etcd_server_heartbeat_send_failures_total"
# description: "The total number of leader heartbeat send failures (likely overloaded from slow disk)."
# type: "counter"
etcd_server_heartbeat_send_failures_total
# name: "etcd_server_id"
# description: "Server or member ID in hexadecimal format. 1 for 'server_id' label with current ID."
# type: "gauge"
etcd_server_id{server_id="221ed218fa16d1fe"}
etcd_server_id{server_id="3f1ca865609d428d"}
# name: "etcd_server_is_leader"
# description: "Whether or not this member is a leader. 1 if is, 0 otherwise."
# type: "gauge"
etcd_server_is_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_is_learner"
# description: "Whether or not this member is a learner. 1 if is, 0 otherwise."
# type: "gauge"
etcd_server_is_learner
# name: "etcd_server_learner_promote_failures"
# description: "The total number of failed learner promotions (likely learner not ready) while this member is leader."
# type: "counter"
etcd_server_learner_promote_failures
# name: "etcd_server_learner_promote_successes"
# description: "The total number of successful learner promotions while this member is leader."
# type: "counter"
etcd_server_learner_promote_successes
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "etcd_server_quota_backend_bytes"
# description: "Current backend storage quota size in bytes."
# type: "gauge"
etcd_server_quota_backend_bytes
# name: "etcd_server_read_indexes_failed_total"
# description: "The total number of failed read indexes seen."
# type: "counter"
etcd_server_read_indexes_failed_total
# name: "etcd_server_slow_apply_total"
# description: "The total number of slow apply requests (likely overloaded from slow disk)."
# type: "counter"
etcd_server_slow_apply_total
# name: "etcd_server_slow_read_indexes_total"
# description: "The total number of pending read indexes not in sync with leader's or timed out read index requests."
# type: "counter"
etcd_server_slow_read_indexes_total
# name: "etcd_server_version"
# description: "Which version is running. 1 for 'server_version' label with current version."
# type: "gauge"
etcd_server_version{server_version="3.3.0+git"}
# name: "etcd_snap_db_fsync_duration_seconds"
# description: "The latency distributions of fsyncing .snap.db file"
# type: "histogram"
etcd_snap_db_fsync_duration_seconds_bucket{le="0.001"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.002"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.004"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.008"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.016"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.032"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.064"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.128"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.256"}
etcd_snap_db_fsync_duration_seconds_bucket{le="0.512"}
etcd_snap_db_fsync_duration_seconds_bucket{le="1.024"}
etcd_snap_db_fsync_duration_seconds_bucket{le="2.048"}
etcd_snap_db_fsync_duration_seconds_bucket{le="4.096"}
etcd_snap_db_fsync_duration_seconds_bucket{le="8.192"}
etcd_snap_db_fsync_duration_seconds_bucket{le="+Inf"}
etcd_snap_db_fsync_duration_seconds_sum
etcd_snap_db_fsync_duration_seconds_count
# name: "etcd_snap_db_save_total_duration_seconds"
# description: "The total latency distributions of v3 snapshot save"
# type: "histogram"
etcd_snap_db_save_total_duration_seconds_bucket{le="0.1"}
etcd_snap_db_save_total_duration_seconds_bucket{le="0.2"}
etcd_snap_db_save_total_duration_seconds_bucket{le="0.4"}
etcd_snap_db_save_total_duration_seconds_bucket{le="0.8"}
etcd_snap_db_save_total_duration_seconds_bucket{le="1.6"}
etcd_snap_db_save_total_duration_seconds_bucket{le="3.2"}
etcd_snap_db_save_total_duration_seconds_bucket{le="6.4"}
etcd_snap_db_save_total_duration_seconds_bucket{le="12.8"}
etcd_snap_db_save_total_duration_seconds_bucket{le="25.6"}
etcd_snap_db_save_total_duration_seconds_bucket{le="51.2"}
etcd_snap_db_save_total_duration_seconds_bucket{le="+Inf"}
etcd_snap_db_save_total_duration_seconds_sum
etcd_snap_db_save_total_duration_seconds_count
# name: "etcd_snap_fsync_duration_seconds"
# description: "The latency distributions of fsync called by snap."
# type: "histogram"
etcd_snap_fsync_duration_seconds_bucket{le="0.001"}
etcd_snap_fsync_duration_seconds_bucket{le="0.002"}
etcd_snap_fsync_duration_seconds_bucket{le="0.004"}
etcd_snap_fsync_duration_seconds_bucket{le="0.008"}
etcd_snap_fsync_duration_seconds_bucket{le="0.016"}
etcd_snap_fsync_duration_seconds_bucket{le="0.032"}
etcd_snap_fsync_duration_seconds_bucket{le="0.064"}
etcd_snap_fsync_duration_seconds_bucket{le="0.128"}
etcd_snap_fsync_duration_seconds_bucket{le="0.256"}
etcd_snap_fsync_duration_seconds_bucket{le="0.512"}
etcd_snap_fsync_duration_seconds_bucket{le="1.024"}
etcd_snap_fsync_duration_seconds_bucket{le="2.048"}
etcd_snap_fsync_duration_seconds_bucket{le="4.096"}
etcd_snap_fsync_duration_seconds_bucket{le="8.192"}
etcd_snap_fsync_duration_seconds_bucket{le="+Inf"}
etcd_snap_fsync_duration_seconds_sum
etcd_snap_fsync_duration_seconds_count
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "Aborted"
# - "AlreadyExists"
# - "Canceled"
# - "DataLoss"
# - "DeadlineExceeded"
# - "FailedPrecondition"
# - "Internal"
# - "InvalidArgument"
# - "NotFound"
# - "OK"
# - "OutOfRange"
# - "PermissionDenied"
# - "ResourceExhausted"
# - "Unauthenticated"
# - "Unavailable"
# - "Unimplemented"
# - "Unknown"
grpc_server_handled_total{grpc_code="CODE",grpc_method="Alarm",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="AuthDisable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="AuthEnable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Authenticate",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Check",grpc_service="grpc.health.v1.Health",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Compact",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Defragment",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Hash",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="HashKV",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="LeaseGrant",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="LeaseKeepAlive",grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="LeaseLeases",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="LeaseRevoke",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="LeaseTimeToLive",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="MemberAdd",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="MemberList",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="MemberRemove",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="MemberUpdate",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="MoveLeader",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleGrantPermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="RoleRevokePermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Snapshot",grpc_service="etcdserverpb.Maintenance",grpc_type="server_stream"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Status",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Txn",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserChangePassword",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserGrantRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="UserRevokeRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="Alarm",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="AuthDisable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="AuthEnable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Authenticate",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Check",grpc_service="grpc.health.v1.Health",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Compact",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Defragment",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Hash",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="HashKV",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="LeaseGrant",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="LeaseKeepAlive",grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}
grpc_server_msg_received_total{grpc_method="LeaseLeases",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="LeaseRevoke",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="LeaseTimeToLive",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="MemberAdd",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="MemberList",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="MemberRemove",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="MemberUpdate",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="MoveLeader",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleGrantPermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="RoleRevokePermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Snapshot",grpc_service="etcdserverpb.Maintenance",grpc_type="server_stream"}
grpc_server_msg_received_total{grpc_method="Status",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Txn",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserChangePassword",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserGrantRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="UserRevokeRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="Alarm",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="AuthDisable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="AuthEnable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Authenticate",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Check",grpc_service="grpc.health.v1.Health",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Compact",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Defragment",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Hash",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="HashKV",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="LeaseGrant",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="LeaseKeepAlive",grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}
grpc_server_msg_sent_total{grpc_method="LeaseLeases",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="LeaseRevoke",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="LeaseTimeToLive",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="MemberAdd",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="MemberList",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="MemberRemove",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="MemberUpdate",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="MoveLeader",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleGrantPermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="RoleRevokePermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Snapshot",grpc_service="etcdserverpb.Maintenance",grpc_type="server_stream"}
grpc_server_msg_sent_total{grpc_method="Status",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Txn",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserChangePassword",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserGrantRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="UserRevokeRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="Alarm",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="AuthDisable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="AuthEnable",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="Authenticate",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="Check",grpc_service="grpc.health.v1.Health",grpc_type="unary"}
grpc_server_started_total{grpc_method="Compact",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Defragment",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Hash",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="HashKV",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="LeaseGrant",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_started_total{grpc_method="LeaseKeepAlive",grpc_service="etcdserverpb.Lease",grpc_type="bidi_stream"}
grpc_server_started_total{grpc_method="LeaseLeases",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_started_total{grpc_method="LeaseRevoke",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_started_total{grpc_method="LeaseTimeToLive",grpc_service="etcdserverpb.Lease",grpc_type="unary"}
grpc_server_started_total{grpc_method="MemberAdd",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_started_total{grpc_method="MemberList",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_started_total{grpc_method="MemberRemove",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_started_total{grpc_method="MemberUpdate",grpc_service="etcdserverpb.Cluster",grpc_type="unary"}
grpc_server_started_total{grpc_method="MoveLeader",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleGrantPermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="RoleRevokePermission",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="Snapshot",grpc_service="etcdserverpb.Maintenance",grpc_type="server_stream"}
grpc_server_started_total{grpc_method="Status",grpc_service="etcdserverpb.Maintenance",grpc_type="unary"}
grpc_server_started_total{grpc_method="Txn",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserAdd",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserChangePassword",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserDelete",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserGet",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserGrantRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserList",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="UserRevokeRole",grpc_service="etcdserverpb.Auth",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "OK"
# - "Unavailable"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "OK"
# - "Unavailable"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "Canceled"
# - "OK"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "OK"
# - "Unavailable"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "OK"
# - "Unavailable"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,511 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "OK"
# - "Unavailable"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

View File

@ -1,516 +0,0 @@
# server version: unknown
# name: "etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of db compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="2048"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="4096"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_compaction_total_duration_milliseconds"
# description: "Bucketed histogram of db compaction total duration."
# type: "histogram"
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="100"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="1600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="3200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="6400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="12800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="25600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="51200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="102400"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="204800"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="409600"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="819200"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_sum
etcd_debugging_mvcc_db_compaction_total_duration_milliseconds_count
# name: "etcd_debugging_mvcc_db_total_size_in_bytes"
# description: "Total size of the underlying database in bytes."
# type: "gauge"
etcd_debugging_mvcc_db_total_size_in_bytes
# name: "etcd_debugging_mvcc_delete_total"
# description: "Total number of deletes seen by this member."
# type: "counter"
etcd_debugging_mvcc_delete_total
# name: "etcd_debugging_mvcc_events_total"
# description: "Total number of events sent by this member."
# type: "counter"
etcd_debugging_mvcc_events_total
# name: "etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds"
# description: "Bucketed histogram of index compaction pause duration."
# type: "histogram"
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="0.5"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="2"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="4"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="8"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="16"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="32"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="64"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="128"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="256"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="512"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="1024"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_bucket{le="+Inf"}
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_sum
etcd_debugging_mvcc_index_compaction_pause_duration_milliseconds_count
# name: "etcd_debugging_mvcc_keys_total"
# description: "Total number of keys."
# type: "gauge"
etcd_debugging_mvcc_keys_total
# name: "etcd_debugging_mvcc_pending_events_total"
# description: "Total number of pending events to be sent."
# type: "gauge"
etcd_debugging_mvcc_pending_events_total
# name: "etcd_debugging_mvcc_put_total"
# description: "Total number of puts seen by this member."
# type: "counter"
etcd_debugging_mvcc_put_total
# name: "etcd_debugging_mvcc_range_total"
# description: "Total number of ranges seen by this member."
# type: "counter"
etcd_debugging_mvcc_range_total
# name: "etcd_debugging_mvcc_slow_watcher_total"
# description: "Total number of unsynced slow watchers."
# type: "gauge"
etcd_debugging_mvcc_slow_watcher_total
# name: "etcd_debugging_mvcc_txn_total"
# description: "Total number of txns seen by this member."
# type: "counter"
etcd_debugging_mvcc_txn_total
# name: "etcd_debugging_mvcc_watch_stream_total"
# description: "Total number of watch streams."
# type: "gauge"
etcd_debugging_mvcc_watch_stream_total
# name: "etcd_debugging_mvcc_watcher_total"
# description: "Total number of watchers."
# type: "gauge"
etcd_debugging_mvcc_watcher_total
# name: "etcd_debugging_snap_save_marshalling_duration_seconds"
# description: "The marshalling cost distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_marshalling_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_marshalling_duration_seconds_sum
etcd_debugging_snap_save_marshalling_duration_seconds_count
# name: "etcd_debugging_snap_save_total_duration_seconds"
# description: "The total latency distributions of save called by snapshot."
# type: "histogram"
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.001"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.002"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.004"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.008"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.016"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.032"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.064"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.128"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.256"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="0.512"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="1.024"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="2.048"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="4.096"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="8.192"}
etcd_debugging_snap_save_total_duration_seconds_bucket{le="+Inf"}
etcd_debugging_snap_save_total_duration_seconds_sum
etcd_debugging_snap_save_total_duration_seconds_count
# name: "etcd_debugging_store_expires_total"
# description: "Total number of expired keys."
# type: "counter"
etcd_debugging_store_expires_total
# name: "etcd_debugging_store_reads_total"
# description: "Total number of reads action by (get/getRecursive), local to this member."
# type: "counter"
etcd_debugging_store_reads_total{action="getRecursive"}
# name: "etcd_debugging_store_watch_requests_total"
# description: "Total number of incoming watch requests (new or reestablished)."
# type: "counter"
etcd_debugging_store_watch_requests_total
# name: "etcd_debugging_store_watchers"
# description: "Count of currently active watchers."
# type: "gauge"
etcd_debugging_store_watchers
# name: "etcd_debugging_store_writes_total"
# description: "Total number of writes (e.g. set/compareAndDelete) seen by this member."
# type: "counter"
etcd_debugging_store_writes_total{action="create"}
etcd_debugging_store_writes_total{action="set"}
# name: "etcd_disk_backend_commit_duration_seconds"
# description: "The latency distributions of commit called by backend."
# type: "histogram"
etcd_disk_backend_commit_duration_seconds_bucket{le="0.001"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.002"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.004"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.008"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.016"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.032"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.064"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.128"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.256"}
etcd_disk_backend_commit_duration_seconds_bucket{le="0.512"}
etcd_disk_backend_commit_duration_seconds_bucket{le="1.024"}
etcd_disk_backend_commit_duration_seconds_bucket{le="2.048"}
etcd_disk_backend_commit_duration_seconds_bucket{le="4.096"}
etcd_disk_backend_commit_duration_seconds_bucket{le="8.192"}
etcd_disk_backend_commit_duration_seconds_bucket{le="+Inf"}
etcd_disk_backend_commit_duration_seconds_sum
etcd_disk_backend_commit_duration_seconds_count
# name: "etcd_disk_wal_fsync_duration_seconds"
# description: "The latency distributions of fsync called by wal."
# type: "histogram"
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.001"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.002"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.004"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.008"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.016"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.032"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.064"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.128"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.256"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="0.512"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="1.024"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="2.048"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="4.096"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="8.192"}
etcd_disk_wal_fsync_duration_seconds_bucket{le="+Inf"}
etcd_disk_wal_fsync_duration_seconds_sum
etcd_disk_wal_fsync_duration_seconds_count
# name: "etcd_grpc_proxy_cache_hits_total"
# description: "Total number of cache hits"
# type: "gauge"
etcd_grpc_proxy_cache_hits_total
# name: "etcd_grpc_proxy_cache_misses_total"
# description: "Total number of cache misses"
# type: "gauge"
etcd_grpc_proxy_cache_misses_total
# name: "etcd_grpc_proxy_events_coalescing_total"
# description: "Total number of events coalescing"
# type: "counter"
etcd_grpc_proxy_events_coalescing_total
# name: "etcd_grpc_proxy_watchers_coalescing_total"
# description: "Total number of current watchers coalescing"
# type: "gauge"
etcd_grpc_proxy_watchers_coalescing_total
# name: "etcd_network_client_grpc_received_bytes_total"
# description: "The total number of bytes received from grpc clients."
# type: "counter"
etcd_network_client_grpc_received_bytes_total
# name: "etcd_network_client_grpc_sent_bytes_total"
# description: "The total number of bytes sent to grpc clients."
# type: "counter"
etcd_network_client_grpc_sent_bytes_total
# name: "etcd_network_peer_received_bytes_total"
# description: "The total number of bytes received from peers."
# type: "counter"
etcd_network_peer_received_bytes_total{From="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_round_trip_time_seconds"
# description: "Round-Trip-Time histogram between peers."
# type: "histogram"
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="+Inf"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0001"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0002"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0004"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0008"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0016"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0032"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0064"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0128"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0256"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.0512"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.1024"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.2048"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.4096"}
etcd_network_peer_round_trip_time_seconds_bucket{To="REMOTE_PEER_NODE_ID",le="0.8192"}
etcd_network_peer_round_trip_time_seconds_count{To="REMOTE_PEER_NODE_ID"}
etcd_network_peer_round_trip_time_seconds_sum{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_network_peer_sent_bytes_total"
# description: "The total number of bytes sent to peers."
# type: "counter"
etcd_network_peer_sent_bytes_total{To="REMOTE_PEER_NODE_ID"}
# name: "etcd_server_has_leader"
# description: "Whether or not a leader exists. 1 is existence, 0 is not."
# type: "gauge"
etcd_server_has_leader
# name: "etcd_server_is_leader"
# description: "Whether or not this member is a leader. 1 if is, 0 otherwise."
# type: "gauge"
etcd_server_is_leader
# name: "etcd_server_leader_changes_seen_total"
# description: "The number of leader changes seen."
# type: "counter"
etcd_server_leader_changes_seen_total
# name: "etcd_server_proposals_applied_total"
# description: "The total number of consensus proposals applied."
# type: "gauge"
etcd_server_proposals_applied_total
# name: "etcd_server_proposals_committed_total"
# description: "The total number of consensus proposals committed."
# type: "gauge"
etcd_server_proposals_committed_total
# name: "etcd_server_proposals_failed_total"
# description: "The total number of failed proposals seen."
# type: "counter"
etcd_server_proposals_failed_total
# name: "etcd_server_proposals_pending"
# description: "The current number of pending proposals to commit."
# type: "gauge"
etcd_server_proposals_pending
# name: "go_gc_duration_seconds"
# description: "A summary of the GC invocation durations."
# type: "summary"
go_gc_duration_seconds{quantile="0"}
go_gc_duration_seconds{quantile="0.25"}
go_gc_duration_seconds{quantile="0.5"}
go_gc_duration_seconds{quantile="0.75"}
go_gc_duration_seconds{quantile="1"}
go_gc_duration_seconds_sum
go_gc_duration_seconds_count
# name: "go_goroutines"
# description: "Number of goroutines that currently exist."
# type: "gauge"
go_goroutines
# name: "go_memstats_alloc_bytes"
# description: "Number of bytes allocated and still in use."
# type: "gauge"
go_memstats_alloc_bytes
# name: "go_memstats_alloc_bytes_total"
# description: "Total number of bytes allocated, even if freed."
# type: "counter"
go_memstats_alloc_bytes_total
# name: "go_memstats_buck_hash_sys_bytes"
# description: "Number of bytes used by the profiling bucket hash table."
# type: "gauge"
go_memstats_buck_hash_sys_bytes
# name: "go_memstats_frees_total"
# description: "Total number of frees."
# type: "counter"
go_memstats_frees_total
# name: "go_memstats_gc_sys_bytes"
# description: "Number of bytes used for garbage collection system metadata."
# type: "gauge"
go_memstats_gc_sys_bytes
# name: "go_memstats_heap_alloc_bytes"
# description: "Number of heap bytes allocated and still in use."
# type: "gauge"
go_memstats_heap_alloc_bytes
# name: "go_memstats_heap_idle_bytes"
# description: "Number of heap bytes waiting to be used."
# type: "gauge"
go_memstats_heap_idle_bytes
# name: "go_memstats_heap_inuse_bytes"
# description: "Number of heap bytes that are in use."
# type: "gauge"
go_memstats_heap_inuse_bytes
# name: "go_memstats_heap_objects"
# description: "Number of allocated objects."
# type: "gauge"
go_memstats_heap_objects
# name: "go_memstats_heap_released_bytes_total"
# description: "Total number of heap bytes released to OS."
# type: "counter"
go_memstats_heap_released_bytes_total
# name: "go_memstats_heap_sys_bytes"
# description: "Number of heap bytes obtained from system."
# type: "gauge"
go_memstats_heap_sys_bytes
# name: "go_memstats_last_gc_time_seconds"
# description: "Number of seconds since 1970 of last garbage collection."
# type: "gauge"
go_memstats_last_gc_time_seconds
# name: "go_memstats_lookups_total"
# description: "Total number of pointer lookups."
# type: "counter"
go_memstats_lookups_total
# name: "go_memstats_mallocs_total"
# description: "Total number of mallocs."
# type: "counter"
go_memstats_mallocs_total
# name: "go_memstats_mcache_inuse_bytes"
# description: "Number of bytes in use by mcache structures."
# type: "gauge"
go_memstats_mcache_inuse_bytes
# name: "go_memstats_mcache_sys_bytes"
# description: "Number of bytes used for mcache structures obtained from system."
# type: "gauge"
go_memstats_mcache_sys_bytes
# name: "go_memstats_mspan_inuse_bytes"
# description: "Number of bytes in use by mspan structures."
# type: "gauge"
go_memstats_mspan_inuse_bytes
# name: "go_memstats_mspan_sys_bytes"
# description: "Number of bytes used for mspan structures obtained from system."
# type: "gauge"
go_memstats_mspan_sys_bytes
# name: "go_memstats_next_gc_bytes"
# description: "Number of heap bytes when next garbage collection will take place."
# type: "gauge"
go_memstats_next_gc_bytes
# name: "go_memstats_other_sys_bytes"
# description: "Number of bytes used for other system allocations."
# type: "gauge"
go_memstats_other_sys_bytes
# name: "go_memstats_stack_inuse_bytes"
# description: "Number of bytes in use by the stack allocator."
# type: "gauge"
go_memstats_stack_inuse_bytes
# name: "go_memstats_stack_sys_bytes"
# description: "Number of bytes obtained from system for stack allocator."
# type: "gauge"
go_memstats_stack_sys_bytes
# name: "go_memstats_sys_bytes"
# description: "Number of bytes obtained by system. Sum of all system allocations."
# type: "gauge"
go_memstats_sys_bytes
# name: "grpc_server_handled_total"
# description: "Total number of RPCs completed on the server, regardless of success or failure."
# type: "counter"
# gRPC codes:
# - "Canceled"
# - "OK"
grpc_server_handled_total{grpc_code="CODE",grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_handled_total{grpc_code="CODE",grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_received_total"
# description: "Total number of RPC stream messages received on the server."
# type: "counter"
grpc_server_msg_received_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_received_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_msg_sent_total"
# description: "Total number of gRPC stream messages sent by the server."
# type: "counter"
grpc_server_msg_sent_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_msg_sent_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "grpc_server_started_total"
# description: "Total number of RPCs started on the server."
# type: "counter"
grpc_server_started_total{grpc_method="DeleteRange",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Put",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Range",grpc_service="etcdserverpb.KV",grpc_type="unary"}
grpc_server_started_total{grpc_method="Watch",grpc_service="etcdserverpb.Watch",grpc_type="bidi_stream"}
# name: "http_request_duration_microseconds"
# description: "The HTTP request latencies in microseconds."
# type: "summary"
http_request_duration_microseconds{handler="prometheus",quantile="0.5"}
http_request_duration_microseconds{handler="prometheus",quantile="0.9"}
http_request_duration_microseconds{handler="prometheus",quantile="0.99"}
http_request_duration_microseconds_sum{handler="prometheus"}
http_request_duration_microseconds_count{handler="prometheus"}
# name: "http_request_size_bytes"
# description: "The HTTP request sizes in bytes."
# type: "summary"
http_request_size_bytes{handler="prometheus",quantile="0.5"}
http_request_size_bytes{handler="prometheus",quantile="0.9"}
http_request_size_bytes{handler="prometheus",quantile="0.99"}
http_request_size_bytes_sum{handler="prometheus"}
http_request_size_bytes_count{handler="prometheus"}
# name: "http_response_size_bytes"
# description: "The HTTP response sizes in bytes."
# type: "summary"
http_response_size_bytes{handler="prometheus",quantile="0.5"}
http_response_size_bytes{handler="prometheus",quantile="0.9"}
http_response_size_bytes{handler="prometheus",quantile="0.99"}
http_response_size_bytes_sum{handler="prometheus"}
http_response_size_bytes_count{handler="prometheus"}

Some files were not shown because too many files have changed in this diff Show More