Commit Graph

579 Commits

Author SHA1 Message Date
0d2dd291bd tools: rw-heatmaps output format bug fix 2021-06-06 14:58:49 -07:00
8a0142c8b1 server: enable getting raft request from cache to avoid extra unmarshalling 2021-06-05 17:37:01 -07:00
852fae73ae Merge pull request #13082 from wilsonwang371/main_stable
tools: update mixed read/write perf testing script for a larger range of value size
2021-06-04 13:29:31 +02:00
fe2c05275b tools: update mixed read/write perf testing script for a larger range of value size 2021-06-03 22:46:47 -07:00
af6f3dcba5 Merge pull request #13074 from wilsonwang371/rw-bm-bugfix
tools: fix small rw-heatmaps python plot script argument parsing bug
2021-06-02 13:59:48 -07:00
d3ae6f25dd tools: fix small rw-heatmaps python plot script argument parsing bug
(cherry picked from commit 5b6b3ffe0b9fca7c72edfc2218e98357d1152872)
2021-06-02 13:32:02 -07:00
c7cbc6befa Merge pull request #13027 from lilic/benchmark-readme
tools/benchmark/README.md: Update installation as go get does not work
2021-05-31 10:37:49 +02:00
312908c068 tools/benchmark/README.md: Update installation as go get does not work 2021-05-31 10:25:37 +02:00
8389ab8751 tool: save test parameters in mixed read/write benchmark script. 2021-05-26 20:38:49 -07:00
2ab2b4035c tools: no hardcoded repeat times in rw-benchmark scripts 2021-05-25 15:43:46 -07:00
db8ce62984 tools: update mixed read/write benchmark script 2021-05-25 13:30:56 -07:00
79b2777482 tools: add mixed read-write performance evaluation scripts 2021-05-25 12:03:42 -07:00
66752fef2f Represent bucket as object instead of []byte name.
Thanks to this change:
  - all the maps bucket -> buffer are indexed by int's instead of
string. No need to do: byte[] -> string -> hash conversion on each
access.
  - buckets are strongly typed in backend/mvcc API.
2021-05-18 18:58:53 +02:00
6bab63e96b tools: rename "master" branch references to "main"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2021-05-12 10:39:53 -07:00
a78d072b9a Simplify KVstore dependency on cindex. 2021-05-04 18:21:24 +02:00
ffea1537d4 ClientV3 tests use integration.NewClient that configures proper logger. 2021-04-29 18:18:34 +02:00
aa6597384b etcd-dump-logs: Print full confState as json for debugging purposes. 2021-04-27 17:17:31 +02:00
3423a949c0 Update go for 3.5: 1.15 -> 1.16.(3).
https://github.com/etcd-io/etcd/issues/12732
2021-04-19 16:50:54 +02:00
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
19f7c6ef3e *: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
2021-03-23 11:48:06 -06:00
9312d1b077 clientv3: Bring back ETCD_CLIENT_DEBUG variable interpretation.
env ETCD_CLIENT_DEBUG supports log levels (debug, info, warn, error, dpanic, panic, fatal).
Only when set, overrides application-wide grpc logging settings.
2021-03-18 18:43:07 +01:00
a7f340216d Reformat code according to 'gotip' rules.
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
2021-02-26 10:14:46 +01:00
598ca6caab Modernize release script:
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
  - better interaction with ./test_lib.sh script.
  - more consistent logging
  - bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
2021-01-21 17:47:36 +01:00
577c898fee scripts: Integrate ./scripts/release with new code for tagging modules.
Changes:
  - signing tags.
  - allows to override BRANCH and REPOSITORY using env variables.

Tested by a release in my private fork:
  BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
2021-01-15 12:31:44 +01:00
3eea37cf31 tools/etcd-dump-metrics: validate exec cmd args
To prevent arbitrary command invocations.

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2020-11-25 10:37:52 -08:00
aaf423e962 server: Update imports.
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/|etcd/server/v3/|g'
2020-10-26 13:02:32 +01:00
0ba16d8ee1 *: Convert tabulators to whitespaces in bash scripts.
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t|  |g'
```
2020-10-26 10:59:40 +01:00
e33c6dd9df client/v3: Rename of imports 2020-10-20 10:13:06 +02:00
e62417297d *: Rename of imports of raft (as its now a module)
% find -name '*.go' -o -name '*.md' -o -name '*.sh' | xargs sed -i --follow-symlinks 's|etcd/v3/raft|etcd/raft/v3|g'
2020-10-16 13:58:18 +02:00
b7f0f52a16 *: Refresh of dependencies (#12399)
This PR focuses on dependencies that have new stable versions.
2020-10-15 15:32:00 -07:00
da5ca2c0d5 tools: Update of go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6)
This brings consistency between proto-generation code and actual versions of libraries being used in runtime:

github.com/gogo/protobuf                        v1.2.1,v1.0.0 -> v1.3.1
github.com/golang/protobuf                             v1.3.2 -> v1.3.5
github.com/grpc-ecosystem/grpc-gateway  v1.9.5,v1.4.1,v1.15.2 -> v1.14.6
google.golang.org/grpc                                v1.26.0 -> v1.29.1

Moved as far as possible, without bumping on grpc 1.30.0 "naming" decomissioning.
Please also notice that gogo/protobuf is likely to reach EOL: https://github.com/gogo/protobuf/issues/691
2020-10-14 18:46:38 +02:00
bc9e433ca2 tools: Migrate remaining tools to gobin
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.

Side changes:
  - Expose /scripts/fix.sh that fixes formatting and bom across modules
  - Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
  - Make stderr output of commands explicit and make commands use
different color than callouts.
2020-10-13 19:33:01 +02:00
de55bb6331 pkg: Rename imports after making 'pkg' a module
find -name '*.go' | xargs sed --follow-symlinks -i 's|go.etcd.io/etcd/v3/pkg/|go.etcd.io/etcd/pkg/v3/|g'
go fmt ./...
2020-10-13 00:09:27 +02:00
dfdda47bd8 script/genproto.sh: Refactor to be explicit about versions.
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
  - https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
  - uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
  - caches tools between executions
  - guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)

The change is no-op for the generated code.

The commit reveals a few 'worring things':
  1  We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
  2. And also     : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
  3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
2020-10-08 19:52:27 +02:00
28f2b07623 *: Update references to code moved to the api/ dir.
Follow up to file-moves done in the previous commit.

The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):

  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'

  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'

  I also modified manually paths in scripts/genproto.sh.

  % go fmt ./...
2020-10-06 11:56:16 +02:00
205a656cc5 Merge pull request #11853 from viviyww/dev1
tools: fix test case errors in etcd-dump-logs
2020-09-21 09:13:31 -07:00
c88d1497ee tools: fix test case errors in etcd-dump-logs
Fix test case errors in etcd-dump-logs and the error is the time zone.
When the GOST time zone is CST or PST and the test case will be failed.
So we should set UTC as the standard time zone.
2020-09-21 15:54:47 +08:00
96cce208c2 go.mod: use go.etcd.io/etcd/v3 versioning
This change makes the etcd package compatible with the existing Go
ecosystem for module versioning.

Used this tool to update package imports:
  https://github.com/KSubedi/gomove
2020-04-28 00:57:35 +00:00
08a8b80e32 tools/etcd-dump-db: add auth decoder, optimize print format 2020-02-29 13:31:44 +08:00
725e09023a *: set zap as default logger, remove capnslog
Set zap as default logger. Remove capnslog and deprecated logging
flags.
2020-02-04 04:57:49 -08:00
6ffd9f659e tools/benchmark: exit when txn-ops is larger than key-space-size to avoid key duplicated error. (#11499) 2020-01-06 16:36:51 -08:00
57aa68af5a etcdserver: trace compaction request; add return parameter 'trace' to applierV3.Compaction()
mvcc: trace compaction request; add input parameter 'trace' to KV.Compact()
2019-10-07 09:55:27 -07:00
e98479ddfe tools/etcd-dump-logs: Fixed test cases
Fixed failing test cases due to changes in previous commit
2019-09-06 20:11:41 +01:00
4e551d06ed tools/etcd-dump-logs: Fixed default values for -entry-type flag
The tool takes default values but it was not visible which default
values were taken. Added default values in proper place, and added a
newline at the end of output.
2019-09-06 20:11:41 +01:00
9b51febaf5 *: Add experimental-compaction-batch-limit flag 2019-08-15 11:47:23 -07:00
b5ee1de076 cleanup: correct summary message in put.go 2019-08-12 21:13:58 +08:00
867b31e01a etcd-dump-logs: correct logging message word
Signed-off-by: Guangming Wang <guangming.wang@daocloud.io>
2019-08-10 17:01:57 +08:00
3fc62ca586 tools,Documentation: move "etcd-dump-metrics" output to "Documentation"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2019-07-28 21:53:50 -07:00
b9c051e7a7 raftpb: clean up naming in ConfChange 2019-07-23 10:40:03 +02:00
5a67dd788d *: support creating a user without password
This commit adds a feature for creating a user without password. The
purpose of the feature is reducing attack surface by configuring bad
passwords (CN based auth will be allowed for the user).

The feature can be used with `--no-password` of `etcdctl user add`
command.

Fix https://github.com/coreos/etcd/issues/9590
2019-05-30 21:59:30 +09:00