mirrors/etcd
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,759 Commits 124 Branches 418 Tags 106 MiB
main
Commit Graph

440 Commits

Author SHA1 Message Date
Marek Siarkowicz
b8347edf3b makefile: Split fmt into separate verify commands 
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
 2022-09-16 22:37:10 +02:00
Bogdan Kanivets
c31d758189 functional: remove SIGQUIT_ETCD_AND_REMOVE_DATA_AND_STOP_AGENT command 
Problem: both SIGQUIT_ETCD_AND_REMOVE_DATA_AND_STOP_AGENT and test.sh
will attempt to stop agents and remove directories.

Solution: since test.sh creates directories and starts test, it should be
responsible for cleanup.

See https://github.com/etcd-io/etcd/issues/14384

Signed-off-by: Bogdan Kanivets <bkanivets@apple.com>
 2022-09-15 10:55:32 -07:00
Benjamin Wang
d8ee37641d Cleanup all etcd-agent and etcd processes when the test finishes 
There are two cases, when interrupted by users, then forcibly kill
all processes. Otherwise, gracefully terminate all processes.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-08-18 17:15:39 +08:00
Benjamin Wang
7400d6c363 Update scripts/release.sh to remove message related to travis.yml 
Signed-off-by: Benjamin Wang <wachao@vmware.com>
 2022-07-11 10:31:51 +08:00
Eduardo Patrocinio
5a93b087a6 Separate the build library functions and add a top level tools script 
The current Makefile doesn't allow the compilation of the tools directory.
This commit creates a build library file, updates the Makefile
and a top level script fod building tools.

To build the tools, you can run make build_tools.

As before, you can run make build to build etcd binaries.
 2022-06-10 11:43:10 -04:00
Eduardo Patrocinio
73766a92fb Update build file to run with an argument 
The current Makefile doesn't allow the compilation of the tools directory.
This commit updates the Makefile and the build file to add an option
to build the tools.

To build the tools, you can run make build_tools.
 2022-06-07 11:47:28 -04:00
Marek Siarkowicz
63e47656e0 scripts: Detect staged files before building release 2022-05-18 12:21:43 +02:00
Marek Siarkowicz
e737f4296c scripts: Avoid additional repo clone 
This PR removes additional clone when building artifacts.

When releasing v3.5.4 this clone was main cause of issues and
confusion about what release script is doing.

release.sh script already clones repo in /tmp/ directory, so clonning
before build is not needed. As precautions for bug in script leaving
/tmp/ clone in bad state  I moved "Verify the latest commit has the
version tag" and added "Verify the clean working tree" to be always run
before build.
 2022-05-16 14:37:47 +02:00
Marek Siarkowicz
5472e589cf scripts: Add tests for release scripts 2022-04-25 10:12:01 +02:00
Piotr Tabor
8d36956d0e
Merge pull request #13955 from ptabor/20220417-tests-improvements 
test.sh script improvement (OsX centric)
 2022-04-19 12:18:45 +02:00
Vimal Kumar
12ab72a6ff smoke test:fix license header check 2022-04-19 08:35:22 +05:30
Piotr Tabor
8a22d640a7 add OUTPUT_FILE='foo.log' env variable support to test.sh script. 
Side consequence of this is that `make test-smoke` can become interactive again.
 2022-04-18 22:54:51 +02:00
Piotr Tabor
55a7c91de9 test.sh: Tools run correctly on OsX 
- The location of installed tools is found using `go list`
- ETCD_VERIFY env variable is printed in repro command line.
 2022-04-17 23:28:26 +02:00
Piotr Tabor
cf2d98cc88 Fix PASSES="build" on OSX 
`readlink -f` is supported since OsX 12.3.

```
% env GO_BUILD_FLAGS=-v CGO_ENABLED=0 go build -v -installsuffix=cgo -ldflags='-X=go.etcd.io/etcd/api/v3/version.GitSHA=4ac59dbe3' -o=bin/tools/local-tester/bridge ./tools/local-tester/bridge
readlink: illegal option -- m
usage: readlink [-fn] [file ...]
mkdir: /functional: Read-only file system
% ./tests/functional/build.sh
stderr: go build go.etcd.io/etcd/tests/v3/functional/cmd/etcd-agent: mkdir /functional: read-only file system
stderr: go build go.etcd.io/etcd/tests/v3/functional/cmd/etcd-proxy: mkdir /functional: read-only file system
stderr: go build go.etcd.io/etcd/tests/v3/functional/cmd/etcd-runner: mkdir /functional: read-only file system
stderr: go test go.etcd.io/etcd/tests/v3/functional/cmd/etcd-tester.test: mkdir /functional: read-only file system
```
 2022-04-17 23:21:37 +02:00
Marek Siarkowicz
98764fadef scripts: Remove Windows build scripts 2022-04-12 10:34:10 +02:00
Marek Siarkowicz
ae57fe5d30
Merge pull request #13885 from serathius/verify 
server: Add verification of whether lock was called within out outside of apply
 2022-04-05 16:22:52 +02:00
Marek Siarkowicz
1d3517020b server: Add verification of whether lock was called within out outside of apply 2022-04-05 15:34:45 +02:00
Marek Siarkowicz
8d8271f6d1
Merge pull request #13175 from karuppiah7890/issue-13167-measure-flakyness 
scripts: add script to measure percentage of commits with failed status
 2022-04-05 15:25:47 +02:00
Piotr Tabor
d4dcd3061d Fix flakes in TestV3LeaseCheckpoint/Checkpointing_disabled,_lease_TTL_is_reset 
I think strong (not-equal) relationship was too restrictive when expressed with 1s granularity.

```
        logger.go:130: 2022-04-03T22:15:15.242+0200	WARN	m1	leader failed to send out heartbeat on time; took too long, leader is overloaded likely from slow disk	{"member": "m1", "to": "cb785755eb80ac1", "heartbeat-interval": "10ms", "expected-duration": "20ms", "exceeded-duration": "24.666613ms"}
        logger.go:130: 2022-04-03T22:15:15.262+0200	INFO	m-1	published local member to cluster through raft	{"member": "m-1", "local-member-id": "e2dd9f523aa7be87", "local-member-attributes": "{Name:m-1 ClientURLs:[unix://127.0.0.1:2196386040]}", "cluster-id": "b4b8e7e41c23c8b5", "publish-timeout": "5.2s"}
        v3_lease_test.go:415: Expected lease ttl (4m58s) to be greather than (4m58s)
```
 2022-04-03 23:13:01 +02:00
Piotr Tabor
90796720c1 Reduce integration test parallelism to 2 packages at once. 
Especially with 'race' detection, running O(cpu) integrational tests was causing CPU overloads and timeouts.
 2022-04-03 14:48:36 +02:00
ahrtr
edce939f6e add one more field storageVersion into StatusResponse 
When performing the downgrade operation, users can confirm whether each member
is ready to be downgraded using the field 'storageVersion'. If it's equal to the
'target version' in the downgrade command, then it's ready to be downgraded;
otherwise, the etcd member is still in progress of processing the db file.
 2022-03-18 07:04:44 +08:00
cuishuang
b7824b794b fix some typos 
Signed-off-by: cuishuang <imcusg@gmail.com>

fix some typos

Signed-off-by: cuishuang <imcusg@gmail.com>
 2022-03-07 20:28:53 +08:00
Marek Siarkowicz
778c95fdf4 scripts: Fix release scripts 2022-02-25 13:30:34 +01:00
Marek Siarkowicz
744dd077cd tests: Create common framework for e2e and integration tests and migrate TestKVPut test 2022-02-23 16:47:02 +01:00
sitchenchao
b683aa1254 fix ctl_v3_kv_no_quorum_test.go and coverage failures 2022-02-12 18:00:02 -08:00
Sahdev Zala
bba3937f69
Merge pull request #13649 from serathius/scripts 
Cleanup scripts in root directory
 2022-02-11 15:40:53 -05:00
ahrtr
900fc8dd7e replace gobin with go install 
The repository github.com/myitcv/gobin has already been archived,
and the `go install` command accepts arguments with version suffixs
starting from 1.16 (for example, go install example.com/cmd@v1.0.0).
So there is no reason to continue to use gobin.
 2022-02-08 05:41:21 +08:00
Marek Siarkowicz
396efd9608 Cleanup scripts in root directory 2022-02-02 13:50:44 +01:00
Sahdev Zala
1986357b4c
Merge pull request #13663 from serathius/post-3.5.2 
release: Post v3.5.2 release updates and fixes
 2022-02-01 18:45:00 -05:00
Marek Siarkowicz
aa76b32a05 release: Post v3.5.2 release updates and fixes 2022-02-01 15:36:45 +01:00
Marek Siarkowicz
3df14fc24f Document proto annotations scripts 2022-01-28 16:08:28 +01:00
Marek Siarkowicz
6d808e5d7d *: Add static validation to etcd_version proto annotation 2022-01-26 15:50:14 +01:00
Dirkjan Bussink
ddb9554eec
server/etcdmain: add build support for Apple M1 
This has been additionally verified by running the tests locally as a
basic smoke test. GitHub Actions doesn't provide MacOS M1 (arm64) yet,
so there's no good way to automate testing.

Ran `TMPDIR=/tmp make test` locally. The `TMPDIR` bit is needed so
there's no really long path used that breaks Unix socket setup in one of
the tests.
 2021-12-17 17:35:36 +01:00
Chao Chen
9038e65359 scripts/genproto.sh: fix module path of wal proto files 2021-10-15 22:00:16 -07:00
Marek Siarkowicz
1b4e54c238 api: Annotate proto messages and use it to detect etcd version that generated wal 
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
 2021-08-04 13:36:08 +02:00
Karuppiah Natarajan
2b796a0c9a
scripts: add script to measure percentage of commits with failed status 
This is to start measuring the test flakiness and see the numbers improving once we improve and deflake flaky tests

Fixes #13167
 2021-07-13 18:42:06 +05:30
Marek Siarkowicz
4c5efc970d *: Upload test junit results 2021-07-01 14:41:13 +02:00
Rajalakshmi-Girish
8bfce5dc01 Add option to generate junit xml reports 2021-06-16 07:25:50 +00:00
Piotr Tabor
b00803a9c1
Merge pull request #13071 from serathius/genproto 
tests: Switch regenerating functional/rpcpb proto to scripts/genproto.sh
 2021-06-04 13:30:02 +02:00
Nate W
9e4dd4dc4e updating scripts/genproto.sh 
Changing API-reference generator

- matching changes mentioned in https://github.com/etcd-io/website/issues/330
- generating page frontmater
- updating .gitignore

Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
 2021-06-02 10:29:35 -07:00
Marek Siarkowicz
af5cf1ac1c tests: Switch regenerating functional/rpcpb proto to scripts/genproto.sh 2021-06-02 16:42:35 +02:00
Sam Batschelet
80ccb27ea5
Merge pull request #12997 from hexfusion/fix-etcdutl-build 
scripts: add missing etcdutl to release pipeline
 2021-05-18 07:36:22 -04:00
Sam Batschelet
86393ecfcd scripts: add missing etcdutl to release pipeline 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2021-05-18 07:16:28 -04:00
Piotr Tabor
3f7a038656 Fixed documentation and scripts. 2021-05-17 11:54:03 +02:00
Piotr Tabor
c09aca1ba4 Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files) 
Motivation is as follows:

  - etcdctl we only depend on clientv3 APIs, no dependencies of bolt, backend, mvcc, file-layout
  - etcdctl can be officially supported across wide range of versions, while etcdutl is pretty specific to file format at particular version.
it's step towards desired modules layout, documented in: https://etcd.io/docs/next/dev-internal/modules/
 2021-05-17 11:54:03 +02:00
Piotr Tabor
2712c787ab Fix release script: add go mod tidy 
Seems that additinal refresh of go.sum files is needed during the
release process after editing the versions of modules.

Discovered the problem while I was testing whether etcdutl is
properly included in the target binaries using:

```
DRY_RUN=true REPOSITORY=\`pwd\` BRANCH='branch-with-etcdutl' ./scripts/release 3.5.0-foobar.2
```
 2021-05-16 20:43:12 +02:00
Gyuho Lee
3d4629c1fb hack,scripts: rename "master" branch references to "main" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-12 10:40:28 -07:00
Gyuho Lee
3a04b2b61f github action: add ARM64 test with README 
README for self-hosted-linux-arm64-graviton2-tests.yml

Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2021-05-10 00:28:41 -07:00
Piotr Tabor
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
Piotr Tabor
f290ab2e60 Update dependecies: 
github.com/grpc-ecosystem/grpc-gateway v1.14.6 -> grpc-gateway v1.16.0
  golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e->v0.0.0-20210220033141-f8bda1e9f3ba
 2021-03-27 20:48:33 +01:00
Piotr Tabor
7665bd764e scripts/update_dep.sh: Script to update selected versions of dependencies. 2021-03-27 11:17:56 +01:00
Piotr Tabor
9180e8e56e wal: Update WAL Snapshot format to contain ConfState. 2021-03-25 00:31:44 +01:00
Joel Smith
19f7c6ef3e *: Update gogo/protobuf to v1.3.2, rerun ./scripts/genproto.sh 
While it appears that etcd is not vulnerable to CVE-2021-3121,
it is a good idea to update to the new generator so that new
vulnerable code isn't generated in any future APIs. Also, this
lays the issue to rest of whether there is any issue with
etcd and CVE-2021-3121.
 2021-03-23 11:48:06 -06:00
Piotr Tabor
a84bd093b0 Integration with grpc-settable logger. 2021-03-16 22:50:41 +01:00
Piotr Tabor
f7a2389992 Update version of certifi/gocertifi to get rid of WTF Public license 
Seems old versions of https://github.com/certifi/gocertifi where
categorized as "Do What The F*ck You Want To Public License".

Update to newer version that is explicit `Mozilla Public License` 2.0 (MPL 2.0).
 2021-03-04 09:48:34 +01:00
Piotr Tabor
a7f340216d Reformat code according to 'gotip' rules. 
In practices adds annotations in the new syntax:
```
+//go:build !linux
 // +build !linux
```

Fixes failing gotip PASSES='fmt' check:
https://travis-ci.com/github/etcd-io/etcd/jobs/486453806
 2021-02-26 10:14:46 +01:00
Piotr Tabor
60a669762f Improve release scripts: 
- Fix script that creates manifest-list based multi-arch-images.
    The images need to be pushed first.
  - Use docker instead of gcloud docker helper
  - Make sure docker pushes are properly 'dry run'
  - Added preparation instruction to the release script.
 2021-02-25 12:28:52 +00:00
Piotr Tabor
c640957e2d Release scripts: Minor fixes discovered during attempt for 3.5.0-alpha.0 2021-02-24 19:16:27 +00:00
Piotr Tabor
1a9c81abda Update grpc dependency to 1.32. 
Simplify grpc testing infrastructure to align with upstream changes.
 2021-02-23 11:31:50 +01:00
Sahdev Zala
6bbef10721 Clarify the root directory comment 
A contributor had an issue when ran script changing the directory
name after clone. This was discussed on the etcd slack channel.
 2021-02-01 15:41:36 -05:00
Piotr Tabor
90d1b838ad Fix shellcheck warnings: Escaping. 
```
build:3:65: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
test:3:64: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:46: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:30:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:38:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:53:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:73:23: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:73:33: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:74:69: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:74:78: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
scripts/genproto.sh:75:40: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:75:52: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:91:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:115:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
./test.sh:71:29: warning: Quote to prevent word splitting, or split robustly with mapfile or read -a. [SC2206]
./test.sh:400:28: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:406:33: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:415:36: note: Backslash is literal in "\*". Prefer explicit escaping: "\\*". [SC1117]
./test.sh:415:176: note: Backslash is literal in "\'". Prefer explicit escaping: "\\'". [SC1117]
./test.sh:493:56: note: Backslash is literal in "\.". Prefer explicit escaping: "\\.". [SC1117]
./test.sh:601:57: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:64: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:67: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:74: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:77: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:84: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:91: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
./test.sh:601:94: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
./test.sh:601:103: note: Backslash is literal in "\3". Prefer explicit escaping: "\\3". [SC1117]
```
 2021-01-29 22:32:43 +00:00
Piotr Tabor
88ee005a0f Reduce verbosity of coverage collection. 2021-01-23 08:59:19 +01:00
Piotr Tabor
598ca6caab Modernize release script: 
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
  - better interaction with ./test_lib.sh script.
  - more consistent logging
  - bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
 2021-01-21 17:47:36 +01:00
Gyuho Lee
1d73d2e85e
Merge pull request #12632 from ptabor/20210119-fix-coverage-report 
Fix code coverage measurement
 2021-01-19 10:19:39 -08:00
Piotr Tabor
486687843b Fix code coverage measurement. 2021-01-19 15:41:36 +01:00
Angus Lees
2529fee958 Use /build instead of deprecated /build.sh 2021-01-19 12:46:56 +11:00
Angus Lees
7f79ffb409 Generate and push multi-arch docker manifest list 
Change amd64 docker image to use a `-amd64` suffix, consistent with
other architectures.  Introduce a multiarch docker manifest list with
the original (no-arch) docker tag.

Fixes #10380
 2021-01-19 12:46:56 +11:00
Piotr Tabor
577c898fee scripts: Integrate ./scripts/release with new code for tagging modules. 
Changes:
  - signing tags.
  - allows to override BRANCH and REPOSITORY using env variables.

Tested by a release in my private fork:
  BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
 2021-01-15 12:31:44 +01:00
Piotr Tabor
84b5b87fb2 Make test.sh scripts OSX/BSD compatible: 
- build & test scripts deprecated. Call *.sh variants.
  This will avoid delete the symlinks and get rid of
  subtle dependency on 'sed --follow-symlinks' on OsX/BSD sed.

- Fix parameters to mktemp
 2021-01-12 16:29:00 +01:00
Piotr Tabor
23340bb62a Refresh proto generation script after moving modules files. 
With modulatiozation server protos get moved into ./server directory,
but it was not reflected in scripts/genproto.sh.
 2021-01-08 16:33:12 +01:00
Piotr Tabor
fd2f34fd13 Release: Scripts to change versions in all go.mod files and push tags upstream. 
Examplar invocations:

Edit go.mod files such that all etcd modules are pointing on given version:

```
% DRY_RUN=false TARGET_VERSION="v3.5.13" ./scripts/release_mod.sh update_versions
```

Tag latest commit with current version number for all the modules and push upstream:
```
% DRY_RUN=true REMOTE_REPO="origin" ./scripts/release_mod.sh push_mod_tags
```
 2020-11-04 15:16:36 +01:00
Piotr Tabor
6e800b9b01
20201103 no commit title check (#12447) 
* Turn off checking of format of commit message.

* scripts/fix.sh: Fix fixing whitespaces in *.sh scripts

Aparently there is a difference between:
  find ./ -print0 -name *.sh and
  find ./ -name *.sh -print0

* etcdserver unit tests: Do not call .Fatalf(...) from not test's goroutine.

Fixes following test failures:
https://travis-ci.com/github/etcd-io/etcd/jobs/425920416
```
% (cd server && go vet ./...)
stderr: # go.etcd.io/etcd/server/v3/etcdserver
stderr: etcdserver/server_test.go:1002:4: call to (*T).Fatalf from a non-test goroutine
stderr: etcdserver/server_test.go:1166:4: call to (*T).Fatalf from a non-test goroutine
FAIL: (code:2):
  % (cd server && go vet ./...)
FAIL: 'run go vet ./...' checking failed (!=0 return code)
FAIL: 'govet' failed at Tue Nov  3 04:07:47 UTC 2020
```
 2020-11-03 07:59:42 -08:00
CFC4N
2e55875cc7
scripts: install github.com/myitcv/gobin while gobin doesn't exist. 2020-10-30 20:58:00 +08:00
Gyuho Lee
7da5182f1d
Merge pull request #12422 from tangcong/fix-realpath 
scripts: fix realpath command not found in mac os
 2020-10-26 10:42:26 -07:00
tangcong
8277395e1b scripts: use manual scripts to replace realpath 2020-10-27 00:36:28 +08:00
Piotr Tabor
6c1efd6ba5 server: Update go.mod 2020-10-26 13:02:32 +01:00
Piotr Tabor
0ba16d8ee1 *: Convert tabulators to whitespaces in bash scripts. 
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t|  |g'
```
 2020-10-26 10:59:40 +01:00
Piotr Tabor
c035df5317 test: Detect indention done using tab (\t) in *.sh 2020-10-26 10:59:40 +01:00
Piotr Tabor
09679d29ad etcdctl: Rename of imports after making etcdctl a module. 
```
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/etcdctl|etcd/etcdctl/v3|g'
```
 2020-10-21 11:15:35 +02:00
Piotr Tabor
df48e499ea etcdctl: Make etcdctl a module (go.mod, LICENSE) 2020-10-20 12:07:27 +02:00
Piotr Tabor
cf795c0ae6 client/v3: Create and update go.mod 2020-10-20 10:09:12 +02:00
Piotr Tabor
bea5f379bc raft: Make raft a module 
Establishing a module: Creating a go.mod files
 2020-10-16 09:15:48 +02:00
Piotr Tabor
45e4306508 client: Move client to client/v2 as a module. 
We make v2 client code a module go.etcd.io/etcd/client/v2.

Pretty mechanical change that can be summarized as:

  mkdir client/v2
  cd client/v2 && git mod init go.etcd.io/etcd/client/v2

  git mv client/*.go client/v2/
  find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'

  + fixing changelog, bom, go.mod etc.
 2020-10-15 14:39:43 +02:00
Piotr Tabor
da5ca2c0d5 tools: Update of go.mod for protos (gogo/protobuf 1.3.1, grpc 1.29.1, grpc-gateway 1.14.6) 
This brings consistency between proto-generation code and actual versions of libraries being used in runtime:

github.com/gogo/protobuf                        v1.2.1,v1.0.0 -> v1.3.1
github.com/golang/protobuf                             v1.3.2 -> v1.3.5
github.com/grpc-ecosystem/grpc-gateway  v1.9.5,v1.4.1,v1.15.2 -> v1.14.6
google.golang.org/grpc                                v1.26.0 -> v1.29.1

Moved as far as possible, without bumping on grpc 1.30.0 "naming" decomissioning.
Please also notice that gogo/protobuf is likely to reach EOL: https://github.com/gogo/protobuf/issues/691
 2020-10-14 18:46:38 +02:00
Piotr Tabor
bc9e433ca2 tools: Migrate remaining tools to gobin 
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.

Side changes:
  - Expose /scripts/fix.sh that fixes formatting and bom across modules
  - Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
  - Make stderr output of commands explicit and make commands use
different color than callouts.
 2020-10-13 19:33:01 +02:00
Piotr Tabor
bfa91d48af pkg: Estabilish pkg as a module 2020-10-12 23:58:09 +02:00
Gyuho Lee
0b95e8cef1
Merge pull request #12374 from ptabor/20201009-new-genproto 
script/genproto.sh: Refactor to be explicit about versions.
 2020-10-12 14:25:43 -07:00
Piotr Tabor
00e49d0c10
build: Fix updating 'go.etcd.io/etcd/api/v3/version.GitSHA' + test (#12382) 
During move of code to 'api' (0aab02e7b5),
I overlooked that the ./build script is setting the version on the 'symbol'.

I added a code to ./build script that checks whether the symbol's are in sync.
 2020-10-09 17:32:15 -07:00
Piotr Tabor
dfdda47bd8 script/genproto.sh: Refactor to be explicit about versions. 
Refactoring script/genproto.sh around state-of-the-art techniques of
managing tooling in go:
  - https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module
  - uses https://github.com/myitcv/gobin instead of customly created gopath.proto dir
  - caches tools between executions
  - guaratees hermetics runs (it was not guaranteed for protoc_grpc_gateway that used latest)

The change is no-op for the generated code.

The commit reveals a few 'worring things':
  1  We depend on : github.com/grpc-ecosystem/grpc-gateway/@v/v1.4.1/protoc-gen-grpc-gateway
  2. And also     : github.com/grpc-ecosystem/grpc-gateway/@v/v1.15.0/protoc-gen-swagger/protoc-gen-swagger
  3. And on extremely old: github.com/gogo/protobuf@v1.0.0 protoc-gen-gofast that is out of sync with the library linked to binaries: github.com/gogo/protobuf@v1.2.1
 2020-10-08 19:52:27 +02:00
Piotr Tabor
313087f8c5 ./test,./scripts: Update go.sum and bill-of-materials generation logic (and refresh bom). 2020-10-07 15:38:39 +02:00
Piotr Tabor
7886ec8574 test: Modules: PASSES="integration" ./test understand new file locations 2020-10-07 15:38:39 +02:00
Piotr Tabor
4fb48b7c14 tests: Estabilishing module for tests 
For now it contains only e2e tests, but integration & functional will follow.
 2020-10-07 15:03:51 +02:00
Piotr Tabor
28f2b07623 *: Update references to code moved to the api/ dir. 
Follow up to file-moves done in the previous commit.

The commit contains purely mechanical consequences of execution (apart
of scripts/genproto.sh):

  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/v3rpc/rpctypes|v3/api/v3rpc/rpctypes|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/version|v3/api/version|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/mvcc/mvccpb|v3/api/mvccpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/etcdserverpb|v3/api/etcdserverpb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/etcdserver/api/membership/membershippb|v3/api/membershippb|g'
  % find ./ -name '*.go'  | xargs sed --follow-symlinks -i 's|v3/auth/authpb|v3/api/authpb|g'

  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/mvcc/mvccpb/kv.proto|/api/mvccpb/kv.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/auth/authpb/auth.proto|/api/authpb/auth.proto|g'
  % find ./ -name '*.proto' -o -name '*.md'  | xargs -L 1 sed --follow-symlinks -i 's|/etcdserver/api/membership/membershippb/membership.proto|/api/membershippb/membership.proto|g'

  I also modified manually paths in scripts/genproto.sh.

  % go fmt ./...
 2020-10-06 11:56:16 +02:00
Piotr Tabor
2edb08642c api: Make api/ a module that will contain proto-definitions. 
The module is supposed to contain minimal set of files that establish
public etcd server API. In particular client libraries for etcd built in
different languages might want to depend on this file.
 2020-10-06 11:54:50 +02:00
Piotr Tabor
0e5d81704f .travis.yml, scripts: Fix minor bugs in the test script. 
1. setting environment variable cannot be in quote
2. "--race" testing for unit tests is supposed to be part of linux-amd64-unit-4-cpu-race config.
3. 'run' function in test script should log_error in case of failed
command (wrong operator for ints comparison in bash).
 2020-10-01 14:34:58 +02:00
Piotr Tabor
f1d4593241 ./test: Refactoring of test script for modularization 
This refactoring offers following benefits:

  - Unified way how go test commands are being called (in terms of flags intepretation)
  - Uses standard go mechanisms (like go lists) to find files/packages that are subject for test. The mechanism are module aware.
  - Added instruction how to install tools needed for the tests/checkers.
  - Added colors to the output to make it easier to spot any failure.

Confirmed to work using:
- COVERDIR="./coverage" CPU="4" RACE=false COVER=false PASSES="build build_cov cov" ./test
- CPU="4" RACE=false COVER=false PASSES="e2e functional integration" ./test
- COVERDIR="./coverage" COVER="false" CPU="4" RACE="false" PASSES="fmt build unit build_cov integration e2e integration_e2e grpcproxy cov" ./test
- PASSES=unit PKG=./wal TIMEOUT=1m ./test
- PASSES=integration PKG=./clientv3 TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE=TestNew TIMEOUT=1m ./test
- PASSES=unit PKG=./wal TESTCASE="\bTestNew\b" TIMEOUT=1m ./test
- PASSES=integration PKG=./client/integration TESTCASE="\bTestV2NoRetryEOF\b" TIMEOUT=1m ./test
- COVERDIR=coverage PASSES="build_cov cov" ./test
 2020-09-28 11:07:50 +02:00
Piotr Tabor
138a312643 scripts: Removed updatedep.sh. 
As we get rid of ./vendor there is no need to update the dependencies.

If anyone needs up-to-date vendor directory locally, getting it is as
simple as:

```go mod vendor```
 2020-09-16 13:04:58 +02:00
Piotr Tabor
fb2e96e651 vendor: Get rid of ./vendor cleanup 
Updated scripts and documentation to not recommend vendoring.
Implemented best practices for tools installation.

Performed multiple tests to confirm its not breaking any workflows and
has no negative performance impact. Rather see 3x speedup.

1. PASSES="fmt unit integration e2e functional" ./test
2. ./scripts/updatebom.sh
3. ./scripts/updatedep.sh
4. ./scripts/genproto.sh - works - ca be simplified - in follow up PR
5. Installation without explicit GOPATH:

```
% unset GOPATH
% [sudo] rm -rf ~/go
% git clone https://github.com/etcd-io/etcd.git
% time ./build
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading go.uber.org/zap v1.15.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading github.com/google/uuid v1.0.0
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/btree v1.0.0
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/urfave/cli v1.20.0
go: downloading github.com/mattn/go-runewidth v0.0.2
./build  8.22s user 2.31s system 117% cpu 8.961 total
```

Before:
```
% git clone https://github.com/etcd-io/etcd.git && cd etcd && time ./build
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 19.85 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.

./build  34.97s user 4.15s system 236% cpu 16.555 total
```

6. Rebuild without changes:

```
% time ./build
./build  1.43s user 0.83s system 168% cpu 1.336 total
```

7. Instantation of vendor directory (assuming ./build loaded them to
$GOPATH/pkg):

```
time go mod vendor
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  0.51s user 0.44s system 110% cpu 0.861 total
```

8. Fresh instantation of vendor:

```
% rm -rf vendor
% [sudo] rm -rf ~/go

% time go mod vendor
go: downloading github.com/coreos/go-systemd/v22 v22.0.0
go: downloading github.com/spf13/cobra v0.0.3
go: downloading github.com/prometheus/client_golang v1.0.0
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4
go: downloading github.com/gogo/protobuf v1.2.1
go: downloading sigs.k8s.io/yaml v1.1.0
go: downloading google.golang.org/grpc v1.26.0
go: downloading github.com/urfave/cli v1.20.0
go: downloading go.uber.org/zap v1.15.0
go: downloading github.com/spf13/pflag v1.0.1
go: downloading github.com/soheilhy/cmux v0.1.4
go: downloading github.com/json-iterator/go v1.1.7
go: downloading github.com/coreos/go-semver v0.2.0
go: downloading github.com/prometheus/common v0.4.1
go: downloading github.com/prometheus/procfs v0.0.2
go: downloading go.uber.org/atomic v1.6.0
go: downloading github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5
go: downloading github.com/golang/protobuf v1.3.2
go: downloading github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa
go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
go: downloading github.com/modern-go/reflect2 v1.0.1
go: downloading github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: downloading go.uber.org/multierr v1.5.0
go: downloading github.com/creack/pty v1.1.11
go: downloading github.com/mattn/go-runewidth v0.0.2
go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
go: downloading golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5
go: downloading github.com/jonboulle/clockwork v0.1.0
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/etcd-io/gofail v0.0.0-20190801230047-ad7f989257ca
go: downloading github.com/grpc-ecosystem/grpc-gateway v1.9.5
go: downloading github.com/google/btree v1.0.0
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading github.com/beorn7/perks v1.0.0
go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible
go: downloading github.com/google/uuid v1.0.0
go: downloading golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2
go: downloading github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8
go: downloading go.etcd.io/bbolt v1.3.5
go: downloading golang.org/x/text v0.3.3
go: downloading gopkg.in/cheggaaa/pb.v1 v1.0.25
go: downloading github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4
go: downloading github.com/inconshreveable/mousetrap v1.0.0
go: downloading github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bgentry/speakeasy v0.1.0
go: downloading github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go mod vendor  3.62s user 1.30s system 127% cpu 3.854 total
```

9. Size of the repository - before: 39M, after: 18M

Before:

```
% time git clone https://github.com/etcd-io/etcd.git
Cloning into 'etcd'...
remote: Enumerating objects: 97872, done.
remote: Total 97872 (delta 0), reused 0 (delta 0), pack-reused 97872
Receiving objects: 100% (97872/97872), 58.97 MiB | 20.53 MiB/s, done.
Resolving deltas: 100% (63091/63091), done.
git clone https://github.com/etcd-io/etcd.git  4.66s user 1.02s system 93% cpu 6.068 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
52K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
21M	./vendor
39M
```

After:
```
% time git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor
Cloning into 'etcd'...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 98489 (delta 10), reused 8 (delta 1), pack-reused 98451
Receiving objects: 100% (98489/98489), 59.23 MiB | 21.26 MiB/s, done.
Resolving deltas: 100% (63572/63572), done.
git clone https://github.com/ptabor/etcd.git -b 20200908-no-vendor  5.56s user 1.05s system 105% cpu 6.260 total

% du -h --exclude .git -d 1
944K	./clientv3
108K	./etcdmain
5.4M	./Documentation
384K	./security
384K	./mvcc
28K	./.github
8.0K	./version
144K	./contrib
240K	./proxy
2.5M	./etcdserver
112K	./embed
536K	./integration
332K	./tools
116K	./lease
108K	./logos
896K	./tests
960K	./raft
216K	./client
56K	./scripts
100K	./hack
464K	./etcdctl
3.0M	./pkg
620K	./functional
136K	./wal
152K	./auth
19M	.
```
 2020-09-15 14:38:54 +02:00
Gyuho Lee
d93c36bbb9 *: run "gofmt -s" on genproto complete 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-05-09 23:14:32 -07:00
Brandon Philips
03248a9045 bill-of-materials: updates for /etcd/v3 packaging 
bill-of-materials:
- update to new package names

test/updatebom.sh scripts:
- Update to the right package names
- Don't add bom tool to go.mod
 2020-04-28 00:57:52 +00:00
Brandon Philips
2ea4d88299 *: use go.etcd.io/etcd/v3 where needed 
Found potential locations via

```
git grep 'go.etcd.io/etcd$'
```
 2020-04-28 00:57:52 +00:00
Gyuho Lee
3ffb3aed3a vendor: update bbolt v1.3.4, zap v1.14.1 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2020-03-19 13:36:43 -07:00
Nirman Narang
17a220eaee scripts: Added s390x support for docker image release 2020-01-20 07:09:59 -08:00
Jingyi Hu
dcd622b2c7 etcdserver: add v3 request type for cluster attr 
Added ClusterVersionSetRequest for setting cluster version via v3 apply.

Added ClusterMemberAttrSetRequest for setting clsuter member attributes
via v3 apply.
 2019-12-05 16:25:13 -08:00
Nirman Narang
cabf5942b2 scripts/build-binary: Added s390x support for binary release 2019-11-13 22:57:23 -08:00
Gyuho Lee
fd5a25fcec scripts/release: list GPG key only when tagging is needed 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-10-23 10:33:39 -07:00
Joe Betz
36696fe2cc
Add version, tag and branch checks to release script 2019-10-11 15:47:15 -07:00
Gyuho Lee
0824b08cab
Merge pull request #11237 from YoyinZyc/prevent-darwin-build 
scripts: avoid release builds on darwin machine.
 2019-10-11 10:39:14 -07:00
yoyinzyc
b9358a0bc8 scripts: avoid release builds on darwin machine. 2019-10-11 09:56:48 -07:00
yoyinzyc
55efdfe364 scripts: fix read failure prompt in release; use https for git clone. 2019-10-10 13:23:36 -07:00
Gyuho Lee
46f3e8349b scripts/build-binary: fix darwin tar commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-27 23:35:34 -07:00
Gyuho Lee
838315283d scripts/release: fix SHA256SUMS command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-17 14:12:50 -07:00
Gyuho Lee
c327120ba0 scripts/release: fix docker push command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-17 13:52:22 -07:00
Gyuho Lee
1dff148f69 scripts/updatedep: use Go 1.13 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-09-06 11:39:03 -07:00
Manuel Rüger
d981fe6b6b scripts/release: Apply shellcheck findings 
I run https://github.com/koalaman/shellcheck/ over scripts/* and fixed
the findings it returned.

Signed-off-by: Manuel Rüger <manuel@rueg.eu>
 2019-09-02 17:50:04 +02:00
Brandon Philips
f108a32393
scripts/release: rename SHA256SUM to SHA256SUMS 
These files are commonly called SHA256SUMS and with this change rget
works for v3.4.0 as well.
 2019-08-30 13:31:17 -07:00
Gyuho Lee
837fe89d0b scripts/release: fix sha256sum 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-29 09:39:18 -07:00
Sam Batschelet
2386b945c6 scripts/release: add sha256sum summary of release assets 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2019-08-28 15:56:39 +00:00
Gyuho Lee
fd0d43dad9 scripts/release: clean up minor tag docker commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-13 22:00:07 -07:00
Gyuho Lee
8d823e70dd
Merge pull request #11018 from gyuho/yq 
scripts/release: update "yq" command
 2019-08-12 11:03:37 -07:00
Gyuho Lee
c72fa51b2a scripts: remove ".aci" commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:21 -07:00
Gyuho Lee
af38185f92 scripts/release: fix version check commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:13 -07:00
Gyuho Lee
ac613c481f scripts: fix build docker commands, add more logging 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 10:20:05 -07:00
Gyuho Lee
7299a6a106 *: remove "acbuild" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 09:31:22 -07:00
Gyuho Lee
1f4f8ae731 scripts/release: update "yq" command 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-12 09:26:04 -07:00
Gyuho Lee
f7992237ce scripts/release: remove acbuild commands 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-08-04 23:20:41 -07:00
Tobias Schottdorf
14625b847c scripts: have genproto.sh clean up after itself 
We don't want it to leave gopath.proto around for reasons detailed in
the previous commit (messing up vgo).
 2019-07-16 14:01:04 +02:00
Tobias Schottdorf
f63984bb33 scripts: fail explicitly in updatedep.sh when gopath.proto exists 
I had been dealing with these intermittent failures for a while and
finally figured out why. The real solution is making genproto.sh less
ugly but that won't happen for a while.
 2019-07-16 13:54:09 +02:00
Gyuho Lee
b40597ce46 module: require 1.12, remove "v3" import paths 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-06-03 11:15:19 -07:00
Gyuho Lee
c8ffa36d9e scripts/genproto: bump up protoc 3.7.1 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2019-05-28 15:39:35 -07:00
Davanum Srinivas
86e3481ba2
scripts: Remove generated code and script 
Change-Id: Iac4601443bcad71920fd96b97bfe21c16116577a
 2019-04-23 16:54:44 -04:00
Gyuho Lee
3e0f0ba40e
Merge pull request #10401 from markmc/doc-drop-etcdctl-v3-flag 
Eliminate some ETCDCTL_API=3 usage
 2019-02-01 11:25:55 -08:00
Sam Batschelet
a011b2c4c4 scripts: disable go mod and bump protoc to 3.6.1 
Signed-off-by: Sam Batschelet <sbatsche@redhat.com>
 2019-01-22 16:55:01 +00:00
Mark McLoughlin
e53324db3b scripts/release: stop using ETCDCTL_API=3 
Note: v3 has been the default since 25bc65794.
 2019-01-14 14:46:16 +00:00
Gyuho Lee
d5967b40db scripts/updatedep: fix shellcheck errors 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-09-21 02:08:25 -07:00
Gyuho Lee
ad2d18aeff scripts/updatedep: use Go 1.11 module for dependency management 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-09-21 01:47:24 -07:00
Vimal K
bcde798fdd *: path changes for moving to github/etcd-io/etcd 2018-09-03 21:57:23 +05:30
Gyuho Lee
a8a3efd27a scripts: update import paths in "go.etcd.io/etcd" 
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
 2018-08-28 17:47:55 -07:00
Gyuho Lee
981644af7d
scripts/updatedep: remove "dep prune" command 
"Pruning is now performed automatically by dep ensure."

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-08-02 08:06:27 -07:00
Gyuho Lee
453db62d84 scripts/genproto: use protoc 3.6.0 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-07-10 10:16:37 -07:00
Joe Betz
a8990e2e18
scripts: Fix release script to always build etcd binary before checking version 2018-06-06 10:48:45 -07:00
Joe Betz
e2821a16ff
scripts/release: Fix docker push for 3.1 releases, remove inaccurate warning at the end of release script 2018-05-31 13:56:07 -07:00
Gyuho Lee
0db7ea7337 scripts/release: disable minor version docker push 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-31 12:08:01 -07:00
Sam Batschelet
fe15c3250e scripts/genproto: use latest "grpc-ecosystem/grpc-gateway" 2018-05-23 08:36:58 -04:00
Sam Batschelet
05554119c9 vendor: use latest "grpc-ecosystem/grpc-gateway" 2018-05-22 17:35:15 -04:00
Gyuho Lee
d29a0c7172 scripts/genproto: use latest "gogo/protobuf" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-22 13:42:43 -07:00
Gyuho Lee
2d542d336a scripts/genproto.sh: rename 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-05-21 07:45:34 -07:00
Joe Betz
4319a8a2ba
scripts: Fix remote tag check, gcloud login and umask in release script 2018-05-09 10:37:24 -07:00
Joe Betz
bee6f34197
scripts: Fix a few etcd release script bugs and make it reenterant. 2018-04-24 17:03:14 -07:00
Joe Betz
7750974760
scripts: Add scripts/release that performs 'etcd-release-runbook' (https://goo.gl/Gxwysq) style release workflow 2018-04-18 13:35:41 -07:00
Gyuho Lee
140374e25d *: move "scripts/docker-*" to "tests" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-09 10:31:00 -07:00
Gyuho Lee
6f2adc8a3d scripts: add dev scripts 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-04-02 11:27:39 -07:00
Gyuho Lee
f5c56401d7 scripts/update-clientv2: initial commit 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-27 10:33:27 -07:00
Gyuho Lee
906a40bced scripts/genproto: fix shellcheck 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-22 10:39:31 -07:00
Gyuho Lee
27f0aa9cf0 scripts: document "updatedep.sh" script 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-03-07 13:54:56 -08:00
Gyuho Lee
b24d81c30d scripts: remove "internal" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-26 17:11:40 -08:00
Gyuho Lee
dac6948774 scripts/genproto: run "protodoc" by default 
Do not skip "protodoc" by default.

Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-02-13 13:51:13 -08:00
Gyuho Lee
264f8f6577 *: regenerate proto 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-28 23:43:15 -08:00
Gyuho Lee
2c91007591 *: regenerate proto 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-26 11:21:40 -08:00
Gyuho Lee
698ec7f7d5 scripts/updatebom.sh: remove "./cmd/vendor" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-24 15:26:17 -08:00
Gyuho Lee
505d24c463 scripts/updatedep.sh: use "golang/dep" 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-24 15:26:16 -08:00
Gyuho Lee
2b6fd3094f *: rename "snap" to "raftsnap" package 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-24 10:26:07 -08:00
Gyuho Lee
fd16656af1 scripts/genproto: use "grpc-gateway" v1.3.1 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2018-01-09 12:41:49 -08:00
Gyuho Lee
7800fd9fff scripts/genproto: require protoc 3.5.1 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2017-12-31 13:20:40 -08:00
Gyuho Lee
3518351a24 scripts: remove ACI build scripts 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2017-12-20 15:35:11 -08:00
Gyuho Lee
b1c6b98f3d scripts/genproto: require protoc 3.5, update gogo/proto 
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
 2017-12-07 21:31:13 -08:00
Gyu-Ho Lee
aca39e2ae1 scripts/build-docker: build both gcr.io and quay.io images 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-11-28 15:02:09 -08:00
Gyu-Ho Lee
ec881b0507 scripts/genproto: upgrade protoc to 3.4 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-11-08 18:50:29 -08:00
tamal
c0c19465fc *: upgrade grpc-gateway to v1.3 2017-11-08 18:38:41 -08:00
Anthony Romano
9abe9da9db *: fix shellcheck warnings 
Fixes scripts and removes shellcheck warning suppressions.

* regexp warnings
* use ./*glob* so names don't become options
* use $(..) instead of legacy `..`
* read with -r to avoid mangling backslashes
* double quote to prevent globbing and word splitting
 2017-09-06 19:18:04 -07:00
Sam Batschelet
e5aa938fec scripts: generate swagger with authorization support 2017-06-15 13:41:43 -04:00
Geoff Levand
e402606f02 build-aci: Fix ACI image name 
The appc discovery spec states that the architecture specifier in the ACI
image file name will be an ACI architecture value.  Our build scripts were
using GOARCH in the image name, which is incorrect for arm64/aarch64.
See: https://github.com/appc/spec/blob/master/spec/discovery.md

Fixes errors like these on arm64 machines:

  $ rkt --debug --insecure-options=image fetch coreos.com/etcd:v3.2.0-rc.1
  image: remote fetching from URL "https://github.com/coreos/etcd/releases/download/v3.2.0-rc.1/etcd-v3.2.0-rc.1-linux-aarch64.aci"
  fetch: bad HTTP status code: 404

Signed-off-by: Geoff Levand <geoff@infradead.org>
 2017-06-13 13:09:02 -07:00
Anthony Romano
4ebeba0e18 *: regen protofiles with latest protobuf tools 2017-06-12 15:14:43 -07:00
Anthony Romano
85f433232a *: clear rarer shellcheck errors on scripts 
Clean up the tail of the warnings
 2017-06-06 09:36:25 -07:00
Anthony Romano
1be245269e scripts, Documentation: fix swagger generation 
Changes to the genproto to support splitting out the grpc-gateway broke
swagger generation.
 2017-06-02 10:54:05 -07:00
Anthony Romano
156612bb25 Documentation, scripts: regen RPC docs 
Was missing the new cancel_reason field. Also includes updated protodoc
sha to fix generating documentation for upcoming txn compare range patchset.
 2017-06-02 10:15:12 -07:00
Anthony Romano
10b1ba7886 Documentation, scripts: regenerate protobuf docs with updated protodoc 2017-05-19 14:57:16 -07:00
Gyu-Ho Lee
8c9f01ef53 scripts: add 'BOM' update script 
Need this script when we add external dependencies.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-05-08 17:59:11 -07:00
Gyu-Ho Lee
e9a63473a0 scripts,travis: install 'marker' for CI tests 
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
 2017-05-04 08:26:00 -07:00
Anthony Romano
6c1ce697a6 scripts: annotate with acbuild with supports-systemd-notify 
Fixes #7840
 2017-05-01 12:59:08 -07:00
Anthony Romano
76fb6ebcbb scripts: remove testify hack in updatedep 2017-04-22 20:29:58 -07:00
Geoff Levand
07c07cea25 release: Add multi arch support 
Signed-off-by: Geoff Levand <geoff@infradead.org>
 2017-04-21 10:04:41 -07:00
Geoff Levand
0c8988aa07 build-docker: Updates for multi-arch release 
o Set -e to abort script if a command fails.
 o Allow custom docker 'TAG' from the environment.
 o Move arch suffix to version to allow all images to
   be put into a single repository.
 o Enable cross builds.  When doing cross builds where the
   host and target architectures are different 'RUN mkdir'
   will fail since the target container cannot be run on
   the host.  To work around this, create the directories
   in build-docker, then use ADD in the Dockerfile.
 o Add Dockerfile-release.arm64

Signed-off-by: Geoff Levand <geoff@infradead.org>
 2017-04-21 10:04:41 -07:00
Geoff Levand
8309ca92d7 build-aci: Add multi arch support 
Uses GOARCH to build for a targeted arch.

Usage: GOARCH=... BINARYDIR=... BUILDDIR=... ./scripts/build-aci version

Signed-off-by: Geoff Levand <geoff@infradead.org>
 2017-04-21 10:04:41 -07:00
Geoff Levand
fb6287240f build-binary: Add arm64 
Signed-off-by: Geoff Levand <geoff@infradead.org>
 2017-04-21 10:04:41 -07:00
Anthony Romano
fd11523af9 scripts: move gateway stubs into gw/ packages 2017-04-19 12:50:04 -07:00
Anthony Romano
78a5eb79b5 *: add swagger and grpc-gateway assets for v3lock and v3election 2017-04-10 15:21:07 -07:00
Anthony Romano
9ba69ff317 scripts: update genproto.sh to include v3election 2017-04-07 16:36:38 -07:00
Anthony Romano
1ff0b71b30 *: use protoc 3.2.0 
Fixes #7631
 2017-03-30 13:43:10 -07:00
Anthony Romano
c737bf3d2a scripts: generate lock service rpc stubs 2017-03-13 15:23:26 -07:00
Manjunath A Kumatagi
633cfbe241 script: Build docker image for ppc64le 2017-02-27 19:04:32 -05:00
Manjunath A Kumatagi
5d41e7f09b scripts: Add support to build ppc64le binary for release 2017-02-02 00:45:56 +05:30
Anthony Romano
de58a9c733 scripts: use glide update if repo exists in glide.lock 2016-12-08 14:26:29 -08:00
Gyu-Ho Lee
baa4e4ee56 scripts/genproto: update gogo/protobuf, grpc-gateway 2016-11-10 12:02:00 -08:00
Gyu-Ho Lee
4cf5b76d18 scripts/updatedep: work around 'testify/assert' 
'glide vc --no-tests' flag removes 'testify/assert' deps
in v2 client. Until we deprecate v2 tests, just copy the
necessary files as workaround.

And remove '--skip-tests' flags in case we add dependencies
in test files.
 2016-11-09 15:59:34 -08:00
fanmin shi
c2fd42b556 etcdserver, clientv3: add "!=" to txn 
adding != to compare is a requested functionality from a etcd user

FIX #6719
 2016-11-09 14:28:36 -08:00
Gyu-Ho Lee
791aeb39a6 Merge pull request #6653 from gyuho/acbuild 
acbuild: add symlinks to /usr/local/bin/etcd*
 2016-10-21 10:48:08 -07:00
Gyu-Ho Lee
0bd9179835 release: build binary without symbols for debug 2016-10-19 09:45:10 -07:00
Gyu-Ho Lee
522be31192 acbuild: add symlinks to /usr/local/bin/etcd* 
And uses latest acbuild (v0.4.0, --to-dir flag is deprecated).

For https://github.com/coreos/etcd/issues/6057.
 2016-10-14 10:35:26 -07:00
Gyu-Ho Lee
def1a3b77f script/updatedep: update glide, glide-vc version 2016-10-10 11:11:58 -07:00