Commit Graph

59 Commits

Author SHA1 Message Date
cd077a806d refactor: split test_lib.sh into test_utils.sh for basic functionalities and test_lib.sh
Signed-off-by: joshjms <joshjms1607@gmail.com>

fix shell scripts

Signed-off-by: joshjms <joshjms1607@gmail.com>
2025-03-20 10:34:59 +08:00
9fc3b2ad40 Merge pull request #18590 from henrybear327/ci/fix
Add tools/mod to module_dirs
2024-09-19 10:36:12 +01:00
5b2faec983 fea: Remane modules_exp to modules_for_bom to improve code and clarity
Signed-off-by: mevrin <matthieu.evrin@gmail.com>
2024-09-18 21:45:07 -04:00
fc901bd019 Add tools/mod to module_dirs
As `tools/mod` also contains the `go.mod` file. We should add it to the
`module_dirs` variable, so that when executing `./scripts/fix.sh`, the
proper checks and fixes can be applied.

To address the issue of broken unit tests and code coverage due to the
directory's lack of Go code, we've introduced a new doc.go file. This
file acts as a placeholder, enabling tools like golangci-lint and
go test to function correctly.

---

Discovered when working on https://github.com/etcd-io/etcd/pull/18575

The directories are checked against the following:
- Command: `find . -type f -name go.mod -exec dirname {} \;`
- Output:
```
./etcdutl
.
./tools/testgrid-analysis
./tools/rw-heatmaps
./tools/mod
./etcdctl
./tests
./server
./api
./client/internal/v2
./client/v3
./client/pkg
./pkg
```

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-09-18 14:33:13 +02:00
e094139b05 scripts: default to using .go-version's version for tests and builds
Additionally, provide ability to opt-out of the .go-version and use a
custom one via env vars: FORCE_HOST_GO and GO_VERSION.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-07 17:52:56 +05:30
a988f7edc5 script/test_lib: add description about modules
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:08:05 +08:00
aa53afdce2 test: add tools/{rw-heatmaps,testgrid-analysis} to module_dirs
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:07:59 +08:00
fbda591866 fix some typos
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-25 10:34:44 +08:00
f454342bfe fix missing json flag when using VERBOSE=1
Signed-off-by: upodroid <upodroid@users.noreply.github.com>
2023-11-23 12:33:29 +00:00
ac61b36e06 Add an option to keep_going with run for modules on failure
Signed-off-by: Rajalakshmi Girish <rajalakshmi.girish1@ibm.com>
2023-05-07 00:43:30 -07:00
81fccc13da Add -failfast flag when the mode is fail_fast
Signed-off-by: Rajalakshmi Girish <rajalakshmi.girish1@ibm.com>
2023-04-27 05:26:38 -07:00
dc2cf29598 chore: add strict mode to scripts/test_lib.sh
REF: #15514

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-04-13 12:05:39 +08:00
be7be34800 client: Hide v2 client package
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-31 10:26:11 +02:00
5223d09d41 Merge pull request #14838 from serathius/linearizability-docs
tests: Document robustness tests
2023-03-28 16:22:09 +02:00
d03ac88b36 tests: Document robustness tests
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-03-28 15:08:43 +02:00
1fcb782780 chore: introduce strict bash mode for scripts,.github
REF: #15514

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-03-22 18:00:41 +08:00
5d78d6d4b1 release: support kick off release in current branch
Currently when triggering release, it always pull remote repo and
checkout main branch. Any changes which are merged into the target
release branch (e.g. release-3.5) will be ignored. It isn't
convenient for test, including in github workflow and local environment.
So we need to support triggering release in current branch.

Note: --current-branch should only be called with DRY_RUN=true

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-12 09:35:03 +08:00
b1cdf16988 scripts: cleanup raft from all script files
No need to generate proto file;
No need to test coverage for raft;
No need to run any test for raft module;
NO need to run any test for raftexample;

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-12-02 13:58:11 +08:00
18463081ad scripts: go_srcs_in_module to list test packages files
The shell func go_srcs_in_module will now list
1. go src files
2. go test files belong to same packages
3. go test files that belong to _test packages

Fixes #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 23:50:21 +05:30
cc77eb1011 scripts: go_srcs_in_module to list go source + test files
The shell function go_srcs_in_module will list go test files
along with go src files in the specified module. This helps
in identifying the copyright header misses in test files.

Fixes #14827

Signed-off-by: Bhargav Ravuri <bhargav.ravuri@infracloud.io>
2022-11-23 21:52:09 +05:30
8d36956d0e Merge pull request #13955 from ptabor/20220417-tests-improvements
test.sh script improvement (OsX centric)
2022-04-19 12:18:45 +02:00
12ab72a6ff smoke test:fix license header check 2022-04-19 08:35:22 +05:30
55a7c91de9 test.sh: Tools run correctly on OsX
- The location of installed tools is found using `go list`
- ETCD_VERIFY env variable is printed in repro command line.
2022-04-17 23:28:26 +02:00
bba3937f69 Merge pull request #13649 from serathius/scripts
Cleanup scripts in root directory
2022-02-11 15:40:53 -05:00
900fc8dd7e replace gobin with go install
The repository github.com/myitcv/gobin has already been archived,
and the `go install` command accepts arguments with version suffixs
starting from 1.16 (for example, go install example.com/cmd@v1.0.0).
So there is no reason to continue to use gobin.
2022-02-08 05:41:21 +08:00
396efd9608 Cleanup scripts in root directory 2022-02-02 13:50:44 +01:00
4c5efc970d *: Upload test junit results 2021-07-01 14:41:13 +02:00
8bfce5dc01 Add option to generate junit xml reports 2021-06-16 07:25:50 +00:00
c09aca1ba4 Split etcdctl into etcdctl (public API access) & etcdutl (direct surgery on files)
Motivation is as follows:

  - etcdctl we only depend on clientv3 APIs, no dependencies of bolt, backend, mvcc, file-layout
  - etcdctl can be officially supported across wide range of versions, while etcdutl is pretty specific to file format at particular version.
it's step towards desired modules layout, documented in: https://etcd.io/docs/next/dev-internal/modules/
2021-05-17 11:54:03 +02:00
3bb7acc8cf Migrate dependencies pkg/foo -> client/pkg/foo 2021-04-07 00:38:47 +02:00
a84bd093b0 Integration with grpc-settable logger. 2021-03-16 22:50:41 +01:00
c640957e2d Release scripts: Minor fixes discovered during attempt for 3.5.0-alpha.0 2021-02-24 19:16:27 +00:00
90d1b838ad Fix shellcheck warnings: Escaping.
```
build:3:65: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
test:3:64: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:23:46: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/fix.sh:30:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:38:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:53:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:73:23: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:73:33: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:74:69: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:74:78: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
scripts/genproto.sh:75:40: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
scripts/genproto.sh:75:52: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
scripts/genproto.sh:91:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
scripts/genproto.sh:115:17: note: Backslash is literal in "\n". Prefer explicit escaping: "\\n". [SC1117]
./test.sh:71:29: warning: Quote to prevent word splitting, or split robustly with mapfile or read -a. [SC2206]
./test.sh:400:28: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:406:33: note: Backslash is literal in "\t". Prefer explicit escaping: "\\t". [SC1117]
./test.sh:415:36: note: Backslash is literal in "\*". Prefer explicit escaping: "\\*". [SC1117]
./test.sh:415:176: note: Backslash is literal in "\'". Prefer explicit escaping: "\\'". [SC1117]
./test.sh:493:56: note: Backslash is literal in "\.". Prefer explicit escaping: "\\.". [SC1117]
./test.sh:601:57: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:64: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:67: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:74: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:77: note: Backslash is literal in "\(". Prefer explicit escaping: "\\(". [SC1117]
./test.sh:601:84: note: Backslash is literal in "\)". Prefer explicit escaping: "\\)". [SC1117]
./test.sh:601:91: note: Backslash is literal in "\1". Prefer explicit escaping: "\\1". [SC1117]
./test.sh:601:94: note: Backslash is literal in "\2". Prefer explicit escaping: "\\2". [SC1117]
./test.sh:601:103: note: Backslash is literal in "\3". Prefer explicit escaping: "\\3". [SC1117]
```
2021-01-29 22:32:43 +00:00
88ee005a0f Reduce verbosity of coverage collection. 2021-01-23 08:59:19 +01:00
598ca6caab Modernize release script:
- making sure the DRY_RUN mode can finish e2e, so e.g. commits to
local copy of repository are OK in dry-run (while git pushes are NOT).
  - better interaction with ./test_lib.sh script.
  - more consistent logging
  - bringing back s390x architecture that on go 1.14.3 seems to work as
expected.
2021-01-21 17:47:36 +01:00
486687843b Fix code coverage measurement. 2021-01-19 15:41:36 +01:00
577c898fee scripts: Integrate ./scripts/release with new code for tagging modules.
Changes:
  - signing tags.
  - allows to override BRANCH and REPOSITORY using env variables.

Tested by a release in my private fork:
  BRANCH="20201126-ptabor-release" REPOSITORY="git@github.com:ptabor/etcd.git" ./scripts/release 3.5.0-alpha.20
2021-01-15 12:31:44 +01:00
fd2f34fd13 Release: Scripts to change versions in all go.mod files and push tags upstream.
Examplar invocations:

Edit go.mod files such that all etcd modules are pointing on given version:

```
% DRY_RUN=false TARGET_VERSION="v3.5.13" ./scripts/release_mod.sh update_versions
```

Tag latest commit with current version number for all the modules and push upstream:
```
% DRY_RUN=true REMOTE_REPO="origin" ./scripts/release_mod.sh push_mod_tags
```
2020-11-04 15:16:36 +01:00
2e55875cc7 scripts: install github.com/myitcv/gobin while gobin doesn't exist. 2020-10-30 20:58:00 +08:00
7da5182f1d Merge pull request #12422 from tangcong/fix-realpath
scripts: fix realpath command not found in mac os
2020-10-26 10:42:26 -07:00
8277395e1b scripts: use manual scripts to replace realpath 2020-10-27 00:36:28 +08:00
6c1efd6ba5 server: Update go.mod 2020-10-26 13:02:32 +01:00
0ba16d8ee1 *: Convert tabulators to whitespaces in bash scripts.
Execution of `./scripts/fix.sh` that executed:
```
find ./ -name '*.sh' | xargs sed --follow-symlinks -i 's|\t|  |g'
```
2020-10-26 10:59:40 +01:00
09679d29ad etcdctl: Rename of imports after making etcdctl a module.
```
find -name '*.go' | xargs sed -i --follow-symlinks 's|etcd/v3/etcdctl|etcd/etcdctl/v3|g'
```
2020-10-21 11:15:35 +02:00
df48e499ea etcdctl: Make etcdctl a module (go.mod, LICENSE) 2020-10-20 12:07:27 +02:00
cf795c0ae6 client/v3: Create and update go.mod 2020-10-20 10:09:12 +02:00
bea5f379bc raft: Make raft a module
Establishing a module: Creating a go.mod files
2020-10-16 09:15:48 +02:00
45e4306508 client: Move client to client/v2 as a module.
We make v2 client code a module go.etcd.io/etcd/client/v2.

Pretty mechanical change that can be summarized as:

  mkdir client/v2
  cd client/v2 && git mod init go.etcd.io/etcd/client/v2

  git mv client/*.go client/v2/
  find -name '*.go' | xargs sed -i --follow-symlinks 's|/v3/client["]|/client/v2\"|g'

  + fixing changelog, bom, go.mod etc.
2020-10-15 14:39:43 +02:00
bc9e433ca2 tools: Migrate remaining tools to gobin
Replace ./scripts/install_tool.sh with `gobin`, such that we have
consistent handling for all tools needed for build and consistent
versioning within ./tools/mod/go.mod.

Side changes:
  - Expose /scripts/fix.sh that fixes formatting and bom across modules
  - Expose *.sh variants of scripts like build and ./test (first step
towards replacement).
  - Make stderr output of commands explicit and make commands use
different color than callouts.
2020-10-13 19:33:01 +02:00
bfa91d48af pkg: Estabilish pkg as a module 2020-10-12 23:58:09 +02:00