Commit Graph

411 Commits

Author SHA1 Message Date
9fc3b2ad40 Merge pull request #18590 from henrybear327/ci/fix
Add tools/mod to module_dirs
2024-09-19 10:36:12 +01:00
5b2faec983 fea: Remane modules_exp to modules_for_bom to improve code and clarity
Signed-off-by: mevrin <matthieu.evrin@gmail.com>
2024-09-18 21:45:07 -04:00
fc901bd019 Add tools/mod to module_dirs
As `tools/mod` also contains the `go.mod` file. We should add it to the
`module_dirs` variable, so that when executing `./scripts/fix.sh`, the
proper checks and fixes can be applied.

To address the issue of broken unit tests and code coverage due to the
directory's lack of Go code, we've introduced a new doc.go file. This
file acts as a placeholder, enabling tools like golangci-lint and
go test to function correctly.

---

Discovered when working on https://github.com/etcd-io/etcd/pull/18575

The directories are checked against the following:
- Command: `find . -type f -name go.mod -exec dirname {} \;`
- Output:
```
./etcdutl
.
./tools/testgrid-analysis
./tools/rw-heatmaps
./tools/mod
./etcdctl
./tests
./server
./api
./client/internal/v2
./client/v3
./client/pkg
./pkg
```

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-09-18 14:33:13 +02:00
7e8e0bf1f9 Ensure commits are signed in scripts/release.sh.
Signed-off-by: James Blair <mail@jamesblair.net>
2024-09-16 11:31:30 +12:00
ce916fbb38 tests: remove Dockerfile
The tests/Dockerfile was outdated (using an obsolete Ubuntu
distribution), and it's not being used by any workflow or prow job. This
Dockerfile was the only one using `scripts/install-marker.sh`, which
used a vendorized binary generated by the etcd project. This is not
needed anymore, as marker releases now provide the binaries.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-26 15:31:38 -07:00
4f9cd336a6 marker: use released binary when running markdown lint
The markdown linter, marker, now publishes a released version with the
tool. Given that this project is written in Rust, running this check can
now be done by using the statically compiled binary without having the
Rust local environment.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-13 15:54:46 -07:00
059371a02f Merge pull request #18250 from jmhbnz/main
Remove gsutil acl command for bucket permissions from release.sh
2024-07-04 13:43:54 +12:00
ba4b2bffeb make: Include tools/mod when checking dependency versions
As tools/mod is not part of the test library's modules, the check to verify
consistent dependency versions ignored it. Explicitly get the dependencies from
this module when running verify-dep.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-07-03 15:21:31 -07:00
f0246a9f84 Remove gsutil acl command for bucket permissions.
The bucket is already fully public so new releases don't need to run this command.

This will prevent significant terminal spam during the release process as there are thousands of objects in the bucket.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-06-29 17:39:06 +12:00
29b4ff2fd8 Bump shellcheck to latest 2024 release.
Signed-off-by: James Blair <mail@jamesblair.net>
2024-06-24 11:24:31 +12:00
dfda61bbbf make: fix verify-dep
Makefile's target `verify-dep` current behavior is to use `go list` to
check consistent dependency versions from direct dependencies. Ignoring
indirect dependencies in a multi-module project could lead to version
mismatches. If module A imports module B, module B's dependency will be
an indirect dependency in module A. Which can potentially have a version
mismatch. Therefore, use `go mod edit` with indirect dependencies, too.
So it can work with all dependencies defined in go.mod.

Fix displaying dependencies with mismatches, as the old code was
searching with grep just for the prefix, which would show other
dependencies that shared the same prefix.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-06-19 09:02:17 -07:00
043096067f Add run-govulncheck Makefile target
Add a `Makefile` target to run govuln across the submodules. So, it can
be imported into a prow job.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-06-15 15:47:13 -07:00
1579c57729 Update codecov config to support commenting code coverage in PRs
Add missing directory fixing go.etcd.io/etcd/etcdutl/v3/::etcdutl/

Note: we have some of the tests written in a way that is
non-deterministic across runs, thus, even when there is no code changes
there might still have slight variation for test coverage [2]

Reference:
[1] https://github.com/etcd-io/etcd/issues/18131
[2] https://docs.codecov.com/docs/unexpected-coverage-changes#reasons-for-indirect-changes

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-06-12 11:23:31 +02:00
4881e53677 scripts/release: shallow clone repository
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-05-29 15:14:53 -07:00
b915323be6 split out subroutine from fix.sh to makefile target
Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-14 15:18:30 +05:30
3375e89b11 Fix issue reported by static analyzer
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-05-09 13:58:29 +02:00
f6a35f8786 .*: sync go toolchain version and add ability to verify versions
This commit adds a script to sync the version present in .go-version
across all go.mod files as the toolchain directive. As part of that,
this commit also modifies go.mod files that did not have synced toolchain
directives.

Additionally, this also adds a script to verify all toolchain and go
directives against the version present in .go-version as follows:
(1) The go directive <= version in .go-version
(2) The toolchain directive == version in .go-version

This script runs as part of the `make verify` target, making it run
as a presbumit by default.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-07 17:52:56 +05:30
e094139b05 scripts: default to using .go-version's version for tests and builds
Additionally, provide ability to opt-out of the .go-version and use a
custom one via env vars: FORCE_HOST_GO and GO_VERSION.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-05-07 17:52:56 +05:30
13c0f75b7d Merge pull request #17877 from tjungblu/17872_status
add quota to endpoint status response
2024-05-03 13:03:23 +01:00
2b1914c262 Merge pull request #17770 from tico88612/feat/update-tools
Test: add tools/rw-heatmaps & tools/testgrid-analysis to module_dirs
2024-04-29 18:35:05 +01:00
0bd0c4a21b github workflow: drop measure test flakiness
Signed-off-by: zgzhang4 <zgzhang4@iflytek.com>
2024-04-29 23:20:45 +08:00
a988f7edc5 script/test_lib: add description about modules
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:08:05 +08:00
aa53afdce2 test: add tools/{rw-heatmaps,testgrid-analysis} to module_dirs
Signed-off-by: tico88612 <17496418+tico88612@users.noreply.github.com>
2024-04-26 20:07:59 +08:00
bdcff246cb add quota to endpoint status response
This adds the configured backend quota bytes to the endpoint status response.
related discussion in #17821

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-04-25 16:44:20 +02:00
b03368485c testgrid: print out all failed tests for visibility.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-12 12:39:38 -07:00
0168c3e1ce Merge pull request #17662 from siyuanfoundation/flaky
Add script and workflow to detect flaky tests in testgrid.
2024-04-05 16:58:45 +01:00
5cd14a6031 Revert "tests: temporary disable coverage redesign Go experiment"
With the update to Go 1.22.2, this workaround is no longer needed.

This reverts commit da7ab15f80.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-04-04 11:17:06 -07:00
3f37cd2d73 Add script to detect flaky tests in testgrid.
Signed-off-by: Siyuan Zhang <sizhang@google.com>
2024-04-01 09:41:42 -07:00
b82f882c3d Update release script to use ssh.
From 2021-08-13, GitHub is no longer accepting account passwords when authenticating Git operations. You need to add a PAT (Personal Access Token) instead for HTTPS. We should intead keep things simple and just use ssh to push to GitHub.

Signed-off-by: James Blair <mail@jamesblair.net>
2024-03-31 10:09:51 +13:00
fbda591866 fix some typos
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-25 10:34:44 +08:00
da7ab15f80 tests: temporary disable coverage redisign Go experiment
Go v1.22 has an error generating the coverage output. Disable it
temporarily so GitHub workflows can run in the meantime.

Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-09 15:07:33 -08:00
20005e0e84 Make CGO_ENABLED configurable
Signed-off-by: Arka Saha <arkas1@vmware.com>
2024-02-13 13:24:55 +05:30
=
dbfacb4260 Ensure bin directory exists for protoc install
Co-authored-by: James Blair <mail@jamesblair.net>
Signed-off-by: = <abby.crimlis@outlook.com>
2024-01-05 10:29:34 +13:00
=
580ace4d17 Automating install of protoc
Signed-off-by: = <abby.crimlis@outlook.com>
2023-12-10 19:42:30 +13:00
f454342bfe fix missing json flag when using VERBOSE=1
Signed-off-by: upodroid <upodroid@users.noreply.github.com>
2023-11-23 12:33:29 +00:00
2b7e1c6f82 fix scripts
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
2023-11-08 00:27:13 -06:00
649b365937 Merge pull request #16883 from mjlshen/16760
scripts: macOS support for scripts/genproto.sh
2023-11-08 01:56:45 +01:00
c4499adfd5 replace archived repo in BOM check
Signed-off-by: Tessa Pham <hpham111@bloomberg.net>
2023-11-07 17:41:11 -06:00
27272c3ed0 scripts: macOS support for scripts/genproto.sh
This commit will allow scripts/genproto.sh to fall back to gsed if
available, which is required for the script to work on macOS.

Signed-off-by: Michael Shen <mishen@umich.edu>
2023-11-07 18:00:36 -05:00
aea1cd0077 feat: enable unparam lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-17 21:24:13 +08:00
81fe610bd5 *: enable markdown_marker checker
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-13 14:00:25 +08:00
8e1713dda1 Merge pull request #16714 from fuweid/clean-unconvert_pass
*: cleanup unconvert_pass and remove revive tools
2023-10-09 19:38:19 +01:00
0ec6181466 scripts: remove unconvert_pass
The `unconvert` has been migrated into golingci-lint.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-09 23:27:48 +08:00
093ec7a11d scripts: remove the markdown_you_pass
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-08 15:20:30 +08:00
5878f346ba *: migrate nakedret into golangci-lint
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-10-07 22:29:43 +08:00
07effc4d0a *: fix revive linter
Remove old revive_pass in the bash scripts and migirate the revive.toml
into golangci linter_settings.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-24 14:21:11 +08:00
46df6abb03 *: remove goimports(_fix)_pass and add lint_fix_pass
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 21:18:36 +08:00
502879405f *: lint_pass should use global golangci.yaml
Disable failed linters and enable it by #16610.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 16:10:45 +08:00
2b1e181c8b *: Use golangcilint_pass to run staticcheck and ineffassign
Copy the tools/.golangci.yaml and run the linters for which we have
already fixed. The temp .golangci.yaml will be removed when we fixes all
the linters' issues.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 12:34:17 +08:00
9c3edfa0af *: fix staticcheck lint
Changed TraceKey/StartTimeKey/TokenFieldNameGRPCKey to struct{} to
follow the correct usage of context. Similar patch to #8901.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-09-21 11:24:26 +08:00