Commit Graph

70 Commits

Author SHA1 Message Date
1d7639f796 etcdserver: added more debug log for the purgeFile goroutine
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2022-10-12 19:39:20 +08:00
e4deb09c9e etcdserver,pkg: remove temp files in snap dir when etcdserver starting
- Backporting: https://github.com/etcd-io/etcd/pull/12846
- Reference: https://github.com/etcd-io/etcd/issues/14232

Signed-off-by: vivekpatani <9080894+vivekpatani@users.noreply.github.com>
2022-07-21 15:50:27 -07:00
bea35fd2c6 pkg/fileutil: fix F_OFD_ constants
Use golang.org/x/sys/unix for F_OFD_* constants.

This fixes the issue that F_OFD_GETLK was defined incorrectly,
resulting in bugs such as https://github.com/moby/moby/issues/31182

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-12-14 10:42:13 -08:00
d5ebbbceb8 pkg: file stat warning
Provide warning and doc instead of enforcing file permission.
2020-08-24 11:21:29 -04:00
0207d1df66 pkg/fileutil: print desired file permission in error log 2020-06-29 09:59:19 +08:00
e5424fc474 pkg: Fix dir permission check on Windows 2020-06-25 20:20:55 -04:00
434f7e83f0 pkg: check file stats
modify file util.
2020-06-20 16:29:47 -04:00
4f3c81d81d etcdserver: wait purge file loop during shutdown
To prevent the purge file loop from accidentally acquiring the file lock
and remove the files during server shutdowm.
2019-10-30 16:04:41 -07:00
1caaa9ed4a test: test update for Go 1.12.5 and related changes
Update to Go 1.12.5 testing. Remove deprecated unused and gosimple
pacakges, and mask staticcheck 1006. Also, fix unconvert errors related
to unnecessary type conversions and following staticcheck errors:
- remove redundant return statements
- use for range instead of for select
- use time.Since instead of time.Now().Sub
- omit comparison to bool constant
- replace T.Fatal and T.Fatalf in tests with T.Error and T.Fatalf respectively because the goroutine calls T.Fatal must be called in the same goroutine as the test
- fix error strings that should not be capitalized
- use sort.Strings(...) instead of sort.Sort(sort.StringSlice(...))
- use he status code of Canceled instead of grpc.ErrClientConnClosing which is deprecated
- use use status.Errorf instead of grpc.Errorf which is deprecated

Related #10528 #10438
2019-06-05 17:02:05 -04:00
d37f1521b7 *: update import paths to "go.etcd.io/etcd"
Signed-off-by: Gyuho Lee <leegyuho@amazon.com>
2018-08-28 17:47:55 -07:00
fa6d2e2d71 pkg/fileutil: add "ReadDirOption"
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-18 10:28:27 -07:00
2bb1a268b3 pkg/fileutil: clarify flock errors
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-05-03 14:00:42 -07:00
0dad8abb6f pkg: support structured logger
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-16 17:36:00 -07:00
b8bf42cc5a pkg/fileutil: test "Exist" on directory
Signed-off-by: Gyuho Lee <gyuhox@gmail.com>
2018-04-08 14:26:30 -07:00
962976f2df pkg/fileutil: fix preallocate under OS X kernel
ftruncate changes st_blocks, and following fallocate
syscalls would return EINVAL when allocated block size
is already greater than requested block size
(e.g. st_blocks==8, requested blocks are 2).

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-12-14 10:36:57 -08:00
75110dd839 *: fix naked returns
Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-11-10 18:46:15 -08:00
a36d62a30c Merge pull request #8049 from heyitsanthony/flock-base-test
fileutil: test some fallback functionality
2017-06-07 16:12:38 -07:00
c3fcf0f339 fileutil: test some fallback functionality
syscall.Flock fallback and preallocExtendTrunc
2017-06-07 11:22:40 -07:00
87a3c87e45 fileutil: return immediately if preallocating 0 bytes
fallocate will return EINVAL, causing zeroing to the end of a
0 byte file to fail.

Fixes #8045
2017-06-07 09:57:14 -07:00
aca2abd8fe *: use 'io.Seek*' for go1.7+
For https://github.com/coreos/etcd/issues/6174.

Signed-off-by: Gyu-Ho Lee <gyuhox@gmail.com>
2017-03-20 15:15:24 -07:00
2f1542c06d *: use filepath.Join for files 2017-03-16 07:46:06 -07:00
9473e9c30e pkg: use etcd as logging repo 2016-09-29 15:29:38 +08:00
c449da6ff9 fileutil: windows OpenDir
Windows needs to open a directory with write access to fsync but the go
runtime won't open directories that way.
2016-09-08 00:09:16 -07:00
58a36cb651 fileutil: add ZeroToEnd for zeroing files 2016-08-25 14:24:46 -07:00
c920ce0453 fileutil: rework purge tests so they don't poll
Fixes #5966
2016-07-18 14:19:09 -07:00
187faba3e0 pkg/fileutil: fix TouchDirAll, add CreateDirAll
os.MkdirAll never returns os.ErrExist.
And add another function to ensure deepest
directory is empty.
2016-06-22 15:54:17 -07:00
6480066054 *: misc typos and go vet fixes 2016-06-22 10:32:13 -07:00
47d5257622 pkg/fileutil: expose PrivateFileMode/DirMode 2016-06-10 15:22:14 -07:00
4f7622fb9a fileutil: avoid double preallocation 2016-06-10 00:27:59 -04:00
54aac4ab7e pkg/fileutil: fall back to truncate() if fallocate is interrupted
Fixes #5558
2016-06-06 09:52:34 -07:00
8b77de4e99 pkg: update LICENSE header 2016-05-12 20:48:53 -07:00
4ab1500a6d pkg/fileutil: wait up to 300ms for purge test
Fix https://github.com/coreos/etcd/issues/5231.

The issue shows that slow CI can take more than 200ms
for purging. This increase the loop iteration to wait
up to 300ms in case the disk is being slow.
2016-04-29 15:24:44 -07:00
8b6de5f85d fileutil: Sync on HFS/OSX needs to be handled differently.
A call file.Sync on OSX doesn't guarantee actual persistence on
physical drive media as the data can be cached in physical drive's
buffers. Hence calls to file.Sync need to be replaced with
fcntl(F_FULLFSYNC).
2016-04-18 21:49:04 -07:00
d4ff9364d4 Merge pull request #4861 from heyitsanthony/nfs-lock
pkg/fileutil: fix linux file locks over NFS
2016-04-16 08:59:10 -07:00
b0cc0e443c *: clean up if, bool comparison 2016-04-02 12:55:11 -07:00
3f1a1c3192 pkg/fileutil: lock file on Windows 2016-03-27 00:35:44 -07:00
877030ea9d pkg/fileutil: fix linux file locks over NFS
Fixes #4853
2016-03-25 16:28:29 -07:00
bd832e5b0a *: migrate Godeps to vendor/ 2016-03-22 17:10:28 -07:00
aafe717f2f fileutil: support file extending preallocate 2016-03-21 09:42:30 -07:00
7397e14c0a fileutil, wal: refactor file locking
File lock interface was more verbose than it needed to be while
simultaneously making it difficult to support systems (e.g., Windows)
that only permit locked writes on a single fd holding the lock.
2016-03-16 15:02:15 -07:00
e59efe45a1 wal: support fadatasync on linux 2016-03-13 17:22:53 -07:00
22a8bbd3b1 pkg/fileutil: clean up interface, comments 2016-03-09 09:19:56 -08:00
20461ab11a *: fix many typos 2016-01-31 21:42:39 -08:00
150e646b05 etcdserver: always check if the data dir is writable before starting etcd 2015-12-29 11:29:01 -08:00
8d368c4dba pkg/fileutil: fix error var shadow
Go tip complains about error variable shadowing at
https://travis-ci.org/coreos/etcd/jobs/98636879#L291-L292.
2015-12-23 23:56:26 -08:00
058f1449d6 pkg/fileutil: skip TestIsDirWriteable when running as root 2015-12-24 14:52:40 +09:00
96731f4525 Merge pull request #3991 from xiang90/fix_lock
pkg/fileutil: make TestLockAndUnlock less flaky on CI
2015-12-14 11:58:14 -08:00
be3e87b238 pkg/fileutil: make TestLockAndUnlock less flaky on CI
CI is slow sometime. To make the test less flaky, we increases the
timeout. This does not affect the correctness, but the test might
take longer to finish or to fail.
2015-12-14 11:38:07 -08:00
4e06510280 pkg/fileutil: fix shadowed variables 2015-12-12 09:38:26 -08:00
e93c07ba91 pkg/fileutil: make purge test more reliable 2015-12-09 10:34:38 -08:00