Commit Graph

91 Commits

Author SHA1 Message Date
53e7ddbc66 wal: do not call fsync when it is not necessary 2016-03-14 11:52:06 -07:00
e59efe45a1 wal: support fadatasync on linux 2016-03-13 17:22:53 -07:00
71c2a9bb3c *: fix minor typos, comments 2016-01-30 18:15:56 -08:00
a4de207d53 wal: fix shadowed variables
Fixes for https://github.com/coreos/etcd/issues/3954.
2015-12-12 09:38:26 -08:00
55adfcb428 wal: minor typo in wal pkg
Fixes a minor typo in wal.go.
Thanks!
2015-11-12 15:23:12 -08:00
1c7f52d931 wal: use Histogram for syncDuration 2015-10-17 12:45:43 -07:00
39a4b6a5e5 pkg/fileutil: support perallocate 2015-08-06 10:10:58 -07:00
ba76e27875 wal: log errors in wal.Close()
This patch adds error logging in wal.Close() if unlocking and
destroying fail. Though it is hard to handling the errors, logging
would be helpful for trouble shooting.
2015-08-05 15:03:45 +09:00
f59da0e453 *:fix point-in-time backup
Backup process should be able to read all WALs until io.EOF to
generate a point-in-time backup.

Our WAL file is append-only. And the backup process will lock all
files before start reading, which can prevent the gc routine from
removing any files in the middle.
2015-06-15 11:12:28 -07:00
711451ce2d *: rename logger to plog 2015-06-02 14:58:24 -07:00
185d2bced4 wal: use leveled logger 2015-06-01 13:38:50 -07:00
89242d4659 wal: better log msg 2015-04-09 09:54:20 -07:00
53792ccbdc wal: never leave a corrupted wal file
If the process dies during wal.cut(), it might leave a corrupted wal
file. This commit solves the problem by creating a temp wal file first,
then atomically rename it to a wal file when we are sure it is vaild.
2015-04-08 15:57:20 -07:00
44de670de7 wal: allow at most one WAL function called at one time
SaveSnap and Save are called in separate goroutines now. Allow at most
one WAL function being called at one time to protect internal fields and
guarantee execution order.
Or one possible bug is that the new cut file is started with snapshot
entry instead of crc entry.
2015-04-08 00:34:30 -07:00
c32cca3a4f wal: reduce allocation when encoding entries 2015-03-30 19:20:46 -07:00
8b4eed29e5 wal: fix the unexpectedEOF error in the last wal.
It is safe to repair the unexpectedEOF error in the last wal. raft
will not send out message before the entry successfully comitted
into wal. Thus we can safely truncate the last entry in the wal
to repair.
2015-03-28 21:08:14 -07:00
6e6669d696 wal: releastTo should work with large release index 2015-03-24 22:34:26 -07:00
b66eb3d81c wal: fix ReleaseLockTo
ReleaseLockTo should not release the lock on the WAL
segment that is right before the given index. When
restarting etcd, etcd needs to read from the WAL segment
that has a smaller index than the snapshot index.

The correct behavior is that ReleaseLockTo releases
the locks w is holding so that w only holds one lock
that has an index smaller than the given index.
2015-03-09 19:52:54 -07:00
86429264fb wal: support auto-cut in wal
WAL should control the cut logic itself. We want to do falloc to
per allocate the space for a segmented wal file at the beginning
and cut it when it size reaches the limit.
2015-02-28 11:18:59 -08:00
84485643fe *: expose wal metrics at /metrics 2015-02-28 11:06:11 -08:00
fb1a28c65d *: vendor prometheus 2015-02-28 11:06:11 -08:00
9e63b1fb63 wal: record metrics 2015-02-28 10:12:35 -08:00
e966e565c4 etcdctl/backup_command: handle datadir with missed snapshot mark
This helps to recover from the data dir created in v2.0.0-rc1.
2015-01-29 13:32:59 -08:00
f1ed69e883 *: switch to line comments for copyright
Build tags are not compatible with block comments.
Also adds copyright header to a few places it was missing.
2015-01-26 09:53:30 -08:00
270e67db84 wal: not export unnecessary public functions 2015-01-09 14:55:10 -08:00
6460e49a33 wal: save empty snapshot when create
So caller can open at empty snapshot to read all entries.
2015-01-06 19:48:21 -08:00
84f62f21ee wal: record and check snapshot 2015-01-06 16:27:40 -08:00
1ec98cb795 pkg/fileutil: sort filenames during ReadDir 2014-12-18 16:36:11 -08:00
53bf7e4b5e wal: rename openAtIndex -> open; OpenAtIndexUntilUsing -> openNotInUse 2014-12-14 19:33:06 -08:00
f538cba272 *: do not backup files still in use 2014-12-14 19:27:22 -08:00
ea94d19147 *: lock the in using files; do not purge locked the wal files 2014-12-14 19:27:22 -08:00
b9bf957c6d wal: sync after writing data to disk in Cut function 2014-12-04 22:56:34 -08:00
af0f34c595 wal: save latest state into new WAL
So we could always read out state when open at valid index.
2014-12-04 12:19:21 -08:00
aa61009560 wal: not return ErrIndexNotFound in ReadAll
This IndexNotFound case is reasonable now because we don't write dummy
entries into wals any more.
2014-12-02 00:28:54 -08:00
d3db010190 *: support purging old wal/snap files 2014-12-01 11:50:17 -08:00
5a964f49a5 wal: propagate errors 2014-11-08 17:16:23 -08:00
075ab6415f Merge pull request #1587 from xiangli-cmu/fix_wal
wal: sync before returning from create
2014-11-03 15:58:47 -08:00
ff1f5a9d57 wal: sync before returning from create 2014-11-03 14:28:59 -08:00
433b4138c5 etcdserver: panic on storage error
It is a critical error to etcd, and etcd is not able to recover it now.
2014-11-03 10:46:04 -08:00
7a4d42166b *: add license header to all source files 2014-10-17 15:41:22 -07:00
f98fbbfc14 *: proto refactoring 2014-10-14 21:07:23 +08:00
447caf1afc etcdserver/wal: record info at the head of WAL file 2014-10-10 11:57:09 -07:00
8bbbaa88b2 *: raft related int64 -> uint64 2014-10-09 14:29:21 +08:00
1d5d2e3726 *: Id -> ID for protobuf types
We use ID instead of Id in this project based on golang conventions.
2014-09-26 11:49:30 -07:00
cf4af47f7e wal: remove logging messages
These are noisy and unhelpful by default. We can re-add if at some point
we reintroduce debugging/multiple log levels.
2014-09-22 19:08:47 -07:00
df70f653a4 Merge pull request #1099 from unihorn/128
wal: OpenFromIndex fails if it cannot find previous index
2014-09-17 16:00:51 -07:00
29f6d8a9e6 wal: add ErrFileNotFound and ErrIndexNotFound 2014-09-17 15:58:06 -07:00
7160b5ae26 wal: OpenFromIndex fails if it cannot find previous index
Example:
We save entry 1, 2, 3 to WAL.
If we try to open 100, it should fail.
2014-09-17 15:07:07 -07:00
ab61a8aa9a *: init for on disk snap support 2014-09-17 13:56:12 -07:00
de21c39ca5 raft: isStateEqual -> isHardStateEqual, IsEmptyState -> IsEmptyHardState 2014-09-16 13:55:00 -07:00