Commit Graph

6838 Commits

Author SHA1 Message Date
516be7a781 travis: only run unit tests
Travis has chronic problems successfully running the integration suite -
and we've successfully moved to Semaphore for that purpose - but can
still be useful as a fail-fast option for testing unit tests and formatting.
2015-10-26 12:47:15 -07:00
52782cf8ee etcdmain: make flags and formats idential
This makes flagsline and config.go identical in its flag description and some
punctuation conventions.
2015-10-25 06:31:37 -07:00
d44b79c3c9 Merge pull request #3748 from coreos/revert-3737-rafthttp-continue
Revert "rafthttp: fix wrong return in pipeline.handle"
2015-10-24 21:05:52 -07:00
5eda45ece6 Revert "rafthttp: fix wrong return in pipeline.handle" 2015-10-24 20:25:56 -07:00
dbba5bb373 Merge pull request #3737 from yichengq/rafthttp-continue
rafthttp: fix wrong return in pipeline.handle
2015-10-24 19:42:38 -07:00
7e38f05ceb Merge pull request #3742 from yichengq/save-index
etcdserver: save consistent index into v3 storage
2015-10-24 09:48:28 -07:00
15ed6d8268 etcdserver: save consistent index into v3 storage
This helps to recover consistent index when restart in the future.
2015-10-24 09:27:24 -07:00
f648d52afe rafthttp: fix wrong return in pipeline.handle
pipeline.handle is a long-living one, and should continue to receive
next message to send out when current message fails to send. So it
should `continue` instead of `return` here.
2015-10-23 17:00:03 -07:00
41cb39b68a storage: Get -> ConsistentIndex in ConsistentIndexGetter
To make the method name more specific in the context.
2015-10-23 16:40:55 -07:00
4f47b08cf6 Merge pull request #3744 from yichengq/fix-sem
raft: extend wait timeout in TestMultiNodeAdvance
2015-10-23 13:20:52 -07:00
bf3057e5bd raft: extend wait timeout in TestMultiNodeAdvance
This fixes the failure met in semaphore CI:

```
--- FAIL: TestMultiNodeAdvance-2 (0.01s)
		multinode_test.go:458: expect Ready after Advance, but there is
		no Ready available
```
2015-10-23 12:08:24 -07:00
01559fafeb Merge pull request #3741 from yichengq/receive-restore
etcdserver: restore KV snapshot when receiving snapshot
2015-10-23 09:24:17 -07:00
cacc0d6432 etcdserver: restore KV snapshot when receiving snapshot
When a slow follower receives the snapshot sent from the leader, it
should rename the snapshot file to the default KV file path, and
restore KV snapshot.

Have tested it manually and it works pretty well.
2015-10-23 08:43:26 -07:00
d33c26c20a Merge pull request #3730 from yichengq/storage-consistent
storage: add consistentWatchableStore
2015-10-23 08:15:04 -07:00
4fb4bc3ca8 storage: add consistentWatchableStore
consistentWatchableStore maintains an index that is always consistent
with the latest txn. The index could be used to indicate the progress
of the store so far when recovery.
2015-10-22 22:54:51 -07:00
ae62a77de6 Merge pull request #3729 from xiang90/mem_bench
doc: add benchmark doc for new storage pkg
2015-10-22 10:54:43 -07:00
e3cedeeb12 doc: add benchmark doc for new storage pkg 2015-10-22 13:53:03 -04:00
2feccd3fa4 Merge pull request #3733 from yichengq/fix-wait-timeout
pkg/transport: extend wait timeout for write
2015-10-22 13:07:14 -04:00
d3ebecdddd pkg/transport: extend wait timeout for write
This helps the test to pass safely in semaphore CI.

Based on my manual testing, it may take at most 500ms to return
error in semaphore CI, so I set 1s as a safe value.
2015-10-21 18:27:21 -07:00
8b08fff1e9 Merge pull request #3731 from yichengq/storage-kv
storage: fix WatchableKV interface and refine comment
2015-10-21 17:27:24 -07:00
01b163e77d Merge pull request #3588 from gyuho/storage/watchable_store.go-use-map-for-unsynced
storage/watchable_store.go: use map for unsynced
2015-10-21 16:50:15 -07:00
44cecb8624 Merge pull request #3732 from yichengq/config-header
docs/configuration: fix heading hierarchy
2015-10-21 15:50:38 -07:00
f73d0ed1d9 storage: use map for watchable store unsynced
This is for `TODO: use map to reduce cancel cost`.
I switched slice to map, and benchmark results show
that map implementation performs better, as follows:

```
[1]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       215212        1307          -99.39%
BenchmarkWatchableStoreUnsyncedCancel-2     120453        710           -99.41%
BenchmarkWatchableStoreUnsyncedCancel-4     120765        748           -99.38%
BenchmarkWatchableStoreUnsyncedCancel-8     121391        719           -99.41%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       200           1             -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[2]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       212550        1117          -99.47%
BenchmarkWatchableStoreUnsyncedCancel-2     120927        691           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4     120752        699           -99.42%
BenchmarkWatchableStoreUnsyncedCancel-8     121012        688           -99.43%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       197           1             -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[3]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       214268        1183          -99.45%
BenchmarkWatchableStoreUnsyncedCancel-2     120763        759           -99.37%
BenchmarkWatchableStoreUnsyncedCancel-4     120321        708           -99.41%
BenchmarkWatchableStoreUnsyncedCancel-8     121628        680           -99.44%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       200           1             -99.50%
BenchmarkWatchableStoreUnsyncedCancel-2     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     138           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[4]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       208332        1089          -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     121011        691           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-4     120678        681           -99.44%
BenchmarkWatchableStoreUnsyncedCancel-8     121303        721           -99.41%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       194           1             -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     139           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     139           0             -100.00%

[5]:
benchmark                                   old ns/op     new ns/op     delta
BenchmarkWatchableStoreUnsyncedCancel       211900        1097          -99.48%
BenchmarkWatchableStoreUnsyncedCancel-2     121795        753           -99.38%
BenchmarkWatchableStoreUnsyncedCancel-4     123182        700           -99.43%
BenchmarkWatchableStoreUnsyncedCancel-8     122820        688           -99.44%

benchmark                                   old allocs     new allocs     delta
BenchmarkWatchableStoreUnsyncedCancel       0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-2     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-4     0              0              +0.00%
BenchmarkWatchableStoreUnsyncedCancel-8     0              0              +0.00%

benchmark                                   old bytes     new bytes     delta
BenchmarkWatchableStoreUnsyncedCancel       198           1             -99.49%
BenchmarkWatchableStoreUnsyncedCancel-2     140           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-4     141           0             -100.00%
BenchmarkWatchableStoreUnsyncedCancel-8     141           0             -100.00%
```
2015-10-21 15:30:15 -07:00
f95d18f766 docs/configuration: fix heading hierarchy
to make it consistent with other sections in the doc.
2015-10-21 15:23:48 -07:00
027c073d55 storage: refine Watch comment in WatchableKV
Explain explicitly how these arguments are used.
2015-10-21 14:47:52 -07:00
56b7584418 Merge pull request #3725 from joshix/hdinghier-mulligan
Documentation: Fix heading hierarchy.
2015-10-21 13:52:57 -07:00
2673e657e6 storage: fix WatchableKV interface
We delete endRev from the watch functionality, so the interface needs
to be fixed.
2015-10-21 11:50:25 -07:00
35eb26ef5d Merge pull request #3726 from yichengq/watch-store
storage: add store field in watchableStore
2015-10-21 11:07:45 -07:00
0f7374ce89 storage: KV field -> store field in watchableStore
We need to access the underlying store to use its RangeEvents function.
It is not good to use unnecessary type conversion.

The underlying store is also needed for further store upon
watchableStore.
2015-10-20 19:23:20 -07:00
8d3ed0176c Merge pull request #3727 from yichengq/govet
raft: fix malformed example name
2015-10-20 16:51:47 -07:00
01806c3e80 raft: fix malformed example name
It is reported by latest govet:
```
gopath/src/github.com/coreos/etcd/raft/example_test.go:26: Example_Node
has malformed example suffix: Node
```
2015-10-20 16:40:01 -07:00
98bdeab53b Documentation: Fix heading hierarchy.
Correct the hierarchy of Markdown symbols in document headings.
2015-10-20 15:26:49 -07:00
704bff0c77 Merge pull request #3724 from coreos/philips-patch-1
README: fix language for release binaries
2015-10-20 14:22:02 -07:00
5b5b0ef060 README: attempt to make it even clearer 2015-10-20 14:17:33 -07:00
b38e21a9e9 README: fix language for release binaries
Address confusion on where to find stuff and make it easier to find with keywords for various operating systems.
2015-10-20 14:07:40 -07:00
9635d8d94c Merge pull request #3720 from yichengq/clean-streamAppV1
rafthttp: deprecate streamTypeMsgApp and remove msgApp stream sent restriction due to streamTypeMsgApp
2015-10-20 10:37:51 -07:00
de669be6d6 Merge pull request #3683 from yichengq/raft-block
etcdserver: fix raft state machine may block
2015-10-20 09:44:34 -07:00
ab5df57ecf etcdserver: fix raft state machine may block
When snapshot store requests raft snapshot from etcdserver apply loop,
it may block on the channel for some time, or wait some time for KV to
snapshot. This is unexpected because raft state machine should be unblocked.

Even worse, this block may lead to deadlock:
1. raft state machine waits on getting snapshot from raft memory storage
2. raft memory storage waits snapshot store to get snapshot
3. snapshot store requests raft snapshot from apply loop
4. apply loop is applying entries, and waits raftNode loop to finish
messages sending
5. raftNode loop waits peer loop in Transport to send out messages
6. peer loop in Transport waits for raft state machine to process message

Fix it by changing the logic of getSnap to be asynchronously creation.
2015-10-20 09:19:34 -07:00
b61eaf3335 rafthttp: msgApp{Reader/Writer} -> msgAppV2{Reader/Writer}
To make what it serves more clear.
2015-10-20 08:28:06 -07:00
5060b2f322 rafthttp: send all MsgApp on stream msgAppV2
For stream msgAppV2, as long as the message is MsgApp type, it should be sent
through stream msgAppV2.
2015-10-20 08:23:36 -07:00
33231fccdd rafthttp: fix wrong stream name returned by pick
msgAppWriter uses streamAppV2 type, and it should return the correct name.
2015-10-20 08:17:06 -07:00
f725f6a552 rafthttp: deprecate streamTypeMsgApp
streamTypeMsgApp is only used in etcd 2.0. etcd 2.3 should not talk to
etcd 2.0, either send or receive requests. So I deprecate streamTypeMsgApp
and its related stuffs from rafthttp package.

updating term is only used from streamTypeMsgApp, so it is removed too.
2015-10-20 08:15:54 -07:00
eb7bce893e Merge pull request #3721 from mitake/servevars
etcdserver: don't allow methods other than GET in /debug/vars
2015-10-20 08:01:38 -07:00
1b0c65c299 etcdserver: don't allow methods other than GET in /debug/vars
Currently, /debug/vars seems to allow all types of methods e.g. PUT,
POST, etc. However, this path is a readonly stuff so it should allow
GET only.
2015-10-20 17:19:42 +09:00
7dcb99b60e Merge pull request #3656 from endocode/kayrus/client_doc
Added example on how to get node's value
2015-10-19 20:41:12 -07:00
e5082fce54 Merge pull request #3718 from gyuho/gyuho_README
README: fix typo
2015-10-19 17:14:46 -07:00
7d326087f9 README: fix typo
It looks like a typo. Or is it elided sentence?
Thanks,
2015-10-19 17:03:40 -07:00
4e9f137d1b Merge pull request #3716 from yichengq/add-sem-badge
README: add semaphore CI status badge into README
2015-10-19 16:25:21 -07:00
6ebd62a869 README: add semaphore CI status badge into README 2015-10-19 16:08:40 -07:00
32dd4d5de3 Merge pull request #3657 from xiang90/fix_remove
etcdserver: skip updating attr if the member does not exist
2015-10-19 13:35:57 -07:00