Commit Graph

66 Commits

Author SHA1 Message Date
449c116a44 storage: shadowed err var fix at go-tip 2016-02-14 19:49:03 -08:00
bfa5e310a9 *: detach keys from leases
1. deatch when a key is removed
2. deatch when the key's lease changes
3. potentially deatch when restroing a tombstone key
2016-02-11 15:31:25 -08:00
d314345e6d *: add a scheduler and use it to schedule compaction 2016-02-10 14:27:08 -08:00
4ba1ec6a4d Merge pull request #4363 from xiang90/watch
storage: simplify watch store
2016-01-31 16:49:44 -08:00
611751aee2 storage: simplify watch store
We decided that we will not support modifing the same
key in one txn multiple times. That can simlify the current
code/design a lot.
2016-01-31 16:33:07 -08:00
f6215574f2 *: kv range to return current revision
This changes the behavior of KV's range and tx range to return
current revision rather than range revision. This makes populating
range response header easier.
2016-01-30 17:37:34 -08:00
128b5e7387 etcdserver: check invalid range in txn 2016-01-25 20:21:17 -08:00
ecba9b61cf storage: start initial revision at 1
When the start revision was 0, there was no way to safely watch
starting from the first store revision.
2016-01-14 17:05:56 -08:00
f01c8188f8 *: rename lease.DeleteableRange to lease.RangeDeleter 2016-01-09 11:01:58 -08:00
f5753f2f51 *: support lease Attach
Now we can attach keys to leases. And revoking the lease removes all
the attached keys of that lease.
2016-01-09 11:01:58 -08:00
8774d53459 Merge pull request #4158 from heyitsanthony/nolease-to-leasepkg
lease: move storage.NoLease to lease package
2016-01-07 10:43:51 -08:00
f9af744be3 lease: move storage.NoLease to lease package 2016-01-07 10:33:44 -08:00
43a777b7a2 *: get snapshot from backend
We should get snapshot from backend, not just KV.
We plan to store the lease data into backend too.
2016-01-06 22:40:23 -08:00
1714290f4e storage: support recovering from backend
We want the KV to support recovering from backend to avoid
additional pointer swap. Or we have to do coordination between
etcdserver and API layer, since API layer might have access to
kv pointer and use a closed kv.
2016-01-06 21:16:55 -08:00
5dd3f91903 *: make backend outside kv
KV and lease will share the same backend. Thus we need to make
backend outside KV.
2016-01-05 19:55:29 -08:00
09b420f08c *: move leaseID typedef to lease pkg 2016-01-05 10:18:17 -08:00
4336278b44 *: support put with lease 2016-01-04 15:54:06 -08:00
0b2d31f3bc storage: decouple default parameters and storage creation
newStore() uses constants for some important parameters
e.g. batchInerval. It is not suitable for other storage users
(e.g. tools/benchmark). This commit decouples the default parameters
from storage creation. etcd should use a newly added function
newDefaultStore() for creating a store with default parameters.
2015-12-31 22:28:59 +09:00
ecc3e15a46 storage: delete RangeHistory
This has been replaced by operations inside `syncWatchings`.
2015-12-28 11:37:32 -08:00
84d777305d storage: use only one mutex for store struct
Mutex is a variable, which means there needs to be only one mutex
value per scope. We don't need a separate mutex inside store struct,
**if we assume that `TxnPut` and `TxnRange` are called ONLY ONCE
per transaction (between `TxnBegin` and `TxnEnd`)**, as documented.
2015-12-21 11:42:40 -08:00
2f74f76025 storage: remove the event concept from key-value layer
The point is to decouple the key-value storage layer and the
event notification layer clearly. It gives the watchableKV the
flexibility to define whatever event structure it wants without
breaking the ondisk format at key-value storage layer.

Changes:

1. change the format of key and value stored in backend

Store KeyValue struct instead of Event struct in backend value for
better abstraction as xiang suggests. And record the corresponded
action in the backend key.

2. Remove word 'event' from functions
2015-11-17 20:35:49 -08:00
263b270708 etcdserver: commit v3 storage before releasing WAL
This ensures that v3 storage could always find the following log entries
when restart.
2015-10-26 21:06:08 -07:00
15ed6d8268 etcdserver: save consistent index into v3 storage
This helps to recover consistent index when restart in the future.
2015-10-24 09:27:24 -07:00
6556bf1643 storage: remove the endRev of watcher 2015-10-17 15:59:49 -07:00
21179d929f Merge pull request #3616 from yichengq/storage-txn
storage: hold batchTx lock during KV txn
2015-10-05 17:12:52 -07:00
0aa2f1192a storage: add metrics for db total size 2015-10-05 16:56:30 -07:00
c97dda766e storage: hold batchTx lock during KV txn
One txn is treated as atomic, and might contain multiple Put/Delete/Range
operations. For now, between these operations, we might call forecCommit
to sync the change to disk, or backend may commit it in background.
Thus the snapshot state might contains an unfinished multiple objects
transaction, which is dangerous if database is restored from the snapshot.

This PR makes KV txn hold batchTx lock during the process and avoids
commit to happen.
2015-10-03 16:01:05 -07:00
36f4303fc3 storage/etcdserver: update KV.Snapshot function
When using Snapshot function, it is expected:
1. know the size of snapshot before writing data
2. split snapshot-ready phase and write-data phase. so we could cut
snapshot first and write data later.

Update its interface to fit the requirement of etcdserver.
2015-10-03 10:15:23 -07:00
385e17583f storage: fix hash by iterating kv 2015-09-23 11:28:33 -07:00
d72914c36f storage: clarify comment for store.RangeEvents and fix related bugs
Change to the function:
1. specify the meaning of startRev and endRev parameters
2. specify the meaning of returned nextRev

Moreover, it adds unit tests for the function.
2015-09-19 23:17:03 -07:00
7848ac3979 *: add missing license headers 2015-09-15 14:09:01 -07:00
6d1f0ce89f storage: add rev into kv interface 2015-09-15 12:11:00 -07:00
c082488e23 Merge pull request #3507 from yichengq/watch
storage: support basic watch
2015-09-15 00:04:36 -07:00
ec43e0a4c3 storage: introduce WatchableKV and watch feature
WatchableKV is an interface upon KV, and supports watch feature.
2015-09-14 23:53:03 -07:00
714b5e0b08 storage: support hash state 2015-09-13 21:34:58 -07:00
3f18ded10a *: v3api index->revision 2015-09-04 10:41:20 -07:00
9175df7c71 storage: correct revision for range when deleteRange
to make it logically reasonable.
2015-09-03 19:12:27 -07:00
797a4796d9 storage: remove check for DELETE type KeyValue
kvindex always returns kvs that exist at given revision, so there is no
need to check for whether the KeyValue range from backend is DELETE type.
2015-09-03 19:12:27 -07:00
00e31f13a6 storage: remove unnecessary rev parameter 2015-09-03 19:12:27 -07:00
b5838edb93 storage: add initial metrics for kv 2015-08-28 13:41:42 -07:00
9960651c3f storage: let range work in the process of txn
range should work in the process of txn to help check the status during the
txn.
2015-08-23 20:59:06 -07:00
6d97dcaf3f storage: ensure that desired compaction is persisted
It needs to persist the desired compaction, so it won't forget the compaction
if it crashes later.
2015-08-23 20:59:06 -07:00
353f10ca2b storage: reject to compact on future rev
Compaction on future rev is unreasonable.
2015-08-23 20:59:06 -07:00
47b243be5d storage: let TxnDeleteRange return rev if no error
If it doesn't return error, it should return valid rev.
2015-08-23 20:59:06 -07:00
acd7a92f03 storage: reversion -> revision 2015-08-20 08:39:07 -07:00
53a77fa519 *: tnx -> txn 2015-07-24 23:21:09 +08:00
883bb47dcf Merge pull request #3074 from xiang90/storage_restore
storage: correctly restore create and ver
2015-06-30 09:20:19 -07:00
f8b947a00b storage: fix small issues 2015-06-29 22:02:21 -07:00
433f2ee1bc storage: correctly restore create and ver
Add a restore func to correctly restore create reversion and
version of keys for the index.
2015-06-29 13:44:43 -07:00
ccca2b04da storage: save version 2015-06-29 13:15:09 -07:00