Commit Graph

23 Commits

Author SHA1 Message Date
43c6316420 robustness: remove explicit random seed in robustness tests.
Signed-off-by: redwrasse <mail@redwrasse.io>
2025-01-23 16:02:02 -08:00
d961c81813 Add jitter to failpoint injection to cover periodily executed compaction
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-11-20 19:58:01 +01:00
4da20aa06c fix: use require.NoError instead of t.Fatal(err) in tests package (part 1)
Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com>
2024-10-24 09:47:52 +00:00
8cf1121584 Move scenarios to subpackage
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-10-05 13:02:38 +02:00
279ffd57af Move watch util to client package.
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-10-05 13:02:37 +02:00
fdf8fde387 Remove flake caused failpoint in watch disrupting progress notifies
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-08-01 21:19:03 +02:00
c41e02f7b6 Add failpoint name to test name allowing us to track a per failpoint failures in testgrid
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-06-27 18:35:04 +02:00
2e04ee77b6 Avoid sending Compact request when LazyFS is enabled
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-06-18 08:36:24 +02:00
fc1863086c tests/robustness: unlock Delete/LeaseRevoke ops
We should return token to that bucket if `nonUniqueWriteLimiter.Take()`
return true. After unlock Delete/LeaseRevoke ops, the model should be
updated for replay function. There are two updates for `toWatchEvents`.

1. When leaveRevokes op has deleted few keys, we should generate
   `delete-operation` events based on alphabetical order of deleted
   keys.
2. When putWithLease op hits non-exist lease, we should ignore that
   update event.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-06-06 20:40:08 +08:00
d8bb19327b Prevent picking a failpoint that waiting till snapshot that doesn't support lower snapshot catchup entries but allow reproducing issue #15271
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-13 12:08:42 +02:00
c4e3b61a1c Record operation from failpoint injection
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-05-01 19:20:22 +02:00
9fcde37447 Persist member data with lazyfs enabled
Discovered turning off LazyFS before creating the report might result in
empty server directory. This PR moves cluster shutdown to defer executed
after we generate report and copies the data from lazyfs directory.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-23 10:19:05 +02:00
f285330d46 Don't require minimal for failpoint injection period
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-20 10:34:51 +02:00
569693be8d Utilize WAL to patch operation history
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-14 12:09:38 +02:00
3a23994fbf Make no failpoint error more readable
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-07 15:13:59 +02:00
e2bb8c698f Limit a timeout in testing robustness validation
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-04-06 12:28:57 +02:00
0976398964 tests/robustness: address golangci var-naming issues
Signed-off-by: Ivan Valdes <ivan@vald.es>
2024-03-25 16:27:05 -07:00
3a351c2fec Revert "tests/robustness: check for compaction before prevKV validation"
This reverts commit 5d7f58d14b.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-02-21 16:15:40 +01:00
5d7f58d14b tests/robustness: check for compaction before prevKV validation
We can check for the condition that Kubernetes checks for, i.e.
prevKV can be nil iff the event is not a create a event, only if
we know whether compaction has occured or not. If compaction has
occured, prevKV can be nil and that is completely valid.

This commit checks if compaction took place during the test run
by querying the /metrics endpoint. Based on if compaction occured,
we now check the Kubernetes condition in the prevKV robustness test.

Signed-off-by: Madhav Jivrajani <madhav.jiv@gmail.com>
2024-02-19 17:05:59 +05:30
f0d73c9d12 Separate robustness test scenarios and increase number of times we run exploratory tests in nightly
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2024-01-16 17:17:54 +01:00
5175652a8e Abort if failpoint injecton failed
If one of nodes is unhealthy the test would never finish as watchers
would never reach max revision.

Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-12-03 17:26:51 +01:00
452e820516 Make the main_test the entrypoint and move senario generation to separate file
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-10-16 22:10:41 +02:00
d475cf81a0 tests: Rename linearizability tests to robustness
Signed-off-by: Marek Siarkowicz <siarkowicz@google.com>
2023-02-26 14:36:18 +01:00