--experimental-memory-mlock support

The flag protects etcd memory from being swapped out to disk.
This can happen in memory constrained systems where mmaped bbolt
area is natural condidate for swapping out.

This flag should provide better tail latency on the cost of higher RSS
ram usage. If the experiment is successful, the logic should get moved
into bbolt layer, where we can protect specific bbolt instances
(e.g. avoid protecting both during defragmentation).

server/embed/config.go

@@ -345,6 +345,14 @@ type Config struct {
 	UnsafeNoFsync bool `json:"unsafe-no-fsync"`
 
 	ExperimentalDowngradeCheckTime time.Duration `json:"experimental-downgrade-check-time"`
+
+	// ExperimentalMemoryMlock enables mlocking of etcd owned memory pages.
+	// The setting improves etcd tail latency in environments were:
+	//   - memory pressure might lead to swapping pages to disk
+	//   - disk latency might be unstable
+	// Currently all etcd memory gets mlocked, but in future the flag can
+	// be refined to mlock in-use area of bbolt only.
+	ExperimentalMemoryMlock bool `json:"experimental-memory-mlock"`
 }
 
 // configYAML holds the config suitable for yaml parsing
@@ -433,6 +441,7 @@ func NewConfig() *Config {
 		EnableGRPCGateway: true,
 
 		ExperimentalDowngradeCheckTime: DefaultDowngradeCheckTime,
+		ExperimentalMemoryMlock:        false,
 	}
 	cfg.InitialCluster = cfg.InitialClusterFromName(cfg.Name)
 	return cfg