reftable/stack: perform auto-compaction with transactional interface
Whenever updating references or reflog entries in the reftable stack, we need to add a new table to the stack, thus growing the stack's length by one. The stack can grow to become quite long rather quickly, leading to performance issues when trying to read records. But besides performance issues, this can also lead to exhaustion of file descriptors very rapidly as every single table requires a separate descriptor when opening the stack. While git-pack-refs(1) fixes this issue for us by merging the tables, it runs too irregularly to keep the length of the stack within reasonable limits. This is why the reftable stack has an auto-compaction mechanism: `reftable_stack_add()` will call `reftable_stack_auto_compact()` after its added the new table, which will auto-compact the stack as required. But while this logic works alright for `reftable_stack_add()`, we do not do the same in `reftable_addition_commit()`, which is the transactional equivalent to the former function that allows us to write multiple updates to the stack atomically. Consequentially, we will easily run into file descriptor exhaustion in code paths that use many separate transactions like e.g. non-atomic fetches. Fix this issue by calling `reftable_stack_auto_compact()`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:

committed by
Junio C Hamano

parent
15f98b602f
commit
5c086453ff
@ -584,6 +584,12 @@ int reftable_addition_commit(struct reftable_addition *add)
|
||||
add->new_tables_len = 0;
|
||||
|
||||
err = reftable_stack_reload(add->stack);
|
||||
if (err)
|
||||
goto done;
|
||||
|
||||
if (!add->stack->disable_auto_compact)
|
||||
err = reftable_stack_auto_compact(add->stack);
|
||||
|
||||
done:
|
||||
reftable_addition_close(add);
|
||||
return err;
|
||||
|
Reference in New Issue
Block a user