Compare commits

..

9 Commits

Author SHA1 Message Date
bcd6bc478a Git 2.38-rc2
We have small updates since -rc1 but none of them is about a new
thing and there is no updates to the release notes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-27 11:25:52 -07:00
2a7d63a245 Merge branch 'ds/bitmap-lookup-remove-tracing'
Perf-fix.

* ds/bitmap-lookup-remove-tracing:
  pack-bitmap: remove trace2 region from hot path
2022-09-26 21:46:51 -07:00
89a1ab8fb5 pack-bitmap: remove trace2 region from hot path
The trace2 region around the call to lazy_bitmap_for_commit() in
bitmap_for_commit() was added in 28cd730680 (pack-bitmap: prepare to
read lookup table extension, 2022-08-14). While adding trace2 regions is
typically helpful for tracking performance, this method is called
possibly thousands of times as a commit walk explores commit history
looking for a matching bitmap. When trace2 output is enabled, this
region is emitted many times and performance is throttled by that
output.

For now, remove these regions entirely.

This is a critical path, and it would be valuable to measure that the
time spent in bitmap_for_commit() does not increase when using the
commit lookup table. The best way to do that would be to use a mechanism
that sums the time spent in a region and reports a single value at the
end of the process. This technique was introduced but not merged by [1]
so maybe this example presents some justification to revisit that
approach.

[1] https://lore.kernel.org/git/pull.1099.v2.git.1640720202.gitgitgadget@gmail.com/

To help with the 'git blame' output in this region, add a comment that
warns against adding a trace2 region. Delete a test from t5310 that used
that trace output to check that this lookup optimization was activated.
To create this kind of test again in the future, the stopwatch traces
mentioned earlier could be used as a signal that we activated this code
path.

Helpedy-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-26 12:09:18 -07:00
4fd6c5e444 Merge branch 'ac/bitmap-lookup-table'
Grammofix.

* ac/bitmap-lookup-table:
  pack-bitmap: improve grammar of "xor chain" error message
2022-09-23 11:07:49 -07:00
0d14f80f94 Merge branch 'ma/scalar-to-main-fix'
Fix manpage generation.

* ma/scalar-to-main-fix:
  cmd-list.perl: fix identifying man sections
2022-09-23 11:07:48 -07:00
32c6fff4b8 cmd-list.perl: fix identifying man sections
We attribute each documentation text file to a man section by finding a
line in the file that looks like "gitfoo(<digit>)". Commit cc75e556a9
("scalar: add to 'git help -a' command list", 2022-09-02) updated this
logic to look not only for "gitfoo" but also "scalarfoo". In doing so,
it forgot to account for the fact that after the updated regex has found
a match, the man section is no longer to be found in `$1` but now lives
in `$2`.

This makes our git(1) manpage look as follows:

  Main porcelain commands
       git-add(git)
           Add file contents to the index.

  [...]

       gitk(git)
           The Git repository browser.

       scalar(scalar)
           A tool for managing large Git repositories.

Restore the man sections by not capturing the (git|scalar) part of the
match into `$1`.

As noted by Ævar [1], we could even match any "foo" rather than just
"gitfoo" and "scalarfoo", but that's a larger change. For now, just fix
the regression in cc75e556a9.

[1] https://lore.kernel.org/git/220923.86wn9u4joo.gmgdl@evledraar.gmail.com/#t

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-23 10:01:07 -07:00
711340c797 pack-bitmap: improve grammar of "xor chain" error message
Signed-off-by: Alex Henrie <alexhenrie24@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-23 08:54:05 -07:00
4b79ee4b0c Merge branch 'jk/list-objects-filter-cleanup'
Fix uninitialized memory access in a recent fix-up that is already
in -rc1.

* jk/list-objects-filter-cleanup:
  list-objects-filter: initialize sub-filter structs
2022-09-22 15:30:47 -07:00
4eaed7c2f2 list-objects-filter: initialize sub-filter structs
Since commit c54980ab83 (list-objects-filter: convert filter_spec to a
strbuf, 2022-09-11), building with SANITIZE=undefined triggers an error
in t5616.

The problem is that we end up with a strbuf that has been
zero-initialized instead of via STRBUF_INIT. Feeding that strbuf to
strbuf_addbuf() in list_objects_filter_copy() means we will call memcpy
like:

   memcpy(some_actual_buffer, NULL, 0);

This works on most systems because we're copying zero bytes, but it is
technically undefined behavior to ever pass NULL to memcpy.

Even though c54980ab83 is where the bug manifests, that is only because
we switched away from a string_list, which is OK with being
zero-initialized (though it may cause other problems by not duplicating
the strings, it happened to be OK in this instance).

The actual bug is caused by the commit before that, 2a01bdedf8
(list-objects-filter: add and use initializers, 2022-09-11). There we
consistently initialize the top-level filter structs, but we forgot the
dynamically allocated ones we stick in filter_options->sub when creating
combined filters.

Note that we need to fix two spots here: where we parse a "combine:"
filter, but also where we transform from a single-filter into a combined
one after seeing multiple "--filter" options. In the second spot, we'll
do some minor refactoring to avoid repeating our very-long array index.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-22 12:43:04 -07:00
5 changed files with 11 additions and 15 deletions

View File

@ -10,7 +10,7 @@ sub format_one {
$state = 0;
open I, '<', "$name.txt" or die "No such file $name.txt";
while (<I>) {
if (/^(git|scalar)[a-z0-9-]*\(([0-9])\)$/) {
if (/^(?:git|scalar)[a-z0-9-]*\(([0-9])\)$/) {
$mansection = $1;
next;
}

View File

@ -1,7 +1,7 @@
#!/bin/sh
GVF=GIT-VERSION-FILE
DEF_VER=v2.38.0-rc1
DEF_VER=v2.38.0-rc2
LF='
'

View File

@ -143,6 +143,7 @@ static int parse_combine_subfilter(
ALLOC_GROW_BY(filter_options->sub, filter_options->sub_nr, 1,
filter_options->sub_alloc);
list_objects_filter_init(&filter_options->sub[new_index]);
decoded = url_percent_decode(subspec->buf);
@ -263,6 +264,8 @@ void parse_list_objects_filter(
parse_error = gently_parse_list_objects_filter(
filter_options, arg, &errbuf);
} else {
struct list_objects_filter_options *sub;
/*
* Make filter_options an LOFC_COMBINE spec so we can trivially
* add subspecs to it.
@ -273,10 +276,11 @@ void parse_list_objects_filter(
filter_spec_append_urlencode(filter_options, arg);
ALLOC_GROW_BY(filter_options->sub, filter_options->sub_nr, 1,
filter_options->sub_alloc);
sub = &filter_options->sub[filter_options->sub_nr - 1];
parse_error = gently_parse_list_objects_filter(
&filter_options->sub[filter_options->sub_nr - 1], arg,
&errbuf);
list_objects_filter_init(sub);
parse_error = gently_parse_list_objects_filter(sub, arg,
&errbuf);
}
if (parse_error)
die("%s", errbuf.buf);

View File

@ -723,7 +723,7 @@ static struct stored_bitmap *lazy_bitmap_for_commit(struct bitmap_index *bitmap_
ALLOC_GROW(xor_items, xor_items_nr + 1, xor_items_alloc);
if (xor_items_nr + 1 >= bitmap_git->entry_count) {
error(_("corrupt bitmap lookup table: xor chain exceed entry count"));
error(_("corrupt bitmap lookup table: xor chain exceeds entry count"));
goto corrupt;
}
@ -830,10 +830,9 @@ struct ewah_bitmap *bitmap_for_commit(struct bitmap_index *bitmap_git,
if (!bitmap_git->table_lookup)
return NULL;
trace2_region_enter("pack-bitmap", "reading_lookup_table", the_repository);
/* this is a fairly hot codepath - no trace2_region please */
/* NEEDSWORK: cache misses aren't recorded */
bitmap = lazy_bitmap_for_commit(bitmap_git, commit);
trace2_region_leave("pack-bitmap", "reading_lookup_table", the_repository);
if (!bitmap)
return NULL;
return lookup_stored_bitmap(bitmap);

View File

@ -455,13 +455,6 @@ test_expect_success 'verify writing bitmap lookup table when enabled' '
grep "\"label\":\"writing_lookup_table\"" trace2
'
test_expect_success 'lookup table is actually used to traverse objects' '
git repack -adb &&
GIT_TRACE2_EVENT="$(pwd)/trace3" \
git rev-list --use-bitmap-index --count --all &&
grep "\"label\":\"reading_lookup_table\"" trace3
'
test_expect_success 'truncated bitmap fails gracefully (lookup table)' '
test_config pack.writebitmaphashcache false &&
git repack -adb &&