docs: move commit-graph format docs to man section 5
Continue the move of existing Documentation/technical/* protocol and file-format documentation into our main documentation space. By moving the documentation for the commit-graph format into man section 5 and the new "developerinterfaces" category. This change is split from subsequent commits due to the relatively large amount of ASCIIDOC formatting changes that are required. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:
Ævar Arnfjörð Bjarmason
committed by
Junio C Hamano
Junio C Hamano
parent
844739ba27
commit
8cbace93d2
@ -6,7 +6,7 @@ sections of the file. This allows structured access to a large file by
|
||||
scanning a small "table of contents" for the remaining data. This common
|
||||
format is used by the `commit-graph` and `multi-pack-index` files. See
|
||||
link:technical/pack-format.html[the `multi-pack-index` format] and
|
||||
link:technical/commit-graph-format.html[the `commit-graph` format] for
|
||||
the `commit-graph` format in linkgit:gitformat-commit-graph[5] for
|
||||
how they use the chunks to describe structured data.
|
||||
|
||||
A chunk-based file format begins with some header information custom to
|
||||
@ -108,7 +108,7 @@ for future formats:
|
||||
* *commit-graph:* see `write_commit_graph_file()` and `parse_commit_graph()`
|
||||
in `commit-graph.c` for how the chunk-format API is used to write and
|
||||
parse the commit-graph file format documented in
|
||||
link:technical/commit-graph-format.html[the commit-graph file format].
|
||||
the commit-graph file format in linkgit:gitformat-commit-graph[5].
|
||||
|
||||
* *multi-pack-index:* see `write_midx_internal()` and `load_multi_pack_index()`
|
||||
in `midx.c` for how the chunk-format API is used to write and
|
||||
|
||||
@ -1,166 +0,0 @@
|
||||
Git commit graph format
|
||||
=======================
|
||||
|
||||
The Git commit graph stores a list of commit OIDs and some associated
|
||||
metadata, including:
|
||||
|
||||
- The generation number of the commit.
|
||||
|
||||
- The root tree OID.
|
||||
|
||||
- The commit date.
|
||||
|
||||
- The parents of the commit, stored using positional references within
|
||||
the graph file.
|
||||
|
||||
- The Bloom filter of the commit carrying the paths that were changed between
|
||||
the commit and its first parent, if requested.
|
||||
|
||||
These positional references are stored as unsigned 32-bit integers
|
||||
corresponding to the array position within the list of commit OIDs. Due
|
||||
to some special constants we use to track parents, we can store at most
|
||||
(1 << 30) + (1 << 29) + (1 << 28) - 1 (around 1.8 billion) commits.
|
||||
|
||||
== Commit graph files have the following format:
|
||||
|
||||
In order to allow extensions that add extra data to the graph, we organize
|
||||
the body into "chunks" and provide a binary lookup table at the beginning
|
||||
of the body. The header includes certain values, such as number of chunks
|
||||
and hash type.
|
||||
|
||||
All multi-byte numbers are in network byte order.
|
||||
|
||||
HEADER:
|
||||
|
||||
4-byte signature:
|
||||
The signature is: {'C', 'G', 'P', 'H'}
|
||||
|
||||
1-byte version number:
|
||||
Currently, the only valid version is 1.
|
||||
|
||||
1-byte Hash Version
|
||||
We infer the hash length (H) from this value:
|
||||
1 => SHA-1
|
||||
2 => SHA-256
|
||||
If the hash type does not match the repository's hash algorithm, the
|
||||
commit-graph file should be ignored with a warning presented to the
|
||||
user.
|
||||
|
||||
1-byte number (C) of "chunks"
|
||||
|
||||
1-byte number (B) of base commit-graphs
|
||||
We infer the length (H*B) of the Base Graphs chunk
|
||||
from this value.
|
||||
|
||||
CHUNK LOOKUP:
|
||||
|
||||
(C + 1) * 12 bytes listing the table of contents for the chunks:
|
||||
First 4 bytes describe the chunk id. Value 0 is a terminating label.
|
||||
Other 8 bytes provide the byte-offset in current file for chunk to
|
||||
start. (Chunks are ordered contiguously in the file, so you can infer
|
||||
the length using the next chunk position if necessary.) Each chunk
|
||||
ID appears at most once.
|
||||
|
||||
The CHUNK LOOKUP matches the table of contents from
|
||||
link:technical/chunk-format.html[the chunk-based file format].
|
||||
|
||||
The remaining data in the body is described one chunk at a time, and
|
||||
these chunks may be given in any order. Chunks are required unless
|
||||
otherwise specified.
|
||||
|
||||
CHUNK DATA:
|
||||
|
||||
OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes)
|
||||
The ith entry, F[i], stores the number of OIDs with first
|
||||
byte at most i. Thus F[255] stores the total
|
||||
number of commits (N).
|
||||
|
||||
OID Lookup (ID: {'O', 'I', 'D', 'L'}) (N * H bytes)
|
||||
The OIDs for all commits in the graph, sorted in ascending order.
|
||||
|
||||
Commit Data (ID: {'C', 'D', 'A', 'T' }) (N * (H + 16) bytes)
|
||||
* The first H bytes are for the OID of the root tree.
|
||||
* The next 8 bytes are for the positions of the first two parents
|
||||
of the ith commit. Stores value 0x70000000 if no parent in that
|
||||
position. If there are more than two parents, the second value
|
||||
has its most-significant bit on and the other bits store an array
|
||||
position into the Extra Edge List chunk.
|
||||
* The next 8 bytes store the topological level (generation number v1)
|
||||
of the commit and
|
||||
the commit time in seconds since EPOCH. The generation number
|
||||
uses the higher 30 bits of the first 4 bytes, while the commit
|
||||
time uses the 32 bits of the second 4 bytes, along with the lowest
|
||||
2 bits of the lowest byte, storing the 33rd and 34th bit of the
|
||||
commit time.
|
||||
|
||||
Generation Data (ID: {'G', 'D', 'A', '2' }) (N * 4 bytes) [Optional]
|
||||
* This list of 4-byte values store corrected commit date offsets for the
|
||||
commits, arranged in the same order as commit data chunk.
|
||||
* If the corrected commit date offset cannot be stored within 31 bits,
|
||||
the value has its most-significant bit on and the other bits store
|
||||
the position of corrected commit date into the Generation Data Overflow
|
||||
chunk.
|
||||
* Generation Data chunk is present only when commit-graph file is written
|
||||
by compatible versions of Git and in case of split commit-graph chains,
|
||||
the topmost layer also has Generation Data chunk.
|
||||
|
||||
Generation Data Overflow (ID: {'G', 'D', 'O', '2' }) [Optional]
|
||||
* This list of 8-byte values stores the corrected commit date offsets
|
||||
for commits with corrected commit date offsets that cannot be
|
||||
stored within 31 bits.
|
||||
* Generation Data Overflow chunk is present only when Generation Data
|
||||
chunk is present and atleast one corrected commit date offset cannot
|
||||
be stored within 31 bits.
|
||||
|
||||
Extra Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional]
|
||||
This list of 4-byte values store the second through nth parents for
|
||||
all octopus merges. The second parent value in the commit data stores
|
||||
an array position within this list along with the most-significant bit
|
||||
on. Starting at that array position, iterate through this list of commit
|
||||
positions for the parents until reaching a value with the most-significant
|
||||
bit on. The other bits correspond to the position of the last parent.
|
||||
|
||||
Bloom Filter Index (ID: {'B', 'I', 'D', 'X'}) (N * 4 bytes) [Optional]
|
||||
* The ith entry, BIDX[i], stores the number of bytes in all Bloom filters
|
||||
from commit 0 to commit i (inclusive) in lexicographic order. The Bloom
|
||||
filter for the i-th commit spans from BIDX[i-1] to BIDX[i] (plus header
|
||||
length), where BIDX[-1] is 0.
|
||||
* The BIDX chunk is ignored if the BDAT chunk is not present.
|
||||
|
||||
Bloom Filter Data (ID: {'B', 'D', 'A', 'T'}) [Optional]
|
||||
* It starts with header consisting of three unsigned 32-bit integers:
|
||||
- Version of the hash algorithm being used. We currently only support
|
||||
value 1 which corresponds to the 32-bit version of the murmur3 hash
|
||||
implemented exactly as described in
|
||||
https://en.wikipedia.org/wiki/MurmurHash#Algorithm and the double
|
||||
hashing technique using seed values 0x293ae76f and 0x7e646e2 as
|
||||
described in https://doi.org/10.1007/978-3-540-30494-4_26 "Bloom Filters
|
||||
in Probabilistic Verification"
|
||||
- The number of times a path is hashed and hence the number of bit positions
|
||||
that cumulatively determine whether a file is present in the commit.
|
||||
- The minimum number of bits 'b' per entry in the Bloom filter. If the filter
|
||||
contains 'n' entries, then the filter size is the minimum number of 64-bit
|
||||
words that contain n*b bits.
|
||||
* The rest of the chunk is the concatenation of all the computed Bloom
|
||||
filters for the commits in lexicographic order.
|
||||
* Note: Commits with no changes or more than 512 changes have Bloom filters
|
||||
of length one, with either all bits set to zero or one respectively.
|
||||
* The BDAT chunk is present if and only if BIDX is present.
|
||||
|
||||
Base Graphs List (ID: {'B', 'A', 'S', 'E'}) [Optional]
|
||||
This list of H-byte hashes describe a set of B commit-graph files that
|
||||
form a commit-graph chain. The graph position for the ith commit in this
|
||||
file's OID Lookup chunk is equal to i plus the number of commits in all
|
||||
base graphs. If B is non-zero, this chunk must exist.
|
||||
|
||||
TRAILER:
|
||||
|
||||
H-byte HASH-checksum of all of the above.
|
||||
|
||||
== Historical Notes:
|
||||
|
||||
The Generation Data (GDA2) and Generation Data Overflow (GDO2) chunks have
|
||||
the number '2' in their chunk IDs because a previous version of Git wrote
|
||||
possibly erroneous data in these chunks with the IDs "GDAT" and "GDOV". By
|
||||
changing the IDs, newer versions of Git will silently ignore those older
|
||||
chunks and write the new information without trusting the incorrect data.
|
||||
Reference in New Issue
Block a user