mirrors/git
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'bw/protocol-v2' into HEAD

Browse Source 
* bw/protocol-v2: (35 commits)
  remote-curl: don't request v2 when pushing
  remote-curl: implement stateless-connect command
  http: eliminate "# service" line when using protocol v2
  http: don't always add Git-Protocol header
  http: allow providing extra headers for http requests
  remote-curl: store the protocol version the server responded with
  remote-curl: create copy of the service name
  pkt-line: add packet_buf_write_len function
  transport-helper: introduce stateless-connect
  transport-helper: refactor process_connect_service
  transport-helper: remove name parameter
  connect: don't request v2 when pushing
  connect: refactor git_connect to only get the protocol version once
  fetch-pack: support shallow requests
  fetch-pack: perform a fetch using v2
  upload-pack: introduce fetch server command
  push: pass ref prefixes when pushing
  fetch: pass ref prefixes when fetching
  ls-remote: pass ref prefixes when requesting a remote's refs
  transport: convert transport_get_remote_refs to take a list of ref prefixes
  ...
This commit is contained in:
Junio C Hamano
2018-04-24 11:24:22 +09:00
parent fe0a9eaf31 a4d78ce26b
commit bbc39d4020
44 changed files with 3326 additions and 376 deletions
Download Patch File Download Diff File Expand all files Collapse all files

395
Documentation/technical/protocol-v2.txt Normal file
View File

@ -0,0 +1,395 @@
Git Wire Protocol, Version 2
==============================
This document presents a specification for a version 2 of Git's wire
protocol. Protocol v2 will improve upon v1 in the following ways:
* Instead of multiple service names, multiple commands will be
supported by a single service
* Easily extendable as capabilities are moved into their own section
of the protocol, no longer being hidden behind a NUL byte and
limited by the size of a pkt-line
* Separate out other information hidden behind NUL bytes (e.g. agent
string as a capability and symrefs can be requested using 'ls-refs')
* Reference advertisement will be omitted unless explicitly requested
* ls-refs command to explicitly request some refs
* Designed with http and stateless-rpc in mind. With clear flush
semantics the http remote helper can simply act as a proxy
In protocol v2 communication is command oriented. When first contacting a
server a list of capabilities will advertised. Some of these capabilities
will be commands which a client can request be executed. Once a command
has completed, a client can reuse the connection and request that other
commands be executed.
Packet-Line Framing
---------------------
All communication is done using packet-line framing, just as in v1. See
`Documentation/technical/pack-protocol.txt` and
`Documentation/technical/protocol-common.txt` for more information.
In protocol v2 these special packets will have the following semantics:
* '0000' Flush Packet (flush-pkt) - indicates the end of a message
* '0001' Delimiter Packet (delim-pkt) - separates sections of a message
Initial Client Request
------------------------
In general a client can request to speak protocol v2 by sending
`version=2` through the respective side-channel for the transport being
used which inevitably sets `GIT_PROTOCOL`. More information can be
found in `pack-protocol.txt` and `http-protocol.txt`. In all cases the
response from the server is the capability advertisement.
Git Transport
~~~~~~~~~~~~~~~
When using the git:// transport, you can request to use protocol v2 by
sending "version=2" as an extra parameter:
003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0
SSH and File Transport
~~~~~~~~~~~~~~~~~~~~~~~~
When using either the ssh:// or file:// transport, the GIT_PROTOCOL
environment variable must be set explicitly to include "version=2".
HTTP Transport
~~~~~~~~~~~~~~~~
When using the http:// or https:// transport a client makes a "smart"
info/refs request as described in `http-protocol.txt` and requests that
v2 be used by supplying "version=2" in the `Git-Protocol` header.
C: Git-Protocol: version=2
C:
C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0
A v2 server would reply:
S: 200 OK
S: <Some headers>
S: ...
S:
S: 000eversion 2\n
S: <capability-advertisement>
Subsequent requests are then made directly to the service
`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack).
Capability Advertisement
--------------------------
A server which decides to communicate (based on a request from a client)
using protocol version 2, notifies the client by sending a version string
in its initial response followed by an advertisement of its capabilities.
Each capability is a key with an optional value. Clients must ignore all
unknown keys. Semantics of unknown values are left to the definition of
each key. Some capabilities will describe commands which can be requested
to be executed by the client.
capability-advertisement = protocol-version
capability-list
flush-pkt
protocol-version = PKT-LINE("version 2" LF)
capability-list = *capability
capability = PKT-LINE(key[=value] LF)
key = 1*(ALPHA | DIGIT | "-_")
value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;")
Command Request
-----------------
After receiving the capability advertisement, a client can then issue a
request to select the command it wants with any particular capabilities
or arguments. There is then an optional section where the client can
provide any command specific parameters or queries. Only a single
command can be requested at a time.
request = empty-request | command-request
empty-request = flush-pkt
command-request = command
capability-list
[command-args]
flush-pkt
command = PKT-LINE("command=" key LF)
command-args = delim-pkt
*command-specific-arg
command-specific-args are packet line framed arguments defined by
each individual command.
The server will then check to ensure that the client's request is
comprised of a valid command as well as valid capabilities which were
advertised. If the request is valid the server will then execute the
command. A server MUST wait till it has received the client's entire
request before issuing a response. The format of the response is
determined by the command being executed, but in all cases a flush-pkt
indicates the end of the response.
When a command has finished, and the client has received the entire
response from the server, a client can either request that another
command be executed or can terminate the connection. A client may
optionally send an empty request consisting of just a flush-pkt to
indicate that no more requests will be made.
Capabilities
--------------
There are two different types of capabilities: normal capabilities,
which can be used to to convey information or alter the behavior of a
request, and commands, which are the core actions that a client wants to
perform (fetch, push, etc).
Protocol version 2 is stateless by default. This means that all commands
must only last a single round and be stateless from the perspective of the
server side, unless the client has requested a capability indicating that
state should be maintained by the server. Clients MUST NOT require state
management on the server side in order to function correctly. This
permits simple round-robin load-balancing on the server side, without
needing to worry about state management.
agent
~~~~~~~
The server can advertise the `agent` capability with a value `X` (in the
form `agent=X`) to notify the client that the server is running version
`X`. The client may optionally send its own agent string by including
the `agent` capability with a value `Y` (in the form `agent=Y`) in its
request to the server (but it MUST NOT do so if the server did not
advertise the agent capability). The `X` and `Y` strings may contain any
printable ASCII characters except space (i.e., the byte range 32 < x <
127), and are typically of the form "package/version" (e.g.,
"git/1.8.3.1"). The agent strings are purely informative for statistics
and debugging purposes, and MUST NOT be used to programmatically assume
the presence or absence of particular features.
ls-refs
~~~~~~~~~
`ls-refs` is the command used to request a reference advertisement in v2.
Unlike the current reference advertisement, ls-refs takes in arguments
which can be used to limit the refs sent from the server.
Additional features not supported in the base command will be advertised
as the value of the command in the capability advertisement in the form
of a space separated list of features: "<command>=<feature 1> <feature 2>"
ls-refs takes in the following arguments:
symrefs
In addition to the object pointed by it, show the underlying ref
pointed by it when showing a symbolic ref.
peel
Show peeled tags.
ref-prefix <prefix>
When specified, only references having a prefix matching one of
the provided prefixes are displayed.
The output of ls-refs is as follows:
output = *ref
flush-pkt
ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF)
ref-attribute = (symref | peeled)
symref = "symref-target:" symref-target
peeled = "peeled:" obj-id
fetch
~~~~~~~
`fetch` is the command used to fetch a packfile in v2. It can be looked
at as a modified version of the v1 fetch where the ref-advertisement is
stripped out (since the `ls-refs` command fills that role) and the
message format is tweaked to eliminate redundancies and permit easy
addition of future extensions.
Additional features not supported in the base command will be advertised
as the value of the command in the capability advertisement in the form
of a space separated list of features: "<command>=<feature 1> <feature 2>"
A `fetch` request can take the following arguments:
want <oid>
Indicates to the server an object which the client wants to
retrieve. Wants can be anything and are not limited to
advertised objects.
have <oid>
Indicates to the server an object which the client has locally.
This allows the server to make a packfile which only contains
the objects that the client needs. Multiple 'have' lines can be
supplied.
done
Indicates to the server that negotiation should terminate (or
not even begin if performing a clone) and that the server should
use the information supplied in the request to construct the
packfile.
thin-pack
Request that a thin pack be sent, which is a pack with deltas
which reference base objects not contained within the pack (but
are known to exist at the receiving end). This can reduce the
network traffic significantly, but it requires the receiving end
to know how to "thicken" these packs by adding the missing bases
to the pack.
no-progress
Request that progress information that would normally be sent on
side-band channel 2, during the packfile transfer, should not be
sent. However, the side-band channel 3 is still used for error
responses.
include-tag
Request that annotated tags should be sent if the objects they
point to are being sent.
ofs-delta
Indicate that the client understands PACKv2 with delta referring
to its base by position in pack rather than by an oid. That is,
they can read OBJ_OFS_DELTA (ake type 6) in a packfile.
If the 'shallow' feature is advertised the following arguments can be
included in the clients request as well as the potential addition of the
'shallow-info' section in the server's response as explained below.
shallow <oid>
A client must notify the server of all commits for which it only
has shallow copies (meaning that it doesn't have the parents of
a commit) by supplying a 'shallow <oid>' line for each such
object so that the server is aware of the limitations of the
client's history. This is so that the server is aware that the
client may not have all objects reachable from such commits.
deepen <depth>
Requests that the fetch/clone should be shallow having a commit
depth of <depth> relative to the remote side.
deepen-relative
Requests that the semantics of the "deepen" command be changed
to indicate that the depth requested is relative to the client's
current shallow boundary, instead of relative to the requested
commits.
deepen-since <timestamp>
Requests that the shallow clone/fetch should be cut at a
specific time, instead of depth. Internally it's equivalent to
doing "git rev-list --max-age=<timestamp>". Cannot be used with
"deepen".
deepen-not <rev>
Requests that the shallow clone/fetch should be cut at a
specific revision specified by '<rev>', instead of a depth.
Internally it's equivalent of doing "git rev-list --not <rev>".
Cannot be used with "deepen", but can be used with
"deepen-since".
The response of `fetch` is broken into a number of sections separated by
delimiter packets (0001), with each section beginning with its section
header.
output = *section
section = (acknowledgments | shallow-info | packfile)
(flush-pkt | delim-pkt)
acknowledgments = PKT-LINE("acknowledgments" LF)
(nak | *ack)
(ready)
ready = PKT-LINE("ready" LF)
nak = PKT-LINE("NAK" LF)
ack = PKT-LINE("ACK" SP obj-id LF)
shallow-info = PKT-LINE("shallow-info" LF)
*PKT-LINE((shallow | unshallow) LF)
shallow = "shallow" SP obj-id
unshallow = "unshallow" SP obj-id
packfile = PKT-LINE("packfile" LF)
*PKT-LINE(%x01-03 *%x00-ff)
acknowledgments section
* If the client determines that it is finished with negotiations
by sending a "done" line, the acknowledgments sections MUST be
omitted from the server's response.
* Always begins with the section header "acknowledgments"
* The server will respond with "NAK" if none of the object ids sent
as have lines were common.
* The server will respond with "ACK obj-id" for all of the
object ids sent as have lines which are common.
* A response cannot have both "ACK" lines as well as a "NAK"
line.
* The server will respond with a "ready" line indicating that
the server has found an acceptable common base and is ready to
make and send a packfile (which will be found in the packfile
section of the same response)
* If the server has found a suitable cut point and has decided
to send a "ready" line, then the server can decide to (as an
optimization) omit any "ACK" lines it would have sent during
its response. This is because the server will have already
determined the objects it plans to send to the client and no
further negotiation is needed.
shallow-info section
* If the client has requested a shallow fetch/clone, a shallow
client requests a fetch or the server is shallow then the
server's response may include a shallow-info section. The
shallow-info section will be included if (due to one of the
above conditions) the server needs to inform the client of any
shallow boundaries or adjustments to the clients already
existing shallow boundaries.
* Always begins with the section header "shallow-info"
* If a positive depth is requested, the server will compute the
set of commits which are no deeper than the desired depth.
* The server sends a "shallow obj-id" line for each commit whose
parents will not be sent in the following packfile.
* The server sends an "unshallow obj-id" line for each commit
which the client has indicated is shallow, but is no longer
shallow as a result of the fetch (due to its parents being
sent in the following packfile).
* The server MUST NOT send any "unshallow" lines for anything
which the client has not indicated was shallow as a part of
its request.
* This section is only included if a packfile section is also
included in the response.
packfile section
* This section is only included if the client has sent 'want'
lines in its request and either requested that no more
negotiation be done by sending 'done' or if the server has
decided it has found a sufficient cut point to produce a
packfile.
* Always begins with the section header "packfile"
* The transmission of the packfile begins immediately after the
section header
* The data transfer of the packfile is always multiplexed, using
the same semantics of the 'side-band-64k' capability from
protocol version 1. This means that each packet, during the
packfile data stream, is made up of a leading 4-byte pkt-line
length (typical of the pkt-line format), followed by a 1-byte
stream code, followed by the actual data.
The stream code can be one of:
1 - pack data
2 - progress messages
3 - fatal error message just before stream aborts