The current parsing scheme for upload-archive is to pack
arguments into a fixed-size buffer, separated by NULs, and
put a pointer to each argument in the buffer into a
fixed-size argv array.
This works fine, and the limits are high enough that nobody
reasonable is going to hit them, but it makes the code hard
to follow. Instead, let's just stuff the arguments into an
argv_array, which is much simpler. That lifts the "all
arguments must fit inside 4K together" limit.
We could also trivially lift the MAX_ARGS limitation (in
fact, we have to keep extra code to enforce it). But that
would mean a client could force us to allocate an arbitrary
amount of memory simply by sending us "argument" lines. By
limiting the MAX_ARGS, we limit an attacker to about 4
megabytes (64 times a maximum 64K packet buffer). That may
sound like a lot compared to the 4K limit, but it's not a
big deal compared to what git-archive will actually allocate
while working (e.g., to load blobs into memory). The
important thing is that it is bounded.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
According to the comment, enter_repo will modify its input.
However, this has not been the case since 1c64b48
(enter_repo: do not modify input, 2011-10-04). Drop the
now-useless copy.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Usually we do not pass an empty string to the function hash_name()
because we almost always ask for hash values for a path that is a
candidate to be added to the index. However, check-ignore (and most
likely check-attr, but I didn't check) apparently has a callchain
to ask the hash value for an empty path when it was given a "." from
the top-level directory to ask "Is the path . excluded by default?"
Make sure that hash_name() does not overrun the end of the given
pathname even when it is empty.
Remove a sweep-the-issue-under-the-rug conditional in check-ignore
that avoided to pass an empty string to the callchain while at it.
It is a valid question to ask for check-ignore if the top-level is
set to be ignored by default, even though the answer is most likely
no, if only because there is currently no way to specify such an
entry in the .gitignore file. But it is an unusual thing to ask and
it is not worth optimizing for it by special casing at the top level
of the call chain.
Signed-off-by: Adam Spiers <git@adamspiers.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, git will append two newlines to every message supplied via
the -m switch. The purpose of this is to allow -m to be supplied
multiple times and have each supplied string become a paragraph in the
resulting commit message.
Normally, this does not cause a problem since any trailing newlines will
be removed by the cleanup operation. If cleanup=verbatim for example,
then the trailing newlines will not be removed and will survive into the
resulting commit message.
Instead, let's ensure that the string supplied to -m is newline terminated,
but only append a second newline when appending additional messages.
Fixes the test in t7502.
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow the server side to redact the refs/ namespace it shows to the
client.
Will merge to 'master'.
* jc/hidden-refs:
upload/receive-pack: allow hiding ref hierarchies
upload-pack: simplify request validation
upload-pack: share more code
Also issue warnings on loose garbages instead of errors as a result of
using report_garbage() function in count_objects()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prepare_packed_git_one() is modified to allow count-objects to hook a
report function to so we don't need to duplicate the pack searching
logic in count-objects.c. When report_pack_garbage is NULL, the
overhead is insignificant.
The garbage is reported with warning() instead of error() in packed
garbage case because it's not an error to have garbage. Loose garbage
is still reported as errors and will be converted to warnings later.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of requiring the full 40-hex object names on the index
line, we can read submodule commit object names from the textual
diff when synthesizing a fake ancestore tree for "git am -3".
* jc/extended-fake-ancestor-for-gitlink:
apply: verify submodule commit object name better
Currently, verify_signed_buffer() returns the user facing output only.
Allow callers to request the status output also.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a preparation step for merging with append_signoff from
sequencer.c
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Brandon Casey <bcasey@nvidia.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach append_signoff how to detect a duplicate s-o-b in the commit footer.
This is in preparation to unify the append_signoff implementations in
log-tree.c and sequencer.c.
Fixes test in t3511.
Signed-off-by: Brandon Casey <bcasey@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pathspec is the most widely used term, and is the one defined in
gitglossary.txt. <filepattern> was used only in the synopsys for git-add
and git-commit, and in git-add.txt. Get rid of it.
This patch is obtained with by running:
perl -pi -e 's/filepattern/pathspec/' `git grep -l filepattern`
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make sure the similarity value shown in the "apply --summary"
output is sensible, even when the input had a bogus value.
* jk/apply-similaritly-parsing:
builtin/apply: tighten (dis)similarity index parsing
Rebasing the history of superproject with change in the submodule
has been broken since v1.7.12.
* jc/fake-ancestor-with-non-blobs:
apply: diagnose incomplete submodule object name better
apply: simplify build_fake_ancestor()
git-am: record full index line in the patch used while rebasing
Fix various error messages and conditions in "git branch", e.g. we
advertised "branch -d/-D" to remove one or more branches but actually
implemented removal of zero or more branches---request to remove no
branches was not rejected.
* nd/branch-error-cases:
branch: let branch filters imply --list
docs: clarify git-branch --list behavior
branch: mark more strings for translation
branch: give a more helpful message on redundant arguments
branch: reject -D/-d without branch name
Even though "git fetch" has full infrastructure to parse refspecs to
be fetched and match them against the list of refs to come up with
the final list of refs to be fetched, the list of refs that are
requested to be fetched were internally converted to a plain list of
strings at the transport layer and then passed to the underlying
fetch-pack driver.
Stop this conversion and instead pass around an array of refs.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A repository may have refs that are only used for its internal
bookkeeping purposes that should not be exposed to the others that
come over the network.
Teach upload-pack to omit some refs from its initial advertisement
by paying attention to the uploadpack.hiderefs multi-valued
configuration variable. Do the same to receive-pack via the
receive.hiderefs variable. As a convenient short-hand, allow using
transfer.hiderefs to set the value to both of these variables.
Any ref that is under the hierarchies listed on the value of these
variable is excluded from responses to requests made by "ls-remote",
"fetch", etc. (for upload-pack) and "push" (for receive-pack).
Because these hidden refs do not count as OUR_REF, an attempt to
fetch objects at the tip of them will be rejected, and because these
refs do not get advertised, "git push :" will not see local branches
that have the same name as them as "matching" ones to be sent.
An attempt to update/delete these hidden refs with an explicit
refspec, e.g. "git push origin :refs/hidden/22", is rejected. This
is not a new restriction. To the pusher, it would appear that there
is no such ref, so its push request will conclude with "Now that I
sent you all the data, it is time for you to update the refs. I saw
that the ref did not exist when I started pushing, and I want the
result to point at this commit". The receiving end will apply the
compare-and-swap rule to this request and rejects the push with
"Well, your update request conflicts with somebody else; I see there
is such a ref.", which is the right thing to do. Otherwise a push to
a hidden ref will always be "the last one wins", which is not a good
default.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rebasing the history of superproject with change in the submodule
was broken since v1.7.12.
* jc/fake-ancestor-with-non-blobs:
apply: diagnose incomplete submodule object name better
apply: simplify build_fake_ancestor()
git-am: record full index line in the patch used while rebasing
A textual patch also records the submodule commit object name in
full. Make the parsing more robust by reading from there and
verifying the (possibly abbreviated) name on the index line matches.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Turn some boolean fields into bitfields and use uint32_t for name
hash. This shrinks the size of the structure from 128 bytes to 120
bytes.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Clarify the ownership rule for commit->buffer field, which some
callers incorrectly accessed without making sure it is populated.
* jk/read-commit-buffer-data-after-free:
logmsg_reencode: lazily load missing commit buffers
logmsg_reencode: never return NULL
commit: drop useless xstrdup of commit message
Forbid "git add -u" and "git add -A" without pathspec run from a
subdirectory, to train people to type "." (or ":/") to make the
choice of default does not matter.
* mm/add-u-A-sans-pathspec:
add: warn when -u or -A is used without pathspec
Improve error and advice messages given locally when "git push"
refuses when it cannot compute fast-forwardness by separating these
cases from the normal "not a fast-forward; merge first and push
again" case.
* jc/push-reject-reasons:
push: finishing touches to explain REJECT_ALREADY_EXISTS better
push: introduce REJECT_FETCH_FIRST and REJECT_NEEDS_FORCE
push: further simplify the logic to assign rejection reason
push: further clean up fields of "struct ref"
Configuration parsing for tar.* configuration variables were
broken. Introduce a new config-keyname parser API to make the
callers much less error prone.
* jk/config-parsing-cleanup:
reflog: use parse_config_key in config callback
help: use parse_config_key for man config
submodule: simplify memory handling in config parsing
submodule: use parse_config_key when parsing config
userdiff: drop parse_driver function
convert some config callbacks to parse_config_key
archive-tar: use parse_config_key when parsing config
config: add helper function for parsing key names
Allow a configuration variable core.commentchar to customize the
character used to comment out the hint lines in the edited text from
the default '#'.
* jc/custom-comment-char:
Allow custom "comment char"
Attempt to "branch --edit-description" an existing branch, while
being on a detached HEAD, errored out.
* nd/edit-branch-desc-while-detached:
branch: no detached HEAD check when editing another branch's description
* jc/merge-blobs:
Makefile: Replace merge-file.h with merge-blobs.h in LIB_H
merge-tree: fix d/f conflicts
merge-tree: add comments to clarify what these functions are doing
merge-tree: lose unused "resolve_directories"
merge-tree: lose unused "flags" from merge_list
Which merge_file() function do you mean?
This was prompted by an incorrect warning issued by clang [1], and a
suggestion by Linus to restrict the range to check for values greater
than INT_MAX since these will give bogus output after casting to int.
In fact the (dis)similarity index is a percentage, so reject values
greater than 100.
[1] http://article.gmane.org/gmane.comp.version-control.git/213857
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Attempt to "branch --edit-description" an existing branch, while
being on a detached HEAD, errored out.
* nd/edit-branch-desc-while-detached:
branch: no detached HEAD check when editing another branch's description
Help "fetch only" repositories that do not trigger "gc --auto"
often enough.
* jk/gc-auto-after-fetch:
fetch-pack: avoid repeatedly re-scanning pack directory
fetch: run gc --auto after fetching
We stopped paying attention to $GIT_CONFIG environment that points
at a single configuration file from any command other than "git config"
quite a while ago, but "git clone" internally set, exported, and
then unexported the variable during its operation unnecessarily.
* jc/no-git-config-in-clone:
clone: do not export and unexport GIT_CONFIG
"git fetch --depth" was broken in at least three ways. The
resulting history was deeper than specified by one commit, it was
unclear how to wipe the shallowness of the repository with the
command, and documentation was misleading.
* nd/fetch-depth-is-broken:
fetch: elaborate --depth action
upload-pack: fix off-by-one depth calculation in shallow clone
fetch: add --unshallow for turning shallow repo into complete one
"git am -3" uses this function to build a tree that records how the
preimage the patch was created from would have looked like. An
abbreviated object name on the index line is ordinarily sufficient
for us to figure out the object name the preimage tree would have
contained, but a change to a submodule by definition shows an object
name of a submodule commit which our repository should not have, and
get_sha1_blob() is not an appropriate way to read it (or get_sha1()
for that matter).
Use get_sha1_hex() and complain if we do not find a full object name
there.
We could read from the payload part of the patch to learn the full
object name of the commit, but the primary user "git rebase" has
been fixed to give us a full object name, so this should suffice
for now.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The local variable sha1_ptr in the build_fake_ancestor() function
used to either point at the null_sha1[] (if the ancestor did not
have the path) or at sha1[] (if we read the object name into the
local array), but 7a98869 (apply: get rid of --index-info in favor
of --build-fake-ancestor, 2007-09-17) made the "missing in the
ancestor" case unnecessary, hence sha1_ptr, when used, always points
at the local array.
Get rid of the unneeded variable, and restructure the if/else
cascade a bit to make it easier to read. There should be no
behaviour change.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, a branch filter like `--contains`, `--merged`, or
`--no-merged` is ignored when we are not in listing mode.
For example:
git branch --contains=foo bar
will create the branch "bar" from the current HEAD, ignoring
the `--contains` argument entirely. This is not very
helpful. There are two reasonable behaviors for git here:
1. Flag an error; the arguments do not make sense.
2. Implicitly go into `--list` mode
This patch chooses the latter, as it is more convenient, and
there should not be any ambiguity with attempting to create
a branch; using `--contains` and not wanting to list is
nonsensical.
That leaves the case where an explicit modification option
like `-d` is given. We already catch the case where
`--list` is given alongside `-d` and flag an error. With
this patch, we will also catch the use of `--contains` and
other filter options alongside `-d`.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When giving arguments without "--" disambiguation, object names
that come earlier on the command line must not be interpretable as
pathspecs and pathspecs that come later on the command line must
not be interpretable as object names. Tweak the disambiguation
rule so that ":/" (no other string before or after) is always
interpreted as a pathspec, to avoid having to say "git cmd -- :/".
* nd/magic-pathspec-from-root:
grep: avoid accepting ambiguous revision
Update :/abc ambiguity check
A header file that has the definition of a static array was
included in two places, wasting the space.
* jc/help:
help: include <common-cmds.h> only in one file
Most Git commands that can be used with or without pathspec operate
tree-wide by default, the pathspec being used to restrict their
scope. A few exceptions are: 'git grep', 'git clean', 'git add -u'
and 'git add -A'. When run in a subdirectory without pathspec, they
operate only on paths in the current directory.
The inconsistency of 'git add -u' and 'git add -A' is particularly
problematic since other 'git add' subcommands (namely 'git add -p'
and 'git add -e') are tree-wide by default. It also means that "git
add -u && git commit" will record a state that is different from
what is recorded with "git commit -a".
Flipping the default now is unacceptable, so let's start training
users to type 'git add -u|-A :/' or 'git add -u|-A .' explicitly, to
prepare for the next steps:
* forbid 'git add -u|-A' without pathspec (like 'git add' without
option)
* much later, maybe, re-allow 'git add -u|-A' without pathspec, that
will add all tracked and modified files, or all files, tree-wide.
A nice side effect of this patch is that it makes the :/ magic
pathspec easier to discover for users.
When the command is called from the root of the tree, there is no
ambiguity and no need to change the behavior, hence no need to warn.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we look up a sha1 object for reading via parse_object() =>
read_sha1_file() => read_object() callpath, we first check
packfiles, and then loose objects. If we still haven't found it, we
re-scan the list of packfiles in `objects/pack`. This final step
ensures that we can co-exist with a simultaneous repack process
which creates a new pack and then prunes the old object.
This extra re-scan usually does not have a performance impact for
two reasons:
1. If an object is missing, then typically the re-scan will find a
new pack, then no more misses will occur. Or if it truly is
missing, then our next step is usually to die().
2. Re-scanning is cheap enough that we do not even notice.
However, these do not always hold. The assumption in (1) is that the
caller is expecting to find the object. This is usually the case,
but the call to `parse_object` in `everything_local` does not follow
this pattern. It is looking to see whether we have objects that the
remote side is advertising, not something we expect to
have. Therefore if we are fetching from a remote which has many refs
pointing to objects we do not have, we may end up re-scanning the
pack directory many times.
Even with this extra re-scanning, the impact is often not noticeable
due to (2); we just readdir() the packs directory and skip any packs
that are already loaded. However, if there are a large number of
packs, even enumerating the directory can be expensive, especially
if we do it repeatedly.
Having this many packs is a good sign the user should run `git gc`,
but it would still be nice to avoid having to scan the directory at
all.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We generally try to run "gc --auto" after any commands that
might introduce a large number of new objects. An obvious
place to do so is after running "fetch", which may introduce
new loose objects or packs (depending on the size of the
fetch).
While an active developer repository will probably
eventually trigger a "gc --auto" on another action (e.g.,
git-rebase), there are two good reasons why it is nicer to
do it at fetch time:
1. Read-only repositories which track an upstream (e.g., a
continuous integration server which fetches and builds,
but never makes new commits) will accrue loose objects
and small packs, but never coalesce them into a more
efficient larger pack.
2. Fetching is often already perceived to be slow to the
user, since they have to wait on the network. It's much
more pleasant to include a potentially slow auto-gc as
part of the already-long network fetch than in the
middle of productive work with git-rebase or similar.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Usually a commit that makes it to logmsg_reencode will have
been parsed, and the commit->buffer struct member will be
valid. However, some code paths will free commit buffers
after having used them (for example, the log traversal
machinery will do so to keep memory usage down).
Most of the time this is fine; log should only show a commit
once, and then exits. However, there are some code paths
where this does not work. At least two are known:
1. A commit may be shown as part of a regular ref, and
then it may be shown again as part of a submodule diff
(e.g., if a repo contains refs to both the superproject
and subproject).
2. A notes-cache commit may be shown during "log --all",
and then later used to access a textconv cache during a
diff.
Lazily loading in logmsg_reencode does not necessarily catch
all such cases, but it should catch most of them. Users of
the commit buffer tend to be either parsing for structure
(in which they will call parse_commit, and either we will
already have parsed, or we will load commit->buffer lazily
there), or outputting (either to the user, or fetching a
part of the commit message via format_commit_message). In
the latter case, we should always be using logmsg_reencode
anyway (and typically we do so via the pretty-print
machinery).
If there are any cases that this misses, we can fix them up
to use logmsg_reencode (or handle them on a case-by-case
basis if that is inappropriate).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The logmsg_reencode function will return the reencoded
commit buffer, or NULL if reencoding failed or no reencoding
was necessary. Since every caller then ends up checking for NULL
and just using the commit's original buffer, anyway, we can
be a bit more helpful and just return that buffer when we
would have returned NULL.
Since the resulting string may or may not need to be freed,
we introduce a logmsg_free, which checks whether the buffer
came from the commit object or not (callers either
implemented the same check already, or kept two separate
pointers, one to mark the buffer to be used, and one for the
to-be-freed string).
Pushing this logic into logmsg_* simplifies the callers, and
will let future patches lazily load the commit buffer in a
single place.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
