While the only permitted drive letters for physical drives on Windows
are letters of the US-English alphabet, this restriction does not apply
to virtual drives assigned via `subst <letter>: <path>`.
To prevent targeted attacks against systems where "funny" drive letters
such as `1` or `!` are assigned, let's handle them as regular drive
letters on Windows.
This fixes CVE-2019-1351.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, filenames cannot have trailing spaces or periods, when
opening such paths, they are stripped automatically. Read: you can open
the file `README` via the file name `README . . .`. This ambiguity can
be used in combination with other security bugs to cause e.g. remote
code execution during recursive clones. This patch series fixes that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch fixes a vulnerability in the Windows-specific code where a
submodule names ending in a backslash were quoted incorrectly, and that
bug could be abused to insert command-line parameters e.g. to `ssh` in a
recursive clone.
Note: this bug is Windows-only, as we have to construct a command line
for the process-to-spawn, unlike Linux/macOS, where `execv()` accepts an
already-split command line.
While at it, other quoting issues are fixed as well.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Recursive clones are currently affected by a vulnerability that is
caused by too-lax validation of submodule names.
This topic branch fixes that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch series makes it safe to use Git on Windows drives, even if
running on a mounted network share or within the Windows Subsystem for
Linux (WSL).
This topic branch addresses CVE-2019-1353.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Over a decade ago, in 25fe217b86 (Windows: Treat Windows style path
names., 2008-03-05), Git was taught to handle absolute Windows paths,
i.e. paths that start with a drive letter and a colon.
Unbeknownst to us, while drive letters of physical drives are limited to
letters of the English alphabet, there is a way to assign virtual drive
letters to arbitrary directories, via the `subst` command, which is
_not_ limited to English letters.
It is therefore possible to have absolute Windows paths of the form
`1:\what\the\hex.txt`. Even "better": pretty much arbitrary Unicode
letters can also be used, e.g. `ä:\tschibät.sch`.
While it can be sensibly argued that users who set up such funny drive
letters really seek adverse consequences, the Windows Operating System
is known to be a platform where many users are at the mercy of
administrators who have their very own idea of what constitutes a
reasonable setup.
Therefore, let's just make sure that such funny paths are still
considered absolute paths by Git, on Windows.
In addition to Unicode characters, pretty much any character is a valid
drive letter, as far as `subst` is concerned, even `:` and `"` or even a
space character. While it is probably the opposite of smart to use them,
let's safeguard `is_dos_drive_prefix()` against all of them.
Note: `[::1]:repo` is a valid URL, but not a valid path on Windows.
As `[` is now considered a valid drive letter, we need to be very
careful to avoid misinterpreting such a string as valid local path in
`url_is_local_not_ssh()`. To do that, we use the just-introduced
function `is_valid_path()` (which will label the string as invalid file
name because of the colon characters).
This fixes CVE-2019-1351.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch series plugs an attack vector we had overlooked in our
December 2014 work on `core.protectNTFS`.
Essentially, the path `.git::$INDEX_ALLOCATION/config` is interpreted as
`.git/config` when NTFS Alternate Data Streams are available (which they
are on Windows, and at least on network shares that are SMB-mounted on
macOS).
Needless to say: we don't want that.
In fact, we want to stay on the very safe side and not even special-case
the `$INDEX_ALLOCATION` stream type: let's just prevent Git from
touching _any_ explicitly specified Alternate Data Stream of `.git`.
In essence, we'll prevent Git from tracking, or writing to, any path
with a segment of the form `.git:<anything>`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When creating a directory on Windows whose path ends in a space or a
period (or chains thereof), the Win32 API "helpfully" trims those. For
example, `mkdir("abc ");` will return success, but actually create a
directory called `abc` instead.
This stems back to the DOS days, when all file names had exactly 8
characters plus exactly 3 characters for the file extension, and the
only way to have shorter names was by padding with spaces.
Sadly, this "helpful" behavior is a bit inconsistent: after a successful
`mkdir("abc ");`, a `mkdir("abc /def")` will actually _fail_ (because
the directory `abc ` does not actually exist).
Even if it would work, we now have a serious problem because a Git
repository could contain directories `abc` and `abc `, and on Windows,
they would be "merged" unintentionally.
As these paths are illegal on Windows, anyway, let's disallow any
accesses to such paths on that Operating System.
For practical reasons, this behavior is still guarded by the
config setting `core.protectNTFS`: it is possible (and at least two
regression tests make use of it) to create commits without involving the
worktree. In such a scenario, it is of course possible -- even on
Windows -- to create such file names.
Among other consequences, this patch disallows submodules' paths to end
in spaces on Windows (which would formerly have confused Git enough to
try to write into incorrect paths, anyway).
While this patch does not fix a vulnerability on its own, it prevents an
attack vector that was exploited in demonstrations of a number of
recently-fixed security bugs.
The regression test added to `t/t7417-submodule-path-url.sh` reflects
that attack vector.
Note that we have to adjust the test case "prevent git~1 squatting on
Windows" in `t/t7415-submodule-names.sh` because of a very subtle issue.
It tries to clone two submodules whose names differ only in a trailing
period character, and as a consequence their git directories differ in
the same way. Previously, when Git tried to clone the second submodule,
it thought that the git directory already existed (because on Windows,
when you create a directory with the name `b.` it actually creates `b`),
but with this patch, the first submodule's clone will fail because of
the illegal name of the git directory. Therefore, when cloning the
second submodule, Git will take a different code path: a fresh clone
(without an existing git directory). Both code paths fail to clone the
second submodule, both because the the corresponding worktree directory
exists and is not empty, but the error messages are worded differently.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is unfortunate that we need to quote arguments differently on
Windows, depending whether we build a command-line for MSYS2's `sh` or
for other Windows executables.
We already have a test helper to verify the latter, with this patch we
can also verify the former.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Certain characters are not admissible in file names on Windows, even if
Cygwin/MSYS2 (and therefore, Git for Windows' Bash) pretend that they
are, e.g. `:`, `<`, `>`, etc
Let's disallow those characters explicitly in Windows builds of Git.
Note: just like trailing spaces or periods, it _is_ possible on Windows
to create commits adding files with such illegal characters, as long as
the operation leaves the worktree untouched. To allow for that, we
continue to guard `is_valid_win32_path()` behind the config setting
`core.protectNTFS`, so that users _can_ continue to do that, as long as
they turn the protections off via that config setting.
Among other problems, this prevents Git from trying to write to an "NTFS
Alternate Data Stream" (which refers to metadata stored alongside a
file, under a special name: "<filename>:<stream-name>"). This fix
therefore also prevents an attack vector that was exploited in
demonstrations of a number of recently-fixed security bugs.
Further reading on illegal characters in Win32 filenames:
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
A `git clone` will end with exit code 0 when `merged_entry()` returns a
positive value during a call of `unpack_trees()` to `traverse_trees()`.
The reason is that `unpack_trees()` will interpret a positive value not
to be an error.
The problem is, however, that `add_index_entry()` (which is called by
`merged_entry()` can report an error, and we really should fail the
entire clone in such a case.
Let's fix this problem, in preparation for a Windows-specific patch
disallowing `mkdir()` with directory names that contain a trailing space
(which is illegal on NTFS): we want `git clone` to abort when a path
cannot be checked out due to that condition.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the, say, 93rd trial run fails, it is a good idea to have a way to
skip the first 92 trials and dig directly into the 93rd in a debugger.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, file names cannot contain asterisks nor newline characters.
In an upcoming commit, we will make this limitation explicit,
disallowing even the creation of commits that introduce such file names.
However, in the test scripts touched by this patch, we _know_ that those
paths won't be checked out, so we _want_ to allow such file names.
Happily, the stringent path validation will be guarded via the
`core.protectNTFS` flag, so all we need to do is to force that flag off
temporarily.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the stress test reported a problem with quoting certain arguments,
it is helpful to have a facility to play with those arguments in order
to find out whether variations of those arguments are affected, too.
Let's allow `test-run-command quote-stress-test -- <args>` to be used
for that purpose.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, we have to do all the command-line argument quoting
ourselves. Worse: we have to have two versions of said quoting, one for
MSYS2 programs (which have their own dequoting rules) and the rest.
We care mostly about the rest, and to make sure that that works, let's
have a stress test that comes up with all kinds of awkward arguments,
verifying that a spawned sub-process receives those unharmed.
Signed-off-by: Garima Singh <garima.singh@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Currently it is technically possible to let a submodule's git
directory point right into the git dir of a sibling submodule.
Example: the git directories of two submodules with the names `hippo`
and `hippo/hooks` would be `.git/modules/hippo/` and
`.git/modules/hippo/hooks/`, respectively, but the latter is already
intended to house the former's hooks.
In most cases, this is just confusing, but there is also a (quite
contrived) attack vector where Git can be fooled into mistaking remote
content for file contents it wrote itself during a recursive clone.
Let's plug this bug.
To do so, we introduce the new function `validate_submodule_git_dir()`
which simply verifies that no git dir exists for any leading directories
of the submodule name (if there are any).
Note: this patch specifically continues to allow sibling modules names
of the form `core/lib`, `core/doc`, etc, as long as `core` is not a
submodule name.
This fixes CVE-2019-1387.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Back in the DOS days, in the FAT file system, file names always
consisted of a base name of length 8 plus a file extension of length 3.
Shorter file names were simply padded with spaces to the full 8.3
format.
Later, the FAT file system was taught to support _also_ longer names,
with an 8.3 "short name" as primary file name. While at it, the same
facility allowed formerly illegal file names, such as `.git` (empty base
names were not allowed), which would have the "short name" `git~1`
associated with it.
For backwards-compatibility, NTFS supports alternative 8.3 short
filenames, too, even if starting with Windows Vista, they are only
generated on the system drive by default.
We addressed the problem that the `.git/` directory can _also_ be
accessed via `git~1/` (when short names are enabled) in 2b4c6efc82
(read-cache: optionally disallow NTFS .git variants, 2014-12-16), i.e.
since Git v1.9.5, by introducing the config setting `core.protectNTFS`
and enabling it by default on Windows.
In the meantime, Windows 10 introduced the "Windows Subsystem for Linux"
(short: WSL), i.e. a way to run Linux applications/distributions in a
thinly-isolated subsystem on Windows (giving rise to many a "2016 is the
Year of Linux on the Desktop" jokes). WSL is getting increasingly
popular, also due to the painless way Linux application can operate
directly ("natively") on files on Windows' file system: the Windows
drives are mounted automatically (e.g. `C:` as `/mnt/c/`).
Taken together, this means that we now have to enable the safe-guards of
Git v1.9.5 also in WSL: it is possible to access a `.git` directory
inside `/mnt/c/` via the 8.3 name `git~1` (unless short name generation
was disabled manually). Since regular Linux distributions run in WSL,
this means we have to enable `core.protectNTFS` at least on Linux, too.
To enable Services for Macintosh in Windows NT to store so-called
resource forks, NTFS introduced "Alternate Data Streams". Essentially,
these constitute additional metadata that are connected to (and copied
with) their associated files, and they are accessed via pseudo file
names of the form `filename:<stream-name>:<stream-type>`.
In a recent patch, we extended `core.protectNTFS` to also protect
against accesses via NTFS Alternate Data Streams, e.g. to prevent
contents of the `.git/` directory to be "tracked" via yet another
alternative file name.
While it is not possible (at least by default) to access files via NTFS
Alternate Data Streams from within WSL, the defaults on macOS when
mounting network shares via SMB _do_ allow accessing files and
directories in that way. Therefore, we need to enable `core.protectNTFS`
on macOS by default, too, and really, on any Operating System that can
mount network shares via SMB/CIFS.
A couple of approaches were considered for fixing this:
1. We could perform a dynamic NTFS check similar to the `core.symlinks`
check in `init`/`clone`: instead of trying to create a symbolic link
in the `.git/` directory, we could create a test file and try to
access `.git/config` via 8.3 name and/or Alternate Data Stream.
2. We could simply "flip the switch" on `core.protectNTFS`, to make it
"on by default".
The obvious downside of 1. is that it won't protect worktrees that were
clone with a vulnerable Git version already. We considered patching code
paths that check out files to check whether we're running on an NTFS
system dynamically and persist the result in the repository-local config
setting `core.protectNTFS`, but in the end decided that this solution
would be too fragile, and too involved.
The obvious downside of 2. is that everybody will have to "suffer" the
performance penalty incurred from calling `is_ntfs_dotgit()` on every
path, even in setups where.
After the recent work to accelerate `is_ntfs_dotgit()` in most cases,
it looks as if the time spent on validating ten million random
file names increases only negligibly (less than 20ms, well within the
standard deviation of ~50ms). Therefore the benefits outweigh the cost.
Another downside of this is that paths that might have been acceptable
previously now will be forbidden. Realistically, though, this is an
improvement because public Git hosters already would reject any `git
push` that contains such file names.
Note: There might be a similar problem mounting HFS+ on Linux. However,
this scenario has been considered unlikely and in light of the cost (in
the aforementioned benchmark, `core.protectHFS = true` increased the
time from ~440ms to ~610ms), it was decided _not_ to touch the default
of `core.protectHFS`.
This change addresses CVE-2019-1353.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Helped-by: Garima Singh <garima.singh@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We just safe-guarded `.git` against NTFS Alternate Data Stream-related
attack vectors, and now it is time to do the same for `.gitmodules`.
Note: In the added regression test, we refrain from verifying all kinds
of variations between short names and NTFS Alternate Data Streams: as
the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it
is enough to test one in order to know that all of them are guarded
against.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We need to be careful to follow proper quoting rules. For example, if an
argument contains spaces, we have to quote them. Double-quotes need to
be escaped. Backslashes need to be escaped, but only if they are
followed by a double-quote character.
We need to be _extra_ careful to consider the case where an argument
ends in a backslash _and_ needs to be quoted: in this case, we append a
double-quote character, i.e. the backslash now has to be escaped!
The current code, however, fails to recognize that, and therefore can
turn an argument that ends in a single backslash into a quoted argument
that now ends in an escaped double-quote character. This allows
subsequent command-line parameters to be split and part of them being
mistaken for command-line options, e.g. through a maliciously-crafted
submodule URL during a recursive clone.
Technically, we would not need to quote _all_ arguments which end in a
backslash _unless_ the argument needs to be quoted anyway. For example,
`test\` would not need to be quoted, while `test \` would need to be.
To keep the code simple, however, and therefore easier to reason about
and ensure its correctness, we now _always_ quote an argument that ends
in a backslash.
This addresses CVE-2019-1350.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, this function was written without focusing on speed,
intending to make reviewing the code as easy as possible, to avoid any
bugs in this critical code.
Turns out: we can do much better on both accounts. With this patch, we
make it as fast as this developer can make it go:
- We avoid the call to `is_dir_sep()` and make all the character
comparisons explicit.
- We avoid the cost of calling `strncasecmp()` and unroll the test for
`.git` and `git~1`, not even using `tolower()` because it is faster to
compare against two constant values.
- We look for `.git` and `.git~1` first thing, and return early if not
found.
- We also avoid calling a separate function for detecting chains of
spaces and periods.
Each of these improvements has a noticeable impact on the speed of
`is_ntfs_dotgit()`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Probably inspired by HFS' resource streams, NTFS supports "Alternate
Data Streams": by appending `:<stream-name>` to the file name,
information in addition to the file contents can be written and read,
information that is copied together with the file (unless copied to a
non-NTFS location).
These Alternate Data Streams are typically used for things like marking
an executable as having just been downloaded from the internet (and
hence not necessarily being trustworthy).
In addition to a stream name, a stream type can be appended, like so:
`:<stream-name>:<stream-type>`. Unless specified, the default stream
type is `$DATA` for files and `$INDEX_ALLOCATION` for directories. In
other words, `.git::$INDEX_ALLOCATION` is a valid way to reference the
`.git` directory!
In our work in Git v2.2.1 to protect Git on NTFS drives under
`core.protectNTFS`, we focused exclusively on NTFS short names, unaware
of the fact that NTFS Alternate Data Streams offer a similar attack
vector.
Let's fix this.
Seeing as it is better to be safe than sorry, we simply disallow paths
referring to *any* NTFS Alternate Data Stream of `.git`, not just
`::$INDEX_ALLOCATION`. This also simplifies the implementation.
This closes CVE-2019-1352.
Further reading about NTFS Alternate Data Streams:
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c54dec26-1551-4d3a-a0ea-4fa40f848eb3
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The config setting `core.protectNTFS` is specifically designed to work
not only on Windows, but anywhere, to allow for repositories hosted on,
say, Linux servers to be protected against NTFS-specific attack vectors.
As a consequence, `is_ntfs_dotgit()` manually splits backslash-separated
paths (but does not do the same for paths separated by forward slashes),
under the assumption that the backslash might not be a valid directory
separator on the _current_ Operating System.
However, the two callers, `verify_path()` and `fsck_tree()`, are
supposed to feed only individual path segments to the `is_ntfs_dotgit()`
function.
This causes a lot of duplicate scanning (and very inefficient scanning,
too, as the inner loop of `is_ntfs_dotgit()` was optimized for
readability rather than for speed.
Let's simplify the design of `is_ntfs_dotgit()` by putting the burden of
splitting the paths by backslashes as directory separators on the
callers of said function.
Consequently, the `verify_path()` function, which already splits the
path by directory separators, now treats backslashes as directory
separators _explicitly_ when `core.protectNTFS` is turned on, even on
platforms where the backslash is _not_ a directory separator.
Note that we have to repeat some code in `verify_path()`: if the
backslash is not a directory separator on the current Operating System,
we want to allow file names like `\`, but we _do_ want to disallow paths
that are clearly intended to cause harm when the repository is cloned on
Windows.
The `fsck_tree()` function (the other caller of `is_ntfs_dotgit()`) now
needs to look for backslashes in tree entries' names specifically when
`core.protectNTFS` is turned on. While it would be tempting to
completely disallow backslashes in that case (much like `fsck` reports
names containing forward slashes as "full paths"), this would be
overzealous: when `core.protectNTFS` is turned on in a non-Windows
setup, backslashes are perfectly valid characters in file names while we
_still_ want to disallow tree entries that are clearly designed to
exploit NTFS-specific behavior.
This simplification will make subsequent changes easier to implement,
such as turning `core.protectNTFS` on by default (not only on Windows)
or protecting against attack vectors involving NTFS Alternate Data
Streams.
Incidentally, this change allows for catching malicious repositories
that contain tree entries of the form `dir\.gitmodules` already on the
server side rather than only on the client side (and previously only on
Windows): in contrast to `is_ntfs_dotgit()`, the
`is_ntfs_dotgitmodules()` function already expects the caller to split
the paths by directory separators.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In preparation to flipping the default on `core.protectNTFS`, let's have
some way to measure the speed impact of this config setting reliably
(and for comparison, the `core.protectHFS` config setting).
For now, this is a manual performance benchmark:
./t/helper/test-path-utils protect_ntfs_hfs [arguments...]
where the arguments are an optional number of file names to test with,
optionally followed by minimum and maximum length of the random file
names. The default values are one million, 3 and 20, respectively.
Just like `sqrti()` in `bisect.c`, we introduce a very simple function
to approximation the square root of a given value, in order to avoid
having to introduce the first user of `<math.h>` in Git's source code.
Note: this is _not_ implemented as a Unix shell script in t/perf/
because we really care about _very_ precise timings here, and Unix shell
scripts are simply unsuited for precise and consistent benchmarking.
Signed-off-by: Garima Singh <garima.singh@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This patch series fixes an issue where Git could formerly have been
tricked into creating a `.git` file with an unexpected (and therefore
unprotected) NTFS short name.
Incidentally, it also fixes an issue where a tree entry containing a
backslash could be tricked into following a symbolic link, i.e. Git
could be tricked into writing files outside the worktree.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `--export-marks` option of `git fast-import` is exposed also via the
in-stream command `feature export-marks=...` and it allows overwriting
arbitrary paths.
This topic branch prevents the in-stream version, to prevent arbitrary
file accesses by `git fast-import` streams coming from untrusted sources
(e.g. in remote helpers that are based on `git fast-import`).
This fixes CVE-2019-1348.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, this function was completely undocumented. It is worth,
though, to explain what is going on, as it is not really obvious at all.
Suggested-by: Garima Singh <garima.singh@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The backslash character is not a valid part of a file name on Windows.
Hence it is dangerous to allow writing files that were unpacked from
tree objects, when the stored file name contains a backslash character:
it will be misinterpreted as directory separator.
This not only causes ambiguity when a tree contains a blob `a\b` and a
tree `a` that contains a blob `b`, but it also can be used as part of an
attack vector to side-step the careful protections against writing into
the `.git/` directory during a clone of a maliciously-crafted
repository.
Let's prevent that, addressing CVE-2019-1354.
Note: we guard against backslash characters in tree objects' file names
_only_ on Windows (because on other platforms, even on those where NTFS
volumes can be mounted, the backslash character is _not_ a directory
separator), and _only_ when `core.protectNTFS = true` (because users
might need to generate tree objects for other platforms, of course
without touching the worktree, e.g. using `git update-index
--cacheinfo`).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In addition to preventing `.git` from being tracked by Git, on Windows
we also have to prevent `git~1` from being tracked, as the default NTFS
short name (also known as the "8.3 filename") for the file name `.git`
is `git~1`, otherwise it would be possible for malicious repositories to
write directly into the `.git/` directory, e.g. a `post-checkout` hook
that would then be executed _during_ a recursive clone.
When we implemented appropriate protections in 2b4c6efc82 (read-cache:
optionally disallow NTFS .git variants, 2014-12-16), we had analyzed
carefully that the `.git` directory or file would be guaranteed to be
the first directory entry to be written. Otherwise it would be possible
e.g. for a file named `..git` to be assigned the short name `git~1` and
subsequently, the short name generated for `.git` would be `git~2`. Or
`git~3`. Or even `~9999999` (for a detailed explanation of the lengths
we have to go to protect `.gitmodules`, see the commit message of
e7cb0b4455 (is_ntfs_dotgit: match other .git files, 2018-05-11)).
However, by exploiting two issues (that will be addressed in a related
patch series close by), it is currently possible to clone a submodule
into a non-empty directory:
- On Windows, file names cannot end in a space or a period (for
historical reasons: the period separating the base name from the file
extension was not actually written to disk, and the base name/file
extension was space-padded to the full 8/3 characters, respectively).
Helpfully, when creating a directory under the name, say, `sub.`, that
trailing period is trimmed automatically and the actual name on disk
is `sub`.
This means that while Git thinks that the submodule names `sub` and
`sub.` are different, they both access `.git/modules/sub/`.
- While the backslash character is a valid file name character on Linux,
it is not so on Windows. As Git tries to be cross-platform, it
therefore allows backslash characters in the file names stored in tree
objects.
Which means that it is totally possible that a submodule `c` sits next
to a file `c\..git`, and on Windows, during recursive clone a file
called `..git` will be written into `c/`, of course _before_ the
submodule is cloned.
Note that the actual exploit is not quite as simple as having a
submodule `c` next to a file `c\..git`, as we have to make sure that the
directory `.git/modules/b` already exists when the submodule is checked
out, otherwise a different code path is taken in `module_clone()` that
does _not_ allow a non-empty submodule directory to exist already.
Even if we will address both issues nearby (the next commit will
disallow backslash characters in tree entries' file names on Windows,
and another patch will disallow creating directories/files with trailing
spaces or periods), it is a wise idea to defend in depth against this
sort of attack vector: when submodules are cloned recursively, we now
_require_ the directory to be empty, addressing CVE-2019-1349.
Note: the code path we patch is shared with the code path of `git
submodule update --init`, which must not expect, in general, that the
directory is empty. Hence we have to introduce the new option
`--force-init` and hand it all the way down from `git submodule` to the
actual `git submodule--helper` process that performs the initial clone.
Reported-by: Nicolas Joly <Nicolas.Joly@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
As with export-marks in the previous commit, import-marks can access the
filesystem. This is significantly less dangerous than export-marks
because it only involves reading from arbitrary paths, rather than
writing them. However, it could still be surprising and have security
implications (e.g., exfiltrating data from a service that accepts
fast-import streams).
Let's lump it (and its "if-exists" counterpart) in with export-marks,
and enable the in-stream version only if --allow-unsafe-features is set.
Signed-off-by: Jeff King <peff@peff.net>
The fast-import stream command "feature export-marks=<path>" lets the
stream write marks to an arbitrary path. This may be surprising if you
are running fast-import against an untrusted input (which otherwise
cannot do anything except update Git objects and refs).
Let's disallow the use of this feature by default, and provide a
command-line option to re-enable it (you can always just use the
command-line --export-marks as well, but the in-stream version provides
an easy way for exporters to control the process).
This is a backwards-incompatible change, since the default is flipping
to the new, safer behavior. However, since the main users of the
in-stream versions would be import/export-based remote helpers, and
since we trust remote helpers already (which are already running
arbitrary code), we'll pass the new option by default when reading a
remote helper's stream. This should minimize the impact.
Note that the implementation isn't totally simple, as we have to work
around the fact that fast-import doesn't parse its command-line options
until after it has read any "feature" lines from the stream. This is how
it lets command-line options override in-stream. But in our case, it's
important to parse the new --allow-unsafe-features first.
There are three options for resolving this:
1. Do a separate "early" pass over the options. This is easy for us to
do because there are no command-line options that allow the
"unstuck" form (so there's no chance of us mistaking an argument
for an option), though it does introduce a risk of incorrect
parsing later (e.g,. if we convert to parse-options).
2. Move the option parsing phase back to the start of the program, but
teach the stream-reading code never to override an existing value.
This is tricky, because stream "feature" lines override each other
(meaning we'd have to start tracking the source for every option).
3. Accept that we might parse a "feature export-marks" line that is
forbidden, as long we don't _act_ on it until after we've parsed
the command line options.
This would, in fact, work with the current code, but only because
the previous patch fixed the export-marks parser to avoid touching
the filesystem.
So while it works, it does carry risk of somebody getting it wrong
in the future in a rather subtle and unsafe way.
I've gone with option (1) here as simple, safe, and unlikely to cause
regressions.
This fixes CVE-2019-1348.
Signed-off-by: Jeff King <peff@peff.net>
When we parse the --export-marks option, we don't immediately open the
file, but we do create any leading directories. This can be especially
confusing when a command-line option overrides an in-stream one, in
which case we'd create the leading directory for the in-stream file,
even though we never actually write the file.
Let's instead create the directories just before opening the file, which
means we'll create only useful directories. Note that this could change
the handling of relative paths if we chdir() in between, but we don't
actually do so; the only permanent chdir is from setup_git_directory()
which runs before either code path (potentially we should take the
pre-setup dir into account to avoid surprising the user, but that's an
orthogonal change).
The test just adapts the existing "override" test to use paths with
leading directories. This checks both that the correct directory is
created (which worked before but was not tested), and that the
overridden one is not (our new fix here).
While we're here, let's also check the error result of
safe_create_leading_directories(). We'd presumably notice any failure
immediately after when we try to open the file itself, but we can give a
more specific error message in this case.
Signed-off-by: Jeff King <peff@peff.net>
When asked to import marks from "subdir/file.marks", we create the
leading directory "subdir" if it doesn't exist. This makes no sense for
importing marks, where we only ever open the path for reading.
Most of the time this would be a noop, since if the marks file exists,
then the leading directories exist, too. But if it doesn't (e.g.,
because --import-marks-if-exists was used), then we'd create the useless
directory.
This dates back to 580d5f83e7 (fast-import: always create marks_file
directories, 2010-03-29). Even then it was useless, so it seems to have
been added in error alongside the --export-marks case (which _is_
helpful).
Signed-off-by: Jeff King <peff@peff.net>
We parse options like "--max-pack-size=" using skip_prefix(), which
makes sense to get at the bytes after the "=". However, we also parse
"--quiet" and "--stats" with skip_prefix(), which allows things like
"--quiet-nonsense" to behave like "--quiet".
This was a mistaken conversion in 0f6927c229 (fast-import: put option
parsing code in separate functions, 2009-12-04). Let's tighten this to
an exact match, which was the original intent.
Signed-off-by: Jeff King <peff@peff.net>
Our tests confirm that providing two "import-marks" options in a
fast-import stream is an error. However, the invoked command would fail
even without covering this case, because the marks files themselves do
not actually exist. Let's create the files to make sure we fail for the
right reason (we actually do, because the option parsing happens before
we open anything, but this future-proofs our test).
Signed-off-by: Jeff King <peff@peff.net>
We recently banned submodule urls that look like
command-line options. This is the matching change to ban
leading-dash paths.
As with the urls, this should not break any use cases that
currently work. Even with our "--" separator passed to
git-clone, git-submodule.sh gets confused. Without the code
portion of this patch, the clone of "-sub" added in t7417
would yield results like:
/path/to/git-submodule: 410: cd: Illegal option -s
/path/to/git-submodule: 417: cd: Illegal option -s
/path/to/git-submodule: 410: cd: Illegal option -s
/path/to/git-submodule: 417: cd: Illegal option -s
Fetched in submodule path '-sub', but it did not contain b56243f8f4eb91b2f1f8109452e659f14dd3fbe4. Direct fetching of that commit failed.
Moreover, naively adding such a submodule doesn't work:
$ git submodule add $url -sub
The following path is ignored by one of your .gitignore files:
-sub
even though there is no such ignore pattern (the test script
hacks around this with a well-placed "git mv").
Unlike leading-dash urls, though, it's possible that such a
path _could_ be useful if we eventually made it work. So
this commit should be seen not as recommending a particular
policy, but rather temporarily closing off a broken and
possibly dangerous code-path. We may revisit this decision
later.
There are two minor differences to the tests in t7416 (that
covered urls):
1. We don't have a "./-sub" escape hatch to make this
work, since the submodule code expects to be able to
match canonical index names to the path field (so you
are free to add submodule config with that path, but we
would never actually use it, since an index entry would
never start with "./").
2. After this patch, cloning actually succeeds. Since we
ignore the submodule.*.path value, we fail to find a
config stanza for our submodule at all, and simply
treat it as inactive. We still check for the "ignoring"
message.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous commit taught the submodule code to invoke our
"git clone $url $path" with a "--" separator so that we
aren't confused by urls or paths that start with dashes.
However, that's just one code path. It's not clear if there
are others, and it would be an easy mistake to add one in
the future. Moreover, even with the fix in the previous
commit, it's quite hard to actually do anything useful with
such an entry. Any url starting with a dash must fall into
one of three categories:
- it's meant as a file url, like "-path". But then any
clone is not going to have the matching path, since it's
by definition relative inside the newly created clone. If
you spell it as "./-path", the submodule code sees the
"/" and translates this to an absolute path, so it at
least works (assuming the receiver has the same
filesystem layout as you). But that trick does not apply
for a bare "-path".
- it's meant as an ssh url, like "-host:path". But this
already doesn't work, as we explicitly disallow ssh
hostnames that begin with a dash (to avoid option
injection against ssh).
- it's a remote-helper scheme, like "-scheme::data". This
_could_ work if the receiver bends over backwards and
creates a funny-named helper like "git-remote--scheme".
But normally there would not be any helper that matches.
Since such a url does not work today and is not likely to do
anything useful in the future, let's simply disallow them
entirely. That protects the existing "git clone" path (in a
belt-and-suspenders way), along with any others that might
exist.
Our tests cover two cases:
1. A file url with "./" continues to work, showing that
there's an escape hatch for people with truly silly
repo names.
2. A url starting with "-" is rejected.
Note that we expect case (2) to fail, but it would have done
so even without this commit, for the reasons given above.
So instead of just expecting failure, let's also check for
the magic word "ignoring" on stderr. That lets us know that
we failed for the right reason.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we clone a submodule, we call "git clone $url $path".
But there's nothing to say that those components can't begin
with a dash themselves, confusing git-clone into thinking
they're options. Let's pass "--" to make it clear what we
expect.
There's no test here, because it's actually quite hard to
make these names work, even with "git clone" parsing them
correctly. And we're going to restrict these cases even
further in future commits. So we'll leave off testing until
then; this is just the minimal fix to prevent us from doing
something stupid with a badly formed entry.
Reported-by: joernchen <joernchen@phenoelit.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-2.13:
Git 2.13.7
verify_path: disallow symlinks in .gitmodules
update-index: stat updated files earlier
verify_dotfile: mention case-insensitivity in comment
verify_path: drop clever fallthrough
skip_prefix: add case-insensitive variant
is_{hfs,ntfs}_dotgitmodules: add tests
is_ntfs_dotgit: match other .git files
is_hfs_dotgit: match other .git files
is_ntfs_dotgit: use a size_t for traversing string
submodule-config: verify submodule names as paths
A regression fix for 2.11 that made the code to read the list of
alternate object stores overrun the end of the string.
* jk/info-alternates-fix:
read_info_alternates: warn on non-trivial errors
read_info_alternates: read contents into strbuf
"git fetch <there> <src>:<dst>" allows an object name on the <src>
side when the other side accepts such a request since Git v2.5, but
the documentation was left stale.
* jc/fetch-refspec-doc-update:
fetch doc: src side of refspec could be full SHA-1
Many codepaths did not diagnose write failures correctly when disks
go full, due to their misuse of write_in_full() helper function,
which have been corrected.
* jk/write-in-full-fix:
read_pack_header: handle signed/unsigned comparison in read result
config: flip return value of store_write_*()
notes-merge: use ssize_t for write_in_full() return value
pkt-line: check write_in_full() errors against "< 0"
convert less-trivial versions of "write_in_full() != len"
avoid "write_in_full(fd, buf, len) != len" pattern
get-tar-commit-id: check write_in_full() return against 0
config: avoid "write_in_full(fd, buf, len) < len" pattern
The scripts to drive TravisCI has been reorganized and then an
optimization to avoid spending cycles on a branch whose tip is
tagged has been implemented.
* ls/travis-scriptify:
travis-ci: fix "skip_branch_tip_with_tag()" string comparison
travis: dedent a few scripts that are indented overly deeply
travis-ci: skip a branch build if equal tag is present
travis-ci: move Travis CI code into dedicated scripts
The checkpoint command "git fast-import" did not flush updates to
refs and marks unless at least one object was created since the
last checkpoint, which has been corrected, as these things can
happen without any new object getting created.
* er/fast-import-dump-refs-on-checkpoint:
fast-import: checkpoint: dump branches/tags/marks even if object_count==0
"git fast-export" with -M/-C option issued "copy" instruction on a
path that is simultaneously modified, which was incorrect.
* jt/fast-export-copy-modify-fix:
fast-export: do not copy from modified file
"git branch -M a b" while on a branch that is completely unrelated
to either branch a or branch b misbehaved when multiple worktree
was in use. This has been fixed.
* nd/worktree-kill-parse-ref:
branch: fix branch renaming not updating HEADs correctly
This is the "theoretically more correct" approach of simply
stepping back to the state before plumbing commands started paying
attention to "color.ui" configuration variable.
* jk/ref-filter-colors-fix:
tag: respect color.ui config
Revert "color: check color.ui in git_default_config()"
Revert "t6006: drop "always" color config tests"
Revert "color: make "always" the same as "auto" in config"
color: make "always" the same as "auto" in config
provide --color option for all ref-filter users
t3205: use --color instead of color.branch=always
t3203: drop "always" color test
t6006: drop "always" color config tests
t7502: use diff.noprefix for --verbose test
t7508: use test_terminal for color output
t3701: use test-terminal to collect color output
t4015: prefer --color to -c color.diff=always
test-terminal: set TERM=vt100
Improve behaviour of "git fsck" upon finding a missing object.
* rs/fsck-null-return-from-lookup:
fsck: handle NULL return of lookup_blob() and lookup_tree()
Code clean-up.
* jk/validate-headref-fix:
validate_headref: use get_oid_hex for detached HEADs
validate_headref: use skip_prefix for symref parsing
validate_headref: NUL-terminate HEAD buffer
Doc update.
* kd/doc-for-each-ref:
doc/for-each-ref: explicitly specify option names
doc/for-each-ref: consistently use '=' to between argument names and values
Finishing touches to a topic already in 'master'.
* cc/subprocess-handshake-missing-capabilities:
subprocess: loudly die when subprocess asks for an unsupported capability
In the "--format=..." option of the "git for-each-ref" command (and
its friends, i.e. the listing mode of "git branch/tag"), "%(atom:)"
(e.g. "%(refname:)", "%(body:)" used to error out. Instead, treat
them as if the colon and an empty string that follows it were not
there.
* tb/ref-filter-empty-modifier:
ref-filter.c: pass empty-string as NULL to atom parsers
Backports a moral equivalent of 2015 fix to the poll emulation from
the upstream gnulib to fix occasional breakages on HPE NonStop.
* rb/compat-poll-fix:
poll.c: always set revents, even if to zero
Fixes for a handful memory access issues identified by valgrind.
* tg/memfixes:
sub-process: use child_process.args instead of child_process.argv
http-push: fix construction of hex value from path
path.c: fix uninitialized memory access
Spell the name of our system as "Git" in the output from
request-pull script.
* ar/request-pull-phrasofix:
request-pull: capitalise "Git" to make it a proper noun
The documentation for '-X<option>' for merges was misleadingly
written to suggest that "-s theirs" exists, which is not the case.
* jc/merge-x-theirs-docfix:
merge-strategies: avoid implying that "-s theirs" exists
"git mailinfo" was loose in decoding quoted printable and produced
garbage when the two letters after the equal sign are not
hexadecimal. This has been fixed.
* rs/mailinfo-qp-decode-fix:
mailinfo: don't decode invalid =XY quoted-printable sequences
The built-in pattern to detect the "function header" for HTML did
not match <H1>..<H6> elements without any attributes, which has
been fixed.
* ik/userdiff-html-h-element-fix:
userdiff: fix HTML hunk header regexp
"git describe --match" learned to take multiple patterns in v2.13
series, but the feature ignored the patterns after the first one
and did not work at all. This has been fixed.
* jk/describe-omit-some-refs:
describe: fix matching to actually match all patterns
"git gc" tries to avoid running two instances at the same time by
reading and writing pid/host from and to a lock file; it used to
use an incorrect fscanf() format when reading, which has been
corrected.
* aw/gc-lockfile-fscanf-fix:
gc: call fscanf() with %<len>s, not %<len>c, when reading hostname
API error-proofing which happens to also squelch warnings from GCC.
* tg/refs-allowed-flags:
refs: strip out not allowed flags from ref_transaction_update
"git archive", especially when used with pathspec, stored an empty
directory in its output, even though Git itself never does so.
This has been fixed.
* rs/archive-excluded-directory:
archive: don't add empty directories to archives
Unlike "git commit-tree < file", "git commit-tree -F file" did not
pass the contents of the file verbatim and instead completed an
incomplete line at the end, if exists. The latter has been updated
to match the behaviour of the former.
* rk/commit-tree-make-F-verbatim:
commit-tree: do not complete line in -F input
In addition to "cc: <a@dd.re.ss> # cruft", "cc: a@dd.re.ss # cruft"
was taught to "git send-email" as a valid way to tell it that it
needs to also send a carbon copy to <a@dd.re.ss> in the trailer
section.
* mm/send-email-cc-cruft:
send-email: don't use Mail::Address, even if available
send-email: fix garbage removal after address
A helper function to read a single whole line into strbuf
mistakenly triggered OOM error at EOF under certain conditions,
which has been fixed.
* rs/strbuf-getwholeline-fix:
strbuf: clear errno before calling getdelim(3)
Since a9d34933 ("Merge branch 'fm/fetch-raw-sha1'", 2015-06-01) we
allow to fetch by an object name when the other side accepts such a
request, but we never updated the documentation to match.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 11b087adfd (ref-filter: consult want_color() before
emitting colors, 2017-07-13), we expect that setting
"color.ui" to "always" will enable color tag formats even
without a tty. As that commit was built on top of
136c8c8b8f (color: check color.ui in git_default_config(),
2017-07-13) from the same series, we didn't need to touch
tag's config parsing at all.
However, since we reverted 136c8c8b8f, we now need to
explicitly call git_color_default_config() to make this
work.
Let's do so, and also restore the test dropped in 0c88bf5050
(provide --color option for all ref-filter users,
2017-10-03). That commit swapped out our "color.ui=always"
test for "--color" in preparation for "always" going away.
But since it is here to stay, we should test both cases.
Note that for-each-ref also lost its color.ui support as
part of reverting 136c8c8b8f. But as a plumbing command, it
should _not_ respect the color.ui config. Since it also
gained a --color option in 0c88bf5050, that's the correct
way to ask it for color. We'll continue to test that, and
confirm that "color.ui" is not respected.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 136c8c8b8f.
That commit was trying to address a bug caused by 4c7f1819b3
(make color.ui default to 'auto', 2013-06-10), in which
plumbing like diff-tree defaulted to "auto" color, but did
not respect a "color.ui" directive to disable it.
But it also meant that we started respecting "color.ui" set
to "always". This was a known problem, but 4c7f1819b3 argued
that nobody ought to be doing that. However, that turned out
to be wrong, and we got a number of bug reports related to
"add -p" regressing in v2.14.2.
Let's revert 136c8c8b8, fixing the regression to "add -p".
This leaves the problem from 4c7f1819b3 unfixed, but:
1. It's a pretty obscure problem in the first place. I
only noticed it while working on the color code, and we
haven't got a single bug report or complaint about it.
2. We can make a more moderate fix on top by respecting
"never" but not "always" for plumbing commands. This
is just the minimal fix to go back to the working state
we had before v2.14.2.
Note that this isn't a pure revert. We now have a test in
t3701 which shows off the "add -p" regression. This can be
flipped to success.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit c5bdfe677c.
That commit was done primarily to prepare for the weakening
of "always" in 6be4595edb (color: make "always" the same as
"auto" in config, 2017-10-03). But since we've now reverted
6be4595edb, there's no need for us to remove "-c
color.ui=always" from the tests. And in fact it's a good
idea to restore these tests, to make sure that "always"
continues to work.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 6be4595edb.
That commit weakened the "always" setting of color config so
that it acted as "auto". This was meant to solve regressions
in v2.14.2 in which setting "color.ui=always" in the on-disk
config broke scripts like add--interactive, because the
plumbing diff commands began to generate color output.
This was due to 136c8c8b8f (color: check color.ui in
git_default_config(), 2017-07-13), which was in turn trying
to fix issues caused by 4c7f1819b3 (make color.ui default to
'auto', 2013-06-10). But in weakening "always", we created
even more problems, as people expect to be able to use "git
-c color.ui=always" to force color (especially because some
commands don't have their own --color flag). We can fix that
by special-casing the command-line "-c", but now things are
getting pretty confusing.
Instead of piling hacks upon hacks, let's start peeling off
the hacks. The first step is dropping the weakening of
"always", which this revert does.
Note that we could actually revert the whole series merged
in by da15b78e52. Most of that
series consists of preparations to the tests to handle the
weakening of "-c color.ui=always". But it's worth keeping
for a few reasons:
- there are some other preparatory cleanups, like
e433749d86 (test-terminal: set TERM=vt100, 2017-10-03)
- it adds "--color" options more consistently in
0c88bf5050 (provide --color option for all ref-filter
users, 2017-10-03)
- some of the cases dropping "-c" end up being more robust
and realistic tests, as in 01c94e9001 (t7508: use
test_terminal for color output, 2017-10-03)
- the preferred tool for overriding config is "--color",
and we should be modeling that consistently
We can individually revert the few commits necessary to
restore some useful tests (which will be done on top of this
patch).
Note that this isn't a pure revert; we'll keep the test
added in t3701, but mark it as failure for now.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'jk/ui-color-always-to-auto-maint' (early part):
color: make "always" the same as "auto" in config
provide --color option for all ref-filter users
t3205: use --color instead of color.branch=always
t3203: drop "always" color test
t6006: drop "always" color config tests
t7502: use diff.noprefix for --verbose test
t7508: use test_terminal for color output
t3701: use test-terminal to collect color output
t4015: prefer --color to -c color.diff=always
test-terminal: set TERM=vt100
There are "git checkout [-p][<tree-ish>][--][<paths>...]" in the
SYNOPSIS section, and "git checkout [-p][<tree-ish>][--]<paths>..."
as the header for the section that explains the "check out paths
from index/tree-ish" mode. It is unclear if we require at least one
path, or it is entirely optional.
Actually, both are wrong. Without the "-p(atch)" option, you must
have <pathspec> (otherwise, with a commit that is a <tree-ish>, you
would be checking out that commit to build a new history on top of
it). With it, it is already clear that you are checking out paths,
it is optional. In other words, you cannot omit both.
The source of the confusion is that -p(atch) is described as if it
is just another "optional" part and its description is lumped
together with the non patch mode, even though the actual end user
experience is vastly different.
Let's split the entry into two, and describe the regular mode and
the patch mode separately. This allows us to make it clear that the
regular mode MUST be given at least one pathspec, that the patch
mode can be invoked with either '-p' or '--patch' but one of these
must be given, and that the pathspec is entirely optional in the
patch mode.
Also, revamp the explanation of "checkout paths" by removing
extraneous description at the beginning, that says "checking out
paths is not checking out a branch". Explaining what it is for and
when the user wants to use it upfront is the most direct way to help
the readers.
Noticed-by: Robert P J Day
Helped-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the flags for broken and dirty were implemented in
b0176ce6b5 (builtin/describe: introduce --broken flag, 2017-03-21)
and 9f67d2e827 (Teach "git describe" --dirty option, 2009-10-21)
the completion was not updated, although these flags are useful
completions. Add them.
Signed-off-by: Thomas Braun <thomas.braun@virtuell-zuhause.de>
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a caller of sha1_object_info_extended() sets the
"contentp" field in object_info, we call unpack_sha1_rest()
but do not check whether it signaled an error.
This causes two problems:
1. We pass back NULL to the caller via the contentp field,
but the function returns "0" for success. A caller
might reasonably expect after a successful return that
it can access contentp without a NULL check and
segfault.
As it happens, this is impossible to trigger in the
current code. There is exactly one caller which uses
contentp, read_object(). And the only thing it does
after a successful call is to return the content
pointer to its caller, using NULL as a sentinel for
errors. So in effect it converts the success code from
sha1_object_info_extended() back into an error!
But this is still worth addressing avoid problems for
future users of "contentp".
2. Callers of unpack_sha1_rest() are expected to close the
zlib stream themselves on error. Which means that we're
leaking the stream.
The problem in (1) comes from from c84a1f3ed4 (sha1_file:
refactor read_object, 2017-06-21), which added the contentp
field. Before that, we called unpack_sha1_rest() via
unpack_sha1_file(), which directly used the NULL to signal
an error.
But note that the leak in (2) is actually older than that.
The original unpack_sha1_file() directly returned the result
of unpack_sha1_rest() to its caller, when it should have
been closing the zlib stream itself on error.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
lookup_blob() and lookup_tree() can return NULL if they find an object
of an unexpected type. Accessing the object member is undefined in that
case. Cast the result to a struct object pointer instead; we can do
that because object is the first member of all object types. This trick
is already used in other places in the code.
An error message is already shown by object_as_type(), which is called
by the lookup functions. The walk callback functions are expected to
handle NULL object pointers passed to them, but put_object_name() needs
a valid object, so avoid calling it without one.
Suggested-by: SZEDER Gábor <szeder.dev@gmail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Peff points out that different atom parsers handle the empty
"sub-argument" list differently. An example of this is the format
"%(refname:)".
Since callers often use `string_list_split` (which splits the empty
string with any delimiter as a 1-ary string_list containing the empty
string), this makes handling empty sub-argument strings non-ergonomic.
Let's fix this by declaring that atom parser implementations must
not care about distinguishing between the empty string "%(refname:)"
and no sub-arguments "%(refname)". Current code aborts, either with
"unrecognised arg" (e.g. "refname:") or "does not take args"
(e.g. "body:") as an error message.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Reviewed-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
strbuf_release leaves the strbuf in a valid, initialized state, so
there is no need to call strbuf_init after it.
Moreover, this is not likely to change in the future: strbuf_release
leaving the strbuf in a valid state has been easy to maintain and has
been very helpful for Git's robustness and simplicity (e.g.,
preventing use-after-free vulnerabilities).
Document the semantics so the next generation of Git developers can
become familiar with them without reading the implementation. It is
still not advisable to call strbuf_release too often because it is
wasteful, so add a note pointing to strbuf_reset for that.
The same semantics apply to strbuf_detach. Add a similar note to its
docstring to make that clear.
Improved-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our documentation advises to not re-use a strbuf, after strbuf_release
has been called on it. Use the proper reset instead.
Currently 'strbuf_release' releases and re-initializes the strbuf, so it
is safe, but slow. 'strbuf_reset' only resets the internal length variable,
such that this could also be accounted for as a micro-optimization.
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the argv is only allocated on the stack, and then assigned to
process->argv. When the start_subprocess function goes out of scope,
the local argv variable is eliminated from the stack, but the pointer is
still kept around in process->argv.
Much later when we try to access the same process->argv in
finish_command, this leads us to access a memory location that no longer
contains what we want. As argv0 is only used for printing errors, this
is not easily noticed in normal git operations. However when running
t0021-conversion.sh through valgrind, valgrind rightfully complains:
==21024== Invalid read of size 8
==21024== at 0x2ACF64: finish_command (run-command.c:869)
==21024== by 0x2D6B18: subprocess_exit_handler (sub-process.c:72)
==21024== by 0x2AB41E: cleanup_children (run-command.c:45)
==21024== by 0x2AB526: cleanup_children_on_exit (run-command.c:81)
==21024== by 0x54AD487: __run_exit_handlers (in /usr/lib/libc-2.26.so)
==21024== by 0x54AD4D9: exit (in /usr/lib/libc-2.26.so)
==21024== by 0x11A9EF: handle_builtin (git.c:550)
==21024== by 0x11ABCC: run_argv (git.c:602)
==21024== by 0x11AD8E: cmd_main (git.c:679)
==21024== by 0x1BF125: main (common-main.c:43)
==21024== Address 0x1ffeffec00 is on thread 1's stack
==21024== 1504 bytes below stack pointer
==21024==
These days, the child_process structure has its own args array, and
the standard way to set up its argv[] is to use that one, instead of
assigning to process->argv to point at an array that is outside.
Use that facility automatically fixes this issue.
Reported-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The get_oid_hex_from_objpath takes care of creating a oid from a
pathname. It does this by memcpy'ing the first two bytes of the path to
the "hex" string, then skipping the '/', and then copying the rest of the
path to the "hex" string. Currently it fails to increase the pointer to
the hex string, so the second memcpy invocation just mashes over what
was copied in the first one, and leaves the last two bytes in the string
uninitialized.
This breaks valgrind in t5540, although the test passes without
valgrind:
==5490== Use of uninitialised value of size 8
==5490== at 0x13C6B5: hexval (cache.h:1238)
==5490== by 0x13C6DB: hex2chr (cache.h:1247)
==5490== by 0x13C734: get_sha1_hex (hex.c:42)
==5490== by 0x13C78E: get_oid_hex (hex.c:53)
==5490== by 0x118BDA: get_oid_hex_from_objpath (http-push.c:1023)
==5490== by 0x118C92: process_ls_object (http-push.c:1038)
==5490== by 0x118E5B: handle_remote_ls_ctx (http-push.c:1077)
==5490== by 0x118227: xml_end_tag (http-push.c:815)
==5490== by 0x50C1448: ??? (in /usr/lib/libexpat.so.1.6.6)
==5490== by 0x50C221B: ??? (in /usr/lib/libexpat.so.1.6.6)
==5490== by 0x50BFBF2: ??? (in /usr/lib/libexpat.so.1.6.6)
==5490== by 0x50C0B24: ??? (in /usr/lib/libexpat.so.1.6.6)
==5490== Uninitialised value was created by a stack allocation
==5490== at 0x118B63: get_oid_hex_from_objpath (http-push.c:1012)
==5490==
Fix this by correctly incrementing the pointer to the "hex" variable, so
the first two bytes are left untouched by the memcpy call, and the last
two bytes are correctly initialized.
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In cleanup_path we're passing in a char array, run a memcmp on it, and
run through it without ever checking if something is in the array in the
first place. This can lead us to access uninitialized memory, for
example in t5541-http-push-smart.sh test 7, when run under valgrind:
==4423== Conditional jump or move depends on uninitialised value(s)
==4423== at 0x242FA9: cleanup_path (path.c:35)
==4423== by 0x242FA9: mkpath (path.c:456)
==4423== by 0x256CC7: refname_match (refs.c:364)
==4423== by 0x26C181: count_refspec_match (remote.c:1015)
==4423== by 0x26C181: match_explicit_lhs (remote.c:1126)
==4423== by 0x26C181: check_push_refs (remote.c:1409)
==4423== by 0x2ABB4D: transport_push (transport.c:870)
==4423== by 0x186703: push_with_options (push.c:332)
==4423== by 0x18746D: do_push (push.c:409)
==4423== by 0x18746D: cmd_push (push.c:566)
==4423== by 0x1183E0: run_builtin (git.c:352)
==4423== by 0x11973E: handle_builtin (git.c:539)
==4423== by 0x11973E: run_argv (git.c:593)
==4423== by 0x11973E: main (git.c:698)
==4423== Uninitialised value was created by a heap allocation
==4423== at 0x4C2CD8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4423== by 0x4C2F195: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4423== by 0x2C196B: xrealloc (wrapper.c:137)
==4423== by 0x29A30B: strbuf_grow (strbuf.c:66)
==4423== by 0x29A30B: strbuf_vaddf (strbuf.c:277)
==4423== by 0x242F9F: mkpath (path.c:454)
==4423== by 0x256CC7: refname_match (refs.c:364)
==4423== by 0x26C181: count_refspec_match (remote.c:1015)
==4423== by 0x26C181: match_explicit_lhs (remote.c:1126)
==4423== by 0x26C181: check_push_refs (remote.c:1409)
==4423== by 0x2ABB4D: transport_push (transport.c:870)
==4423== by 0x186703: push_with_options (push.c:332)
==4423== by 0x18746D: do_push (push.c:409)
==4423== by 0x18746D: cmd_push (push.c:566)
==4423== by 0x1183E0: run_builtin (git.c:352)
==4423== by 0x11973E: handle_builtin (git.c:539)
==4423== by 0x11973E: run_argv (git.c:593)
==4423== by 0x11973E: main (git.c:698)
==4423==
Avoid this by using skip_prefix(), which knows not to go beyond the
end of the string.
Reported-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Check if the strbuf containing data to sort is empty before attempting
to trim a trailing newline character.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It can be handy to use `--color=always` (or it's synonym
`--color`) on the command-line to convince a command to
produce color even if it's stdout isn't going to the
terminal or a pager.
What's less clear is whether it makes sense to set config
variables like color.ui to `always`. For a one-shot like:
git -c color.ui=always ...
it's potentially useful (especially if the command doesn't
directly support the `--color` option). But setting `always`
in your on-disk config is much muddier, as you may be
surprised when piped commands generate colors (and send them
to whatever is consuming the pipe downstream).
Some people have done this anyway, because:
1. The documentation for color.ui makes it sound like
using `always` is a good idea, when you almost
certainly want `auto`.
2. Traditionally not every command (and especially not
plumbing) respected color.ui in the first place. So
the confusion came up less frequently than it might
have.
The situation changed in 136c8c8b8f (color: check color.ui
in git_default_config(), 2017-07-13), which negated point
(2): now scripts using only plumbing commands (like
add-interactive) are broken by this setting.
That commit was fixing real issues (e.g., by making
`color.ui=never` work, since `auto` is the default), so we
don't want to just revert it. We could turn `always` into a
noop in plumbing commands, but that creates a hard-to-explain
inconsistency between the plumbing and other commands.
Instead, let's just turn `always` into `auto` for all config.
This does break the "one-shot" config shown above, but again,
we're probably better to have simple and consistent rules than
to try to special-case command-line config.
There is one place where `always` should retain its meaning:
on the command line, `--color=always` should continue to be
the same as `--color`, overriding any isatty checks. Since the
command-line parser also depends on git_config_colorbool(), we
can use the existence of the "var" string to deterine whether
we are serving the command-line or the config.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When ref-filter learned about want_color() in 11b087adfd
(ref-filter: consult want_color() before emitting colors,
2017-07-13), it became useful to be able to turn colors off
and on for specific commands. For git-branch, you can do so
with --color/--no-color.
But for git-for-each-ref and git-tag, the other users of
ref-filter, you have no option except to tweak the
"color.ui" config setting. Let's give both of these commands
the usual color command-line options.
This is a bit more obvious as a method for overriding the
config. And it also prepares us for the behavior of "always"
changing (so that we are still left with a way of forcing
color when our output goes to a non-terminal).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To test the color output, we must convince "git branch" to
write colors to a non-terminal. We do that now by setting
the color config to "always". In preparation for the
behavior of "always" changing, let's switch to using the
"--color" command-line option, which is more direct.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In preparation for the behavior of "always" changing to
match "auto", we can simply drop this test. We already check
other forms (like "--color") independently.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We test the %C() format placeholders with a variety of
color-inducing options, including "--color" and
"-c color.ui=always". In preparation for the behavior of
"always" changing, we need to do something with those
"always" tests.
We can drop ones that expect "always" to turn on color even
to a file, as that will become a synonym for "auto", which
is already tested.
For the "--no-color" test, we need to make sure that color
would otherwise be shown. To do this, we can use
test_terminal, which enables colors in the default setup.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To check that "status -v" respects diff config, we set
"color.diff" and look at the output of "status". We could
equally well use any diff config. Since color output depends
on a lot of other factors (like whether stdout is a tty, and
how we interpret "always"), let's use a more mundane option.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This script tests the output of status with various formats
when color is enabled. It uses the "always" setting so that
the output is valid even though we capture it in a file.
Using test_terminal gives us a more realistic environment,
and prepares us for the behavior of "always" changing.
Arguably we are testing less than before, since "auto" is
already the default, and we can no longer tell if the config
is actually doing anything.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When testing whether "add -p" can generate colors, we set
color.ui to "always". This isn't a very good test, as in the
real-world a user typically has "auto" coupled with stdout
going to a terminal (and it's plausible that this could mask
a real bug in add--interactive if we depend on plumbing's
isatty check).
Let's switch to test_terminal, which gives us a more
realistic environment. This also prepare us for future
changes to the "always" color option.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4015 contains many color-related tests which need to
override the "is stdout a tty" check. They do so by setting
the color.diff config, but we can accomplish the same with
the --color option. Besides being shorter to type, switching
will prepare us for upcoming changes to "always" when see it
in config.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The point of the test-terminal script is to simulate in the
test scripts an environment where output is going to a real
terminal.
But since test-lib.sh also sets TERM=dumb, the simulation
isn't very realistic. The color code will skip auto-coloring
for TERM=dumb, leading to us liberally sprinkling
test_terminal env TERM=vt100 git ...
through the test suite to convince the tests to actually
generate colors. Let's set TERM for programs run under
test_terminal, which is one less thing for test-writers to
remember.
In most cases the callers can be simplified, but note there
is one interesting case in t4202. It uses test_terminal to
check the auto-enabling of --decorate, but the expected
output _doesn't_ contain colors (because TERM=dumb
suppresses them). Using TERM=vt100 is closer to what the
real world looks like; adjust the expected output to match.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Of the many ways to spell the three-letter word, the variant "Git"
should be used when referring to a repository in a description; or, in
general, when it is used as a proper noun.
We thus change the pull-request template message so that it reads
"...in the Git repository at:"
Besides, this brings us in line with the documentation, see
Documentation/howto/using-signed-tag-in-pull-request.txt
Signed-off-by: Ann T Ropea <bedhanger@gmx.de>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the macro ALLOC_ARRAY to allocate an array. This is shorter and
easier, as it automatically infers the size of elements.
Patch generated with Coccinelle and contrib/coccinelle/array.cocci.
Signeg-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
lookup_blob() etc. can return NULL if the referenced object isn't of the
expected type. In theory it's wrong to reference the object member in
that case. In practice it's OK because it's located at offset 0 for all
types, so the pointer arithmetic (NULL + 0) is optimized out by the
compiler. The issue is reported by Clang's AddressSanitizer, though.
Avoid the ASan error by casting the results of the lookup functions to
struct object pointers. That works fine with NULL pointers as well. We
already rely on the object member being first in all object types in
other places in the code.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Transformations that hide multiplications can end up with an pair of
parentheses that is no longer needed. E.g. with a rule like this:
@@
expression E;
@@
- E * 2
+ double(E)
... we might get a patch like this:
- x = (a + b) * 2;
+ x = double((a + b));
Add a pair of parentheses to the preimage side of such rules.
Coccinelle will generate patches that remove them if they are present,
and it will still match expressions that lack them.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The checkpoint command cycles packfiles if object_count != 0, a sensible
test or there would be no pack files to write. Since 820b931012, the
command also dumps branches, tags and marks, but still conditionally.
However, it is possible for a command stream to modify refs or create
marks without creating any new objects.
For example, reset a branch (and keep fast-import running):
$ git fast-import
reset refs/heads/master
from refs/heads/master^
checkpoint
but refs/heads/master remains unchanged.
Other example: a commit command that re-creates an object that already
exists in the object database.
The man page also states that checkpoint "updates the refs" and that
"placing a progress command immediately after a checkpoint will inform
the reader when the checkpoint has been completed and it can safely
access the refs that fast-import updated". This wasn't always true
without this patch.
This fix unconditionally calls dump_{branches,tags,marks}() for all
checkpoint commands. dump_branches() and dump_tags() are cheap to call
in the case of a no-op.
Add tests to t9300 that observe the (non-packfiles) effects of
checkpoint.
Signed-off-by: Eric Rannaud <e@nanocritical.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Match what is done to pfd[i].revents when compute_revents() returns
0 to the upstream gnulib's commit d42461c3 ("poll: fixes for large
fds", 2015-02-20). The revents field is set to 0, without
incrementing the value rc to be returned from the function. The
original code left the field to whatever random value the field was
initialized to.
This fixes occasional hangs in git-upload-pack on HPE NonStop.
Signed-off-by: Randall S. Becker <randall.becker@nexbridge.ca>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Leaving spaces around the `-delimeters for commands means asciidoc fails
to parse them as the start of a literal string. Remove an extraneous
space that is causing a literal to not be formatted as such.
Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
Acked-by: Andreas Heiduk <asheiduk@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a candidate HEAD isn't a symref, we check that it
contains a viable sha1. But in a post-sha1 world, we should
be checking whether it has any plausible object-id.
We can do that by switching to get_oid_hex().
Note that both before and after this patch, we only check
for a plausible object id at the start of the file, and then
call that good enough. We ignore any content _after_ the
hex, so a string like:
0123456789012345678901234567890123456789 foo
is accepted. Though we do put extra bytes like this into
some pseudorefs (e.g., FETCH_HEAD), we don't typically do so
with HEAD. We could tighten this up by using parse_oid_hex(),
like:
if (!parse_oid_hex(buffer, &oid, &end) &&
*end++ == '\n' && *end == '\0')
return 0;
But we're probably better to remain on the loose side. We're
just checking here for a plausible-looking repository
directory, so heuristics are acceptable (if we really want
to be meticulous, we should use the actual ref code to parse
HEAD).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since the previous commit guarantees that our symref buffer
is NUL-terminated, we can just use skip_prefix() and friends
to parse it. This is shorter and saves us having to deal
with magic numbers and keeping the "len" counter up to date.
While we're at it, let's name the rather obscure "buf" to
"refname", since that is the thing we are parsing with it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we are checking to see if we have a git repo, we peek
into the HEAD file and see if it's a plausible symlink,
symref, or detached HEAD.
For the latter two, we read the contents with read_in_full(),
which means they aren't NUL-terminated. The symref check is
careful to respect the length we got, but the sha1 check
will happily parse up to 40 bytes, even if we read fewer.
E.g.,:
echo 1234 >.git/HEAD
git rev-parse
will parse 36 uninitialized bytes from our stack buffer.
This isn't a big deal in practice. Our buffer is 256 bytes,
so we know we'll never read outside of it. The worst case is
that the uninitialized bytes look like valid hex, and we
claim a bogus HEAD file is valid. The chances of this
happening randomly are quite slim, but let's be careful.
One option would be to check that "len == 41" before feeding
the buffer to get_sha1_hex(). But we'd like to eventually
prepare for a world with variable-length hashes. Let's
NUL-terminate as soon as we've read the buffer (we already
even leave a spare byte to do so!). That fixes this problem
without depending on the size of an object id.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ability to exclude paths with a negative pathspec is not mentioned
in the man pages for git grep and other commands where it might be
useful.
Add an example and a pointer to the pathspec glossary entry in the man
page for git grep to help the user to discover this ability.
Add similar pointers from the git-add and git-status man pages.
Additionally,
- Add a test for the behaviour when multiple exclusions are present.
- Add a test for the ^ alias.
- Improve name of existing test.
- Improve grammar in glossary description of the exclude pathspec.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Manav Rathi <mnvrth@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
References to multi-word configuration variable names in our
documentation must consistently use camelCase to highlight where
the word boundaries are, even though these are treated case
insensitively.
Fix a few places that spell them in all lowercase, which makes
them harder to read.
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The description of `-Xours` merge option has a parenthetical note
that tells the readers that it is very different from `-s ours`,
which is correct, but the description of `-Xtheirs` that follows it
carelessly says "this is the opposite of `ours`", giving a false
impression that the readers also need to be warned that it is very
different from `-s theirs`, which in reality does not even exist.
Clarify it a bit to avoid misleading readers.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Decode =XY in quoted-printable segments only if X and Y are hexadecimal
digits, otherwise just copy them. That's at least better than
interpreting negative results from hexval() as a character.
Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Current HTML header regexp doesn't match headers without attributes.
So it fails to match <h1>...</h1>, while <h1 class="smth">...</h1> matches.
Make attributes optional to fix this. The regexp is still far from
perfect, but now it at least handles the common case.
Signed-off-by: Ilya Kantor <iliakan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-read-tree manpage has a table that is meant to
be shown with its spacing exactly as it is in the source. We
mark it as a "literal paragraph" by indenting each line by
at least one space. This renders OK with asciidoc for both
the HTML and manpage versions.
But there are two problems when we render it with
asciidoctor.
The first is that some lines mix tabs and spaces. Even if
asciidoctor is correctly configured for 8-space tabs, it
seems to handle this case differently, soaking up some of
the initial literal-paragraph spaces and mis-aligning the
table text.
The second problem is that the table uses blank lines to
group rows. But as blank lines separate paragraphs in
asciidoc, this actually means that each chunk of the table
is rendered in its own pre-formatted <div> block. This
happens even with vanilla asciidoc, but there's no visible
result because the literal paragraphs aren't styled in any
special way. But with asciidoctor (or at least the styles
used on git-scm.com), literal paragraphs are styled with a
different background. This breaks the table into a visually
distracting sequence of chunks.
We can fix both by adding a literal-paragraph block
delimiter. That turns the whole table into a single block
(for both implementations) and causes asciidoctor to render
the indentation as it is in the source.
Reported-at: https://github.com/git/git-scm.com/issues/1023
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit dc944b65f1 (get_sha1_with_context: dynamically
allocate oc->path, 2017-05-19) changed the rules that
callers must follow for seeing if we parsed a path in the
object name. The rules switched from "check if the oc.path
buffer is empty" to "check if the oc.path pointer is NULL".
But that commit forgot to update some sites in
cat_one_file(), meaning we might dereference a NULL pointer.
You can see this by making a path-aware request like
--textconv without specifying --path, and giving an object
name that doesn't have a path in it. Like:
git cat-file --textconv HEAD
which will reliably segfault.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
09f5e97 ("travis-ci: skip a branch build if equal tag is present",
2017-09-17) introduced the "skip_branch_tip_with_tag" function with
a broken string comparison. Fix it!
Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When run with the "-C" option, fast-export writes 'C' commands in its
output whenever the internal diff mechanism detects a file copy,
indicating that fast-import should copy the given existing file to the
given new filename. However, the diff mechanism works against the
prior version of the file, whereas fast-import uses whatever is current.
This causes issues when a commit both modifies a file and uses it as the
source for a copy.
Therefore, teach fast-export to refrain from writing 'C' when it has
already written a modification command for a file.
An existing test in t9350-fast-export is also fixed in this patch. The
existing line "C file6 file7" copies the wrong version of file6, but it
has coincidentally worked because file7 was subsequently overridden.
Reported-by: Juraj Oršulić <juraj.orsulic@fer.hr>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some implementations of `echo` support the '-e' option to enable
backslash interpretation of the following string.
As an addition, they support '-E' to turn it off.
However, none of these are portable, POSIX doesn't even mention them,
and many implementations don't support them.
A check for '-n' is already done in check-non-portable-shell.pl,
extend it to cover '-n', '-e' or '-E'.
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If you pass a newly initialized or newly cleared `string_list` to
`for_each_string_list_item()`, then the latter does
for (
item = (list)->items; /* NULL */
item < (list)->items + (list)->nr; /* NULL + 0 */
++item)
Even though this probably works almost everywhere, it is undefined
behavior, and it could plausibly cause highly-optimizing compilers to
misbehave. C99 section 6.5.6 paragraph 8 explains:
If both the pointer operand and the result point to elements
of the same array object, or one past the last element of the
array object, the evaluation shall not produce an overflow;
otherwise, the behavior is undefined.
and (6.3.2.3.3) a null pointer does not point to anything.
Guard the loop with a NULL check to make the intent crystal clear to
even the most pedantic compiler. A suitably clever compiler could let
the NULL check only run in the first iteration, but regardless, this
overhead is likely to be dwarfed by the work to be done on each item.
This problem was noticed by Coverity.
[jn: using a NULL check instead of a placeholder empty list;
fleshed out the commit message based on mailing list discussion]
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we fail to open $GIT_DIR/info/alternates, we silently
assume there are no alternates. This is the right thing to
do for ENOENT, but not for other errors.
A hard error is probably overkill here. If we fail to read
an alternates file then either we'll complete our operation
anyway, or we'll fail to find some needed object. Either
way, a warning is good idea. And we already have a helper
function to handle this pattern; let's just call
warn_on_fopen_error().
Note that technically the errno from strbuf_read_file()
might be from a read() error, not open(). But since read()
would never return ENOENT or ENOTDIR, and since it produces
a generic "unable to access" error, it's suitable for
handling errors from either.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch fixes a regression in v2.11.1 where we might read
past the end of an mmap'd buffer. It was introduced in
cf3c635210.
The link_alt_odb_entries() function has always taken a
ptr/len pair as input. Until cf3c635210 (alternates: accept
double-quoted paths, 2016-12-12), we made a copy of those
bytes in a string. But after that commit, we switched to
parsing the input left-to-right, and we ignore "len"
totally, instead reading until we hit a NUL.
This has mostly gone unnoticed for a few reasons:
1. All but one caller passes a NUL-terminated string, with
"len" pointing to the NUL.
2. The remaining caller, read_info_alternates(), passes in
an mmap'd file. Unless the file is an exact multiple of
the page size, it will generally be followed by NUL
padding to the end of the page, which just works.
The easiest way to demonstrate the problem is to build with:
make SANITIZE=address NO_MMAP=Nope test
Any test which involves $GIT_DIR/info/alternates will fail,
as the mmap emulation (correctly) does not add an extra NUL,
and ASAN complains about reading past the end of the buffer.
One solution would be to teach link_alt_odb_entries() to
respect "len". But it's actually a bit tricky, since we
depend on unquote_c_style() under the hood, and it has no
ptr/len variant.
We could also just make a NUL-terminated copy of the input
bytes and operate on that. But since all but one caller
already is passing a string, instead let's just fix that
caller to provide NUL-terminated input in the first place,
by swapping out mmap for strbuf_read_file().
There's no advantage to using mmap on the alternates file.
It's not expected to be large (and anyway, we're copying its
contents into an in-memory linked list). Nor is using
git_open() buying us anything here, since we don't keep the
descriptor open for a long period of time.
Let's also drop the "len" parameter entirely from
link_alt_odb_entries(), since it's completely ignored. That
will avoid any new callers re-introducing a similar bug.
Reported-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier in this codepath, we (ab)used "%<len>c" to read the hostname
recorded in the lockfile into locking_host[HOST_NAME_MAX + 1] while
substituting <len> with the actual value of HOST_NAME_MAX.
This turns out to be incorrect, as it is an instruction to read
exactly the specified number of bytes. Because we are trying to
read at most that many bytes, we should be using "%<len>s" instead.
Helped-by: A. Wilcox <awilfox@adelielinux.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`git describe --match` with multiple patterns matches only first pattern.
If it fails, next patterns are not tried.
Fix it, add test cases and update existing test which has wrong
expectation.
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The result of read_in_full() may be -1 if we saw an error.
But in comparing it to a sizeof() result, that "-1" will be
promoted to size_t. In fact, the largest possible size_t
which is much bigger than our struct size. This means that
our "< sizeof(header)" error check won't trigger.
In practice, we'd go on to read uninitialized memory and
compare it to the PACK signature, which is likely to fail.
But we shouldn't get there.
We can fix this by making a direct "!=" comparison to the
requested size, rather than "<". This means that errors get
lumped in with short reads, but that's sufficient for our
purposes here. There's no PH_ERROR tp represent our case.
And anyway, this function reads from pipes and network
sockets. A network error may racily appear as EOF to us
anyway if there's data left in the socket buffers.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The store_write_section() and store_write_pairs() functions
are basically high-level wrappers around write(). But their
return values are flipped from our usual convention, using
"1" for success and "0" for failure.
Let's flip them to follow the usual write() conventions and
update all callers. As these are local to config.c, it's
unlikely that we'd have new callers in any topics in flight
(which would be silently broken by our change). But just to
be on the safe side, let's rename them to just
write_section() and write_pairs(). That also accentuates
their relationship with write().
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We store the return value of write_in_full() in a long,
though the return is actually an ssize_t. This probably
doesn't matter much in practice (since the buffer size is
alredy an unsigned long), but it might if the size if
between what can be represented in "long" and "unsigned
long", and if your size_t is larger than a "long" (as it is
on 64-bit Windows).
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As with the previous two commits, we prefer to check
write_in_full()'s return value to see if it is negative,
rather than comparing it to the input length.
These cases actually flip the logic to check for success,
making conversion a little different than in other cases. We
could of course write:
if (write_in_full(...) >= 0)
return 0;
return error(...);
But our usual method of spelling write() error checks is
just "< 0". So let's flip the logic for each of these
conditionals to our usual style.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The prior commit converted many sites to check the return
value of write_in_full() for negativity, rather than a
mismatch with the input length. This patch covers similar
cases, but where the return value is stored in an
intermediate variable. These should get the same treatment,
but they need to be reviewed more carefully since it would
be a bug if the return value is stored in an unsigned type
(which indeed, it is in one of the cases).
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The return value of write_in_full() is either "-1", or the
requested number of bytes[1]. If we make a partial write
before seeing an error, we still return -1, not a partial
value. This goes back to f6aa66cb95 (write_in_full: really
write in full or return error on disk full., 2007-01-11).
So checking anything except "was the return value negative"
is pointless. And there are a couple of reasons not to do
so:
1. It can do a funny signed/unsigned comparison. If your
"len" is signed (e.g., a size_t) then the compiler will
promote the "-1" to its unsigned variant.
This works out for "!= len" (unless you really were
trying to write the maximum size_t bytes), but is a
bug if you check "< len" (an example of which was fixed
recently in config.c).
We should avoid promoting the mental model that you
need to check the length at all, so that new sites are
not tempted to copy us.
2. Checking for a negative value is shorter to type,
especially when the length is an expression.
3. Linus says so. In d34cf19b89 (Clean up write_in_full()
users, 2007-01-11), right after the write_in_full()
semantics were changed, he wrote:
I really wish every "write_in_full()" user would just
check against "<0" now, but this fixes the nasty and
stupid ones.
Appeals to authority aside, this makes it clear that
writing it this way does not have an intentional
benefit. It's a historical curiosity that we never
bothered to clean up (and which was undoubtedly
cargo-culted into new sites).
So let's convert these obviously-correct cases (this
includes write_str_in_full(), which is just a wrapper for
write_in_full()).
[1] A careful reader may notice there is one way that
write_in_full() can return a different value. If we ask
write() to write N bytes and get a return value that is
_larger_ than N, we could return a larger total. But
besides the fact that this would imply a totally broken
version of write(), it would already invoke undefined
behavior. Our internal remaining counter is an unsigned
size_t, which means that subtracting too many byte will
wrap it around to a very large number. So we'll instantly
begin reading off the end of the buffer, trying to write
gigabytes (or petabytes) of data.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We ask to write 41 bytes and make sure that the return value
is at least 41. This is the same "dangerous" pattern that
was fixed in the prior commit (wherein a negative return
value is promoted to unsigned), though it is not dangerous
here because our "41" is a constant, not an unsigned
variable.
But we should convert it anyway to avoid modeling a
dangerous construct.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The return type of write_in_full() is a signed ssize_t,
because we may return "-1" on failure (even if we succeeded
in writing some bytes). But "len" itself is may be an
unsigned type (the function takes a size_t, but of course we
may have something else in the calling function). So while
it seems like:
if (write_in_full(fd, buf, len) < len)
die_errno("write error");
would trigger on error, it won't if "len" is unsigned. The
compiler sees a signed/unsigned comparison and promotes the
signed value, resulting in (size_t)-1, the highest possible
size_t (or again, whatever type the caller has). This cannot
possibly be smaller than "len", and so the conditional can
never trigger.
I scoured the code base for cases of this, but it turns out
that these two in git_config_set_multivar_in_file_gently()
are the only ones. Here our "len" is the difference between
two size_t variables, making the result an unsigned size_t.
We can fix this by just checking for a negative return value
directly, as write_in_full() will never return any value
except -1 or the full count.
There's no addition to the test suite here, since you need
to convince write() to fail in order to see the problem. The
simplest reproduction recipe I came up with is to trigger
ENOSPC:
# make a limited-size filesystem
dd if=/dev/zero of=small.disk bs=1M count=1
mke2fs small.disk
mkdir mnt
sudo mount -o loop small.disk mnt
cd mnt
sudo chown $USER:$USER .
# make a config file with some content
git config --file=config one.key value
git config --file=config two.key value
# now fill up the disk
dd if=/dev/zero of=fill
# and try to delete a key, which requires copying the rest
# of the file to config.lock, and will fail on write()
git config --file=config --unset two.key
That final command should (and does after this patch)
produce an error message due to the failed write, and leave
the file intact. Instead, it silently ignores the failure
and renames config.lock into place, leaving you with a
totally empty config file!
Reported-by: demerphq <demerphq@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Following are several fixes for duplicated words ("of of") and one
case where an extra article ("a") slipped in.
Signed-off-by: Evan Zacks <zackse@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While git doesn't track empty directories, git archive can be tricked
into putting some into archives. One way is to construct an empty tree
object, as t5004 does. While that is supported by the object database,
it can't be represented in the index and thus it's unlikely to occur in
the wild.
Another way is using the literal name of a directory in an exclude
pathspec -- its contents are are excluded, but the directory stub is
included. That's inconsistent: exclude pathspecs containing wildcards
don't leave empty directories in the archive.
Yet another way is have a few levels of nested subdirectories (e.g.
d1/d2/d3/file1) and ignoring the entries at the leaves (e.g. file1).
The directories with the ignored content are ignored as well (e.g. d3),
but their empty parents are included (e.g. d2).
As empty directories are not supported by git, they should also not be
written into archives. If an empty directory is really needed then it
can be tracked and archived by placing an empty .gitignore file in it.
There already is a mechanism in place for suppressing empty directories.
When read_tree_recursive() encounters a directory excluded by a pathspec
then it enters it anyway because it might contain included entries. It
calls the callback function before it is able to decide if the directory
is actually needed. For that reason git archive adds directories to a
queue and writes entries for them only when it encounters the first
child item -- but currently only if pathspecs with wildcards are used.
Queue *all* directories, no matter if there even are pathspecs present.
This prevents git archive from writing entries for empty directories in
all cases.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Callers are only allowed to pass certain flags into
ref_transaction_update, other flags are internal to it. To prevent
mistakes from the callers, strip the internal only flags out before
continuing.
This was noticed because of a compiler warning gcc 7.1.1 issued about
passing a NULL parameter as second parameter to memcpy (through
hashcpy):
In file included from refs.c:5:0:
refs.c: In function ‘ref_transaction_verify’:
cache.h:948:2: error: argument 2 null where non-null expected [-Werror=nonnull]
memcpy(sha_dst, sha_src, GIT_SHA1_RAWSZ);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from git-compat-util.h:165:0,
from cache.h:4,
from refs.c:5:
/usr/include/string.h:43:14: note: in a call to function ‘memcpy’ declared here
extern void *memcpy (void *__restrict __dest, const void *__restrict __src,
^~~~~~
The call to hascpy in ref_transaction_add_update is protected by the
passed in flags, but as we only add flags there, gcc notices
REF_HAVE_NEW or REF_HAVE_OLD flags could be passed in from the outside,
which would potentially result in passing in NULL as second parameter to
memcpy.
Fix both the compiler warning, and make the interface safer for its
users by stripping the internal flags out.
Suggested-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For count, sort and format, only the argument names were listed under
OPTIONS, not the option names.
Add the option names to make it clear the options exist
Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The synopsis and description inconsistently add a '=' between the
argument name and it's value. Make this consistent.
Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The handshake_capabilities() function first advertises the set of
capabilities it supports, so that the other side can pick and choose
which ones to use and ask us to enable in its response. Then we
read the response that tells us what choice the other side made. If
we saw something that we never advertised, that indicates one of two
things. The other side, i.e. the "upgraded" filter, is not paying
attention of the capabilities advertisement, and asking something
its correct operation relies on, but we are not capable of giving
that unknown feature and operate without it, so after that point the
exchange of data is a garbage-in-garbage-out. Or the other side
wanted to ask for one of the capabilities we advertised, but the
code has typo and their wish to enable a capability that its correct
operation relies on is not understood on this end. The result is
the same garbage-in-garbage-out.
Instead of sweeping such a potential bug under the rug, die loudly
when we see a request for an unsupported capability in order to
force sloppily-written filter scripts to get corrected.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we push a branch and a tag pointing to the HEAD of this branch,
then Travis CI would run the build twice. This wastes resources and
slows the testing.
Add a function to detect this situation and skip the build the branch
if appropriate. Invoke this function on every build.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the Travis CI commands are in the '.travis.yml'. The yml format
does not support functions and therefore code duplication is necessary
to run commands across all builds.
To fix this, add a library for common CI functions. Move all Travis CI
code into dedicated scripts and make them call the library first.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Doc fix.
* mg/format-ref-doc-fix:
Documentation/git-for-each-ref: clarify peeling of tags for --format
Documentation: use proper wording for ref format strings
"git archive" did not work well with pathspecs and the
export-ignore attribute.
We may want to resurrect the "we don't archive an empty directory"
bonus patch, but I do not mind merging the above early to 'next'
and leave it as a separate follow-up enhancement.
cf. <20170820090629.tumvqwzkromcykjf@sigill.intra.peff.net>
* rs/archive-excluded-directory:
archive: don't queue excluded directories
archive: factor out helper functions for handling attributes
t5001: add tests for export-ignore attributes and exclude pathspecs
Killing "git merge --edit" before the editor returns control left
the repository in a state with MERGE_MSG but without MERGE_HEAD,
which incorrectly tells the subsequent "git commit" that there was
a squash merge in progress. This has been fixed.
* mg/killed-merge:
merge: save merge state earlier
merge: split write_merge_state in two
merge: clarify call chain
Documentation/git-merge: explain --continue
"git apply" that is used as a better "patch -p1" failed to apply a
taken from a file with CRLF line endings to a file with CRLF line
endings. The root cause was because it misused convert_to_git()
that tried to do "safe-crlf" processing by looking at the index
entry at the same path, which is a nonsense---in that mode, "apply"
is not working on the data in (or derived from) the index at all.
This has been fixed.
* tb/apply-with-crlf:
apply: file commited with CRLF should roundtrip diff and apply
convert: add SAFE_CRLF_KEEP_CRLF
When handshake with a subprocess filter notices that the process
asked for an unknown capability, Git did not report what program
the offending subprocess was running. This has been corrected.
We may want a follow-up fix to tighten the error checking, though.
* cc/subprocess-handshake-missing-capabilities:
sub-process: print the cmd when a capability is unsupported
"git grep -L" and "git grep --quiet -L" reported different exit
codes; this has been corrected.
* as/grep-quiet-no-match-exit-code-fix:
git-grep: correct exit code with --quiet and -L
bash 4.4 or newer gave a warning on NUL byte in command
substitution done in "git stash"; this has been squelched.
* kd/stash-with-bash-4.4:
stash: prevent warning about null bytes in input
"git svn" used with "--localtime" option did not compute the tz
offset for the timestamp in question and instead always used the
current time, which has been corrected.
* ur/svn-local-zone:
git svn fetch: Create correct commit timestamp when using --localtime
"git am -s" has been taught that some input may end with a trailer
block that is not Signed-off-by: and it should refrain from adding
an extra blank line before adding a new sign-off in such a case.
* pw/am-signoff:
am: fix signoff when other trailers are present
"git clone --recurse-submodules --quiet" did not pass the quiet
option down to submodules.
* bw/clone-recursive-quiet:
clone: teach recursive clones to respect -q
Commands like "git rebase" accepted the --rerere-autoupdate option
from the command line, but did not always use it. This has been
fixed.
* pw/sequence-rerere-autoupdate:
cherry-pick/revert: reject --rerere-autoupdate when continuing
cherry-pick/revert: remember --rerere-autoupdate
t3504: use test_commit
rebase -i: honor --rerere-autoupdate
rebase: honor --rerere-autoupdate
am: remember --rerere-autoupdate setting
"git push --recurse-submodules $there HEAD:$target" was not
propagated down to the submodules, but now it is.
* bw/push-options-recursively-to-submodules:
submodule--helper: teach push-check to handle HEAD
The "tag.pager" configuration variable was useless for those who
actually create tag objects, as it interfered with the use of an
editor. A new mechanism has been introduced for commands to enable
pager depending on what operation is being carried out to fix this,
and then "git tag -l" is made to run pager by default.
If this works out OK, I think there are low-hanging fruits in
other commands like "git branch" that outputs long list in one mode
while taking input in another.
* ma/pager-per-subcommand-action:
git.c: ignore pager.* when launching builtin as dashed external
tag: change default of `pager.tag` to "on"
tag: respect `pager.tag` in list-mode only
t7006: add tests for how git tag paginates
git.c: provide setup_auto_pager()
git.c: let builtins opt for handling `pager.foo` themselves
builtin.h: take over documentation from api-builtin.txt
"git log --tag=no-such-tag" showed log starting from HEAD, which
has been fixed---it now shows nothing.
* jk/rev-list-empty-input:
revision: do not fallback to default when rev_input_given is set
rev-list: don't show usage when we see empty ref patterns
revision: add rev_input_given flag
t6018: flesh out empty input/output rev-list tests
Some versions of GnuPG fails to kill gpg-agent it auto-spawned
and such a left-over agent can interfere with a test. Work it
around by attempting to kill one before starting a new test.
* st/lib-gpg-kill-stray-agent:
t: lib-gpg: flush gpg agent on startup
"git commit-tree -F <file>", unlike "cat <file> | git
commit-tree" (i.e. feeding the same contents from the standard
input), added a missing final newline when the input ended in an
incomplete line.
Correct this inconsistency by leaving the incomplete line as-is,
as erring on the side of not touching the input is preferrable
and expected for a plumbing command like "commit-tree".
Signed-off-by: Ross Kabus <rkabus@aerotech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the RUNTIME_PREFIX compile-time knob isn't set, we
never look at the argv0_path we extract. We can push its
declaration inside the #ifdef to make it more clear that the
extract code is effectively a noop.
This also un-confuses leak-checking of the argv0_path
variable when RUNTIME_PREFIX isn't set. The compiler is free
to drop this static variable that we set but never look at
(and "gcc -O2" does so). But the compiler still must call
strbuf_detach(), since it doesn't know whether that function
has side effects; it just throws away the result rather than
putting it into the global.
Leak-checkers which work by scanning the data segment for
pointers to heap blocks would normally consider the block
as reachable at program end. But if the compiler removes the
variable entirely, there's nothing to find.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The system_path() function has an #ifdef in the middle of
it. Let's move the conditional logic into a sub-function.
This isolates it more, which will make it easier to change
and add to.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When setting the `eol` attribute, paths can change their dirty status
without any change in the working directory. This can cause confusion
and should at least be mentioned with a remedy.
Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
Reviewed-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using for_each_ref_in() with a full refname has always been
a questionable practice, but it became an error with
b9c8e7f2fb (prefix_ref_iterator: don't trim too much,
2017-05-22), making "git rev-parse --bisect" pretty reliably
show a BUG.
Commit 03df567fbf (for_each_bisect_ref(): don't trim
refnames, 2017-06-18) fixed this case for revision.c, but
rev-parse handles this option on its own. We can use the
same solution here (and piggy-back on its test).
Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The static-ness was silently dropped in commit 70428d1a5 ("pkt-line: add
packet_write_fmt_gently()", 2016-10-16). As a result, for each call to
packet_write_fmt_1, we allocate and leak a buffer.
We could keep the strbuf non-static and instead make sure we always
release it before returning (but not before we die, so that we don't
touch errno). That would also prepare us for threaded use. But until
that needs to happen, let's just restore the static-ness so that we get
back to a situation where we (eventually) do not continuosly keep
allocating memory.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When modifying git config, we xcalloc() a struct lock_file
but never free it. This is necessary because the tempfile
code (upon which the locking code is built) requires that
the resulting struct remain valid through the life of the
program. However, it also confuses leak-checkers like
valgrind because only the inner "struct tempfile" is still
reachable; no pointer to the outer lock_file is kept.
Other code paths solve this by using a single static lock
struct. We can do the same here, because we know that we'll
only lock and modify one config file at a time (and
assertions within the lockfile code will ensure that this
remains the case).
That removes a real leak (when we fail to free the struct
after locking fails) as well as removes the valgrind false
positive. It also means that doing N sequential
config-writes will use a constant amount of memory, rather
than leaving stale lock_files for each.
Note that since "lock" is no longer a pointer, it can't be
NULL anymore. But that's OK. We used that feature only to
avoid calling rollback_lock_file() on an already-committed
lock. Since the lockfile code keeps its own "active" flag,
it's a noop to rollback an inactive lock, and we don't have
to worry about this ourselves.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that `make` produces a file, we should have a clean target to remove
it.
Signed-off-by: Daniel Watkins <daniel@daniel-watkins.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier, dddbad728c ("timestamp_t: a new data type for timestamps",
2017-04-26) changed several types to timestamp_t.
5589e87fd8 ("name-rev: change a "long" variable to timestamp_t",
2017-05-20) cleaned up a missed variable, but both missed a _MAX
constant.
Change the remaining constant to the one appropriate for the current
type
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since f1068efefe (sha1_file: drop experimental GIT_USE_LOOKUP search, 2017-08-09)
the definition of sha1_entry_pos() has been removed from "sha1-lookup.c", so
there is no need anymore for its declaration in "sha1-lookup.h".
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using Mail::Address made sense when we didn't have a proper parser. We
now have a reasonable address parser, and using Mail::Address
_if available_ causes much more trouble than it gives benefits:
* Developers typically test one version, not both.
* Users may not be aware that installing Mail::Address will change the
behavior. They may complain about the behavior in one case without
knowing that Mail::Address is involved.
* Having this optional Mail::Address makes it tempting to anwser "please
install Mail::Address" to users instead of fixing our own code. We've
reached the stage where bugs in our parser should be fixed, not worked
around.
Signed-off-by: Matthieu Moy <git@matthieu-moy.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a followup over 9d33439 (send-email: only allow one address
per body tag, 2017-02-20). The first iteration did allow writting
Cc: <foo@example.com> # garbage
but did so by matching the regex ([^>]*>?), i.e. stop after the first
instance of '>'. However, it did not properly deal with
Cc: foo@example.com # garbage
Fix this using a new function strip_garbage_one_address, which does
essentially what the old ([^>]*>?) was doing, but dealing with more
corner-cases. Since we've allowed
Cc: "Foo # Bar" <foobar@example.com>
in previous versions, it makes sense to continue allowing it (but we
still remove any garbage after it). OTOH, when an address is given
without quoting, we just take the first word and ignore everything
after.
Signed-off-by: Matthieu Moy <git@matthieu-moy.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two bugs that sort of work together and cause
problems. Let's start with one in replace_each_worktree_head_symref.
Before fa099d2322 (worktree.c: kill parse_ref() in favor of
refs_resolve_ref_unsafe() - 2017-04-24), this code looks like this:
if (strcmp(oldref, worktrees[i]->head_ref))
continue;
set_worktree_head_symref(...);
After fa099d2322, it is possible that head_ref can be NULL. However,
the updated code takes the wrong exit. In the error case (NULL
head_ref), we should "continue;" to the next worktree. The updated
code makes us _skip_ "continue;" and update HEAD anyway.
The NULL head_ref is triggered by the second bug in add_head_info (in
the same commit). With the flag RESOLVE_REF_READING, resolve_ref_unsafe()
will abort if it cannot resolve the target ref. For orphan checkouts,
HEAD always points to an unborned branch, resolving target ref will
always fail. Now we have NULL head_ref. Now we always update HEAD.
Correct the logic in replace_ function so that we don't accidentally
update HEAD on error. As it turns out, correcting the logic bug above
breaks branch renaming completely, thanks to the second bug.
"git branch -[Mm]" does two steps (on a normal checkout, no orphan!):
- rename the branch on disk (e.g. refs/heads/abc to refs/heads/def)
- update HEAD if it points to the branch being renamed.
At the second step, since the branch pointed to by HEAD (e.g. "abc") no
longer exists on disk, we run into a temporary orphan checkout situation
that has been just corrected to _not_ update HEAD. But we need to update
HEAD since it's not actually an orphan checkout. We need to update HEAD
to move out of that orphan state.
Correct add_head_info(), remove RESOLVE_REF_READING flag. With the flag
gone, we should always return good "head_ref" in orphan checkouts (either
temporary or permanent). With good head_ref, things start to work again.
Noticed-by: Nish Aravamudan <nish.aravamudan@canonical.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Code cleanup.
* jt/subprocess-handshake:
sub-process: refactor handshake to common function
Documentation: migrate sub-process docs to header
convert: add "status=delayed" to filter process protocol
convert: refactor capabilities negotiation
convert: move multiple file filter error handling to separate function
convert: put the flags field before the flag itself for consistent style
t0021: write "OUT <size>" only on success
t0021: make debug log file name configurable
t0021: keep filter log files on comparison
Code clean-up.
* rs/move-array:
ls-files: don't try to prune an empty index
apply: use COPY_ARRAY and MOVE_ARRAY in update_image()
use MOVE_ARRAY
add MOVE_ARRAY
Code clean-up.
* rs/bswap-ubsan-fix:
bswap: convert get_be16, get_be32 and put_be32 to inline functions
bswap: convert to unsigned before shifting in get_be32
A recently added test for the "credential-cache" helper revealed
that EOF detection done around the time the connection to the cache
daemon is torn down were flaky. This was fixed by reacting to
ECONNRESET and behaving as if we got an EOF.
* dl/credential-cache-socket-in-xdg-cache:
credential-cache: interpret an ECONNRESET as an EOF
When a directory is not readable, "gitweb" fails to build the
project list. Work this around by skipping such a directory.
It might end up hiding a problem under the rug and a better
solution might be to loudly complain to the administrator pointing
out the problematic directory, but this will at least make it
"work".
* hb/gitweb-project-list:
gitweb: skip unreadable subdirectories
"git commit" when seeing an totally empty message said "you did not
edit the message", which is clearly wrong. The message has been
corrected.
* ks/commit-abort-on-empty-message-fix:
commit: check for empty message before the check for untouched template
Numerous bugs in walking of reflogs via "log -g" and friends have
been fixed.
* jk/reflog-walk:
reflog-walk: apply --since/--until to reflog dates
reflog-walk: stop using fake parents
rev-list: check reflog_info before showing usage
get_revision_1(): replace do-while with an early return
log: do not free parents when walking reflog
log: clarify comment about reflog cycles
revision: disallow reflog walking with revs->limited
t1414: document some reflog-walk oddities
The http.{sslkey,sslCert} configuration variables are to be
interpreted as a pathname that honors "~[username]/" prefix, but
weren't, which has been fixed.
* jc/http-sslkey-and-ssl-cert-are-paths:
http.c: http.sslcert and http.sslkey are both pathnames
"%C(color name)" in the pretty print format always produced ANSI
color escape codes, which was an early design mistake. They now
honor the configuration (e.g. "color.ui = never") and also tty-ness
of the output medium.
* jk/ref-filter-colors:
ref-filter: consult want_color() before emitting colors
pretty: respect color settings for %C placeholders
rev-list: pass diffopt->use_colors through to pretty-print
for-each-ref: load config earlier
color: check color.ui in git_default_config()
ref-filter: pass ref_format struct to atom parsers
ref-filter: factor out the parsing of sorting atoms
ref-filter: make parse_ref_filter_atom a private function
ref-filter: provide a function for parsing sort options
ref-filter: move need_color_reset_at_eol into ref_format
ref-filter: abstract ref format into its own struct
ref-filter: simplify automatic color reset
t: use test_decode_color rather than literal ANSI codes
docs/for-each-ref: update pointer to color syntax
check return value of verify_ref_format()
Because recent Git for Windows do come with a real msgfmt, the
build procedure for git-gui has been updated to use it instead of a
hand-rolled substitute.
* js/git-gui-msgfmt-on-windows:
git-gui (MinGW): make use of MSys2's msgfmt
git gui: allow for a long recentrepo list
git gui: de-dup selected repo from recentrepo history
git gui: cope with duplicates in _get_recentrepo
git-gui: remove duplicate entries from .gitconfig's gui.recentrepo
These used to be for manipulating the in-memory repo_tree structure,
but nowadays they are convenience wrappers to handle a few git-vs-svn
mismatches:
1. Git does not track empty directories but Subversion does. When
looking up a path in git that Subversion thinks exists and finding
nothing, we can safely assume that the path represents a
directory. This is needed when a later Subversion revision
modifies that directory.
2. Subversion allows deleting a file by copying. In Git fast-import
we have to handle that more explicitly as a deletion.
These are details of the tool's interaction with git fast-import.
Move them to fast_export.c, where other such details are handled.
This way the function names do not start with a repo_ prefix that
would clash with the repository object introduced in
v2.14.0-rc0~38^2~16 (repository: introduce the repository object,
2017-06-22) or an svn_ prefix that would clash with libsvn (in case
someone wants to link this code with libsvn some day).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since v1.7.10-rc0~118^2~4^2~4^2~3 (vcs-svn: pass paths through to
fast-import, 2010-12-13) this is an alias for fast_export_delete.
Remove the unnecessary layer of indirection.
No functional change intended.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the rest of Git, these modes are spelled as S_IFDIR,
S_IFREG | 0644, S_IFREG | 0755, and S_IFLNK. Use the same constants
in svn-fe for simplicity and consistency.
No functional change intended.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
I forgot to remove these in v1.7.10-rc0~118^2~4^2~5^2~4 (vcs-svn:
eliminate repo_tree structure, 2010-12-10).
This finishes what was started in commit 36f63b50 (vcs-svn: remove
unused prototypes, 2017-08-21).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for pack-objects describes that it creates "a packed
archive of objects", which is confusing because it may create multiple
packs if --max-pack-size is set. Update the documentation to clarify
this, and explaining in which cases such a feature would be useful.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a file .tsan-suppressions and list two functions in it: want_color()
and transfer_debug(). Both of these use the pattern
static int foo = -1;
if (foo < 0)
foo = bar();
where bar always returns the same non-negative value. This can cause
ThreadSanitizer to diagnose a race when foo is written from two threads.
That is indeed a race, although it arguably doesn't matter in practice
since it's always the same value that is written.
Add NEEDSWORK-comments to the functions so that this problem is not
forever swept way under the carpet.
The suppressions-file is used by setting the environment variable
TSAN_OPTIONS to, e.g., "suppressions=$(pwd)/.tsan-suppressions". Observe
that relative paths such as ".tsan-suppressions" might not work.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
strbuf_setlen(., 0) writes '\0' to sb.buf[0], where buf is either
allocated and unique to sb, or the global slopbuf. The slopbuf is meant
to provide a guarantee that buf is not NULL and that a freshly
initialized buffer contains the empty string, but it is not supposed to
be written to. That strbuf_setlen writes to slopbuf has at least two
implications:
First, it's wrong in principle. Second, it might be hiding misuses which
are just waiting to wreak havoc. Third, ThreadSanitizer detects a race
when multiple threads write to slopbuf at roughly the same time, thus
potentially making any more critical races harder to spot.
Avoid writing to strbuf_slopbuf in strbuf_setlen. Let's instead assert
on the first byte of slopbuf being '\0', since it helps ensure the
promised invariant of buf[len] == '\0'. (We know that "len" was already
0, or someone has messed with "alloc". If someone has fiddled with the
fields that much beyond the correct interface, they're on their own.)
This is a function which is used in many places, possibly also in hot
code paths. There are two branches in strbuf_setlen already, and we are
adding a third and possibly a fourth (in the assert). In hot code paths,
we hopefully reuse the buffer in order to avoid continous reallocations.
Thus, after a start-up phase, we should always take the same path,
which might help branch prediction, and we would never make the assert.
If a hot code path continuously reallocates, we probably have bigger
performance problems than this new safety-check.
Simple measurements do not contradict this reasoning. 100000000 times
resetting a buffer and adding the empty string takes 5.29/5.26 seconds
with/without this patch (best of three). Releasing at every iteration
yields 18.01/17.87. Adding a 30-character string instead of the empty
string yields 5.61/5.58 and 17.28/17.28(!).
This patch causes the git binary emitted by gcc 5.4.0 -O2 on my machine
to grow from 11389848 bytes to 11497184 bytes, an increase of 0.9%.
I also tried to piggy-back on the fact that we already check alloc,
which should already tell us whether we are using the slopbuf:
if (sb->alloc) {
if (len > sb->alloc - 1)
die("BUG: strbuf_setlen() beyond buffer");
sb->buf[len] = '\0';
} else {
if (len)
die("BUG: strbuf_setlen() beyond buffer");
assert(!strbuf_slopbuf[0]);
}
sb->len = len;
That didn't seem to be much slower (5.38, 18.02, 5.70, 17.32 seconds),
but it does introduce some minor code duplication. The resulting git
binary was 11510528 bytes large (another 0.1% increase).
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the `git merge` process is killed while waiting for the editor to
finish, the merge state is lost but the prepared merge msg and tree is kept.
So, a subsequent `git commit` creates a squashed merge even when the
user asked for proper merge commit originally.
Demonstrate the problem with a test crafted after the in t7502. The test
requires EXECKEEPSPID (thus does not run under MINGW).
Save the merge state earlier (in the non-squash case) so that it does
not get lost. This makes the test pass.
Reported-by: hIpPy <hippy2981@gmail.com>
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
write_merge_state() writes out the merge heads, mode, and msg. But we
may want to write out heads, mode without the msg. So, split out heads
(+mode) into a separate function write_merge_heads() that is called by
write_merge_state().
No funtional change so far, except when these non-atomic writes are
interrupted: we write heads-mode-msg now when we used to write
heads-msg-mode.
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
prepare_to_commit() cannot be reached in the non-squash case:
It is called by merge_trivial() and finish_automerge() only, but the
calls to the latter are somewhat hard to track:
If option_commit is not set, the code in cmd_merge() uses a fake
conflict return code (ret=1) to avoid writing the tree, which also
avoids setting automerge_was_ok (just as in the proper ret==1 case), so
that finish_automerge() is not called.
To ensure that no code change breaks that assumption, safe-guard
prepare_to_commit() by a BUG() statement.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When checking the conditional of "while (me->remaining)", we did not
hold the lock. Calling find_deltas would still be safe, since it checks
"remaining" (after taking the lock) and is able to handle all values. In
fact, this could (currently) not trigger any bug: a bug could happen if
`remaining` transitioning from zero to non-zero races with the evaluation
of the while-condition, but these are always separated by the
data_ready-mechanism.
Make sure we have the lock when we read `remaining`. This does mean we
release it just so that find_deltas can take it immediately again. We
could tweak the contract so that the lock should be taken before calling
find_deltas, but let's defer that until someone can actually show that
"unlock+lock" has a measurable negative impact.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
convert_attrs contains an "if-else". In the "if", we set attr_action
twice, and the first assignment has no effect. In the "else", we do not
set it at all. Since git_check_attr always returns the same value, we'll
always end up in the "if", so there is no problem right now. But
convert_attrs is obviously trying not to rely on such an
implementation-detail of another component.
Make the initialization of attr_action after the if-else. Remove the
earlier assignments.
Suggested-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, 'git merge --continue' is mentioned but not explained.
Explain it.
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were several functions in the Subversion code that started with
"repo_". This namespace is also used by the Git struct repository code.
Rename these functions to start with "svn_repo" to avoid any future
conflicts.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The Subversion code had prototypes for several functions which were not
ever defined or used. These functions all had names starting with
"repo_", some of which conflict with those in repository.h. To avoid
the conflict, remove those unused prototypes.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Saying "the this" is an obvious typo. But while we're here,
let's polish the English on the second half of the sentence,
too.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a file had been commited with CRLF but now .gitattributes say
"* text=auto" (or core.autocrlf is true), the following does not
roundtrip, `git apply` fails:
printf "Added line\r\n" >>file &&
git diff >patch &&
git checkout -- . &&
git apply patch
Before applying the patch, the file from working tree is converted
into the index format (clean filter, CRLF conversion, ...). Here,
when commited with CRLF, the line endings should not be converted.
Note that `git apply --index` or `git apply --cache` doesn't call
convert_to_git() because the source material is already in index
format.
Analyze the patch if there is a) any context line with CRLF, or b)
if any line with CRLF is to be removed. In this case the patch file
`patch` has mixed line endings, for a) it looks like this:
diff --git a/one b/one
index 533790e..c30dea8 100644
--- a/one
+++ b/one
@@ -1 +1,2 @@
a\r
+b\r
And for b) it looks like this:
diff --git a/one b/one
index 533790e..485540d 100644
--- a/one
+++ b/one
@@ -1 +1 @@
-a\r
+b\r
If `git apply` detects that the patch itself has CRLF, (look at the
line " a\r" or "-a\r" above), the new flag crlf_in_old is set in
"struct patch" and two things will happen:
- read_old_data() will not convert CRLF into LF by calling
convert_to_git(..., SAFE_CRLF_KEEP_CRLF);
- The WS_CR_AT_EOL bit is set in the "white space rule",
CRLF are no longer treated as white space.
While at there, make it clear that read_old_data() in apply.c knows
what it wants convert_to_git() to do with respect to CRLF. In fact,
this codepath is about applying a patch to a file in the filesystem,
which may not exist in the index, or may exist but may not match
what is recorded in the index, or in the extreme case, we may not
even be in a Git repository. If convert_to_git() peeked at the
index while doing its work, it *would* be a bug.
Pass NULL instead of &the_index to convert_to_git() to make sure we
catch future bugs to clarify this.
Update the test in t4124: split one test case into 3:
- Detect the " a\r" line in the patch
- Detect the "-a\r" line in the patch
- Use LF in repo and CLRF in the worktree.
Reported-by: Anthony Sottile <asottile@umich.edu>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
53b2c823f6 (revision walker: mini clean-up) added the function in 2007,
but it was never used, so we should be able to get rid of it now.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reject directories with the attribute export-ignore already while
queuing them. This prevents read_tree_recursive() from descending into
them and this avoids write_archive_entry() rejecting them later on,
which queue_or_write_archive_entry() is not prepared for.
Borrow the existing strbuf to build the full path to avoid string
copies and extra allocations; just make sure we restore the original
value before moving on.
Keep checking any other attributes in write_archive_entry() as before,
but avoid checking them twice.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add helpers for accessing attributes that encapsulate the details of how
to retrieve their values.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Demonstrate mishandling of the attribute export-ignore by git archive
when used together with pathspecs. Wildcard pathspecs can even cause it
to abort. And a directory excluded without a wildcard is still included
as an empty folder in the archive.
Test-case-by: David Adam <zanchey@ucc.gu.uwa.edu.au>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`*` in format strings means peeling of tag objects so that object field
names refer to the object that the tag object points at, instead of the
tag object itself.
Currently, this is documented using grammar that is clearly inspired by
classical latin, though missing more than an article in order to be
classical english.
Try and straighten that explanation out a bit.
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Various commands list refs and allow to use a format string for the
output that interpolates from the ref as well as the object it points
at (for-each-ref; branch and tag in list mode).
Currently, the documentation talks about interpolating from the object.
This is confusing because a ref points to an object but not vice versa,
so the object cannot possible know %(refname), for example. Thus, this is
wrong independent of refs being objects (one day, maybe) or not.
Change the wording to make this clearer (and distinguish it from formats
for the log family).
Signed-off-by: Michael J Gruber <git@grubix.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The handling of `status_only` no longer interferes with the handling of
`unmatch_name_only`. `--quiet` no longer affects the exit code when using
`-L`/`--files-without-match`.
Signed-off-by: Anthony Sottile <asottile@umich.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This could have been part of 48308681b0 (git submodule update: have a
dedicated helper for cloning, 2016-02-29).
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When convert_to_git() is called, the caller may want to keep CRLF to
be kept as CRLF (and not converted into LF).
This will be used in the next commit, when apply works with files
that have CRLF and patches are applied onto these files.
Add the new value "SAFE_CRLF_KEEP_CRLF" to safe_crlf.
Prepare convert_to_git() to be able to run the clean filter, skip
the CRLF conversion and run the ident filter.
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In handshake_capabilities() we use warning() when a capability
is not supported, so the exit code of the function is 0 and no
further error is shown. This is a problem because the warning
message doesn't tell us which subprocess cmd failed.
On the contrary if we cannot write a packet from this function,
we use error() and then subprocess_start() outputs:
initialization for subprocess '<cmd>' failed
so we can know which subprocess cmd failed.
Let's improve the warning() message, so that we can know which
subprocess cmd failed.
Helped-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read_info_alternates is not used from outside, so let's make it static.
We have to declare the function before link_alt_odb_entry instead of
moving the code around, link_alt_odb_entry calls read_info_alternates,
which in turn calls link_alt_odb_entry.
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
sum(1) is a command for calculating checksums of the contents of files.
It was part of early editions of Unix ("Research Unix", 1972/1973, [1]).
cksum(1) appeared in 4.4BSD (1993) as a replacement [2], and became part
of POSIX.1-2008 [3]. OpenBSD 5.6 (2014) removed sum(1).
We only use sum(1) in t1002 to check for changes in three files. On
MinGW we use md5sum(1) instead. We could switch to the standard command
cksum(1) for all platforms; MinGW comes with GNU coreutils now, which
provides sum(1), cksum(1) and md5sum(1). Use our standard method for
checking for file changes instead: test_cmp.
It's more convenient because it shows differences nicely, it's faster on
MinGW because we have a special implementation there based only on
shell-internal commands, it's simpler as it allows us to avoid stripping
out unnecessary entries from the checksum file using grep(1), and it's
more consistent with the rest of the test suite.
We already compare changed files with their expected new contents using
diff(1), so we don't need to check with "test_must_fail test_cmp" if
they differ from their original state. A later patch could convert the
direct diff(1) calls to test_cmp as well.
With all sum(1) calls gone, remove the MinGW-specific implementation
from test-lib.sh as well.
[1] http://minnie.tuhs.org/cgi-bin/utree.pl?file=V3/man/man1/sum.1
[2] http://minnie.tuhs.org/cgi-bin/utree.pl?file=4.4BSD/usr/share/man/cat1/cksum.0
[3] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/cksum.html
Signed-off-by: René Scharfe <l.s.r@web.de>
Reviewed-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`git config --bool xxx.yyy` returns `true` for `[xxx]yyy` but
`false` for `[xxx]yyy=` or `[xxx]yyy=""`. This is tested in
t1300-repo-config.sh since 09bc098c2.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `no_changes` function calls the `untracked_files` function through
command substitution. `untracked_files` will return null bytes because it
runs ls-files with the '-z' option.
Bash since version 4.4 warns about these null bytes. As they are not
required for the test that is being done, make sure `untracked_files`
does not output null bytes when not required.
This is achieved by adding a parameter to the `untracked_files` function to
specify wither `-z` should be passed to ls-files or not.
This warning is triggered when running git stash save -u resulting in
two warnings:
git-stash: line 43: warning: command substitution: ignored null byte
in input
Signed-off-by: Kevin Daudt <me@ikke.info>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When unpack_entry() encounters a broken packed object, it returns early.
It adjusts the reference count of the pack window, but leaks the buffer
for a big delta stack in case the small automatic one was not enough.
Jump to the cleanup code at end instead, which takes care of that.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the code for releasing tree buffers and commit buffers in
fsck_obj() to the end of the function and make sure it's executed no
matter of an error is encountered or not.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
getdelim(3) returns -1 at the end of the file and if it encounters an
error, but sets errno only in the latter case. Set errno to zero before
calling it to avoid misdiagnosing an out-of-memory condition due to a
left-over value from some other function call.
Reported-by: Yaroslav Halchenko <yoh@onerussian.com>
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If realloc() fails then the original buffer is still valid. Free it
before exiting the function.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Get rid of a magic string length constant by using skip_prefix() instead
of starts_with().
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Long ago in 628522ec14 (sha1-lookup: more memory efficient
search in sorted list of SHA-1, 2007-12-29) we added
sha1_entry_pos(), a binary search that uses the uniform
distribution of sha1s to scale the selection of mid-points.
As this was a performance experiment, we tied it to the
GIT_USE_LOOKUP environment variable and never enabled it by
default.
This code was successful in reducing the number of steps in
each search. But the overhead of the scaling ends up making
it slower when the cache is warm. Here are best-of-five
timings for running rev-list on linux.git, which will have
to look up every object:
$ time git rev-list --objects --all >/dev/null
real 0m35.357s
user 0m35.016s
sys 0m0.340s
$ time GIT_USE_LOOKUP=1 git rev-list --objects --all >/dev/null
real 0m37.364s
user 0m37.045s
sys 0m0.316s
The USE_LOOKUP version might have more benefit on a cold
cache, as the time to fault in each page would dominate. But
that would be for a single lookup. In practice, most
operations tend to look up many objects, and the whole pack
.idx will end up warm.
It's possible that the code could be better optimized to
compete with a naive binary search for the warm-cache case,
and we could have the best of both worlds. But over the
years nobody has done so, and this is largely dead code that
is rarely run outside of the test suite. Let's drop it in
the name of simplicity.
This lets us remove sha1_entry_pos() entirely, as the .idx
lookup code was the only caller. Note that sha1-lookup.c
still contains sha1_pos(), which differs from
sha1_entry_pos() in two ways:
- it has a different interface; it uses a function pointer
to access sha1 entries rather than a size/offset pair
describing the table's memory layout
- it only scales the initial selection of "mi", rather
than each iteration of the search
We can't get rid of this function, as it's called from
several places. It may be that we could replace it with a
simple binary search, but that's out of scope for this patch
(and would need benchmarking).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 1a812f3a70 (hashcmp(): inline memcmp() by hand to
optimize, 2011-04-28), it was reported that an open-coded
loop outperformed memcmp() for comparing sha1s.
Discussion[1] a few years later in 2013 showed that this
depends on your libc's version of memcmp(). In particular,
glibc 2.13 optimized their memcmp around 2011. Here are
current timings with glibc 2.24 (best-of-five, on
linux.git):
[before this patch, open-coded]
$ time git rev-list --objects --all
real 0m35.357s
user 0m35.016s
sys 0m0.340s
[after this patch, memcmp]
real 0m32.930s
user 0m32.630s
sys 0m0.300s
Now that we've had 6 years for that version of glibc to
make its way onto people's machines, it's worth revisiting
our benchmarks and switching to memcmp().
It may be that there are other non-glibc systems where
memcmp() isn't as well optimized. But since our single data
point in favor of open-coding was on a now-ancient glibc, we
should probably assume the system memcmp is good unless
proven otherwise. We may end up with a SLOW_MEMCMP Makefile
knob, but we can hold off on that until we actually find
such a system in practice.
[1] https://public-inbox.org/git/20130318073229.GA5551@sigill.intra.peff.net/
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use a NULL-and-NUL check to see if we have a prefix and consistently use
C string functions on it instead of storing its length in a member of
struct apply_state. This avoids strlen() calls and simplifies the code.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In addition to adding the missing newline, add the x-ecutable bit
'mode change' character to the error message. This message now has
the same form as similar messages output by 'update-index'.
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
find_pack_entry_one() uses the fan-out table of pack indexes to find out
which entries match the first byte of the searched hash and does a
binary search on this subset of the main index table.
If there are no matching entries then lo and hi will have the same
value. The binary search still starts and compares the hash of the
following entry (which has a non-matching first byte, so won't cause any
trouble), or whatever comes after the sorted list of entries.
The probability of that stray comparison matching by mistake is low, but
let's not take any chances and check when entering the binary search
loop if we're actually done already.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
OpenBSD's regex library has a repetition limit (RE_DUP_MAX) of 255.
That's the minimum acceptable value according to POSIX. In t4062 we use
4096 repetitions in the test "-G matches", though, causing it to fail.
Combine two repetition operators, both less than 256, to arrive at 4096
zeros instead of using a single one, to fix the test on OpenBSD.
Original-patch-by: David Coppa <dcoppa@openbsd.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
76e368c378 (t3700: fix broken test under !SANITY) explains that the test
'git add --chmod=[+-]x changes index with already added file' can fail
if xfoo3 is still present as a symlink from a previous test and deletes
it with rm(1). That still leaves it present in the index, which causes
the test to fail if POSIXPERM is not defined. Get rid of it by calling
"git reset --hard" as well, as 76e368c378 already mentioned in passing.
Helped-by: Adam Dinwoodie <adam@dinwoodie.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If there was no 'Signed-off-by:' trailer but another trailer such as
'Reported-by:' then 'git am --signoff' would add a blank line between
the existing trailers and the added 'Signed-off-by:' line. e.g.
Rebase accepts '--rerere-autoupdate' as an option but only honors
it if '-m' is also given. Fix it for a non-interactive rebase by
passing on the option to 'git am' and 'git cherry-pick'.
Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Fix by using the code provided for this purpose in sequencer.c.
Change the tests so that they check the formatting of the
'Signed-off-by:' lines rather than just grepping for them.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In parse_svn_date() prepend the correct UTC offset to the timestamp
returned. This is the offset in effect at the commit time instead of
the offset in effect at calling time.
Signed-off-by: Urs Thuermann <urs@isnogud.escape.de>
Reviewed-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We want to make sure that people who copy & paste code would see
fewer instances of "git-foo". The use of these dashed forms have
been discouraged since v1.6.0 days.
Signed-off-by: Michael Forney <mforney@mforney.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The parameter to basename(3) and dirname(3) traditionally had the type
"char *", but on OpenBSD it's been "const char *" for years. That
causes (at least) Clang to throw an incompatible-pointer-types warning
for test-path-utils, where we try to pass around pointers to these
functions.
Avoid this warning (which is fatal in DEVELOPER mode) by ignoring the
promise of OpenBSD's implementations to keep input strings unmodified
and enclosing them in POSIX-compatible wrappers.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The sub-test "init in long base path" in t0001 checks the ability to
handle long base paths with restrictive permissions (--x). On OpenBSD
getcwd(3) fails in that case even for short paths. Check the two
aspects separately by trying to use a long base path both with and
without execute-only permissions. Only attempt the former if we know
that getcwd(3) doesn't care.
Original-patch-by: David Coppa <dcoppa@openbsd.org>
Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change an argument to test_line_count (which'll ultimately be turned
into a "test" expression) to use "-gt" instead of ">" for an
arithmetic test.
This broken on e.g. OpenBSD as of v2.13.0 with my commit
ac3f5a3468 ("ref-filter: add --no-contains option to
tag/branch/for-each-ref", 2017-03-24).
Downstream just worked around it by patching git and didn't tell us
about it, I discovered this when reading various Git packaging
implementations: https://github.com/openbsd/ports/commit/7e48bf88a20
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach 'git clone --recurse-submodules' to respect the '-q' option by
passing down the quiet flag to the process which handles cloning of
submodules.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running, e.g., `git -c alias.bar=foo bar`, we expand the alias and
execute `git-foo` as a dashed external. This is true even if git foo is
a builtin. That is on purpose, and is motivated in a comment which was
added in commit 441981bc ("git: simplify environment save/restore
logic", 2016-01-26).
Shortly before we launch a dashed external, and unless we have already
found out whether we should use a pager, we check `pager.foo`. This was
added in commit 92058e4d ("support pager.* for external commands",
2011-08-18). If the dashed external is a builtin, this does not match
that commit's intention and is arguably wrong, since it would be cleaner
if we let the "dashed external builtin" handle `pager.foo`.
This has not mattered in practice, but a recent patch taught `git-tag`
to ignore `pager.tag` under certain circumstances. But, when started
using an alias, it doesn't get the chance to do so, as outlined above.
That recent patch added a test to document this breakage.
Do not check `pager.foo` before launching a builtin as a dashed
external, i.e., if we recognize the name of the external as a builtin.
Change the test to use `test_expect_success`.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous patch taught `git tag` to only respect `pager.tag` in
list-mode. That patch left the default value of `pager.tag` at "off".
After that patch, it makes sense to let the default value be "on"
instead, since it will help with listing many tags, but will not hurt
users of `git tag -a` as it would have before. Make that change. Update
documentation and tests.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using, e.g., `git -c pager.tag tag -a new-tag` results in errors such as
"Vim: Warning: Output is not to a terminal" and a garbled terminal.
Someone who makes use of both `git tag -a` and `git tag -l` will
probably not set `pager.tag`, so that `git tag -a` will actually work,
at the cost of not paging output of `git tag -l`.
Use the mechanisms introduced in two earlier patches to ignore
`pager.tag` in git.c and let the `git tag` builtin handle it on its own.
Only respect `pager.tag` when running in list-mode.
There is a window between where the pager is started before and after
this patch. This means that early errors can behave slightly different
before and after this patch. Since operation-parsing has to happen
inside this window, this can be seen with `git -c pager.tag="echo pager
is used" tag -l --unknown-option`. This change in paging-behavior should
be acceptable since it only affects erroneous usages.
Update the documentation and update tests.
If an alias is used to run `git tag -a`, then `pager.tag` will still be
respected. Document this known breakage. It will be fixed in a later
commit. Add a similar test for `-l`, which works.
Noticed-by: Anatoly Borodin <anatoly.borodin@gmail.com>
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using, e.g., `git -c pager.tag tag -a new-tag` results in errors such as
"Vim: Warning: Output is not to a terminal" and a garbled terminal.
Someone who makes use of both `git tag -a` and `git tag -l` will
probably not set `pager.tag`, so that `git tag -a` will actually work,
at the cost of not paging output of `git tag -l`.
Since we're about to change how `git tag` respects `pager.tag`, add tests
around this, including how the configuration is ignored if --no-pager or
--paginate are used.
Construct tests with a few different subcommands. First, use -l. Second,
use "no arguments" and --contains, since those imply -l. (There are
more arguments which imply -l, but using these two should be enough.)
Third, use -a as a representative for "not -l". Actually, the tests use
`git tag -am` so no editor is launched, but that is irrelevant, since we
just want to see whether the pager is used or not. Make one of the tests
demonstrate the broken behavior mentioned above, where `git tag -a`
respects `pager.tag`.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The previous patch introduced a way for builtins to declare that they
will take responsibility for handling the `pager.foo`-config item. (See
the commit message of that patch for why that could be useful.)
Provide setup_auto_pager(), which builtins can call in order to handle
`pager.<cmd>`, including possibly starting the pager. Make this function
don't do anything if a pager has already been started, as indicated by
use_pager or pager_in_use().
Whenever this function is called from a builtin, git.c will already have
called commit_pager_choice(). Since commit_pager_choice() treats the
special value -1 as "punt" or "not yet decided", it is not a problem
that we might end up calling commit_pager_choice() once in git.c and
once (or more) in the builtin. Make the new function use -1 in the same
way and document it as "punt".
Don't add any users of setup_auto_pager just yet, one will follow in
a later patch.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before launching a builtin git foo and unless mechanisms with precedence
are in use, we check for and handle the `pager.foo` config. This is done
without considering exactly how git foo is being used, and indeed, git.c
cannot (and should not) know what the arguments to git foo are supposed
to achieve.
In practice this means that, e.g., `git -c pager.tag tag -a new-tag`
results in errors such as "Vim: Warning: Output is not to a terminal"
and a garbled terminal. Someone who makes use of both `git tag -a` and
`git tag -l` will probably not set `pager.tag`, so that `git tag -a`
will actually work, at the cost of not paging output of `git tag -l`.
To allow individual builtins to make more informed decisions about when
to respect `pager.foo`, introduce a flag DELAY_PAGER_CONFIG. If the flag
is set, do not check `pager.foo`.
Do not check for DELAY_PAGER_CONFIG in `execv_dashed_external()`. That
call site is arguably wrong, although in a way that is not yet visible,
and will be changed in a slightly different direction in a later patch.
Don't add any users of DELAY_PAGER_CONFIG just yet, one will follow in a
later patch.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Delete Documentation/technical/api-builtin.txt and move its content
into builtin.h. Format it as a comment. Remove a '+' which was needed
when the information was formatted for AsciiDoc. Similarly, change
"::" to ":".
Document SUPPORT_SUPER_PREFIX, thereby bringing the documentation up to
date with the available flags.
While at it, correct '3 more things to do' to '4 more things to do'.
Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If revs->def is set (as it is in "git log") and there are no
pending objects after parsing the user's input, then we show
whatever is in "def". But if the user _did_ ask for some
input that just happened to be empty (e.g., "--glob" that
does not match anything), showing the default revision is
confusing. We should just show nothing, as that is what the
user's request yielded.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the user gives us no starting point for a traversal, we
want to complain with our normal usage message. But if they
tried to do so with "--all" or "--glob", but that happened
not to match any refs, the usage message isn't helpful. We
should just give them the empty output they asked for
instead.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Normally a caller that invokes setup_revisions() has to
check rev.pending to see if anything was actually queued for
the traversal. But they can't tell the difference between
two cases:
1. The user gave us no tip from which to start a
traversal.
2. The user tried to give us tips via --glob, --all, etc,
but their patterns ended up being empty.
Let's set a flag in the rev_info struct that callers can use
to tell the difference. We can set this from the
init_all_refs_cb() function. That's a little funny because
it's not exactly about initializing the "cb" struct itself.
But that function is the common setup place for doing
pattern traversals that is used by --glob, --all, etc.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 751a2ac6e (rev-list --exclude: tests, 2013-11-01), we
added a few tests for handling "empty" inputs with rev-list
(i.e., where the user gave us some pattern but it turned out
not to queue any objects for traversal), all of which were
marked as failing.
In preparation for working on this area of the code, let's
give each test a more descriptive name. Let's also include
one more case which we should cover: feeding a --glob
pattern that doesn't match anything.
We can also drop the explanatory comment; we'll be
converting these to expect_success in the next few patches,
so the discussion isn't necessary.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cherry-pick and revert should not accept --[no-]rerere-autoupdate once
they have started.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When continuing after conflicts, cherry-pick forgot if the user had specified
'--rerere-autoupdate'.
Redo the cherry-pick rerere tests to check --rerere-autoupdate works
as expected.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Using test_commit is simpler than chaining echo && git add &&
test_tick && commit. Also having tags makes it clearer which commit
is being selecting by reset.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Interactive rebase was ignoring '--rerere-autoupdate'. Fix this by
reading it appropriate file when restoring the sequencer state for an
interactive rebase and passing '--rerere-autoupdate' to merge and
cherry-pick when rebasing with '--preserve-merges'.
Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rebase accepts '--rerere-autoupdate' as an option but only honors it
if '-m' is also given. Fix it for a non-interactive rebase by passing
on the option to 'git am' and 'git cherry-pick'. Rework the tests so
that they can be used for each rebase flavor and extend them.
Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Save the rerere-autoupdate setting so that it is remembered after
stopping for the user to resolve conflicts.
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1450-fsck.sh does not have a test that checks fsck's behavior when a
packfile is invalid. It does have a test for when an object in a
packfile is invalid, but in that test, the packfile itself is valid.
Add such a test.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The patch is read from standard input and not from a parameter.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit 612c49e94d ("credential-cache: add tests for XDG
functionality", 17-03-2017), the cygwin build has been failing all the
new tests added by that commit. In particular, the 'git credential-cache
exit' command, as part of the test cleanup code, has been die-ing with
the message:
fatal: read error from cache daemon: Connection reset by peer
As this git command is part of an && chain in a 'test_when_finished'
call, the remaining test cleanup is not happening, so practically all
remaining tests fail due to the unexpected presence of various socket
files and directories.
A simple means of getting the tests to pass, is to simply ignore the
failure of 'git credential-cache exit' command and make sure all test
cleanup is done. For example, the diff for test #12 would look like:
diff --git a/t/t0301-credential-cache.sh b/t/t0301-credential-cache.sh
index fd92533ac..87e5001bb 100755
--- a/t/t0301-credential-cache.sh
+++ b/t/t0301-credential-cache.sh
@@ -17,7 +17,7 @@ helper_test cache
test_expect_success 'socket defaults to ~/.cache/git/credential/socket' '
test_when_finished "
- git credential-cache exit &&
+ (git credential-cache exit || :) &&
rmdir -p .cache/git/credential/
" &&
test_path_is_missing "$HOME/.git-credential-cache" &&
... and so on for all remaining tests. While this does indeed make all
tests pass, it is not really a solution.
As an aside, while looking to debug this issue, I added the '--debug'
option to the invocation of the 'git-credential-cache--daemon' child
process (see the spawn_daemon() function). This not only fixed the tests,
but also stopped git-credential-cache exiting with a failure. Since the
only effect of passing '--debug' was to suppress the redirection of stderr
to the bit-bucket (/dev/null), I have no idea why this seems to fix the
protocol interaction between git and git-credential-cache--daemon. (I
did think that maybe it was a timing issue, so I tried sleeping before
reading from the daemon on Linux, but that only slowed down the tests!)
All descriptions of the "Connection reset by peer" error, that I could
find, say that the peer had destroyed the connection before the client
attempted to perform I/O on the connection. Since the daemon does not
respond to an "exit" message from the client, it just closes the socket
and deletes the socket file (via the atexit handler), it seems that the
expected result is for the client to receive an EOF. Indeed, this is
exactly what seems to be happening on Linux. Also a comment in
credential-cache--daemon.c reads:
else if (!strcmp(action.buf, "exit")) {
/*
* It's important that we clean up our socket first, and then
* signal the client only once we have finished the cleanup.
* Calling exit() directly does this, because we clean up in
* our atexit() handler, and then signal the client when our
* process actually ends, which closes the socket and gives
* them EOF.
*/
exit(0);
}
On cygwin this is not the case, at least when not passing --debug to the
daemon, and the read following the "exit" gets an error with errno set
to ECONNRESET.
In order to suppress the fatal exit in this case, check the read error
for an ECONNRESET and return as if no data was read from the daemon.
This effectively converts an ECONNRESET into an EOF.
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove hard coded sha1 values, obtain the values using
'git rev-parse HEAD' which should be future proof regardless
of the hash function used.
Additionally future-proof the test by hard coding the
abbreviation length of the hash.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor, into a common function, the version and capability negotiation
done when invoking a long-running process as a clean or smudge filter.
This will be useful for other Git code that needs to interact similarly
with a long-running process.
As you can see in the change to t0021, this commit changes the error
message reported when the long-running process does not introduce itself
with the expected "server"-terminated line. Originally, the error
message reports that the filter "does not support filter protocol
version 2", differentiating between the old single-file filter protocol
and the new multi-file filter protocol - I have updated it to something
more generic and useful.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the documentation for the sub-process API from a separate txt file
to its header file.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ls/filter-process-delayed:
convert: add "status=delayed" to filter process protocol
convert: refactor capabilities negotiation
convert: move multiple file filter error handling to separate function
convert: put the flags field before the flag itself for consistent style
t0021: write "OUT <size>" only on success
t0021: make debug log file name configurable
t0021: keep filter log files on comparison
Remove the unused variable "heads" from cmd_fsck().
This variable was made unused in commit c3271a0 ("fsck: do not fallback
"git fsck <bogus>" to "git fsck"", 2017-01-17).
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The values have eluded documentation so far. While at it streamline
the wording by grouping relevant parts together.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'js/msgfmt-on-windows' of ../git-gui:
git-gui (MinGW): make use of MSys2's msgfmt
git gui: allow for a long recentrepo list
git gui: de-dup selected repo from recentrepo history
git gui: cope with duplicates in _get_recentrepo
git-gui: remove duplicate entries from .gitconfig's gui.recentrepo
When Git for Windows was still based on MSys1, we had no gettext, ergo
no msgfmt, either. Therefore, we introduced a small and simple Tcl
script to perform the same task.
However, with MSys2, we no longer need that because we have a proper
msgfmt executable. Plus, the po2msg.sh script somehow manages to hang
when run in parallel in Git for Windows' SDK (symptom: the Continuous
Testing tasks timing out).
Two reasons to use real msgfmt.exe instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Align argument list and place opening brace on its own line.
Signed-off-by: Dimitrios Christidis <dimitrios@christidis.me>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is shorter, makes the logic a bit easier to follow, and is
perhaps a bit faster too.
The logic is to make the final decision only when "subject" is there,
its early part matches "match", and the match is at the slash
boundary (or the whole thing).
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When attempting to blame a non-existing path, git should show an error
message like this:
$ git blame e83c51633 -- nonexisting-file
fatal: no such path nonexisting-file in e83c51633
Since the recent commit 835c49f7d (blame: rework methods that
determine 'final' commit, 2017-05-24) the revision name is either
missing or some scrambled characters are shown instead. The reason is
that the revision name must be duplicated, because it is invalidated
when the pending objects array is cleared in the meantime, but this
commit dropped the duplication.
Restore the duplication of the revision name in the affected functions
(find_single_final() and find_single_initial()).
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It turns out that Cygwin also needs the fopen() wrapper that
returns failure when a directory is opened for reading.
* rj/cygwin-fread-reads-directories:
config.mak.uname: set FREAD_READS_DIRECTORIES for cygwin
We started using "%" PRItime, imitating "%" PRIuMAX and friends, as
a way to format the internal timestamp value, but this does not
play well with gettext(1) i18n framework, and causes "make pot"
that is run by the l10n coordinator to create a broken po/git.pot
file. This is a possible workaround for that problem.
* jc/po-pritime-fix:
Makefile: help gettext tools to cope with our custom PRItime format
By declaring the task_cb parameter of type `void **`, the signature of
the get_next_task method suggests that the "task-specific cookie" can be
defined in that method, and the signatures of the start_failure and of
the task_finished methods declare that parameter of type `void *`,
suggesting that those methods are mere users of said cookie.
That convention makes a total lot of sense, because the tasks are pretty
much dead when one of the latter two methods is called: there would be
little use to reset that cookie at that point because nobody would be
able to see the change afterwards.
However, this is not what the code actually does. For all three methods,
it passes the *address* of pp->children[i].data.
As reasoned above, this behavior makes no sense. So let's change the
implementation to adhere to the convention suggested by the signatures.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use "p4 -G" to make "p4 changes" output more Python-friendly
to parse.
* mt/p4-parse-G-output:
git-p4: filter for {'code':'info'} in p4CmdList
git-p4: parse marshal output "p4 -G" in p4 changes
git-p4: git-p4 tests with p4 triggers
A recent update made it easier to use "-fsanitize=" option while
compiling but supported only one sanitize option. Allow more than
one to be combined, joined with a comma, like "make SANITIZE=foo,bar".
* jk/build-with-asan:
Makefile: allow combining UBSan with other sanitizers
A recent update broke an alias that contained an uppercase letter.
* js/alias-case-sensitivity:
alias: compare alias name *case-insensitively*
t1300: demonstrate that CamelCased aliases regressed
When running gpg-relevant tests, a gpg-daemon is spawned for each
GNUPGHOME used. This daemon may stay running after the test and cache
file descriptors for the trash directories, even after the trash
directory is removed. This leads to ENOENT errors when attempting to
create files if tests are run multiple times.
Add a cleanup script to force flushing the gpg-agent for that GNUPGHOME
(if any) before setting up the GPG relevant-environment.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Santiago Torres <santiago@nyu.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 06bf4ad1d (push: propagate remote and refspec with
--recurse-submodules) push was taught how to propagate a refspec down to
submodules when the '--recurse-submodules' flag is given. The only refspecs
that are allowed to be propagated are ones which name a ref which exists
in both the superproject and the submodule, with the caveat that 'HEAD'
was disallowed.
This patch teaches push-check (the submodule helper which determines if
a refspec can be propagated to a submodule) to permit propagating 'HEAD'
if and only if the superproject and the submodule both have the same
named branch checked out and the submodule is not in a detached head
state.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "used" field in struct object is only used by builtin/fsck. Remove
that field and modify builtin/fsck to use a flag instead.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If obj->type == OBJ_TREE, an invocation of fsck_walk() will invoke
parse_tree() and return quickly if that returns nonzero, so it is of no
use for traverse_one_object() to invoke parse_tree() in this situation
before invoking fsck_walk(). Remove that code.
The behavior of traverse_one_object() is changed slightly in that it now
returns -1 instead of 1 in the case that parse_tree() fails, but this is
not an issue because its only caller (traverse_reachable) does not care
about the value as long as it is nonzero.
This code was introduced in commit 271b8d2 ("builtin-fsck: move away
from object-refs to fsck_walk", 2008-02-25). The same issue existed in
that commit.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If done_pbase_paths is NULL then done_pbase_paths_num must be zero and
done_pbase_path_pos() returns -1 without accessing the array, so the
check is not necessary.
If the invariant was violated then the check would make sure we keep
on going and allocate the necessary amount of memory in the next
ALLOC_GROW call. That sounds nice, but all array entries except for
one would contain garbage data.
If the invariant was violated without the check we'd get a segfault in
done_pbase_path_pos(), i.e. an observable crash, alerting us of the
presence of a bug.
Currently there is no such bug: Only the functions check_pbase_path()
and cleanup_preferred_base() change pointer and counter, and both make
sure to keep them in sync. Get rid of the check anyway to allow us to
see if later changes introduce such a defect, and to simplify the code.
Detected by Coverity Scan.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Back when the modern http_options() codepath was created to parse
various http.* options at 29508e1e ("Isolate shared HTTP request
functionality", 2005-11-18), and then later was corrected for
interation between the multiple configuration files in 7059cd99
("http_init(): Fix config file parsing", 2009-03-09), we parsed
configuration variables like http.sslkey, http.sslcert as plain
vanilla strings, because git_config_pathname() that understands
"~[username]/" prefix did not exist. Later, we converted some of
them (namely, http.sslCAPath and http.sslCAInfo) to use the
function, and added variables like http.cookeyFile http.pinnedpubkey
to use the function from the beginning. Because of that, these
variables all understand "~[username]/" prefix.
Make the remaining two variables, http.sslcert and http.sslkey, also
aware of the convention, as they are both clearly pathnames to
files.
Noticed-by: Victor Toni <victor.toni@gmail.com>
Helped-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To note that merely cloning git.git without --recurse-submodules
doesn't get you a full copy of the code anymore. See
5f6482d642 ("RelNotes: mention "log: make --regexp-ignore-case work
with --perl-regexp"", 2017-07-20).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To inform users that they can use --regexp-ignore-case now, and that
existing scripts which relied on that + PCRE may be buggy. See
9e3cbc59d5 ("log: make --regexp-ignore-case work with --perl-regexp",
2017-05-20).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To inform users that they can use the short form now. See
7531a2dd87 ("log: add -P as a synonym for --perl-regexp", 2017-05-25).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We started using our own timestamp_t type and PRItime format
specifier to go along with it, so that we can later change the
underlying type and output format more easily, but this does not
play well with gettext tools.
Because gettext tools need to keep the *.po file portable across
platforms, they have to special-case the format specifiers like
PRIuMAX that are known types in inttypes.h, instead of letting CPP
handle strings like
"%" PRIuMAX " seconds ago"
as an ordinary string concatenation. They fundamentally cannot do
the same for our own custom type/format.
Given that po/git.pot needs to be generated only once every release
and by only one person, i.e. the l10n coordinator, let's update the
Makefile rule to generate po/git.pot so that gettext tools are run
on a munged set of sources in which all mentions of PRItime are
replaced with PRIuMAX, which is what we happen to use right now.
This way, developers do not have to care that PRItime does not play
well with gettext, and translators do not have to care that we use
our own PRItime.
The credit for the idea to munge the source files goes to Dscho.
Possible bugs are mine.
Helped-by: Jiang Xin <worldhello.net@gmail.com>
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We run an early part of "git gc" that deals with refs before
daemonising (and not under lock) even when running a background
auto-gc, which caused multiple gc processes attempting to run the
early part at the same time. This is now prevented by running the
early part also under the GC lock.
* jk/gc-pre-detach-under-hook:
gc: run pre-detach operations under lock
Code clean-up, that makes us in sync with Debian by one patch.
* jn/hooks-pre-rebase-sample-fix:
pre-rebase hook: capture documentation in a <<here document
The progress meter did not give a useful output when we haven't had
0.5 seconds to measure the throughput during the interval. Instead
show the overall throughput rate at the end, which is a much more
useful number.
* rs/progress-overall-throughput-at-the-end:
progress: show overall rate in last update
On Cygwin, similar to Windows, "git push //server/share/repository"
ought to mean a repository on a network share that can be accessed
locally, but this did not work correctly due to stripping the double
slashes at the beginning.
This may need to be heavily tested before it gets unleashed to the
wild, as the change is at a fairly low-level code and would affect
not just the code to decide if the push destination is local. There
may be unexpected fallouts in the path normalization.
* tb/push-to-cygwin-unc-path:
cygwin: allow pushing to UNC paths
gitweb terminates and shows no project list, if it can not access a
sub-directory in the project root directory while looking for projects
to show.
Work it around by skipping unreadable directories.
Signed-off-by: Hielke Christian Braun <hcb@unco.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The check for whether the template given to 'git commit' is untouched
is done before the empty message check. This results in a wrong error
message being displayed in the following case. When the user removes
everything in template completely to abort the commit he is shown the
"template untouched" error which is wrong. He should be shown the
"empty message" error.
Do the empty message check before checking for an untouched template
thus fixing this issue.
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The untracked cache is stored on disk by concatenating its memory
structures without any padding. Consequently some of the structs are
not aligned at a particular boundary when the whole extension is read
back in one go. That's only OK on platforms without strict alignment
requirements, or for byte-aligned data like strings or hash values.
Decode struct ondisk_untracked_cache carefully from the extension
blob by using explicit pointer arithmetic with offsets, avoiding
alignment issues. Use char pointers for passing stat_data objects to
stat_data_from_disk(), and use memcpy(3) in that function to get the
contents into a properly aligned struct, then perform the byte-order
adjustment in place there.
Found with Clang's UBSan.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Exit early when asked to prune an index that contains no entries to
begin with. This avoids pointer arithmetic on istate->cache, which is
possibly NULL in that case.
Found with Clang's UBSan.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simplify the code by using the helper macros COPY_ARRAY and MOVE_ARRAY,
which also makes them more robust in the case we copy or move no lines,
as they allow using NULL points in that case, while memcpy(3) and
memmove(3) don't.
Found with Clang's UBSan.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simplify the code for moving members inside of an array and make it more
robust by using the helper macro MOVE_ARRAY. It calculates the size
based on the specified number of elements for us and supports NULL
pointers when that number is zero. Raw memmove(3) calls with NULL can
cause the compiler to (over-eagerly) optimize out later NULL checks.
This patch was generated with contrib/coccinelle/array.cocci and spatch
(Coccinelle).
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Similar to COPY_ARRAY (introduced in 60566cbb58), add a safe and
convenient helper for moving potentially overlapping ranges of array
entries. It infers the element size, multiplies automatically and
safely to get the size in bytes, does a basic type safety check by
comparing element sizes and unlike memmove(3) it supports NULL
pointers iff 0 elements are to be moved.
Also add a semantic patch to demonstrate the helper's intended usage.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simplify the implementation and allow callers to use expressions with
side-effects by turning the macros get_be16, get_be32 and put_be32 into
inline functions.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pointer p is dereferenced and we get an unsigned char. Before
shifting it's automatically promoted to int. Left-shifting a signed
32-bit value bigger than 127 by 24 places is undefined. Explicitly
convert to a 32-bit unsigned type to avoid undefined behaviour if
the highest bit is set.
Found with Clang's UBSan.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Multiple sanitizers can be specified as a comma-separated list. Set
the flag NO_UNALIGNED_LOADS even if UndefinedBehaviorSanitizer is not
the only sanitizer to build with.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The build procedure has been improved to allow building and testing
Git with address sanitizer more easily.
* jk/build-with-asan:
Makefile: disable unaligned loads with UBSan
Makefile: turn off -fomit-frame-pointer with sanitizers
Makefile: add helper for compiling with -fsanitize
test-lib: turn on ASan abort_on_error by default
test-lib: set ASAN_OPTIONS variable before we run git
"git pull --rebase --recurse-submodules" learns to rebase the
branch in the submodules to an updated base.
* sb/pull-rebase-submodule:
builtin/fetch cleanup: always set default value for submodule recursing
pull: optionally rebase submodules (remote submodule changes only)
builtin/fetch: parse recurse-submodules-default at default options parsing
builtin/fetch: factor submodule recurse parsing out to submodule config
Update the hashmap API so that data to customize the behaviour of
the comparison function can be specified at the time a hashmap is
initialized.
* sb/hashmap-customize-comparison:
hashmap: migrate documentation from Documentation/technical into header
patch-ids.c: use hashmap correctly
hashmap.h: compare function has access to a data field
Code cleanup.
* ab/grep-lose-opt-regflags:
grep: remove redundant REG_NEWLINE when compiling fixed regex
grep: remove regflags from the public grep_opt API
grep: remove redundant and verbose re-assignments to 0
grep: remove redundant "fixed" field re-assignment to 0
grep: adjust a redundant grep pattern type assignment
grep: remove redundant double assignment to 0
When color placeholders like %(color:red) are used in a
ref-filter format, we unconditionally output the colors,
even if the user has asked us for no colors. This usually
isn't a problem when the user is constructing a --format on
the command line, but it means we may do the wrong thing
when the format is fed from a script or alias. For example:
$ git config alias.b 'branch --format=%(color:green)%(refname)'
$ git b --no-color
should probably omit the green color. Likewise, running:
$ git b >branches
should probably also omit the color, just as we would for
all baked-in coloring (and as we recently started to do for
user-specified colors in --pretty formats).
This commit makes both of those cases work by teaching
the ref-filter code to consult want_color() before
outputting any color. The color flag in ref_format defaults
to "-1", which means we'll consult color.ui, which in turn
defaults to the usual isatty() check on stdout. However,
callers like git-branch which support their own color config
(and command-line options) can override that.
The new tests independently cover all three of the callers
of ref-filter (for-each-ref, tag, and branch). Even though
these seem redundant, it confirms that we've correctly
plumbed through all of the necessary config to make colors
work by default.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The color placeholders have traditionally been
unconditional, showing colors even when git is not otherwise
configured to do so. This was not so bad for their original
use, which was on the command-line (and the user could
decide at that moment whether to add colors or not). But
these days we have configured formats via pretty.*, and
those should operate correctly in multiple contexts.
In 3082517 (log --format: teach %C(auto,black) to respect
color config, 2012-12-17), we gave an extended placeholder
that could be used to accomplish this. But it's rather
clunky to use, because you have to specify it individually
for each color (and their matching resets) in the format.
We shied away from just switching the default to auto,
because it is technically breaking backwards compatibility.
However, there's not really a use case for unconditional
colors. The most plausible reason you would want them is to
redirect "git log" output to a file. But there, the right
answer is --color=always, as it does the right thing both
with custom user-format colors and git-generated colors.
So let's switch to the more useful default. In the
off-chance that somebody really does find a use for
unconditional colors without wanting to enable the rest of
git's colors, we provide a new %C(always,...) to enable the
old behavior. And we can remind them of --color=always in
the documentation.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When rev-list pretty-prints a commit, it creates a new
pretty_print_context and copies items from the rev_info
struct. We don't currently copy the "use_color" field,
though. Nobody seems to have noticed because the only part
of pretty.c that cares is the %C(auto,...) placeholder, and
presumably not many people use that with the rev-list
plumbing (as opposed to with git-log).
It will become more noticeable in a future patch, though,
when we start treating all user-format colors as auto-colors
(in which case it would become impossible to format colors
with rev-list, even with --color=always).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In most commands we load config before parsing command line
options, since it lets the latter override the former with a
simple variable assignment. In the case of for-each-ref,
though, we do it in the reverse order. This is OK with
the current code, since there's no interaction between the
config and command-line options.
However, as the ref-filter code starts to care about config
during verify_ref_format(), we'll want to make sure the
config is loaded. Let's bump the config to the usual spot
near the top of the function.
We can drop the comment there; it's impossible to keep a
"why we load the config" comment like this up to date with
every config option we might be interested in. And indeed,
it's already stale; we'd care about core.abbrev, for
instance, when %(objectname:short) is used.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Back in prehistoric times, our decision on whether or not to
show color by default relied on using a config callback that
either did or didn't load color config like color.diff.
When we introduced color.ui, we put it in the same boat:
commands had to manually respect it by using git_color_config()
or its git_color_default_config() convenience wrapper.
But in 4c7f1819b (make color.ui default to 'auto',
2013-06-10), that changed. Since then, we default color.ui
to auto in all programs, meaning that even plumbing commands
like "git diff-tree --pretty" might colorize the output.
Nobody seems to have complained in the intervening years,
presumably because the "is stdout a tty" check does a good
job of catching the right cases.
But that leaves an interesting curiosity: color.ui defaults
to auto even in plumbing, but you can't actually _disable_
the color via config. So if you really hate color and set
"color.ui" to false, diff-tree will still show color (but
porcelain like git-diff won't). Nobody noticed that either,
probably because very few people disable color.
One could argue that the plumbing should _always_ disable
color unless an explicit --color option is given on the
command line. But in practice, this creates a lot of
complications for scripts which do want plumbing to show
user-visible output. They can't just pass "--color" blindly;
they need to check the user's config and decide what to
send.
Given that nobody has complained about the current behavior,
let's assume it's a good path, and follow it to its
conclusion: supporting color.ui everywhere.
Note that you can create havoc by setting color.ui=always in
your config, but that's more or less already the case. We
could disallow it entirely, but it is handy for one-offs
like:
git -c color.ui=always foo >not-a-tty
when "foo" does not take a --color option itself.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The callback for parsing each formatting atom gets to see
only the atom struct (which it's filling in) and the text to
be parsed. This doesn't leave any room for it to behave
differently based on context known only to the ref_format.
We can solve this by passing in the surrounding ref_format
to each parser. Note that this makes things slightly awkward
for sort strings, which parse atoms without having a
ref_format. We'll solve that by using a dummy ref_format
with default parameters.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We parse sort strings as single formatting atoms, and just
build on parse_ref_filter_atom(). Let's pull this idea into
its own function, since it's about to get a little more
complex. As a bonus, we can give the function a slightly
more natural interface, since our single atoms are in their
own strings.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The parse_ref_filter_atom() function really shouldn't be
exposed outside of ref-filter.c; its return value is an
integer index into an array that is private in that file.
Since the previous commit removed the sole external caller
(and replaced it with a public function at a more
appropriately level), we can just make this static.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ref-filter module currently provides a callback suitable
for parsing command-line --sort options. But since git-tag
also supports the tag.sort config option, it needs a
function whose implementation is quite similar, but with a
slightly different interface. The end result is that
builtin/tag.c has a copy-paste of parse_opt_ref_sorting().
Instead, let's provide a function to parse an arbitrary
sort string, which we can then trivially wrap to make the
parse_opt variant.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Calling verify_ref_format() doesn't just confirm that the
format is sane; it actually sets some global variables that
will be used later when formatting the refs. These logically
should belong to the ref_format, which would make it
possible to use multiple formats within a single program
invocation.
Let's move one such flag into the ref_format struct. There
are still others that would need to be moved before it would
be safe to use multiple formats, but this commit gives a
blueprint for how that should look.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ref-filter module provides routines for formatting a ref
for output. The fundamental interface for the format is a
"const char *" containing the format, and any additional
options need to be passed to each invocation of
show_ref_array_item.
Instead, let's make a ref_format struct that holds the
format, along with any associated format options. That will
make some enhancements easier in the future:
1. new formatting options can be added without disrupting
existing callers
2. some state can be carried in the struct rather than as
global variables
For now this just has the text format itself along with the
quote_style option, but we'll add more fields in future patches.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the user-format doesn't add the closing color reset, we
add one automatically. But we do so by parsing the "reset"
string. We can just use the baked-in string literal, which
is simpler.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we put literal ANSI terminal codes into our test
scripts, it makes diffs on those scripts hard to read (the
colors may be indistinguishable from diff coloring, or in
the case of a reset, may not be visible at all).
Some scripts get around this by including human-readable
names and converting to literal codes with a git-config
hack. This makes the actual code diffs look OK, but test_cmp
output suffers from the same problem.
Let's use test_decode_color instead, which turns the codes
into obvious text tags.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for the %(color) placeholder refers to the
color.branch.* config for more details. But those details
moved to their own section in b92c1a28f
(Documentation/config.txt: describe 'color' value type in
the "Values" section, 2015-03-03). Let's update our
pointer. We can steal the text from 30cfe72d3 (pretty: fix
document link for color specification, 2016-10-11), which
fixed the same problem in a different place.
While we're at it, let's give an example, which makes the
syntax much more clear than just the text.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Users of the ref-filter code must call verify_ref_format()
before formatting any refs, but most ignore its return
value. This means we may print an error on a syntactically
bogus pattern, but keep going anyway.
In most cases this results in a fatal error when we actually
try to format a ref. But if you have no refs to show at all,
then the behavior is confusing: git prints the error from
verify_ref_format(), then exits with code 0 without showing
any output. Let's instead abort immediately if we know we
have a bogus format.
We'll output the usage information if we have it handy (just
like the existing call in cmd_for_each_ref() does), and
otherwise just die().
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function p4CmdList accepts a new argument: skip_info. When set to
True it ignores any 'code':'info' entry (skip_info=False by default).
That allows us to fix some of the tests in t9831-git-p4-triggers.sh
known to be broken with verobse p4 triggers
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The option -G of p4 (python marshal output) gives more context about the
data being output. That's useful when using the command "change -o" as
we can distinguish between warning/error line and real change description.
This fixes the case where a p4 trigger for "p4 change" is set and the command git-p4 submit is run.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some p4 triggers in the server side generate some warnings when
executed. Unfortunately those messages are mixed with the output of
p4 commands. A few git-p4 commands don't expect extra messages or output
lines and may fail with verbose triggers.
New tests added are known to be broken.
Signed-off-by: Miguel Torroja <miguel.torroja@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The rewrite of "git branch --list" using for-each-ref's internals
that happened in v2.13 regressed its handling of color.branch.local;
this has been fixed.
* kn/ref-filter-branch-list:
ref-filter.c: drop return from void function
branch: set remote color in ref-filter branch immediately
branch: use BRANCH_COLOR_LOCAL in ref-filter format
branch: only perform HEAD check for local branches
Convert code that divides and rounds up to use DIV_ROUND_UP to make the
intent clearer and reduce the number of magic constants.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After "git branch --move" of the currently checked out branch, the
code to walk the reflog of HEAD via "log -g" and friends
incorrectly stopped at the reflog entry that records the renaming
of the branch.
* jk/reflog-walk-maint:
reflog-walk: include all fields when freeing complete_reflogs
reflog-walk: don't free reflogs added to cache
reflog-walk: duplicate strings in complete_reflogs list
reflog-walk: skip over double-null oid due to HEAD rename
The "collission-detecting" implementation of SHA-1 hash we borrowed
from is replaced by directly binding the upstream project as our
submodule. Glitches on minority platforms are still being worked out.
* ab/sha1dc:
sha1collisiondetection: automatically enable when submodule is populated
sha1dc: optionally use sha1collisiondetection as a submodule
Code refactoring.
* pw/unquote-path-in-git-pm:
t9700: add tests for Git::unquote_path()
Git::unquote_path(): throw an exception on bad path
Git::unquote_path(): handle '\a'
add -i: move unquote_path() to Git.pm
An old message shown in the commit log template was removed, as it
has outlived its usefulness.
* ks/commit-assuming-only-warning-removal:
commit-template: distinguish status information unconditionally
commit-template: remove outdated notice about explicit paths
The undefined behavior sanitizer complains about unaligned
loads, even if they're OK for a particular platform in
practice. It's possible that they _are_ a problem, of
course, but since it's a known tradeoff the UBSan errors are
just noise.
Let's quiet it automatically by building with
NO_UNALIGNED_LOADS when SANITIZE=undefined is in use.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ASan manual recommends disabling this optimization, as
it can make the backtraces produced by the tool harder to
follow (and since this is a test-debug build, we don't care
about squeezing out every last drop of performance).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
You can already build and test with ASan by doing:
make CFLAGS=-fsanitize=address test
but there are a few slight annoyances:
1. It's a little long to type.
2. It override your CFLAGS completely. You'd probably
still want -O2, for instance.
3. It's a good idea to also turn off "recovery", which
lets the program keep running after a problem is
detected (with the intention of finding as many bugs as
possible in a given run). Since Git's test suite should
generally run without triggering any problems, it's
better to abort immediately and fail the test when we
do find an issue.
With this patch, all of that happens automatically when you
run:
make SANITIZE=address test
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
By default, ASan will exit with code 1 when it sees an
error. This means we'll notice a problem when we expected
git to succeed, but not in a test_must_fail block.
Let's ask it to actually raise SIGABRT instead. That will
give us a signal death that test_must_fail will notice. As a
bonus, it may also leave a coredump, which can be handy for
digging into a failure.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We turn off ASan's leak detection by default in the test
suite because it's too noisy. But we don't do so until
part-way through test-lib. This is before we've run any
tests, but after we do our initial "./git" to see if the
binary has even been built.
When built with clang, this seems to work fine. However,
using "gcc -fsanitize=address", the leak checker seems to
complain more aggressively:
$ ./git
...
==5352==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 2 byte(s) in 1 object(s) allocated from:
#0 0x7f120e7afcf8 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1cf8)
#1 0x559fc2a3ce41 in do_xmalloc /home/peff/compile/git/wrapper.c:60
#2 0x559fc2a3cf1a in do_xmallocz /home/peff/compile/git/wrapper.c:100
#3 0x559fc2a3d0ad in xmallocz /home/peff/compile/git/wrapper.c:108
#4 0x559fc2a3d0ad in xmemdupz /home/peff/compile/git/wrapper.c:124
#5 0x559fc2a3d0ad in xstrndup /home/peff/compile/git/wrapper.c:130
#6 0x559fc274535a in main /home/peff/compile/git/common-main.c:39
#7 0x7f120dabd2b0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202b0)
This is a leak in the sense that we never free it, but it's
in a global that is meant to last the whole program. So it's
not really interesting or in need of fixing. And at any
rate, mentioning leaks outside of the test_expect blocks is
certainly unwelcome, as it pollutes stderr.
Let's bump the setting of ASAN_OPTIONS higher in test-lib.sh
to catch our initial "can we even run git?" test. While
we're at it, we can add a comment to make it a bit less
inscrutable.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Store the pointer to the string allocated by shorten_unambiguous_ref in
a dedicated variable, short_base, and keep base unchanged. A non-const
variable is more appropriate for such an object. It avoids having to
cast const away on free and stops redefining the meaning of base, making
the code slightly clearer.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When doing a reflog walk, we use the commit's date to
do any date limiting. In earlier versions of Git, this could
lead to nonsense results, since a skipped commit would
truncate the traversal. So a sequence like:
git commit ...
git checkout week-old-branch
git checkout -
git log -g --since=1.day.ago
would stop at the week-old-branch, even though the "git
commit" entry further back is still interesting.
As of the prior commit, which uses a parent-less traversal
of the reflog, you get the whole reflog minus any commits
whose dates do not match the specified options. This is
arguably useful, as you could scan the reflogs for commits
that originated in a certain range.
But more likely a user doing a reflog walk wants to limit
based on the reflog entries themselves. You can simulate
--until with:
git log -g @{1.day.ago}
but there's no way to ask Git to traverse only back to a
certain date. E.g.:
# show me reflog entries from the past day
git log -g --since=1.day.ago
This patch teaches the revision machinery to prefer the
reflog entry dates to the commit dates when doing a reflog
walk. Technically this is a change in behavior that affects
plumbing, but the previous behavior was so buggy that it's
unlikely anyone was relying on it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The reflog-walk system works by putting a ref's tip into the
pending queue, and then "traversing" the reflog by
pretending that the parent of each commit is the previous
reflog entry.
This causes a number of user-visible oddities, as documented
in t1414 (and the commit message which introduced it). We
can fix all of them in one go by replacing the fake-reflog
system with a much simpler one: just keeping a list of
reflogs to show, and walking through them entry by entry.
The implementation is fairly straight-forward, but there are
a few items to note:
1. We obviously must skip calling add_parents_to_list()
when we are traversing reflogs, since we do not want to
walk the original parents at all. As a result, we must call
try_to_simplify_commit() ourselves.
There are other parts of add_parents_to_list() we skip,
as well, but none of them should matter for a reflog
traversal:
- We do not allow UNINTERESTING commits, nor
symmetric ranges (and we bail when these are used
with "-g").
- Using --source makes no sense, since we aren't
traversing. The reflog selector shows the same
information with more detail.
- Using --first-parent is still sensible, since you
may want to see the first-parent diff for each
entry. But since we're not traversing, we don't
need to cull the parent list here.
2. Since we now just walk the reflog entries themselves,
rather than starting with the ref tip, we now look at
the "new" field of each entry rather than the "old"
(i.e., we are showing entries, not faking parents).
This removes all of the tricky logic around skipping
past root commits.
But note that we have no way to show an entry with the
null sha1 in its "new" field (because such a commit
obviously does not exist). Normally this would not
happen, since we delete reflogs along with refs, but
there is one special case. When we rename the currently
checked out branch, we write two reflog entries into
the HEAD log: one where the commit goes away, and
another where it comes back.
Prior to this commit, we show both entries with
identical reflog messages. After this commit, we show
only the "comes back" entry. See the update in t3200
which demonstrates this.
Arguably either is fine, as the whole double-entry
thing is a bit hacky in the first place. And until a
recent fix, we truncated the traversal in such a case
anyway, which was _definitely_ wrong.
3. We show individual reflogs in order, but choose which
reflog to show at each stage based on which has the
most recent timestamp. This interleaves the output
from multiple reflogs based on date order, which is
probably what you'd want with limiting like "-n 30".
Note that the implementation aims for simplicity. It
does a linear walk over the reflog queue for each
commit it pulls, which may perform badly if you
interleave an enormous number of reflogs. That seems
like an unlikely use case; if we did want to handle it,
we could probably keep a priority queue of reflogs,
ordered by the timestamp of their current tip entry.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When git-rev-list sees no pending commits, it shows a usage
message. This works even when reflog-walking is requested,
because the reflog-walk code currently puts the reflog tips
into the pending queue.
In preparation for refactoring the reflog-walk code, let's
explicitly check whether we have any reflogs to walk. For
now this is a noop, but the existing reflog tests will make
sure that it kicks in after the refactoring. Likewise, we'll
add a test that "rev-list -g" without specifying any reflogs
continues to fail (so that we know our check does not kick
in too aggressively).
Note that the implementation needs to go into its own
sub-function, as the walk code does not expose its innards
outside of reflog-walk.c.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The get_revision_1() function tries to avoid entering its
main loop at all when there are no commits to look at. But
it's perfectly safe to call pop_commit() on an empty list
(in which case it will return NULL). Switching to an early
return from the loop lets us skip repeating the loop
condition before we enter the do-while. That will get more
important when we start pulling reflog-walk commits from a
source besides the revs->commits queue, as that condition
will get much more complicated.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we're doing a reflog walk (instead of walking the
actual parent pointers), we may see commits multiple times.
For this reason, we hold on to the commit buffer for each
commit rather than freeing it after we've showed the commit.
We should do the same for the parent list. Right now this is
just a minor optimization. But once we refactor how reflog
walks are performed, keeping the parents will avoid
confusing us the second time we see the commit.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we're walking reflogs, we leave the commit buffer and
parents in place. A comment explains that this is due to
"cycles". But the interesting thing is the unsaid
implication: that the cycles (plus our clearing of the SEEN
flag) will cause us to show commits multiple times. Let's
spell it out.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Simplify the code by using hex2chr() to convert and check for invalid
characters at the same time instead of doing that sequentially with
one table lookup for each.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The reflog-walk code doesn't work with limit_list(). That
function traverses down the real history graph, not the fake
reflog history that get_revision() returns. So it's not
going to actually examine all of the commits we're going to
show, because we'd add them to the pending list only during
the actual traversal.
In practice this limitation doesn't really matter, because
the options that require list-limiting generally need
UNINTERESTING endpoints or symmetric ranges, which already
are forbidden for reflog walks. Still, there are likely some
corner cases that would behave oddly. We're better off to
warn the user that we can't fulfill their request than to
generate potentially wrong output.
This will also make it easier to refactor the reflog-walking
code, because it eliminates a whole area of corner cases
we'd have to consider (that already don't work anyway).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since its inception, the general strategy of the reflog-walk
code has been to start with the tip commit for the ref, and
as we traverse replace each commit's parent pointers with
fake parents pointing to the previous reflog entry.
This lets us traverse the reflog as if it were a real
history, but it has some user-visible oddities. Namely:
1. The fake parents are used for commit selection and
display. So for example, "--merges" or "--no-merges"
are not useful, because the history appears as a linear
string of commits. Likewise, pathspec limiting is based
on the diff between adjacent entries, not the changes
actually introduced by a commit.
These are often the same (e.g., because the entry was
just running "git commit" and the adjacent entry _is_
the true parent), but it may not be in several common
cases. For instance, using "git reset" to jump around
history, or "git checkout" to move HEAD.
2. We reverse-map each commit back to its reflog. So when
it comes time to show commit X, we say "a-ha, we added
X because it was at the tip of the 'foo' reflog, so
let's show the foo reflog". But this leads to nonsense
results when you ask to traverse multiple reflogs: if
two reflogs have the same tip commit, we only map back
to one of them. Instead, we should show both.
3. If the tip of the reflog and the ref tip disagree on
the current value, we show the ref tip but give no
indication of the value in the reflog. This situation
isn't supposed to happen (since any ref update should
touch the reflog). But if it does, given that the
requested operation is to show the reflog, it makes
sense to prefer that.
This commit adds a new script with several expect_failure
tests to demonstrate the problems. This could be part of
the existing t1411, but it's a bit easier to start from a
fresh state, where we know exactly what will be in the log.
Since the new multiple-reflog tests are checking the actual
output, we can drop the "make sure we don't segfault" tests
from t1411, which are a strict subset of what we're doing
here.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jk/reflog-walk-maint:
reflog-walk: include all fields when freeing complete_reflogs
reflog-walk: don't free reflogs added to cache
reflog-walk: duplicate strings in complete_reflogs list
reflog-walk: skip over double-null oid due to HEAD rename
strbuf_addftime() is further getting tweaked.
* ab/strbuf-addftime-tzname-boolify:
strbuf: change an always NULL/"" strbuf_addftime() param to bool
strbuf.h comment: discuss strbuf_addftime() arguments in order
"git send-email" learned to overcome some SMTP server limitation
that does not allow many pieces of e-mails to be sent over a single
session.
* xz/send-email-batch-size:
send-email: --batch-size to work around some SMTP server limit
A few tests that tried to verify the contents of push certificates
did not use 'git rev-parse' to formulate the line to look for in
the certificate correctly.
* js/t5534-rev-parse-gives-multi-line-output-fix:
t5534: fix misleading grep invocation
The Makefile rule in contrib/subtree for building documentation
learned to honour USE_ASCIIDOCTOR just like the main documentation
set does.
* aw/contrib-subtree-doc-asciidoctor:
subtree: honour USE_ASCIIDOCTOR when set
The split index code did not honor core.sharedrepository setting
correctly.
* cc/shared-index-permfix:
t1700: make sure split-index respects core.sharedrepository
t1301: move modebits() to test-lib-functions.sh
read-cache: use shared perms when writing shared index
Optimize "what are the object names already taken in an alternate
object database?" query that is used to derive the length of prefix
an object name is uniquely abbreviated to.
* rs/sha1-name-readdir-optim:
sha1_file: guard against invalid loose subdirectory numbers
sha1_file: let for_each_file_in_obj_subdir() handle subdir names
p4205: add perf test script for pretty log formats
sha1_name: cache readdir(3) results in find_short_object_filename()
Introduce a "repository" object to eventually make it easier to
work in multiple repositories (the primary focus is to work with
the superproject and its submodules) in a single process.
* bw/repo-object:
ls-files: use repository object
repository: enable initialization of submodules
submodule: convert is_submodule_initialized to work on a repository
submodule: add repo_read_gitmodules
submodule-config: store the_submodule_cache in the_repository
repository: add index_state to struct repo
config: read config from a repository object
path: add repo_worktree_path and strbuf_repo_worktree_path
path: add repo_git_path and strbuf_repo_git_path
path: worktree_git_path() should not use file relocation
path: convert do_git_path to take a 'struct repository'
path: convert strbuf_git_common_path to take a 'struct repository'
path: always pass in commondir to update_common_dir
path: create path.h
environment: store worktree in the_repository
environment: place key repository state in the_repository
repository: introduce the repository object
environment: remove namespace_len variable
setup: add comment indicating a hack
setup: don't perform lazy initialization of repository state
Some email servers (e.g. smtp.163.com) limit the number emails to be
sent per session (connection) and this will lead to a faliure when
sending many messages.
Teach send-email to disconnect after sending a number of messages
(configurable via the --batch-size=<num> option), wait for a few
seconds (configurable via the --relogin-delay=<seconds> option) and
reconnect, to work around such a limit.
Also add two configuration variables to give these options the default.
Note:
We will use this as a band-aid for now, but in the longer term, we
should look at and react to the SMTP error code from the server;
Xianqiang reports that 450 and 451 are returned by problematic
servers.
cf. https://public-inbox.org/git/7993e188.d18d.15c3560bcaf.Coremail.zxq_yx_007@163.com/
Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a user wants to experiment with the version of collision
detecting sha1 from the submodule, the user needed to not just
populate the submodule but also needed to turn the knob.
A Makefile trick is easy enough to do so, so let's do this. When
somebody with a copy of the submodule populated wants not to use it,
that can be done by overriding it in config.mak or from the command
line.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add an option to use the sha1collisiondetection library from the
submodule in sha1collisiondetection/ instead of in the copy in the
sha1dc/ directory.
This allows us to try out the submodule in sha1collisiondetection
without breaking the build for anyone who's not expecting them as we
work out any kinks.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
strbuf_addftime() allows callers to pass a time zone name for
expanding %Z. The only current caller either passes the empty string
or NULL, in which case %Z is handed over verbatim to strftime(3).
Replace that string parameter with a flag controlling whether to
remove %Z from the format specification. This simplifies the code.
Commit-message-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some `clean` / `smudge` filters may require a significant amount of
time to process a single blob (e.g. the Git LFS smudge filter might
perform network requests). During this process the Git checkout
operation is blocked and Git needs to wait until the filter is done to
continue with the checkout.
Teach the filter process protocol, introduced in edcc8581 ("convert: add
filter.<driver>.process option", 2016-10-16), to accept the status
"delayed" as response to a filter request. Upon this response Git
continues with the checkout operation. After the checkout operation Git
calls "finish_delayed_checkout" which queries the filter for remaining
blobs. If the filter is still working on the completion, then the filter
is expected to block. If the filter has completed all remaining blobs
then an empty response is expected.
Git has a multiple code paths that checkout a blob. Support delayed
checkouts only in `clone` (in unpack-trees.c) and `checkout` operations
for now. The optimization is most effective in these code paths as all
files of the tree are processed.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code to negotiate long running filter capabilities was very
repetitive for new capabilities. Replace the repetitive conditional
statements with a table-driven approach. This is useful for the
subsequent patch 'convert: add "status=delayed" to filter process
protocol'.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git status" has long shown essentially the same message as "git
commit"; the message it gives while preparing for the root commit,
i.e. "Initial commit", was hard to understand for some new users.
Now it says "No commits yet" to stress more on the current status
(rather than the commit the user is preparing for, which is more in
line with the focus of "git commit").
* ks/status-initial-commit:
status: contextually notify user about an initial commit
Traditionally, the default die() routine had a code to prevent it
from getting called multiple times, which interacted badly when a
threaded program used it (one downside is that the real error may
be hidden and instead the only error message given to the user may
end up being "die recursion detected", which is not very useful).
* ab/die-errors-in-threaded:
die(): stop hiding errors due to overzealous recursion guard
Fix a recent regression to "git rebase -i" and add tests that would
have caught it and others.
* pw/rebase-i-regression-fix-tests:
t3420: fix under GETTEXT_POISON build
rebase: add more regression tests for console output
rebase: add regression tests for console output
rebase -i: add test for reflog message
sequencer: print autostash messages to stderr
While at it, clarify the use of `key`, `keydata`, `entry_or_key` as well
as documenting the new data pointer for the compare function.
Rework the example.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As alluded to in the previous patch, the code in patch-ids.c is
using the hashmaps API wrong.
Luckily we do not have a bug, as all hashmap functionality that we use
here (hashmap_get) passes through the keydata. If hashmap_get_next were
to be used, a bug would occur as that passes NULL for the key_data.
So instead use the hashmap API correctly and provide the caller required
data in the compare function via the first argument that always gets
passed and was setup via the hashmap_init function.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using the hashmap a common need is to have access to caller provided
data in the compare function. A couple of times we abuse the keydata field
to pass in the data needed. This happens for example in patch-ids.c.
This patch changes the function signature of the compare function
to have one more void pointer available. The pointer given for each
invocation of the compare function must be defined in the init function
of the hashmap and is just passed through.
Documentation of this new feature is deferred to a later patch.
This is a rather mechanical conversion, just adding the new pass-through
parameter. However while at it improve the naming of the fields of all
compare functions used by hashmaps by ensuring unused parameters are
prefixed with 'unused_' and naming the parameters what they are (instead
of 'unused' make it 'unused_keydata').
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the redundant REG_NEWLINE regcomp() flag from the code that
compiles a fixed-string regular-expression.
The REG_NEWLINE causes metacharacters such as "." to match a newline,
since the basic_regex_quote_buf() function being called here escapes
all metacharacters using REG_NEWLINE is confusing and redundant.
The use of this flag was introduced as an unintended emergent property
of 793dc676e0 ("grep/icase: avoid kwsset when -F is specified",
2016-06-25).
That change amended the existing regflags, which were initialized to
REG_NEWLINE in init_grep_defaults() assuming a subsequent non-fixed
regcomp().
Manual testing reveals that this was always redundant, since no flags
of any use were inherited from opt->regflags even back
then. 793dc676e0 passes all tests with this on top:
diff --git a/grep.c b/grep.c
index 627ae3e3e8..89e84ed7fd 100644
--- a/grep.c
+++ b/grep.c
@@ -407,3 +407,3 @@ static void compile_fixed_regexp(struct grep_pat *p, struct grep_opt *opt)
basic_regex_quote_buf(&sb, p->pattern);
- regflags = opt->regflags & ~REG_EXTENDED;
+ regflags = 0;
if (opt->ignore_case)
Since this isn't used for anything and never was, remove it to reduce
confusion when reading this code.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor calls to the grep machinery to always pass opt.ignore_case &
opt.extended_regexp_option instead of setting the equivalent regflags
bits.
The bug fixed when making -i work with -P in commit 9e3cbc59d5 ("log:
make --regexp-ignore-case work with --perl-regexp", 2017-05-20) was
really just plastering over the code smell which this change fixes.
The reason for adding the extensive commentary here is that I
discovered some subtle complexity in implementing this that really
should be called out explicitly to future readers.
Before this change we'd rely on the difference between
`extended_regexp_option` and `regflags` to serve as a membrane between
our preliminary parsing of grep.extendedRegexp and grep.patternType,
and what we decided to do internally.
Now that those two are the same thing, it's necessary to unset
`extended_regexp_option` just before we commit in cases where both of
those config variables are set. See 84befcd0a4 ("grep: add a
grep.patternType configuration setting", 2012-08-03) for the code and
documentation related to that.
The explanation of why the if/else branches in
grep_commit_pattern_type() are ordered the way they are exists in that
commit message, but I think it's worth calling this subtlety out
explicitly with a comment for future readers.
Even though grep_commit_pattern_type() is the only caller of
grep_set_pattern_type_option() it's simpler to reset the
extended_regexp_option flag in the latter, since 2/3 branches in the
former would otherwise need to reset it, this way we can do it in one
place.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the redundant re-assignments of the fixed/pcre1/pcre2 fields to
zero right after the entire struct has been set to zero via
memset(...).
See an earlier related cleanup commit e0b9f8ae09 ("grep: remove
redundant regflags assignments", 2017-05-25) for an explanation of why
the code was structured like this to begin with.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the redundant re-assignment of the fixed field to zero right
after the entire struct has been set to zero via memset(...).
Unlike some nearby commits this pattern doesn't date back to the
pattern described in e0b9f8ae09 ("grep: remove redundant regflags
assignments", 2017-05-25), instead it was apparently cargo-culted in
9eceddeec6 ("Use kwset in grep", 2011-08-21).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adjust a now-redundant assignment to extended_regexp_option to make it
zero if grep.extendedRegexp is not set. This is always called right
after init_grep_defaults() which memsets the entire structure to 0, so
there's no need to set it again to zero.
However the reason for the if/else pattern is a holdover from[1] where
this was adjusted from a bitfield assignment to a boolean. Rather than
getting rid of the assignment to 0 in all cases, let's just use the
value returned by git_config_bool(), which is more idiomatic and in
sync with the rest of the boolean handling in this function.
This is a logical follow-up to my commit to remove redundant regflags
assignments[2]. This logic was originally introduced in [3], but as
explained in the former commit it's working around a pattern in our
code that no longer exists, and is now confusing as it leads the
reader to think that this needs to be flipped back & forth.
1. 84befcd0a4 ("grep: add a grep.patternType configuration setting",
2012-08-03)
2. e0b9f8ae09 ("grep: remove redundant regflags assignments",
2017-05-25)
3. b22520a37c ("grep: allow -E and -n to be turned on by default via
configuration", 2011-03-30)
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stop assigning 0 to the extended_regexp_option field right after we've
zeroed out the entire struct with memset() just a few lines earlier.
Unlike some of the code being refactored in subsequent commits, this
was always completely redundant. See the original code introduced in
84befcd0a4 ("grep: add a grep.patternType configuration setting",
2012-08-03).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The commit template adds the status information without
adding a new line to distinguish them in the absence
of optional parts. This results in difficulty in interpreting
it's content, specifically for inexperienced users.
Unconditionally, add new lines to separate the status message
from the other parts of the commit-template to make it more
readable.
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The notice that "git commit <paths>" default to "git commit
--only <paths>" was there since 756e3ee0 ("Merge branch
'jc/commit'", 2006-02-14). Back then, existing users of Git
expected the command doing "git commit --include <paths>", and
after the behaviour of the command was changed to align with
other people's "$scm commit <paths>", the text was added to help
them transition their expectations.
Remove the message that now has outlived its usefulness.
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactoring the filter error handling is useful for the subsequent patch
'convert: add "status=delayed" to filter process protocol'.
In addition, replace the parentheses around the empty "if" block with a
single semicolon to adhere to the Git style guide.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"rot13-filter.pl" always writes "OUT <size>" to the debug log at the end
of a response.
This works perfectly for the existing responses "abort", "error", and
"success". A new response "delayed", that will be introduced in a
subsequent patch, accepts the input without giving the filtered result
right away. At this point we cannot know the size of the response.
Therefore, we do not write "OUT <size>" for "delayed" responses.
To simplify the code we do not write "OUT <size>" for "abort" and
"error" responses either as their size is always zero.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are two rules for using FREE_AND_NULL in free.cocci, one for
pointer types and one for expressions. Both cause coccinelle to remove
empty lines and even newline characters between replacements for some
reason; consecutive "free(x);/x=NULL;" sequences end up as multiple
FREE_AND_NULL calls on the same time.
Remove the type rule, as the expression rule already covers it, and
rearrange the lines of the latter to place the addition of FREE_AND_NULL
between the removals, which causes coccinelle to leave surrounding
whitespace untouched.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The check for the default was introduced with 88a21979c5 (fetch/pull:
recurse into submodules when necessary, 2011-03-06), which replaced an
older construct (builtin/fetchs own implementation of the super-prefix)
introduced in be254a0ea9 (Add the 'fetch.recurseSubmodules' config setting,
2010-11-11) which made sense at the time as there was no default fetch
option for submodules at the time.
Set builtin/fetch.c#recurse_submodules_default to the same value as
submodule.c#config_fetch_recurse_submodules which is set via
set_config_fetch_recurse_submodules, such that the condition for checking
whether we have to set the default value becomes unnecessary.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git add -p" were updated in 2.12 timeframe to cope with custom
core.commentchar but the implementation was buggy and a
metacharacter like $ and * did not work.
* jk/add-p-commentchar-fix:
add--interactive: quote commentChar regex
add--interactive: handle EOF in prompt_yesno
Bugfix for a topic that is (only) in 'master'.
* mh/packed-ref-store-prep:
for_each_bisect_ref(): don't trim refnames
lock_packed_refs(): fix cache validity check
"git status" learned to optionally give how many stash entries the
user has in its output.
* lb/status-stash-count:
glossary: define 'stash entry'
status: add optional stash count information
stash: update documentation to use 'stash entry'
The "rot13-filter.pl" helper wrote its debug logs always to "rot13-filter.log".
Make this configurable by defining the log file as first parameter of
"rot13-filter.pl".
This is useful if "rot13-filter.pl" is configured multiple times similar to the
subsequent patch 'convert: add "status=delayed" to filter process protocol'.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The filter log files are modified on comparison. That might be
unexpected by the caller. It would be even undesirable if the caller
wants to reuse the original log files.
Address these issues by using temp files for modifications. This is
useful for the subsequent patch 'convert: add "status=delayed" to
filter process protocol'.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
has_sha1_file_with_flags() implements many mechanisms in common with
sha1_object_info_extended(). Make has_sha1_file_with_flags() a
convenience function for sha1_object_info_extended() instead.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, regardless of the contents of the "struct object_info" passed
to sha1_object_info_extended(), that function always accesses the
packfile whenever it returns information about a packed object, since it
needs to populate "u.packed".
Add the ability to pass NULL, and use NULL-ness of the argument to
activate an optimization in which sha1_object_info_extended() does not
needlessly access the packfile. A subsequent patch will make use of this
optimization.
A similar optimization is not made for the cached and loose cases as it
would not cause a significant performance improvement.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Improve sha1_object_info_extended() by supporting additional
flags. This allows has_sha1_file_with_flags() to be modified to use
sha1_object_info_extended() in a subsequent patch.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A common pattern to free a piece of memory and assign NULL to the
pointer that used to point at it has been replaced with a new
FREE_AND_NULL() macro.
* ab/free-and-null:
*.[ch] refactoring: make use of the FREE_AND_NULL() macro
coccinelle: make use of the "expression" FREE_AND_NULL() rule
coccinelle: add a rule to make "expression" code use FREE_AND_NULL()
coccinelle: make use of the "type" FREE_AND_NULL() rule
coccinelle: add a rule to make "type" code use FREE_AND_NULL()
git-compat-util: add a FREE_AND_NULL() wrapper around free(ptr); ptr = NULL
Using "git add d/i/r" when d/i/r is the top of the working tree of
a separate repository would create a gitlink in the index, which
would appear as a not-quite-initialized submodule to others. We
learned to give warnings when this happens.
* jk/warn-add-gitlink:
t: move "git add submodule" into test blocks
add: warn when adding an embedded repository
Fix configuration codepath to pay proper attention to commondir
that is used in multi-worktree situation, and isolate config API
into its own header file.
* bw/config-h:
config: don't implicitly use gitdir or commondir
config: respect commondir
setup: teach discover_git_directory to respect the commondir
config: don't include config.h by default
config: remove git_config_iter
config: create config.h
Code clean-up.
* bw/ls-files-sans-the-index:
ls-files: factor out tag calculation
ls-files: factor out debug info into a function
ls-files: convert show_files to take an index
ls-files: convert show_ce_entry to take an index
ls-files: convert prune_cache to take an index
ls-files: convert ce_excluded to take an index
ls-files: convert show_ru_info to take an index
ls-files: convert show_other_files to take an index
ls-files: convert show_killed_files to take an index
ls-files: convert write_eolinfo to take an index
ls-files: convert overlay_tree_on_cache to take an index
tree: convert read_tree to take an index parameter
convert: convert renormalize_buffer to take an index
convert: convert convert_to_git to take an index
convert: convert convert_to_git_filter_fd to take an index
convert: convert crlf_to_git to take an index
convert: convert get_cached_convert_stats_ascii to take an index
The code to pick up and execute command alias definition from the
configuration used to switch to the top of the working tree and
then come back when the expanded alias was executed, which was
unnecessarilyl complex. Attempt to simplify the logic by using the
early-config mechanism that does not chdir around.
* js/alias-early-config:
alias: use the early config machinery to expand aliases
t7006: demonstrate a problem with aliases in subdirectories
t1308: relax the test verifying that empty alias values are disallowed
help: use early config when autocorrecting aliases
config: report correct line number upon error
discover_git_directory(): avoid setting invalid git_dir
The pretty-format specifiers like '%h', '%t', etc. had an
optimization that no longer works correctly. In preparation/hope
of getting it correctly implemented, first discard the optimization
that is broken.
* rs/pretty-add-again:
pretty: recalculate duplicate short hashes
The 'diff-highlight' program (in contrib/) has been restructured
for easier reuse by an external project 'diff-so-fancy'.
* jk/diff-highlight-module:
diff-highlight: split code into module
An example in documentation that does not work in multi worktree
configuration has been corrected.
* ah/doc-gitattributes-empty-index:
doc: do not use `rm .git/index` when normalizing line endings
A new test to show the interaction between the pattern [^a-z]
(which matches '/') and a slash in a path has been added. The
pattern should not match the slash with "pathmatch", but should
with "wildmatch".
* ab/wildmatch-glob-slash-test:
wildmatch test: cover a blind spot in "/" matching
"git mergetool" learned to work around a wrapper MacOS X adds
around underlying meld.
* da/mergetools-meld-output-opt-on-macos:
mergetools/meld: improve compatibiilty with Meld on macOS X
Change the comment documenting the strbuf_addftime() function to
discuss the parameters in the order in which they appear, which makes
this easier to read than discussing them out of order.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Loose object subdirectories have hexadecimal names based on the first
byte of the hash of contained objects, thus their numerical
representation can range from 0 (0x00) to 255 (0xff). Change the type
of the corresponding variable in for_each_file_in_obj_subdir() and
associated callback functions to unsigned int and add a range check.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function for_each_file_in_obj_subdir() takes a object subdirectory
number and expects the name of the same subdirectory to be included in
the path strbuf. Avoid this redundancy by letting the function append
the hexadecimal subdirectory name itself. This makes it a bit easier
and safer to use the function -- it becomes impossible to specify
different subdirectories in subdir_nr and path.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add simple performance tests for expanded log format placeholders.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the unused wildopts placeholder struct from being passed to all
wildmatch() invocations, or rather remove all the boilerplate NULL
parameters.
This parameter was added back in commit 9b3497cab9 ("wildmatch: rename
constants and update prototype", 2013-01-01) as a placeholder for
future use. Over 4 years later nothing has made use of it, let's just
remove it. It can be added in the future if we find some reason to
start using such a parameter.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert ls-files to use a repository struct and recurse submodules
inprocess.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce 'repo_submodule_init()' which performs initialization of a
'struct repository' as a submodule of another 'struct repository'.
The resulting submodule 'struct repository' can be in one of three states:
1. The submodule is initialized and has a worktree.
2. The submodule is initialized but does not have a worktree. This
would occur when the submodule's gitdir is present in the
superproject's 'gitdir/modules/' directory yet the submodule has not
been checked out in superproject's worktree.
3. The submodule remains uninitialized due to an error in the
initialization process or there is no matching submodule at the
provided path in the superproject.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach the repo object to be able to populate the submodule_cache by
reading the repository's gitmodules file.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor how 'the_submodule_cache' is handled so that it can be stored
inside of a repository object. Also migrate 'the_submodule_cache' to be
stored in 'the_repository'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach the config machinery to read config information from a repository
object. This involves storing a 'struct config_set' inside the
repository object and adding a number of functions (repo_config*) to be
able to query a repository's config.
The current config API enables lazy-loading of the config. This means
that when 'git_config_get_int()' is called, if the_config_set hasn't
been populated yet, then it will be populated and properly initialized by
reading the necessary config files (system wide .gitconfig, user's home
.gitconfig, and the repository's config). To maintain this paradigm,
the new API to read from a repository object's config will also perform
this lazy-initialization.
Since both APIs (git_config_get* and repo_config_get*) have the same
semantics we can migrate the default config to be stored within
'the_repository' and just have the 'git_config_get*' family of functions
redirect to the 'repo_config_get*' functions.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce 'repo_worktree_path' and 'strbuf_repo_worktree_path' which
take a repository struct and constructs a path relative to the
repository's worktree.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce 'repo_git_path' and 'strbuf_repo_git_path' which take a
repository struct and constructs a path into the repository's git
directory.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git_path is a convenience function that usually produces a string
$GIT_DIR/<path>. Since v2.5.0-rc0~143^2~35 (git_path(): be aware of
file relocation in $GIT_DIR, 2014-11-30), as a side benefit callers
get support for path relocation variables like $GIT_OBJECT_DIRECTORY:
- git_path("index") is $GIT_INDEX_FILE when set
- git_path("info/grafts") is $GIT_GRAFTS_FILE when set
- git_path("objects/<foo>") is $GIT_OBJECT_DIRECTORY/<foo> when set
- git_path("hooks/<foo>") is <foo> under core.hookspath when set
- git_path("refs/<foo>") etc (see path.c::common_list) is relative
to $GIT_COMMON_DIR instead of $GIT_DIR
worktree_git_path, by comparison, is designed to resolve files in a
specific worktree's git dir. Unfortunately, it shares code with
git_path and performs the same relocation. The result is that paths
that are meant to be relative to the specified worktree's git dir end
up replaced by paths from environment variables within the current git
dir.
Luckily, no current callers pass such arguments. The relocation was
noticed when testing the result of merging two patches under review,
one of which introduces a caller:
* The first patch made git prune check the index file in each
worktree's git dir (using worktree_git_path(wt, "index")) for
objects not to prune. This would trigger the unwanted relocation
when GIT_INDEX_FILE is set, causing objects reachable from the
index to be pruned.
* The second patch simplified the relocation logic for index,
info/grafts, objects, and hooks to happen unconditionally instead of
based on whether environment or configuration variables are set.
This caused the relocation to trigger even when GIT_INDEX_FILE is
not set.
[jn: rewrote commit message; skipping all relocation instead of just
GIT_INDEX_FILE]
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In preparation to adding 'git_path' like functions which operate on a
'struct repository' convert 'do_git_path' to take a 'struct repository'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of passing in 'NULL' and having 'update_common_dir()' query for
the commondir, have the callers of 'update_common_dir()' be responsible
for providing the commondir.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move all path related declarations from cache.h to a new path.h header
file. This makes cache.h smaller and makes it easier to add new path
related functions.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Migrate 'work_tree' to be stored in 'the_repository'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Migrate 'git_dir', 'git_common_dir', 'git_object_dir', 'git_index_file',
'git_graft_file', and 'namespace' to be stored in 'the_repository'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce the repository object 'struct repository' which can be used to
hold all state pertaining to a git repository.
Some of the benefits of object-ifying a repository are:
1. Make the code base more readable and easier to reason about.
2. Allow for working on multiple repositories, specifically
submodules, within the same process. Currently the process for
working on a submodule involves setting up an argv_array of options
for a particular command and then launching a child process to
execute the command in the context of the submodule. This is
clunky and can require lots of little hacks in order to ensure
correctness. Ideally it would be nice to simply pass a repository
and an options struct to a command.
3. Eliminating reliance on global state will make it easier to
enable the use of threading to improve performance.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use 'skip_prefix' instead of 'starts_with' so that we can drop the need
to keep around 'namespace_len'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'GIT_TOPLEVEL_PREFIX_ENVIRONMENT' was added in (b58a68c1c setup: allow
for prefix to be passed to git commands) to aid in fixing a bug where
'ls-files' and 'grep' were not able to properly recurse when called from
within a subdirectory. Add a 'NEEDSWORK' comment indicating that this
envvar should be removed once 'ls-files' and 'grep' can recurse
in-process.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Under some circumstances (bogus GIT_DIR value or the discovered gitdir
is '.git') 'setup_git_directory()' won't initialize key repository
state. This leads to inconsistent state after running the setup code.
To account for this inconsistent state, lazy initialization is done once
a caller asks for the repository's gitdir or some other piece of
repository state. This is confusing and can be error prone.
Instead let's tighten the expected outcome of 'setup_git_directory()'
and ensure that it initializes repository state in all cases that would
have been handled by lazy initialization.
This also lets us drop the requirement to have 'have_git_dir()' check if
the environment variable GIT_DIR was set as that will be handled by the
end of the setup code.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bw/ls-files-sans-the-index:
ls-files: factor out tag calculation
ls-files: factor out debug info into a function
ls-files: convert show_files to take an index
ls-files: convert show_ce_entry to take an index
ls-files: convert prune_cache to take an index
ls-files: convert ce_excluded to take an index
ls-files: convert show_ru_info to take an index
ls-files: convert show_other_files to take an index
ls-files: convert show_killed_files to take an index
ls-files: convert write_eolinfo to take an index
ls-files: convert overlay_tree_on_cache to take an index
tree: convert read_tree to take an index parameter
convert: convert renormalize_buffer to take an index
convert: convert convert_to_git to take an index
convert: convert convert_to_git_filter_fd to take an index
convert: convert crlf_to_git to take an index
convert: convert get_cached_convert_stats_ascii to take an index
* bw/config-h:
config: don't implicitly use gitdir or commondir
config: respect commondir
setup: teach discover_git_directory to respect the commondir
config: don't include config.h by default
config: remove git_config_iter
config: create config.h
alias: use the early config machinery to expand aliases
t7006: demonstrate a problem with aliases in subdirectories
t1308: relax the test verifying that empty alias values are disallowed
help: use early config when autocorrecting aliases
config: report correct line number upon error
discover_git_directory(): avoid setting invalid git_dir
Teach pull to optionally update submodules when '--recurse-submodules'
is provided. This will teach pull to run 'submodule update --rebase'
when the '--recurse-submodules' and '--rebase' flags are given under
specific circumstances.
On a rebase workflow:
=====================
1. Both sides change the submodule
------------------------------
Let's assume the following history in a submodule:
H---I---J---K---L local branch
\
M---N---O---P remote branch
and the following in the superproject (recorded submodule in parens):
A(H)---B(I)---F(K)---G(L) local branch
\
C(N)---D(N)---E(P) remote branch
In an ideal world this would rebase the submodule and rewrite
the submodule pointers that the superproject points at such that
the superproject looks like
A(H)---B(I) F(K')---G(L') rebased branch
\ /
C(N)---D(N)---E(P) remote branch
and the submodule as:
J---K---L (old dangeling tip)
/
H---I J'---K'---L' rebased branch
\ /
M---N---O---P remote branch
And if a conflict arises in the submodule the superproject rebase
would stop at that commit at which the submodule conflict occurs.
Currently a "pull --rebase" in the superproject produces
a merge conflict as the submodule pointer changes are
conflicting and cannot be resolved.
2. Local submodule changes only
-----------------------
Assuming histories as above, except that the remote branch
would not contain submodule changes, then a result as
A(H)---B(I) F(K)---G(L) rebased branch
\ /
C(I)---D(I)---E(I) remote branch
is desire-able. This is what currently happens in rebase.
If the recursive flag is given, the ideal git would
produce a superproject as:
A(H)---B(I) F(K')---G(L') rebased branch (incl. sub rebase!)
\ /
C(I)---D(I)---E(I) remote branch
and the submodule as:
J---K---L (old dangeling tip)
/
H---I J'---K'---L' locally rebased branch
\ /
M---N---O---P advanced branch
This patch doesn't address this issue, however
a test is added that this fails up front.
3. Remote submodule changes only
----------------------
Assuming histories as in (1) except that the local superproject branch
would not have touched the submodule the rebase already works out in the
superproject with no conflicts:
A(H)---B(I) F(P)---G(P) rebased branch (no sub changes)
\ /
C(N)---D(N)---E(P) remote branch
The recurse flag as presented in this patch would additionally
update the submodule as:
H---I J'---K'---L' rebased branch
\ /
M---N---O---P remote branch
As neither J, K, L nor J', K', L' are referred to from the superproject,
no rewriting of the superproject commits is required.
Conclusion for 'pull --rebase --recursive'
-----------------------------------------
If there are no local superproject changes it is sufficient to call
"submodule update --rebase" as this produces the desired results. In case
of conflicts, the behavior is the same as in 'submodule update --recursive'
which is assumed to be sane.
This patch implements (3) only.
On a merge workflow:
====================
We'll start off with the same underlying DAG as in (1) in the rebase
workflow. So in an ideal world a 'pull --merge --recursive' would
produce this:
H---I---J---K---L----X
\ /
M---N---O---P
with X as the new merge-commit in the submodule and the superproject
as:
A(H)---B(I)---F(K)---G(L)---Y(X)
\ /
C(N)---D(N)---E(P)
However modifying the submodules on the fly is not supported in git-merge
such that Y(X) is not easy to produce in a single patch. In fact git-merge
doesn't know about submodules at all.
However when at least one side does not contain commits touching the
submodule at all, then we do not need to perform the merge for the
submodule but a fast-forward can be done via checking out either L or P
in the submodule. This strategy is implemented in 68d03e4a6e (Implement
automatic fast-forward merge for submodules, 2010-07-07) already, so
to align with the rebase behavior we need to also update the worktree
of the submodule.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of just storing the string and then later calling our own
parsing function 'parse_fetch_recurse_submodules_arg', make use of the
function callback 'option_fetch_parse_recurse_submodules' that was
introduced in the last patch. Also move all submodule recursing variables
in one spot at the top of the file.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Later we want to access this parsing in builtin/pull as well.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch aims to detangle (a) the usage of `git-submodule`
from (b) the concept of submodules and (c) how the actual
implementation looks like, such as where they are configured
and (d) what the best practices are.
To do so, move the conceptual parts of the 'git-submodule'
man page to a new man page gitsubmodules(7). This new page
is just like gitmodules(5), gitattributes(5), gitcredentials(7),
gitnamespaces(7), gittutorial(7), which introduce a concept
rather than explaining a specific command.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As there is no portable way to pass timezone information to
strftime, some output format from "git log" and friends are
impossible to produce. Teach our own strbuf_addftime to replace %z
and %Z with caller-supplied values to help working around this.
* rs/strbuf-addftime-zZ:
date: use localtime() for "-local" time formats
t0006: check --date=format zone offsets
strbuf: let strbuf_addftime handle %z and %Z itself
"git stash push <pathspec>" did not work from a subdirectory at all.
Bugfix for a topic in v2.13
* ps/stash-push-pathspec-fix:
git-stash: fix pushing stash with pathspec from subdir
Code clean-up.
* sg/revision-parser-skip-prefix:
revision.c: use skip_prefix() in handle_revision_pseudo_opt()
revision.c: use skip_prefix() in handle_revision_opt()
revision.c: stricter parsing of '--early-output'
revision.c: stricter parsing of '--no-{min,max}-parents'
revision.h: turn rev_info.early_output back into an unsigned int
"fast-import" uses a default pack chain depth that is consistent
with other parts of the system.
* mh/fast-import-raise-default-depth:
fast-import: increase the default pack depth to 50
"filter-branch" learned a pseudo filter "--setup" that can be used
to define a common function/variable that can be used by other
filters.
* ah/filter-branch-setup:
filter-branch: add [--] to usage
filter-branch: add `--setup` step
The "add" section for 'git-submodule' is redundant in its
description and the short synopsis line. Fix it.
Remove the redundant mentioning of the 'repository' argument
being mandatory.
The text is hard to read because of back-references, so remove
those.
Replace the word "humanish" by "canonical" as that conveys better
what we do to guess the path.
While at it, quote all occurrences of '.gitmodules' as that is an
important file in the submodule context, also link to it on its
first mention.
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Read each loose object subdirectory at most once when looking for unique
abbreviated hashes. This speeds up commands like "git log --pretty=%h"
considerably, which previously caused one readdir(3) call for each
candidate, even for subdirectories that were visited before.
The new cache is kept until the program ends and never invalidated. The
same is already true for pack indexes. The inherent racy nature of
finding unique short hashes makes it still fit for this purpose -- a
conflicting new object may be added at any time. Tasks with higher
consistency requirements should not use it, though.
The cached object names are stored in an oid_array, which is quite
compact. The bitmap for remembering which subdir was already read is
stored as a char array, with one char per directory -- that's not quite
as compact, but really simple and incurs only an overhead equivalent to
11 hashes after all.
Suggested-by: Jeff King <peff@peff.net>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The existing message, "Initial commit", makes sense for the commit template
notifying users that it's their initial commit, but is confusing when
merely checking the status of a fresh repository (or orphan branch)
without having any commits yet.
Change the output of "status" to say "No commits yet" when "git
status" is run on a fresh repo (or orphan branch), while retaining the
current "Initial commit" message displayed in the template that's
displayed in the editor when the initial commit is being authored.
Correspondingly change the output of "short status" to "No commits yet
on " when "git status -sb" is run on a fresh repo (or orphan branch).
A few alternatives considered were,
* Waiting for initial commit
* Your current branch does not have any commits
* Current branch waiting for initial commit
The most succint one among the alternatives was chosen.
[with help on tests from Ævar]
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Kaartic Sivaraam <kaarticsivaraam91196@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read_object() and sha1_object_info_extended() both implement mechanisms
such as object replacement, retrying the packed store after failing to
find the object in the packed store then the loose store, and being able
to mark a packed object as bad and then retrying the whole process.
Consolidating these mechanisms would be a great help to maintainability.
Therefore, consolidate them by extending sha1_object_info_extended() to
support the functionality needed, and then modifying read_object() to
use sha1_object_info_extended().
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent patch, packed_object_info() will be modified to use the
delta base cache, so move the relevant code to before
packed_object_info().
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The LOOKUP_REPLACE_OBJECT flag controls whether the
lookup_replace_object() function is invoked by
sha1_object_info_extended(), read_sha1_file_extended(), and
lookup_replace_object_extended(), but it is not immediately clear which
functions accept that flag.
Therefore restrict this flag to only sha1_object_info_extended(),
renaming it appropriately to OBJECT_INFO_LOOKUP_REPLACE and adding some
documentation. Update read_sha1_file_extended() to have a boolean
parameter instead, and delete lookup_replace_object_extended().
parse_sha1_header() also passes this flag to
parse_sha1_header_extended() since commit 46f0344 ("sha1_file: support
reading from a loose object of unknown type", 2015-05-03), but that has
had no effect since that commit. Therefore this patch also removes this
flag from that invocation.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The LOOKUP_UNKNOWN_OBJECT flag was introduced in commit 46f0344
("sha1_file: support reading from a loose object of unknown type",
2015-05-03) in order to support a feature in cat-file subsequently
introduced in commit 39e4ae3 ("cat-file: teach cat-file a
'--allow-unknown-type' option", 2015-05-03). Despite its name and
location in cache.h, this flag is used neither in
read_sha1_file_extended() nor in any of the lookup functions, but used
only in sha1_object_info_extended().
Therefore rename this flag to OBJECT_INFO_ALLOW_UNKNOWN_TYPE, taking the
name of the cat-file flag that invokes this feature, and move it closer
to the declaration of sha1_object_info_extended(). Also add
documentation for this flag.
OBJECT_INFO_ALLOW_UNKNOWN_TYPE is defined to 2, not 1, to avoid
conflicting with LOOKUP_REPLACE_OBJECT. Avoidance of this conflict is
necessary because sha1_object_info_extended() supports both flags.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the recursion limit for the default die routine from a *very*
low 1 to 1024. This ensures that infinite recursions are broken, but
doesn't lose the meaningful error messages under threaded execution
where threads concurrently start to die.
The intent of the existing code, as explained in commit
cd163d4b4e ("usage.c: detect recursion in die routines and bail out
immediately", 2012-11-14), is to break infinite recursion in cases
where the die routine itself calls die(), and would thus infinitely
recurse.
However, doing that very aggressively by immediately printing out
"recursion detected in die handler" if we've already called die() once
means that threaded invocations of git can end up only printing out
the "recursion detected" error, while hiding the meaningful error.
An example of this is running a threaded grep which dies on execution
against pretty much any repo, git.git will do:
git grep -P --threads=8 '(*LIMIT_MATCH=1)-?-?-?---$'
With the current version of git this will print some combination of
multiple PCRE failures that caused the abort and multiple "recursion
detected", some invocations will print out multiple "recursion
detected" errors with no PCRE error at all!
Before this change, running the above grep command 1000 times against
git.git[1] and taking the top 20 results will on my system yield the
following distribution of actual errors ("E") and recursion
errors ("R"):
322 E R
306 E
116 E R R
65 R R
54 R E
49 E E
44 R
15 E R R R
9 R R R
7 R E R
5 R R E
3 E R R R R
2 E E R
1 R R R R
1 R R R E
1 R E R R
The exact results are obviously random and system-dependent, but this
shows the race condition in this code. Some small part of the time
we're about to print out the actual error ("E") but another thread's
recursion error beats us to it, and sometimes we print out nothing but
the recursion error.
With this change we get, now with "W" to mean the new warning being
emitted indicating that we've called die() many times:
502 E
160 E W E
120 E E
53 E W
35 E W E E
34 W E E
29 W E E E
16 E E W
16 E E E
11 W E E E E
7 E E W E
4 W E
3 W W E E
2 E W E E E
1 W W E
1 W E W E
1 E W W E E E
1 E W W E E
1 E W W E
1 E W E E W
Which still sucks a bit, due to a still present race-condition in this
code we're sometimes going to print out several errors still, or
several warnings, or two duplicate errors without the warning.
But we will never have a case where we completely hide the actual
error as we do now.
Now, git-grep could make use of the pluggable error facility added in
commit c19a490e37 ("usage: allow pluggable die-recursion checks",
2013-04-16). There's other threaded code that calls set_die_routine()
or set_die_is_recursing_routine().
But this is about fixing the general die() behavior with threading
when we don't have such a custom routine yet. Right now the common
case is not an infinite recursion in the handler, but us losing error
messages by default because we're overly paranoid about our recursion
check.
So let's just set the recursion limit to a number higher than the
number of threads we're ever likely to spawn. Now we won't lose
errors, and if we have a recursing die handler we'll still die within
microseconds.
There are race conditions in this code itself, in particular the
"dying" variable is not thread mutexed, so we e.g. won't be dying at
exactly 1024, or for that matter even be able to accurately test
"dying == 2", see the cases where we print out more than one "W"
above.
But that doesn't really matter, for the recursion guard we just need
to die "soon", not at exactly 1024 calls, and for printing the correct
error and only one warning most of the time in the face of threaded
death this is good enough and a net improvement on the current code.
1. for i in {1..1000}; do git grep -P --threads=8 '(*LIMIT_MATCH=1)-?-?-?---$' 2>&1|perl -pe 's/^fatal: r.*/R/; s/^fatal: p.*/E/; s/^warning.*/W/' | tr '\n' ' '; echo; done | sort | uniq -c | sort -nr | head -n 20
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We bumped the default in be4ca2905 (Increase
core.packedGitLimit, 2017-04-20) but never adjusted the
documentation to match.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix an erroneously copy/pasted check for the pcre2_jit_stack variable
to check pcre2_match_context instead. The former was already checked
in the preceding "if" statement.
This is a trivial and obvious error introduced in my commit
94da9193a6 ("grep: add support for PCRE v2", 2017-06-01).
In practice if pcre2_match_context_create() returned NULL we were
likely in a situation where malloc() was returning NULL, and were thus
screwed anyway, but if only the pcre2_match_context_create() call
returned NULL (through some transitory bug) PCRE v2 would just
allocate and supply its own context object when matching, and we'd run
normally at the trivial expense of not getting a slight speedup by
sharing the context object between successive matches.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git $cmd -h" for builtin commands calls the implementation of the
command (i.e. cmd_$cmd() function) without doing any repository
set-up, and the commands that expect RUN_SETUP is done by the Git
potty needs to be prepared to show the help text without barfing.
* jk/consistent-h:
t0012: test "-h" with builtins
git: add hidden --list-builtins option
version: convert to parse-options
diff- and log- family: handle "git cmd -h" early
submodule--helper: show usage for "-h"
remote-{ext,fd}: print usage message on invalid arguments
upload-archive: handle "-h" option early
credential: handle invalid arguments earlier
When an existing repository is used for t/perf testing, we first
create bit-for-bit copy of it, which may grab a transient state of
the repository and freeze it into the repository used for testing,
which then may cause Git operations to fail. Single out "the index
being locked" case and forcibly drop the lock from the copy.
* ab/perf-remove-index-lock:
perf: work around the tested repo having an index.lock
Update "perl-compatible regular expression" support to enable JIT
and also allow linking with the newer PCRE v2 library.
* ab/pcre-v2:
grep: add support for PCRE v2
grep: un-break building with PCRE >= 8.32 without --enable-jit
grep: un-break building with PCRE < 8.20
grep: un-break building with PCRE < 8.32
grep: add support for the PCRE v1 JIT API
log: add -P as a synonym for --perl-regexp
grep: skip pthreads overhead when using one thread
grep: don't redundantly compile throwaway patterns under threading
The convention for a command line is to follow "git cmdname
--options" with revisions followed by an optional "--"
disambiguator and then finally pathspecs. When "--" is not there,
we make sure early ones are all interpretable as revs (and do not
look like paths) and later ones are the other way around. A
pathspec with "magic" (e.g. ":/p/a/t/h" that matches p/a/t/h from
the top-level of the working tree, no matter what subdirectory you
are working from) are conservatively judged as "not a path", which
required disambiguation more often. The command line parser
learned to say "it's a pathspec" a bit more often when the syntax
looks like so.
* jk/pathspec-magic-disambiguation:
verify_filename(): flip order of checks
verify_filename(): treat ":(magic)" as a pathspec
check_filename(): handle ":^" path magic
check_filename(): use skip_prefix
check_filename(): refactor ":/" handling
t4208: add check for ":/" without matching file
Introduce '--show-stash' and its configuration option 'status.showStash'
to allow git-status to show information about currently stashed entries.
Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Most of the time, a 'stash entry' is called a 'stash'. Lets try to make
this more consistent and use 'stash entry' instead.
Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
`for_each_bisect_ref()` is called by `for_each_bad_bisect_ref()` with
a term "bad". This used to make it call `for_each_ref_in_submodule()`
with a prefix "refs/bisect/bad". But the latter is the name of the
reference that is being sought, so the empty string was being passed
to the callback as the trimmed refname. Moreover, this questionable
practice was turned into an error by
b9c8e7f2fb prefix_ref_iterator: don't trim too much, 2017-05-22
It makes more sense (and agrees better with the documentation of
`--bisect`) for the callers to receive the full reference names. So
* Add a new function, `for_each_fullref_in_submodule()`, to the refs
API. This plugs a gap in the existing functionality, analogous to
`for_each_fullref_in()` but accepting a `submodule` argument.
* Change `for_each_bad_bisect_ref()` to call the new function rather
than `for_each_ref_in_submodule()`.
* Add a test.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace occurrences of `free(ptr); ptr = NULL` which weren't caught by
the coccinelle rule. These fall into two categories:
- free/NULL assignments one after the other which coccinelle all put
on one line, which is functionally equivalent code, but very ugly.
- manually spotted occurrences where the NULL assignment isn't right
after the free() call.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A follow-up to the existing "expression" rule added in an earlier
change. This manually excludes a few occurrences, mostly things that
resulted in many FREE_AND_NULL() on one line, that'll be manually
fixed in a subsequent change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A follow-up to the existing "type" rule added in an earlier
change. This catches some occurrences that are missed by the previous
rule.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Apply the result of the just-added coccinelle rule. This manually
excludes a few occurrences, mostly things that resulted in many
FREE_AND_NULL() on one line, that'll be manually fixed in a subsequent
change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A negated character class that does not include '/', e.g. [^a-z]:
- Should match '/' when doing "wildmatch"
- Should not match '/' when doing "pathmatch"
Add two tests to cover these cases.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a FREE_AND_NULL() wrapper marco for the common pattern of freeing
a pointer and assigning NULL to it right afterwards.
The implementation is similar to the (currently unused) XDL_PTRFREE
macro in xdiff/xmacros.h added in commit 3443546f6e ("Use a *real*
built-in diff generator", 2006-03-24). The only difference is that
free() is called unconditionally, see [1].
See [2] for a suggested alternative which does this via a function
instead of a macro. As covered in replies to that message, while it's
a viable approach, it would introduce caveats which this approach
doesn't have, so that potential change is left to a future follow-up
change.
This merely allows us to translate exactly what we're doing now to a
less verbose & idiomatic form using a macro, while guaranteeing that
we don't introduce any functional changes.
1. <alpine.DEB.2.20.1608301948310.129229@virtualbox>
(http://public-inbox.org/git/alpine.DEB.2.20.1608301948310.129229@virtualbox/)
2. <20170610032143.GA7880@starla>
(https://public-inbox.org/git/20170610032143.GA7880@starla/)
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we convert seconds-since-epochs timestamps into a
broken-down "struct tm", we do so by adjusting the timestamp
according to the known offset and then using gmtime() to
break down the result. This means that the resulting struct
"knows" that it's in GMT, even though the time it represents
is adjusted for a different zone. The fields where it stores
this data are not portably accessible, so we have no way to
override them to tell them the real zone info.
For the most part, this works. Our date-formatting routines
don't pay attention to these inaccessible fields, and use
the same tz info we provided for adjustment. The one
exception is when we call strftime(), whose %Z format
reveals this hidden timezone data.
We solved that by always showing the empty string for %Z.
This is allowed by POSIX, but not very helpful to the user.
We can't make this work in the general case, as there's no
portable function for setting an arbitrary timezone (and
anyway, we don't have the zone name for the author zones,
only their offsets).
But for the special case of the "-local" formats, we can
just skip the adjustment and use localtime() instead of
gmtime(). This makes --date=format-local:%Z work correctly,
showing the local timezone instead of an empty string.
The new test checks the result for "UTC", our default
test-lib value for $TZ. Using something like EST5 might be
more interesting, but the actual zone string is
system-dependent (for instance, on my system it expands to
just EST). Hopefully "UTC" is vanilla enough that every
system treats it the same.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We already test that "%z" and "%Z" show the right thing, but
we don't actually check that the time we display is the
correct one. Let's add two new tests:
1. Test that "format:" shows the time in the author's
timezone, just like the other time formats.
2. Test that "format-local:" shows time in the local
timezone. We don't want to use our normal UTC for this,
because its offset is zero (so the result would be
"correct" even if the code forgot to apply the offset
or applied it in the wrong direction).
We'll use the EST5 zone, which is already used
elsewhere in the script (and so is assumed to be
available everywhere).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no portable way to pass timezone information to strftime. Add
parameters for timezone offset and name to strbuf_addftime and let it
handle the timezone-related format specifiers %z and %Z internally.
Callers can opt out for %Z by passing NULL as timezone name. %z is
always handled internally -- this helps on Windows, where strftime would
expand it to a timezone name (same as %Z), in violation of POSIX.
Modifiers are not handled, e.g. %Ez is still passed to strftime.
Use an empty string as timezone name in show_date (the only current
caller) for now because we only have the timezone offset in non-local
mode. POSIX allows %Z to resolve to an empty string in case of missing
information.
Helped-by: Ulrich Mueller <ulm@gentoo.org>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We added an FREAD_READS_DIRECTORIES Makefile knob long ago
in cba22528f (Add compat/fopen.c which returns NULL on
attempt to open directory, 2008-02-08) to handle systems
where reading from a directory returned garbage. This works
by catching the problem at the fopen() stage and returning
NULL.
More recently, we found that there is a class of systems
(including Linux) where fopen() succeeds but fread() fails.
Since the solution is the same (having fopen return NULL),
they use the same Makefile knob as of e2d90fd1c
(config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and
FreeBSD, 2017-05-03).
This works fine except for one thing: the autoconf test in
configure.ac to set FREAD_READS_DIRECTORIES actually checks
whether fread succeeds. Which means that on Linux systems,
the knob isn't set (and we even override the config.mak.uname
default). t1308 catches the failure.
We can fix this by tweaking the autoconf test to cover both
cases. In theory we might care about the distinction between
the traditional "fread reads directories" case and the new
"fopen opens directories". But since our solution catches
the problem at the fopen stage either way, we don't actually
need to know the difference. The "fopen" case is a superset.
This does mean the FREAD_READS_DIRECTORIES name is slightly
misleading. Probably FOPEN_OPENS_DIRECTORIES would be more
accurate. But it would be disruptive to simply change the
name (people's existing build configs would fail), and it's
not worth the complexity of handling both. Let's just add a
comment in the knob description.
Reported-by: Øyvind A. Holm <sunny@sunbase.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'git_config_with_options()' takes a 'config_options' struct which
contains feilds for 'git_dir' and 'commondir'. If those feilds happen
to be NULL the config machinery falls back to querying global repository
state. Let's change this and instead use these fields in the
'config_options' struct explicilty all the time. Since the API is
slightly changing to require these two fields to be set if callers want
the config machinery to load the repository's config, let's change the
name to 'config_with_optison()'. This allows the config machinery to
not implicitly rely on any global repository state.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Worktrees present an interesting problem when it comes to the config.
Historically we could assume that the per-repository config lives at
'gitdir/config', but since worktrees were introduced this isn't the case
anymore. There is currently no way to specify per-worktree
configuration, and as such the repository config is shared with all
worktrees and is located at 'commondir/config'.
Many users of the config machinery correctly set
'config_options.git_dir' with the repository's commondir, allowing the
config to be properly loaded when operating in a worktree. But other's,
like 'read_early_config()', set 'config_options.git_dir' with the
repository's gitdir which can be incorrect when using worktrees.
To fix this issue, and to make things less ambiguous, lets add a
'commondir' field to the 'config_options' struct and have all callers
properly set both the 'git_dir' and 'commondir' fields so that the
config machinery is able to properly find the repository's config.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently 'discover_git_directory' only looks at the gitdir to determine
if a git directory was discovered. This causes a problem in the event
that the gitdir which was discovered was in fact a per-worktree git
directory and not the common git directory. This is because the
repository config, which is checked to verify the repository's format,
is stored in the commondir and not in the per-worktree gitdir. Correct
this behavior by checking the config stored in the commondir.
It will also be of use for callers to have access to the commondir, so
lets also return that upon successfully discovering a git directory.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stop including config.h by default in cache.h. Instead only include
config.h in those files which require use of the config system.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since there is no implementation of the function 'git_config_iter',
let's stop exporting it and remove the prototype from config.h.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move all config related declarations from cache.h to a new config.h
header file. This makes cache.h smaller and allows for the opportunity
in a following patch to only include config.h when needed.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit 46f0344 ("sha1_file: support reading from a loose object of
unknown type", 2015-05-06), "struct object_info" gained a "typename"
field that could represent a type name from a loose object file, whether
valid or invalid, as opposed to the existing "typep" which could only
represent valid types. Some relatively complex manipulations were added
to avoid breaking packed_object_info() without modifying it, but it is
much easier to just teach packed_object_info() about the new field.
Therefore, teach packed_object_info() as described above.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some submodule tests do some setup outside of a test_expect
block. This is bad because we won't actually check the
outcome of those commands. But it's doubly so because "git
add submodule" now produces a warning to stderr, which is
not suppressed by the test scripts in non-verbose mode.
This patch does the minimal to fix the annoying warnings.
All three of these scripts could use more cleanup of related
setup.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's an easy mistake to add a repository inside another
repository, like:
git clone $url
git add .
The resulting entry is a gitlink, but there's no matching
.gitmodules entry. Trying to use "submodule init" (or clone
with --recursive) doesn't do anything useful. Prior to
v2.13, such an entry caused git-submodule to barf entirely.
In v2.13, the entry is considered "inactive" and quietly
ignored. Either way, no clone of your repository can do
anything useful with the gitlink without the user manually
adding the submodule config.
In most cases, the user probably meant to either add a real
submodule, or they forgot to put the embedded repository in
their .gitignore file.
Let's issue a warning when we see this case. There are a few
things to note:
- the warning will go in the git-add porcelain; anybody
wanting to do low-level manipulation of the index is
welcome to create whatever funny states they want.
- we detect the case by looking for a newly added gitlink;
updates via "git add submodule" are perfectly reasonable,
and this avoids us having to investigate .gitmodules
entirely
- there's a command-line option to suppress the warning.
This is needed for git-submodule itself (which adds the
entry before adding any submodule config), but also
provides a mechanism for other scripts doing
submodule-like things.
We could make this a hard error instead of a warning.
However, we do add lots of sub-repos in our test suite. It's
not _wrong_ to do so. It just creates a state where users
may be surprised. Pointing them in the right direction with
a gentle hint is probably the best option.
There is a config knob that can disable the (long) hint. But
I intentionally omitted a config knob to disable the warning
entirely. Whether the warning is sensible or not is
generally about context, not about the user's preferences.
If there's a tool or workflow that adds gitlinks without
matching .gitmodules, it should probably be taught about the
new command-line option, rather than blanket-disabling the
warning.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some options specific for `git svn commit-diff` where not documented
so far.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Eric Wong <e@80x24.org>
We often try to open a file for reading whose existence is
optional, and silently ignore errors from open/fopen; report such
errors if they are not due to missing files.
* nd/fopen-errors:
mingw_fopen: report ENOENT for invalid file names
mingw: verify that paths are not mistaken for remote nicknames
log: fix memory leak in open_next_file()
rerere.c: move error_errno() closer to the source system call
print errno when reporting a system call error
wrapper.c: make warn_on_inaccessible() static
wrapper.c: add and use fopen_or_warn()
wrapper.c: add and use warn_on_fopen_errors()
config.mak.uname: set FREAD_READS_DIRECTORIES for Darwin, too
config.mak.uname: set FREAD_READS_DIRECTORIES for Linux and FreeBSD
clone: use xfopen() instead of fopen()
use xfopen() in more places
git_fopen: fix a sparse 'not declared' warning
Many commands learned to pay attention to submodule.recurse
configuration.
* sb/submodule-blanket-recursive:
builtin/fetch.c: respect 'submodule.recurse' option
builtin/push.c: respect 'submodule.recurse' option
builtin/grep.c: respect 'submodule.recurse' option
Introduce 'submodule.recurse' option for worktree manipulators
submodule loading: separate code path for .gitmodules and config overlay
reset/checkout/read-tree: unify config callback for submodule recursion
submodule test invocation: only pass additional arguments
submodule recursing: do not write a config variable twice
Our code often opens a path to an optional file, to work on its
contents when we can successfully open it. We can ignore a failure
to open if such an optional file does not exist, but we do want to
report a failure in opening for other reasons (e.g. we got an I/O
error, or the file is there, but we lack the permission to open).
The exact errors we need to ignore are ENOENT (obviously) and
ENOTDIR (less obvious). Instead of repeating comparison of errno
with these two constants, introduce a helper function to do so.
* jc/noent-notdir:
treewide: use is_missing_file_error() where ENOENT and ENOTDIR are checked
compat-util: is_missing_file_error()
Similar functions exist in apply.c and builtin/show-branch.c for
counting the number of slashes in a string. Also in the later
patches, we introduce a third caller for the same. Hence, we unify
it now by cleaning the existing functions and declaring a common
function count_slashes in dir.h and implementing it in dir.c to
remove this code duplication.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 28ed9830b1 (get_packed_ref_cache(): assume "packed-refs" won't
change while locked, 2017-05-22) assumes that the "packed-refs" file
cannot change while we hold the lock. That assumption is
justified *if* the lock has been held the whole time since the
"packed-refs" file was last read.
But in `lock_packed_refs()`, we ourselves lock the "packed-refs" file
and then call `get_packed_ref_cache()` to ensure that the cache agrees
with the file. The intent is to guard against the possibility that
another process changed the "packed-refs" file the moment before we
locked it.
This check was defeated because `get_packed_ref_cache()` saw that the
file was locked, and therefore didn't do the `stat_validity_check()`
that we want.
The mistake was compounded with a misleading comment in
`lock_packed_refs()` claiming that it was doing the right thing. That
comment came from an earlier draft of the mh/packed-ref-store-prep
patch series when the commits were in a different order.
So instead:
* Extract a function `validate_packed_ref_cache()` that does the
validity check independent of whether the lock is held.
* Change `get_packed_ref_cache()` to call the new function, but only
if the lock *isn't* held.
* Change `lock_packed_refs()` to call the new function in any case
before calling `get_packed_ref_cache()`.
* Fix the comment in `lock_packed_refs()`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In 618e613a70, 10 years ago, the default for pack depth used for
git-pack-objects and git-repack was changed from 10 to 50, while
leaving fast-import's default to 10.
There doesn't seem to be a reason besides oversight for the change not
having happened in fast-import as well.
Interestingly, fast-import uses pack.depth when it's set, and the
git-config manual says the default for pack.depth is 50. While the
git-fast-import manual does say the default depth is 10, the
inconsistency is also confusing.
Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A `--setup` step in `git filter-branch` makes it much easier to
define the initial values of variables used in the real filters.
Also sourcing/defining utility functions here instead of
`--env-filter` improves performance and minimizes clogging the
output in case of errors.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the missing documentation for `git svn init --ignore-refs`.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Eric Wong <e@80x24.org>
Since commit 99caeed05 (Let 'git <command> -h' show usage
without a git dir, 2009-11-09), the git wrapper handles "-h"
specially, skipping any repository setup but still calling
the builtin's cmd_foo() function. This means that every
cmd_foo() must be ready to handle this case, but we don't
have any systematic tests. This led to "git am -h" being
broken for some time without anybody noticing.
This patch just tests that "git foo -h" works for every
builtin, where we see a 129 exit code (the normal code for
our usage() helper), and that the word "usage" appears in
the output.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It can be useful in the test suite to be able to iterate
over the list of builtins. We could do this with some
Makefile magic. But since the authoritative list is in the
commands array inside git.c, and since this could also be
handy for debugging, let's add a hidden command-line option
to dump that list.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "git version" command didn't traditionally accept any
options, and in fact ignores any you give it. When we added
simple option parsing for "--build-options" in 6b9c38e14, we
didn't improve this; we just loop over the arguments and
pick out the one we recognize.
Instead, let's move to a real parsing loop, complain about
nonsense options, and recognize conventions like "-h".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git $builtin -h" bypasses the usual repository setup and calls the
cmd_$builtin() function, expecting it to show the help text.
Unfortunately the commands in the log- and the diff- family want to
call into the revisions machinery, which by definition needs to have
a repository already discovered. Strictly speaking, they may not
need a repository only for parsing "-h", but it is a good discipline
to future-proof codepath to ensure that setup_revisions() is called
after we know that a repository is there.
Handle the "git $builtin -h" special case very early in these
commands to work around potential issues.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert notes_merge and notes_merge_commit to use struct object_id.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the tested repo has an index.lock file it should be removed. This
file may be present if e.g. git-status previously crashed in that
repo, and it will make a lot of git commands fail. Let's try harder
and remove the lock.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git pull --rebase --autostash" didn't auto-stash when the local history
fast-forwards to the upstream.
* tb/pull-ff-rebase-autostash:
pull: ff --rebase --autostash works in dirty repo
The internal logic used in "git blame" has been libified to make it
easier to use by cgit.
* js/blame-lib: (29 commits)
blame: move entry prepend to libgit
blame: move scoreboard setup to libgit
blame: move scoreboard-related methods to libgit
blame: move fake-commit-related methods to libgit
blame: move origin-related methods to libgit
blame: move core structures to header
blame: create entry prepend function
blame: create scoreboard setup function
blame: create scoreboard init function
blame: rework methods that determine 'final' commit
blame: wrap blame_sort and compare_blame_final
blame: move progress updates to a scoreboard callback
blame: make sanity_check use a callback in scoreboard
blame: move no_whole_file_rename flag to scoreboard
blame: move xdl_opts flags to scoreboard
blame: move show_root flag to scoreboard
blame: move reverse flag to scoreboard
blame: move contents_from to scoreboard
blame: move copy/move thresholds to scoreboard
blame: move stat counters to scoreboard
...
The implementation of "ref" API around the "packed refs" have been
cleaned up, in preparation for further changes.
* mh/packed-ref-store-prep: (25 commits)
cache_ref_iterator_begin(): avoid priming unneeded directories
ref-filter: limit traversal to prefix
create_ref_entry(): remove `check_name` option
refs_ref_iterator_begin(): handle `GIT_REF_PARANOIA`
read_packed_refs(): report unexpected fopen() failures
read_packed_refs(): do more of the work of reading packed refs
get_packed_ref_cache(): assume "packed-refs" won't change while locked
should_pack_ref(): new function, extracted from `files_pack_refs()`
ref_update_reject_duplicates(): add a sanity check
ref_update_reject_duplicates(): use `size_t` rather than `int`
ref_update_reject_duplicates(): expose function to whole refs module
ref_transaction_prepare(): new optional step for reference updates
ref_transaction_commit(): check for valid `transaction->state`
files_transaction_cleanup(): new helper function
files_ref_store: put the packed files lock directly in this struct
files-backend: move `lock` member to `files_ref_store`
lockfile: add a new method, is_lock_file_locked()
ref_store: take a `msg` parameter when deleting references
refs: use `size_t` indexes when iterating over ref transaction updates
refs_ref_iterator_begin(): don't check prefixes redundantly
...
Make the "indent" heuristics the default in "diff" and diff.indentHeuristics
configuration variable an escape hatch for those who do no want it.
* mb/diff-default-to-indent-heuristics:
add--interactive: drop diff.indentHeuristic handling
diff: enable indent heuristic by default
diff: have the diff-* builtins configure diff before initializing revisions
diff: make the indent heuristic part of diff's basic configuration
The timestamp of the index file is now taken after the file is
closed, to help Windows, on which a stale timestamp is reported by
fstat() on a file that is opened for writing and data was written
but not yet closed.
* jh/close-index-before-stat:
read-cache: close index.lock in do_write_index
The interaction of "url.*.insteadOf" and custom URL scheme's
whitelisting is now documented better.
* jk/url-insteadof-config:
docs/config: mention protocol implications of url.insteadOf
Update the C style recommendation for notes for translators, as
recent versions of gettext tools can work with our style of
multi-line comments.
* ab/c-translators-comment-style:
C style: use standard style for "TRANSLATORS" comments
The "collision detecting" SHA-1 implementation shipped with 2.13
was quite broken on some big-endian platforms and/or platforms that
do not like unaligned fetches. Update to the upstream code which
has already fixed these issues.
* ab/sha1dc-maint:
sha1dc: update from upstream
The internal implementation of "git grep" has seen some clean-up.
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
The result from "git diff" that compares two blobs, e.g. "git diff
$commit1:$path $commit2:$path", used to be shown with the full
object name as given on the command line, but it is more natural to
use the $path in the output and use it to look up .gitattributes.
* jk/diff-blob:
diff: use blob path for blob/file diffs
diff: use pending "path" if it is available
diff: use the word "path" instead of "name" for blobs
diff: pass whole pending entry in blobinfo
handle_revision_arg: record paths for pending objects
handle_revision_arg: record modes for "a..b" endpoints
t4063: add tests of direct blob diffs
get_sha1_with_context: dynamically allocate oc->path
get_sha1_with_context: always initialize oc->symlink_path
sha1_name: consistently refer to object_context as "oc"
handle_revision_arg: add handle_dotdot() helper
handle_revision_arg: hoist ".." check out of range parsing
handle_revision_arg: stop using "dotdot" as a generic pointer
handle_revision_arg: simplify commit reference lookups
handle_revision_arg: reset "dotdot" consistently
"git clean -d" used to clean directories that has ignored files,
even though the command should not lose ignored ones without "-x".
"git status --ignored" did not list ignored and untracked files
without "-uall". These have been corrected.
* sl/clean-d-ignored-fix:
clean: teach clean -d to preserve ignored paths
dir: expose cmp_name() and check_contains()
dir: hide untracked contents of untracked dirs
dir: recurse into untracked dirs for ignored files
t7061: status --ignored should search untracked dirs
t7300: clean -d should skip dirs with ignored files
"foo\bar\baz" in "git fetch foo\bar\baz", even though there is no
slashes in it, cannot be a nickname for a remote on Windows, as
that is likely to be a pathname on a local filesystem.
* js/bs-is-a-dir-sep-on-windows:
Windows: do not treat a path with backslashes as a remote's nick name
mingw.h: permit arguments with side effects for is_dir_sep
Add missing completions for git config advice:
* advice.amWorkDir
* advice.pushAlreadyExists
* advice.pushFetchFirst
* advice.pushNeedsForce
* advice.pushNonFFCurrent
* advice.pushNonFFMatching
* advice.pushUpdateRejected
* advice.rmHints
* advice.statusUoption
Remove completion for git config advice.pushNonFastForward,
since it was renamed to pushUpdateRejected in 1184564eac.
The config still works, but is no longer part of the documentation.
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add missing completions for git config core:
* core.checkStat
* core.commentChar
* core.hideDotFiles
* core.hooksPath
* core.packedRefsTimeout
* core.precomposeUnicode
* core.protectHFS
* core.protectNTFS
* core.splitIndex
* core.sshCommand
Note that some configs are only used for some platforms
(hideDotFiles on Windows and precomposeUnicode on Mac).
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unless --no-validate is passed, send-email will invoke
$repo->repo_path() in its search for a validate hook regardless of
whether a Git repo is actually present. Teach send-email to first check
for repo existence.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows, certain characters are prohibited in file names, most
prominently the colon. When fopen() is called with such an invalid file
name, the underlying Windows API actually reports a particular error,
but since there is no suitable errno value, this error is translated
to EINVAL. Detect the case and report ENOENT instead.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This added test case simply verifies that users will not be bothered
with bogus complaints à la
warning: unable to access '.git/remotes/D:\repo': Invalid argument
when fetching from a Windows path (in this case, D:\repo).
[j6t: mark the new test as test_expect_failure]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert find_paths_generic and find_paths_multitree to use struct
object_id.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert diff_change to take a struct object_id. In addition convert the
function pointer type 'change_fn_t' to also take a struct object_id.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert diff_addremove to take a struct object_id. In addtion convert
the function pointer type 'add_remove_fn_t' to also take a struct
object_id.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the remaining parts of grep to use struct object_id.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert most of the static functions to use struct object_id. In
addition, convert copy_notes_for_rewrite and its callers.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make get_note return a pointer to a const struct object_id. Add a
defensive check to ensure we don't accidentally dereference a NULL
pointer.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert several portions of the internals of the code to struct
object_id. Introduce two macros to denote the different constants in
the code: KEY_INDEX for the last byte of the object ID, and
FANOUT_PATH_SEPARATORS for the number of possible path separators (on
Unix, "/"). While these constants are both 19 (one less than the number
of bytes in the hash), distinguish them to make the code more
understandable, and define them logically based on their intended
purpose.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add support for v2 of the PCRE API. This is a new major version of
PCRE that came out in early 2015[1].
The regular expression syntax is the same, but while the API is
similar, pretty much every function is either renamed or takes
different arguments. Thus using it via entirely new functions makes
sense, as opposed to trying to e.g. have one compile_pcre_pattern()
that would call either PCRE v1 or v2 functions.
Git can now be compiled with either USE_LIBPCRE1=YesPlease or
USE_LIBPCRE2=YesPlease, with USE_LIBPCRE=YesPlease currently being a
synonym for the former. Providing both is a compile-time error.
With earlier patches to enable JIT for PCRE v1 the performance of the
release versions of both libraries is almost exactly the same, with
PCRE v2 being around 1% slower.
However after I reported this to the pcre-dev mailing list[2] I got a
lot of help with the API use from Zoltán Herczeg, he subsequently
optimized some of the JIT functionality in v2 of the library.
Running the p7820-grep-engines.sh performance test against the latest
Subversion trunk of both, with both them and git compiled as -O3, and
the test run against linux.git, gives the following results. Just the
/perl/ tests shown:
$ GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_COMMAND='grep -q LIBPCRE2 Makefile && make -j8 USE_LIBPCRE2=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre2/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre2/inst/lib || make -j8 USE_LIBPCRE=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre/inst/lib' ./run HEAD~5 HEAD~ HEAD p7820-grep-engines.sh
[...]
Test HEAD~5 HEAD~ HEAD
-----------------------------------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.31(1.10+0.48) 0.21(0.35+0.56) -32.3% 0.21(0.34+0.55) -32.3%
7820.7: perl grep '^how to' 0.56(2.70+0.40) 0.24(0.64+0.52) -57.1% 0.20(0.28+0.60) -64.3%
7820.11: perl grep '[how] to' 0.56(2.66+0.38) 0.29(0.95+0.45) -48.2% 0.23(0.45+0.54) -58.9%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 1.02(5.77+0.42) 0.31(1.02+0.54) -69.6% 0.23(0.50+0.54) -77.5%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.38(1.57+0.42) 0.27(0.85+0.46) -28.9% 0.21(0.33+0.57) -44.7%
See commit ("perf: add a comparison test of grep regex engines",
2017-04-19) for details on the machine the above test run was executed
on.
Here HEAD~2 is git with PCRE v1 without JIT, HEAD~ is PCRE v1 with
JIT, and HEAD is PCRE v2 (also with JIT). See previous commits of mine
mentioning p7820-grep-engines.sh for more details on the test setup.
For ease of readability, a different run just of HEAD~ (PCRE v1 with
JIT v.s. PCRE v2), again with just the /perl/ tests shown:
[...]
Test HEAD~ HEAD
----------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.21(0.42+0.52) 0.21(0.31+0.58) +0.0%
7820.7: perl grep '^how to' 0.25(0.65+0.50) 0.20(0.31+0.57) -20.0%
7820.11: perl grep '[how] to' 0.30(0.90+0.50) 0.23(0.46+0.53) -23.3%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 0.30(1.19+0.38) 0.23(0.51+0.51) -23.3%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.27(0.84+0.48) 0.21(0.34+0.57) -22.2%
I.e. the two are either neck-to-neck, but PCRE v2 usually pulls ahead,
when it does it's around 20% faster.
A brief note on thread safety: As noted in pcre2api(3) & pcre2jit(3)
the compiled pattern can be shared between threads, but not some of
the JIT context, however the grep threading support does all pattern &
JIT compilation in separate threads, so this code doesn't need to
concern itself with thread safety.
See commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09) for the
initial addition of PCRE v1. This change follows some of the same
patterns it did (and which were discussed on list at the time),
e.g. mocking up types with typedef instead of ifdef-ing them out when
USE_LIBPCRE2 isn't defined. This adds some trivial memory use to the
program, but makes the code look nicer.
1. https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html
2. https://lists.exim.org/lurker/thread/20170419.172322.833ee099.en.html
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Amend my change earlier in this series ("grep: add support for the
PCRE v1 JIT API", 2017-04-11) to un-break the build on PCRE v1
versions later than 8.31 compiled without --enable-jit.
As explained in that change and a later compatibility change in this
series ("grep: un-break building with PCRE < 8.32", 2017-05-10) the
pcre_jit_exec() function is a faster path to execute the JIT.
Unfortunately there's no compatibility stub for that function compiled
into the library if pcre_config(PCRE_CONFIG_JIT, &ret) would return 0,
and no macro that can be used to check for it, so the only portable
option to support builds without --enable-jit is via a new
NO_LIBPCRE1_JIT=UnfortunatelyYes Makefile option[1].
Another option would be to make the JIT opt-in via
USE_LIBPCRE1_JIT=YesPlease, after all it's not a default option of
PCRE v1.
I think it makes more sense to make it opt-out since even though it's
not a default option, most packagers of PCRE seem to turn it on by
default, with the notable exception of the MinGW package.
Make the MinGW platform work by default by changing the build defaults
to turn on NO_LIBPCRE1_JIT=UnfortunatelyYes. It is the only platform
that turns on USE_LIBPCRE=YesPlease by default, see commit
df5218b4c3 ("config.mak.uname: support MSys2", 2016-01-13) for that
change.
1. "How do I support pcre1 JIT on all
versions?" (https://lists.exim.org/lurker/thread/20170601.103148.10253788.en.html)
2. https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-pcre/PKGBUILD
(referenced from "Re: PCRE v2 compile error, was Re: What's cooking
in git.git (May 2017, #01; Mon, 1)";
<alpine.DEB.2.20.1705021756530.3480@virtualbox>)
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The closest mapping from the boolean 'submodule.recurse' set to "yes"
to the variety of submodule push modes is "on-demand", so implement that.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In builtin/grep.c we parse the config before evaluating the command line
options. This makes the task of teaching grep to respect the new config
option 'submodule.recurse' very easy by just parsing that option.
As an alternative I had implemented a similar structure to treat
submodules as the fetch/push command have, including
* aligning the meaning of the 'recurse_submodules' to possible submodule
values RECURSE_SUBMODULES_* as defined in submodule.h.
* having a callback to parse the value and
* reacting to the RECURSE_SUBMODULES_DEFAULT state that was the initial
state.
However all this is not needed for a true boolean value, so let's keep
it simple. However this adds another place where "submodule.recurse" is
parsed.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Any command that understands '--recurse-submodules' can have its
default changed to true, by setting the new 'submodule.recurse'
option.
This patch includes read-tree/checkout/reset for working tree
manipulating commands. Later patches will cover other commands.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Normal users shouldn't ever call submodule--helper, but it
doesn't hurt to give them a normal usage message if they try
"-h".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We just say "Expected two arguments" when we get a different
number of arguments, but we can be slightly friendlier.
People shouldn't generally be running remote helpers
themselves, but curious users might say "git remote-ext -h".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Normally upload-archive forks off upload-archive--writer to
do the real work, and relays any errors back over the
sideband channel. This is a good thing when the command is
properly invoked remotely via ssh or git-daemon. But it's
confusing to curious users who try "git upload-archive -h".
Let's catch this invocation early and give a real usage
message, rather than spewing "-h does not appear to be a git
repository" amidst packet-lines. The chance of a false
positive due to a real client asking for the repo "-h" is
quite small.
Likewise, we'll catch "-h" in upload-archive--writer. People
shouldn't be invoking it manually, but it doesn't hurt to
give a sane message if they do.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-credential command only takes one argument: the
operation to perform. If we don't have one, we complain
immediately. But if we have one that we don't recognize, we
don't notice until after we've read the credential from
stdin. This is likely to confuse a user invoking "git
credential -h", as the program will hang waiting for their
input before showing anything.
Let's detect this case early. Likewise, we never noticed
when there are extra arguments beyond the one we're
expecting. Let's catch this with the same conditional.
Note that we don't need to handle "--help" similarly,
because the git wrapper does this before even calling
cmd_credential().
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The .gitmodules file is not supposed to have all the options available,
that are available in the configuration so separate it out.
A configuration option such as the hypothetical submodule.color.diff
that determines in which color a submodule change is printed,
is a very user specific thing, that the .gitmodules file should
not tamper with.
The .gitmodules file should only be used for settings that required
to setup the project in which the .gitmodules file is tracked. As the
minimum this would only include the name<->path mapping of the
submodule and its URL and branch.
Any further setting (such as 'fetch.recursesubmodules' or
'submodule.<name>.{update, ignore, shallow}') is not specific
to the project setup requirements, but rather is a distribution
of suggested developer configurations. In other areas of Git
a suggested developer configuration is not transported in-tree
but via other means. In an organisation this could be done
by deploying an opinionated system wide config (/etc/gitconfig)
or by putting the settings in the users home directory when
they start at the organisation. In open source projects this
is often accomplished via extensive READMEs (cf. our
SubmittingPatches/CodingGuidlines).
As a later patch in this series wants to introduce
a generic submodule recursion option, we want to make
sure that switch is not exposed via the gitmodules file.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The callback function is essentially duplicated 3 times. Remove all
of them and offer a new callback function, that lives in submodule.c
By putting the callback function there, we no longer need the function
'set_config_update_recurse_submodules', nor duplicate the global variable
in each builtin as well as submodule.c
In the three builtins we have different 2 ways how to load the .gitmodules
and config file, which are slightly different. git-checkout has to load
the submodule config all the time due to 23b4c7bcc5 (checkout: Use
submodule.*.ignore settings from .git/config and .gitmodules, 2010-08-28)
git-reset and git-read-tree do not respect these diff settings, so loading
the submodule configuration is optional. Also put that into submodule.c
for code deduplication.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a later patch we want to introduce a config option to trigger the
submodule recursing by default. As this option should be available and
uniform across all commands that deal with submodules we'd want to test
for this option in the submodule update library.
So instead of calling the whole test set again for
"git -c submodule.recurse foo" instead of "git foo --recurse-submodules",
we'd only want to introduce one basic test that tests if the option is
recognized and respected to not overload the test suite.
Change the test functions by taking only the argument and assemble the
command inside the test function by embedding the arguments into the
command that is "git $arguments --recurse-submodules".
It would be nice to do this for all functions in lib-submodule-update,
but we cannot do that for the non-recursing tests, as there we do not
just pass in a git command but whole functions. (See t3426 for example)
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The command line option for '--recurse-submodules' is implemented
using an OPTION_CALLBACK, which takes both the callback (that sets
the file static global variable) as well as passes the same file
static global variable to the option parsing machinery to assign it.
This is fixed in this commit by passing NULL as the variable. The
callback sets it instead
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ab/grep-preparatory-cleanup: (31 commits)
grep: assert that threading is enabled when calling grep_{lock,unlock}
grep: given --threads with NO_PTHREADS=YesPlease, warn
pack-objects: fix buggy warning about threads
pack-objects & index-pack: add test for --threads warning
test-lib: add a PTHREADS prerequisite
grep: move is_fixed() earlier to avoid forward declaration
grep: change internal *pcre* variable & function names to be *pcre1*
grep: change the internal PCRE macro names to be PCRE1
grep: factor test for \0 in grep patterns into a function
grep: remove redundant regflags assignments
grep: catch a missing enum in switch statement
perf: add a comparison test of log --grep regex engines with -F
perf: add a comparison test of log --grep regex engines
perf: add a comparison test of grep regex engines with -F
perf: add a comparison test of grep regex engines
perf: emit progress output when unpacking & building
perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do
grep: add tests to fix blind spots with \0 patterns
grep: prepare for testing binary regexes containing rx metacharacters
grep: add a test helper function for less verbose -f \0 tests
...
When "git checkout", "git merge", etc. manipulates the in-core
index, various pieces of information in the index extensions are
discarded from the original state, as it is usually not the case
that they are kept up-to-date and in-sync with the operation on the
main index. The untracked cache extension is copied across these
operations now, which would speed up "git status" (as long as the
cache is properly invalidated).
* dt/unpack-save-untracked-cache-extension:
unpack-trees: preserve index extensions
"git send-email" now uses Net::SMTP::SSL, which is obsolete, only
when needed. Recent versions of Net::SMTP can do TLS natively.
* dk/send-email-avoid-net-smtp-ssl-when-able:
send-email: Net::SMTP::SSL is obsolete, use only when necessary
A recent update to t5545-push-options.sh started skipping all the
tests in the script when a web server testing is disabled or
unavailable, not just the ones that require a web server. Non HTTP
tests have been salvaged to always run in this script.
* jc/skip-test-in-the-middle:
t5545: enhance test coverage when no http server is installed
test: allow skipping the remainder
The recently introduced "[includeIf "gitdir:$dir"] path=..."
mechansim has further been taught to take symlinks into account.
The directory "$dir" specified in "gitdir:$dir" may be a symlink to
a real location, not something that $(getcwd) may return. In such
a case, a realpath of "$dir" is compared with the real path of the
current repository to determine if the contents from the named path
should be included.
* ab/conditional-config-with-symlinks:
config: match both symlink & realpath versions in IncludeIf.gitdir:*
There is no good reason why "git fetch $there $sha1" should fail
when the $sha1 names an object at the tip of an advertised ref,
even when the other side hasn't enabled allowTipSHA1InWant.
* jt/fetch-allow-tip-sha1-implicitly:
fetch-pack: always allow fetching of literal SHA1s
"git send-email" learned to run sendemail-validate hook to inspect
and reject a message before sending it out.
* jt/send-email-validate-hook:
send-email: support validate hook
perf-test update.
* jh/memihash-opt:
p0004: don't error out if test repo is too small
p0004: don't abort if multi-threaded is too slow
p0004: use test_perf
p0004: avoid using pipes
p0004: simplify calls of test-lazy-init-name-hash
Code from "conversion using external process" codepath has been
extracted to a separate sub-process.[ch] module.
* bp/sub-process-convert-filter:
convert: update subprocess_read_status() to not die on EOF
sub-process: move sub-process functions into separate files
convert: rename reusable sub-process functions
convert: update generic functions to only use generic data structures
convert: separate generic structures and variables from the filter specific ones
convert: split start_multi_file_filter() into two separate functions
pkt-line: annotate packet_writel with LAST_ARG_MUST_BE_NULL
convert: move packet_write_line() into pkt-line as packet_writel()
pkt-line: add packet_read_line_gently()
pkt-line: fix packet_read_line() to handle len < 0 errors
convert: remove erroneous tests for errno == EPIPE
The "run-command" API implementation has been made more robust
against dead-locking in a threaded environment.
* bw/forking-and-threading:
usage.c: drop set_error_handle()
run-command: restrict PATH search to executable files
run-command: expose is_executable function
run-command: block signals between fork and execve
run-command: add note about forking and threading
run-command: handle dup2 and close errors in child
run-command: eliminate calls to error handling functions in child
run-command: don't die in child when duping /dev/null
run-command: prepare child environment before forking
string-list: add string_list_remove function
run-command: use the async-signal-safe execv instead of execvp
run-command: prepare command before forking
t0061: run_command executes scripts without a #! line
t5550: use write_script to generate post-update hook
Add perf-test for wildmatch.
* ab/perf-wildmatch:
perf: add test showing exponential growth in path globbing
perf: add function to setup a fresh test repo
"git describe --contains" penalized light-weight tags so much that
they were almost never considered. Instead, give them about the
same chance to be considered as an annotated tag that is the same
age as the underlying commit would.
* jc/name-rev-lw-tag:
name-rev: favor describing with tags and use committer date to tiebreak
name-rev: refactor logic to see if a new candidate is a better name
Simplify parse_pathspec() codepath and stop it from looking at the
default in-core index.
* bw/pathspec-sans-the-index:
pathspec: convert find_pathspecs_matching_against_index to take an index
pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP
ls-files: prevent prune_cache from overeagerly pruning submodules
pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag
submodule: add die_in_unpopulated_submodule function
pathspec: provide a more descriptive die message
Using the is_missing_file_error() helper introduced in the previous
step, update all hits from
$ git grep -e ENOENT --and -e ENOTDIR
There are codepaths that only check ENOENT, and it is possible that
some of them should be checking both. Updating them is kept out of
this step deliberately, as we do not want to change behaviour in this
step.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our code often opens a path to an optional file, to work on its
contents when we can successfully open it. We can ignore a failure
to open if such an optional file does not exist, but we do want to
report a failure in opening for other reasons (e.g. we got an I/O
error, or the file is there, but we lack the permission to open).
The exact errors we need to ignore are ENOENT (obviously) and
ENOTDIR (less obvious). Instead of repeating comparison of errno
with these two constants, introduce a helper function to do so.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tag objects, which are not reachable from any ref, that point at
missing objects were mishandled by "git gc" and friends (they
should silently be ignored instead)
* jk/ignore-broken-tags-when-ignoring-missing-links:
revision.c: ignore broken tags with ignore_missing_links
A test allowed both "git push" and "git receive-pack" on the other
end write their traces into the same file. This is OK on platforms
that allows atomically appending to a file opened with O_APPEND,
but on other platforms led to a mangled output, causing
intermittent test failures. This has been fixed by disabling
traces from "receive-pack" in the test.
* jk/alternate-ref-optim:
t5400: avoid concurrent writes into a trace file
"git interpret-trailers", when used as GIT_EDITOR for "git commit
-v", looked for and appended to a trailer block at the very end,
i.e. at the end of the "diff" output. The command has been
corrected to pay attention to the cut-mark line "commit -v" adds to
the buffer---the real trailer block should appear just before it.
* bm/interpret-trailers-cut-line-is-eom:
interpret-trailers: honor the cut line
Regression fix to topic recently merged to 'master'.
* pw/rebase-i-regression-fix:
rebase -i: add missing newline to end of message
rebase -i: silence stash apply
rebase -i: fix reflog message
"git for-each-ref --format=..." with %(HEAD) in the format used to
resolve the HEAD symref as many times as it had processed refs,
which was wasteful, and "git branch" shared the same problem.
* kn/ref-filter-branch-list:
ref-filter: resolve HEAD when parsing %(HEAD) atom
A few http:// links that are redirected to https:// in the
documentation have been updated to https:// links.
* jk/update-links-in-docs:
doc: use https links to Wikipedia to avoid http redirects
Git sometimes gives an advice in a rhetorical question that does
not require an answer, which can confuse new users and non native
speakers. Attempt to rephrase them.
* ja/do-not-ask-needless-questions:
git-filter-branch: be more direct in an error message
read-tree -m: make error message for merging 0 trees less smart aleck
usability: don't ask questions if no reply is required
Clarify documentation for include.path and includeIf.<condition>.path
configuration variables.
* jk/doc-config-include:
docs/config: consistify include.path examples
docs/config: avoid the term "expand" for includes
docs/config: give a relative includeIf example
docs/config: clarify include/includeIf relationship
Introduce the BUG() macro to improve die("BUG: ...").
* jk/bug-to-abort:
usage: add NORETURN to BUG() function definitions
config: complain about --local outside of a git repo
setup_git_env: convert die("BUG") to BUG()
usage.c: add BUG() function
Make sure our tests would pass when the sources are checked out
with "platform native" line ending convention by default on
Windows. Some "text" files out tests use and the test scripts
themselves that are meant to be run with /bin/sh, ought to be
checked out with eol=LF even on Windows.
* js/eol-on-ourselves:
t4051: mark supporting files as requiring LF-only line endings
Fix the remaining tests that failed with core.autocrlf=true
t3901: move supporting files into t/t3901/
completion: mark bash script as LF-only
git-new-workdir: mark script as LF-only
Fix build with core.autocrlf=true
"git read-tree -m" (no tree-ish) gave a nonsense suggestion "use
--empty if you want to clear the index". With "-m", such a request
will still fail anyway, as you'd need to name at least one tree-ish
to be merged.
* jc/read-tree-empty-with-m:
read-tree: "read-tree -m --empty" does not make sense
"pack-objects" can stream a slice of an existing packfile out when
the pack bitmap can tell that the reachable objects are all needed
in the output, without inspecting individual objects. This
strategy however would not work well when "--local" and other
options are in use, and need to be disabled.
* jk/disable-pack-reuse-when-broken:
t5310: fix "; do" style
pack-objects: disable pack reuse for object-selection options
Conversion from uchar[20] to struct object_id continues.
* bc/object-id: (53 commits)
object: convert parse_object* to take struct object_id
tree: convert parse_tree_indirect to struct object_id
sequencer: convert do_recursive_merge to struct object_id
diff-lib: convert do_diff_cache to struct object_id
builtin/ls-tree: convert to struct object_id
merge: convert checkout_fast_forward to struct object_id
sequencer: convert fast_forward_to to struct object_id
builtin/ls-files: convert overlay_tree_on_cache to object_id
builtin/read-tree: convert to struct object_id
sha1_name: convert internals of peel_onion to object_id
upload-pack: convert remaining parse_object callers to object_id
revision: convert remaining parse_object callers to object_id
revision: rename add_pending_sha1 to add_pending_oid
http-push: convert process_ls_object and descendants to object_id
refs/files-backend: convert many internals to struct object_id
refs: convert struct ref_update to use struct object_id
ref-filter: convert some static functions to struct object_id
Convert struct ref_array_item to struct object_id
Convert the verify_pack callback to struct object_id
Convert lookup_tag to struct object_id
...
Plug some leaks and updates internal API used to implement the
split index feature to make it easier to avoid such a leak in the
future.
* nd/split-index-unshare:
p3400: add perf tests for rebasing many changes
split-index: add and use unshare_split_index()
"git repack" learned to accept the --threads=<n> option and pass it
to pack-objects.
* jc/repack-threads:
repack: accept --threads=<n> and pass it down to pack-objects
API update.
* bw/dir-c-stops-relying-on-the-index:
dir: convert fill_directory to take an index
dir: convert read_directory to take an index
dir: convert read_directory_recursive to take an index
dir: convert open_cached_dir to take an index
dir: convert is_excluded to take an index
dir: convert prep_exclude to take an index
dir: convert add_excludes to take an index
dir: convert is_excluded_from_list to take an index
dir: convert last_exclude_matching_from_list to take an index
dir: convert dir_add* to take an index
dir: convert get_dtype to take index
dir: convert directory_exists_in_index to take index
dir: convert read_skip_worktree_file_from_index to take an index
dir: stop using the index compatibility macros
"git checkout --recurse-submodules" did not quite work with a
submodule that itself has submodules.
* sb/checkout-recurse-submodules:
submodule: properly recurse for read-tree and checkout
submodule: avoid auto-discovery in new working tree manipulator code
submodule_move_head: reuse child_process structure for futher commands
The looks_like_pathspec() check is much cheaper than
check_filename(), which actually stats the file. Since
either is sufficient for our return value, we should do the
cheaper one first, potentially short-circuiting the other.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
For commands that take revisions and pathspecs, magic
pathspecs like ":(exclude)foo" require the user to specify
a disambiguating "--", since they do not match a file in the
filesystem, like:
git grep foo -- :(exclude)bar
This makes them more annoying to use than they need to be.
We loosened the rules for wildcards in 28fcc0b71 (pathspec:
avoid the need of "--" when wildcard is used, 2015-05-02).
Let's do the same for pathspecs with long-form magic.
We already handle the short-forms ":/" and ":^" specially in
check_filename(), so we don't need to handle them here. And
in fact, we could do the same with long-form magic, parsing
out the actual filename and making sure it exists. But there
are a few reasons not to do it that way:
- the parsing gets much more complicated, and we'd want to
hand it off to the pathspec code. But that code isn't
ready to do this kind of speculative parsing (it's happy
to die() when it sees a syntactically invalid pathspec).
- not all pathspec magic maps to a filesystem path. E.g.,
:(attr) should be treated as a pathspec regardless of
what is in the filesystem
- we can be a bit looser with ":(" than with the
short-form ":/", because it is much less likely to have
a false positive. Whereas ":/" also means "search for a
commit with this regex".
Note that because the change is in verify_filename() and not
in its helper check_filename(), this doesn't affect the
verify_non_filename() case. I.e., if an item that matches
our new rule doesn't resolve as an object, we may fallback
to treating it as a pathspec (rather than complaining it
doesn't exist). But if it does resolve (e.g., as a file in
the index that starts with an open-paren), we won't then
complain that it's also a valid pathspec. This matches the
wildcard-exception behavior.
And of course in either case, one can always insert the "--"
to get more precise results.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We special-case "git log :/foo" to work when "foo" exists in
the working tree. But :^ (and its alias :!) do not get the
same treatment, requiring the user to supply a
disambiguating "--". Let's make them work without requiring
the user to type the "--".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This avoids some magic numbers (and we'll be adding more
similar calls in a minute).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We handle arguments with the ":/" pathspec magic specially,
making sure the name exists at the top-level. We'll want to
handle more pathspec magic in future patches, so let's do a
little rearranging to make that easier.
Instead of relying on an if/else cascade to avoid the
prefix_filename() call, we'll just set prefix to NULL.
Likewise, we'll get rid of the "name" variable entirely, and
just push the "arg" pointer forward to skip past the magic.
That means by the time we get to the prefix-handling, we're
set up appropriately whether we saw ":/" or not.
Note that this does impact the final error message we
produce when stat() fails, as it shows "arg" (which we'll
have modified to skip magic and include the prefix). This is
a good thing; the original message would say something like
"failed to stat ':/foo'", which is confusing (we tried to
stat "foo").
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The DWIM magic in check_filename() doesn't just recognize
":/". It actually makes sure that the file it points to
exists. t4208 checks only the case where the path is
present, not the opposite. Since the next patches will be
touching this area, let's add a test to make sure it
continues working.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Amend my change earlier in this series ("grep: add support for the
PCRE v1 JIT API", 2017-04-11) to un-break the build on PCRE v1
versions earlier than 8.20.
The 8.20 release was the first release to have JIT & pcre_jit_stack in
the headers, so a mock type needs to be provided for it on those
releases.
Now git should compile with all PCRE versions that it supported before
my JIT change.
I've tested it as far back as version 7.5 released on 2008-01-10, once
I got down to version 7.0 it wouldn't build anymore with GCC 7.1.1,
and I couldn't be bothered to anything older than 7.5 as I'm confident
that if the build breaks on those older versions it's not because of
my JIT change.
See the "un-break" change in this series ("grep: un-break building
with PCRE < 8.32", 2017-05-10) for why this isn't squashed into the
main PCRE JIT commit.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Amend my change earlier in this series ("grep: add support for the
PCRE v1 JIT API", 2017-04-11) to un-break the build on PCRE v1
versions earlier than 8.32.
The JIT support was added in version 8.20 released on 2011-10-21, but
it wasn't until 8.32 released on 2012-11-30 that the fast code path to
use the JIT via pcre_jit_exec() was added[1] (see also [2]).
This means that versions 8.20 through 8.31 could still use the JIT,
but supporting it on those versions would add to the already verbose
macro soup around JIT support it, and I don't expect that the use-case
of compiling a brand new git against a 5 year old PCRE is particularly
common, and if someone does that they can just get the existing
pre-JIT slow codepath.
So just take the easy way out and disable the JIT on any version older
than 8.32.
The reason this change isn't part of the initial change PCRE JIT
support is to have a cleaner history showing which parts of the
implementation are only used for ancient PCRE versions. This also
makes it easier to revert this change if we ever decide to stop
supporting those old versions.
1. http://www.pcre.org/original/changelog.txt ("28. Introducing a
native interface for JIT. Through this interface, the
compiled[...]")
2. https://bugs.exim.org/show_bug.cgi?id=2121
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the grep PCRE v1 code to use JIT when available. When PCRE
support was initially added in commit 63e7e9d8b6 ("git-grep: Learn
PCRE", 2011-05-09) PCRE had no JIT support, it was integrated into
8.20 released on 2011-10-21.
Enabling JIT support usually improves performance by more than
40%. The pattern compilation times are relatively slower, but those
relative numbers are tiny, and are easily made back in all but the
most trivial cases of grep. Detailed benchmarks & overview of
compilation times is at: http://sljit.sourceforge.net/pcre.html
With this change the difference in a t/perf/p7820-grep-engines.sh run
is, with just the /perl/ tests shown:
$ GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_OPTS='-j8 USE_LIBPCRE=YesPlease CC=~/perl5/installed/bin/gcc NO_R_TO_GCC_LINKER=YesPlease CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst LDFLAGS=-Wl,-rpath,/home/avar/g/pcre/inst/lib' ./run HEAD~ HEAD p7820-grep-engines.sh
Test HEAD~ HEAD
---------------------------------------------------------------------------------------
7820.3: perl grep 'how.to' 0.35(1.11+0.43) 0.23(0.42+0.46) -34.3%
7820.7: perl grep '^how to' 0.64(2.71+0.36) 0.27(0.66+0.44) -57.8%
7820.11: perl grep '[how] to' 0.63(2.51+0.42) 0.33(0.98+0.39) -47.6%
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 1.17(5.61+0.35) 0.34(1.08+0.46) -70.9%
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.43(1.52+0.44) 0.30(0.88+0.42) -30.2%
The conditional support for JIT is implemented as suggested in the
pcrejit(3) man page. E.g. defining PCRE_STUDY_JIT_COMPILE to 0 if it's
not present.
The implementation is relatively verbose because even if
PCRE_CONFIG_JIT is defined only a call to pcre_config() can determine
if the JIT is available, and if so the faster pcre_jit_exec() function
should be called instead of pcre_exec(), and a different (but not
complimentary!) function needs to be called to free pcre1_extra_info.
There's no graceful fallback if pcre_jit_stack_alloc() fails under
PCRE_CONFIG_JIT, instead the program will simply abort. I don't think
this is worth handling gracefully, it'll only fail in cases where
malloc() doesn't work, in which case we're screwed anyway.
That there's no assignment of `p->pcre1_jit_on = 0` when
PCRE_CONFIG_JIT isn't defined isn't a bug. The create_grep_pat()
function allocates the grep_pat allocates it with calloc(), so it's
guaranteed to be 0 when PCRE_CONFIG_JIT isn't defined.
I you're bisecting and find this change, check that your PCRE isn't
older than 8.32. This change intentionally broke really old versions
of PCRE, but that's fixed in follow-up commits.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a short -P option as a synonym for the longer --perl-regexp, for
consistency with the options the corresponding grep invocations
accept.
This was intentionally omitted in commit 727b6fc3ed ("log --grep:
accept --basic-regexp and --perl-regexp", 2012-10-03) for unspecified
future use.
Make it consistent with "grep" rather than to keep it open for future
use, and to avoid the confusion of -P meaning different things for
grep & log, as is the case with the -G option.
As noted in the aforementioned commit the --basic-regexp option can't
have a corresponding -G argument, as the log command already uses that
for -G<regex>.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Skip the administrative overhead of using pthreads when only using one
thread. Instead take the non-threaded path which would be taken under
NO_PTHREADS.
The threading support was initially added in commit
5b594f457a ("Threaded grep", 2010-01-25) with a hardcoded compile-time
number of 8 threads. Later the number of threads was made configurable
in commit 89f09dd34e ("grep: add --threads=<num> option and
grep.threads configuration", 2015-12-15).
That change did not add any special handling for --threads=1. Now we
take a slightly faster path by skipping thread handling entirely when
1 thread is requested.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the pattern compilation logic under threading so that grep
doesn't compile a pattern it never ends up using on the non-threaded
code path, only to compile it again N times for N threads which will
each use their own copy, ignoring the initially compiled pattern.
This redundant compilation dates back to the initial introduction of
the threaded grep in commit 5b594f457a ("Threaded grep",
2010-01-25).
There was never any reason for doing this redundant work other than an
oversight in the initial commit. Jeff King suggested on-list in
<20170414212325.fefrl3qdjigwyitd@sigill.intra.peff.net> that this
might be needed to check the pattern for sanity before threaded
execution commences.
That's not the case. The pattern is compiled under threading in
start_threads() before any concurrent execution has started by calling
pthread_create(), so if the pattern contains an error we still do the
right thing. I.e. die with one error before any threaded execution has
commenced, instead of e.g. spewing out an error for each N threads,
which could be a regression a change like this might inadvertently
introduce.
This change is not meant as an optimization, any performance gains
from this are in the hundreds to thousands of nanoseconds at most. If
we wanted more performance here we could just re-use the compiled
patterns in multiple threads (regcomp(3) is thread-safe), or partially
re-use them and the associated structures in the case of later PCRE
JIT changes.
Rather, it's just to make the code easier to reason about. It's
confusing to debug this under threading & non-threading when the
threading codepaths redundantly compile a pattern which is never used.
The reason the patterns are recompiled is as a side-effect of
duplicating the whole grep_opt structure, which is not thread safe,
writable, and munged during execution. The grep_opt structure then
points to the grep_pat structure where pattern or patterns are stored.
I looked into e.g. splitting the API into some "do & alloc threadsafe
stuff", "spawn thread", "do and alloc non-threadsafe stuff", but the
execution time of grep_opt_dup() & pattern compilation is trivial
compared to actually executing the grep, so there was no point. Even
with the more expensive JIT changes to follow the most expensive PCRE
patterns take something like 0.0X milliseconds to compile at most[1].
The undocumented --debug mode added in commit 17bf35a3c7 ("grep: teach
--debug option to dump the parse tree", 2012-09-13) still works
properly with this change. It only emits debugging info during pattern
compilation, which is now dumped by the pattern compiled just before
the first thread is started.
1. http://sljit.sourceforge.net/pcre.html
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the grep_{lock,unlock} functions to assert that num_threads is
true, instead of only locking & unlocking the pthread mutex lock when
it is.
These functions are never called when num_threads isn't true, this
logic has gone through multiple iterations since the initial
introduction of grep threading in commit 5b594f457a ("Threaded grep",
2010-01-25), but ever since then they'd only be called if num_threads
was true, so this check made the code confusing to read.
Replace the check with an assertion, so that it's clear to the reader
that this code path is never taken unless we're spawning threads.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a warning about missing thread support when grep.threads or
--threads is set to a non 0 (default) or 1 (no parallelism) value
under NO_PTHREADS=YesPlease.
This is for consistency with the index-pack & pack-objects commands,
which also take a --threads option & are configurable via
pack.threads, and have long warned about the same under
NO_PTHREADS=YesPlease.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix a buggy warning about threads under NO_PTHREADS=YesPlease. Due to
re-using the delta_search_threads variable for both the state of the
"pack.threads" config & the --threads option, setting "pack.threads"
but not supplying --threads would trigger the warning for both
"pack.threads" & --threads.
Solve this bug by resetting the delta_search_threads variable in
git_pack_config(), it might then be set by --threads again and be
subsequently warned about, as the test I'm changing here asserts.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test for the warning that's emitted when --threads or
pack.threads is provided under NO_PTHREADS=YesPlease. This uses the
new PTHREADS prerequisite.
The assertion for C_LOCALE_OUTPUT in the latter test is currently
redundant, since unlike index-pack the pack-objects warnings aren't
i18n'd. However they might be changed to be i18n'd in the future, and
there's no harm in future-proofing the test.
There's an existing bug in the implementation of pack-objects which
this test currently tests for as-is. Details about the bug & the fix
are included in a follow-up change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a PTHREADS prerequisite which is false when git is compiled with
NO_PTHREADS=YesPlease.
There's lots of custom code that runs when threading isn't available,
but before this prerequisite there was no way to test it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the is_fixed() function which are currently only used in
compile_regexp() earlier so it can be used in the PCRE family of
functions in a later change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the internal PCRE variable & function names to have a "1"
suffix. This is for preparation for libpcre2 support, where having
non-versioned names would be confusing.
An earlier change in this series ("grep: change the internal PCRE
macro names to be PCRE1", 2017-04-07) elaborates on the motivations
behind this change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the internal USE_LIBPCRE define, & build options flag to use a
naming convention ending in PCRE1, without changing the long-standing
USE_LIBPCRE Makefile flag which enables this code.
This is for preparation for libpcre2 support where having things like
USE_LIBPCRE and USE_LIBPCRE2 in any more places than we absolutely
need to for backwards compatibility with old Makefile arguments would
be confusing.
In some ways it would be better to change everything that now uses
USE_LIBPCRE to use USE_LIBPCRE1, and to make specifying
USE_LIBPCRE (or --with-pcre) an error. This would impose a one-time
burden on packagers of git to s/USE_LIBPCRE/USE_LIBPCRE1/ in their
build scripts.
However I'd like to leave the door open to making
USE_LIBPCRE=YesPlease eventually mean USE_LIBPCRE2=YesPlease,
i.e. once PCRE v2 is ubiquitous enough that it makes sense to make it
the default.
This code and the USE_LIBPCRE Makefile argument was added in commit
63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09). At the time there was
no indication that the PCRE project would release an entirely new &
incompatible API around 3 years later.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor the test for \0 in grep patterns into a function. Since commit
9eceddeec6 ("Use kwset in grep", 2011-08-21) any pattern containing a
\0 is considered fixed as regcomp() can't handle it.
This change makes later changes that make use of either has_null() or
is_fixed() (but not both) smaller.
While I'm at it make the comment conform to the style guide, i.e. add
an opening "/*\n".
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove redundant assignments to the "regflags" variable. This variable
is only used set under GREP_PATTERN_TYPE_ERE, so there's no need to
un-set it under GREP_PATTERN_TYPE_{FIXED,BRE,PCRE}.
Back in 5010cb5fcc[1], we did do "opt.regflags &= ~REG_EXTENDED" upon
seeing "-G" on the command line and flipped the bit on upon seeing
"-E", but I think that was perfectly sensible and it would have been a
bug if we didn't. They were part of the command line parsing that
could have seen "-E" on the command line earlier.
When cca2c172 ("git-grep: do not die upon -F/-P when
grep.extendedRegexp is set.", 2011-05-09) switched the command line
parsing to "read into a 'tentatively this is what we saw the last'
variable and then finally commit just once", we didn't touch
opt.regflags for PCRE and FIXED, but we still had to flip regflags
between BRE and ERE, because parsing of grep.extendedregexp
configuration variable directly touched opt.regflags back then, which
was done by b22520a3 ("grep: allow -E and -n to be turned on by
default via configuration", 2011-03-30).
When 84befcd0 ("grep: add a grep.patternType configuration setting",
2012-08-03) introduced extended_regexp_option field, we stopped
flipping regflags while reading the configuration, and that was when
we should have noticed and stopped dropping REG_EXTENDED bit in the
"now we can commit what type to use" helper function.
There is no reason to do this anymore, so stop doing it, more to
reduce "wait this is used under fixed/BRE/PCRE how?" confusion when
reading the code, than to to save ourselves trivial CPU cycles by
removing one assignment.
1. "built-in "git grep"", 2006-04-30.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a die(...) to a default case for the switch statement selecting
between grep pattern types under --recurse-submodules.
Normally this would be caught by -Wswitch, but the grep_pattern_type
type is converted to int by going through parse_options(). Changing
the argument type passed to compile_submodule_options() won't work,
the value will just get coerced. The -Wswitch-default warning will
warn about it, but that produces a lot of noise across the codebase,
this potential issue would be drowned in that noise.
Thus catching this at runtime is the least bad option. This won't ever
trigger in practice, but if a new pattern type were to be added this
catches an otherwise silent bug during development.
See commit 0281e487fd ("grep: optionally recurse into submodules",
2016-12-16) for the initial addition of this code.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a performance comparison test of grep regex engines given fixed
strings.
The current logic in compile_regexp() ignores the engine parameter and
uses kwset() to search for these, so this test shows no difference
between engines right now:
$ GIT_PERF_REPEAT_COUNT=10 GIT_PERF_LARGE_REPO=~/g/linux ./run p7821-grep-engines-fixed.sh
[...]
Test this tree
------------------------------------------------
7821.1: fixed grep int 0.56(1.67+0.68)
7821.2: basic grep int 0.57(1.70+0.57)
7821.3: extended grep int 0.59(1.76+0.51)
7821.4: perl grep int 1.08(1.71+0.55)
7821.6: fixed grep uncommon 0.23(0.55+0.50)
7821.7: basic grep uncommon 0.24(0.55+0.50)
7821.8: extended grep uncommon 0.26(0.55+0.52)
7821.9: perl grep uncommon 0.24(0.58+0.47)
7821.11: fixed grep æ 0.36(1.30+0.42)
7821.12: basic grep æ 0.36(1.32+0.40)
7821.13: extended grep æ 0.38(1.30+0.42)
7821.14: perl grep æ 0.35(1.24+0.48)
Only when run with -i via GIT_PERF_7821_GREP_OPTS=' -i' do we avoid
avoid going through the same kwset.[ch] codepath, see the "Even when
-F..." comment in grep.c. This only kicks for the non-ASCII case:
$ GIT_PERF_REPEAT_COUNT=10 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_7821_GREP_OPTS=' -i' ./run p7821-grep-engines-fixed.sh
[...]
Test this tree
---------------------------------------------------
7821.1: fixed grep -i int 0.62(2.10+0.57)
7821.2: basic grep -i int 0.68(1.90+0.61)
7821.3: extended grep -i int 0.78(1.94+0.57)
7821.4: perl grep -i int 0.98(1.78+0.74)
7821.6: fixed grep -i uncommon 0.24(0.44+0.64)
7821.7: basic grep -i uncommon 0.25(0.56+0.54)
7821.8: extended grep -i uncommon 0.27(0.62+0.45)
7821.9: perl grep -i uncommon 0.24(0.59+0.49)
7821.11: fixed grep -i æ 0.30(0.96+0.39)
7821.12: basic grep -i æ 0.27(0.92+0.44)
7821.13: extended grep -i æ 0.28(0.90+0.46)
7821.14: perl grep -i æ 0.28(0.74+0.49)
I'm planning to change how fixed-string searching happens. This test
gives a baseline for comparing performance before & after any such
change.
See commit ("perf: add a comparison test of grep regex engines",
2017-04-19) for details on the machine the above test run was executed
on.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a very basic performance comparison test comparing the POSIX
basic, extended and perl engines.
In theory the "basic" and "extended" engines should be implemented
using the same underlying code with a slightly different pattern
parser, but some implementations may not do this. Jump through some
slight hoops to test both, which is worthwhile since "basic" is the
default.
Running this on an i7 3.4GHz Linux 4.9.0-2 Debian testing against a
checkout of linux.git & latest upstream PCRE, both PCRE and git
compiled with -O3 using gcc 7.1.1:
$ GIT_PERF_REPEAT_COUNT=10 GIT_PERF_LARGE_REPO=~/g/linux ./run p7820-grep-engines.sh
[...]
Test this tree
---------------------------------------------------------------
7820.1: basic grep 'how.to' 0.34(1.24+0.53)
7820.2: extended grep 'how.to' 0.33(1.23+0.45)
7820.3: perl grep 'how.to' 0.31(1.05+0.56)
7820.5: basic grep '^how to' 0.32(1.24+0.42)
7820.6: extended grep '^how to' 0.33(1.20+0.44)
7820.7: perl grep '^how to' 0.57(2.67+0.42)
7820.9: basic grep '[how] to' 0.51(2.16+0.45)
7820.10: extended grep '[how] to' 0.49(2.20+0.43)
7820.11: perl grep '[how] to' 0.56(2.60+0.43)
7820.13: basic grep '\(e.t[^ ]*\|v.ry\) rare' 0.66(3.25+0.40)
7820.14: extended grep '(e.t[^ ]*|v.ry) rare' 0.65(3.19+0.46)
7820.15: perl grep '(e.t[^ ]*|v.ry) rare' 1.05(5.74+0.34)
7820.17: basic grep 'm\(ú\|u\)lt.b\(æ\|y\)te' 0.34(1.28+0.47)
7820.18: extended grep 'm(ú|u)lt.b(æ|y)te' 0.34(1.38+0.38)
7820.19: perl grep 'm(ú|u)lt.b(æ|y)te' 0.39(1.56+0.44)
Options can also be passed to git-grep via the GIT_PERF_7820_GREP_OPTS
environment variable. There are various modes such as "-v" that have
very different performance profiles, but handling the combinatorial
explosion of testing all those options would make this script much
more complex and harder to maintain. Instead just add the ability to
do one-shot runs with arbitrary options, e.g.:
$ GIT_PERF_REPEAT_COUNT=10 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_7820_GREP_OPTS=" -i" ./run p7820-grep-engines.sh
[...]
Test this tree
------------------------------------------------------------------
7820.1: basic grep -i 'how.to' 0.49(1.72+0.38)
7820.2: extended grep -i 'how.to' 0.46(1.64+0.42)
7820.3: perl grep -i 'how.to' 0.44(1.45+0.45)
7820.5: basic grep -i '^how to' 0.47(1.76+0.38)
7820.6: extended grep -i '^how to' 0.47(1.70+0.42)
7820.7: perl grep -i '^how to' 0.65(2.72+0.37)
7820.9: basic grep -i '[how] to' 0.86(3.64+0.42)
7820.10: extended grep -i '[how] to' 0.84(3.62+0.46)
7820.11: perl grep -i '[how] to' 0.73(3.06+0.39)
7820.13: basic grep -i '\(e.t[^ ]*\|v.ry\) rare' 1.63(8.13+0.36)
7820.14: extended grep -i '(e.t[^ ]*|v.ry) rare' 1.64(8.01+0.44)
7820.15: perl grep -i '(e.t[^ ]*|v.ry) rare' 1.44(6.88+0.44)
7820.17: basic grep -i 'm\(ú\|u\)lt.b\(æ\|y\)te' 0.66(2.67+0.44)
7820.18: extended grep -i 'm(ú|u)lt.b(æ|y)te' 0.66(2.67+0.43)
7820.19: perl grep -i 'm(ú|u)lt.b(æ|y)te' 0.59(2.31+0.37)
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We are supposed to report errno from fopen(). fclose() between fopen()
and the report function could either change errno or reset it.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After the last patch, this function is not used outside anymore. Keep it
static.
Noticed-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When fopen() returns NULL, it could be because the given path does not
exist, but it could also be some other errors and the caller has to
check. Add a wrapper so we don't have to repeat the same error check
everywhere.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In many places, Git warns about an inaccessible file after a fopen()
failed. To discern these cases from other cases where we want to warn
about inaccessible files, introduce a new helper specifically to test
whether fopen() failed because the current user lacks the permission to
open file in question.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This variable is added [1] with the assumption that on a sane system,
fopen(<dir>, "r") should return NULL. Linux and FreeBSD do not meet this
expectation while at least Windows and AIX do. Let's make sure they
behave the same way.
I only tested one version on Linux (4.7.0 with glibc 2.22) and
FreeBSD (11.0) but since GNU/kFreeBSD is fbsd kernel with gnu userspace,
I'm pretty sure it shares the same problem.
[1] cba22528fa (Add compat/fopen.c which returns NULL on attempt to open
directory - 2008-02-08)
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
copy_alternates() called fopen() without handling errors. By switching
to xfopen(), this bug is fixed.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
xfopen()
- provides error details
- explains error on reading, or writing, or whatever operation
- has l10n support
- prints file name in the error
Some of these are missing in the places that are replaced with xfopen(),
which is a clear win. In some other places, it's just less code (not as
clearly a win as the previous case but still is).
The only slight regresssion is in remote-testsvn, where we don't report
the file class (marks files) in the error messages anymore. But since
this is a _test_ svn remote transport, I'm not too concerned.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If git is built with the FREAD_READS_DIRECTORIES build variable set, this
would cause sparse to issue a 'not declared, should it be static?' warning
on Linux. This is a result of the method employed by 'compat/fopen.c' to
suppress the (possible) redefinition of the (system) fopen macro, which
also removes the extern declaration of the git_fopen function.
In order to suppress the warning, introduce a new macro to suppress the
definition (or possibly the re-definition) of the fopen symbol as a macro
override. This new macro (SUPPRESS_FOPEN_REDEFINITION) is only defined in
'compat/fopen.c', just prior to the inclusion of the 'git-compat-util.h'
header file.
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The origin, entry, and scoreboard structures are core to the blame
interface and need to be exposed for blame functionality.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create function that populates a blame_entry and prepends it to a list.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create function that completes setting up blame_scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create function that initializes blame_scoreboard to default values.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Either prepare_initial or prepare_final is used to determine which
commit is marked as 'final'. Call the underlying methods directly to
make this more clear.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The new method's interface is marginally cleaner than blame_sort, and
will avoid the need to expose the compare_blame_final method.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow the interface user to decide how to handle a progress update.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When iterating over references, reference priming is used to make sure
that loose references are read into the ref-cache before packed
references, to avoid races. It used to be that the prefix passed to
reference iterators almost always ended in `/`, for example
`refs/heads/`. In that case, the priming code would read all loose
references under `find_containing_dir("refs/heads/")`, which is
"refs/heads/". That's just what we want.
But now that `ref-filter` knows how to pass refname prefixes to
`for_each_fullref_in()`, the prefix might come from user input; for
example,
git for-each-ref refs/heads
Since the argument doesn't include a trailing slash, the reference
iteration code would prime all of the loose references under
`find_containing_dir("refs/heads")`, which is "refs/". Thus we would
unnecessarily read tags, remote-tracking references, etc., when the
user is only interested in branches.
It is a bit awkward to get around this problem. We can't just append a
slash to the argument, because we don't know ab initio whether an
argument like `refs/tags/release` corresponds to a single tag or to a
directory containing tags.
Moreover, until now a `prefix_ref_iterator` was used to make the final
decision about which references fall within the prefix (the
`cache_ref_iterator` only did a rough cut). This is also inefficient,
because the `prefix_ref_iterator` can't know, for example, that while
you are in a subdirectory that is completely within the prefix, you
don't have to do the prefix check.
So:
* Move the responsibility for doing the prefix check directly to
`cache_ref_iterator`. This means that `cache_ref_iterator_begin()`
never has to wrap its return value in a `prefix_ref_iterator`.
* Teach `cache_ref_iterator_begin()` (and `prime_ref_dir()`) to be
stricter about what they iterate over and what directories they
prime.
* Teach `cache_ref_iterator` to keep track of whether the current
`cache_ref_iterator_level` is fully within the prefix. If so, skip
the prefix checks entirely.
The main benefit of these optimizations is for loose references, since
packed references are always read all at once.
Note that after this change, `prefix_ref_iterator` is only ever used
for its trimming feature and not for its "prefix" feature. But I'm not
ripping out the latter yet, because it might be useful for another
patch series that I'm working on.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow the interface user to decide how to handle a failed sanity check,
whether that be to output with the current state or to do nothing.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The no_whole_file_rename flag is used in parts of blame that are being
moved to libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The xdl_opts flags are used in parts of blame that are being moved to
libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The show_root flag is used in parts of blame that are being moved to
libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The reverse flag is used in parts of blame that are being moved to
libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The argument from --contents is used in parts of blame that are being
moved to libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Copy and move score thresholds are used in parts of blame that are being
moved to libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Statistic counters are used in parts of blame that are being moved to
libgit, and should be accessible via the scoreboard structure.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Functions that will be publicly exposed should have names that better
reflect what they are a part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Functions that will be publicly exposed should have names that better
reflect what they are a part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Functions that will be publicly exposed should have names that better
reflect what they are a part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Functions related to blame_origin that will be publicly exposed should
have names that better reflect what they are a part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The scoreboard structure is core to the blame interface. Since
scoreboard will become more exposed, rename it to blame_scoreboard to
clarify what it is a part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The origin structure is core to the blame interface. Since origin will
become more exposed, rename it to blame_origin to clarify what it is a
part of.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
textconv_object is used in places other than blame.c and should be moved
to a more appropriate location. Other textconv related functions are
located in diff.c so that seems as good a place as any.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With commit 21666f1 ("convert object type handling from a string to a
number", 2007-02-26), there was no longer a need for blame.c to include
blob.h but it was not removed.
Signed-off-by: Jeff Smith <whydoubt@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On Windows the environment variable PATH contains a semicolon-separated
list of directories to search for, in order, when looking for the
location of a binary to run. get_path_split() parses it and returns an
array of string copies, which is iterated by path_lookup(), which in
turn passes each entry to lookup_prog().
Change lookup_prog() to take the directory name as a length-limited
string instead of as a NUL-terminated one and parse PATH directly in
path_lookup(). This avoids memory allocations, simplifying the code.
Helped-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we are matching refnames against a pattern, then we know that the
beginning of any refname that can match the pattern has to match the
part of the pattern up to the first glob character. For example, if
the pattern is `refs/heads/foo*bar`, then it can only match a
reference that has the prefix `refs/heads/foo`.
So pass that prefix to `for_each_fullref_in()`. This lets the ref code
avoid passing us the full set of refs, and in some cases avoid reading
them in the first place.
Note that this applies only when the `match_as_path` flag is set
(i.e., when `for-each-ref` is the caller), as the matching rules for
git-branch and git-tag are subtly different.
This could be generalized to the case of multiple patterns, but (a) it
probably doesn't come up that often, and (b) it is more awkward to
deal with multiple patterns (e.g., the patterns might not be
disjoint). So, since this is just an optimization, punt on the case of
multiple patterns.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Only one caller was using it, so move the check to that caller.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of handling `GIT_REF_PARANOIA` in
`files_ref_iterator_begin()`, handle it in
`refs_ref_iterator_begin()`, where it will cover all reference stores.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The old code ignored any errors encountered when trying to fopen the
"packed-refs" file, treating all such failures as if the file didn't
exist. But it could be that there is some other error opening the
file (e.g., permissions problems), and we don't want to silently
ignore such problems. So report any failures that are not due to
ENOENT.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach `read_packed_refs()` to also
* Allocate and initialize the new `packed_ref_cache`
* Open and close the `packed-refs` file
* Update the `validity` field of the new object
This decreases the coupling between `packed_refs_cache` and
`files_ref_store` by a little bit.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we've got the "packed-refs" file locked, then it can't change;
there's no need to keep calling `stat_validity_check()` on it.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extract a function for deciding whether a reference should be packed.
It is a self-contained bit of logic, so splitting it out improves
readability.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's pretty cheap to make sure that the caller didn't pass us an
unsorted list by accident, so do so.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Eliminate a theoretical risk of integer overflow if the two types have
different sizes.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the future, compound reference stores will sometimes need to modify
references in two different reference stores at the same time, meaning
that a single logical reference transaction might have to be
implemented as two internal sub-transactions. They won't want to call
`ref_transaction_commit()` for the two sub-transactions one after the
other, because that wouldn't be atomic (the first commit could succeed
and the second one fail). Instead, they will want to prepare both
sub-transactions (i.e., obtain any necessary locks and do any
pre-checks), and only if both prepare steps succeed, then commit both
sub-transactions.
Start preparing for that day by adding a new, optional
`ref_transaction_prepare()` step to the reference transaction
sequence, which obtains the locks and does any prechecks, reporting
any errors that occur. Also add a `ref_transaction_abort()` function
that can be used to abort a sub-transaction even if it has already
been prepared.
That is on the side of the public-facing API. On the side of the
`ref_store` VTABLE, get rid of `transaction_commit` and instead add
methods `transaction_prepare`, `transaction_finish`, and
`transaction_abort`. A `ref_transaction_commit()` now basically calls
methods `transaction_prepare` then `transaction_finish`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the check that `transaction->state` is valid from
`files_transaction_commit()` to `ref_transaction_commit()`, where
other future reference backends can benefit from it as well.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extract the cleanup functionality from `files_transaction_commit()`
into a new function. It will soon have another caller.
Use the common cleanup code even on early exit if the transaction is
empty, to reduce code duplication.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of using a global `lock_file` instance for the main
"packed-refs" file and using a pointer in `files_ref_store` to keep
track of whether it is locked, embed the `lock_file` instance directly
in the `files_ref_store` struct and use the new
`is_lock_file_locked()` function to keep track of whether it is
locked. This keeps related data together and makes the main reference
store less of a special case.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the `lock` member from `packed_ref_cache` to `files_ref_store`,
since at most one cache can have a locked "packed-refs" file
associated with it. Rename it to `packed_refs_lock` to make its
purpose clearer in its new home. More changes are coming here shortly.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Just because the files backend can't retain reflogs for deleted
references is no reason that they shouldn't be supported by the
virtual method interface. Also, `delete_ref()` and `refs_delete_ref()`
have already gained `msg` parameters. Now let's add them to
`delete_refs()` and `refs_delete_refs()`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Eliminate any chance of integer overflow on platforms where the two
types have different sizes.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The backend already correctly restricts its output to references whose
names start with the prefix. By passing the prefix again to
`prefix_ref_iterator`, we were forcing that iterator to do redundant
prefix comparisons. So set it to the empty string.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `trim` parameter can be set independently of `prefix`. So if some
caller were to set `trim` to be greater than `strlen(prefix)`, we
could end up pointing the `refname` field of the iterator past the NUL
of the actual reference name string.
That can't happen currently, because `trim` is always set either to
zero or to `strlen(prefix)`. But even the latter could lead to
confusion, if a refname is exactly equal to the prefix, because then
we would set the outgoing `refname` to the empty string.
And we're about to decouple the `prefix` and `trim` arguments even
more, so let's be cautious here. Report a bug if ever asked to trim a
reference whose name is not longer than `trim`.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The former is by far more common in our codebase.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The iterator returned by this function only includes references whose
names start with the whole prefix, not all of those in
`find_containing_dir(prefix)` as the old docstring claimed. This
docstring was probably copy-pasted from old ref-cache code, which had
the old specification. But now, `cache_ref_iterator_begin()`
(from which the files reference iterator gets its values)
automatically wraps its output using `prefix_ref_iterator_begin()`
when necessary, so it has the stricter behavior.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In particular, make it clear that they make copies of the sha1
arguments.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test of failing `git rm -f` removes the write permissions on the
test directory, but fails to restore them if the test fails. This
means that the test temporary directory cannot be cleaned up, which
means that subsequent attempts to run the test fail mysteriously.
Instead, do the cleanup in a `test_when_finished` block so that it
can't be skipped.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bc/object-id: (53 commits)
object: convert parse_object* to take struct object_id
tree: convert parse_tree_indirect to struct object_id
sequencer: convert do_recursive_merge to struct object_id
diff-lib: convert do_diff_cache to struct object_id
builtin/ls-tree: convert to struct object_id
merge: convert checkout_fast_forward to struct object_id
sequencer: convert fast_forward_to to struct object_id
builtin/ls-files: convert overlay_tree_on_cache to object_id
builtin/read-tree: convert to struct object_id
sha1_name: convert internals of peel_onion to object_id
upload-pack: convert remaining parse_object callers to object_id
revision: convert remaining parse_object callers to object_id
revision: rename add_pending_sha1 to add_pending_oid
http-push: convert process_ls_object and descendants to object_id
refs/files-backend: convert many internals to struct object_id
refs: convert struct ref_update to use struct object_id
ref-filter: convert some static functions to struct object_id
Convert struct ref_array_item to struct object_id
Convert the verify_pack callback to struct object_id
Convert lookup_tag to struct object_id
...
Update tests to pass under GETTEXT_POISON (a mechanism to ensure
that output strings that should not be translated are not
translated by mistake), and tell TravisCI to run them.
* ab/fix-poison-tests:
travis-ci: add job to run tests with GETTEXT_POISON
travis-ci: setup "prove cache" in "script" step
tests: fix tests broken under GETTEXT_POISON=YesPlease
The receive-pack program now makes sure that the push certificate
records the same set of push options used for pushing.
* jt/push-options-doc:
receive-pack: verify push options in cert
docs: correct receive.advertisePushOptions default
The Web interface to gmane news archive is long gone, even though
the articles are still accessible via NTTP. Replace the links with
ones to public-inbox.org. Because their message identification is
based on the actual message-id, it is likely that it will be easier
to migrate away from it if/when necessary.
* ab/doc-replace-gmane-links:
doc: replace more gmane links
doc: replace a couple of broken gmane links
A few codepaths in "checkout" and "am" working on an unborn branch
tried to access an uninitialized piece of memory.
* rs/checkout-am-fix-unborn:
am: check return value of resolve_refdup before using hash
checkout: check return value of resolve_refdup before using hash
* ls/travis-relays-for-windows-ci:
travis-ci: retry if Git for Windows CI returns HTTP error 502 or 503
travis-ci: handle Git for Windows CI status "failed" explicitly
Setting "log.decorate=false" in the configuration file did not take
effect in v2.13, which has been corrected.
* ah/log-decorate-default-to-auto:
builtin/log: honor log.decorate
Amend the t/perf/run output so that in addition to the "Running N
tests" heading currently being emitted, it also emits "Unpacking $rev"
and "Building $rev" when setting up the build/$rev directory & when
building it, respectively.
This makes it easier to see what's going on and what revision is being
tested as the output scrolls by.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a git GIT_PERF_MAKE_COMMAND variable to compliment the existing
GIT_PERF_MAKE_OPTS facility. This allows specifying an arbitrary shell
command to execute instead of 'make'.
This is useful e.g. in cases where the name, semantics or defaults of
a Makefile flag have changed over time. It can even be used to change
the contents of the tree, useful for monkeypatching ancient versions
of git to get them to build.
This opens Pandora's box in some ways, it's now possible to
"jailbreak" the perf environment and e.g. modify the source tree via
this arbitrary instead of just issuing a custom "make" command, such a
command has to be re-entrant in the sense that subsequent perf runs
will re-use the possibly modified tree.
It would be pointless to try to mitigate or work around that caveat in
a tool purely aimed at Git developers, so this change makes no attempt
to do so.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Address a big blind spot in the tests for patterns containing \0. The
is_fixed() function considers any string that contains \0 fixed, even
if it contains regular expression metacharacters, those patterns are
currently matched with kwset.
Before this change removing that memchr(s, 0, len) check from
is_fixed() wouldn't change the result of any of the tests, since
regcomp() will happily match the part before the \0.
The kwset path is dependent on whether the the -i flag is on, and
whether the pattern has any non-ASCII characters, but none of this was
tested for.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add setup code needed for testing regexes that contain both binary
data and regex metacharacters.
The POSIX regcomp() function inherently can't support that, because it
takes a \0-delimited char *, but other regex engines APIs like PCRE v2
take a pattern/length pair, and are thus able to handle \0s in
patterns as well as any other character.
When kwset was imported in commit 9eceddeec6 ("Use kwset in grep",
2011-08-21) this limitation was fixed, but at the expense of
introducing the undocumented limitation that any pattern containing \0
implicitly becomes a fixed match (equivalent to -F having been
provided).
That's not something we'd like to keep in the future. The inability to
match patterns containing \0 is a leaky implementation detail.
So add tests as a first step towards changing that. In order to test
that \0-patterns can properly match as regexes the test string needs
to have some regex metacharacters in it.
There were other blind spots in the tests. The code around kwset
specially handles case-insensitive & non-ASCII data, but there were no
tests for this.
Fix all of that by amending the text being matched to contain both
regex metacharacters & non-ASCII data.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a helper function to make the tests which check for patterns with
\0 in them more succinct. Right now this isn't a big win, but
subsequent commits will add a lot more of these tests.
The helper is based on the match() function in t3070-wildmatch.sh.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add testing for grep pattern types being correctly passed to
submodules. The pattern "(.|.)[\d]" matches differently under
fixed (not at all), and then matches different lines under
basic/extended & perl regular expressions, so this change asserts that
the pattern type is passed along correctly.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Amend the submodule recursion test to prepare it for subsequent tests
of whether it passes along the grep.patternType to the submodule
greps.
This is the result of searching & replacing:
foobar -> (1|2)d(3|4)
foo -> (1|2)
bar -> (3|4)
Currently there's no tests for whether e.g. -P or -E is correctly
passed along, tests for that will be added in a follow-up change, but
first add content to the tests which will match differently under
different regex engines.
Reuse the pattern established in an earlier commit of mine in this
series ("log: add exhaustive tests for pattern style options &
config", 2017-04-07). The pattern "(.|.)[\d]" will match this content
differently under fixed/basic/extended & perl.
This test code was originally added in commit 0281e487fd ("grep:
optionally recurse into submodules", 2016-12-16).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add tests for --threads=N being supplied on the command-line, or when
grep.threads=N being supplied in the configuration.
When the threading support was made run-time configurable in commit
89f09dd34e ("grep: add --threads=<num> option and grep.threads
configuration", 2015-12-15) no tests were added for it.
In developing a change to the grep code I was able to make
'--threads=1 <pat>` segfault, while the test suite still passed. This
change fixes that blind spot in the tests.
In addition to asserting that asking for N threads shouldn't segfault,
test that the grep output given any N is the same.
The choice to test only 1..10 as opposed to 1..8 or 1..16 or whatever
is arbitrary. Testing 1..1024 works locally for me (but gets
noticeably slower as more threads are spawned). Given the structure of
the code there's no reason to test an arbitrary number of threads,
only 0, 1 and >=2 are special modes of operation.
A later patch introduces a PTHREADS test prerequisite which is true
under NO_PTHREADS=UnfortunatelyYes, but even under NO_PTHREADS it's
fine to test --threads=N, we'll just ignore it and not use
threading. So these tests also make sense under that mode to assert
that --threads=N without pthreads still returns expected results.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change a non-ASCII case-insensitive test case to stop using --debug,
and instead simply test for the expected results.
The test coverage remains the same with this change, but the test
won't break due to internal refactoring.
This test was added in commit 793dc676e0 ("grep/icase: avoid kwsset
when -F is specified", 2016-06-25). It was asserting that the regex
must be compiled with compile_fixed_regexp(), instead test for the
expected results, allowing the underlying implementation to change.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test for backreferences such as (.)\1 in PCRE patterns. This
test ensures that the PCRE_NO_AUTO_CAPTURE option isn't turned
on. Before this change turning it on would break these sort of
patterns, but wouldn't break any tests.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test asserting that when --perl-regexp (and -P for grep) is
given to git-grep & git-log that we die with an error.
In developing the PCRE v2 series I introduced a regression where -P
would (through control-flow fall-through) become synonymous with basic
POSIX matching. I.e. 'git grep -P '[\d]' would match "d" instead of
digits.
The entire test suite would still pass with this serious regression,
since everything that tested for --perl-regexp would be guarded by the
PCRE prerequisite, fix that blind-spot by adding tests under !PCRE
asserting that git must die when given --perl-regexp or -P.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make the --regexp-ignore-case option work with --perl-regexp. This
never worked, and there was no test for this. Fix the bug and add a
test.
When PCRE support was added in commit 63e7e9d8b6 ("git-grep: Learn
PCRE", 2011-05-09) compile_pcre_regexp() would only check
opt->ignore_case, but when the --perl-regexp option was added in
commit 727b6fc3ed ("log --grep: accept --basic-regexp and
--perl-regexp", 2012-10-03) the code didn't set the opt->ignore_case.
Change the test suite to test for -i and --invert-regexp with
basic/extended/perl patterns in addition to fixed, which was the only
patternType that was tested for before in combination with those
options.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add exhaustive tests for how the different grep.patternType options &
the corresponding command-line options affect git-log.
Before this change it was possible to patch revision.c so that the
--basic-regexp option was synonymous with --extended-regexp, and
--perl-regexp wasn't recognized at all, and still have 100% of the
test suite pass.
This was because the first test being modified here, added in commit
34a4ae55b2 ("log --grep: use the same helper to set -E/-F options as
"git grep"", 2012-10-03), didn't actually check whether we'd enabled
extended regular expressions as distinct from re-toggling non-fixed
string support.
Fix that by changing the pattern to a pattern that'll only match if
--extended-regexp option is provided, but won't match under the
default --basic-regexp option.
Other potential regressions were possible since there were no tests
for the rest of the combinations of grep.patternType configuration
toggles & corresponding git-log command-line options. Add exhaustive
tests for those.
The patterns being passed to fixed/basic/extended/PCRE are carefully
crafted to return the wrong thing if the grep engine were to pick any
other matching method than the one it's told to use.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename the LIBPCRE prerequisite to PCRE. This is for preparation for
libpcre2 support, where having just "LIBPCRE" would be confusing as it
implies v1 of the library.
None of these tests are incompatible between versions 1 & 2 of
libpcre, it's less confusing to give them a more general name to make
it clear that they work on both library versions.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stop promising in our grep & rev-list options documentation that we're
always going to be using libpcre when given the --perl-regexp option.
Instead talk about using "Perl-compatible regular expressions" and
using these types of patterns using "a compile-time dependency".
Saying "libpcre" means that we're talking about libpcre.so, which is
always going to be v1. This change is part of an ongoing saga to add
support for libpcre2, which comes with PCRE v2.
In the future we might use some completely unrelated library to
provide perl-compatible regular expression support. By wording the
documentation differently and not promising any specific version of
PCRE or even PCRE at all we have more wiggle room to change the
implementation.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reword an outdated & inaccurate comment which suggests that only
git-grep can use PCRE.
This comment was added back when PCRE support was initially added in
commit 63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09), and was true
at the time.
It hasn't been telling the full truth since git-log learned to use
PCRE with --grep in commit 727b6fc3ed ("log --grep: accept
--basic-regexp and --perl-regexp", 2012-10-03), and more importantly
is likely to get more inaccurate over time as more use is made of PCRE
in other areas.
Reword it to be more future-proof, and to more clearly explain that
this enables user-initiated runtime behavior.
Copy/pasting this so much in configure.ac is lame, these Makefile-like
flags aren't even used by autoconf, just the corresponding
--with[out]-* options. But copy/pasting the comments that make sense
for the Makefile to configure.ac where they make less sense is the
pattern everything else follows in that file. I'm not going to war
against that as part of this change, just following the existing
pattern.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make git checkout (and other unpack_tree operations) preserve the
untracked cache. This is valuable for two reasons:
1. Often, an unpack_tree operation will not touch large parts of the
working tree, and thus most of the untracked cache will continue to be
valid.
2. Even if the untracked cache were entirely invalidated by such an
operation, the user has signaled their intention to have such a cache,
and we don't want to throw it away.
[jes: backed out the watchman-specific parts]
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier dddbad72 ("timestamp_t: a new data type for timestamps",
2017-04-26) updated all in-core variables, fields and function
return values that are used to store "seconds since epoch" to a new
type timestamp_t. Unfortunately one variable "cutoff", which is
used to keep track of the oldest timestamp of commit we saw on the
command line, was "long" and left behind.
Update it to timestamp_t as well.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the conditional inclusion mechanism to support
e.g. gitdir:~/git_tree/repo where ~/git_tree is a symlink to
/mnt/stuff/repo.
This worked in the initial version of this facility[1], but regressed
later in the series while solving a related bug[2].
Now gitdir: will match against the symlinked
path (e.g. gitdir:~/git_tree/repo) in addition to the current
/mnt/stuff/repo path.
Since this is already in a release version note in the documentation
that this behavior changed, so users who expect their configuration to
work on both v2.13.0 and some future version of git with this fix
aren't utterly confused.
1. commit 3efd0bedc6 ("config: add conditional include", 2017-03-01)
2. commit 86f9515708 ("config: resolve symlinks in conditional
include's patterns", 2017-04-05)
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some platforms have ulong that is smaller than time_t, and our
historical use of ulong for timestamp would mean they cannot
represent some timestamp that the platform allows. Invent a
separate and dedicated timestamp_t (so that we can distingiuish
timestamps and a vanilla ulongs, which along is already a good
move), and then declare uintmax_t is the type to be used as the
timestamp_t.
* js/larger-timestamps:
archive-tar: fix a sparse 'constant too large' warning
use uintmax_t for timestamps
date.c: abort if the system time cannot handle one of our timestamps
timestamp_t: a new data type for timestamps
PRItime: introduce a new "printf format" for timestamps
parse_timestamp(): specify explicitly where we parse timestamps
t0006 & t5000: skip "far in the future" test when time_t is too limited
t0006 & t5000: prepare for 64-bit timestamps
ref-filter: avoid using `unsigned long` for catch-all data type
"git clone --config var=val" is a way to populate the
per-repository configuration file of the new repository, but it did
not work well when val is an empty string. This has been fixed.
* jn/clone-add-empty-config-from-command-line:
clone: handle empty config values in -c
Travis CI gained a task to format the documentation with both
AsciiDoc and AsciiDoctor.
* ls/travis-doc-asciidoctor:
travis-ci: check AsciiDoc/AsciiDoctor stderr output
travis-ci: unset compiler for jobs that do not need one
travis-ci: parallelize documentation build
travis-ci: build documentation with AsciiDoc and Asciidoctor
"git archive --format=zip" learned to use zip64 extension when
necessary to go beyond the 4GB limit.
* rs/large-zip:
t5004: require 64-bit support for big ZIP tests
archive-zip: set version field for big files correctly
archive-zip: support files bigger than 4GB
archive-zip: support archives bigger than 4GB
archive-zip: write ZIP dir entry directly to strbuf
archive-zip: use strbuf for ZIP directory
archive-zip: add tests for big ZIP archives
"git clone" learned the "--no-tags" option not to fetch all tags
initially, and also set up the tagopt not to follow any tags in
subsequent fetches.
* ab/clone-no-tags:
tests: rename a test having to do with shallow submodules
clone: add a --no-tags option to clone without tags
tests: change "cd ... && git fetch" to "cd &&\n\tgit fetch"
The colors in which "git status --short --branch" showed the names
of the current branch and its remote-tracking branch are now
configurable.
* sk/status-short-branch-color-config:
status: add color config slots for branch info in "--short --branch"
status: fix missing newline when comment chars are disabled
The codepath in "git am" that is used when running "git rebase"
leaked memory held for the log message of the commits being rebased.
* jk/am-leakfix:
am: shorten ident_split variable name in get_commit_info()
am: simplify allocations in get_commit_info()
am: fix commit buffer leak in get_commit_info()
"git cherry-pick" and other uses of the sequencer machinery
mishandled a trailer block whose last line is an incomplete line.
This has been fixed so that an additional sign-off etc. are added
after completing the existing incomplete line.
* jt/use-trailer-api-in-commands:
sequencer: add newline before adding footers
"git gc" did not interact well with "git worktree"-managed
per-worktree refs.
* nd/worktree-kill-parse-ref:
refs: kill set_worktree_head_symref()
worktree.c: kill parse_ref() in favor of refs_resolve_ref_unsafe()
refs: introduce get_worktree_ref_store()
refs: add REFS_STORE_ALL_CAPS
refs.c: make submodule ref store hashmap generic
environment.c: fix potential segfault by get_git_common_dir()
Attempt to allow us notice "fishy" situation where we fail to
remove the temporary directory used during the test.
* dt/gc-ignore-old-gc-logs:
test-lib: retire $remove_trash variable
test-lib.sh: do not barf under --debug at the end of the test
test-lib: abort when can't remove trash directory
Code to update the cache-tree has been tightened so that we won't
accidentally write out any 0{40} entry in the tree object.
* jk/no-null-sha1-in-cache-tree:
cache-tree: reject entries with null sha1
The default packed-git limit value has been raised on larger
platforms to save "git fetch" from a (recoverable) failure while
"gc" is running in parallel.
* dt/raise-core-packed-git-limit:
Increase core.packedGitLimit
Currently, send-email has support for rudimentary e-mail validation.
Allow the user to add support for more validation by providing a
sendemail-validate hook.
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch-pack, when fetching a literal SHA-1 from a server that is not
configured with uploadpack.allowtipsha1inwant (or similar), always
returns an error message of the form "Server does not allow request for
unadvertised object %s". However, it is sometimes the case that such
object is advertised. This situation would occur, for example, if a user
or a script was provided a SHA-1 instead of a branch or tag name for
fetching, and wanted to invoke "git fetch" or "git fetch-pack" using
that SHA-1.
Teach fetch-pack to also check the SHA-1s of the refs in the received
ref advertisement if a literal SHA-1 was given by the user.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Enable sub-processes to gracefully handle when the process dies by
updating subprocess_read_status to return an error on EOF instead of
dying.
Update apply_multi_file_filter to take advantage of the revised
subprocess_read_status.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the sub-proces functions into sub-process.h/c. Add documentation
for the new module in Documentation/technical/api-sub-process.txt
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Do a mechanical rename of the functions that will become the reusable
sub-process module.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update all functions that are going to be moved into a reusable module
so that they only work with the reusable data structures. Move code
that is specific to the filter out into the filter specific functions.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To enable future reuse of the filter.<driver>.process infrastructure,
split the cmd2process structure into two separate parts.
subprocess_entry will now contain the generic data required to manage
the creation and tracking of the child process in a hashmap.
cmd2process is a filter protocol specific structure that is used to
track the negotiated capabilities of the filter.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To enable future reuse of the filter.<driver>.process infrastructure,
split start_multi_file_filter() into two separate parts.
start_multi_file_filter() will now only contain the generic logic to
manage the creation and tracking of the child process in a hashmap.
start_multi_file_filter_fn() is a protocol specific initialization
function that will negotiate the multi-file-filter interface version
and capabilities.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
packet_writel() takes a variable-sized list and reads to
the first NULL. Let's let the compiler know so that it can
help us catch mistakes in the callers.
This should have been annotated similarly when it was a
static function, but it's doubly important now that the
function is available to the whole code-base.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert find_pathspecs_matching_against_index to take an index
parameter.
In addition mark pathspec.c with NO_THE_INDEX_COMPATIBILITY_MACROS now
that it doesn't use any cache macros or reference 'the_index'.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since (ae8d08242 pathspec: pass directory indicator to
match_pathspec_item()) the path matching logic has been able to cope
with submodules without needing to strip off a trailing slash if a path
refers to a submodule.
Since stripping the slash is no longer necessary, remove the
PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP flag.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since (ae8d08242 pathspec: pass directory indicator to
match_pathspec_item()) the path matching logic has been able to cope
with submodules without needing to strip off a trailing slash if a path
refers to a submodule.
ls-files is the only caller of 'parse_pathspec()' which relies on the
behavior of the PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP flag because it
uses the result to construct a common prefix of all provided pathspecs
which is then used to prune the index of all entries which don't have
that prefix. Since submodules entries in the index don't have a
trailing slash 'prune_cache()' will be overeager and prune a submodule
'sub' if the common prefix is 'sub/'. To correct this behavior, only
prune entries which don't match up to, but not including, a trailing
slash of the common prefix.
This is in preparation to remove the
PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP flag in a later patch.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since (ae8d08242 pathspec: pass directory indicator to
match_pathspec_item()) the path matching logic has been able to cope
with submodules without needing to strip off a trailing slash if a path
refers to a submodule.
Since the stripping the trailing slash is no longer necessary, remove
the PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag. In addition, factor
out the logic which dies if a path decends into a submodule so that it
can still be used as a check after a pathspec struct has been
initialized.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a test showing that runtimes of the wildmatch() function used for
globbing in git grow exponentially in the face of some pathological
globs.
This issue affects both globs matching filenames via e.g. ls-files,
and globs matching refnames via e.g. for-each-ref.
As noted in the test description this is a test to see whether Git
suffers from the issue noted in an article Russ Cox posted today about
common bugs in various glob implementations:
https://research.swtch.com/glob
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a function to setup a fresh test repo via 'git init' to compliment
the existing functions to copy over a normal & large repo.
Some performance tests don't need any existing repository data at all
to be significant, e.g. tests which stress glob matches against single
pathological revisions or files, which I'm about to add in a
subsequent commit.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently 'git add' is the only command which dies when launched from an
unpopulated submodule (the place-holder directory for a submodule which
hasn't been checked out). This is triggered implicitly by passing the
PATHSPEC_STRIP_SUBMODULE_SLASH_EXPENSIVE flag to 'parse_pathspec()'.
Instead make this desire more explicit by creating a function
'die_in_unpopulated_submodule()' which dies if the provided 'prefix' has
a leading path component which matches a submodule in the the index.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current message displayed upon an internal error in
'init_pathspec_item()' isn't very descriptive and doesn't provide much
context to where the error occurred. Update the error message to
provide more context to where the error occured.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Now that diff.indentHeuristic is handled automatically by the plumbing
commands, there's no need to propagate it manually.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The feature was included in v2.11 (released 2016-11-29) and we got no
negative feedback. Quite the opposite, all feedback we got was positive.
Turn it on by default. Users who dislike the feature can turn it off
by setting diff.indentHeuristic (which also configures plumbing commands,
see prior patches).
The change to t/t4051-diff-function-context.sh is needed because the
heuristic shifts the changed hunk in the patch. To get the same result
regardless of the heuristic configuration, we modify the test file
differently: We insert a completely new line after line 2, instead of
simply duplicating it.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This matches how the diff Porcelain works. It makes the plumbing commands
respect diff's configuration options, such as indentHeuristic, because
init_revisions() calls diff_setup() which fills in the diff_options struct.
Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This heuristic was originally introduced as an experimental feature,
and therefore part of the UI configuration.
But the user often sees diffs generated by plumbing commands like
diff-tree. Moving the indent heuristic into diff's basic configuration
prepares the way for diff plumbing commands to respect the setting.
The heuristic itself merely makes the diffs more aesthetically
pleasing, without changing their correctness. Scripts that rely on
the diff plumbing commands should not care whether or not the heuristic
is employed.
Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit dddbad728c ("timestamp_t: a new data type for timestamps",
26-04-2017) introduced a new typedef 'timestamp_t', as a synonym for an
unsigned long, which was used at the time to represent timestamps in
git. A later commit 28f4aee3fb ("use uintmax_t for timestamps",
26-04-2017) changed the typedef to use an 'uintmax_t' for the timestamp
representation type.
When building on a 32-bit Linux system, sparse complains that a constant
(USTAR_MAX_MTIME) used to detect a 'far-future mtime' timestamp, is too
large; 'warning: constant 077777777777UL is so big it is unsigned long
long' on lines 335 and 338 of archive-tar.c. Note that both gcc and
clang only issue a warning if this constant is used in a context that
requires an 'unsigned long' (rather than an uintmax_t). (Since TIME_MAX
is no longer equal to 0xFFFFFFFF, even on a 32-bit system, the macro
USTAR_MAX_MTIME is set to 077777777777UL, which cannot be represented as
an 'unsigned long' constant).
In order to suppress the warning, change the definition of the macro
constant USTAR_MAX_MTIME to use an 'ULL' type suffix.
In a similar vein, on systems which use a 64-bit representation of the
'unsigned long' type, the USTAR_MAX_SIZE constant macro is defined with
the value 077777777777ULL. Although this does not cause any warning
messages to be issued, it would be more appropriate for this constant
to use an 'UL' type suffix rather than 'ULL'.
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert parse_tree_indirect to take a pointer to struct object_id.
Update all the callers. This transformation was achieved using the
following semantic patch and manual updates to the declaration and
definition. Update builtin/checkout.c manually as well, since it uses a
ternary expression not handled by the semantic patch.
@@
expression E1;
@@
- parse_tree_indirect(E1.hash)
+ parse_tree_indirect(&E1)
@@
expression E1;
@@
- parse_tree_indirect(E1->hash)
+ parse_tree_indirect(E1)
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a prerequisite to convert do_diff_cache, which is required to
convert parse_tree_indirect.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Converting checkout_fast_forward is required to convert
parse_tree_indirect.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast_forward_to is required for checkout_fast_fowrard, which is required
for parse_tree_indirect.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a caller of parse_tree_indirect, which must be converted in
order to convert parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the remaining parse_object callers to struct object_id. Use
named constants for several hard-coded values. In addition, rename
got_sha1 to got_oid to reflect the new argument.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename this function and convert it to take a pointer to struct
object_id.
This is a prerequisite for converting get_reference, which is needed to
convert parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename one function to reflect that it now uses struct object_id. This
conversion is a prerequisite for converting parse_object.
Note that while the use of a buffer that is exactly forty bytes long
looks questionable, get_oid_hex reads exactly the right number of bytes
and does not require the data to be NUL-terminated.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert many of the internals of the files backend to use struct
object_id. Avoid converting public APIs (except one change to
refs/ref-cache.c) to limit the scope of the changes.
Convert one use of get_sha1_hex to parse_oid_hex, and rely on the fact
that a strbuf will be NUL-terminated and that parse_oid_hex will fail on
truncated input to avoid the need to check the length.
This is a requirement to convert parse_object later on.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert struct ref_array_item to use struct object_id by changing the
definition and applying the following semantic patch, plus the standard
object_id transforms:
@@
struct ref_update E1;
@@
- E1.new_sha1
+ E1.new_oid.hash
@@
struct ref_update *E1;
@@
- E1->new_sha1
+ E1->new_oid.hash
@@
struct ref_update E1;
@@
- E1.old_sha1
+ E1.old_oid.hash
@@
struct ref_update *E1;
@@
- E1->old_sha1
+ E1->old_oid.hash
This transformation allows us to convert write_ref_to_lockfile, which is
required to convert parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Among the converted functions is a caller of parse_object_buffer, which
we will convert later.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert struct ref_array_item to use struct object_id by changing the
definition and applying the following semantic patch, plus the standard
object_id transforms:
@@
struct ref_array_item E1;
@@
- E1.objectname
+ E1.objectname.hash
@@
struct ref_array_item *E1;
@@
- E1->objectname
+ E1->objectname.hash
This transformation allows us to convert get_obj, which is needed to
convert parse_object_buffer.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make the verify_pack_callback take a pointer to struct object_id.
Change the pack checksum to use GIT_MAX_RAWSZ, even though it is not
strictly an object ID. Doing so ensures resilience against future hash
size changes, and allows us to remove hard-coded assumptions about how
big the buffer needs to be.
Also, use a union to convert the pointer from nth_packed_object_sha1 to
to a pointer to struct object_id. This behavior is compatible with GCC
and clang and explicitly sanctioned by C11. The alternatives are to
just perform a cast, which would run afoul of strict aliasing rules, but
should just work, and changing the pointer into an instance of struct
object_id and copying the value. The latter operation could seriously
bloat memory usage on fsck, which already uses a lot of memory on some
repositories.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the remaining functions to take pointers to struct object_id
instead of pointers to unsigned char, and update the internals of these
functions as well. Among these functions is a caller of lookup_tag,
which we will convert shortly.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the lookup_tree function to take a pointer to struct object_id.
The commit was created with manual changes to tree.c, tree.h, and
object.c, plus the following semantic patch:
@@
@@
- lookup_tree(EMPTY_TREE_SHA1_BIN)
+ lookup_tree(&empty_tree_oid)
@@
expression E1;
@@
- lookup_tree(E1.hash)
+ lookup_tree(&E1)
@@
expression E1;
@@
- lookup_tree(E1->hash)
+ lookup_tree(E1)
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert lookup_blob to take a pointer to struct object_id.
The commit was created with manual changes to blob.c and blob.h, plus
the following semantic patch:
@@
expression E1;
@@
- lookup_blob(E1.hash)
+ lookup_blob(&E1)
@@
expression E1;
@@
- lookup_blob(E1->hash)
+ lookup_blob(E1)
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All but a few callers of lookup_blob have been converted to struct
object_id. Introduce a temporary, which will be removed later, into
parse_object to ease the transition, and convert the remaining callers
so that we can update lookup_blob to take struct object_id *.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert struct delta_info and struct object_info, as well as the various
functions, to use struct object_id. Convert several hard-coded 20
values to GIT_SHA1_RAWSZ. Among the functions converted is a caller of
lookup_blob, which we will convert shortly.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert struct pack_idx_entry to use struct object_id by changing the
definition and applying the following semantic patch, plus the standard
object_id transforms:
@@
struct pack_idx_entry E1;
@@
- E1.sha1
+ E1.oid.hash
@@
struct pack_idx_entry *E1;
@@
- E1->sha1
+ E1->oid.hash
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert lookup_commit, lookup_commit_or_die,
lookup_commit_reference, and lookup_commit_reference_gently to take
struct object_id arguments.
Introduce a temporary in parse_object buffer in order to convert this
function. This is required since in order to convert parse_object and
parse_object_buffer, lookup_commit_reference_gently and
lookup_commit_or_die would need to be converted. Not introducing a
temporary would therefore require that lookup_commit_or_die take a
struct object_id *, but lookup_commit would take unsigned char *,
leaving a confusing and hard-to-use interface.
parse_object_buffer will lose this temporary in a later patch.
This commit was created with manual changes to commit.c, commit.h, and
object.c, plus the following semantic patch:
@@
expression E1, E2;
@@
- lookup_commit_reference_gently(E1.hash, E2)
+ lookup_commit_reference_gently(&E1, E2)
@@
expression E1, E2;
@@
- lookup_commit_reference_gently(E1->hash, E2)
+ lookup_commit_reference_gently(E1, E2)
@@
expression E1;
@@
- lookup_commit_reference(E1.hash)
+ lookup_commit_reference(&E1)
@@
expression E1;
@@
- lookup_commit_reference(E1->hash)
+ lookup_commit_reference(E1)
@@
expression E1;
@@
- lookup_commit(E1.hash)
+ lookup_commit(&E1)
@@
expression E1;
@@
- lookup_commit(E1->hash)
+ lookup_commit(E1)
@@
expression E1, E2;
@@
- lookup_commit_or_die(E1.hash, E2)
+ lookup_commit_or_die(&E1, E2)
@@
expression E1, E2;
@@
- lookup_commit_or_die(E1->hash, E2)
+ lookup_commit_or_die(E1, E2)
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are a small number of remaining callers of lookup_commit_reference
and lookup_commit_reference_gently that still need to be converted to
struct object_id. Convert these.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Parts of this module call lookup_commit_reference, which we want to
convert. The module is small and mostly self-contained, so convert the
rest of it while we're at it.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert update_squash_messages and is_index_unchanged to struct
object_id. These are callers of lookup_commit and
lookup_commit_reference, which we want to convert.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert register_shallow and unregister_shallow to take struct
object_id. register_shallow is a caller of lookup_commit, which we will
convert later. It doesn't make sense for the registration and
unregistration functions to have incompatible interfaces, so convert
them both.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a caller of lookup_commit_or_die, which we will convert later
on.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the internals of create_notes_comit and commit_notes to use
struct object_id.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Among the functions converted is a caller of lookup_commit_or_die, which
we will convert later on.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Specify some constants in terms of GIT_SHA1_HEXSZ, and convert a
get_sha1_hex into parse_oid_hex to avoid needing to specify additional
constants.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a prerequisite to convert to lookup_commit, which we will
convert later.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Adjust the callback functions to take struct object_id * instead of
unsigned char *, and modify related static functions accordingly.
Introduce a temporary object_id instance into files_reflog_expire and
copy the SHA-1 value passed in. This is necessary because the sha1
parameter can come indirectly from get_sha1. Without the temporary, it
would require much more refactoring to be able to convert this function.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a caller of lookup_commit_reference, which we will soon convert.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert as many instances of unsigned char [20] as possible. Update the
callers of notes_cache_get and notes_cache_put to use the new interface.
Among the functions updated are callers of
lookup_commit_reference_gently, which we will soon convert.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is a caller of lookup_commit_reference, which we will convert
later.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the remaining parts of fast-import.c to use struct object_id.
Convert several instances of get_sha1_hex to parse_oid_hex to avoid
needing to specify constants. Convert other hardcoded values to named
constants. Finally, use the is_empty_tree_oid function instead of a
direct comparison against a fixed string.
Note that the odd computation with GIT_MAX_HEXSZ is due to the insertion
of a slash between every two hex digits in the path, plus one for the
terminating NUL.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add packet_writel() which writes multiple lines in a single call and
then calls packet_flush_gently(). Update convert.c to use the new
packet_writel() function from pkt-line.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add packet_read_line_gently() to enable reading a line without dying on
EOF.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update packet_read_line() to test for len > 0 to avoid potential bug
if read functions return lengths less than zero to indicate errors.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Found/Fixed-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
start_multi_file_filter() and apply_multi_file_filter() currently test
for errno == EPIPE but treating EPIPE as an error is already happening
from one of the packet_write() functions.
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Found/Fixed-by: Jeff King <peff@peff.net>
Acked-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rebasing onto many changes is interesting, but it's also
interesting to see what happens when rebasing many changes.
And while at it, let's also look at the impact of using a
split index.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When split-index is being used, we have two cache_entry arrays in
index_state->cache[] and index_state->split_index->base->cache[].
index_state->cache[] may share the same entries with base->cache[] so
we can quickly determine what entries are shared. This makes memory
management tricky, we can't free base->cache[] until we know
index_state->cache[] does not point to any of those entries.
unshare_split_index() is added for this purpose, to find shared
entries and either duplicate them in index_state->cache[], or discard
them. Either way it should be safe to free base->cache[] after
unshare_split_index().
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When fd47ae6a5b (diff: teach diff to display submodule difference with an
inline diff, 2016-08-31) was introduced, we did not think of recursing
into nested submodules.
When showing the inline diff for submodules, automatically recurse
into nested submodules as well with inline submodule diffs.
Signed-off-by: Stefan Beller <sbeller@google.com>
Acked-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order to make it clearer where the_index is being referenced, stop
using the index compatibility macros in dir.c. This is to make it
easier to identify the functions which need to be convert to taking in a
'struct index_state' as a parameter.
The end goal would be to eliminate the need to reference global index
state in dir.c.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git for Windows CI returns "completed: failed" if a build or test
failure happened. This case was processed as "Unhandled status".
Handle the case explicitly.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some of the functions converted are callers of lookup_commit_reference.
However, the changes involved in converting the entire thing are not too
large, so we might as well convert it all.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function is a caller of lookup_commit_reference_gently, which we
will convert later.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the bundle code, plus the sole external user of struct
ref_list_entry, to use struct object_id. Include cache.h from within
bundle.h to provide the definition. Convert some of the hash parsing
code to use parse_oid_hex to avoid needing to hard-code constant values.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the sole instance of unsigned char [20] to struct object_id.
cmd_prune is a caller of parse_object, which we will convert later.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert all the uses of unsigned char [20] to struct object_id. Also,
convert some hard-coded integers into constants.
name_rev_line accepts a wide variety of free-form input and only
interprets 40-character hex values, passing through everything else.
Consequently, it is not a good candidate for parse_oid_hex, which is
much stricter.
This change is a prerequisite for converting parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert the sha1 member of struct cache_tree to struct object_id by
changing the definition and applying the following semantic patch, plus
the standard object_id transforms:
@@
struct cache_tree E1;
@@
- E1.sha1
+ E1.oid.hash
@@
struct cache_tree *E1;
@@
- E1->sha1
+ E1->oid.hash
Fix up one reference to active_cache_tree which was not automatically
caught by Coccinelle. These changes are prerequisites for converting
parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The semantic patch for standard object_id transforms found two
outstanding places where we could make a transformation automatically.
Apply these changes.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Convert all uses of unsigned char [20] to struct object_id. Switch one
use of get_sha1_hex to parse_oid_hex to avoid the need for a constant.
This change is necessary in order to convert parse_object.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are currently two instances (fetch and push) where we want to
determine if submodules have changed given some revision specification.
These two instances don't use the same logic to generate a list of
changed submodules and as a result there is a fair amount of code
duplication.
This patch refactors these two code paths such that they both use the
same logic to generate a list of changed submodules. This also makes it
easier for future callers to be able to reuse this logic as they only
need to create an argv_array with the revision specification to be using
during the revision walk.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach 'submodule_has_commits()' to ensure that if a commit exists in a
submodule, that it is also reachable from a ref.
This is a preparatory step prior to merging the logic which checks for
changed submodules when fetching or pushing.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Eliminate a call to 'xstrdup()' by changing the string_list
'changed_submodule_paths' to duplicated strings added to it.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function 'add_oid_to_argv()' provides the same functionality as
'append_oid_to_argv()'. Remove this duplicate function and instead use
'append_oid_to_argv()' where 'add_oid_to_argv()' was previously used.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename 'free_submodules_sha1s()' to 'free_submodules_oids()' since the
function frees a 'struct string_list' which has a 'struct oid_array'
stored in the 'util' field.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename 'add_sha1_to_array()' to 'append_oid_to_array()' to more
accurately describe what the function does, since it handles
'struct object_id' and not sha1 character arrays.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Rename the t5614-clone-submodules.sh test to
t5614-clone-submodules-shallow.sh. It's not a general test of
submodules, but of shallow cloning in relation to submodules. Move it
to create another similar t56*-clone-submodules-*.sh test.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a --no-tags option to clone without fetching any tags.
Without this change there's no easy way to clone a repository without
also fetching its tags.
When supplying --single-branch the primary remote branch will be
cloned, but in addition tags will be followed & retrieved. Now
--no-tags can be added --single-branch to clone a repository without
tags, and which only tracks a single upstream branch.
This option works without --single-branch as well, and will do a
normal clone but not fetch any tags.
Many git commands pay some fixed overhead as a function of the number
of references. E.g. creating ~40k tags in linux.git will cause a
command like `git log -1 >/dev/null` to run in over a second instead
of in a matter of milliseconds, in addition numerous other things will
slow down, e.g. "git log <TAB>" with the bash completion will slowly
show ~40k references instead of 1.
The user might want to avoid all of that overhead to simply use a
repository like that to browse the "master" branch, or something like
a CI tool might want to keep that one branch up-to-date without caring
about any other references.
Without this change the only way of accomplishing this was either by
manually tweaking the config in a fresh repository:
git init git &&
cat >git/.git/config <<EOF &&
[remote "origin"]
url = git@github.com:git/git.git
tagOpt = --no-tags
fetch = +refs/heads/master:refs/remotes/origin/master
[branch "master"]
remote = origin
merge = refs/heads/master
EOF
cd git &&
git pull
Which requires hardcoding the "master" name, which may not be the main
--single-branch would have retrieved, or alternatively by setting
tagOpt=--no-tags right after cloning & deleting any existing tags:
git clone --single-branch git@github.com:git/git.git &&
cd git &&
git config remote.origin.tagOpt --no-tags &&
git tag -l | xargs git tag -d
Which of course was also subtly buggy if --branch was pointed at a
tag, leaving the user in a detached head:
git clone --single-branch --branch v2.12.0 git@github.com:git/git.git &&
cd git &&
git config remote.origin.tagOpt --no-tags &&
git tag -l | xargs git tag -d
Now all this complexity becomes the much simpler:
git clone --single-branch --no-tags git@github.com:git/git.git
Or in the case of cloning a single tag "branch":
git clone --single-branch --branch v2.12.0 --no-tags git@github.com:git/git.git
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change occurrences "cd" followed by "fetch" on a single line to be on
two lines.
This is purely a stylistic change pointed out in code review for an
unrelated patch. Change the these tests use so new tests added later
using the more common style don't look out of place.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Check if unzip supports the ZIP64 format and skip the tests that create
big archives otherwise. Also skip the test that archives a big file on
32-bit platforms because the git object systems can't unpack files
bigger than 4GB there.
Reported-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add color config slots to be used in the status short-format when
displaying local and remote tracking branch information.
[jc: rebased on top of Peff's fix to 'git status' and tweaked the
test to check both local and remote-tracking branch output]
Signed-off-by: Stephen Kent <smkent@smkent.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When git-status shows tracking data for the current branch
in the long format, we try to end the stanza with a blank
line. When status.displayCommentPrefix is true, we call
color_fprintf_ln() to do so. But when it's false, we call
the enigmatic:
fputs("", s->fp);
which does nothing at all! This is a bug from 7d7d68022
(silence a bunch of format-zero-length warnings,
2014-05-04). Prior to that, we called fprintf_ln() with an
empty string. Switching to fputs() meant we needed to
include the "newline in the string, but we didn't.
So you see:
On branch jk/status-tracking-newline
Your branch is ahead of 'origin/master' by 1 commit.
Changes not staged for commit:
modified: foo
Untracked files:
bar
whereas there should be a blank line before the "Changes not
staged" line.
The fix itself is a one-liner. But we never noticed this
bug because t7508 doesn't exercise the ahead/behind code at
all. So let's configure an upstream during the initial
setup, which means that the code will be exercised as part
of all of the various invocations in that script. This makes
the diff rather noisy, but should give us good coverage.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signal that extractors need to implement spec version 4.5 (or higher)
for files with sizes of 4GB and more. Older unzippers might produce
truncated results otherwise; they should rather refuse to extract.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, we used `unsigned long` for timestamps. This was only a good
choice on Linux, where we know implicitly that `unsigned long` is what is
used for `time_t`.
However, we want to use a different data type for timestamps for two
reasons:
- there is nothing that says that `unsigned long` should be the same data
type as `time_t`, and indeed, on 64-bit Windows for example, it is not:
`unsigned long` is 32-bit but `time_t` is 64-bit.
- even on 32-bit Linux, where `unsigned long` (and thereby `time_t`) is
32-bit, we *want* to be able to encode timestamps in Git that are
currently absurdly far in the future, *even if* the system library is
not able to format those timestamps into date strings.
So let's just switch to the maximal integer type available, which should
be at least 64-bit for all practical purposes these days. It certainly
cannot be worse than `unsigned long`, so...
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We are about to switch to a new data type for time stamps that is
definitely not smaller or equal, but larger or equal to time_t.
So before using the system functions to process or format timestamps,
let's make extra certain that they can handle what we feed them.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git's source code assumes that unsigned long is at least as precise as
time_t. Which is incorrect, and causes a lot of problems, in particular
where unsigned long is only 32-bit (notably on Windows, even in 64-bit
versions).
So let's just use a more appropriate data type instead. In preparation
for this, we introduce the new `timestamp_t` data type.
By necessity, this is a very, very large patch, as it has to replace all
timestamps' data type in one go.
As we will use a data type that is not necessarily identical to `time_t`,
we need to be very careful to use `time_t` whenever we interact with the
system functions, and `timestamp_t` everywhere else.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We already do so for --window=<n> and --depth=<n>; this will help
when the user wants to force --threads=1 for reproducible testing
without getting affected by racing multiple threads.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The convention "$remove_trash is set to the trash directory that is
used during the test, so that it will be removed at the end, but
under --debug option we set the varilable to empty string to
preserve the directory" made sense back when it was introduced, as
there was no $TRASH_DIRECTORY variable. These days, since no tests
looks at the variable, it is obscure and even risks that by mistake
the variable gets used for something else (e.g. remove_trash=yes)
and cause us misbehave. Worse yet, remove_trash was not initialized
to an empty string at the beginning, so a stray environment variable
the user has could have affected the logic when "--debug" is in use.
Rewrite the clean-up sequence in test_done helper to explicitly
check the $debug condition and remove the trash directory using
the $TRASH_DIRECTORY variable.
Note that "go to the directory one level above the trash and then
remove it" is kept and this is deliverate; test_at_end_hook_ will
keep running from the expected location, and also some platforms may
not like a directory that is serving as the $cwd of a still-active
process removed.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The original did "does $remove_trash exist? Then go one level above
and remove it". There was no problem under "--debug", where
the variable is left empty, as the first "test -d $remove_trash" would
have said "No, it doesn't".
With the check implemented in the previous step, we'd always get an
error under "--debug".
Noticed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Write a zip64 extended information extra field for big files as part of
their local headers and as part of their central directory headers.
Also write a zip64 version of the data descriptor in that case.
If we're streaming then we don't know the compressed size at the time we
write the header. Deflate can end up making a file bigger instead of
smaller if we're unlucky. Write a local zip64 header already for files
with a size of 2GB or more in this case to be on the safe side.
Both sizes need to be included in the local zip64 header, but the extra
field for the directory must only contain 64-bit equivalents for 32-bit
values of 0xffffffff.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a zip64 extended information extra field to the central directory
and emit the zip64 end of central directory records as well as locator
if the offset of an entry within the archive exceeds 4GB.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Write all fields of the ZIP directory record for an archive entry
in the right order directly into the strbuf instead of taking a detour
through a struct. Do that at end, when we have all necessary data like
checksum and compressed size. The fields are documented just as well,
the code becomes shorter and we save an extra copy.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Keep the ZIP central directory, which is written after all archive
entries, in a strbuf instead of a custom-managed buffer. It contains
binary data, so we can't (and don't want to) use the full range of
strbuf functions and we don't need the terminating NUL, but the result
is shorter and simpler code.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Test the creation of ZIP archives bigger than 4GB and containing files
bigger than 4GB. They are marked as EXPENSIVE because they take quite a
while and because the first one needs a bit more than 4GB of disk space
to store the resulting archive.
The big archive in the first test is made up of a tree containing
thousands of copies of a small file. Yet the test has to write out the
full archive because unzip doesn't offer a way to read from stdin.
The big file in the second test is provided as a zipped pack file to
avoid writing another 4GB file to disk and then adding it.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
70999e9cec (branch -m: update all per-worktree HEADs - 2016-03-27)
added this function in order to update HEADs of all relevant
worktrees, when a branch is renamed.
It, as a public ref api, kind of breaks abstraction when it uses
internal functions of files backend. With the introduction of
refs_create_symref(), we can move back pretty close to the code before
70999e9cec, where create_symref() was used for updating HEAD.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The manual parsing code is replaced with a call to refs_resolve_ref_unsafe().
The manual parsing code must die because only refs/files-backend.c
should do that.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
files-backend at this point is still aware of the per-repo/worktree
separation in refs, so it can handle a linked worktree.
Some refs operations are known not working when current files-backend is
used in a linked worktree (e.g. reflog). Tests will be written when
refs_* functions start to be called with worktree backend to verify that
they work as expected.
Note: accessing a worktree of a submodule remains unaddressed. Perhaps
after get_worktrees() can access submodule (or rather a new function
get_submodule_worktrees(), that lists worktrees of a submodule), we can
update this function to work with submodules as well.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, Git's source code treats all timestamps as if they were
unsigned longs. Therefore, it is okay to write "%lu" when printing them.
There is a substantial problem with that, though: at least on Windows,
time_t is *larger* than unsigned long, and hence we will want to switch
away from the ill-specified `unsigned long` data type.
So let's introduce the pseudo format "PRItime" (currently simply being
defined to "lu") to make it easier to change the data type used for
timestamps.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, Git's source code represents all timestamps as `unsigned
long`. In preparation for using a more appropriate data type, let's
introduce a symbol `parse_timestamp` (currently being defined to
`strtoul`) where appropriate, so that we can later easily switch to,
say, use `strtoull()` instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We generally disallow null sha1s from entering the index,
due to 4337b5856 (do not write null sha1s to on-disk index,
2012-07-28). However, we loosened that in 83bd7437c
(write_index: optionally allow broken null sha1s,
2013-08-27) so that tools like filter-branch could be used
to repair broken history.
However, we should make sure that these broken entries do
not get propagated into new trees. For most entries, we'd
catch them with the missing-object check (since presumably
the null sha1 does not exist in our object database). But
gitlink entries do not need reachability, so we may blindly
copy the entry into a bogus tree.
This patch rejects all null sha1s (with the same "invalid
entry" message that missing objects get) when building trees
from the index. It does so even for non-gitlinks, and even
when "write-tree" is given the --missing-ok flag. The null
sha1 is a special sentinel value that is already rejected in
trees by fsck; whether the object exists or not, it is an
error to put it in a tree.
Note that for this to work, we must also avoid reusing an
existing cache-tree that contains the null sha1. This patch
does so by just refusing to write out any cache tree when
the index contains a null sha1. This is blunter than we need
to be; we could just reject the subtree that contains the
offending entry. But it's not worth the complexity. The
behavior is unchanged unless you have a broken index entry,
and even then we'd refuse the whole index write unless the
emergency GIT_ALLOW_NULL_SHA1 is in use. And even then the
end result is only a performance drop (any write-tree will
have to generate the whole cache-tree from scratch).
The tests bear some explanation.
The existing test in t7009 doesn't catch this problem,
because our index-filter runs "git rm --cached", which will
try to rewrite the updated index and barf on the bogus
entry. So we never even make it to write-tree. The new test
there adds a noop index-filter, which does show the problem.
The new tests in t1601 are slightly redundant with what
filter-branch is doing under the hood in t7009. But as
they're much more direct, they're easier to reason about.
And should filter-branch ever change or go away, we'd want
to make sure that these plumbing commands behave sanely.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-reset is yet another working tree manipulator, which should
be taught about submodules.
When a user uses git-reset and requests to recurse into submodules,
this will reset the submodules to the object name as recorded in the
superproject, detaching the HEADs.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We had two similar bugs in the tests sporadically triggering error
messages during the removal of the trash directory, see commits
bb05510e5 (t5510: run auto-gc in the foreground, 2016-05-01) and
ef09036cf (t6500: wait for detached auto gc at the end of the test
script, 2017-04-13). The test script succeeded nonetheless, because
these errors are ignored during housekeeping in 'test_done'.
However, such an error is a sign that something is fishy in the test
script. Print an error message and abort the test script when the
trash directory can't be removed successfully or is already removed,
because that's unexpected and we would prefer somebody notice and
figure out why.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git's source code refers to timestamps as unsigned long, which is
ill-defined, as there is no guarantee about the number of bits that
data type has.
In preparation of switching to another data type that is large enough
to hold "far in the future" dates, we need to prepare the t0006-date.sh
script for the case where we *still* cannot format those dates if the
system library uses 32-bit time_t.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git's source code refers to timestamps as unsigned longs. On 32-bit
platforms, as well as on Windows, unsigned long is not large enough to
capture dates that are "absurdly far in the future".
It is perfectly valid by the C standard, of course, for the `long` data
type to refer to 32-bit integers. That is why the `time_t` data type
exists: so that it can be 64-bit even if `long` is 32-bit. Git's source
code simply uses an incorrect data type for timestamps, is all.
The earlier quick fix 6b9c38e14c (t0006: skip "far in the future" test
when unsigned long is not long enough, 2016-07-11) papered over this
issue simply by skipping the respective test cases on platforms where
they would fail due to the data type in use.
This quick fix, however, tests for *long* to be 64-bit or not. What we
need, though, is a test that says whether *whatever data type we use for
timestamps* is 64-bit or not.
The same quick fix was used to handle the similar problem where Git's
source code uses `unsigned long` to represent size, instead of `size_t`,
conflating the two issues.
So let's just add another prerequisite to test specifically whether
timestamps are represented by a 64-bit data type or not. Later, after we
switch to a larger data type, we can flip that prerequisite to test
`time_t` instead of `long`.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In its `atom_value` struct, the ref-filter source code wants to store
different values in a field called `ul` (for `unsigned long`), e.g.
timestamps.
However, as we are about to switch the data type of timestamps away from
`unsigned long` (because it may be 32-bit even when `time_t` is 64-bit),
that data type is not large enough.
Simply change that field to use `uintmax_t` instead.
This patch is a bit larger than the mere change of the data type
because the field's name was tied to its data type, which has been fixed
at the same time.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When core.packedGitLimit is exceeded, git will close packs. If there
is a repack operation going on in parallel with a fetch, the fetch
might open a pack, and then be forced to close it due to
packedGitLimit being hit. The repack could then delete the pack
out from under the fetch, causing the fetch to fail.
Increase core.packedGitLimit's default value to prevent
this.
On current 64-bit x86_64 machines, 48 bits of address space are
available. It appears that 64-bit ARM machines have no standard
amount of address space (that is, it varies by manufacturer), and IA64
and POWER machines have the full 64 bits. So 48 bits is the only
limit that we can reasonably care about. We reserve a few bits of the
48-bit address space for the kernel's use (this is not strictly
necessary, but it's better to be safe), and use up to the remaining
45. No git repository will be anywhere near this large any time soon,
so this should prevent the failure.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: David Turner <dturner@twosigma.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Early on in submodule_move_head just after the check if the submodule is
initialized, we need to check if the submodule is populated correctly.
If the submodule is initialized but doesn't look like it is populated,
this is a red flag and can indicate multiple sorts of failures:
(1) The submodule may be recorded at an object name, that is missing.
(2) The submodule '.git' file link may be broken and it is not pointing
at a repository.
In both cases we want to complain to the user in the non-forced mode,
and in the forced mode ignoring the old state and just moving the
submodule into its new state with a fixed '.git' file link.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This was an oversight when working on the working tree modifying commands
recursing into submodules.
To test for uninitialized submodules, introduce another submodule
"uninitialized_sub". Adding it via `submodule add` will activate the
submodule in the preparation area (in create_lib_submodule_repo we
setup all the things in submodule_update_repo), but the later tests
will use a new testing repo that clones the preparation repo
in which the new submodule is not initialized.
By adding it to the branch "add_sub1", which is the starting point of
all other branches, we have wide coverage.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In case of a non-forced worktree update, the submodule movement is tested
in a dry run first, such that it doesn't matter if the actual update is
done via the force flag. However for correctness, we want to give the
flag as specified by the user. All callers have been inspected and updated
if needed.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This removes the "submodule" from submodule_hash_entry and other
function names. The goal is to reuse the same code and data structure
for other ref store types. The first one is worktree ref stores.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
setup_git_env() must be called before this function to initialize
git_common_dir so that it returns a non NULL string. And it must return
a non NULL string or segfault can happen because all callers expect so.
It does not do so explicitly though and depends on get_git_dir() being
called first (which will guarantee setup_git_env()). Avoid this
dependency and call setup_git_env() by itself.
test-ref-store.c will hit this problem because it's very lightweight,
just enough initialization to exercise refs code, and get_git_dir() will
never be called until get_worktrees() is, which uses get_git_common_dir
and hits a segfault.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The gui.recentrepo list may be longer than the maxrecent setting.
Allow extra space to show any extra entries.
In an ideal world, the git gui would limit the number of entries
to the maxrecent setting, however the recentrepo config list may
have been extended outwith the gui, or the maxrecent setting changed
to a reduced value. Further, when testing the gui's recentrepo
logic it is useful to show these extra, but valid, entries.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
When the gui/user selects a repo for display, that repo is brought to
the end of the recentrepo config list. The logic can fail if there are
duplicate old entries for the repo (you cannot unset a single config
entry when duplicates are present).
Similarly, the maxrecentrepo logic could fail if older duplicate entries
are present.
The first commit of this series ({this}~2) fixed the config unsetting
issue. Rather than manipulating a local copy of the $recent list (one
cannot know how many entries were removed), simply re-read it.
We must also catch the error when the attempt to remove the second copy
from the re-read list is performed.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
_get_recentrepo will fail if duplicate invalid entries are present
in the recentrepo config list. The previous commit fixed the
'git config' limitations in _unset_recentrepo by unsetting all config
entries, however this code would fail on the second attempt to unset it.
Refactor the code to pre-sort and de-duplicate the recentrepo list to
avoid a potential second unset attempt.
Signed-off-by: Philip Oakley <philipoakley@iee.org>
The git gui's recent repo list may become contaminated with duplicate
entries. The git gui would barf when attempting to remove one entry.
Remove them all - there is no option within 'git config' to selectively
remove one of the entries.
This issue was reported on the 'Git User' list
(https://groups.google.com/forum/#!topic/git-users/msev4KsQGFc,
Warning: gui.recentrepo has multiply values while executing).
And also by zosrothko as a Git-for-Windows issue
https://github.com/git-for-windows/git/issues/1014.
On startup the gui checks that entries in the recentrepo list are still
valid repos and deletes thoses that are not. If duplicate entries are
present the 'git config --unset' will barf and this prevents the gui
from starting.
Subsequent patches fix other parts of recentrepo logic used for syncing
internal lists with the external .gitconfig.
Reported-by: Alexey Astakhov <asstv7@gmail.com>
Signed-off-by: Philip Oakley <philipoakley@iee.org>
2017-01-20 20:55:01 +00:00
516 changed files with 51015 additions and 40066 deletions
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
