Compare commits

...

101 Commits

Author SHA1 Message Date
7556e5d737 Git 2.38.4
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:43:30 +01:00
6487e9c459 Sync with 2.37.6
* maint-2.37:
  Git 2.37.6
  Git 2.36.5
  Git 2.35.7
  Git 2.34.7
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  Git 2.33.7
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:43:28 +01:00
eb88fe1ff5 Git 2.37.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:38:32 +01:00
16004682f9 Sync with 2.36.5
* maint-2.36:
  Git 2.36.5
  Git 2.35.7
  Git 2.34.7
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  Git 2.33.7
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:38:31 +01:00
673472a963 Git 2.36.5
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:37:53 +01:00
40843216c5 Sync with 2.35.7
* maint-2.35:
  Git 2.35.7
  Git 2.34.7
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  Git 2.33.7
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:37:52 +01:00
b7a92d078b Git 2.35.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:29:45 +01:00
6a53a59bf9 Sync with 2.34.7
* maint-2.34:
  Git 2.34.7
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
  Git 2.33.7
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:29:44 +01:00
91da4a29e1 Git 2.34.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:29:17 +01:00
a7237f5ae9 Sync with 2.33.7
* maint-2.33:
  Git 2.33.7
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:29:16 +01:00
bd6d3de01f Merge branch 'jk/curl-avoid-deprecated-api'
Deal with a few deprecation warning from cURL library.

* jk/curl-avoid-deprecated-api:
  http: support CURLOPT_PROTOCOLS_STR
  http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
  http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
2023-02-06 09:27:41 +01:00
f44e6a2105 http: support CURLOPT_PROTOCOLS_STR
The CURLOPT_PROTOCOLS (and matching CURLOPT_REDIR_PROTOCOLS) flag was
deprecated in curl 7.85.0, and using it generate compiler warnings as of
curl 7.87.0. The path forward is to use CURLOPT_PROTOCOLS_STR, but we
can't just do so unilaterally, as it was only introduced less than a
year ago in 7.85.0.

Until that version becomes ubiquitous, we have to either disable the
deprecation warning or conditionally use the "STR" variant on newer
versions of libcurl. This patch switches to the new variant, which is
nice for two reasons:

  - we don't have to worry that silencing curl's deprecation warnings
    might cause us to miss other more useful ones

  - we'd eventually want to move to the new variant anyway, so this gets
    us set up (albeit with some extra ugly boilerplate for the
    conditional)

There are a lot of ways to split up the two cases. One way would be to
abstract the storage type (strbuf versus a long), how to append
(strbuf_addstr vs bitwise OR), how to initialize, which CURLOPT to use,
and so on. But the resulting code looks pretty magical:

  GIT_CURL_PROTOCOL_TYPE allowed = GIT_CURL_PROTOCOL_TYPE_INIT;
  if (...http is allowed...)
	GIT_CURL_PROTOCOL_APPEND(&allowed, "http", CURLOPT_HTTP);

and you end up with more "#define GIT_CURL_PROTOCOL_TYPE" macros than
actual code.

On the other end of the spectrum, we could just implement two separate
functions, one that handles a string list and one that handles bits. But
then we end up repeating our list of protocols (http, https, ftp, ftp).

This patch takes the middle ground. The run-time code is always there to
handle both types, and we just choose which one to feed to curl.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:27:09 +01:00
4bd481e0ad http: prefer CURLOPT_SEEKFUNCTION to CURLOPT_IOCTLFUNCTION
The IOCTLFUNCTION option has been deprecated, and generates a compiler
warning in recent versions of curl. We can switch to using SEEKFUNCTION
instead. It was added in 2008 via curl 7.18.0; our INSTALL file already
indicates we require at least curl 7.19.4.

But there's one catch: curl says we should use CURL_SEEKFUNC_{OK,FAIL},
and those didn't arrive until 7.19.5. One workaround would be to use a
bare 0/1 here (or define our own macros).  But let's just bump the
minimum required version to 7.19.5. That version is only a minor version
bump from our existing requirement, and is only a 2 month time bump for
versions that are almost 13 years old. So it's not likely that anybody
cares about the distinction.

Switching means we have to rewrite the ioctl functions into seek
functions. In some ways they are simpler (seeking is the only
operation), but in some ways more complex (the ioctl allowed only a full
rewind, but now we can seek to arbitrary offsets).

Curl will only ever use SEEK_SET (per their documentation), so I didn't
bother implementing anything else, since it would naturally be
completely untested. This seems unlikely to change, but I added an
assertion just in case.

Likewise, I doubt curl will ever try to seek outside of the buffer sizes
we've told it, but I erred on the defensive side here, rather than do an
out-of-bounds read.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:27:09 +01:00
4fab049258 http-push: prefer CURLOPT_UPLOAD to CURLOPT_PUT
The two options do exactly the same thing, but the latter has been
deprecated and in recent versions of curl may produce a compiler
warning. Since the UPLOAD form is available everywhere (it was
introduced in the year 2000 by curl 7.1), we can just switch to it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:27:08 +01:00
ed4404af3c Git 2.33.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:25:58 +01:00
87248c5933 Sync with 2.32.6
* maint-2.32:
  Git 2.32.6
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:25:56 +01:00
2aedeff35f Git 2.32.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:25:09 +01:00
aeb93d7da2 Sync with 2.31.7
* maint-2.31:
  Git 2.31.7
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:25:08 +01:00
0bbcf95194 Git 2.31.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:24:07 +01:00
e14d6b8408 Sync with 2.30.8
* maint-2.30:
  Git 2.30.8
  apply: fix writing behind newly created symbolic links
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:24:06 +01:00
394a759d2b Git 2.30.8
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-06 09:14:45 +01:00
a3033a68ac Merge branch 'ps/apply-beyond-symlink' into maint-2.30
Fix a vulnerability (CVE-2023-23946) that allows crafted input to trick
`git apply` into writing files outside of the working tree.

* ps/apply-beyond-symlink:
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
2023-02-06 09:12:16 +01:00
2c9a4c7310 Merge branch 'tb/clone-local-symlinks' into maint-2.30
Resolve a security vulnerability (CVE-2023-22490) where `clone_local()`
is used in conjunction with non-local transports, leading to arbitrary
path exfiltration.

* tb/clone-local-symlinks:
  dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
  clone: delay picking a transport until after get_repo_path()
  t5619: demonstrate clone_local() with ambiguous transport
2023-02-06 09:09:14 +01:00
fade728df1 apply: fix writing behind newly created symbolic links
When writing files git-apply(1) initially makes sure that none of the
files it is about to create are behind a symlink:

```
 $ git init repo
 Initialized empty Git repository in /tmp/repo/.git/
 $ cd repo/
 $ ln -s dir symlink
 $ git apply - <<EOF
 diff --git a/symlink/file b/symlink/file
 new file mode 100644
 index 0000000..e69de29
 EOF
 error: affected file 'symlink/file' is beyond a symbolic link
```

This safety mechanism is crucial to ensure that we don't write outside
of the repository's working directory. It can be fooled though when the
patch that is being applied creates the symbolic link in the first
place, which can lead to writing files in arbitrary locations.

Fix this by checking whether the path we're about to create is
beyond a symlink or not. Tightening these checks like this should be
fine as we already have these precautions in Git as explained
above. Ideally, we should update the check we do up-front before
starting to reflect the computed changes to the working tree so that
we catch this case as well, but as part of embargoed security work,
adding an equivalent check just before we try to write out a file
should serve us well as a reasonable first step.

Digging back into history shows that this vulnerability has existed
since at least Git v2.9.0. As Git v2.8.0 and older don't build on my
system anymore I cannot tell whether older versions are affected, as
well.

Reported-by: Joern Schneeweisz <jschneeweisz@gitlab.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-02-03 14:41:31 -08:00
bffc762f87 dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS
When using the dir_iterator API, we first stat(2) the base path, and
then use that as a starting point to enumerate the directory's contents.

If the directory contains symbolic links, we will immediately die() upon
encountering them without the `FOLLOW_SYMLINKS` flag. The same is not
true when resolving the top-level directory, though.

As explained in a previous commit, this oversight in 6f054f9fb3
(builtin/clone.c: disallow `--local` clones with symlinks, 2022-07-28)
can be used as an attack vector to include arbitrary files on a victim's
filesystem from outside of the repository.

Prevent resolving top-level symlinks unless the FOLLOW_SYMLINKS flag is
given, which will cause clones of a repository with a symlink'd
"$GIT_DIR/objects" directory to fail.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24 16:52:16 -08:00
cf8f6ce02a clone: delay picking a transport until after get_repo_path()
In the previous commit, t5619 demonstrates an issue where two calls to
`get_repo_path()` could trick Git into using its local clone mechanism
in conjunction with a non-local transport.

That sequence is:

 - the starting state is that the local path https:/example.com/foo is a
   symlink that points to ../../../.git/modules/foo. So it's dangling.

 - get_repo_path() sees that no such path exists (because it's
   dangling), and thus we do not canonicalize it into an absolute path

 - because we're using --separate-git-dir, we create .git/modules/foo.
   Now our symlink is no longer dangling!

 - we pass the url to transport_get(), which sees it as an https URL.

 - we call get_repo_path() again, on the url. This second call was
   introduced by f38aa83f9a (use local cloning if insteadOf makes a
   local URL, 2014-07-17). The idea is that we want to pull the url
   fresh from the remote.c API, because it will apply any aliases.

And of course now it sees that there is a local file, which is a
mismatch with the transport we already selected.

The issue in the above sequence is calling `transport_get()` before
deciding whether or not the repository is indeed local, and not passing
in an absolute path if it is local.

This is reminiscent of a similar bug report in [1], where it was
suggested to perform the `insteadOf` lookup earlier. Taking that
approach may not be as straightforward, since the intent is to store the
original URL in the config, but to actually fetch from the insteadOf
one, so conflating the two early on is a non-starter.

Note: we pass the path returned by `get_repo_path(remote->url[0])`,
which should be the same as `repo_name` (aside from any `insteadOf`
rewrites).

We *could* pass `absolute_pathdup()` of the same argument, which
86521acaca (Bring local clone's origin URL in line with that of a remote
clone, 2008-09-01) indicates may differ depending on the presence of
".git/" for a non-bare repo. That matters for forming relative submodule
paths, but doesn't matter for the second call, since we're just feeding
it to the transport code, which is fine either way.

[1]: https://lore.kernel.org/git/CAMoD=Bi41mB3QRn3JdZL-FGHs4w3C2jGpnJB-CqSndO7FMtfzA@mail.gmail.com/

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24 16:52:16 -08:00
58325b93c5 t5619: demonstrate clone_local() with ambiguous transport
When cloning a repository, Git must determine (a) what transport
mechanism to use, and (b) whether or not the clone is local.

Since f38aa83f9a (use local cloning if insteadOf makes a local URL,
2014-07-17), the latter check happens after the remote has been
initialized, and references the remote's URL instead of the local path.
This is done to make it possible for a `url.<base>.insteadOf` rule to
convert a remote URL into a local one, in which case the `clone_local()`
mechanism should be used.

However, with a specially crafted repository, Git can be tricked into
using a non-local transport while still setting `is_local` to "1" and
using the `clone_local()` optimization. The below test case
demonstrates such an instance, and shows that it can be used to include
arbitrary (known) paths in the working copy of a cloned repository on a
victim's machine[^1], even if local file clones are forbidden by
`protocol.file.allow`.

This happens in a few parts:

 1. We first call `get_repo_path()` to see if the remote is a local
    path. If it is, we replace the repo name with its absolute path.

 2. We then call `transport_get()` on the repo name and decide how to
    access it. If it was turned into an absolute path in the previous
    step, then we should always treat it like a file.

 3. We use `get_repo_path()` again, and set `is_local` as appropriate.
    But it's already too late to rewrite the repo name as an absolute
    path, since we've already fed it to the transport code.

The attack works by including a submodule whose URL corresponds to a
path on disk. In the below example, the repository "sub" is reachable
via the dumb HTTP protocol at (something like):

    http://127.0.0.1:NNNN/dumb/sub.git

However, the path "http:/127.0.0.1:NNNN/dumb" (that is, a top-level
directory called "http:", then nested directories "127.0.0.1:NNNN", and
"dumb") exists within the repository, too.

To determine this, it first picks the appropriate transport, which is
dumb HTTP. It then uses the remote's URL in order to determine whether
the repository exists locally on disk. However, the malicious repository
also contains an embedded stub repository which is the target of a
symbolic link at the local path corresponding to the "sub" repository on
disk (i.e., there is a symbolic link at "http:/127.0.0.1/dumb/sub.git",
pointing to the stub repository via ".git/modules/sub/../../../repo").

This stub repository fools Git into thinking that a local repository
exists at that URL and thus can be cloned locally. The affected call is
in `get_repo_path()`, which in turn calls `get_repo_path_1()`, which
locates a valid repository at that target.

This then causes Git to set the `is_local` variable to "1", and in turn
instructs Git to clone the repository using its local clone optimization
via the `clone_local()` function.

The exploit comes into play because the stub repository's top-level
"$GIT_DIR/objects" directory is a symbolic link which can point to an
arbitrary path on the victim's machine. `clone_local()` resolves the
top-level "objects" directory through a `stat(2)` call, meaning that we
read through the symbolic link and copy or hardlink the directory
contents at the destination of the link.

In other words, we can get steps (1) and (3) to disagree by leveraging
the dangling symlink to pick a non-local transport in the first step,
and then set is_local to "1" in the third step when cloning with
`--separate-git-dir`, which makes the symlink non-dangling.

This can result in data-exfiltration on the victim's machine when
sensitive data is at a known path (e.g., "/home/$USER/.ssh").

The appropriate fix is two-fold:

 - Resolve the transport later on (to avoid using the local
   clone optimization with a non-local transport).

 - Avoid reading through the top-level "objects" directory when
   (correctly) using the clone_local() optimization.

This patch merely demonstrates the issue. The following two patches will
implement each part of the above fix, respectively.

[^1]: Provided that any target directory does not contain symbolic
  links, in which case the changes from 6f054f9fb3 (builtin/clone.c:
  disallow `--local` clones with symlinks, 2022-07-28) will abort the
  clone.

Reported-by: yvvdwf <yvvdwf@gmail.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-24 16:52:16 -08:00
b78628d426 Sync with maint-2.37
* maint-2.37:
  attr: adjust a mismatched data type
2023-01-19 13:48:26 -08:00
f2027d2626 Sync with maint-2.36
* maint-2.36:
  attr: adjust a mismatched data type
2023-01-19 13:48:17 -08:00
5c1fc48d68 Sync with maint-2.35
* maint-2.35:
  attr: adjust a mismatched data type
2023-01-19 13:48:08 -08:00
c508c30968 Sync with maint-2.34
* maint-2.34:
  attr: adjust a mismatched data type
2023-01-19 13:48:00 -08:00
f39fe8fcb2 Sync with maint-2.33
* maint-2.33:
  attr: adjust a mismatched data type
2023-01-19 13:47:42 -08:00
25d7cb600c Sync with maint-2.32
* maint-2.32:
  attr: adjust a mismatched data type
2023-01-19 13:46:04 -08:00
012e0d76dc Sync with maint-2.31
* maint-2.31:
  attr: adjust a mismatched data type
2023-01-19 13:45:37 -08:00
f8bf6b8f3d Sync with maint-2.30
* maint-2.30:
  attr: adjust a mismatched data type
2023-01-19 13:45:23 -08:00
0227130244 attr: adjust a mismatched data type
On platforms where `size_t` does not have the same width as `unsigned
long`, passing a pointer to the former when a pointer to the latter is
expected can lead to problems.

Windows and 32-bit Linux are among the affected platforms.

In this instance, we want to store the size of the blob that was read in
that variable. However, `read_blob_data_from_index()` passes that
pointer to `read_object_file()` which expects an `unsigned long *`.
Which means that on affected platforms, the variable is not fully
populated and part of its value is left uninitialized. (On Big-Endian
platforms, this problem would be even worse.)

The consequence is that depending on the uninitialized memory's
contents, we may erroneously reject perfectly fine attributes.

Let's address this by passing a pointer to a variable of the expected
data type.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-01-19 13:38:06 -08:00
37ed7bf0f1 Git 2.38.3
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:24:14 +09:00
fea9f607a8 Sync with Git 2.37.5 2022-12-13 21:23:36 +09:00
e43ac5f23d Git 2.37.5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:20:47 +09:00
431f6e67e6 Merge branch 'maint-2.36' into maint-2.37 2022-12-13 21:20:35 +09:00
ad949b24f8 Git 2.36.4
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:19:24 +09:00
8253c00421 Merge branch 'maint-2.35' into maint-2.36 2022-12-13 21:19:11 +09:00
02f4981723 Git 2.35.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:17:26 +09:00
fbabbc30e7 Merge branch 'maint-2.34' into maint-2.35 2022-12-13 21:17:10 +09:00
6c9466944c Git 2.34.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:15:39 +09:00
3748b5b7f5 Merge branch 'maint-2.33' into maint-2.34 2022-12-13 21:15:22 +09:00
7fe9bf55b8 Git 2.33.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:13:48 +09:00
5f22dcc02d Sync with Git 2.32.5 2022-12-13 21:13:11 +09:00
d96ea538e8 Git 2.32.5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:10:27 +09:00
32e357b6df Merge branch 'ps/attr-limits-with-fsck' into maint-2.32 2022-12-13 21:09:56 +09:00
8a755eddf5 Sync with Git 2.31.6 2022-12-13 21:09:40 +09:00
82689d5e5d Git 2.31.6
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 21:04:03 +09:00
16128765d7 Sync with Git 2.30.7 2022-12-13 21:02:20 +09:00
b7b37a3371 Git 2.30.7
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-13 20:56:43 +09:00
8706a59933 Git 2.38.2
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-11 09:32:48 +09:00
0ddd73fa9f ci: use a newer github-script version
The old version we currently use runs in node.js v12.x, which is being
deprecated in GitHub Actions. The new version uses node.js v16.x.

Incidentally, this also avoids the warning about the deprecated
`::set-output::` workflow command because the newer version of the
`github-script` Action uses the recommended new way to specify outputs.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-12-10 16:32:16 +09:00
e71f00f73f Merge branch 'jx/ci-ubuntu-fix' into maint-2.38
Adjust the GitHub CI to newer ubuntu release.

* jx/ci-ubuntu-fix:
  ci: install python on ubuntu
  ci: use the same version of p4 on both Linux and macOS
  ci: remove the pipe after "p4 -V" to catch errors
  github-actions: run gcc-8 on ubuntu-20.04 image
2022-12-10 16:17:47 +09:00
ec9816c6b3 Merge branch 'js/ci-use-newer-up-down-artifact' into maint-2.38
CI fix.

* js/ci-use-newer-up-down-artifact:
  ci: avoid using deprecated {up,down}load-artifacts Action
2022-12-10 14:02:09 +09:00
75efbc1372 Merge branch 'ab/ci-use-macos-12' into maint-2.38
CI fix.

* ab/ci-use-macos-12:
  CI: upgrade to macos-12, and pin OSX version
2022-12-10 14:02:09 +09:00
634d026866 Merge branch 'ab/ci-retire-set-output' into maint-2.38
CI fix.

* ab/ci-retire-set-output:
  CI: migrate away from deprecated "set-output" syntax
2022-12-10 14:02:09 +09:00
8972be0252 Merge branch 'ab/ci-musl-bash-fix' into maint-2.38
CI fix.

* ab/ci-musl-bash-fix:
  CI: don't explicitly pick "bash" shell outside of Windows, fix regression
2022-12-10 14:02:09 +09:00
78c5de91f2 Merge branch 'od/ci-use-checkout-v3-when-applicable' into maint-2.38
Update GitHub CI to use actions/checkout@v3; use of the older
checkout@v2 gets annoying deprecation notices.

* od/ci-use-checkout-v3-when-applicable:
  ci(main): upgrade actions/checkout to v3
2022-12-10 14:02:09 +09:00
93a7bc8b28 rebase --update-refs: avoid unintended ref deletion
In b3b1a21d1a (sequencer: rewrite update-refs as user edits todo list,
2022-07-19), the 'todo_list_filter_update_refs()' step was added to handle
the removal of 'update-ref' lines from a 'rebase-todo'. Specifically, it
removes potential ref updates from the "update refs state" if a ref does not
have a corresponding 'update-ref' line.

However, because 'write_update_refs_state()' will not update the state if
the 'refs_to_oids' list was empty, removing *all* 'update-ref' lines will
result in the state remaining unchanged from how it was initialized (with
all refs' "after" OID being null). Then, when the ref update is applied, all
refs will be updated to null and consequently deleted.

To fix this, delete the 'update-refs' state file when 'refs_to_oids' is
empty. Additionally, add a tests covering "all update-ref lines removed"
cases.

Reported-by: herr.kaste <herr.kaste@gmail.com>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Victoria Dye <vdye@github.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
2022-12-09 19:31:45 +09:00
27ab4784d5 fsck: implement checks for gitattributes
Recently, a vulnerability was reported that can lead to an out-of-bounds
write when reading an unreasonably large gitattributes file. The root
cause of this error are multiple integer overflows in different parts of
the code when there are either too many lines, when paths are too long,
when attribute names are too long, or when there are too many attributes
declared for a pattern.

As all of these are related to size, it seems reasonable to restrict the
size of the gitattributes file via git-fsck(1). This allows us to both
stop distributing known-vulnerable objects via common hosting platforms
that have fsck enabled, and users to protect themselves by enabling the
`fetch.fsckObjects` config.

There are basically two checks:

    1. We verify that size of the gitattributes file is smaller than
       100MB.

    2. We verify that the maximum line length does not exceed 2048
       bytes.

With the preceding commits, both of these conditions would cause us to
either ignore the complete gitattributes file or blob in the first case,
or the specific line in the second case. Now with these consistency
checks added, we also grow the ability to stop distributing such files
in the first place when `receive.fsckObjects` is enabled.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 17:07:04 +09:00
f8587c31c9 fsck: move checks for gitattributes
Move the checks for gitattributes so that they can be extended more
readily.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 17:05:00 +09:00
a59a8c687f fsck: pull out function to check a set of blobs
In `fsck_finish()` we check all blobs for consistency that we have found
during the tree walk, but that haven't yet been checked. This is only
required for gitmodules right now, but will also be required for a new
check for gitattributes.

Pull out a function `fsck_blobs()` that allows the caller to check a set
of blobs for consistency.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 17:05:00 +09:00
bb3a9265e5 fsck: refactor fsck_blob() to allow for more checks
In general, we don't need to validate blob contents as they are opaque
blobs about whose content Git doesn't need to care about. There are some
exceptions though when blobs are linked into trees so that they would be
interpreted by Git. We only have a single such check right now though,
which is the one for gitmodules that has been added in the context of
CVE-2018-11235.

Now we have found another vulnerability with gitattributes that can lead
to out-of-bounds writes and reads. So let's refactor `fsck_blob()` so
that it is more extensible and can check different types of blobs.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 17:05:00 +09:00
e0bfc0b3b9 Merge branch 'ps/attr-limits' into maint-2.32 2022-12-09 17:03:49 +09:00
6662a836eb Merge branch 'ps/attr-limits' into maint-2.30 2022-12-09 16:05:52 +09:00
3305300f4c Merge branch 'ps/format-padding-fix' into maint-2.30 2022-12-09 16:02:39 +09:00
304a50adff pretty: restrict input lengths for padding and wrapping formats
Both the padding and wrapping formatting directives allow the caller to
specify an integer that ultimately leads to us adding this many chars to
the result buffer. As a consequence, it is trivial to e.g. allocate 2GB
of RAM via a single formatting directive and cause resource exhaustion
on the machine executing this logic. Furthermore, it is debatable
whether there are any sane usecases that require the user to pad data to
2GB boundaries or to indent wrapped data by 2GB.

Restrict the input sizes to 16 kilobytes at a maximum to limit the
amount of bytes that can be requested by the user. This is not meant
as a fix because there are ways to trivially amplify the amount of
data we generate via formatting directives; the real protection is
achieved by the changes in previous steps to catch and avoid integer
wraparound that causes us to under-allocate and access beyond the
end of allocated memory reagions. But having such a limit
significantly helps fuzzing the pretty format, because the fuzzer is
otherwise quite fast to run out-of-memory as it discovers these
formatters.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
f930a23943 utf8: refactor strbuf_utf8_replace to not rely on preallocated buffer
In `strbuf_utf8_replace`, we preallocate the destination buffer and then
use `memcpy` to copy bytes into it at computed offsets. This feels
rather fragile and is hard to understand at times. Refactor the code to
instead use `strbuf_add` and `strbuf_addstr` so that we can be sure that
there is no possibility to perform an out-of-bounds write.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
81c2d4c3a5 utf8: fix checking for glyph width in strbuf_utf8_replace()
In `strbuf_utf8_replace()`, we call `utf8_width()` to compute the width
of the current glyph. If the glyph is a control character though it can
be that `utf8_width()` returns `-1`, but because we assign this value to
a `size_t` the conversion will cause us to underflow. This bug can
easily be triggered with the following command:

    $ git log --pretty='format:xxx%<|(1,trunc)%x10'

>From all I can see though this seems to be a benign underflow that has
no security-related consequences.

Fix the bug by using an `int` instead. When we see a control character,
we now copy it into the target buffer but don't advance the current
width of the string.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
937b71cc8b utf8: fix overflow when returning string width
The return type of both `utf8_strwidth()` and `utf8_strnwidth()` is
`int`, but we operate on string lengths which are typically of type
`size_t`. This means that when the string is longer than `INT_MAX`, we
will overflow and thus return a negative result.

This can lead to an out-of-bounds write with `--pretty=format:%<1)%B`
and a commit message that is 2^31+1 bytes long:

    =================================================================
    ==26009==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000001168 at pc 0x7f95c4e5f427 bp 0x7ffd8541c900 sp 0x7ffd8541c0a8
    WRITE of size 2147483649 at 0x603000001168 thread T0
        #0 0x7f95c4e5f426 in __interceptor_memcpy /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
        #1 0x5612bbb1068c in format_and_pad_commit pretty.c:1763
        #2 0x5612bbb1087a in format_commit_item pretty.c:1801
        #3 0x5612bbc33bab in strbuf_expand strbuf.c:429
        #4 0x5612bbb110e7 in repo_format_commit_message pretty.c:1869
        #5 0x5612bbb12d96 in pretty_print_commit pretty.c:2161
        #6 0x5612bba0a4d5 in show_log log-tree.c:781
        #7 0x5612bba0d6c7 in log_tree_commit log-tree.c:1117
        #8 0x5612bb691ed5 in cmd_log_walk_no_free builtin/log.c:508
        #9 0x5612bb69235b in cmd_log_walk builtin/log.c:549
        #10 0x5612bb6951a2 in cmd_log builtin/log.c:883
        #11 0x5612bb56c993 in run_builtin git.c:466
        #12 0x5612bb56d397 in handle_builtin git.c:721
        #13 0x5612bb56db07 in run_argv git.c:788
        #14 0x5612bb56e8a7 in cmd_main git.c:923
        #15 0x5612bb803682 in main common-main.c:57
        #16 0x7f95c4c3c28f  (/usr/lib/libc.so.6+0x2328f)
        #17 0x7f95c4c3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #18 0x5612bb5680e4 in _start ../sysdeps/x86_64/start.S:115

    0x603000001168 is located 0 bytes to the right of 24-byte region [0x603000001150,0x603000001168)
    allocated by thread T0 here:
        #0 0x7f95c4ebe7ea in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:85
        #1 0x5612bbcdd556 in xrealloc wrapper.c:136
        #2 0x5612bbc310a3 in strbuf_grow strbuf.c:99
        #3 0x5612bbc32acd in strbuf_add strbuf.c:298
        #4 0x5612bbc33aec in strbuf_expand strbuf.c:418
        #5 0x5612bbb110e7 in repo_format_commit_message pretty.c:1869
        #6 0x5612bbb12d96 in pretty_print_commit pretty.c:2161
        #7 0x5612bba0a4d5 in show_log log-tree.c:781
        #8 0x5612bba0d6c7 in log_tree_commit log-tree.c:1117
        #9 0x5612bb691ed5 in cmd_log_walk_no_free builtin/log.c:508
        #10 0x5612bb69235b in cmd_log_walk builtin/log.c:549
        #11 0x5612bb6951a2 in cmd_log builtin/log.c:883
        #12 0x5612bb56c993 in run_builtin git.c:466
        #13 0x5612bb56d397 in handle_builtin git.c:721
        #14 0x5612bb56db07 in run_argv git.c:788
        #15 0x5612bb56e8a7 in cmd_main git.c:923
        #16 0x5612bb803682 in main common-main.c:57
        #17 0x7f95c4c3c28f  (/usr/lib/libc.so.6+0x2328f)

    SUMMARY: AddressSanitizer: heap-buffer-overflow /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827 in __interceptor_memcpy
    Shadow bytes around the buggy address:
      0x0c067fff81d0: fd fd fd fa fa fa fd fd fd fa fa fa fd fd fd fa
      0x0c067fff81e0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
      0x0c067fff81f0: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
      0x0c067fff8200: fd fd fd fa fa fa fd fd fd fd fa fa 00 00 00 fa
      0x0c067fff8210: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
    =>0x0c067fff8220: fd fa fa fa fd fd fd fa fa fa 00 00 00[fa]fa fa
      0x0c067fff8230: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff8240: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff8250: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff8260: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff8270: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==26009==ABORTING

Now the proper fix for this would be to convert both functions to return
an `size_t` instead of an `int`. But given that this commit may be part
of a security release, let's instead do the minimal viable fix and die
in case we see an overflow.

Add a test that would have previously caused us to crash.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
17d23e8a38 utf8: fix returning negative string width
The `utf8_strnwidth()` function calls `utf8_width()` in a loop and adds
its returned width to the end result. `utf8_width()` can return `-1`
though in case it reads a control character, which means that the
computed string width is going to be wrong. In the worst case where
there are more control characters than non-control characters, we may
even return a negative string width.

Fix this bug by treating control characters as having zero width.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
522cc87fdc utf8: fix truncated string lengths in utf8_strnwidth()
The `utf8_strnwidth()` function accepts an optional string length as
input parameter. This parameter can either be set to `-1`, in which case
we call `strlen()` on the input. Or it can be set to a positive integer
that indicates a precomputed length, which callers typically compute by
calling `strlen()` at some point themselves.

The input parameter is an `int` though, whereas `strlen()` returns a
`size_t`. This can lead to implementation-defined behaviour though when
the `size_t` cannot be represented by the `int`. In the general case
though this leads to wrap-around and thus to negative string sizes,
which is sure enough to not lead to well-defined behaviour.

Fix this by accepting a `size_t` instead of an `int` as string length.
While this takes away the ability of callers to simply pass in `-1` as
string length, it really is trivial enough to convert them to instead
pass in `strlen()` instead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
48050c42c7 pretty: fix integer overflow in wrapping format
The `%w(width,indent1,indent2)` formatting directive can be used to
rewrap text to a specific width and is designed after git-shortlog(1)'s
`-w` parameter. While the three parameters are all stored as `size_t`
internally, `strbuf_add_wrapped_text()` accepts integers as input. As a
result, the casted integers may overflow. As these now-negative integers
are later on passed to `strbuf_addchars()`, we will ultimately run into
implementation-defined behaviour due to casting a negative number back
to `size_t` again. On my platform, this results in trying to allocate
9000 petabyte of memory.

Fix this overflow by using `cast_size_t_to_int()` so that we reject
inputs that cannot be represented as an integer.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
1de69c0cdd pretty: fix adding linefeed when placeholder is not expanded
When a formatting directive has a `+` or ` ` after the `%`, then we add
either a line feed or space if the placeholder expands to a non-empty
string. In specific cases though this logic doesn't work as expected,
and we try to add the character even in the case where the formatting
directive is empty.

One such pattern is `%w(1)%+d%+w(2)`. `%+d` expands to reference names
pointing to a certain commit, like in `git log --decorate`. For a tagged
commit this would for example expand to `\n (tag: v1.0.0)`, which has a
leading newline due to the `+` modifier and a space added by `%d`. Now
the second wrapping directive will cause us to rewrap the text to
`\n(tag:\nv1.0.0)`, which is one byte shorter due to the missing leading
space. The code that handles the `+` magic now notices that the length
has changed and will thus try to insert a leading line feed at the
original posititon. But as the string was shortened, the original
position is past the buffer's boundary and thus we die with an error.

Now there are two issues here:

    1. We check whether the buffer length has changed, not whether it
       has been extended. This causes us to try and add the character
       past the string boundary.

    2. The current logic does not make any sense whatsoever. When the
       string got expanded due to the rewrap, putting the separator into
       the original position is likely to put it somewhere into the
       middle of the rewrapped contents.

It is debatable whether `%+w()` makes any sense in the first place.
Strictly speaking, the placeholder never expands to a non-empty string,
and consequentially we shouldn't ever accept this combination. We thus
fix the bug by simply refusing `%+w()`.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
f6e0b9f389 pretty: fix out-of-bounds read when parsing invalid padding format
An out-of-bounds read can be triggered when parsing an incomplete
padding format string passed via `--pretty=format` or in Git archives
when files are marked with the `export-subst` gitattribute.

This bug exists since we have introduced support for truncating output
via the `trunc` keyword a7f01c6b4d (pretty: support truncating in %>, %<
and %><, 2013-04-19). Before this commit, we used to find the end of the
formatting string by using strchr(3P). This function returns a `NULL`
pointer in case the character in question wasn't found. The subsequent
check whether any character was found thus simply checked the returned
pointer. After the commit we switched to strcspn(3P) though, which only
returns the offset to the first found character or to the trailing NUL
byte. As the end pointer is now computed by adding the offset to the
start pointer it won't be `NULL` anymore, and as a consequence the check
doesn't do anything anymore.

The out-of-bounds data that is being read can in fact end up in the
formatted string. As a consequence, it is possible to leak memory
contents either by calling git-log(1) or via git-archive(1) when any of
the archived files is marked with the `export-subst` gitattribute.

    ==10888==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000398 at pc 0x7f0356047cb2 bp 0x7fff3ffb95d0 sp 0x7fff3ffb8d78
    READ of size 1 at 0x602000000398 thread T0
        #0 0x7f0356047cb1 in __interceptor_strchrnul /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:725
        #1 0x563b7cec9a43 in strbuf_expand strbuf.c:417
        #2 0x563b7cda7060 in repo_format_commit_message pretty.c:1869
        #3 0x563b7cda8d0f in pretty_print_commit pretty.c:2161
        #4 0x563b7cca04c8 in show_log log-tree.c:781
        #5 0x563b7cca36ba in log_tree_commit log-tree.c:1117
        #6 0x563b7c927ed5 in cmd_log_walk_no_free builtin/log.c:508
        #7 0x563b7c92835b in cmd_log_walk builtin/log.c:549
        #8 0x563b7c92b1a2 in cmd_log builtin/log.c:883
        #9 0x563b7c802993 in run_builtin git.c:466
        #10 0x563b7c803397 in handle_builtin git.c:721
        #11 0x563b7c803b07 in run_argv git.c:788
        #12 0x563b7c8048a7 in cmd_main git.c:923
        #13 0x563b7ca99682 in main common-main.c:57
        #14 0x7f0355e3c28f  (/usr/lib/libc.so.6+0x2328f)
        #15 0x7f0355e3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #16 0x563b7c7fe0e4 in _start ../sysdeps/x86_64/start.S:115

    0x602000000398 is located 0 bytes to the right of 8-byte region [0x602000000390,0x602000000398)
    allocated by thread T0 here:
        #0 0x7f0356072faa in __interceptor_strdup /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:439
        #1 0x563b7cf7317c in xstrdup wrapper.c:39
        #2 0x563b7cd9a06a in save_user_format pretty.c:40
        #3 0x563b7cd9b3e5 in get_commit_format pretty.c:173
        #4 0x563b7ce54ea0 in handle_revision_opt revision.c:2456
        #5 0x563b7ce597c9 in setup_revisions revision.c:2850
        #6 0x563b7c9269e0 in cmd_log_init_finish builtin/log.c:269
        #7 0x563b7c927362 in cmd_log_init builtin/log.c:348
        #8 0x563b7c92b193 in cmd_log builtin/log.c:882
        #9 0x563b7c802993 in run_builtin git.c:466
        #10 0x563b7c803397 in handle_builtin git.c:721
        #11 0x563b7c803b07 in run_argv git.c:788
        #12 0x563b7c8048a7 in cmd_main git.c:923
        #13 0x563b7ca99682 in main common-main.c:57
        #14 0x7f0355e3c28f  (/usr/lib/libc.so.6+0x2328f)
        #15 0x7f0355e3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #16 0x563b7c7fe0e4 in _start ../sysdeps/x86_64/start.S:115

    SUMMARY: AddressSanitizer: heap-buffer-overflow /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:725 in __interceptor_strchrnul
    Shadow bytes around the buggy address:
      0x0c047fff8020: fa fa fd fd fa fa 00 06 fa fa 05 fa fa fa fd fd
      0x0c047fff8030: fa fa 00 02 fa fa 06 fa fa fa 05 fa fa fa fd fd
      0x0c047fff8040: fa fa 00 07 fa fa 03 fa fa fa fd fd fa fa 00 00
      0x0c047fff8050: fa fa 00 01 fa fa fd fd fa fa 00 00 fa fa 00 01
      0x0c047fff8060: fa fa 00 06 fa fa 00 06 fa fa 05 fa fa fa 05 fa
    =>0x0c047fff8070: fa fa 00[fa]fa fa fd fa fa fa fd fd fa fa fd fd
      0x0c047fff8080: fa fa fd fd fa fa 00 00 fa fa 00 fa fa fa fd fa
      0x0c047fff8090: fa fa fd fd fa fa 00 00 fa fa fa fa fa fa fa fa
      0x0c047fff80a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff80b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff80c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==10888==ABORTING

Fix this bug by checking whether `end` points at the trailing NUL byte.
Add a test which catches this out-of-bounds read and which demonstrates
that we used to write out-of-bounds data into the formatted message.

Reported-by: Markus Vervier <markus.vervier@x41-dsec.de>
Original-patch-by: Markus Vervier <markus.vervier@x41-dsec.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
b49f309aa1 pretty: fix out-of-bounds read when left-flushing with stealing
With the `%>>(<N>)` pretty formatter, you can ask git-log(1) et al to
steal spaces. To do so we need to look ahead of the next token to see
whether there are spaces there. This loop takes into account ANSI
sequences that end with an `m`, and if it finds any it will skip them
until it finds the first space. While doing so it does not take into
account the buffer's limits though and easily does an out-of-bounds
read.

Add a test that hits this behaviour. While we don't have an easy way to
verify this, the test causes the following failure when run with
`SANITIZE=address`:

    ==37941==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x603000000baf at pc 0x55ba6f88e0d0 bp 0x7ffc84c50d20 sp 0x7ffc84c50d10
    READ of size 1 at 0x603000000baf thread T0
        #0 0x55ba6f88e0cf in format_and_pad_commit pretty.c:1712
        #1 0x55ba6f88e7b4 in format_commit_item pretty.c:1801
        #2 0x55ba6f9b1ae4 in strbuf_expand strbuf.c:429
        #3 0x55ba6f88f020 in repo_format_commit_message pretty.c:1869
        #4 0x55ba6f890ccf in pretty_print_commit pretty.c:2161
        #5 0x55ba6f7884c8 in show_log log-tree.c:781
        #6 0x55ba6f78b6ba in log_tree_commit log-tree.c:1117
        #7 0x55ba6f40fed5 in cmd_log_walk_no_free builtin/log.c:508
        #8 0x55ba6f41035b in cmd_log_walk builtin/log.c:549
        #9 0x55ba6f4131a2 in cmd_log builtin/log.c:883
        #10 0x55ba6f2ea993 in run_builtin git.c:466
        #11 0x55ba6f2eb397 in handle_builtin git.c:721
        #12 0x55ba6f2ebb07 in run_argv git.c:788
        #13 0x55ba6f2ec8a7 in cmd_main git.c:923
        #14 0x55ba6f581682 in main common-main.c:57
        #15 0x7f2d08c3c28f  (/usr/lib/libc.so.6+0x2328f)
        #16 0x7f2d08c3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #17 0x55ba6f2e60e4 in _start ../sysdeps/x86_64/start.S:115

    0x603000000baf is located 1 bytes to the left of 24-byte region [0x603000000bb0,0x603000000bc8)
    allocated by thread T0 here:
        #0 0x7f2d08ebe7ea in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:85
        #1 0x55ba6fa5b494 in xrealloc wrapper.c:136
        #2 0x55ba6f9aefdc in strbuf_grow strbuf.c:99
        #3 0x55ba6f9b0a06 in strbuf_add strbuf.c:298
        #4 0x55ba6f9b1a25 in strbuf_expand strbuf.c:418
        #5 0x55ba6f88f020 in repo_format_commit_message pretty.c:1869
        #6 0x55ba6f890ccf in pretty_print_commit pretty.c:2161
        #7 0x55ba6f7884c8 in show_log log-tree.c:781
        #8 0x55ba6f78b6ba in log_tree_commit log-tree.c:1117
        #9 0x55ba6f40fed5 in cmd_log_walk_no_free builtin/log.c:508
        #10 0x55ba6f41035b in cmd_log_walk builtin/log.c:549
        #11 0x55ba6f4131a2 in cmd_log builtin/log.c:883
        #12 0x55ba6f2ea993 in run_builtin git.c:466
        #13 0x55ba6f2eb397 in handle_builtin git.c:721
        #14 0x55ba6f2ebb07 in run_argv git.c:788
        #15 0x55ba6f2ec8a7 in cmd_main git.c:923
        #16 0x55ba6f581682 in main common-main.c:57
        #17 0x7f2d08c3c28f  (/usr/lib/libc.so.6+0x2328f)
        #18 0x7f2d08c3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #19 0x55ba6f2e60e4 in _start ../sysdeps/x86_64/start.S:115

    SUMMARY: AddressSanitizer: heap-buffer-overflow pretty.c:1712 in format_and_pad_commit
    Shadow bytes around the buggy address:
      0x0c067fff8120: fa fa fd fd fd fa fa fa fd fd fd fa fa fa fd fd
      0x0c067fff8130: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa
      0x0c067fff8140: fd fd fd fa fa fa fd fd fd fa fa fa fd fd fd fa
      0x0c067fff8150: fa fa fd fd fd fd fa fa 00 00 00 fa fa fa fd fd
      0x0c067fff8160: fd fa fa fa fd fd fd fa fa fa fd fd fd fa fa fa
    =>0x0c067fff8170: fd fd fd fa fa[fa]00 00 00 fa fa fa 00 00 00 fa
      0x0c067fff8180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff8190: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff81a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff81b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c067fff81c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb

Luckily enough, this would only cause us to copy the out-of-bounds data
into the formatted commit in case we really had an ANSI sequence
preceding our buffer. So this bug likely has no security consequences.

Fix it regardless by not traversing past the buffer's start.

Reported-by: Patrick Steinhardt <ps@pks.im>
Reported-by: Eric Sesterhenn <eric.sesterhenn@x41-dsec.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
81dc898df9 pretty: fix out-of-bounds write caused by integer overflow
When using a padding specifier in the pretty format passed to git-log(1)
we need to calculate the string length in several places. These string
lengths are stored in `int`s though, which means that these can easily
overflow when the input lengths exceeds 2GB. This can ultimately lead to
an out-of-bounds write when these are used in a call to memcpy(3P):

        ==8340==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f1ec62f97fe at pc 0x7f2127e5f427 bp 0x7ffd3bd63de0 sp 0x7ffd3bd63588
    WRITE of size 1 at 0x7f1ec62f97fe thread T0
        #0 0x7f2127e5f426 in __interceptor_memcpy /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
        #1 0x5628e96aa605 in format_and_pad_commit pretty.c:1762
        #2 0x5628e96aa7f4 in format_commit_item pretty.c:1801
        #3 0x5628e97cdb24 in strbuf_expand strbuf.c:429
        #4 0x5628e96ab060 in repo_format_commit_message pretty.c:1869
        #5 0x5628e96acd0f in pretty_print_commit pretty.c:2161
        #6 0x5628e95a44c8 in show_log log-tree.c:781
        #7 0x5628e95a76ba in log_tree_commit log-tree.c:1117
        #8 0x5628e922bed5 in cmd_log_walk_no_free builtin/log.c:508
        #9 0x5628e922c35b in cmd_log_walk builtin/log.c:549
        #10 0x5628e922f1a2 in cmd_log builtin/log.c:883
        #11 0x5628e9106993 in run_builtin git.c:466
        #12 0x5628e9107397 in handle_builtin git.c:721
        #13 0x5628e9107b07 in run_argv git.c:788
        #14 0x5628e91088a7 in cmd_main git.c:923
        #15 0x5628e939d682 in main common-main.c:57
        #16 0x7f2127c3c28f  (/usr/lib/libc.so.6+0x2328f)
        #17 0x7f2127c3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #18 0x5628e91020e4 in _start ../sysdeps/x86_64/start.S:115

    0x7f1ec62f97fe is located 2 bytes to the left of 4831838265-byte region [0x7f1ec62f9800,0x7f1fe62f9839)
    allocated by thread T0 here:
        #0 0x7f2127ebe7ea in __interceptor_realloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:85
        #1 0x5628e98774d4 in xrealloc wrapper.c:136
        #2 0x5628e97cb01c in strbuf_grow strbuf.c:99
        #3 0x5628e97ccd42 in strbuf_addchars strbuf.c:327
        #4 0x5628e96aa55c in format_and_pad_commit pretty.c:1761
        #5 0x5628e96aa7f4 in format_commit_item pretty.c:1801
        #6 0x5628e97cdb24 in strbuf_expand strbuf.c:429
        #7 0x5628e96ab060 in repo_format_commit_message pretty.c:1869
        #8 0x5628e96acd0f in pretty_print_commit pretty.c:2161
        #9 0x5628e95a44c8 in show_log log-tree.c:781
        #10 0x5628e95a76ba in log_tree_commit log-tree.c:1117
        #11 0x5628e922bed5 in cmd_log_walk_no_free builtin/log.c:508
        #12 0x5628e922c35b in cmd_log_walk builtin/log.c:549
        #13 0x5628e922f1a2 in cmd_log builtin/log.c:883
        #14 0x5628e9106993 in run_builtin git.c:466
        #15 0x5628e9107397 in handle_builtin git.c:721
        #16 0x5628e9107b07 in run_argv git.c:788
        #17 0x5628e91088a7 in cmd_main git.c:923
        #18 0x5628e939d682 in main common-main.c:57
        #19 0x7f2127c3c28f  (/usr/lib/libc.so.6+0x2328f)
        #20 0x7f2127c3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #21 0x5628e91020e4 in _start ../sysdeps/x86_64/start.S:115

    SUMMARY: AddressSanitizer: heap-buffer-overflow /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827 in __interceptor_memcpy
    Shadow bytes around the buggy address:
      0x0fe458c572a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0fe458c572b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0fe458c572c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0fe458c572d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0fe458c572e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    =>0x0fe458c572f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
      0x0fe458c57300: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe458c57310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe458c57320: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe458c57330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
      0x0fe458c57340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
    ==8340==ABORTING

The pretty format can also be used in `git archive` operations via the
`export-subst` attribute. So this is what in our opinion makes this a
critical issue in the context of Git forges which allow to download an
archive of user supplied Git repositories.

Fix this vulnerability by using `size_t` instead of `int` to track the
string lengths. Add tests which detect this vulnerability when Git is
compiled with the address sanitizer.

Reported-by: Joern Schneeweisz <jschneeweisz@gitlab.com>
Original-patch-by: Joern Schneeweisz <jschneeweisz@gitlab.com>
Modified-by: Taylor  Blau <me@ttalorr.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:21 +09:00
a244dc5b0a test-lib: add prerequisite for 64-bit platforms
Allow tests that assume a 64-bit `size_t` to be skipped in 32-bit
platforms and regardless of the size of `long`.

This imitates the `LONG_IS_64BIT` prerequisite.

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-09 14:26:04 +09:00
f115c96e7a CI: migrate away from deprecated "set-output" syntax
As noted in [1] and the warnings the CI itself is spewing echoing
outputs to stdout is deprecated, and they should be written to
"$GITHUB_OUTPUT" instead.

1. https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:47:22 +09:00
1f398446c3 ci: avoid using deprecated {up,down}load-artifacts Action
The deprecated versions of these Actions still use node.js 12 whereas
workflows will need to use node.js 16 to avoid problems going forward.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:15:23 +09:00
d8b21a0fe2 CI: don't explicitly pick "bash" shell outside of Windows, fix regression
When the "js/ci-github-workflow-markup" topic was originally merged in
[1] it included a change to get rid of the "ci/print-test-failures.sh"
step[2]. This was then brought back in [3] as part of a fix-up patches
on top[4].

The problem was that [3] was not a revert of the relevant parts of
[2], but rather copy/pasted the "ci/print-test-failures.sh" step that
was present for the Windows job to all "ci/print-test-failures.sh"
steps. The Windows steps specified "shell: bash", but the non-Windows
ones did not.

This broke the "ci/print/test-failures.sh" step for the "linux-musl"
job, where we don't have a "bash" shell, just a "/bin/sh" (a
"dash"). This breakage was reported at the time[5], but hadn't been
fixed.

It would be sufficient to change this only for "linux-musl", but let's
change this for both "regular" and "dockerized" to omit the "shell"
line entirely, as we did before [2].

Let's also change undo the "name" change that [3] made while
copy/pasting the "print test failures" step for the Windows job. These
steps are now the same as they were before [2], except that the "if"
includes the "env.FAILED_TEST_ARTIFACTS" test.

1. fc5a070f59 (Merge branch 'js/ci-github-workflow-markup', 2022-06-07)
2. 08dccc8fc1 (ci: make it easier to find failed tests' logs in the
   GitHub workflow, 2022-05-21)
3. 5aeb145780 (ci(github): bring back the 'print test failures' step,
   2022-06-08)
4. d0d96b8280 (Merge branch 'js/ci-github-workflow-markup', 2022-06-17)
5. https://lore.kernel.org/git/220725.86sfmpneqp.gmgdl@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-08 08:06:00 +09:00
d11192255d CI: upgrade to macos-12, and pin OSX version
Per [1] and the warnings our CI is emitting GitHub is phasing in
"macos-12" as their "macos-latest".

As with [2], let's pin our image to a specific version so that we're
not having it swept from under us, and our upgrade cycle can be more
predictable than whenever GitHub changes their images.

1. https://github.com/actions/runner-images/issues/6384
2. 0178420b9c (github-actions: run gcc-8 on ubuntu-20.04 image,
   2022-11-25)

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-07 13:36:22 +09:00
6cf4d908a9 ci(main): upgrade actions/checkout to v3
To be up to date with actions/checkout opens the door to use the latest
features if necessary and get the latest security patches.

This also avoids a couple of deprecation warnings in the CI runs.

Note: The `actions/checkout` Action has been known to be broken in i686
containers as of v2, therefore we keep forcing it to v1 there. See
actions/runner#2115 for more details.

Signed-off-by: Oscar Dominguez <dominguez.celada@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-06 08:22:15 +09:00
3c50032ff5 attr: ignore overly large gitattributes files
Similar as with the preceding commit, start ignoring gitattributes files
that are overly large to protect us against out-of-bounds reads and
writes caused by integer overflows. Unfortunately, we cannot just define
"overly large" in terms of any preexisting limits in the codebase.

Instead, we choose a very conservative limit of 100MB. This is plenty of
room for specifying gitattributes, and incidentally it is also the limit
for blob sizes for GitHub. While we don't want GitHub to dictate limits
here, it is still sensible to use this fact for an informed decision
given that it is hosting a huge set of repositories. Furthermore, over
at GitLab we scanned a subset of repositories for their root-level
attribute files. We found that 80% of them have a gitattributes file
smaller than 100kB, 99.99% have one smaller than 1MB, and only a single
repository had one that was almost 3MB in size. So enforcing a limit of
100MB seems to give us ample of headroom.

With this limit in place we can be reasonably sure that there is no easy
way to exploit the gitattributes file via integer overflows anymore.
Furthermore, it protects us against resource exhaustion caused by
allocating the in-memory data structures required to represent the
parsed attributes.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:50:03 +09:00
dfa6b32b5e attr: ignore attribute lines exceeding 2048 bytes
There are two different code paths to read gitattributes: once via a
file, and once via the index. These two paths used to behave differently
because when reading attributes from a file, we used fgets(3P) with a
buffer size of 2kB. Consequentially, we silently truncate line lengths
when lines are longer than that and will then parse the remainder of the
line as a new pattern. It goes without saying that this is entirely
unexpected, but it's even worse that the behaviour depends on how the
gitattributes are parsed.

While this is simply wrong, the silent truncation saves us with the
recently discovered vulnerabilities that can cause out-of-bound writes
or reads with unreasonably long lines due to integer overflows. As the
common path is to read gitattributes via the worktree file instead of
via the index, we can assume that any gitattributes file that had lines
longer than that is already broken anyway. So instead of lifting the
limit here, we can double down on it to fix the vulnerabilities.

Introduce an explicit line length limit of 2kB that is shared across all
paths that read attributes and ignore any line that hits this limit
while printing a warning.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:33:07 +09:00
d74b1fd54f attr: fix silently splitting up lines longer than 2048 bytes
When reading attributes from a file we use fgets(3P) with a buffer size
of 2048 bytes. This means that as soon as a line exceeds the buffer size
we split it up into multiple parts and parse each of them as a separate
pattern line. This is of course not what the user intended, and even
worse the behaviour is inconsistent with how we read attributes from the
index.

Fix this bug by converting the code to use `strbuf_getline()` instead.
This will indeed read in the whole line, which may theoretically lead to
an out-of-memory situation when the gitattributes file is huge. We're
about to reject any gitattributes files larger than 100MB in the next
commit though, which makes this less of a concern.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:29:30 +09:00
a60a66e409 attr: harden allocation against integer overflows
When parsing an attributes line, we need to allocate an array that holds
all attributes specified for the given file pattern. The calculation to
determine the number of bytes that need to be allocated was prone to an
overflow though when there was an unreasonable amount of attributes.

Harden the allocation by instead using the `st_` helper functions that
cause us to die when we hit an integer overflow.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
e1e12e97ac attr: fix integer overflow with more than INT_MAX macros
Attributes have a field that tracks the position in the `all_attrs`
array they're stored inside. This field gets set via `hashmap_get_size`
when adding the attribute to the global map of attributes. But while the
field is of type `int`, the value returned by `hashmap_get_size` is an
`unsigned int`. It can thus happen that the value overflows, where we
would now dereference teh `all_attrs` array at an out-of-bounds value.

We do have a sanity check for this overflow via an assert that verifies
the index matches the new hashmap's size. But asserts are not a proper
mechanism to detect against any such overflows as they may not in fact
be compiled into production code.

Fix this by using an `unsigned int` to track the index and convert the
assert to a call `die()`.

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
447ac906e1 attr: fix out-of-bounds read with unreasonable amount of patterns
The `struct attr_stack` tracks the stack of all patterns together with
their attributes. When parsing a gitattributes file that has more than
2^31 such patterns though we may trigger multiple out-of-bounds reads on
64 bit platforms. This is because while the `num_matches` variable is an
unsigned integer, we always use a signed integer to iterate over them.

I have not been able to reproduce this issue due to memory constraints
on my systems. But despite the out-of-bounds reads, the worst thing that
can seemingly happen is to call free(3P) with a garbage pointer when
calling `attr_stack_free()`.

Fix this bug by using unsigned integers to iterate over the array. While
this makes the iteration somewhat awkward when iterating in reverse, it
is at least better than knowingly running into an out-of-bounds read.
While at it, convert the call to `ALLOC_GROW` to use `ALLOC_GROW_BY`
instead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
34ace8bad0 attr: fix out-of-bounds write when parsing huge number of attributes
It is possible to trigger an integer overflow when parsing attribute
names when there are more than 2^31 of them for a single pattern. This
can either lead to us dying due to trying to request too many bytes:

     blob=$(perl -e 'print "f" . " a=" x 2147483649' | git hash-object -w --stdin)
     git update-index --add --cacheinfo 100644,$blob,.gitattributes
     git attr-check --all file

    =================================================================
    ==1022==ERROR: AddressSanitizer: requested allocation size 0xfffffff800000032 (0xfffffff800001038 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
        #0 0x7fd3efabf411 in __interceptor_calloc /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77
        #1 0x5563a0a1e3d3 in xcalloc wrapper.c:150
        #2 0x5563a058d005 in parse_attr_line attr.c:384
        #3 0x5563a058e661 in handle_attr_line attr.c:660
        #4 0x5563a058eddb in read_attr_from_index attr.c:769
        #5 0x5563a058ef12 in read_attr attr.c:797
        #6 0x5563a058f24c in bootstrap_attr_stack attr.c:867
        #7 0x5563a058f4a3 in prepare_attr_stack attr.c:902
        #8 0x5563a05905da in collect_some_attrs attr.c:1097
        #9 0x5563a059093d in git_all_attrs attr.c:1128
        #10 0x5563a02f636e in check_attr builtin/check-attr.c:67
        #11 0x5563a02f6c12 in cmd_check_attr builtin/check-attr.c:183
        #12 0x5563a02aa993 in run_builtin git.c:466
        #13 0x5563a02ab397 in handle_builtin git.c:721
        #14 0x5563a02abb2b in run_argv git.c:788
        #15 0x5563a02ac991 in cmd_main git.c:926
        #16 0x5563a05432bd in main common-main.c:57
        #17 0x7fd3ef82228f  (/usr/lib/libc.so.6+0x2328f)

    ==1022==HINT: if you don't care about these errors you may set allocator_may_return_null=1
    SUMMARY: AddressSanitizer: allocation-size-too-big /usr/src/debug/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77 in __interceptor_calloc
    ==1022==ABORTING

Or, much worse, it can lead to an out-of-bounds write because we
underallocate and then memcpy(3P) into an array:

    perl -e '
        print "A " . "\rh="x2000000000;
        print "\rh="x2000000000;
        print "\rh="x294967294 . "\n"
    ' >.gitattributes
    git add .gitattributes
    git commit -am "evil attributes"

    $ git clone --quiet /path/to/repo
    =================================================================
    ==15062==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000002550 at pc 0x5555559884d5 bp 0x7fffffffbc60 sp 0x7fffffffbc58
    WRITE of size 8 at 0x602000002550 thread T0
        #0 0x5555559884d4 in parse_attr_line attr.c:393
        #1 0x5555559884d4 in handle_attr_line attr.c:660
        #2 0x555555988902 in read_attr_from_index attr.c:784
        #3 0x555555988902 in read_attr_from_index attr.c:747
        #4 0x555555988a1d in read_attr attr.c:800
        #5 0x555555989b0c in bootstrap_attr_stack attr.c:882
        #6 0x555555989b0c in prepare_attr_stack attr.c:917
        #7 0x555555989b0c in collect_some_attrs attr.c:1112
        #8 0x55555598b141 in git_check_attr attr.c:1126
        #9 0x555555a13004 in convert_attrs convert.c:1311
        #10 0x555555a95e04 in checkout_entry_ca entry.c:553
        #11 0x555555d58bf6 in checkout_entry entry.h:42
        #12 0x555555d58bf6 in check_updates unpack-trees.c:480
        #13 0x555555d5eb55 in unpack_trees unpack-trees.c:2040
        #14 0x555555785ab7 in checkout builtin/clone.c:724
        #15 0x555555785ab7 in cmd_clone builtin/clone.c:1384
        #16 0x55555572443c in run_builtin git.c:466
        #17 0x55555572443c in handle_builtin git.c:721
        #18 0x555555727872 in run_argv git.c:788
        #19 0x555555727872 in cmd_main git.c:926
        #20 0x555555721fa0 in main common-main.c:57
        #21 0x7ffff73f1d09 in __libc_start_main ../csu/libc-start.c:308
        #22 0x555555723f39 in _start (git+0x1cff39)

    0x602000002552 is located 0 bytes to the right of 2-byte region [0x602000002550,0x602000002552) allocated by thread T0 here:
        #0 0x7ffff768c037 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
        #1 0x555555d7fff7 in xcalloc wrapper.c:150
        #2 0x55555598815f in parse_attr_line attr.c:384
        #3 0x55555598815f in handle_attr_line attr.c:660
        #4 0x555555988902 in read_attr_from_index attr.c:784
        #5 0x555555988902 in read_attr_from_index attr.c:747
        #6 0x555555988a1d in read_attr attr.c:800
        #7 0x555555989b0c in bootstrap_attr_stack attr.c:882
        #8 0x555555989b0c in prepare_attr_stack attr.c:917
        #9 0x555555989b0c in collect_some_attrs attr.c:1112
        #10 0x55555598b141 in git_check_attr attr.c:1126
        #11 0x555555a13004 in convert_attrs convert.c:1311
        #12 0x555555a95e04 in checkout_entry_ca entry.c:553
        #13 0x555555d58bf6 in checkout_entry entry.h:42
        #14 0x555555d58bf6 in check_updates unpack-trees.c:480
        #15 0x555555d5eb55 in unpack_trees unpack-trees.c:2040
        #16 0x555555785ab7 in checkout builtin/clone.c:724
        #17 0x555555785ab7 in cmd_clone builtin/clone.c:1384
        #18 0x55555572443c in run_builtin git.c:466
        #19 0x55555572443c in handle_builtin git.c:721
        #20 0x555555727872 in run_argv git.c:788
        #21 0x555555727872 in cmd_main git.c:926
        #22 0x555555721fa0 in main common-main.c:57
        #23 0x7ffff73f1d09 in __libc_start_main ../csu/libc-start.c:308

    SUMMARY: AddressSanitizer: heap-buffer-overflow attr.c:393 in parse_attr_line
    Shadow bytes around the buggy address:
      0x0c047fff8450: fa fa 00 02 fa fa 00 07 fa fa fd fd fa fa 00 00
      0x0c047fff8460: fa fa 02 fa fa fa fd fd fa fa 00 06 fa fa 05 fa
      0x0c047fff8470: fa fa fd fd fa fa 00 02 fa fa 06 fa fa fa 05 fa
      0x0c047fff8480: fa fa 07 fa fa fa fd fd fa fa 00 01 fa fa 00 02
      0x0c047fff8490: fa fa 00 03 fa fa 00 fa fa fa 00 01 fa fa 00 03
    =>0x0c047fff84a0: fa fa 00 01 fa fa 00 02 fa fa[02]fa fa fa fa fa
      0x0c047fff84b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff84c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff84d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff84e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c047fff84f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==15062==ABORTING

Fix this bug by using `size_t` instead to count the number of attributes
so that this value cannot reasonably overflow without running out of
memory before already.

Reported-by: Markus Vervier <markus.vervier@x41-dsec.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
2455720950 attr: fix integer overflow when parsing huge attribute names
It is possible to trigger an integer overflow when parsing attribute
names that are longer than 2^31 bytes because we assign the result of
strlen(3P) to an `int` instead of to a `size_t`. This can lead to an
abort in vsnprintf(3P) with the following reproducer:

    blob=$(perl -e 'print "A " . "B"x2147483648 . "\n"' | git hash-object -w --stdin)
    git update-index --add --cacheinfo 100644,$blob,.gitattributes
    git check-attr --all path

    BUG: strbuf.c:400: your vsnprintf is broken (returned -1)

But furthermore, assuming that the attribute name is even longer than
that, it can cause us to silently truncate the attribute and thus lead
to wrong results.

Fix this integer overflow by using a `size_t` instead. This fixes the
silent truncation of attribute names, but it only partially fixes the
BUG we hit: even though the initial BUG is fixed, we can still hit a BUG
when parsing invalid attribute lines via `report_invalid_attr()`.

This is due to an underlying design issue in vsnprintf(3P) which only
knows to return an `int`, and thus it may always overflow with large
inputs. This issue is benign though: the worst that can happen is that
the error message is misreported to be either truncated or too long, but
due to the buffer being NUL terminated we wouldn't ever do an
out-of-bounds read here.

Reported-by: Markus Vervier <markus.vervier@x41-dsec.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
8d0d48cf21 attr: fix out-of-bounds read with huge attribute names
There is an out-of-bounds read possible when parsing gitattributes that
have an attribute that is 2^31+1 bytes long. This is caused due to an
integer overflow when we assign the result of strlen(3P) to an `int`,
where we use the wrapped-around value in a subsequent call to
memcpy(3P). The following code reproduces the issue:

    blob=$(perl -e 'print "a" x 2147483649 . " attr"' | git hash-object -w --stdin)
    git update-index --add --cacheinfo 100644,$blob,.gitattributes
    git check-attr --all file

    AddressSanitizer:DEADLYSIGNAL
    =================================================================
    ==8451==ERROR: AddressSanitizer: SEGV on unknown address 0x7f93efa00800 (pc 0x7f94f1f8f082 bp 0x7ffddb59b3a0 sp 0x7ffddb59ab28 T0)
    ==8451==The signal is caused by a READ memory access.
        #0 0x7f94f1f8f082  (/usr/lib/libc.so.6+0x176082)
        #1 0x7f94f2047d9c in __interceptor_strspn /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:752
        #2 0x560e190f7f26 in parse_attr_line attr.c:375
        #3 0x560e190f9663 in handle_attr_line attr.c:660
        #4 0x560e190f9ddd in read_attr_from_index attr.c:769
        #5 0x560e190f9f14 in read_attr attr.c:797
        #6 0x560e190fa24e in bootstrap_attr_stack attr.c:867
        #7 0x560e190fa4a5 in prepare_attr_stack attr.c:902
        #8 0x560e190fb5dc in collect_some_attrs attr.c:1097
        #9 0x560e190fb93f in git_all_attrs attr.c:1128
        #10 0x560e18e6136e in check_attr builtin/check-attr.c:67
        #11 0x560e18e61c12 in cmd_check_attr builtin/check-attr.c:183
        #12 0x560e18e15993 in run_builtin git.c:466
        #13 0x560e18e16397 in handle_builtin git.c:721
        #14 0x560e18e16b2b in run_argv git.c:788
        #15 0x560e18e17991 in cmd_main git.c:926
        #16 0x560e190ae2bd in main common-main.c:57
        #17 0x7f94f1e3c28f  (/usr/lib/libc.so.6+0x2328f)
        #18 0x7f94f1e3c349 in __libc_start_main (/usr/lib/libc.so.6+0x23349)
        #19 0x560e18e110e4 in _start ../sysdeps/x86_64/start.S:115

    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV (/usr/lib/libc.so.6+0x176082)
    ==8451==ABORTING

Fix this bug by converting the variable to a `size_t` instead.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
eb22e7dfa2 attr: fix overflow when upserting attribute with overly long name
The function `git_attr_internal()` is called to upsert attributes into
the global map. And while all callers pass a `size_t`, the function
itself accepts an `int` as the attribute name's length. This can lead to
an integer overflow in case the attribute name is longer than `INT_MAX`.

Now this overflow seems harmless as the first thing we do is to call
`attr_name_valid()`, and that function only succeeds in case all chars
in the range of `namelen` match a certain small set of chars. We thus
can't do an out-of-bounds read as NUL is not part of that set and all
strings passed to this function are NUL-terminated. And furthermore, we
wouldn't ever read past the current attribute name anyway due to the
same reason. And if validation fails we will return early.

On the other hand it feels fragile to rely on this behaviour, even more
so given that we pass `namelen` to `FLEX_ALLOC_MEM()`. So let's instead
just do the correct thing here and accept a `size_t` as line length.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-12-05 15:14:16 +09:00
0d3507f3e7 ci: install python on ubuntu
Python is missing from the default ubuntu-22.04 runner image, which
prevents git-p4 from working. To install python on ubuntu, we need
to provide the correct package names:

 * On Ubuntu 18.04 (bionic), "/usr/bin/python2" is provided by the
   "python" package, and "/usr/bin/python3" is provided by the "python3"
   package.

 * On Ubuntu 20.04 (focal) and above, "/usr/bin/python2" is provided by
   the "python2" package which has a different name from bionic, and
   "/usr/bin/python3" is provided by "python3".

Since the "ubuntu-latest" runner image has a higher version, its
safe to use "python2" or "python3" package name.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27 09:33:43 +09:00
31a1952bbd ci: use the same version of p4 on both Linux and macOS
There would be a segmentation fault when running p4 v16.2 on ubuntu
22.04 which is the latest version of ubuntu runner image for github
actions.

By checking each version from [1], p4d version 21.1 and above can work
properly on ubuntu 22.04. But version 22.x will break some p4 test
cases. So p4 version 21.x is exactly the version we can use.

With this update, the versions of p4 for Linux and macOS happen to be
the same. So we can add the version number directly into the "P4WHENCE"
variable, and reuse it in p4 installation for macOS.

By removing the "LINUX_P4_VERSION" variable from "ci/lib.sh", the
comment left above has nothing to do with p4, but still applies to
git-lfs. Since we have a fixed version of git-lfs installed on Linux,
we may have a different version on macOS.

[1]: https://cdist2.perforce.com/perforce/

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27 09:32:56 +09:00
4137c84198 ci: remove the pipe after "p4 -V" to catch errors
When installing p4 as a dependency, we used to pipe output of "p4 -V"
and "p4d -V" to validate the installation and output a condensed version
information. But this would hide potential errors of p4 and would stop
with an empty output. E.g.: p4d version 16.2 running on ubuntu 22.04
causes sigfaults, even before it produces any output.

By removing the pipe after "p4 -V" and "p4d -V", we may get a
verbose output, and stop immediately on errors because we have "set
-e" in "ci/lib.sh". Since we won't look at these trace logs unless
something fails, just including the raw output seems most sensible.

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27 09:31:59 +09:00
0178420b9c github-actions: run gcc-8 on ubuntu-20.04 image
GitHub starts to upgrade its runner image "ubuntu-latest" from version
"ubuntu-20.04" to version "ubuntu-22.04". It will fail to find and
install "gcc-8" package on the new runner image.

Change some of the runner images from "ubuntu-latest" to "ubuntu-20.04"
in order to install "gcc-8" as a dependency.

The first revision of this patch tried to replace "$runs_on_pool" in
"ci/*.sh" with a new "$runs_on_os" environment variable based on the
"os" field in the matrix strategy. But these "os" fields in matrix
strategies are obsolete legacies from commit [1] and commit [2], and
are no longer useful. So remove these unused "os" fields.

[1]: c08bb26010 (CI: rename the "Linux32" job to lower-case "linux32",
                 2021-11-23)
[2]: 25715419bf (CI: don't run "make test" twice in one job, 2021-11-23)

Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-11-27 09:31:12 +09:00
53 changed files with 1162 additions and 230 deletions

View File

@ -23,8 +23,8 @@ jobs:
base=${{ github.event.before }}
head=${{ github.event.after }}
fi
echo "::set-output name=base::$base"
echo "::set-output name=head::$head"
echo base=$base >>$GITHUB_OUTPUT
echo head=$head >>$GITHUB_OUTPUT
- name: Run partial clone
run: |
git -c init.defaultBranch=master init --bare .

View File

@ -34,17 +34,17 @@ jobs:
then
enabled=no
fi
echo "::set-output name=enabled::$enabled"
echo "enabled=$enabled" >>$GITHUB_OUTPUT
- name: skip if the commit or tree was already tested
id: skip-if-redundant
uses: actions/github-script@v3
uses: actions/github-script@v6
if: steps.check-ref.outputs.enabled == 'yes'
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
try {
// Figure out workflow ID, commit and tree
const { data: run } = await github.actions.getWorkflowRun({
const { data: run } = await github.rest.actions.getWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
@ -54,7 +54,7 @@ jobs:
const tree_id = run.head_commit.tree_id;
// See whether there is a successful run for that commit or tree
const { data: runs } = await github.actions.listWorkflowRuns({
const { data: runs } = await github.rest.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 500,
@ -83,7 +83,7 @@ jobs:
if: needs.ci-config.outputs.enabled == 'yes'
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: build
shell: bash
@ -94,7 +94,7 @@ jobs:
- name: zip up tracked files
run: git archive -o artifacts/tracked.tar.gz HEAD
- name: upload tracked files and build artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: windows-artifacts
path: artifacts
@ -108,7 +108,7 @@ jobs:
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
steps:
- name: download tracked files and build artifacts
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
with:
name: windows-artifacts
path: ${{github.workspace}}
@ -125,7 +125,7 @@ jobs:
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
@ -138,10 +138,10 @@ jobs:
GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: initialize vcpkg
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
repository: 'microsoft/vcpkg'
path: 'compat/vcbuild/vcpkg'
@ -177,7 +177,7 @@ jobs:
- name: zip up tracked files
run: git archive -o artifacts/tracked.tar.gz HEAD
- name: upload tracked files and build artifacts
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: vs-artifacts
path: artifacts
@ -192,7 +192,7 @@ jobs:
steps:
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: download tracked files and build artifacts
uses: actions/download-artifact@v2
uses: actions/download-artifact@v3
with:
name: vs-artifacts
path: ${{github.workspace}}
@ -210,7 +210,7 @@ jobs:
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
@ -227,24 +227,22 @@ jobs:
pool: ubuntu-latest
- jobname: linux-sha256
cc: clang
os: ubuntu
pool: ubuntu-latest
- jobname: linux-gcc
cc: gcc
cc_package: gcc-8
pool: ubuntu-latest
pool: ubuntu-20.04
- jobname: linux-TEST-vars
cc: gcc
os: ubuntu
cc_package: gcc-8
pool: ubuntu-latest
pool: ubuntu-20.04
- jobname: osx-clang
cc: clang
pool: macos-latest
pool: macos-12
- jobname: osx-gcc
cc: gcc
cc_package: gcc-9
pool: macos-latest
pool: macos-12
- jobname: linux-gcc-default
cc: gcc
pool: ubuntu-latest
@ -258,16 +256,14 @@ jobs:
runs_on_pool: ${{matrix.vector.pool}}
runs-on: ${{matrix.vector.pool}}
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- run: ci/install-dependencies.sh
- run: ci/run-build-and-tests.sh
- name: print test failures
- run: ci/print-test-failures.sh
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
shell: bash
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v3
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
@ -282,7 +278,6 @@ jobs:
- jobname: linux-musl
image: alpine
- jobname: linux32
os: ubuntu32
image: daald/ubuntu32:xenial
- jobname: pedantic
image: fedora
@ -291,15 +286,22 @@ jobs:
runs-on: ubuntu-latest
container: ${{matrix.vector.image}}
steps:
- uses: actions/checkout@v3
if: matrix.vector.jobname != 'linux32'
- uses: actions/checkout@v1
if: matrix.vector.jobname == 'linux32'
- run: ci/install-docker-dependencies.sh
- run: ci/run-build-and-tests.sh
- name: print test failures
- run: ci/print-test-failures.sh
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
shell: bash
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname != 'linux32'
uses: actions/upload-artifact@v3
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != '' && matrix.vector.jobname == 'linux32'
uses: actions/upload-artifact@v1
with:
name: failed-tests-${{matrix.vector.jobname}}
@ -311,7 +313,7 @@ jobs:
jobname: StaticAnalysis
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- run: ci/install-dependencies.sh
- run: ci/run-static-analysis.sh
- run: ci/check-directional-formatting.bash
@ -331,7 +333,7 @@ jobs:
artifact: sparse-20.04
- name: Install the current `sparse` package
run: sudo dpkg -i sparse-20.04/sparse_*.deb
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Install other dependencies
run: ci/install-dependencies.sh
- run: make sparse
@ -343,6 +345,6 @@ jobs:
jobname: Documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- run: ci/install-dependencies.sh
- run: ci/test-documentation.sh

View File

@ -0,0 +1,86 @@
Git v2.30.7 Release Notes
=========================
This release addresses the security issues CVE-2022-41903 and
CVE-2022-23521.
Fixes since v2.30.6
-------------------
* CVE-2022-41903:
git log has the ability to display commits using an arbitrary
format with its --format specifiers. This functionality is also
exposed to git archive via the export-subst gitattribute.
When processing the padding operators (e.g., %<(, %<|(, %>(,
%>>(, or %><( ), an integer overflow can occur in
pretty.c::format_and_pad_commit() where a size_t is improperly
stored as an int, and then added as an offset to a subsequent
memcpy() call.
This overflow can be triggered directly by a user running a
command which invokes the commit formatting machinery (e.g., git
log --format=...). It may also be triggered indirectly through
git archive via the export-subst mechanism, which expands format
specifiers inside of files within the repository during a git
archive.
This integer overflow can result in arbitrary heap writes, which
may result in remote code execution.
* CVE-2022-23521:
gitattributes are a mechanism to allow defining attributes for
paths. These attributes can be defined by adding a `.gitattributes`
file to the repository, which contains a set of file patterns and
the attributes that should be set for paths matching this pattern.
When parsing gitattributes, multiple integer overflows can occur
when there is a huge number of path patterns, a huge number of
attributes for a single pattern, or when the declared attribute
names are huge.
These overflows can be triggered via a crafted `.gitattributes` file
that may be part of the commit history. Git silently splits lines
longer than 2KB when parsing gitattributes from a file, but not when
parsing them from the index. Consequentially, the failure mode
depends on whether the file exists in the working tree, the index or
both.
This integer overflow can result in arbitrary heap reads and writes,
which may result in remote code execution.
Credit for finding CVE-2022-41903 goes to Joern Schneeweisz of GitLab.
An initial fix was authored by Markus Vervier of X41 D-Sec. Credit for
finding CVE-2022-23521 goes to Markus Vervier and Eric Sesterhenn of X41
D-Sec. This work was sponsored by OSTIF.
The proposed fixes have been polished and extended to cover additional
findings by Patrick Steinhardt of GitLab, with help from others on the
Git security mailing list.
Patrick Steinhardt (21):
attr: fix overflow when upserting attribute with overly long name
attr: fix out-of-bounds read with huge attribute names
attr: fix integer overflow when parsing huge attribute names
attr: fix out-of-bounds write when parsing huge number of attributes
attr: fix out-of-bounds read with unreasonable amount of patterns
attr: fix integer overflow with more than INT_MAX macros
attr: harden allocation against integer overflows
attr: fix silently splitting up lines longer than 2048 bytes
attr: ignore attribute lines exceeding 2048 bytes
attr: ignore overly large gitattributes files
pretty: fix out-of-bounds write caused by integer overflow
pretty: fix out-of-bounds read when left-flushing with stealing
pretty: fix out-of-bounds read when parsing invalid padding format
pretty: fix adding linefeed when placeholder is not expanded
pretty: fix integer overflow in wrapping format
utf8: fix truncated string lengths in `utf8_strnwidth()`
utf8: fix returning negative string width
utf8: fix overflow when returning string width
utf8: fix checking for glyph width in `strbuf_utf8_replace()`
utf8: refactor `strbuf_utf8_replace` to not rely on preallocated buffer
pretty: restrict input lengths for padding and wrapping formats

View File

@ -0,0 +1,52 @@
Git v2.30.8 Release Notes
=========================
This release addresses the security issues CVE-2023-22490 and
CVE-2023-23946.
Fixes since v2.30.7
-------------------
* CVE-2023-22490:
Using a specially-crafted repository, Git can be tricked into using
its local clone optimization even when using a non-local transport.
Though Git will abort local clones whose source $GIT_DIR/objects
directory contains symbolic links (c.f., CVE-2022-39253), the objects
directory itself may still be a symbolic link.
These two may be combined to include arbitrary files based on known
paths on the victim's filesystem within the malicious repository's
working copy, allowing for data exfiltration in a similar manner as
CVE-2022-39253.
* CVE-2023-23946:
By feeding a crafted input to "git apply", a path outside the
working tree can be overwritten as the user who is running "git
apply".
* A mismatched type in `attr.c::read_attr_from_index()` which could
cause Git to errantly reject attributes on Windows and 32-bit Linux
has been corrected.
Credit for finding CVE-2023-22490 goes to yvvdwf, and the fix was
developed by Taylor Blau, with additional help from others on the
Git security mailing list.
Credit for finding CVE-2023-23946 goes to Joern Schneeweisz, and the
fix was developed by Patrick Steinhardt.
Johannes Schindelin (1):
attr: adjust a mismatched data type
Patrick Steinhardt (1):
apply: fix writing behind newly created symbolic links
Taylor Blau (3):
t5619: demonstrate clone_local() with ambiguous transport
clone: delay picking a transport until after get_repo_path()
dir-iterator: prevent top-level symlinks without FOLLOW_SYMLINKS

View File

@ -0,0 +1,5 @@
Git v2.31.6 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,6 @@
Git v2.31.7 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8 to
address the security issues CVE-2023-22490 and CVE-2023-23946;
see the release notes for that version for details.

View File

@ -0,0 +1,8 @@
Git v2.32.5 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.
In addition, included are additional code for "git fsck" to check
for questionable .gitattributes files.

View File

@ -0,0 +1,6 @@
Git v2.32.6 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8 and v2.31.7
to address the security issues CVE-2023-22490 and CVE-2023-23946;
see the release notes for these versions for details.

View File

@ -0,0 +1,5 @@
Git v2.33.6 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.33.7 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7
and v2.32.6 to address the security issues CVE-2023-22490 and
CVE-2023-23946; see the release notes for these versions for
details.

View File

@ -0,0 +1,5 @@
Git v2.34.6 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.34.7 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7,
v2.32.6 and v2.33.7 to address the security issues CVE-2023-22490
and CVE-2023-23946; see the release notes for these versions
for details.

View File

@ -0,0 +1,5 @@
Git v2.35.6 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.35.7 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7,
v2.32.6, v2.33.7 and v2.34.7 to address the security issues
CVE-2023-22490 and CVE-2023-23946; see the release notes for
these versions for details.

View File

@ -0,0 +1,5 @@
Git v2.36.4 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.36.5 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7,
v2.32.6, v2.33.7, v2.34.7 and v2.35.7 to address the security
issues CVE-2023-22490 and CVE-2023-23946; see the release notes
for these versions for details.

View File

@ -0,0 +1,5 @@
Git v2.37.5 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.37.6 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7,
v2.32.6, v2.33.7, v2.34.7, v2.35.7 and v2.36.5 to address the
security issues CVE-2023-22490 and CVE-2023-23946; see the release
notes for these versions for details.

View File

@ -57,4 +57,11 @@ Fixes since v2.38.1
* "git branch --edit-description" on an unborh branch misleadingly
said that no such branch exists, which has been corrected.
* GitHub CI settings have been adjusted to recent reality, merging
and cherry-picking necessary topics that have been prepared for Git
2.39.
* `git rebase --update-refs` would delete references when all `update-ref`
commands in the sequencer were removed, which has been corrected.
Also contains various documentation updates and code clean-ups.

View File

@ -0,0 +1,5 @@
Git v2.38.3 Release Notes
=========================
This release merges the security fix that appears in v2.30.7; see
the release notes for that version for details.

View File

@ -0,0 +1,7 @@
Git v2.38.4 Release Notes
=========================
This release merges up the fixes that appear in v2.30.8, v2.31.7,
v2.32.6, v2.33.7, v2.34.7, v2.35.7, v2.36.5 and v2.37.6 to
address the security issues CVE-2023-22490 and CVE-2023-23946;
see the release notes for these versions for details.

View File

@ -1,7 +1,7 @@
#!/bin/sh
GVF=GIT-VERSION-FILE
DEF_VER=v2.38.1
DEF_VER=v2.38.4
LF='
'

View File

@ -143,7 +143,7 @@ Issues of note:
not need that functionality, use NO_CURL to build without
it.
Git requires version "7.19.4" or later of "libcurl" to build
Git requires version "7.19.5" or later of "libcurl" to build
without NO_CURL. This version requirement may be bumped in
the future.

View File

@ -1 +1 @@
Documentation/RelNotes/2.38.2.txt
Documentation/RelNotes/2.38.4.txt

27
apply.c
View File

@ -4408,6 +4408,33 @@ static int create_one_file(struct apply_state *state,
if (state->cached)
return 0;
/*
* We already try to detect whether files are beyond a symlink in our
* up-front checks. But in the case where symlinks are created by any
* of the intermediate hunks it can happen that our up-front checks
* didn't yet see the symlink, but at the point of arriving here there
* in fact is one. We thus repeat the check for symlinks here.
*
* Note that this does not make the up-front check obsolete as the
* failure mode is different:
*
* - The up-front checks cause us to abort before we have written
* anything into the working directory. So when we exit this way the
* working directory remains clean.
*
* - The checks here happen in the middle of the action where we have
* already started to apply the patch. The end result will be a dirty
* working directory.
*
* Ideally, we should update the up-front checks to catch what would
* happen when we apply the patch before we damage the working tree.
* We have all the information necessary to do so. But for now, as a
* part of embargoed security work, having this check would serve as a
* reasonable first step.
*/
if (path_is_beyond_symlink(state, path))
return error(_("affected file '%s' is beyond a symbolic link"), path);
res = try_create_file(state, path, mode, buf, size);
if (res < 0)
return -1;

91
attr.c
View File

@ -24,7 +24,7 @@ static const char git_attr__unknown[] = "(builtin)unknown";
#define ATTR__UNKNOWN git_attr__unknown
struct git_attr {
int attr_nr; /* unique attribute number */
unsigned int attr_nr; /* unique attribute number */
char name[FLEX_ARRAY]; /* attribute name */
};
@ -206,7 +206,7 @@ static void report_invalid_attr(const char *name, size_t len,
* dictionary. If no entry is found, create a new attribute and store it in
* the dictionary.
*/
static const struct git_attr *git_attr_internal(const char *name, int namelen)
static const struct git_attr *git_attr_internal(const char *name, size_t namelen)
{
struct git_attr *a;
@ -222,8 +222,8 @@ static const struct git_attr *git_attr_internal(const char *name, int namelen)
a->attr_nr = hashmap_get_size(&g_attr_hashmap.map);
attr_hashmap_add(&g_attr_hashmap, a->name, namelen, a);
assert(a->attr_nr ==
(hashmap_get_size(&g_attr_hashmap.map) - 1));
if (a->attr_nr != hashmap_get_size(&g_attr_hashmap.map) - 1)
die(_("unable to add additional attribute"));
}
hashmap_unlock(&g_attr_hashmap);
@ -268,7 +268,7 @@ struct match_attr {
const struct git_attr *attr;
} u;
char is_macro;
unsigned num_attr;
size_t num_attr;
struct attr_state state[FLEX_ARRAY];
};
@ -289,7 +289,7 @@ static const char *parse_attr(const char *src, int lineno, const char *cp,
struct attr_state *e)
{
const char *ep, *equals;
int len;
size_t len;
ep = cp + strcspn(cp, blank);
equals = strchr(cp, '=');
@ -333,8 +333,7 @@ static const char *parse_attr(const char *src, int lineno, const char *cp,
static struct match_attr *parse_attr_line(const char *line, const char *src,
int lineno, unsigned flags)
{
int namelen;
int num_attr, i;
size_t namelen, num_attr, i;
const char *cp, *name, *states;
struct match_attr *res = NULL;
int is_macro;
@ -345,6 +344,11 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,
return NULL;
name = cp;
if (strlen(line) >= ATTR_MAX_LINE_LENGTH) {
warning(_("ignoring overly long attributes line %d"), lineno);
return NULL;
}
if (*cp == '"' && !unquote_c_style(&pattern, name, &states)) {
name = pattern.buf;
namelen = pattern.len;
@ -381,10 +385,9 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,
goto fail_return;
}
res = xcalloc(1,
sizeof(*res) +
sizeof(struct attr_state) * num_attr +
(is_macro ? 0 : namelen + 1));
res = xcalloc(1, st_add3(sizeof(*res),
st_mult(sizeof(struct attr_state), num_attr),
is_macro ? 0 : namelen + 1));
if (is_macro) {
res->u.attr = git_attr_internal(name, namelen);
} else {
@ -447,11 +450,12 @@ struct attr_stack {
static void attr_stack_free(struct attr_stack *e)
{
int i;
unsigned i;
free(e->origin);
for (i = 0; i < e->num_matches; i++) {
struct match_attr *a = e->attrs[i];
int j;
size_t j;
for (j = 0; j < a->num_attr; j++) {
const char *setto = a->state[j].setto;
if (setto == ATTR__TRUE ||
@ -660,8 +664,8 @@ static void handle_attr_line(struct attr_stack *res,
a = parse_attr_line(line, src, lineno, flags);
if (!a)
return;
ALLOC_GROW(res->attrs, res->num_matches + 1, res->alloc);
res->attrs[res->num_matches++] = a;
ALLOC_GROW_BY(res->attrs, res->num_matches, 1, res->alloc);
res->attrs[res->num_matches - 1] = a;
}
static struct attr_stack *read_attr_from_array(const char **list)
@ -701,11 +705,12 @@ void git_attr_set_direction(enum git_attr_direction new_direction)
static struct attr_stack *read_attr_from_file(const char *path, unsigned flags)
{
struct strbuf buf = STRBUF_INIT;
int fd;
FILE *fp;
struct attr_stack *res;
char buf[2048];
int lineno = 0;
struct stat st;
if (flags & READ_ATTR_NOFOLLOW)
fd = open_nofollow(path, O_RDONLY);
@ -717,15 +722,26 @@ static struct attr_stack *read_attr_from_file(const char *path, unsigned flags)
return NULL;
}
fp = xfdopen(fd, "r");
if (fstat(fd, &st)) {
warning_errno(_("cannot fstat gitattributes file '%s'"), path);
fclose(fp);
return NULL;
}
if (st.st_size >= ATTR_MAX_FILE_SIZE) {
warning(_("ignoring overly large gitattributes file '%s'"), path);
fclose(fp);
return NULL;
}
CALLOC_ARRAY(res, 1);
while (fgets(buf, sizeof(buf), fp)) {
char *bufp = buf;
if (!lineno)
skip_utf8_bom(&bufp, strlen(bufp));
handle_attr_line(res, bufp, path, ++lineno, flags);
while (strbuf_getline(&buf, fp) != EOF) {
if (!lineno && starts_with(buf.buf, utf8_bom))
strbuf_remove(&buf, 0, strlen(utf8_bom));
handle_attr_line(res, buf.buf, path, ++lineno, flags);
}
fclose(fp);
strbuf_release(&buf);
return res;
}
@ -736,6 +752,7 @@ static struct attr_stack *read_attr_from_index(struct index_state *istate,
struct attr_stack *res;
char *buf, *sp;
int lineno = 0;
unsigned long size;
if (!istate)
return NULL;
@ -754,9 +771,13 @@ static struct attr_stack *read_attr_from_index(struct index_state *istate,
if (!path_in_cone_mode_sparse_checkout(path, istate))
return NULL;
buf = read_blob_data_from_index(istate, path, NULL);
buf = read_blob_data_from_index(istate, path, &size);
if (!buf)
return NULL;
if (size >= ATTR_MAX_FILE_SIZE) {
warning(_("ignoring overly large gitattributes blob '%s'"), path);
return NULL;
}
CALLOC_ARRAY(res, 1);
for (sp = buf; *sp; ) {
@ -999,12 +1020,12 @@ static int macroexpand_one(struct all_attrs_item *all_attrs, int nr, int rem);
static int fill_one(struct all_attrs_item *all_attrs,
const struct match_attr *a, int rem)
{
int i;
size_t i;
for (i = a->num_attr - 1; rem > 0 && i >= 0; i--) {
const struct git_attr *attr = a->state[i].attr;
for (i = a->num_attr; rem > 0 && i > 0; i--) {
const struct git_attr *attr = a->state[i - 1].attr;
const char **n = &(all_attrs[attr->attr_nr].value);
const char *v = a->state[i].setto;
const char *v = a->state[i - 1].setto;
if (*n == ATTR__UNKNOWN) {
*n = v;
@ -1020,11 +1041,11 @@ static int fill(const char *path, int pathlen, int basename_offset,
struct all_attrs_item *all_attrs, int rem)
{
for (; rem > 0 && stack; stack = stack->prev) {
int i;
unsigned i;
const char *base = stack->origin ? stack->origin : "";
for (i = stack->num_matches - 1; 0 < rem && 0 <= i; i--) {
const struct match_attr *a = stack->attrs[i];
for (i = stack->num_matches; 0 < rem && 0 < i; i--) {
const struct match_attr *a = stack->attrs[i - 1];
if (a->is_macro)
continue;
if (path_matches(path, pathlen, basename_offset,
@ -1055,11 +1076,11 @@ static void determine_macros(struct all_attrs_item *all_attrs,
const struct attr_stack *stack)
{
for (; stack; stack = stack->prev) {
int i;
for (i = stack->num_matches - 1; i >= 0; i--) {
const struct match_attr *ma = stack->attrs[i];
unsigned i;
for (i = stack->num_matches; i > 0; i--) {
const struct match_attr *ma = stack->attrs[i - 1];
if (ma->is_macro) {
int n = ma->u.attr->attr_nr;
unsigned int n = ma->u.attr->attr_nr;
if (!all_attrs[n].macro) {
all_attrs[n].macro = ma;
}
@ -1111,7 +1132,7 @@ void git_check_attr(struct index_state *istate,
collect_some_attrs(istate, path, check);
for (i = 0; i < check->nr; i++) {
size_t n = check->items[i].attr->attr_nr;
unsigned int n = check->items[i].attr->attr_nr;
const char *value = check->all_attrs[n].value;
if (value == ATTR__UNKNOWN)
value = ATTR__UNSET;

12
attr.h
View File

@ -107,6 +107,18 @@
* - Free the `attr_check` struct by calling `attr_check_free()`.
*/
/**
* The maximum line length for a gitattributes file. If the line exceeds this
* length we will ignore it.
*/
#define ATTR_MAX_LINE_LENGTH 2048
/**
* The maximum size of the giattributes file. If the file exceeds this size we
* will ignore it.
*/
#define ATTR_MAX_FILE_SIZE (100 * 1024 * 1024)
struct index_state;
/**

View File

@ -1165,10 +1165,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
refspec_appendf(&remote->fetch, "+%s*:%s*", src_ref_prefix,
branch_top.buf);
transport = transport_get(remote, remote->url[0]);
transport_set_verbosity(transport, option_verbosity, option_progress);
transport->family = family;
path = get_repo_path(remote->url[0], &is_bundle);
is_local = option_local != 0 && path && !is_bundle;
if (is_local) {
@ -1190,6 +1186,10 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
}
if (option_local > 0 && !is_local)
warning(_("--local is ignored"));
transport = transport_get(remote, path ? path : remote->url[0]);
transport_set_verbosity(transport, option_verbosity, option_progress);
transport->family = family;
transport->cloning = 1;
if (is_bundle) {

View File

@ -5,17 +5,17 @@
. ${0%/*}/lib.sh
P4WHENCE=https://cdist2.perforce.com/perforce/r$LINUX_P4_VERSION
P4WHENCE=https://cdist2.perforce.com/perforce/r21.2
LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
UBUNTU_COMMON_PKGS="make libssl-dev libcurl4-openssl-dev libexpat-dev
tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl
libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl"
case "$runs_on_pool" in
ubuntu-latest)
ubuntu-*)
sudo apt-get -q update
sudo apt-get -q -y install language-pack-is libsvn-perl apache2 \
$UBUNTU_COMMON_PKGS $CC_PACKAGE
$UBUNTU_COMMON_PKGS $CC_PACKAGE $PYTHON_PACKAGE
mkdir --parents "$P4_PATH"
pushd "$P4_PATH"
wget --quiet "$P4WHENCE/bin.linux26x86_64/p4d"
@ -30,7 +30,7 @@ ubuntu-latest)
cp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .
popd
;;
macos-latest)
macos-*)
export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
# Uncomment this if you want to run perf tests:
# brew install gnu-time
@ -40,7 +40,7 @@ macos-latest)
mkdir -p $HOME/bin
(
cd $HOME/bin
wget -q "https://cdist2.perforce.com/perforce/r21.2/bin.macosx1015x86_64/helix-core-server.tgz" &&
wget -q "$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz" &&
tar -xf helix-core-server.tgz &&
sudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true
)
@ -83,9 +83,9 @@ esac
if type p4d >/dev/null 2>&1 && type p4 >/dev/null 2>&1
then
echo "$(tput setaf 6)Perforce Server Version$(tput sgr0)"
p4d -V | grep Rev.
p4d -V
echo "$(tput setaf 6)Perforce Client Version$(tput sgr0)"
p4 -V | grep Rev.
p4 -V
else
echo >&2 "WARNING: perforce wasn't installed, see above for clues why"
fi

View File

@ -226,18 +226,18 @@ export GIT_TEST_CLONE_2GB=true
export SKIP_DASHED_BUILT_INS=YesPlease
case "$runs_on_pool" in
ubuntu-latest)
ubuntu-*)
if test "$jobname" = "linux-gcc-default"
then
break
fi
if [ "$jobname" = linux-gcc ]
PYTHON_PACKAGE=python2
if test "$jobname" = linux-gcc
then
MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3"
else
MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python2"
PYTHON_PACKAGE=python3
fi
MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE"
export GIT_TEST_HTTPD=true
@ -246,14 +246,13 @@ ubuntu-latest)
# were recorded in the Homebrew database upon creating the OS X
# image.
# Keep that in mind when you encounter a broken OS X build!
export LINUX_P4_VERSION="16.2"
export LINUX_GIT_LFS_VERSION="1.5.2"
P4_PATH="$HOME/custom/p4"
GIT_LFS_PATH="$HOME/custom/git-lfs"
export PATH="$GIT_LFS_PATH:$P4_PATH:$PATH"
;;
macos-latest)
macos-*)
if [ "$jobname" = osx-gcc ]
then
MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=$(which python3)"

View File

@ -23,7 +23,7 @@ struct column_data {
/* return length of 's' in letters, ANSI escapes stripped */
static int item_length(const char *s)
{
return utf8_strnwidth(s, -1, 1);
return utf8_strnwidth(s, strlen(s), 1);
}
/*

View File

@ -203,7 +203,7 @@ struct dir_iterator *dir_iterator_begin(const char *path, unsigned int flags)
{
struct dir_iterator_int *iter = xcalloc(1, sizeof(*iter));
struct dir_iterator *dir_iterator = &iter->base;
int saved_errno;
int saved_errno, err;
strbuf_init(&iter->base.path, PATH_MAX);
strbuf_addstr(&iter->base.path, path);
@ -213,10 +213,15 @@ struct dir_iterator *dir_iterator_begin(const char *path, unsigned int flags)
iter->flags = flags;
/*
* Note: stat already checks for NULL or empty strings and
* inexistent paths.
* Note: stat/lstat already checks for NULL or empty strings and
* nonexistent paths.
*/
if (stat(iter->base.path.buf, &iter->base.st) < 0) {
if (iter->flags & DIR_ITERATOR_FOLLOW_SYMLINKS)
err = stat(iter->base.path.buf, &iter->base.st);
else
err = lstat(iter->base.path.buf, &iter->base.st);
if (err < 0) {
saved_errno = errno;
goto error_out;
}

View File

@ -61,6 +61,11 @@
* not the symlinks themselves, which is the default behavior. Broken
* symlinks are ignored.
*
* Note: setting DIR_ITERATOR_FOLLOW_SYMLINKS affects resolving the
* starting path as well (e.g., attempting to iterate starting at a
* symbolic link pointing to a directory without FOLLOW_SYMLINKS will
* result in an error).
*
* Warning: circular symlinks are also followed when
* DIR_ITERATOR_FOLLOW_SYMLINKS is set. The iteration may end up with
* an ELOOP if they happen and DIR_ITERATOR_PEDANTIC is set.

135
fsck.c
View File

@ -2,6 +2,7 @@
#include "object-store.h"
#include "repository.h"
#include "object.h"
#include "attr.h"
#include "blob.h"
#include "tree.h"
#include "tree-walk.h"
@ -614,17 +615,22 @@ static int fsck_tree(const struct object_id *tree_oid,
".gitmodules is a symbolic link");
}
if (is_hfs_dotgitattributes(name) || is_ntfs_dotgitattributes(name)) {
if (!S_ISLNK(mode))
oidset_insert(&options->gitattributes_found,
entry_oid);
else
retval += report(options, tree_oid, OBJ_TREE,
FSCK_MSG_GITATTRIBUTES_SYMLINK,
".gitattributes is a symlink");
}
if (S_ISLNK(mode)) {
if (is_hfs_dotgitignore(name) ||
is_ntfs_dotgitignore(name))
retval += report(options, tree_oid, OBJ_TREE,
FSCK_MSG_GITIGNORE_SYMLINK,
".gitignore is a symlink");
if (is_hfs_dotgitattributes(name) ||
is_ntfs_dotgitattributes(name))
retval += report(options, tree_oid, OBJ_TREE,
FSCK_MSG_GITATTRIBUTES_SYMLINK,
".gitattributes is a symlink");
if (is_hfs_dotmailmap(name) ||
is_ntfs_dotmailmap(name))
retval += report(options, tree_oid, OBJ_TREE,
@ -1159,38 +1165,70 @@ static int fsck_gitmodules_fn(const char *var, const char *value, void *vdata)
static int fsck_blob(const struct object_id *oid, const char *buf,
unsigned long size, struct fsck_options *options)
{
struct fsck_gitmodules_data data;
struct config_options config_opts = { 0 };
if (!oidset_contains(&options->gitmodules_found, oid))
return 0;
oidset_insert(&options->gitmodules_done, oid);
int ret = 0;
if (object_on_skiplist(options, oid))
return 0;
if (!buf) {
/*
* A missing buffer here is a sign that the caller found the
* blob too gigantic to load into memory. Let's just consider
* that an error.
*/
return report(options, oid, OBJ_BLOB,
FSCK_MSG_GITMODULES_LARGE,
".gitmodules too large to parse");
if (oidset_contains(&options->gitmodules_found, oid)) {
struct config_options config_opts = { 0 };
struct fsck_gitmodules_data data;
oidset_insert(&options->gitmodules_done, oid);
if (!buf) {
/*
* A missing buffer here is a sign that the caller found the
* blob too gigantic to load into memory. Let's just consider
* that an error.
*/
return report(options, oid, OBJ_BLOB,
FSCK_MSG_GITMODULES_LARGE,
".gitmodules too large to parse");
}
data.oid = oid;
data.options = options;
data.ret = 0;
config_opts.error_action = CONFIG_ERROR_SILENT;
if (git_config_from_mem(fsck_gitmodules_fn, CONFIG_ORIGIN_BLOB,
".gitmodules", buf, size, &data, &config_opts))
data.ret |= report(options, oid, OBJ_BLOB,
FSCK_MSG_GITMODULES_PARSE,
"could not parse gitmodules blob");
ret |= data.ret;
}
data.oid = oid;
data.options = options;
data.ret = 0;
config_opts.error_action = CONFIG_ERROR_SILENT;
if (git_config_from_mem(fsck_gitmodules_fn, CONFIG_ORIGIN_BLOB,
".gitmodules", buf, size, &data, &config_opts))
data.ret |= report(options, oid, OBJ_BLOB,
FSCK_MSG_GITMODULES_PARSE,
"could not parse gitmodules blob");
if (oidset_contains(&options->gitattributes_found, oid)) {
const char *ptr;
return data.ret;
oidset_insert(&options->gitattributes_done, oid);
if (!buf || size > ATTR_MAX_FILE_SIZE) {
/*
* A missing buffer here is a sign that the caller found the
* blob too gigantic to load into memory. Let's just consider
* that an error.
*/
return report(options, oid, OBJ_BLOB,
FSCK_MSG_GITATTRIBUTES_LARGE,
".gitattributes too large to parse");
}
for (ptr = buf; *ptr; ) {
const char *eol = strchrnul(ptr, '\n');
if (eol - ptr >= ATTR_MAX_LINE_LENGTH) {
ret |= report(options, oid, OBJ_BLOB,
FSCK_MSG_GITATTRIBUTES_LINE_LENGTH,
".gitattributes has too long lines to parse");
break;
}
ptr = *eol ? eol + 1 : eol;
}
}
return ret;
}
int fsck_object(struct object *obj, void *data, unsigned long size,
@ -1229,19 +1267,21 @@ int fsck_error_function(struct fsck_options *o,
return 1;
}
int fsck_finish(struct fsck_options *options)
static int fsck_blobs(struct oidset *blobs_found, struct oidset *blobs_done,
enum fsck_msg_id msg_missing, enum fsck_msg_id msg_type,
struct fsck_options *options, const char *blob_type)
{
int ret = 0;
struct oidset_iter iter;
const struct object_id *oid;
oidset_iter_init(&options->gitmodules_found, &iter);
oidset_iter_init(blobs_found, &iter);
while ((oid = oidset_iter_next(&iter))) {
enum object_type type;
unsigned long size;
char *buf;
if (oidset_contains(&options->gitmodules_done, oid))
if (oidset_contains(blobs_done, oid))
continue;
buf = read_object_file(oid, &type, &size);
@ -1249,25 +1289,36 @@ int fsck_finish(struct fsck_options *options)
if (is_promisor_object(oid))
continue;
ret |= report(options,
oid, OBJ_BLOB,
FSCK_MSG_GITMODULES_MISSING,
"unable to read .gitmodules blob");
oid, OBJ_BLOB, msg_missing,
"unable to read %s blob", blob_type);
continue;
}
if (type == OBJ_BLOB)
ret |= fsck_blob(oid, buf, size, options);
else
ret |= report(options,
oid, type,
FSCK_MSG_GITMODULES_BLOB,
"non-blob found at .gitmodules");
ret |= report(options, oid, type, msg_type,
"non-blob found at %s", blob_type);
free(buf);
}
oidset_clear(blobs_found);
oidset_clear(blobs_done);
return ret;
}
int fsck_finish(struct fsck_options *options)
{
int ret = 0;
ret |= fsck_blobs(&options->gitmodules_found, &options->gitmodules_done,
FSCK_MSG_GITMODULES_MISSING, FSCK_MSG_GITMODULES_BLOB,
options, ".gitmodules");
ret |= fsck_blobs(&options->gitattributes_found, &options->gitattributes_done,
FSCK_MSG_GITATTRIBUTES_MISSING, FSCK_MSG_GITATTRIBUTES_BLOB,
options, ".gitattributes");
oidset_clear(&options->gitmodules_found);
oidset_clear(&options->gitmodules_done);
return ret;
}

12
fsck.h
View File

@ -55,6 +55,10 @@ enum fsck_msg_type {
FUNC(GITMODULES_URL, ERROR) \
FUNC(GITMODULES_PATH, ERROR) \
FUNC(GITMODULES_UPDATE, ERROR) \
FUNC(GITATTRIBUTES_MISSING, ERROR) \
FUNC(GITATTRIBUTES_LARGE, ERROR) \
FUNC(GITATTRIBUTES_LINE_LENGTH, ERROR) \
FUNC(GITATTRIBUTES_BLOB, ERROR) \
/* warnings */ \
FUNC(EMPTY_NAME, WARN) \
FUNC(FULL_PATHNAME, WARN) \
@ -129,6 +133,8 @@ struct fsck_options {
struct oidset skiplist;
struct oidset gitmodules_found;
struct oidset gitmodules_done;
struct oidset gitattributes_found;
struct oidset gitattributes_done;
kh_oid_map_t *object_names;
};
@ -136,18 +142,24 @@ struct fsck_options {
.skiplist = OIDSET_INIT, \
.gitmodules_found = OIDSET_INIT, \
.gitmodules_done = OIDSET_INIT, \
.gitattributes_found = OIDSET_INIT, \
.gitattributes_done = OIDSET_INIT, \
.error_func = fsck_error_function \
}
#define FSCK_OPTIONS_STRICT { \
.strict = 1, \
.gitmodules_found = OIDSET_INIT, \
.gitmodules_done = OIDSET_INIT, \
.gitattributes_found = OIDSET_INIT, \
.gitattributes_done = OIDSET_INIT, \
.error_func = fsck_error_function, \
}
#define FSCK_OPTIONS_MISSING_GITMODULES { \
.strict = 1, \
.gitmodules_found = OIDSET_INIT, \
.gitmodules_done = OIDSET_INIT, \
.gitattributes_found = OIDSET_INIT, \
.gitattributes_done = OIDSET_INIT, \
.error_func = fsck_error_cb_print_missing_gitmodules, \
}

View File

@ -1015,6 +1015,14 @@ static inline unsigned long cast_size_t_to_ulong(size_t a)
return (unsigned long)a;
}
static inline int cast_size_t_to_int(size_t a)
{
if (a > INT_MAX)
die("number too large to represent as int on this platform: %"PRIuMAX,
(uintmax_t)a);
return (int)a;
}
/*
* Limit size of IO chunks, because huge chunks only cause pain. OS X
* 64-bit is buggy, returning EINVAL if len >= INT_MAX; and even in

View File

@ -126,4 +126,12 @@
#define GIT_CURL_HAVE_CURLSSLSET_NO_BACKENDS
#endif
/**
* CURLOPT_PROTOCOLS_STR and CURLOPT_REDIR_PROTOCOLS_STR were added in 7.85.0,
* released in August 2022.
*/
#if LIBCURL_VERSION_NUM >= 0x075500
#define GIT_CURL_HAVE_CURLOPT_PROTOCOLS_STR 1
#endif
#endif

View File

@ -198,13 +198,13 @@ static void curl_setup_http(CURL *curl, const char *url,
const char *custom_req, struct buffer *buffer,
curl_write_callback write_fn)
{
curl_easy_setopt(curl, CURLOPT_PUT, 1);
curl_easy_setopt(curl, CURLOPT_UPLOAD, 1);
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_INFILE, buffer);
curl_easy_setopt(curl, CURLOPT_INFILESIZE, buffer->buf.len);
curl_easy_setopt(curl, CURLOPT_READFUNCTION, fread_buffer);
curl_easy_setopt(curl, CURLOPT_IOCTLFUNCTION, ioctl_buffer);
curl_easy_setopt(curl, CURLOPT_IOCTLDATA, buffer);
curl_easy_setopt(curl, CURLOPT_SEEKFUNCTION, seek_buffer);
curl_easy_setopt(curl, CURLOPT_SEEKDATA, buffer);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_fn);
curl_easy_setopt(curl, CURLOPT_NOBODY, 0);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, custom_req);

79
http.c
View File

@ -157,21 +157,19 @@ size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_)
return size / eltsize;
}
curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp)
int seek_buffer(void *clientp, curl_off_t offset, int origin)
{
struct buffer *buffer = clientp;
switch (cmd) {
case CURLIOCMD_NOP:
return CURLIOE_OK;
case CURLIOCMD_RESTARTREAD:
buffer->posn = 0;
return CURLIOE_OK;
default:
return CURLIOE_UNKNOWNCMD;
if (origin != SEEK_SET)
BUG("seek_buffer only handles SEEK_SET");
if (offset < 0 || offset >= buffer->buf.len) {
error("curl seek would be outside of buffer");
return CURL_SEEKFUNC_FAIL;
}
buffer->posn = offset;
return CURL_SEEKFUNC_OK;
}
size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *buffer_)
@ -731,20 +729,37 @@ void setup_curl_trace(CURL *handle)
curl_easy_setopt(handle, CURLOPT_DEBUGDATA, NULL);
}
static long get_curl_allowed_protocols(int from_user)
static void proto_list_append(struct strbuf *list, const char *proto)
{
long allowed_protocols = 0;
if (!list)
return;
if (list->len)
strbuf_addch(list, ',');
strbuf_addstr(list, proto);
}
if (is_transport_allowed("http", from_user))
allowed_protocols |= CURLPROTO_HTTP;
if (is_transport_allowed("https", from_user))
allowed_protocols |= CURLPROTO_HTTPS;
if (is_transport_allowed("ftp", from_user))
allowed_protocols |= CURLPROTO_FTP;
if (is_transport_allowed("ftps", from_user))
allowed_protocols |= CURLPROTO_FTPS;
static long get_curl_allowed_protocols(int from_user, struct strbuf *list)
{
long bits = 0;
return allowed_protocols;
if (is_transport_allowed("http", from_user)) {
bits |= CURLPROTO_HTTP;
proto_list_append(list, "http");
}
if (is_transport_allowed("https", from_user)) {
bits |= CURLPROTO_HTTPS;
proto_list_append(list, "https");
}
if (is_transport_allowed("ftp", from_user)) {
bits |= CURLPROTO_FTP;
proto_list_append(list, "ftp");
}
if (is_transport_allowed("ftps", from_user)) {
bits |= CURLPROTO_FTPS;
proto_list_append(list, "ftps");
}
return bits;
}
#ifdef GIT_CURL_HAVE_CURL_HTTP_VERSION_2
@ -888,10 +903,26 @@ static CURL *get_curl_handle(void)
curl_easy_setopt(result, CURLOPT_MAXREDIRS, 20);
curl_easy_setopt(result, CURLOPT_POSTREDIR, CURL_REDIR_POST_ALL);
#ifdef GIT_CURL_HAVE_CURLOPT_PROTOCOLS_STR
{
struct strbuf buf = STRBUF_INIT;
get_curl_allowed_protocols(0, &buf);
curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS_STR, buf.buf);
strbuf_reset(&buf);
get_curl_allowed_protocols(-1, &buf);
curl_easy_setopt(result, CURLOPT_PROTOCOLS_STR, buf.buf);
strbuf_release(&buf);
}
#else
curl_easy_setopt(result, CURLOPT_REDIR_PROTOCOLS,
get_curl_allowed_protocols(0));
get_curl_allowed_protocols(0, NULL));
curl_easy_setopt(result, CURLOPT_PROTOCOLS,
get_curl_allowed_protocols(-1));
get_curl_allowed_protocols(-1, NULL));
#endif
if (getenv("GIT_CURL_VERBOSE"))
http_trace_curl_no_data();
setup_curl_trace(result);

2
http.h
View File

@ -40,7 +40,7 @@ struct buffer {
size_t fread_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf);
size_t fwrite_buffer(char *ptr, size_t eltsize, size_t nmemb, void *strbuf);
size_t fwrite_null(char *ptr, size_t eltsize, size_t nmemb, void *strbuf);
curlioerr ioctl_buffer(CURL *handle, int cmd, void *clientp);
int seek_buffer(void *clientp, curl_off_t offset, int origin);
/* Slot lifecycle functions */
struct active_request_slot *get_active_slot(void);

View File

@ -14,6 +14,13 @@
#include "trailer.h"
#include "run-command.h"
/*
* The limit for formatting directives, which enable the caller to append
* arbitrarily many bytes to the formatted buffer. This includes padding
* and wrapping formatters.
*/
#define FORMATTING_LIMIT (16 * 1024)
static char *user_format;
static struct cmt_fmt_map {
const char *name;
@ -994,7 +1001,9 @@ static void strbuf_wrap(struct strbuf *sb, size_t pos,
if (pos)
strbuf_add(&tmp, sb->buf, pos);
strbuf_add_wrapped_text(&tmp, sb->buf + pos,
(int) indent1, (int) indent2, (int) width);
cast_size_t_to_int(indent1),
cast_size_t_to_int(indent2),
cast_size_t_to_int(width));
strbuf_swap(&tmp, sb);
strbuf_release(&tmp);
}
@ -1120,9 +1129,18 @@ static size_t parse_padding_placeholder(const char *placeholder,
const char *end = start + strcspn(start, ",)");
char *next;
int width;
if (!end || end == start)
if (!*end || end == start)
return 0;
width = strtol(start, &next, 10);
/*
* We need to limit the amount of padding, or otherwise this
* would allow the user to pad the buffer by arbitrarily many
* bytes and thus cause resource exhaustion.
*/
if (width < -FORMATTING_LIMIT || width > FORMATTING_LIMIT)
return 0;
if (next == start || width == 0)
return 0;
if (width < 0) {
@ -1405,6 +1423,16 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */
if (*next != ')')
return 0;
}
/*
* We need to limit the format here as it allows the
* user to prepend arbitrarily many bytes to the buffer
* when rewrapping.
*/
if (width > FORMATTING_LIMIT ||
indent1 > FORMATTING_LIMIT ||
indent2 > FORMATTING_LIMIT)
return 0;
rewrap_message_tail(sb, c, width, indent1, indent2);
return end - placeholder + 1;
} else
@ -1670,19 +1698,21 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
struct format_commit_context *c)
{
struct strbuf local_sb = STRBUF_INIT;
int total_consumed = 0, len, padding = c->padding;
size_t total_consumed = 0;
int len, padding = c->padding;
if (padding < 0) {
const char *start = strrchr(sb->buf, '\n');
int occupied;
if (!start)
start = sb->buf;
occupied = utf8_strnwidth(start, -1, 1);
occupied = utf8_strnwidth(start, strlen(start), 1);
occupied += c->pretty_ctx->graph_width;
padding = (-padding) - occupied;
}
while (1) {
int modifier = *placeholder == 'C';
int consumed = format_commit_one(&local_sb, placeholder, c);
size_t consumed = format_commit_one(&local_sb, placeholder, c);
total_consumed += consumed;
if (!modifier)
@ -1694,7 +1724,7 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
placeholder++;
total_consumed++;
}
len = utf8_strnwidth(local_sb.buf, -1, 1);
len = utf8_strnwidth(local_sb.buf, local_sb.len, 1);
if (c->flush_type == flush_left_and_steal) {
const char *ch = sb->buf + sb->len - 1;
@ -1709,7 +1739,7 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
if (*ch != 'm')
break;
p = ch - 1;
while (ch - p < 10 && *p != '\033')
while (p > sb->buf && ch - p < 10 && *p != '\033')
p--;
if (*p != '\033' ||
ch + 1 - p != display_mode_esc_sequence_len(p))
@ -1748,7 +1778,7 @@ static size_t format_and_pad_commit(struct strbuf *sb, /* in UTF-8 */
}
strbuf_addbuf(sb, &local_sb);
} else {
int sb_len = sb->len, offset = 0;
size_t sb_len = sb->len, offset = 0;
if (c->flush_type == flush_left)
offset = padding - len;
else if (c->flush_type == flush_both)
@ -1771,8 +1801,7 @@ static size_t format_commit_item(struct strbuf *sb, /* in UTF-8 */
const char *placeholder,
void *context)
{
int consumed;
size_t orig_len;
size_t consumed, orig_len;
enum {
NO_MAGIC,
ADD_LF_BEFORE_NON_EMPTY,
@ -1793,9 +1822,21 @@ static size_t format_commit_item(struct strbuf *sb, /* in UTF-8 */
default:
break;
}
if (magic != NO_MAGIC)
if (magic != NO_MAGIC) {
placeholder++;
switch (placeholder[0]) {
case 'w':
/*
* `%+w()` cannot ever expand to a non-empty string,
* and it potentially changes the layout of preceding
* contents. We're thus not able to handle the magic in
* this combination and refuse the pattern.
*/
return 0;
};
}
orig_len = sb->len;
if (((struct format_commit_context *)context)->flush_type != no_flush)
consumed = format_and_pad_commit(sb, placeholder, context);

View File

@ -717,25 +717,23 @@ static size_t rpc_out(void *ptr, size_t eltsize,
return avail;
}
static curlioerr rpc_ioctl(CURL *handle, int cmd, void *clientp)
static int rpc_seek(void *clientp, curl_off_t offset, int origin)
{
struct rpc_state *rpc = clientp;
switch (cmd) {
case CURLIOCMD_NOP:
return CURLIOE_OK;
if (origin != SEEK_SET)
BUG("rpc_seek only handles SEEK_SET, not %d", origin);
case CURLIOCMD_RESTARTREAD:
if (rpc->initial_buffer) {
rpc->pos = 0;
return CURLIOE_OK;
if (rpc->initial_buffer) {
if (offset < 0 || offset > rpc->len) {
error("curl seek would be outside of rpc buffer");
return CURL_SEEKFUNC_FAIL;
}
error(_("unable to rewind rpc post data - try increasing http.postBuffer"));
return CURLIOE_FAILRESTART;
default:
return CURLIOE_UNKNOWNCMD;
rpc->pos = offset;
return CURL_SEEKFUNC_OK;
}
error(_("unable to rewind rpc post data - try increasing http.postBuffer"));
return CURL_SEEKFUNC_FAIL;
}
struct check_pktline_state {
@ -959,8 +957,8 @@ retry:
rpc->initial_buffer = 1;
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, rpc_ioctl);
curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, rpc);
curl_easy_setopt(slot->curl, CURLOPT_SEEKFUNCTION, rpc_seek);
curl_easy_setopt(slot->curl, CURLOPT_SEEKDATA, rpc);
if (options.verbosity > 1) {
fprintf(stderr, "POST %s (chunked)\n", rpc->service_name);
fflush(stderr);

View File

@ -4130,11 +4130,14 @@ static int write_update_refs_state(struct string_list *refs_to_oids)
struct string_list_item *item;
char *path;
if (!refs_to_oids->nr)
return 0;
path = rebase_path_update_refs(the_repository->gitdir);
if (!refs_to_oids->nr) {
if (unlink(path) && errno != ENOENT)
result = error_errno(_("could not unlink: %s"), path);
goto cleanup;
}
if (safe_create_leading_directories(path)) {
result = error(_("unable to create leading directories of %s"),
path);

View File

@ -376,4 +376,63 @@ test_expect_success SYMLINKS 'symlinks not respected in-tree' '
test_i18ngrep "unable to access.*gitattributes" err
'
test_expect_success 'large attributes line ignored in tree' '
test_when_finished "rm .gitattributes" &&
printf "path %02043d" 1 >.gitattributes &&
git check-attr --all path >actual 2>err &&
echo "warning: ignoring overly long attributes line 1" >expect &&
test_cmp expect err &&
test_must_be_empty actual
'
test_expect_success 'large attributes line ignores trailing content in tree' '
test_when_finished "rm .gitattributes" &&
# older versions of Git broke lines at 2048 bytes; the 2045 bytes
# of 0-padding here is accounting for the three bytes of "a 1", which
# would knock "trailing" to the "next" line, where it would be
# erroneously parsed.
printf "a %02045dtrailing attribute\n" 1 >.gitattributes &&
git check-attr --all trailing >actual 2>err &&
echo "warning: ignoring overly long attributes line 1" >expect &&
test_cmp expect err &&
test_must_be_empty actual
'
test_expect_success EXPENSIVE 'large attributes file ignored in tree' '
test_when_finished "rm .gitattributes" &&
dd if=/dev/zero of=.gitattributes bs=101M count=1 2>/dev/null &&
git check-attr --all path >/dev/null 2>err &&
echo "warning: ignoring overly large gitattributes file ${SQ}.gitattributes${SQ}" >expect &&
test_cmp expect err
'
test_expect_success 'large attributes line ignored in index' '
test_when_finished "git update-index --remove .gitattributes" &&
blob=$(printf "path %02043d" 1 | git hash-object -w --stdin) &&
git update-index --add --cacheinfo 100644,$blob,.gitattributes &&
git check-attr --cached --all path >actual 2>err &&
echo "warning: ignoring overly long attributes line 1" >expect &&
test_cmp expect err &&
test_must_be_empty actual
'
test_expect_success 'large attributes line ignores trailing content in index' '
test_when_finished "git update-index --remove .gitattributes" &&
blob=$(printf "a %02045dtrailing attribute\n" 1 | git hash-object -w --stdin) &&
git update-index --add --cacheinfo 100644,$blob,.gitattributes &&
git check-attr --cached --all trailing >actual 2>err &&
echo "warning: ignoring overly long attributes line 1" >expect &&
test_cmp expect err &&
test_must_be_empty actual
'
test_expect_success EXPENSIVE 'large attributes file ignored in index' '
test_when_finished "git update-index --remove .gitattributes" &&
blob=$(dd if=/dev/zero bs=101M count=1 2>/dev/null | git hash-object -w --stdin) &&
git update-index --add --cacheinfo 100644,$blob,.gitattributes &&
git check-attr --cached --all path >/dev/null 2>err &&
echo "warning: ignoring overly large gitattributes blob ${SQ}.gitattributes${SQ}" >expect &&
test_cmp expect err
'
test_done

View File

@ -110,7 +110,9 @@ test_expect_success SYMLINKS 'setup dirs with symlinks' '
mkdir -p dir5/a/c &&
ln -s ../c dir5/a/b/d &&
ln -s ../ dir5/a/b/e &&
ln -s ../../ dir5/a/b/f
ln -s ../../ dir5/a/b/f &&
ln -s dir4 dir6
'
test_expect_success SYMLINKS 'dir-iterator should not follow symlinks by default' '
@ -146,4 +148,27 @@ test_expect_success SYMLINKS 'dir-iterator should follow symlinks w/ follow flag
test_cmp expected-follow-sorted-output actual-follow-sorted-output
'
test_expect_success SYMLINKS 'dir-iterator does not resolve top-level symlinks' '
test_must_fail test-tool dir-iterator ./dir6 >out &&
grep "ENOTDIR" out
'
test_expect_success SYMLINKS 'dir-iterator resolves top-level symlinks w/ follow flag' '
cat >expected-follow-sorted-output <<-EOF &&
[d] (a) [a] ./dir6/a
[d] (a/f) [f] ./dir6/a/f
[d] (a/f/c) [c] ./dir6/a/f/c
[d] (b) [b] ./dir6/b
[d] (b/c) [c] ./dir6/b/c
[f] (a/d) [d] ./dir6/a/d
[f] (a/e) [e] ./dir6/a/e
EOF
test-tool dir-iterator --follow-symlinks ./dir6 >out &&
sort out >actual-follow-sorted-output &&
test_cmp expected-follow-sorted-output actual-follow-sorted-output
'
test_done

View File

@ -999,4 +999,28 @@ test_expect_success 'fsck error and recovery on invalid object type' '
)
'
test_expect_success 'fsck error on gitattributes with excessive line lengths' '
blob=$(printf "pattern %02048d" 1 | git hash-object -w --stdin) &&
test_when_finished "remove_object $blob" &&
tree=$(printf "100644 blob %s\t%s\n" $blob .gitattributes | git mktree) &&
test_when_finished "remove_object $tree" &&
cat >expected <<-EOF &&
error in blob $blob: gitattributesLineLength: .gitattributes has too long lines to parse
EOF
test_must_fail git fsck --no-dangling >actual 2>&1 &&
test_cmp expected actual
'
test_expect_success 'fsck error on gitattributes with excessive size' '
blob=$(test-tool genzeros $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&
test_when_finished "remove_object $blob" &&
tree=$(printf "100644 blob %s\t%s\n" $blob .gitattributes | git mktree) &&
test_when_finished "remove_object $tree" &&
cat >expected <<-EOF &&
error in blob $blob: gitattributesLarge: .gitattributes too large to parse
EOF
test_must_fail git fsck --no-dangling >actual 2>&1 &&
test_cmp expected actual
'
test_done

View File

@ -1964,6 +1964,113 @@ test_expect_success 'respect user edits to update-ref steps' '
test_cmp_rev HEAD refs/heads/no-conflict-branch
'
test_expect_success '--update-refs: all update-ref lines removed' '
git checkout -b test-refs-not-removed no-conflict-branch &&
git branch -f base HEAD~4 &&
git branch -f first HEAD~3 &&
git branch -f second HEAD~3 &&
git branch -f third HEAD~1 &&
git branch -f tip &&
test_commit test-refs-not-removed &&
git commit --amend --fixup first &&
git rev-parse first second third tip no-conflict-branch >expect-oids &&
(
set_cat_todo_editor &&
test_must_fail git rebase -i --update-refs base >todo.raw &&
sed -e "/^update-ref/d" <todo.raw >todo
) &&
(
set_replace_editor todo &&
git rebase -i --update-refs base
) &&
# Ensure refs are not deleted and their OIDs have not changed
git rev-parse first second third tip no-conflict-branch >actual-oids &&
test_cmp expect-oids actual-oids
'
test_expect_success '--update-refs: all update-ref lines removed, then some re-added' '
git checkout -b test-refs-not-removed2 no-conflict-branch &&
git branch -f base HEAD~4 &&
git branch -f first HEAD~3 &&
git branch -f second HEAD~3 &&
git branch -f third HEAD~1 &&
git branch -f tip &&
test_commit test-refs-not-removed2 &&
git commit --amend --fixup first &&
git rev-parse first second third >expect-oids &&
(
set_cat_todo_editor &&
test_must_fail git rebase -i \
--autosquash --update-refs \
base >todo.raw &&
sed -e "/^update-ref/d" <todo.raw >todo
) &&
# Add a break to the end of the todo so we can edit later
echo "break" >>todo &&
(
set_replace_editor todo &&
git rebase -i --autosquash --update-refs base &&
echo "update-ref refs/heads/tip" >todo &&
git rebase --edit-todo &&
git rebase --continue
) &&
# Ensure first/second/third are unchanged, but tip is updated
git rev-parse first second third >actual-oids &&
test_cmp expect-oids actual-oids &&
test_cmp_rev HEAD tip
'
test_expect_success '--update-refs: --edit-todo with no update-ref lines' '
git checkout -b test-refs-not-removed3 no-conflict-branch &&
git branch -f base HEAD~4 &&
git branch -f first HEAD~3 &&
git branch -f second HEAD~3 &&
git branch -f third HEAD~1 &&
git branch -f tip &&
test_commit test-refs-not-removed3 &&
git commit --amend --fixup first &&
git rev-parse first second third tip no-conflict-branch >expect-oids &&
(
set_cat_todo_editor &&
test_must_fail git rebase -i \
--autosquash --update-refs \
base >todo.raw &&
sed -e "/^update-ref/d" <todo.raw >todo
) &&
# Add a break to the beginning of the todo so we can resume with no
# update-ref lines
echo "break" >todo.new &&
cat todo >>todo.new &&
(
set_replace_editor todo.new &&
git rebase -i --autosquash --update-refs base &&
# Make no changes when editing so update-refs is still empty
cat todo >todo.new &&
git rebase --edit-todo &&
git rebase --continue
) &&
# Ensure refs are not deleted and their OIDs have not changed
git rev-parse first second third tip no-conflict-branch >actual-oids &&
test_cmp expect-oids actual-oids
'
test_expect_success '--update-refs: check failed ref update' '
git checkout -B update-refs-error no-conflict-branch &&
git branch -f base HEAD~4 &&

View File

@ -45,4 +45,85 @@ test_expect_success 'apply --index symlink patch' '
'
test_expect_success 'symlink setup' '
ln -s .git symlink &&
git add symlink &&
git commit -m "add symlink"
'
test_expect_success SYMLINKS 'symlink escape when creating new files' '
test_when_finished "git reset --hard && git clean -dfx" &&
cat >patch <<-EOF &&
diff --git a/symlink b/renamed-symlink
similarity index 100%
rename from symlink
rename to renamed-symlink
--
diff --git /dev/null b/renamed-symlink/create-me
new file mode 100644
index 0000000..039727e
--- /dev/null
+++ b/renamed-symlink/create-me
@@ -0,0 +1,1 @@
+busted
EOF
test_must_fail git apply patch 2>stderr &&
cat >expected_stderr <<-EOF &&
error: affected file ${SQ}renamed-symlink/create-me${SQ} is beyond a symbolic link
EOF
test_cmp expected_stderr stderr &&
! test_path_exists .git/create-me
'
test_expect_success SYMLINKS 'symlink escape when modifying file' '
test_when_finished "git reset --hard && git clean -dfx" &&
touch .git/modify-me &&
cat >patch <<-EOF &&
diff --git a/symlink b/renamed-symlink
similarity index 100%
rename from symlink
rename to renamed-symlink
--
diff --git a/renamed-symlink/modify-me b/renamed-symlink/modify-me
index 1111111..2222222 100644
--- a/renamed-symlink/modify-me
+++ b/renamed-symlink/modify-me
@@ -0,0 +1,1 @@
+busted
EOF
test_must_fail git apply patch 2>stderr &&
cat >expected_stderr <<-EOF &&
error: renamed-symlink/modify-me: No such file or directory
EOF
test_cmp expected_stderr stderr &&
test_must_be_empty .git/modify-me
'
test_expect_success SYMLINKS 'symlink escape when deleting file' '
test_when_finished "git reset --hard && git clean -dfx && rm .git/delete-me" &&
touch .git/delete-me &&
cat >patch <<-EOF &&
diff --git a/symlink b/renamed-symlink
similarity index 100%
rename from symlink
rename to renamed-symlink
--
diff --git a/renamed-symlink/delete-me b/renamed-symlink/delete-me
deleted file mode 100644
index 1111111..0000000 100644
EOF
test_must_fail git apply patch 2>stderr &&
cat >expected_stderr <<-EOF &&
error: renamed-symlink/delete-me: No such file or directory
EOF
test_cmp expected_stderr stderr &&
test_path_is_file .git/delete-me
'
test_done

View File

@ -1018,4 +1018,80 @@ test_expect_success '%(describe:abbrev=...) vs git describe --abbrev=...' '
test_cmp expect actual
'
test_expect_success 'log --pretty with space stealing' '
printf mm0 >expect &&
git log -1 --pretty="format:mm%>>|(1)%x30" >actual &&
test_cmp expect actual
'
test_expect_success 'log --pretty with invalid padding format' '
printf "%s%%<(20" "$(git rev-parse HEAD)" >expect &&
git log -1 --pretty="format:%H%<(20" >actual &&
test_cmp expect actual
'
test_expect_success 'log --pretty with magical wrapping directives' '
commit_id=$(git commit-tree HEAD^{tree} -m "describe me") &&
git tag describe-me $commit_id &&
printf "\n(tag:\ndescribe-me)%%+w(2)" >expect &&
git log -1 --pretty="format:%w(1)%+d%+w(2)" $commit_id >actual &&
test_cmp expect actual
'
test_expect_success SIZE_T_IS_64BIT 'log --pretty with overflowing wrapping directive' '
printf "%%w(2147483649,1,1)0" >expect &&
git log -1 --pretty="format:%w(2147483649,1,1)%x30" >actual &&
test_cmp expect actual &&
printf "%%w(1,2147483649,1)0" >expect &&
git log -1 --pretty="format:%w(1,2147483649,1)%x30" >actual &&
test_cmp expect actual &&
printf "%%w(1,1,2147483649)0" >expect &&
git log -1 --pretty="format:%w(1,1,2147483649)%x30" >actual &&
test_cmp expect actual
'
test_expect_success SIZE_T_IS_64BIT 'log --pretty with overflowing padding directive' '
printf "%%<(2147483649)0" >expect &&
git log -1 --pretty="format:%<(2147483649)%x30" >actual &&
test_cmp expect actual
'
test_expect_success 'log --pretty with padding and preceding control chars' '
printf "\20\20 0" >expect &&
git log -1 --pretty="format:%x10%x10%>|(4)%x30" >actual &&
test_cmp expect actual
'
test_expect_success 'log --pretty truncation with control chars' '
test_commit "$(printf "\20\20\20\20xxxx")" file contents commit-with-control-chars &&
printf "\20\20\20\20x.." >expect &&
git log -1 --pretty="format:%<(3,trunc)%s" commit-with-control-chars >actual &&
test_cmp expect actual
'
test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '
# We only assert that this command does not crash. This needs to be
# executed with the address sanitizer to demonstrate failure.
git log -1 --pretty="format:%>(2147483646)%x41%41%>(2147483646)%x41" >/dev/null
'
test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '
test-tool genzeros 2147483649 | tr "\000" "1" >expect &&
huge_commit=$(git commit-tree -F expect HEAD^{tree})
'
test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '
git log -1 --format="%B%<(1)%x30" $huge_commit >actual &&
echo 0 >>expect &&
test_cmp expect actual
'
test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '
test_must_fail git log -1 --format="%<(1)%B" $huge_commit 2>error &&
cat >expect <<-EOF &&
fatal: number too large to represent as int on this platform: 2147483649
EOF
test_cmp expect error
'
test_done

View File

@ -344,4 +344,20 @@ test_expect_success SYMLINKS 'clone repo with symlinked or unknown files at obje
test_must_be_empty T--shared.objects-symlinks.raw
'
test_expect_success SYMLINKS 'clone repo with symlinked objects directory' '
test_when_finished "rm -fr sensitive malicious" &&
mkdir -p sensitive &&
echo "secret" >sensitive/file &&
git init malicious &&
rm -fr malicious/.git/objects &&
ln -s "$(pwd)/sensitive" ./malicious/.git/objects &&
test_must_fail git clone --local malicious clone 2>err &&
test_path_is_missing clone &&
grep "failed to start iterator over" err
'
test_done

View File

@ -0,0 +1,70 @@
#!/bin/sh
test_description='test local clone with ambiguous transport'
. ./test-lib.sh
. "$TEST_DIRECTORY/lib-httpd.sh"
if ! test_have_prereq SYMLINKS
then
skip_all='skipping test, symlink support unavailable'
test_done
fi
start_httpd
REPO="$HTTPD_DOCUMENT_ROOT_PATH/sub.git"
URI="$HTTPD_URL/dumb/sub.git"
test_expect_success 'setup' '
mkdir -p sensitive &&
echo "secret" >sensitive/secret &&
git init --bare "$REPO" &&
test_commit_bulk -C "$REPO" --ref=main 1 &&
git -C "$REPO" update-ref HEAD main &&
git -C "$REPO" update-server-info &&
git init malicious &&
(
cd malicious &&
git submodule add "$URI" &&
mkdir -p repo/refs &&
touch repo/refs/.gitkeep &&
printf "ref: refs/heads/a" >repo/HEAD &&
ln -s "$(cd .. && pwd)/sensitive" repo/objects &&
mkdir -p "$HTTPD_URL/dumb" &&
ln -s "../../../.git/modules/sub/../../../repo/" "$URI" &&
git add . &&
git commit -m "initial commit"
) &&
# Delete all of the references in our malicious submodule to
# avoid the client attempting to checkout any objects (which
# will be missing, and thus will cause the clone to fail before
# we can trigger the exploit).
git -C "$REPO" for-each-ref --format="delete %(refname)" >in &&
git -C "$REPO" update-ref --stdin <in &&
git -C "$REPO" update-server-info
'
test_expect_success 'ambiguous transport does not lead to arbitrary file-inclusion' '
git clone malicious clone &&
test_must_fail git -C clone submodule update --init 2>err &&
test_path_is_missing clone/.git/modules/sub/objects/secret &&
# We would actually expect "transport .file. not allowed" here,
# but due to quirks of the URL detection in Git, we mis-parse
# the absolute path as a bogus URL and die before that step.
#
# This works for now, and if we ever fix the URL detection, it
# is OK to change this to detect the transport error.
grep "protocol .* is not supported" err
'
test_done

77
utf8.c
View File

@ -206,26 +206,34 @@ int utf8_width(const char **start, size_t *remainder_p)
* string, assuming that the string is utf8. Returns strlen() instead
* if the string does not look like a valid utf8 string.
*/
int utf8_strnwidth(const char *string, int len, int skip_ansi)
int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
{
int width = 0;
const char *orig = string;
size_t width = 0;
if (len == -1)
len = strlen(string);
while (string && string < orig + len) {
int skip;
int glyph_width;
size_t skip;
while (skip_ansi &&
(skip = display_mode_esc_sequence_len(string)) != 0)
string += skip;
width += utf8_width(&string, NULL);
glyph_width = utf8_width(&string, NULL);
if (glyph_width > 0)
width += glyph_width;
}
return string ? width : len;
/*
* TODO: fix the interface of this function and `utf8_strwidth()` to
* return `size_t` instead of `int`.
*/
return cast_size_t_to_int(string ? width : len);
}
int utf8_strwidth(const char *string)
{
return utf8_strnwidth(string, -1, 0);
return utf8_strnwidth(string, strlen(string), 0);
}
int is_utf8(const char *text)
@ -357,51 +365,52 @@ void strbuf_add_wrapped_bytes(struct strbuf *buf, const char *data, int len,
void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
const char *subst)
{
struct strbuf sb_dst = STRBUF_INIT;
char *src = sb_src->buf;
char *end = src + sb_src->len;
char *dst;
int w = 0, subst_len = 0;
const char *src = sb_src->buf, *end = sb_src->buf + sb_src->len;
struct strbuf dst;
int w = 0;
if (subst)
subst_len = strlen(subst);
strbuf_grow(&sb_dst, sb_src->len + subst_len);
dst = sb_dst.buf;
strbuf_init(&dst, sb_src->len);
while (src < end) {
char *old;
const char *old;
int glyph_width;
size_t n;
while ((n = display_mode_esc_sequence_len(src))) {
memcpy(dst, src, n);
strbuf_add(&dst, src, n);
src += n;
dst += n;
}
if (src >= end)
break;
old = src;
n = utf8_width((const char**)&src, NULL);
if (!src) /* broken utf-8, do nothing */
glyph_width = utf8_width((const char**)&src, NULL);
if (!src) /* broken utf-8, do nothing */
goto out;
if (n && w >= pos && w < pos + width) {
/*
* In case we see a control character we copy it into the
* buffer, but don't add it to the width.
*/
if (glyph_width < 0)
glyph_width = 0;
if (glyph_width && w >= pos && w < pos + width) {
if (subst) {
memcpy(dst, subst, subst_len);
dst += subst_len;
strbuf_addstr(&dst, subst);
subst = NULL;
}
w += n;
continue;
} else {
strbuf_add(&dst, old, src - old);
}
memcpy(dst, old, src - old);
dst += src - old;
w += n;
w += glyph_width;
}
strbuf_setlen(&sb_dst, dst - sb_dst.buf);
strbuf_swap(sb_src, &sb_dst);
strbuf_swap(sb_src, &dst);
out:
strbuf_release(&sb_dst);
strbuf_release(&dst);
}
/*
@ -796,7 +805,7 @@ int skip_utf8_bom(char **text, size_t len)
void strbuf_utf8_align(struct strbuf *buf, align_type position, unsigned int width,
const char *s)
{
int slen = strlen(s);
size_t slen = strlen(s);
int display_len = utf8_strnwidth(s, slen, 0);
int utf8_compensation = slen - display_len;

2
utf8.h
View File

@ -7,7 +7,7 @@ typedef unsigned int ucs_char_t; /* assuming 32bit int */
size_t display_mode_esc_sequence_len(const char *s);
int utf8_width(const char **start, size_t *remainder_p);
int utf8_strnwidth(const char *string, int len, int skip_ansi);
int utf8_strnwidth(const char *string, size_t len, int skip_ansi);
int utf8_strwidth(const char *string);
int is_utf8(const char *text);
int is_encoding_utf8(const char *name);