Files
git/.github/workflows/main.yml
Johannes Schindelin 0eb6c189a3 ci: use the new GitHub Action to download git-sdk-64-minimal
In our continuous builds, Windows is the odd cookie that requires a
complete development environment to be downloaded because there is no
suitable one installed by default on Windows.

Side note: technically, there _is_ a development environment present in
GitHub Actions' build agents: MSYS2. But it differs from Git for
Windows' SDK in subtle points, unfortunately enough so to prevent Git's
test suite from running without failures.

Traditionally, we support downloading this environment (which we
nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses
the build artifacts of a dedicated Azure Pipeline (which packages a tiny
subset of the full Git for Windows SDK, containing just enough to build
Git and run its test suite).

This PowerShell script is unfortunately not very robust and sometimes
fails due to network issues.

Of course, we could add code to detect that situation, wait a little,
try again, if it fails again wait a little longer, lather, rinse and
repeat.

Instead of doing all of this in Git's own `.github/workflows/`, though,
let's offload this logic to the new GitHub Action at
https://github.com/marketplace/actions/setup-git-for-windows-sdk

This Action not only downloads and extracts git-sdk-64-minimal _outside_
the worktree (making it no longer necessary to meddle with
`.gitignore` or `.git/info/exclude`), it also adds the `bash.exe` to the
`PATH` and sets the environment variable `MSYSTEM` (an implementation
detail that Git's workflow should never have needed to know about).

This allows us to convert all those funny PowerShell tasks that wanted
to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash`
scriptlets.

This finally lets us get rid of the funny quoting and escaping where we
had to pay attention not only to quote and escape the Bash scriptlets
properly, but also to add a second level of escaping (with backslashes
for double quotes and backticks for dollar signs) to stop PowerShell
from doing unintended things.

Further, this Action uses a fast caching strategy native to GitHub
Actions that should accelerate the download across CI runs:
git-sdk-64-minimal is usually updated once per 24h, and needs to be
cached only once within that period. Caching it (unfortunately only on
a per-branch basis) speeds up the download step, and makes it much more
robust at the same time by virtue of accessing a cache location that is
closer in the network topology.

With this we can drop the home-rolled caching where we try to accelerate
the test phase by uploading git-sdk-64-minimal as a workflow artifact
after using it to build Git, and then download it as workflow artifact
in the test phase.

Even better: the `vs-test` job no longer needs to depend on the
`windows-build` job. The only reason it depended on it was to ensure
that the `git-sdk-64-minimal` workflow artifact was available.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28 20:35:39 -07:00

298 lines
9.6 KiB
YAML

name: CI/PR
on: [push, pull_request]
env:
DEVELOPER: 1
jobs:
ci-config:
runs-on: ubuntu-latest
outputs:
enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}
steps:
- name: try to clone ci-config branch
run: |
git -c protocol.version=2 clone \
--no-tags \
--single-branch \
-b ci-config \
--depth 1 \
--no-checkout \
--filter=blob:none \
https://github.com/${{ github.repository }} \
config-repo &&
cd config-repo &&
git checkout HEAD -- ci/config || : ignore
- id: check-ref
name: check whether CI is enabled for ref
run: |
enabled=yes
if test -x config-repo/ci/config/allow-ref &&
! config-repo/ci/config/allow-ref '${{ github.ref }}'
then
enabled=no
fi
echo "::set-output name=enabled::$enabled"
- name: skip if the commit or tree was already tested
id: skip-if-redundant
uses: actions/github-script@v3
if: steps.check-ref.outputs.enabled == 'yes'
with:
github-token: ${{secrets.GITHUB_TOKEN}}
script: |
try {
// Figure out workflow ID, commit and tree
const { data: run } = await github.actions.getWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
});
const workflow_id = run.workflow_id;
const head_sha = run.head_sha;
const tree_id = run.head_commit.tree_id;
// See whether there is a successful run for that commit or tree
const { data: runs } = await github.actions.listWorkflowRuns({
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 500,
status: 'success',
workflow_id,
});
for (const run of runs.workflow_runs) {
if (head_sha === run.head_sha) {
core.warning(`Successful run for the commit ${head_sha}: ${run.html_url}`);
core.setOutput('enabled', ' but skip');
break;
}
if (run.head_commit && tree_id === run.head_commit.tree_id) {
core.warning(`Successful run for the tree ${tree_id}: ${run.html_url}`);
core.setOutput('enabled', ' but skip');
break;
}
}
} catch (e) {
core.warning(e);
}
windows-build:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
runs-on: windows-latest
steps:
- uses: actions/checkout@v1
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: build
shell: bash
env:
HOME: ${{runner.workspace}}
NO_PERL: 1
run: ci/make-test-artifacts.sh artifacts
- name: upload build artifacts
uses: actions/upload-artifact@v1
with:
name: windows-artifacts
path: artifacts
windows-test:
runs-on: windows-latest
needs: [windows-build]
strategy:
fail-fast: false
matrix:
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
steps:
- uses: actions/checkout@v1
- name: download build artifacts
uses: actions/download-artifact@v1
with:
name: windows-artifacts
path: ${{github.workspace}}
- name: extract build artifacts
shell: bash
run: tar xf artifacts.tar.gz
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: test
shell: bash
run: ci/run-test-slice.sh ${{matrix.nr}} 10
- name: ci/print-test-failures.sh
if: failure()
shell: bash
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v1
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
vs-build:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
env:
NO_PERL: 1
GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
runs-on: windows-latest
steps:
- uses: actions/checkout@v1
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: initialize vcpkg
uses: actions/checkout@v2
with:
repository: 'microsoft/vcpkg'
path: 'compat/vcbuild/vcpkg'
- name: download vcpkg artifacts
shell: powershell
run: |
$urlbase = "https://dev.azure.com/git/git/_apis/build/builds"
$id = ((Invoke-WebRequest -UseBasicParsing "${urlbase}?definitions=9&statusFilter=completed&resultFilter=succeeded&`$top=1").content | ConvertFrom-JSON).value[0].id
$downloadUrl = ((Invoke-WebRequest -UseBasicParsing "${urlbase}/$id/artifacts").content | ConvertFrom-JSON).value[0].resource.downloadUrl
(New-Object Net.WebClient).DownloadFile($downloadUrl, "compat.zip")
Expand-Archive compat.zip -DestinationPath . -Force
Remove-Item compat.zip
- name: add msbuild to PATH
uses: microsoft/setup-msbuild@v1
- name: copy dlls to root
shell: powershell
run: |
& compat\vcbuild\vcpkg_copy_dlls.bat release
if (!$?) { exit(1) }
- name: generate Visual Studio solution
shell: bash
run: |
cmake `pwd`/contrib/buildsystems/ -DCMAKE_PREFIX_PATH=`pwd`/compat/vcbuild/vcpkg/installed/x64-windows \
-DMSGFMT_EXE=C:/git-sdk-64-minimal/mingw64/bin/msgfmt.exe -DPERL_TESTS=OFF -DPYTHON_TESTS=OFF -DCURL_NO_CURL_CMAKE=ON
- name: MSBuild
run: msbuild git.sln -property:Configuration=Release -property:Platform=x64 -maxCpuCount:4 -property:PlatformToolset=v142
- name: bundle artifact tar
shell: bash
env:
MSVC: 1
VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
run: |
mkdir -p artifacts &&
eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts 2>&1 | grep ^tar)"
- name: upload build artifacts
uses: actions/upload-artifact@v1
with:
name: vs-artifacts
path: artifacts
vs-test:
runs-on: windows-latest
needs: vs-build
strategy:
fail-fast: false
matrix:
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
steps:
- uses: actions/checkout@v1
- uses: git-for-windows/setup-git-for-windows-sdk@v1
- name: download build artifacts
uses: actions/download-artifact@v1
with:
name: vs-artifacts
path: ${{github.workspace}}
- name: extract build artifacts
shell: bash
run: tar xf artifacts.tar.gz
- name: test
shell: bash
env:
NO_SVN_TESTS: 1
GIT_TEST_SKIP_REBASE_P: 1
run: ci/run-test-slice.sh ${{matrix.nr}} 10
- name: ci/print-test-failures.sh
if: failure()
shell: bash
run: ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v1
with:
name: failed-tests-windows
path: ${{env.FAILED_TEST_ARTIFACTS}}
regular:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
strategy:
fail-fast: false
matrix:
vector:
- jobname: linux-clang
cc: clang
pool: ubuntu-latest
- jobname: linux-gcc
cc: gcc
pool: ubuntu-latest
- jobname: osx-clang
cc: clang
pool: macos-latest
- jobname: osx-gcc
cc: gcc
pool: macos-latest
- jobname: linux-gcc-default
cc: gcc
pool: ubuntu-latest
env:
CC: ${{matrix.vector.cc}}
jobname: ${{matrix.vector.jobname}}
runs-on: ${{matrix.vector.pool}}
steps:
- uses: actions/checkout@v1
- run: ci/install-dependencies.sh
- run: ci/run-build-and-tests.sh
- run: ci/print-test-failures.sh
if: failure()
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v1
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
dockerized:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
strategy:
fail-fast: false
matrix:
vector:
- jobname: linux-musl
image: alpine
- jobname: Linux32
image: daald/ubuntu32:xenial
env:
jobname: ${{matrix.vector.jobname}}
runs-on: ubuntu-latest
container: ${{matrix.vector.image}}
steps:
- uses: actions/checkout@v1
- run: ci/install-docker-dependencies.sh
- run: ci/run-build-and-tests.sh
- run: ci/print-test-failures.sh
if: failure()
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v1
with:
name: failed-tests-${{matrix.vector.jobname}}
path: ${{env.FAILED_TEST_ARTIFACTS}}
static-analysis:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
env:
jobname: StaticAnalysis
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v1
- run: ci/install-dependencies.sh
- run: ci/run-static-analysis.sh
documentation:
needs: ci-config
if: needs.ci-config.outputs.enabled == 'yes'
env:
jobname: Documentation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- run: ci/install-dependencies.sh
- run: ci/test-documentation.sh