Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.
$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all
pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.
Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
Replace .gitlab-ci.yml with a minimal .build.yml that runs a script in
the .ci dir for each task. Each of these scripts runs as root first,
installs dependencies, and then drops rights to the testuser and runs
the actual test. I went with this design, so we can add a
'pmbootstrap ci' command in following patches that run these in a
pmbootstrap chroot (see pmaports#2169).
Looking at flake8, we currently ignore W504 and W604. Would be nice to
fix these instead, but let's do that in another patch.
I've added a minver package to Alpine, so we don't need to install it
form pip anymore. Use minver's new --lint argument to simplify the
script, and as recommended by minver itself, use --no-parse-comments as
speed optimization since we don't have minver specific comments.
Using doas instead of sudo in build.yml would be nice, but this doesn't
work out of the box. Can be changed in the future.
Add a note to the pytest script, that 'pmbootstrap log' can be used to
follow the log while tests are running.
When running the testsuite, most logging gets written to a separate
log_testsuite.txt file. Check if it exists, and if so, instruct tail to
print its output as well. This allows immediatelly figuring out what the
testsuite is doing without manually attaching to log_testsuite.txt
(which I often did while running the testsuite).
At the moment the "envkernel.sh hasn't run, assuming the kernel was
cross compiled on host and using current dir as source" code path
triggers even when using envkernel.sh, which works somewhat but
requires sourcing envkernel.sh again after each invocation of
"pmbootstrap build --envkernel ...".
The reason is that os.path.ismount() does not work for bind mounts
(see https://bugs.python.org/issue29707). There is a workaround for
that already in pmbootstrap but it is not used here for some reason.
Enable pmbootstrap to work on riscv64 packages, and at the same time
also expand the alpine_to_hostspec list to match upstream (adding
riscv32 and loongarch*).
While introducing multiple pmb:kconfigcheck-* APKBUILD options we missed
adding them to this array leading to potential apkbuild-lint failures.
This hasn't been noticed much because the check in apkbuild-lint is
relatively buggy, see https://gitlab.alpinelinux.org/Leo/atools/-/issues/48
Add support for packaging a kernel that was compiled outside of
envkernel. The envkernel.sh wrapper is great for someone new to kernel
development, but it makes it difficult to do things like "make
dt_binding_check". This lets you avoid the wrapper and build on your
host machine but still use pmbootstrap to package your kernel for easier
testing.
The plan was to drop this workaround, once alpine releases a new gcc
version, and we regenerate our cross compilers against that version
_and_ drop options="!tracedeps", as latest pmbootstrap aportgen would
do. However !tracedeps wasn't dropped in the latest upgrade. This was
too easy to miss, I'll add a comment to the gcc-cross APKBUILDs so we
see it next time. Let's just update the workaround for this version.
Device packages shouldn't depend on it anymore, in order to end up with
a smaller installation for ui=none this package only gets pulled in by
postmarketos-base if mesa is installed.
Related: pmaports MR 3478
Set the env var, so python programs running inside pmbootstrap chroot
don't buffer their output and only end up printing everything when they
are done. I've seen this with meson. This is bad for usability because
we don't see output, but also a problem because pmbootstrap kills
commands if they don't print any output for some time (default: 15 min).
Install isl25 for the specific gcc versions we currently have in
pmaports master that are missing it; this is easier than building new
packages and going out of sync with Alpine's versioning. This can be
reverted as soon as there is a newer gcc.
Related: pmaports issue 1732
abuild traces shared library dependencies for binaries in built
packages, and adds all of them to depends. This is needed for the cross
gcc builds, as otherwise it will not add the isl dependency (currently
isl25).
With the crossdirect compilation method we mount the cross-gcc from the
native chroot in the foreign arch chroot. Due to the missing dependency,
isl was not installed in the native chroot, and so when attempting to
use that native cross gcc from the foreign arch chroot, it would try to
load the foreign arch isl instead of the native one. Resulting in lots
of the following errors (and of course not a working gcc):
Error relocating /usr/lib/libisl.so.23: unsupported relocation type 1026
I had probably added '!tracedeps' because abuild complains about it
otherwise... but as noted in [1] these complaints are safe to ignore.
Keep !strip for now even though it's not in Alpine's APKBUILD. I've
tried to build without and the build fails with:
strip: Unable to recognise the format of the input file `./usr/aarch64-alpine-linux-musl/lib/libgcc_s.so.1'
[1]: 0415ebf6f9
Related: pmaports issue 1732
Add the subpackage, because g++-$arch depends on it since
12.1.1_git20220630-r6. This fix was already applied to pmaports, thanks
to ungeskriptet! Adjust the aportgen code to make sure this is added in
the future as well.
Related: pmaports MR 3464
Related: f1044b1dae
Do not build the foreign arch libgcc. We are using the one provided by
Alpine's binary repository instead, and when building it here and trying
to use it, we actually run into a conflict.
I tried setting BOOTSTRAP="nolibc" first since there is already logic to
then disable libgcc in the APKBUILD. But that also disables g++ which we
do want to build, so we can't use this.
Fixes: issue 2168
Remove it from depends, as the package has been removed in Alpine and
since quite some time we shouldn't have depended on it anyway.
Related: issue 2167
We'll be enabling the kernel options for waydroid which would break this
test. Change it to UEFI check instead which will probably really never
be used on N900.
Add the new option that will be mandatory for all devices in
community/main category. This is just a combination of anbox + iwd +
nftables + containers + zram + netboot.
While the existing options could be removed we're keeping it for now
given that also some devices with downstream kernel might find some
options useful.
Do not only parse triggers, but also emit them in the packages dicts in
pmb.parse.apkbuild. This is the first half to fix pmaports issue 1690,
the other is in pmaports MR 3386.
Set CTARGET and CTARGET_ARCH on top of the generated APKBUILD and use
the existing code in the APKBUILD for build(), package(), makedepends,
subpackages. This is now possible, because the pmbootstrap parser for
APKBUILDs has been improved a lot since this was originally written and
because the Alpine APKBUILD does everything that was hacked in here,
like removing man, info directories and bfd-plugins if doing a cross
build.
This is now very similar to bootstrapping binutils for another
architecture via scripts/bootstrap.sh in aports.git.
Most importantly this sets --disable-gprofng, which fails to build
against musl and is the reason why our cross/binutils-* packages
generated from binutils 2.39.x didn't build anymore.
Related: https://git.alpinelinux.org/aports/tree/main/binutils/APKBUILD
Related: https://sourceware.org/bugzilla/show_bug.cgi?id=29477
Even add_cmd has exited with code 99, it will be overwritten by clean_cmd
later. Exit with code returned by add_cmd to raise a runtime error when
an error occurs.
Don't pass stdin to commands that aren't supposed to be used
interactively (output: log, background, pipe).
This fixes an inconsistency between building packages in CI on gitlab
and building them via bpo on sourcehut or locally. In gitlab, apparently
there is no stdin for the entire build job and so unanswered kernel
config prompts will just use the default. In local builds and on
sourcehut stdin is available and so it just hangs at the prompt until
pmbootstrap kills the build job due to no output being written.
I considered adding an additional check to pmaports to ensure that there
are no unanswered kernel config prompts just in case users run abuild
manually on the kernel APKBUILD with stdin available. But I think
forcing the users to answer all the prompts even if it's not really
needed just creates additional work / makes the workflow worse without
real benefit.
Related: https://builds.sr.ht/~postmarketos/job/824373#task-pmbootstrap_build-432
Fixes: pmaports issue 1225
Here's an example of the properties this could generate:
deviceinfo_header_version="2"
deviceinfo_append_dtb="false"
deviceinfo_flash_offset_dtb="0x0bc08000"
This includes setting header_version="2" as well as dtb_offset
according to the input boot.img when header v2 is detected.
Also adds the following previously missed deviceinfo_attributes:
* "header_version"
* "bootimg_custom_args"
And fixes failing tests now that header_version is always parsed from
boot.img files.