Python passes all arguments by reference. For mutable objects such as
a list used here, changing the object will change the original one. If
components_list is not set, this means the default value gets modified.
This lead to kernels getting checked with the wrong required components.
For example, when checking a kernel from community first with the
default component_list, it would get extended with all options needed
for a kernel in community. When checking another kernel from the
testing category, also with the default component_list, it would now
check for the community options in the testing kernel.
Related: https://stackoverflow.com/a/986145
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230402124338.64886-1-ollieparanoid@postmarketos.org%3E
Don't set the config_path_pretty variable and pass it through various
functions until it ends up being used in check_option. This was just the
directory name of the kernel config and the kernel config filename (or
in case of pmbootstrap kconfig check --file, just the same as the
config path).
Instead we can just print the filename of the kernel config, for example
"config-postmarketos-qcom-sdm845.aarch64". It is shorter and already
obvious to which package it belongs.
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-3-ollieparanoid@postmarketos.org%3E
When looking at the table for possible output modes, it only makes sense
to pass stdin to "interactive" and "tui". The output mode "stdout" is
for non-interactive commands.
This fixes apk going interactive (asking for confirmation) when running
pmbootstrap with --details-to-stdout and building a package that depends
on postmarketos-base.
Fixes: issue 2208
Tested-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230301204112.4351-1-ollieparanoid@postmarketos.org%3E
pmOS cannot boot without these two. There was recently someone in the
unofficial postmarketOS Telegram group who used some minimal config and
couldn't figure out why the initramfs couldn't start up, and these
options being disabled turned out to be the culprit. As such, let's
ensure these are enabled to make life easier for people setting up new
kernel configurations.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230128203716.152281-1-newbyte@postmarketos.org%3E
Always set GOCACHE during build and let it point to pmbootstrap's work
dir. This has a similar effect as using ccache for C/C++.
Set GOMODCACHE conditionally: this is for caching the go modules (git
repositories) that get cloned during a build if they are not bundled
with the source. Usually APKBUILDs should cache them, but when using
pmbootstrap build --src, they would get downloaded over and over again.
Set GOMODCACHE automatically for --src and allow enabling/disabling this
manually with new --go-mod-cache / --no-go-mod-cache.
This speeds up multiple iterations of building the same package
significantly. I'm using it for:
$ pmbootstrap build postmarketos-mkinitfs --src=...
I've verified that using the same GOCACHE dir for multiple architectures
doesn't result in build artifacts for the wrong arch getting used.
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230223064743.1773-1-ollieparanoid@postmarketos.org%3E
Instead of letting the mount -t binfmt_misc … command fail, and simply
telling the user that the command failed: rely on the following check
that prints a better error message on error. If it fails at this point,
the kernel option isn't enabled. Add two comments explaining why
check=False is used in this function while at it.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230123064516.1607-1-ollieparanoid@postmarketos.org%3E
Recent changes to qemu and Alpine packaging now require using the
virtio-vga-gl device and installing -gl packages to get virglrenderer
support.
Without this, wlroots fails to get an EGL context (among other problems
you'd expect by not having a useful GPU around...)
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221210185021.3546-1-clayton@craftyguy.net%3E
Running abuild on the host directly creates directories in the
aport where it gets built. Interrupting abuild results in those
working-dirs not getting deleted.
We don't want to copy those entries to our builder.
It's only really noticeable if pmbootstrap tries to copy a broken
symlink in src/ and thus fails.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221207205201.22139-1-jane400@bingo-ev.de%3E
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.
Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.
As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
Adjust to avbtool now being part of android-tools in alpine edge.
Instead of trying to install both (which fails on edge), take the
dependencies from a new pmaports.cfg variable
supported_fastboot_depends, which only contains android-tools in
pmaports.git master branch.
Related: https://postmarketos.org/pmaports.cfg
The typical workflow for upgrading cross/gcc-* is:
$ cd pmaports/cross
$ pmbootstrap aportgen gcc-*
Currently this is failing because the APKBUILD has been updated in
Alpine for gcc, but gcc has not been built for all arches yet. This
shouldn't prevent us from generating the proper updated APKBUILDs in
pmaports so just print a note and don't fail here.
I'm committing this directly to master as this currently breaks
test/test_aportgen.py::test_aportgen.
On qemu-system-riscv64 the -nic option doesn't seem to work correctly.
qemu-system-riscv64: warning: requested NIC (anonymous, model virtio-net-pci) was not created (not supported by this machine?)
Using -netdev and -device provides the same functionality and also works
on riscv64.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-1-luca@z3ntu.xyz%3E
Commit d8f2f20186 removed
the requirement of selecting MEMCG_SWAP for Kernels >= 6.1.
However, it did not fully account for the behavior change,
as MEMCG_SWAP depends on both MEMCG and SWAP.
Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.
$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all
pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.
Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
When running the testsuite, most logging gets written to a separate
log_testsuite.txt file. Check if it exists, and if so, instruct tail to
print its output as well. This allows immediatelly figuring out what the
testsuite is doing without manually attaching to log_testsuite.txt
(which I often did while running the testsuite).
At the moment the "envkernel.sh hasn't run, assuming the kernel was
cross compiled on host and using current dir as source" code path
triggers even when using envkernel.sh, which works somewhat but
requires sourcing envkernel.sh again after each invocation of
"pmbootstrap build --envkernel ...".
The reason is that os.path.ismount() does not work for bind mounts
(see https://bugs.python.org/issue29707). There is a workaround for
that already in pmbootstrap but it is not used here for some reason.
Enable pmbootstrap to work on riscv64 packages, and at the same time
also expand the alpine_to_hostspec list to match upstream (adding
riscv32 and loongarch*).
While introducing multiple pmb:kconfigcheck-* APKBUILD options we missed
adding them to this array leading to potential apkbuild-lint failures.
This hasn't been noticed much because the check in apkbuild-lint is
relatively buggy, see https://gitlab.alpinelinux.org/Leo/atools/-/issues/48
Add support for packaging a kernel that was compiled outside of
envkernel. The envkernel.sh wrapper is great for someone new to kernel
development, but it makes it difficult to do things like "make
dt_binding_check". This lets you avoid the wrapper and build on your
host machine but still use pmbootstrap to package your kernel for easier
testing.
The plan was to drop this workaround, once alpine releases a new gcc
version, and we regenerate our cross compilers against that version
_and_ drop options="!tracedeps", as latest pmbootstrap aportgen would
do. However !tracedeps wasn't dropped in the latest upgrade. This was
too easy to miss, I'll add a comment to the gcc-cross APKBUILDs so we
see it next time. Let's just update the workaround for this version.
Device packages shouldn't depend on it anymore, in order to end up with
a smaller installation for ui=none this package only gets pulled in by
postmarketos-base if mesa is installed.
Related: pmaports MR 3478
Set the env var, so python programs running inside pmbootstrap chroot
don't buffer their output and only end up printing everything when they
are done. I've seen this with meson. This is bad for usability because
we don't see output, but also a problem because pmbootstrap kills
commands if they don't print any output for some time (default: 15 min).
Install isl25 for the specific gcc versions we currently have in
pmaports master that are missing it; this is easier than building new
packages and going out of sync with Alpine's versioning. This can be
reverted as soon as there is a newer gcc.
Related: pmaports issue 1732
abuild traces shared library dependencies for binaries in built
packages, and adds all of them to depends. This is needed for the cross
gcc builds, as otherwise it will not add the isl dependency (currently
isl25).
With the crossdirect compilation method we mount the cross-gcc from the
native chroot in the foreign arch chroot. Due to the missing dependency,
isl was not installed in the native chroot, and so when attempting to
use that native cross gcc from the foreign arch chroot, it would try to
load the foreign arch isl instead of the native one. Resulting in lots
of the following errors (and of course not a working gcc):
Error relocating /usr/lib/libisl.so.23: unsupported relocation type 1026
I had probably added '!tracedeps' because abuild complains about it
otherwise... but as noted in [1] these complaints are safe to ignore.
Keep !strip for now even though it's not in Alpine's APKBUILD. I've
tried to build without and the build fails with:
strip: Unable to recognise the format of the input file `./usr/aarch64-alpine-linux-musl/lib/libgcc_s.so.1'
[1]: 0415ebf6f9
Related: pmaports issue 1732
Add the subpackage, because g++-$arch depends on it since
12.1.1_git20220630-r6. This fix was already applied to pmaports, thanks
to ungeskriptet! Adjust the aportgen code to make sure this is added in
the future as well.
Related: pmaports MR 3464
Related: f1044b1dae
Do not build the foreign arch libgcc. We are using the one provided by
Alpine's binary repository instead, and when building it here and trying
to use it, we actually run into a conflict.
I tried setting BOOTSTRAP="nolibc" first since there is already logic to
then disable libgcc in the APKBUILD. But that also disables g++ which we
do want to build, so we can't use this.
Fixes: issue 2168
Remove it from depends, as the package has been removed in Alpine and
since quite some time we shouldn't have depended on it anyway.
Related: issue 2167
Add the new option that will be mandatory for all devices in
community/main category. This is just a combination of anbox + iwd +
nftables + containers + zram + netboot.
While the existing options could be removed we're keeping it for now
given that also some devices with downstream kernel might find some
options useful.
Do not only parse triggers, but also emit them in the packages dicts in
pmb.parse.apkbuild. This is the first half to fix pmaports issue 1690,
the other is in pmaports MR 3386.
Set CTARGET and CTARGET_ARCH on top of the generated APKBUILD and use
the existing code in the APKBUILD for build(), package(), makedepends,
subpackages. This is now possible, because the pmbootstrap parser for
APKBUILDs has been improved a lot since this was originally written and
because the Alpine APKBUILD does everything that was hacked in here,
like removing man, info directories and bfd-plugins if doing a cross
build.
This is now very similar to bootstrapping binutils for another
architecture via scripts/bootstrap.sh in aports.git.
Most importantly this sets --disable-gprofng, which fails to build
against musl and is the reason why our cross/binutils-* packages
generated from binutils 2.39.x didn't build anymore.
Related: https://git.alpinelinux.org/aports/tree/main/binutils/APKBUILD
Related: https://sourceware.org/bugzilla/show_bug.cgi?id=29477
Even add_cmd has exited with code 99, it will be overwritten by clean_cmd
later. Exit with code returned by add_cmd to raise a runtime error when
an error occurs.
Newbyte
Oliver Smith
Luca Weiss
Petr Hodina
Clayton Craft
Jami Kettunen
Pablo Correa Gómez
Jane Rachinger
Weijia Wang
Sicelo
vaino
Caleb Connolly
Alexander Martinz
Minecrell
Caleb Connolly
David Wronek
Song Fuchang
Raymond Hackley
Salvatore Stella