When looking at the table for possible output modes, it only makes sense
to pass stdin to "interactive" and "tui". The output mode "stdout" is
for non-interactive commands.
This fixes apk going interactive (asking for confirmation) when running
pmbootstrap with --details-to-stdout and building a package that depends
on postmarketos-base.
Fixes: issue 2208
Tested-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230301204112.4351-1-ollieparanoid@postmarketos.org%3E
Instead of letting the mount -t binfmt_misc … command fail, and simply
telling the user that the command failed: rely on the following check
that prints a better error message on error. If it fails at this point,
the kernel option isn't enabled. Add two comments explaining why
check=False is used in this function while at it.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230123064516.1607-1-ollieparanoid@postmarketos.org%3E
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.
$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all
pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.
Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
When running the testsuite, most logging gets written to a separate
log_testsuite.txt file. Check if it exists, and if so, instruct tail to
print its output as well. This allows immediatelly figuring out what the
testsuite is doing without manually attaching to log_testsuite.txt
(which I often did while running the testsuite).
Add the new option that will be mandatory for all devices in
community/main category. This is just a combination of anbox + iwd +
nftables + containers + zram + netboot.
While the existing options could be removed we're keeping it for now
given that also some devices with downstream kernel might find some
options useful.
Don't pass stdin to commands that aren't supposed to be used
interactively (output: log, background, pipe).
This fixes an inconsistency between building packages in CI on gitlab
and building them via bpo on sourcehut or locally. In gitlab, apparently
there is no stdin for the entire build job and so unanswered kernel
config prompts will just use the default. In local builds and on
sourcehut stdin is available and so it just hangs at the prompt until
pmbootstrap kills the build job due to no output being written.
I considered adding an additional check to pmaports to ensure that there
are no unanswered kernel config prompts just in case users run abuild
manually on the kernel APKBUILD with stdin available. But I think
forcing the users to answer all the prompts even if it's not really
needed just creates additional work / makes the workflow worse without
real benefit.
Related: https://builds.sr.ht/~postmarketos/job/824373#task-pmbootstrap_build-432
Fixes: pmaports issue 1225
Stock bootloader on these devices boots kernel (it is intended to boot
kernel, but i place secondary bootloader there) from special ChromeOS
kernel partition on special GPT which is created with cgpt utility.
This MR adds initial support for it introducing new deviceinfo options:
- cgpt_kpart - path to file to be flashed to ChromeOS partition;
- cgpt_kpart_start - offset from the start in sectors;
- cgpt_kpart_size - partition size in sectors.
For example:
deviceinfo_cgpt_kpart="/usr/share/u-boot/google-peach-pit/u-boot-dtb.img.kpart"
deviceinfo_cgpt_kpart_start="8192"
deviceinfo_cgpt_kpart_size="32768"
cgpt requires start and size values of partition, so these values
are calculated for each partition.
Reserved size and on-device installer are not yet supported.
Reference: https://archlinuxarm.org/platforms/armv7/samsung/samsung-chromebook
pmbootstrap netboot command exposes the generated vendor-codename.img
rootfs through nbd interface so that device can mount it and boot
postmarketOS without having any storage medium at all.
Co-authored-by: Luca Weiss <luca@z3ntu.xyz>
Replace "args.cache" with a global variable in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
The provider selection for "pmbootstrap init" added in this commit
is a flexible way to offer UI/device-specific configuration options
in "pmbootstrap init", without hardcoding them in pmbootstrap.
Instead, the options are defined entirely in pmaports using APK's
virtual package provider mechanism. The code in pmbootstrap searches
for available providers and displays them together with their pkgdesc.
There are many possible use cases for this but I have tested two so far:
1. Selecting root provider (sudo vs doas). This can be defined entirely
in postmarketos-base, without having to handle this specifically in
pmbootstrap.
$ pmbootstrap init
[...]
Available providers for postmarketos-root (2):
* sudo: Use sudo to run root commands (**default**)
* doas: Use doas (minimal replacement for sudo) to run root commands
(Note: Does not support all functionality of sudo)
Provider [default]: doas
2. Device-specific options. My main motivation for working on this
feature is a new configuration option for the MSM8916-based devices.
It allows more control about which firmware to enable:
$ pmbootstrap init
[...]
Available providers for soc-qcom-msm8916-rproc (3):
* all: Enable all remote processors (audio goes through modem) (default)
* no-modem: Disable only modem (audio bypasses modem, ~80 MiB more RAM)
* none: Disable all remote processors (no WiFi/BT/modem, ~90 MiB more RAM)
Provider [default]: no-modem
The configuration prompts show up dynamically by defining
_pmb_select="<virtual packages>" in postmarketos-base, a UI PKGBUILD
or the device APKBUILD. Selecting "default" (just pressing enter)
means that no provider is selected. This allows APK to choose it
automatically based on the "provider_priority". It also provides
compatibility with existing installation; APK will just choose the
default provider when upgrading. The selection can still be changed
after installation by installing another provider using "apk".
Note that at the end this is just a more convenient interface for the
already existing "extra packages" prompt. When using pmbootstrap in
automated scripts the providers (e.g. "postmarketos-root-doas") can be
simply selected through the existing "extra_packages" option.
At the moment, "provides" are only checked in the root package and not
in subpackages of APKBUILDs. Fix this by looking through the subpackages
as well.
At the moment we have to parse all APKBUILDs to find subpackages,
even if they are guessed easily shortly after. To speed this up,
let's guess first but verify the guess by only parsing that particular
APKBUILD. If the subpackage/provides is in there we seem to have found it.
iwd seems like a promising alternative to wpa_supplicant. It uses crypto
implementations from the kernel, so let's make kconfig check aware of
the options it needs.
Replace "args.arch_native" with the direct function call in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
Replace "args.logfd" with "pmb.helpers.logging.logfd" in order to avoid
passing "args" to all functions that only use it to write to logfd. This
is the first step to get rid of this args-passed-to-all-functions
pattern in pmbootstrap.
With this option you can run
$ pmbootstrap kconfig migrate --arch <arch> linux-postmarketos-xxx-xxx
to perform safe kconfig upgrades between kernel releases.
"make oldconfig" will ask question for every new/renamed kconfig option,
so you have no chance to miss anything.
Before this commit, package folders were copied into the chroot one by
one in order to run apkbuild-lint on them. This logic is replaced by
mounting pmaports.git into the chroot and using a single apkbuild-lint
invocation to lint the supplied packages.
Both of these changes result in a performance improvement, especially
when linting multiple packages at once.
Before this change:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m5,261s
user 0m7,046s
sys 0m1,842s
Using the pmaports.git mount but calling apkbuild-lint in a loop:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m4,089s
user 0m6,418s
sys 0m1,219s
After this change:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m3,518s
user 0m5,968s
sys 0m0,959s
Additionally, running apkbuild-lint from the pmaports.git mount point
has the benefit that every printed violation contains a nice source
identifier à la "./cross/grub-x86/APKBUILD". This makes it possible to
differentiate between different packages even though only a single
apkbuild-lint invocation is used.
Relates: postmarketOS/pmaports#564
kernel is named /boot/vmlinuz now, looking at the filename will no
longer tell us what flavor it is. This now will look at
/usr/share/kernel, which has always contained the kernel 'flavor', and
since we currently only install 1 kernel these days, guarding this with
pmaports.cfg should be unnecessary. In the worst case (if there are
multiple kernel 'flavors' installed), it'll just grab the first one and
return it.