Replace "args.logfd" with "pmb.helpers.logging.logfd" in order to avoid
passing "args" to all functions that only use it to write to logfd. This
is the first step to get rid of this args-passed-to-all-functions
pattern in pmbootstrap.
With this option you can run
$ pmbootstrap kconfig migrate --arch <arch> linux-postmarketos-xxx-xxx
to perform safe kconfig upgrades between kernel releases.
"make oldconfig" will ask question for every new/renamed kconfig option,
so you have no chance to miss anything.
Before this commit, package folders were copied into the chroot one by
one in order to run apkbuild-lint on them. This logic is replaced by
mounting pmaports.git into the chroot and using a single apkbuild-lint
invocation to lint the supplied packages.
Both of these changes result in a performance improvement, especially
when linting multiple packages at once.
Before this change:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m5,261s
user 0m7,046s
sys 0m1,842s
Using the pmaports.git mount but calling apkbuild-lint in a loop:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m4,089s
user 0m6,418s
sys 0m1,219s
After this change:
$ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
> /dev/null
real 0m3,518s
user 0m5,968s
sys 0m0,959s
Additionally, running apkbuild-lint from the pmaports.git mount point
has the benefit that every printed violation contains a nice source
identifier à la "./cross/grub-x86/APKBUILD". This makes it possible to
differentiate between different packages even though only a single
apkbuild-lint invocation is used.
Relates: postmarketOS/pmaports#564
kernel is named /boot/vmlinuz now, looking at the filename will no
longer tell us what flavor it is. This now will look at
/usr/share/kernel, which has always contained the kernel 'flavor', and
since we currently only install 1 kernel these days, guarding this with
pmaports.cfg should be unnecessary. In the worst case (if there are
multiple kernel 'flavors' installed), it'll just grab the first one and
return it.
This can be used for example to sideload packages to
pmbootstrap's QEMU which is running on the port 2222
by default, as follows:
pmbootstrap sideload --host localhost --port 2222 --user user <pkg>
This adds a `--port` parameter to sideload subcommand.
If not specified, port defaults to 22.
Do not attempt to install with a filesystem that is not supported by the
initramfs code in the checked out pmaports branch.
Previously we would have increased the pmaports.cfg version and require
that new version by pmbootstrap, however this will break compatibility
with release branches where we won't roll out this feature (v20.05).
Therefore don't change the version, but add a new
"supported_root_filesystems" key to pmaports.cfg, which defaults to
"ext4".
Related: https://postmarketos.org/pmaports.cfg
The sideload command runs the supplied names through the pmbootstrap
buildsystem to make sure they're up-to-date, then uses scp from the host
to copy the built apks to /tmp on the phone and installs them through
ssh.
If the --install-key option is set then it will also copy over the apk
key that's used for signing the packages built by pmbootstrap in case
the postmarketOS install on the device isn't build by the same machine
as you're sideloading from.
Let tail attempt to open the file again, if it becomes inaccessible.
This is useful, when writing a reproducer that deletes pmbootstrap's
log.txt while at the same time running 'pmbootstrap log'.
(027724) [17:57:34] Done
tail: '/home/user/.local/var/pmbootstrap/log.txt' has become inaccessible: No such file or directory
tail: '/home/user/.local/var/pmbootstrap/log.txt' has appeared; following new file
(003493) [17:57:35] % cd /home/user/.local/var/pmbootstrap/cache_git/pmaports; git remote -v
This is likely to fail with the new default cryptsetup cipher of
aes-xts-plain64, as many downstream kernels used in recovery OS (like
TWRP) do not have CRYPTO_XTS set.
Add initial support for the on-device installer in pmbootstrap. Let
pmbootstrap create a regular split image, then prepare a new installer
rootfs and copy the previously generated rootfs image into the installer
rootfs. Put the installer rootfs into a new image, with reserved space.
There is more to do from here, such as disabling the generation of the
user account when using --ondev. But this requires support in
postmarketos-ondev first, so let's build that iteratively.
Related: https://wiki.postmarketos.org/wiki/On-device_installer
Related: https://gitlab.com/postmarketOS/postmarketos-ondev/-/issues
Prepare for a future patch, that adds reserved space in MiB, by changing
size_boot and size_root from bytes to MB everywhere. This is what we need
most of the time and allows to drop some /1024**2 statements.
Previously these two commands would both print the current value:
pmbootstrap config extra_packages
pmbootstrap config extra_packages ''
With this change, the second command will instead set the given config
value to the empty string.
The kconfig check searches the aport with the "linux-" prefix to the
package name passed as argument. This is not working with the full
package name like linux-device-name because it searches a
linux-linux-device-name and fails.
In the future, device ports will be located in a subdirectory
below device/... (e.g. device/testing/device-...).
Replace all occurrences of device/* with a glob that checks the
subdirectories instead.
Note: To ensure that this always works properly we should also add some
checks that all devices are indeed located under one of the supported
subdirectories (i.e. testing/community/main).
Change the glob for pmaports to <aports>/**/APKBUILD.
This allows using subdirectories for organization outside of device/
as well.
Add new "pmbootstrap status" command, which does a quick health check
for the work dir. As first health check, verify that the chroots are not
too old. Replace the reminder text at the end of "pmbootstrap init" to
tell users to run "pmbootstrap status" instead of "pmbootstrap zap" once
a day before working with pmbootstrap.
Related: #1829
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
The gist of this action is upgrading the specified aport to the latest
version. There are implementations for both stable packages (which check
via the release-monitoring.org API for new versions) and git packages
(which check the GitLab/GitHub API for new commits on the main branch).
There's also the possibility to pass --all, --all-stable & --all-git to
the action which either loops through all packages, or just stable or
git packages and upgrades them.
The --dry argument is also respected.
Note, that the implementation does update the variables pkgver, pkgrel
and _commit but it doesn't update the checksums because that would slow
down the process a lot, and is potentially undesirable.
asus-me176c has a Fastboot interface that can be used for flashing,
but in postmarketOS we do not use Android boot images for it.
This is because is it not very practical - the boot partition is
quite small and there is a (custom) EFI bootloader that can boot
directly from any other FAT32 partition.
At the moment the installation process is manual:
1. pmbootstrap install --split to have separated boot (FAT32)
and rootfs images
2. pmbootstrap export
3. Flash boot and rootfs images manually using Fastboot
The "fastboot-bootpart" flasher implements that process in a more
convenient way. When a device uses the "fastboot-bootpart" flasher:
- We generate --split images on "pmbootstrap install" by default.
(This can be disabled using --no-split instead.)
- pmbootstrap flasher flash_kernel flashes the raw boot partition
(not an Android boot image) using Fastboot, just like the rootfs.
There are some limitations that could be improved in the future:
- "fastboot-bootpart" is not offered in the device wizard.
I think it is special enough that no-one will be starting with it,
and the difference to normal "fastboot" might be confusing.
- Support "pmbootstrap flasher boot". asus-me176c does not support
"fastboot boot" properly, but theoretically we could still generate
Android boot images to use when booting an image directly.
- At the moment the boot partition image is not regenerated when
using "pmbootstrap flasher flash_kernel" (unlike when using Android
boot images). "pmbootstrap install" needs to be run manually first.
At the moment, pmbootstrap updates the kernel and the initfs whenever
using the flasher or export. This is useful, but sometimes you just want
to boot exactly the same kernel several times. In that case, having to wait
several seconds for the (redundant) update to complete is quite annoying.
Add a --no-install option that allows skipping the kernel/initfs update.
Add a shortcut for "git pull --ff-only" in all repositories cloned by
pmbootstrap (currently pmaports and aports_upstream, new pmdevices
repository coming soon).
'pmbootstrap pull' will only update the repositories, if:
* they are on an officially supported branch (e.g. master)
* the history is not conflicting (fast-forward is possible)
* the git workdirs are clean
Otherwise it shows the user a descriptive message about what to do. The
list of supported branches is only "master" right now, and will be
extended in later commits, so we can have a stable branch for pmaports
based on Alpine's releases. More about that in the project direction
2020 issue.
Closes: #1858
Rust packaging is new and still a bit weird in Alpine and postmarketOS.
As of writing, we only have one package (squeekboard), and use cargo to
download the source of all dependencies at build time (several git
repositories!) and compile it. Usually, this is a no-go, but at least
until this is resolved properly, let's cache the downloads as suggested
in: https://doc.rust-lang.org/cargo/guide/cargo-home.html
Related: #1861
This allows for example for me to call the kconfig check function on the
.config file in my Linux tree: $ pmbootstrap kconfig check --file
.config and it reports me which kconfig options I need to enable.
Download all sources and verify their checksums.
This will be used in pmaports.git CI, if ci:skip-build is set in the
commit message (currently it just skips the build, and we don't test if
the source checksums are valid or not).
Ignore APKBUILDs that have "!pmb:kconfigcheck" in their options by
default in "pmbootstrap kconfig check", but print a note that they have
been skipped. Check all kernels with "pmbootstrap kconfig check -f".
This is necessary, because the Librem 5 devboard kernel's config does
not have CONFIG_DM_CRYPT enabled in their config, and we check for
that. As the device is still under heavy development, we will make our
lives easier by just using the upstream kernel config without any
changes and ignoring it in our check by default.
BO41
Alexey Min
Johannes Marbach
Clayton Craft
Caio Fontes
Oliver Smith
Bobby The Builder
Martijn Braam
HenriDellal
timbz
Eyal Sawady
Mark Hargreaves
Antoine Fontaine
Niklas Cathor
Minecrell
Daniele Debernardi
Luca Weiss
Danct12
Idan Horo
Oliver Smith