The apkbuild parser could not handle cases where a line ends in a
comment but the value is not quoted.
E.g. this:
pkgver=1.0 # I'm a comment, look at me
was being parsed to a value like this:
1.0 # I'm a comment, look at me
... which is obviously wrong. This strips off any trailing comment on
the line, so it's parsed to the correct value.
fixes#2087
Replace "args.cache" with a global variable in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
So I can finally run `pmbootstrap install --password 147147` and go and
make a cup of tea.
Based on MR 1919.
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
Support full linux-* package names in argument completion for
"pmbootstrap kconfig ..." command-lines and get rid of related PROTIP
messages:
PROTIP: You can simply do 'pmbootstrap kconfig check postmarketos-allwinner'
This improves consistency, as in other places we expect the user to
supply full package names as well (e.g. pmbootstrap build).
When running pmbootstrap on debian bullseye with the distro's
python3-argcomplete 1.8 from 2017, tab completion was broken. After
disabling stderr redirect to /dev/null, the error appeared:
TypeError: package_completer() missing 1 required positional argument: 'parser'
Support this ancient version of argcomplete too by setting None as
default value for parser (we don't use it anyway).
Related: https://wiki.postmarketos.org/wiki/Pmbootstrap_development_guide#Debugging_tab_completion_.28argparse.29
iwd seems like a promising alternative to wpa_supplicant. It uses crypto
implementations from the kernel, so let's make kconfig check aware of
the options it needs.
Extend the kconfig check code to not only support booleans and arrays,
but also strings. This will be used for CONFIG_LSM with apparmor where
it's important that "apparmor" has a certain spot in the list.
Replace "args.arch_native" with the direct function call in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
With this option you can run
$ pmbootstrap kconfig migrate --arch <arch> linux-postmarketos-xxx-xxx
to perform safe kconfig upgrades between kernel releases.
"make oldconfig" will ask question for every new/renamed kconfig option,
so you have no chance to miss anything.
Some Mediatek recovery images have headers named
"RECOVERY". Accept those too.
Signed-off-by: Boris Lysov <arzamas-16@mail.ee>
Co-Authored-By: Alexey Min <alexeymin@postmarketos.org>
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
Move all code that verifies the labels of the kernel and ramdisk inside
the boot.img file into a separate function, so it is easier to extend it
to allow recovery images too.
Fix this weird help output:
-mp URL, --mirror-pmOS URL
postmarketOS mirror, disable with: -mp='', specify
multiple with: -mp='one' -mp='two', default: h, t, t,
p, :, /, /, m, i, r, r, o, r, ., p, o, s, t, m, a, r,
k, e, t, o, s, ., o, r, g, /, p, o, s, t, m, a, r, k,
e, t, o, s, /
The option, --no-firewall, will disable nftables on boot in the image,
and print a warning message if it's being disabled in a device image
where the device's kernel should support running the firewall.
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
Adjust to mkbootimg-osm0sis 2021.04.27, where the output files have been
renamed:
"use correct output names matching mkbootimg args
(zImage=Image.gz=kernel, ramdisk.gz=ramdisk)"
Related: 5a01ae54a9
This can be used for example to sideload packages to
pmbootstrap's QEMU which is running on the port 2222
by default, as follows:
pmbootstrap sideload --host localhost --port 2222 --user user <pkg>
This adds a `--port` parameter to sideload subcommand.
If not specified, port defaults to 22.
Fix issues occurring when using pmbootstrap qemu with proprietary
Nvidia drivers as well as mouse misalignment issues on Phosh UI.
Signed-off-by: Mark Hargreaves <clashclanacc2602@gmail.coM>
Unmaintained devices are device packages that:
- Are known to be broken in some way without an active maintainer
who can investigate how to fix it, or
- Have not received any updates for a very long time, or
- Are discouraged from using because they are just intended for testing.
An example for this are ports using the downstream kernel for devices
which have a mainline port that is working quite well.
Unmaintained devices are still built by bpo (otherwise it would not make
sense to keep them), but they do not show up in "pmbootstrap init".
However, it is possible to manually select them by entering the name.
pmbootstrap will warn in that case.
Unmaintained packages should have a # Unmaintained: <reason> comment
in the APKBUILD, this comment is displayed in "pmbootstrap init"
so that the user knows why the device should not be used unless they
know what they are doing.
Made changes to limit the line length in following files,
- pmb/parse/bootimg.py
- pmb/parse/depends.py
- pmb/parse/kconfig.py
- test/test_parse_depends.py
Added the above files in E501 flake8 command list.
Substitute f-string for string concatenation.
Made changes to limit the line length in following files,
- pmb/parse/_apkbuild.py
- pmb/parse/apkindex.py
- pmb/parse/binfmt_info.py
- pmb/parse/deviceinfo.py
- test/test_parse_apkbuild.py
Added the above files in E501 flake8 command list.
Substitute f-string for string concatenation.
The sideload command runs the supplied names through the pmbootstrap
buildsystem to make sure they're up-to-date, then uses scp from the host
to copy the built apks to /tmp on the phone and installs them through
ssh.
If the --install-key option is set then it will also copy over the apk
key that's used for signing the packages built by pmbootstrap in case
the postmarketOS install on the device isn't build by the same machine
as you're sideloading from.
Move the numerous "install" arguments into an own function (as it was
done with actions added later). Categorize the options and update the
help output, so the options are easier to understand.
This adds a new commandline flag -E / --extra-space for
specifying the amount of additional space to be added to
the image size to work around cases where the automatically
determined size turns out to not actually be enough.
The value is also asked for in the "Additional options"
section of the interactive mode.
Fixes: #1904
Alpine indicates with arch="", that a package should temporarily not be
built for any architecture. Support this in postmarketOS too by not
complaining in the APKBUILD parser if arch is empty.
Adjust pmb.build.autodetect.arch and pmb.build.menuconfig.get_arch, so
both don't fail with an IndexError when encountering a disabled package.
Co-Authored-By: Luca Weiss <luca@z3ntu.xyz>
This adds support for specifying an arbitrary git ref (e.g. commits,
tags, branches) to upgrade to. This can be useful if a specific commit
needs to be packaged instead of the latest available. Alternatively you
can also specify a branch to be used if the default branch is 'stable'
but 'develop' should be packaged.
This also removes old code to use the 'bionic' branch for UBports Lomiri
(formerly Unity 8) packages.
Some Mediatek devices have a special 512-byte header around the zImage
which must be generated so the device boots.
Support for that exists for a while in postmarketOS but detection was
missing. Add that.
Let UI meta-packages specify apps in "pmb_recommends" to be explicitly
installed by default, and not implicitly as dependency of the UI
meta-package ("depends"). Therefore make these apps uninstallable,
without removing the meta-package.
Add pmbootstrap install --no-recommends to disable this feature.
Add initial support for the on-device installer in pmbootstrap. Let
pmbootstrap create a regular split image, then prepare a new installer
rootfs and copy the previously generated rootfs image into the installer
rootfs. Put the installer rootfs into a new image, with reserved space.
There is more to do from here, such as disabling the generation of the
user account when using --ondev. But this requires support in
postmarketos-ondev first, so let's build that iteratively.
Related: https://wiki.postmarketos.org/wiki/On-device_installer
Related: https://gitlab.com/postmarketOS/postmarketos-ondev/-/issues
Alpine APKBUILDs have the concept of "provider priorities" that affect
the choice of the provider to install when multiple packages provide
a virtual package.
One use case for this is to allow installation of different firmware
versions. bq-paella can run unsigned firmware, therefore you have the
choice between using the original firmware from the manufacturer, or
a slightly newer version from Qualcomm for the Dragonboard 410c.
We add provides="firmware-qcom-msm8916-wcnss" (the "virtual package")
to both firmware-qcom-db410c-wcnss and firmware-bq-picmt-wcnss.
At this point, attempting to install "firmware-qcom-msm8916-wcnss"
would still fail with apk. (Because it does not know which provider
to install.)
To pick a default we can set e.g. provider_priority=100 for
firmware-qcom-db410c-wcnss (the slightly newer version).
In that case, firmware-qcom-db410c-wcnss should be installed by default.
However, the user can choose to do "apk add firmware-bq-picmt-wcnss"
to override the default choice in case of problems. In that case,
the conflicting firmware-qcom-db410c-wcnss will be automatically removed.
At the moment, pmbootstrap does not respect the "provider_priority" at all.
In the above case, it would always install "firmware-bq-picmt-wcnss"
during "pmbootstrap install" since that has the shortest name.
Extend the pmbootstrap code to pick a provider with the highest priority
(if any of the providers has a priority set).
Use mirrordir_pmos and mirrordir_alpine from channels.cfg to generate
the mirror URLs for postmarketOS and Alpine, which get written to
/etc/apk/repositories and which postmarketOS uses to download the
APKINDEX files.
Remove hardcoded "master" at the end of the postmarketOS mirror and use
mirrordir_pmos instead (which is "master" for the edge channel). Let the
postmarketOS mirror end in a '/' for consistency with the Alpine mirror
in pmb/config/__init__.py.
Remove obsolete --alpine-version. To experiment with a different Alpine
version, one should pass a custom --config-channels from now on.
Prepare to base postmarketOS on Alpine stable by parsing the new
channels.cfg file in pmaports.git, that describes which channel
needs which branches and mirror dirs from postmarketOS and Alpine.
Use the information in pmb.helpers.git.get_branches_official() first,
more is coming in follow-up commits.
Read the file from origin/master, so we get the latest fetched version
even if the last checked out master branch is not up-to-date (think of
currently checked out release branch instead of master, master will
never be updated to point to latest origin/master). Allow to override
the file with a new --config-channels parameter.
Related: https://postmarketos.org/channels.cfg
Use pmb.helpers.args.add_cache() with the args used during
autocompletion, so pmb.helpers.pmaports._find_apkbuilds() does not fail
when trying to access args.cache.
Flashes device vbmeta partition (can be overriden with
"flash_fastboot_partition_vbmeta" setting in deviceinfo)
with custom vbmeta.img which has verity flag disabled,
so device can boot postmarketOS with no problems.
In the future, device ports will be located in a subdirectory
below device/... (e.g. device/testing/device-...).
Replace all occurrences of device/* with a glob that checks the
subdirectories instead.
Note: To ensure that this always works properly we should also add some
checks that all devices are indeed located under one of the supported
subdirectories (i.e. testing/community/main).
Change the glob for pmaports to <aports>/**/APKBUILD.
This allows using subdirectories for organization outside of device/
as well.
For KVM the code is run pretty much natively on the host CPU, so all
CPU extensions available on the host CPU can be also used inside the VM.
To expose that information to the VM we should pass "-cpu host", so the
VM is aware of which CPU is in use.
For CPU emulation, QEMU uses a rather minimal CPU on x86_64 by default.
It does not have support for SSE3/4 etc, which may be required for some
applications to work properly (e.g. Android in Anbox). Add a --cpu flag
to make the emulated CPU configurable. Useful values are for example
--cpu max to emulate all implemented CPU features.
To test QEMU's CPU emulation it is useful to have a switch to disable
KVM, even when it is available (and potentially working fine).
Add --no-kvm for that purpose.
For some reason, the SDL display backend changes the video resolution
to 1024x768, while the GTK display keeps it at 640x480.
This is annoying, because at the moment we can only set one display
resolution for a device in postmarketOS (e.g. for the splash screen).
At the moment, the resolution for the splash screen is set to 640x480,
which therefore shows up too small with the default SDL display.
It seems like the display resolution can be only changed in the guest
directly. Linux has a video= kernel parameter that can be used to
implement this. (See: https://www.kernel.org/doc/html/latest/fb/modedb.html)
Let's set 1024x768 by default, but make it configurable through --video.
The QEMU 'tablet' input device reports absolute positions instead
of relative mouse pointer movements. This can be used to automatically
grab/release the mouse pointer when entering/leaving the QEMU window,
instead of having to release it with CTRL + ALT + G manually.
This is quite convenient and should be the default option normally,
but at least on my PC the mouse pointer is reported with some vertical
offset for some reason (you can't reach the top and it extends below
the QEMU window...). Let's add it as an optional --tablet option for now.
When using pmbootstrap, you usually select the device you want to work
on using 'pmbootstrap init', generate the rootfs and can then run more
commands in the context of the device.
The same needs to be done before using QEMU (to generate the rootfs).
But for some reason 'pmbootstrap qemu' requires setting the --arch
parameter when running QEMU for a foreign architecture, even when the
device is still selected in pmbootstrap.
Even more confusing is that setting "--arch arm" always selects
device-qemu-vexpress, but this is not immediately clear from the name.
Let's make this a lot more intuitive by making sure there is a QEMU
device selected when running 'pmbootstrap qemu'. We can then use the
device information to infer the architecture automatically.
At the moment, the --display argument is a bit complicated to use.
A common use would be to switch between the UIs (sdl, gtk, none)
or to enable the software rasterizer. Split the two use cases
to separate arguments to make it more intuitive.
mesa-dri-swrast and mesa-dri-virtio are both provided by mesa-dri-gallium
now, so this option does not have much use anymore. With both selections,
exactly the same packages are installed.
The SPICE UI option tends to be broken (see #1836), and even when it is
working, it is not working particularly well. QXL requires special handling
in our QEMU packages, when now virtio-gpu (virgl) is working quite well overall.
Apparently it is possible to use virgl with SPICE; but only when using
a Unix socket instead of a TCP port. That again is a bit complicated
because we run QEMU outside the chroot and the SPICE client within.
Overall it does no longer seem to be worth the effort.
The default QEMU UI is working just fine (for the purposes of testing
postmarketOS at least).
Add new "pmbootstrap status" command, which does a quick health check
for the work dir. As first health check, verify that the chroots are not
too old. Replace the reminder text at the end of "pmbootstrap init" to
tell users to run "pmbootstrap status" instead of "pmbootstrap zap" once
a day before working with pmbootstrap.
Related: #1829
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
At the moment we have a simple subpkgdesc() function that can only
parse "pkgdesc" from subpackages, without support for any variables.
But we have a quite nice variable parser now that can be extended
to work for subpackages.
Simply put this works by:
- Finding the lines that belong to the subpackage function
- Stripping indentation (tab)
- Parsing relevant attributes similar to the apkbuild() function
The "subpackages" in the parsed APKBUILD are replaced by a dict
of subpkgname: {"pkgdesc": "...", "depends": "..."} which are
parsed from the subpackage function (if found).
This makes it possible to get the "depends" of a subpackage.
Clayton Craft
BO41
Caleb Connolly
Ultracoolguy
Oliver Smith
Minecrell
Alexey Min
Bart Ribbers
Newbyte
Boris Lysov
afeuerstein
Anri Dellal
Mark Hargreaves
Bobby The Builder
JuniorJPDJ
Shubham Naik
Martijn Braam
Johannes Marbach
timbz
yarl
Luca Weiss
Anjandev Momi
Antoine Fontaine
Daniele Debernardi
Niklas Cathor