Commit Graph

1270 Commits

Author SHA1 Message Date
Oliver Smith 147b3ce4b9
pmb ci: fix typo in URL
Fix a typo in the URL https://postmarketos.org/pmb-ci that is being used
to detect if a script is compatible with 'pmbootstrap ci' or not.
2022-11-20 16:38:37 +01:00
Luca Weiss 0f6c6238f9
pmb.qemu.run: stop forcing bios for riscv64
We decided now not to force the bios firmware being used for riscv64 but
instead just use the one built into QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221118203424.106861-1-luca@z3ntu.xyz%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 315621d5b8
pmb ci: add --fast argument
Make it easy to only run the fast tests.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-3-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 3fd22104a8
pmb ci: error on using --all with script names
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 9a84ad20b1
pmb ci: fix arg desc for --all
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-1-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:05 +01:00
vaino ed7b0273f5
flasher: heimdall: take depends from pmaports.cfg
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Co-developed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221114115705.1107886-1-vaino@vke.fi%3E
2022-11-16 09:01:32 +01:00
Caleb Connolly ab0aa7f956
pmb: sideload: wait for apk database lock
When installing the APK wait for a while for the APK database lock,
services like apk-polkit-server sometimes hog it for a while but there's
no point bailing out immediately.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111212251.1360612-1-kc@postmarketos.org%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 4a6c5657d5
pmb.parse.kconfig: don't enforce non-core checks for testing devices
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.

Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.

As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 381a1ca907
pmb.parse.kconfig: add wireguard, filesystems & more to community check
Add wireguard options, supported file systems and some extra options to
the community kconfig check.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-1-luca@z3ntu.xyz%3E
2022-11-12 14:28:40 +01:00
Oliver Smith 6f45a5d5fb
Prepare 1.49.0 release 2022-11-09 09:19:33 +01:00
Oliver Smith 30055c14d2
flasher: fastboot: take depends from pmaports.cfg
Adjust to avbtool now being part of android-tools in alpine edge.
Instead of trying to install both (which fails on edge), take the
dependencies from a new pmaports.cfg variable
supported_fastboot_depends, which only contains android-tools in
pmaports.git master branch.

Related: https://postmarketos.org/pmaports.cfg
2022-11-09 09:19:33 +01:00
Oliver Smith b64641bb1c
pmb.flasher.init.install_depends: new function
Move logic to install depends into one shared function.
2022-11-09 09:19:32 +01:00
Oliver Smith 04f8f59208
pmb.config.required_programs: add tar
Add tar as it is required for 'pmbootstrap ci' to get the source into
the pmbootstrap chroot.

Reviewed-by: Caleb Connolly <kc@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-2-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:32 +01:00
Oliver Smith f8d186e776
pmbootstrap ci: fix error with deleted files
Check if each file listed by the two git ls-files commands still exists,
and only add the existing ones to the tarball. Otherwise it would crash
when a file has been deleted without having the change commited yet.

Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-1-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:26 +01:00
Oliver Smith e8b0b4ba78
aportgen: don't fail if binary ver < APKBUILD ver
The typical workflow for upgrading cross/gcc-* is:

$ cd pmaports/cross
$ pmbootstrap aportgen gcc-*

Currently this is failing because the APKBUILD has been updated in
Alpine for gcc, but gcc has not been built for all arches yet. This
shouldn't prevent us from generating the proper updated APKBUILDs in
pmaports so just print a note and don't fail here.

I'm committing this directly to master as this currently breaks
test/test_aportgen.py::test_aportgen.
2022-11-02 21:09:28 +01:00
Luca Weiss 4771fbac65
pmb.qemu.run: support riscv64
We're using the standard virt machine, adding virtio-gpu-pci for
graphics and are using the bios firmware that is installed from the
device package, instead of the one built-in to QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-2-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Luca Weiss 0624a1ae33
pmb.qemu.run: replace -nic option with -netdev and -device
On qemu-system-riscv64 the -nic option doesn't seem to work correctly.

  qemu-system-riscv64: warning: requested NIC (anonymous, model virtio-net-pci) was not created (not supported by this machine?)

Using -netdev and -device provides the same functionality and also works
on riscv64.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-1-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Alexander Martinz 89c2ff0926
pmb.parse.kconfig: explicitly add dependency on SWAP
Commit d8f2f20186 removed
the requirement of selecting MEMCG_SWAP for Kernels >= 6.1.

However, it did not fully account for the behavior change,
as MEMCG_SWAP depends on both MEMCG and SWAP.

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-25 23:18:47 +02:00
Oliver Smith 0bcd58765c
pmbootstrap ci: new command
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.

$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all

pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.

Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
2022-10-25 20:31:19 +02:00
Oliver Smith 6ac39d17e7
pmb/helpers/frontend: order imports alphabetically 2022-10-25 20:31:19 +02:00
Oliver Smith 4428c7bcdc
pmbootstrap log: show testsuite log too
When running the testsuite, most logging gets written to a separate
log_testsuite.txt file. Check if it exists, and if so, instruct tail to
print its output as well. This allows immediatelly figuring out what the
testsuite is doing without manually attaching to log_testsuite.txt
(which I often did while running the testsuite).
2022-10-25 20:31:14 +02:00
Alexander Martinz 2ca4518799
pmb.parse.kconfig: drop remaining "_rc1" references (MR 2220)
As discussed at: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2220#note_1146631834

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-24 17:36:34 +02:00
Alexander Martinz d8f2f20186
pmb.parse.kconfig: fix container check for kernels >= 6.1 (MR 2220)
Kernel 6.1 dropped CONFIG_MEMCG_SWAP[1].

[1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e55b9f96860f6c6026cff97966a740576285e07b

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-24 11:14:47 +02:00
Minecrell 3b5492d91e
pmb: build: envkernel: Fix bind mount detection (MR 2217)
At the moment the "envkernel.sh hasn't run, assuming the kernel was
cross compiled on host and using current dir as source" code path
triggers even when using envkernel.sh, which works somewhat but
requires sourcing envkernel.sh again after each invocation of
"pmbootstrap build --envkernel ...".

The reason is that os.path.ismount() does not work for bind mounts
(see https://bugs.python.org/issue29707). There is a workaround for
that already in pmbootstrap but it is not used here for some reason.
2022-10-20 11:41:27 +02:00
Luca Weiss dfada2a522
pmb: enable riscv64 architecture (MR 2215)
Enable pmbootstrap to work on riscv64 packages, and at the same time
also expand the alpine_to_hostspec list to match upstream (adding
riscv32 and loongarch*).
2022-10-19 09:11:54 +02:00
Luca Weiss 61ce6069ed
pmb.config: add missing pmb:kconfigcheck-* for lint (MR 2214)
While introducing multiple pmb:kconfigcheck-* APKBUILD options we missed
adding them to this array leading to potential apkbuild-lint failures.

This hasn't been noticed much because the check in apkbuild-lint is
relatively buggy, see https://gitlab.alpinelinux.org/Leo/atools/-/issues/48
2022-10-18 21:10:56 +02:00
Caleb Connolly 66706f896d
pmb.parse.build: mention host compiled kernels in envkenel help (MR 2175) 2022-10-18 20:08:41 +02:00
Caleb Connolly 6502d8aa28
build: envkernel: support packaging kernels that were built on the host (MR 2175)
Add support for packaging a kernel that was compiled outside of
envkernel. The envkernel.sh wrapper is great for someone new to kernel
development, but it makes it difficult to do things like "make
dt_binding_check". This lets you avoid the wrapper and build on your
host machine but still use pmbootstrap to package your kernel for easier
testing.
2022-10-18 20:08:36 +02:00
Oliver Smith d2c39ce5d8
Prepare 1.48.0 release 2022-10-16 18:22:31 +02:00
Oliver Smith f41659d4ae
pmb.build._package: update isl workaround (MR 2213)
The plan was to drop this workaround, once alpine releases a new gcc
version, and we regenerate our cross compilers against that version
_and_ drop options="!tracedeps", as latest pmbootstrap aportgen would
do. However !tracedeps wasn't dropped in the latest upgrade. This was
too easy to miss, I'll add a comment to the gcc-cross APKBUILDs so we
see it next time. Let's just update the workaround for this version.
2022-10-16 18:02:26 +02:00
Oliver Smith 35784a5fd7
pmb.aportgen.device: no depend on mesa-dri-gallium (MR 2210)
Device packages shouldn't depend on it anymore, in order to end up with
a smaller installation for ui=none this package only gets pulled in by
postmarketos-base if mesa is installed.

Related: pmaports MR 3478
2022-10-12 08:24:53 +02:00
Oliver Smith 248fe447bd
pmb.chroot.root: set PYTHONUNBUFFERED=1 (MR 2209)
Set the env var, so python programs running inside pmbootstrap chroot
don't buffer their output and only end up printing everything when they
are done. I've seen this with meson. This is bad for usability because
we don't see output, but also a problem because pmbootstrap kills
commands if they don't print any output for some time (default: 15 min).
2022-10-07 09:02:53 +02:00
Oliver Smith 40679f7126
pmb.chroot.root: order env vars alphabetically (MR 2209) 2022-10-07 09:02:53 +02:00
Oliver Smith aaeff96d23
Prepare 1.47.1 release 2022-10-06 07:59:42 +02:00
Oliver Smith 54268b72e1
pmb.build.package: add workaround for missing isl
Install isl25 for the specific gcc versions we currently have in
pmaports master that are missing it; this is easier than building new
packages and going out of sync with Alpine's versioning. This can be
reverted as soon as there is a newer gcc.

Related: pmaports issue 1732
2022-10-05 20:44:36 +02:00
Oliver Smith 550b9b3466
pmb.aportgen.gcc: remove !tracedeps option
abuild traces shared library dependencies for binaries in built
packages, and adds all of them to depends. This is needed for the cross
gcc builds, as otherwise it will not add the isl dependency (currently
isl25).

With the crossdirect compilation method we mount the cross-gcc from the
native chroot in the foreign arch chroot. Due to the missing dependency,
isl was not installed in the native chroot, and so when attempting to
use that native cross gcc from the foreign arch chroot, it would try to
load the foreign arch isl instead of the native one. Resulting in lots
of the following errors (and of course not a working gcc):

  Error relocating /usr/lib/libisl.so.23: unsupported relocation type 1026

I had probably added '!tracedeps' because abuild complains about it
otherwise... but as noted in [1] these complaints are safe to ignore.

Keep !strip for now even though it's not in Alpine's APKBUILD. I've
tried to build without and the build fails with:
  strip: Unable to recognise the format of the input file `./usr/aarch64-alpine-linux-musl/lib/libgcc_s.so.1'

[1]: 0415ebf6f9
Related: pmaports issue 1732
2022-10-05 20:44:31 +02:00
Oliver Smith dac8b27738
Prepare 1.47.0 release 2022-10-04 09:00:44 +02:00
David Wronek b9e7a98c07
pmb.config.apkbuild_attributes: Add _pkgbase and _pkgsnap (MR 2208)
Parse _pkgbase and _pkgsnap variables in pkgver for gcc.
2022-09-30 22:04:16 +02:00
Oliver Smith b262609199
pmb.aportgen.gcc: add subpkg libstdc++-dev-$arch
Add the subpackage, because g++-$arch depends on it since
12.1.1_git20220630-r6. This fix was already applied to pmaports, thanks
to ungeskriptet! Adjust the aportgen code to make sure this is added in
the future as well.

Related: pmaports MR 3464
Related: f1044b1dae
2022-09-30 22:02:31 +02:00
Oliver Smith 22e0f64095
pmb.aportgen.gcc: set libgcc=false (MR 2203)
Do not build the foreign arch libgcc. We are using the one provided by
Alpine's binary repository instead, and when building it here and trying
to use it, we actually run into a conflict.

I tried setting BOOTSTRAP="nolibc" first since there is already logic to
then disable libgcc in the APKBUILD. But that also disables g++ which we
do want to build, so we can't use this.

Fixes: issue 2168
2022-09-13 23:42:15 -07:00
Oliver Smith 4c7d2459f5
pmb.aportgen.gcc: remove isl from depends (MR 2203)
Remove it from depends, as the package has been removed in Alpine and
since quite some time we shouldn't have depended on it anyway.

Related: issue 2167
2022-09-13 23:42:15 -07:00
Luca Weiss f7fb1fe078
pmb.parse.kconfig: rename anbox check to waydroid (MR 2204)
Change the naming to fit what software is actually used in postmarketOS.
2022-09-13 09:30:23 +02:00
Luca Weiss e468fc518e
pmb.parse.kconfig: add 'community' option (MR 2204)
Add the new option that will be mandatory for all devices in
community/main category. This is just a combination of anbox + iwd +
nftables + containers + zram + netboot.

While the existing options could be removed we're keeping it for now
given that also some devices with downstream kernel might find some
options useful.
2022-09-13 09:30:23 +02:00
Luca Weiss 1fee644dce
pmb.parse.kconfig: fix containers check for x86 (MR 2204)
Options in this list are not supposed to have CONFIG_ prefix, remove it.
2022-09-13 09:30:18 +02:00
Oliver Smith 0c18c664aa
pmb.config.apkbuild_package_attributes: + triggers (MR 2202)
Do not only parse triggers, but also emit them in the packages dicts in
pmb.parse.apkbuild. This is the first half to fix pmaports issue 1690,
the other is in pmaports MR 3386.
2022-09-11 13:49:24 +02:00
Song Fuchang d650ed4a14
pmb.chroot.root: preserve proxy environment variables (MR 2201)
This fixes #457.
2022-09-08 18:11:54 +08:00
Luca Weiss 52530caaf8
pmb.parse.kconfig: remove apparmor check (MR 2200)
Remove the check as apparmor is not the way we want to go for privilege
separation anymore.
See https://gitlab.com/postmarketOS/pmaports/-/issues/1596
2022-09-04 19:10:48 +02:00
Oliver Smith b41e4a418f
aportgen binutils: modernize (MR 2199)
Set CTARGET and CTARGET_ARCH on top of the generated APKBUILD and use
the existing code in the APKBUILD for build(), package(), makedepends,
subpackages. This is now possible, because the pmbootstrap parser for
APKBUILDs has been improved a lot since this was originally written and
because the Alpine APKBUILD does everything that was hacked in here,
like removing man, info directories and bfd-plugins if doing a cross
build.

This is now very similar to bootstrapping binutils for another
architecture via scripts/bootstrap.sh in aports.git.

Most importantly this sets --disable-gprofng, which fails to build
against musl and is the reason why our cross/binutils-* packages
generated from binutils 2.39.x didn't build anymore.

Related: https://git.alpinelinux.org/aports/tree/main/binutils/APKBUILD
Related: https://sourceware.org/bugzilla/show_bug.cgi?id=29477
2022-08-24 08:38:49 +02:00
Raymond Hackley e5d580e98a
pmb: sideload: Handle ssh_install_apks() errors (MR 2195)
Even add_cmd has exited with code 99, it will be overwritten by clean_cmd
later. Exit with code returned by add_cmd to raise a runtime error when
an error occurs.
2022-08-22 09:45:30 +02:00
Salvatore Stella 3567b7c123
Fix wrong output of pmbootstrap bootimg_analyze (MR 2198) 2022-08-21 14:25:11 +02:00
Oliver Smith 21b86f7f64
pmb.helpers.run: only pass stdin where useful (MR 2197)
Don't pass stdin to commands that aren't supposed to be used
interactively (output: log, background, pipe).

This fixes an inconsistency between building packages in CI on gitlab
and building them via bpo on sourcehut or locally. In gitlab, apparently
there is no stdin for the entire build job and so unanswered kernel
config prompts will just use the default. In local builds and on
sourcehut stdin is available and so it just hangs at the prompt until
pmbootstrap kills the build job due to no output being written.

I considered adding an additional check to pmaports to ensure that there
are no unanswered kernel config prompts just in case users run abuild
manually on the kernel APKBUILD with stdin available. But I think
forcing the users to answer all the prompts even if it's not really
needed just creates additional work / makes the workflow worse without
real benefit.

Related: https://builds.sr.ht/~postmarketos/job/824373#task-pmbootstrap_build-432
Fixes: pmaports issue 1225
2022-08-19 10:40:25 +02:00
Oliver Smith c88a18d3f6
treewide: fix various lint errors (MR 2197)
Fix errors found by new flake8 version
2022-08-19 10:40:25 +02:00
Oliver Smith 8d736c5fcc
Prepare 1.46.0 release 2022-08-15 19:32:12 +02:00
Newbyte b9485902cf
pmb.qemu.run: drop removed depedency (MR 2196)
mesa-dri-gallium provides all available drivers in Mesa nowadays.

Closes https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2157
2022-08-11 21:17:20 -04:00
Jami Kettunen 5b3cd7a7a6
pmb.aportgen.linux: Install DTBs on header v2 (MR 2194)
A single DTB is required for "mkbootimg --dtb ..."
2022-07-18 14:49:38 -07:00
Jami Kettunen de890c83e7
pmb.aportgen.device: Generate appropriate header v2 deviceinfo (MR 2194)
Here's an example of the properties this could generate:

  deviceinfo_header_version="2"
  deviceinfo_append_dtb="false"
  deviceinfo_flash_offset_dtb="0x0bc08000"
2022-07-18 14:49:33 -07:00
Jami Kettunen 255e69be5e
pmb.parse.bootimg: Add preliminary support for header v2 (MR 2194)
This includes setting header_version="2" as well as dtb_offset
according to the input boot.img when header v2 is detected.

Also adds the following previously missed deviceinfo_attributes:

* "header_version"
* "bootimg_custom_args"

And fixes failing tests now that header_version is always parsed from
boot.img files.
2022-07-18 14:49:16 -07:00
Oliver Smith e527a159ad
Prepare 1.45.0 release 2022-07-03 17:51:32 +02:00
Luca Weiss 58c39f2cb2
pmb.config.init: fix init when work path is an empty directory (MR 2192)
In case a user removes all contents of the work path then pmbootstrap
init will fail and show

  WARNING: Your work folder version needs to be migrated (from version 0 to 6)!

Later the migration would fail with the error

  ERROR: We have split the aports repository from the pmbootstrap repository (#383). Please run 'pmbootstrap init' again to clone it.

This is due to the existing check not accounting for e.g. log.txt being
written in the work path before we get to this check. Now change it so
we always create the version file if it doesn't exist yet.

Test plan:
$ grep work ~/.config/pmbootstrap.cfg
work = /tmp/pmbootstrap-work
$ rm -rf /tmp/pmbootstrap-work
$ mkdir /tmp/pmbootstrap-work
$ pmbootstrap init

This was previously attempted to be fixed in !1975
2022-06-24 14:57:20 +02:00
Shinjo Park 87f7520e51
pmb/export: Fix Odin export filenames (MR 2191)
Since pmaports@50ee94d8 the kernel filename won't contain the flavor
information, but this is not correctly synchronized with the Odin
export. This fixes Odin export.
2022-06-23 17:45:47 +02:00
Jami Kettunen c0cec06df8
pmb/config: vndbinder ANDROID_BINDER_DEVICE is required for Waydroid (MR 2190)
Fixes 'ERROR: Binder node "vndbinder" for waydroid not found' during
'waydroid init'.
2022-06-15 20:26:05 +03:00
Anton Bambura dc1433ead0
Rename Chrome OS kernel partition to pmOS_kernel (MR 2187)
Make it consistent with pmOS_boot and pmOS_root and
make itmore pmOS-specific to prepare for automatic
kernel upgrades in the future.
2022-06-08 23:16:17 +03:00
Minecrell f1cbcb7b3b
pmb: flasher: frontend: don't fail if kernel config cannot be found (MR 2184)
When using a kernel from Alpine the kernel configuration cannot be
found in pmaports. We cannot check the kernel config for missing
options in that case, but that's no reason to break the flasher
entirely.
2022-06-06 09:52:17 +02:00
Oliver Smith cc90bc81f0
pmb.chroot.apk.install: move pkgname sanitization (MR 2185)
Check if the pkgnames are sane in install_run_apk, right before running
apk. This makes sure that we really run it on all arguments that are
supposed to be packages / files and not options to apk.
2022-05-30 19:09:01 +02:00
Oliver Smith 7b09cc7546
pmb.chroot.apk.install: fix empty install messages (MR 2185)
Previously pmbootstrap would only show the packages that are about to be
installed. In case all packages were already installed, this would lead
to weird empty install messages:

  (rootfs_asus-me176c) install

Show all packages that we want to install, even if they are already
installed in the given chroot.

  (rootfs_asus-me176c) install device-asus-me176c
2022-05-30 19:09:01 +02:00
Oliver Smith 6a74109154
pmb.chroot.apk.install: let apk figure out depends (MR 2185)
Previously to this patch, pmbootstrap would pass a full dependency tree
to "apk add". It would use a virtual package to ensure only the right
packages get added to /etc/apk/world. For example:
  apk add -u --virtual .pmbootstrap postmarketos-base device-asus-me176c \
  postmarketos-ui-sxmo-de-sway device-asus-me176c-nonfree-firmware w3m \
  sfeed clickclack firefox-esr font-noto font-noto-emoji gnome-icon-theme \
  imv megapixels mobile-config-firefox ttyescape postmarketos-base-nofde \
  eudev openssh postmarketos-mkinitfs postmarketos-mvcfg postmarketos-keys \
  ...

Instead of doing that, only pass the packages we want to install and let
apk figure out the dependencies. Most of the time we can even avoid
using the virtual package now.

== Remaining edge case: locally built packages
apk will only upgrade a package with the same pkgver + pkgrel but a
different build date if the full path to an apk file gets passed as
argument. So if the user built a package locally that will be installed,
or one of its dependencies then we still need to use a virtual package
and possibly pass a dependency to apk. Replace
replace_aports_packages_with_path() with packages_get_locally_built_apks()
to get a list of such packages and adjust install() and
install_run_apk() to use it.
2022-05-30 19:09:00 +02:00
Oliver Smith 903ed4ee30
pmb.chroot.apk.packages_split_to_add_del: new func (MR 2185)
Make the code easier to read by moving split_to_add_del() to a separate
function and do some related refactoring. A future patch will use it
twice in install().

Move "arch = ..." to the top of the function while at it, since it's
needed later in the function in 2 places and is not needed for figuring
out packages_with_depends, to_add and to_del.

Remove "# Add depends to packages" because it's obvious from the
packages_with_depends variable name, and getting to_add/to_del is a
different action that stood under the same comment.
2022-05-30 19:09:00 +02:00
Oliver Smith 682ee74ea6
pmb.chroot.apk.install_run_apk: new func (MR 2185)
Split out the part that builds the apk commands and runs them out from
install() to a new function install_run_apk(). This makes install()
easier to read.
2022-05-30 19:09:00 +02:00
Oliver Smith 12948eeb3d
pmb.chroot.apk.install: move empty packages check (MR 2185)
Since the previous commit that adds install_build(), all packages either
end up in to_add or to_del. Move the check for empty packages to the top
of the function, and directly check the packages variable.

I think it's worth keeping this check because it's shorter to add this
check once here than having it a few times in other place where we may
have or may not have something to install. And so we can avoid printing
an empty "install" message with no packages.
2022-05-30 19:09:00 +02:00
Oliver Smith e91dbefd16
pmb.chroot.apk.install_{is_necessary -> build} (MR 2185)
install_is_necessary used to do the following things:

1. Error out if there's no binary package but pmb was invoked as
   "pmbootstrap install" and build_pkgs_on_install is disabled.
2. Build the package if necessary.
3. Return if a package "needs to be installed" (Boolean or Float).

The only caller of the function is pmb.chroot.apk.install. It would not
add the package to the long "apk add" command if according to 3. it does
not need to be installed.

When I implemented this a few years ago, I probably thought it would be
useful to not unnecessarily pass packages to apk. But this actually
makes it more complicated and doesn't have a benefit, apk is perfectly
capable of recognizing which packages it had already installed.

Replace the function with a much simpler pmb.chroot.apk.install_build,
which only does 1. and 2. Change the order of the package, arch
arguments to match called functions pmb.parse.apkindex.package and
pmb.build.package.
2022-05-30 19:09:00 +02:00
Oliver Smith c898b13296
pmb.chroot.apk.install: rename to{add,del} vars (MR 2185)
Put underscores between the words for consistency with other variable
names. Since the whole function is about dealing with packages, remove
the "packages_" prefix to make them shorter.

	packages_toadd -> to_add
	packages_todel -> to_del
2022-05-30 19:09:00 +02:00
Oliver Smith 6b520d2d26
pmb.chroot.apk.install: update func desc (MR 2185) 2022-05-30 19:09:00 +02:00
Oliver Smith 03303ef7bc
Prepare 1.44.1 release
Tag 1.44.0 was done without bumping pmb.config.version, so tag another
one.
2022-05-24 20:16:07 +02:00
Bart Ribbers 741be5f521
Prepare pmbootstrap for pmOS v22.06 release (MR 2183) 2022-05-24 08:59:13 +02:00
Oliver Smith 8c7e99acd9
Prepare 1.43.1 release 2022-05-15 18:47:46 +02:00
David Wronek be9aab895b
pmb.install: Check for alpine-conf version (MR 2182)
Upstream Alpine change the way how `setup-timezone` works, it no longer
uses the `-z` argument to change the timezone.
2022-05-15 16:51:53 +02:00
Clayton Craft 9901cb31ea
pmb.parse.depends: ignore conflict dependency that no longer exists (MR 2181)
So it looks like some packages might be mentioned in the "depends=" of
other packages and marked as conflict, for example:
    depends="!foo"

..and the conflicting package can be dropped from the repo entirely if
it is no longer needed, however the reference to it in `depends=` still
exists. This handles that situation by just ignoring the missing package
if it is only a conflicting dependency.

fixes https://gitlab.com/postmarketOS/pmaports/-/issues/1525
2022-05-12 22:26:53 -07:00
Oliver Smith 6fbe916e0d
Prepare 1.43.0 release 2022-04-14 09:33:20 +02:00
Oliver Smith 7abb281296
partitions_mount: fix finding the partition (MR 2178)
Instead of trying both "${dev}1" and "${dev}p1" as partition paths, only
try one of them depending on if "${dev}" ends in a number or not. This
fixes getting a wrong /dev/loop11 partition by accident if there are
many loop devices, instead of only looking for the correct path
/dev/loop1p1. People reported this happening with snaps on ubuntu.
2022-04-14 09:09:46 +02:00
Minecrell faf523911a
pmb.install: Treat empty "flash_sparse_samsung_format" as disabled (MR 2180)
Deviceinfo variables are empty strings by default if they are unset
in the deviceinfo file, so the "is not None" check currently enables
the sm_sparse_image_tool for all existing devices using sparse format.

Simplify the check to convert the string to a bool. Empty strings
evaluate to False so this should work as originally intended.
2022-04-13 21:01:20 +02:00
Enrico Lumetti dee8e34298
pmb.config, pmb.install: add option to handle Samsung sparse image format (MR 2177) 2022-04-13 08:59:29 +02:00
Newbyte ff569ece14
config: Move ASHMEM into conditional (MR 2176)
Option was dropped and is no longer available.

See https://lore.kernel.org/all/20220315123457.2354812-1-hch@lst.de
And 721412ed3d
2022-04-05 21:04:22 +02:00
Newbyte 9a74848f19
pmb.parse.kconfig: print a helpful error message on invalid kconfig name (MR 2172)
Currently pmbootstrap prints a stacktrace if there's no . in the
kernel configuration name. Let's make it more helpful.
2022-04-02 16:36:17 +02:00
Alexey Min 0bab8ab6d2
install: fix alignment of information messages (MR 2174)
Make it in line with other information messages in this section.
Otherwise lk2nd flashing info paragraph looks shifted to the right.
2022-03-29 20:06:40 +03:00
Alexey Min fd74b54001
config: allow using flash_lk2nd with heimdall too (MR 2174)
One of the main purposes of lk2nd is to have a replacement for
the stock bootloader "Odin mode" on many Samsung devices with
sane fastboot implementation, which supports many more features
like live booting and many others.

This is done by using heimdall to flash lk2nd to device's boot
partition. Allow this method to be used with this flash action.
2022-03-29 15:36:08 +03:00
Bart Ribbers 1eef7fbdeb
pmb.parse.kconfig: implement UEFI kconfig check (MR 2169) 2022-03-16 09:25:00 +01:00
Oliver Smith 301f0995fb
pmb.parse.kconfig.check_file: fix broken f-string (MR 2169) 2022-03-16 09:24:55 +01:00
Laszlo Molnar 2e39912790
pmb.parse.arguments: add zap --all flag (MR 2117)
Fixes: #1692
2022-02-28 17:09:21 +03:00
Luca Weiss 5252d8de78
config: Move NFT_COUNTER into conditional (MR 2171)
The option has been removed and is now always enabled with nftables
core.

See also https://git.kernel.org/torvalds/c/023223d
2022-02-24 08:54:50 +01:00
Oliver Smith e45e8f932c
Prepare 1.42.0 release 2022-02-20 15:29:40 +01:00
Clayton Craft a367e0d95b
pmb.chroot.mount: create /dev/fd (MR 2170)
Specifically, this fixes issues with process substitution in bash in the
chroot.

For example, the following lines from a NetworkManager build script:

    ... |
        grep -Fx -f <(get_symbols_explicit) -v |
        grep -Fx -f <(get_symbols_nm)

fail with:
     ninja: job failed: /home/pmos/build/src/NetworkManager-1.35.92/tools/create-exports-NetworkManager.sh --called-from-build /home/pmos/build/src/NetworkManager-1.35.92
     grep: /dev/fd/63: No such file or directory
     grep: /dev/fd/63: No such file or directory
2022-02-20 15:03:09 +01:00
jenneron 7b2ffc3e5f
Support ChromeOS partition table and kernel partition (MR 2163)
Stock bootloader on these devices boots kernel (it is intended to boot
kernel, but i place secondary bootloader there) from special ChromeOS
kernel partition on special GPT which is created with cgpt utility.
This MR adds initial support for it introducing new deviceinfo options:

- cgpt_kpart - path to file to be flashed to ChromeOS partition;
- cgpt_kpart_start - offset from the start in sectors;
- cgpt_kpart_size - partition size in sectors.

For example:

deviceinfo_cgpt_kpart="/usr/share/u-boot/google-peach-pit/u-boot-dtb.img.kpart"
deviceinfo_cgpt_kpart_start="8192"
deviceinfo_cgpt_kpart_size="32768"

cgpt requires start and size values of partition, so these values
are calculated for each partition.

Reserved size and on-device installer are not yet supported.

Reference: https://archlinuxarm.org/platforms/armv7/samsung/samsung-chromebook
2022-02-20 14:49:56 +01:00
Anton Bambura 2363732645
pmb.install: add get_partition_layout() (MR 2163)
"Keeping track of which partition number is what is hard to understand now.
 I think this should be refactored, so we have it defined only in one
 place, and easy to read. Since this merge request increases the complexity
 of the partitions again, let's do it here before merging." - Oliver

Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
2022-02-20 14:48:18 +01:00
Caleb Connolly 18a16c7899
pmb: flasher: add support for rk partitions (MR 2165) 2022-02-13 19:54:29 +01:00
Martijn Braam 884174ce3c
config: use partition names for rkdeveloptool (MR 2165) 2022-02-13 19:54:23 +01:00
Clayton Craft 100fd332df
pmb.qemu.run: add option to set serial to mon:stdio (MR 1980)
This makes QEMU trap signals like Ctrl-C and send it to the guest
instead of terminating QEMU.

To quit QEMU with this option you can use [Ctrl-A] [x]
  or
[Ctrl-A] [c] and type 'quit' at the prompt.

This behavior is disabled by default, and can be enabled by setting a
new option in the pmbootstrap.cfg (using "pmbootstrap config
qemu_redir_stdio true")

Co-authored-by: Luca Weiss <luca@z3ntu.xyz>
2022-02-13 19:22:49 +01:00
Tony Garnock-Jones 3c5effa0e7
Allow override of SSH key glob (MR 2167) 2022-02-08 12:29:27 +01:00
Oliver Smith 5d3cbbdb86
pmb.helpers.pkgrel_bump: !-deps are not missing (MR 2166)
Do not assume that conflicting dependencies are missing.

Fix for:
  Increase 'unl0kr' pkgrel (0 -> 1), missing depend(s): !osk-sdl

Related: https://gitlab.com/postmarketOS/monitoring/-/jobs/2058087076
2022-02-05 11:54:32 +01:00
Caleb Connolly 47410eafb9
config: fix missing comma in flash methods (MR 2164)
o/ hi Martijn
2022-02-02 21:20:11 +00:00
Marian Stramm 80c988236f
pmb.install: add support for creating btrfs root filesystem (MR 2155)
in preparation for reversible upgrades, this change adds the option of
creating a btrfs root filesystem
2022-01-29 11:47:29 +03:00
Bart Ribbers 9e63d5820c
Support generating GPT partition tables (MR 2160)
The PPP will, in combination with tow-boot, boot with GPT on the
internal storage. We need to support that on postmarketOS and as a first
step, let's make it possible to generate GPT images and boot from that
2022-01-28 10:01:46 +01:00
Bart Ribbers 36aabcc4fe
Print return code when subprocess fails (MR 2161)
It can be very useful to figure out why a particular subprocess might be
failing ;)
2022-01-22 15:45:12 +01:00
Mark Hargreaves ff0942b12d
pmbootstrap kconfig check: add netboot check (MR 2064)
Allow checking for the necessary kernel option for netboot.

Co-authored-by: Luca Weiss <luca@z3ntu.xyz>
2022-01-20 22:29:46 +01:00
Mark Hargreaves 47539f1bef
pmb/netboot: new feature (MR 2064)
pmbootstrap netboot command exposes the generated vendor-codename.img
rootfs through nbd interface so that device can mount it and boot
postmarketOS without having any storage medium at all.

Co-authored-by: Luca Weiss <luca@z3ntu.xyz>
2022-01-20 22:29:41 +01:00
Rudraksha Gupta 997e3fb1f5
pmb.install: Remove pass to conform to code style (MR 2159) 2022-01-20 21:01:46 +01:00
Rudraksha Gupta bea18e03f3
pmb.install: password for root is already locked (MR 2159)
When running `pmbootstrap install`, pmbootstrap does not unlock root
when it completes. This patch allows `pmbootstrap install` to run two or
more times (without zap).

Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
2022-01-20 21:01:41 +01:00
Oliver Smith d698aa15ad
Prepare 1.41.0 release 2022-01-18 21:46:34 +01:00
Johannes Marbach 6d7d113040
pmb.chroot.apk: Delete conflicting dependencies (MR 2157)
When installing dependencies for a package, conflicting (!) dependencies
are now deleted (with `apk del pkg`) whereas before a constraint for
their _absence_ was added (with `apk add !pkg`). Doing it the new way
around prevents creating deadlocks because a `!pkg` constraint will
prevent pkg from ever being installed without an explicit `apk del`
call.

Fixes: #2092
2022-01-18 20:06:23 +01:00
Newbyte 614cb72a2a
pmb.install: update UID 10000 explanation (MR 2158) 2022-01-11 00:17:55 +03:00
Johannes Marbach 2ad5427126
pmb.install: automatically pick unlocker package (MR 2154)
This removes the hard-coded installation of osk-sdl when specifying
--fde (or using the on-device installer) and instead determines the
unlocker package by taking the most suitable provider of
postmarketos-fde-unlocker (factoring provider priority and packages
selected for installation).

With this change applied, one can manually select an unlocker package to
be installed via

    pmbootstrap install --fde --add other-unlocker

Relates to: postmarketOS/pmaports#1309
2022-01-02 22:40:23 +01:00
Oliver Smith 6f6a3b0408
Happy new year 2022! 2022-01-02 22:39:14 +01:00
Andreas Kemnade 8ca71ed3a0
install: do not choke on blockdevices without fs (MR 2153)
when blkid returns 2 while searching for existing pmos installations,
do not abort. That might happen if there is a block device without a
filesystem.

Error given:
(1653851) [21:39:19] (native) % blkid -s LABEL -o value /dev/sdcardp1
(1653851) [21:39:19] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(1653851) [21:39:19] NOTE: The failed command's output is above the ^^^ line in the log file: /home/andi/.local/var/pmbootstrap/log.txt
(1653851) [21:39:19] ERROR: Command failed: (native) % blkid -s LABEL -o value /dev/sdcardp1
(1653851) [21:39:19] See also: <https://postmarketos.org/troubleshooting>
(1653851) [21:39:19] Traceback (most recent call last):
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/__init__.py", line 49, in main
    getattr(frontend, args.action)(args)
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/helpers/frontend.py", line 314, in install
    pmb.install.install(args)
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/install/_install.py", line 944, in install
    install_system_image(args, 0, f"rootfs_{args.device}", step, steps,
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/install/_install.py", line 586, in install_system_image
    pmb.install.blockdevice.create(args, size_boot, size_root,
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/install/blockdevice.py", line 138, in create
    mount_sdcard(args, sdcard)
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/install/blockdevice.py", line 51, in mount_sdcard
    if previous_install(args, path):
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/install/blockdevice.py", line 27, in previous_install
    label = pmb.chroot.root(args, ["blkid", "-s", "LABEL", "-o", "value",
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/chroot/root.py", line 76, in root
    return pmb.helpers.run_core.core(args, msg, cmd_sudo, None, output,
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/helpers/run_core.py", line 343, in core
    check_return_code(args, code, log_message)
  File "/home/andi/.local/lib/python3.9/site-packages/pmb/helpers/run_core.py", line 219, in check_return_code
    raise RuntimeError("Command failed: " + log_message)
RuntimeError: Command failed: (native) % blkid -s LABEL -o value /dev/sdcardp1
2022-01-02 22:14:06 +01:00
Johannes Marbach a9d1049a3c
build: add support for conflicting dependencies (MR 2146)
This adds support for the depends="!conflict ..." syntax for explicitly
marking another package as conflicting with the current one.

Fixes: #2085
2021-12-30 20:25:02 +01:00
Oliver Smith 3e5f27d5e3
pmb.parse.depends.recurse: tweak message (MR 2151)
Fix the lint error. While at it, change the wording so it's clear it's
looking in pmaports instead of aports, use '' around the source package
just as the message does about the pkgname_depend, remove <> from the
link URL.

New message:
  Could not find dependency 'so:libPocoData.so.80' in checked out pmaports dir or any APKINDEX. Required by 'nymphcast'. See: https://postmarketos.org/depends

Fixes: 81dc4c ("pmb.parse: show which package require the missing package")
2021-12-19 18:16:51 +01:00
Martijn Braam 81dc4c17f6
pmb.parse: show which package require the missing package (MR 2150)
When a dependency is not found it now shows all the pacakges that
require that dependency in the error message.
2021-12-17 17:17:25 +01:00
Oliver Smith fd7050835f
Prepare 1.40.0 release 2021-12-12 19:07:21 +01:00
Anjandev Momi c1407f921b
Use doas instead of sudo if installed (MR 2091)
Prefer using doas over sudo if both are installed. Let the user override
the sudo tool with PMB_SUDO.
2021-12-12 18:46:17 +01:00
Tim Zimmermann 2801b5d687
kconfig-check: only check TZDEV/SAMSUNG_TUI on kernels older than 4.14 (MR 2148)
* On newer samsung devices disabling these does the exact opposite of
  what's mentioned in 50cdf04760
* With these enabled the device boots up just fine
2021-12-12 17:23:21 +03:00
Alexey Min 66d11d6d34
pmb.helpers.ui: clarify "none" UI purpose
And suggest users to use "console" UI instead.
2021-12-12 16:51:39 +03:00
Clayton Craft 58e4e86b6b
pmb.parse.parse_attribute: strip off trailing comment on attrib line (MR 2149)
The apkbuild parser could not handle cases where a line ends in a
comment but the value is not quoted.

E.g. this:

    pkgver=1.0 # I'm a comment, look at me

was being parsed to a value like this:

    1.0 # I'm a comment, look at me

... which is obviously wrong. This strips off any trailing comment on
the line, so it's parsed to the correct value.

fixes #2087
2021-12-12 16:16:30 +03:00
Anjandev Momi 8563efb9da
aportupgrade: if $pkgver=9999, use $_pkgver variable instead (MR 2127)
fixes #2071
2021-11-27 15:05:25 +01:00
Oliver Smith 54fade131c
Fix 'kconfig edit linux-...' (MR 2143)
Properly assemble the required pkgname to fix:

  $ pmbootstrap kconfig linux-postmarketos-allwinner
  ERROR: Could not find aport for package: postmarketos-allwinner

Fixes: 6557e6 ("kconfig argcomplete: support full pkgname")
2021-11-27 14:45:09 +01:00
BO41 3f2bd03d33
remove unused args argument (MR 2136) 2021-11-27 14:13:33 +01:00
BO41 379991aa62
pmb.parse.apkindex.parse: remove unused args argument (MR 2136) 2021-11-27 14:13:33 +01:00
BO41 99bed38272
pmb.parse.apkbuild: remove unused args argument (MR 2136) 2021-11-27 14:13:33 +01:00
BO41 ce0f1c2d4a
args.cache: remove (MR 2136)
Replace "args.cache" with a global variable in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
2021-11-27 14:13:27 +01:00
Oliver Smith f30b1cc3f2
pmb.config.apk_tools_min_version: add alpine 3.15 (MR 2144)
Prepare for postmarketOS v21.12 release.
2021-11-26 00:51:11 +01:00
Jami Kettunen b3d91f274d
pmb/config: check CROSS_MEMORY_ATTACH config required for Waydroid (MR 2140)
Without this syscall zygote keeps constantly crashing in the container.
2021-11-18 19:43:51 +02:00
Caleb Connolly 4c4bd77c87
install: support password as cmdline arg (MR 2125)
So I can finally run `pmbootstrap install --password 147147` and go and
make a cup of tea.

Based on MR 1919.

Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
2021-11-06 22:45:00 +01:00
Oliver Smith 498738abcc
Prepare 1.39.0 release 2021-11-06 21:06:53 +01:00
Ultracoolguy 54afa35d70
Add 'flasher flash_lk2nd' command (MR 2089)
Many Qualcomm devices need this second bootloader for booting a mainline
kernel. Add an option to flash one if it exists.
2021-11-06 20:51:58 +01:00
Luca Weiss f4990cfc7a
aportupgrade: support gitlab.gnome.org (MR 2119)
Also sort the gitlab hosts alphabetically
2021-11-06 20:31:25 +01:00
Luca Weiss f996f3ed5a
aportupgrade: prefer explicit package mapping (MR 2119)
This solves some edge cases with packages like 'date', 'fmt' and others.
2021-11-06 20:31:19 +01:00
Oliver Smith 6557e6892a
kconfig argcomplete: support full pkgname (MR 2134)
Support full linux-* package names in argument completion for
"pmbootstrap kconfig ..." command-lines and get rid of related PROTIP
messages:
  PROTIP: You can simply do 'pmbootstrap kconfig check postmarketos-allwinner'

This improves consistency, as in other places we expect the user to
supply full package names as well (e.g. pmbootstrap build).
2021-11-06 15:23:37 +01:00
Oliver Smith 152bd6753c
arguments: fix compat with argcomplete 1.8 (MR 2134)
When running pmbootstrap on debian bullseye with the distro's
python3-argcomplete 1.8 from 2017, tab completion was broken. After
disabling stderr redirect to /dev/null, the error appeared:
  TypeError: package_completer() missing 1 required positional argument: 'parser'

Support this ancient version of argcomplete too by setting None as
default value for parser (we don't use it anyway).

Related: https://wiki.postmarketos.org/wiki/Pmbootstrap_development_guide#Debugging_tab_completion_.28argparse.29
2021-11-06 15:23:31 +01:00
Minecrell 8ace36113c
pmb.config/install: add flexible provider selection for "pmbootstrap init" (MR 2132)
The provider selection for "pmbootstrap init" added in this commit
is a flexible way to offer UI/device-specific configuration options
in "pmbootstrap init", without hardcoding them in pmbootstrap.
Instead, the options are defined entirely in pmaports using APK's
virtual package provider mechanism. The code in pmbootstrap searches
for available providers and displays them together with their pkgdesc.

There are many possible use cases for this but I have tested two so far:

  1. Selecting root provider (sudo vs doas). This can be defined entirely
     in postmarketos-base, without having to handle this specifically in
     pmbootstrap.

     $ pmbootstrap init
     [...]
     Available providers for postmarketos-root (2):
      * sudo: Use sudo to run root commands (**default**)
      * doas: Use doas (minimal replacement for sudo) to run root commands
              (Note: Does not support all functionality of sudo)
     Provider [default]: doas

  2. Device-specific options. My main motivation for working on this
     feature is a new configuration option for the MSM8916-based devices.
     It allows more control about which firmware to enable:

     $ pmbootstrap init
     [...]
     Available providers for soc-qcom-msm8916-rproc (3):
      * all: Enable all remote processors (audio goes through modem) (default)
      * no-modem: Disable only modem (audio bypasses modem, ~80 MiB more RAM)
      * none: Disable all remote processors (no WiFi/BT/modem, ~90 MiB more RAM)
     Provider [default]: no-modem

The configuration prompts show up dynamically by defining
_pmb_select="<virtual packages>" in postmarketos-base, a UI PKGBUILD
or the device APKBUILD. Selecting "default" (just pressing enter)
means that no provider is selected. This allows APK to choose it
automatically based on the "provider_priority". It also provides
compatibility with existing installation; APK will just choose the
default provider when upgrading. The selection can still be changed
after installation by installing another provider using "apk".

Note that at the end this is just a more convenient interface for the
already existing "extra packages" prompt. When using pmbootstrap in
automated scripts the providers (e.g. "postmarketos-root-doas") can be
simply selected through the existing "extra_packages" option.
2021-11-06 15:04:34 +01:00
Minecrell 47645f41b1
pmb.parse._apkbuild: parse provider_priority as int (MR 2132) 2021-11-06 15:04:25 +01:00
Minecrell 19b232969d
pmb.helpers.pmaports: search for provides in both root and subpackages (MR 2132)
At the moment, "provides" are only checked in the root package and not
in subpackages of APKBUILDs. Fix this by looking through the subpackages
as well.
2021-11-06 15:04:25 +01:00
Minecrell 5ed807c064
pmb.helpers.pmaports: parse guessed APKBUILD first for speed up (MR 2132)
At the moment we have to parse all APKBUILDs to find subpackages,
even if they are guessed easily shortly after. To speed this up,
let's guess first but verify the guess by only parsing that particular
APKBUILD. If the subpackage/provides is in there we seem to have found it.
2021-11-06 15:04:25 +01:00
Minecrell eb3e38d15c
pmb.helpers.pmaports: separate finding pkg in APKBUILD to function (MR 2132)
Make it possible to call this in a different place as well by placing
it in a separate function.
2021-11-06 15:04:25 +01:00
Minecrell 7246c32539
pmb.install: move install_recommends check into _install.py (MR 2132)
The current install code looks a bit confusing, there is an existing
if statement for the ui and ui-extras package but the recommended
packages are already installed before with a check in a completely
different file. Make this a bit more clear by moving this to the
ui if statement instead.
2021-11-06 15:04:25 +01:00
Minecrell 9d724d5d3d
pmb.config.init: parse deviceinfo only once (MR 2132)
Instead of parsing the deviceinfo over and over again, parse it once
and pass it to all the functions that make use of it.
2021-11-06 15:04:20 +01:00
Oliver Smith 05f257295d
pmb.config: add comment: kconfig check value types (MR 2133) 2021-11-06 14:28:54 +01:00
Oliver Smith aead36d5ac
pmbootstrap kconfig check: add iwd check (MR 2133)
iwd seems like a promising alternative to wpa_supplicant. It uses crypto
implementations from the kernel, so let's make kconfig check aware of
the options it needs.
2021-11-06 14:28:53 +01:00
Oliver Smith d856e21673
pmb.config.necessary_kconfig_*: tweak comments (MR 2133)
Improve the comments for consistency and to give a little more info than
just repeating the variable name.
2021-11-06 14:28:53 +01:00
Oliver Smith 93e7a1d876
pmbootstrap kconfig check: add apparmor check (MR 2133)
SECURITY_APPARMOR_BOOTPARAM_VALUE was required to enable it by default
until 5.1 where the option was removed.

Related: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=0102fb83f90050b86ce37aec810ea17bb4448e0c
Related: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/security/Kconfig?id=fe91c4725aeed35023ba4f7a1e1adfebb6878c23#n285
2021-11-06 14:28:53 +01:00
Oliver Smith dcedc4bc1f
kconfig check: support strings too (MR 2133)
Extend the kconfig check code to not only support booleans and arrays,
but also strings. This will be used for CONFIG_LSM with apparmor where
it's important that "apparmor" has a certain spot in the list.
2021-11-06 14:28:53 +01:00
Oliver Smith 6b8fa93d37
pmb.parse.kconfig.check: git friendly arguments (MR 2133)
Put each component argument into a separate line, as these get changed
whenever adding a new kernel config check. I'm about to add two new
ones.
2021-11-06 14:28:49 +01:00
Alexey Min 0664a38190
pmb.qemu.run: workaround for new mkinitfs requirements (MR 2138)
It's been 3 months since we switched to new mkinitfs
and we are still fixing consequences.

linux-lts and linux-virt used by qemu-amd64 device package
install vmlinuz-lts (vmlinuz-virt). And qemu run cmdline
passed -kernel vmlinuz which makes it impossible to run
qemu without this change.

With this change proper kernel arg is passed to qemu.

Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
Signed-off-by: Alexey Min <alexeymin@postmarketos.org>
2021-11-06 13:36:34 +01:00
Jami Kettunen 931675450d
pmb/config: check CRYPTO_AES and INPUT_EVDEV configs required for FDE (MR 2137)
AES is needed on top of XTS to unlock a disk with FDE enabled and evdev
is used in osk-sdl to input the passphrase.
2021-11-01 23:07:05 +03:00