Commit Graph

2372 Commits

Author SHA1 Message Date
Oliver Smith 032296b7ab
test: rework kconfig check tests
Replace the old kconfig check tests with new ones, that do not rely on:
* any kernel config in pmaports
* the currently required kconfig options set in pmb/config/__init__.py

This will make it much easier to change the required options, and moves
the testsuite a bit more towards making it completely independent from
current pmaports state.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-13-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 3b1ec76395
kconfig check: extract_arch: support riscv64
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-12-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 905ccbb840
kconfig check: extract_version: replace - with _
Replace the dash in rc kernels with an underscore, so the resulting
version of e.g. 6.1.0-rc3 becomes a valid alpine pkgver (6.1.0_rc3).
This is needed so pmbootstrap can properly parse and compare the
version.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-11-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith f393bbdab1
kconfig check: less errors for non-detailed mode
Don't print more than one non-detailed error per component, as it
results in the same message. Adjust the description of the details
parameter to reflect this.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-10-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith dffb8614bc
kconfig check: add --no-details argument
Provide a fast way to go through the no-details mode for kconfig check,
which is otherwise only triggered after running "kconfig edit". This is
useful for development.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-9-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 9bbb1c11fc
kconfig check: add descriptions to more functions
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-8-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 23014ef76f
kconfig check: move check_config_options_set up
In this file and throughout most of the rest of the pmbootstrap code,
functions that get called by other functions are defined on top of them
if they are in the same file. check_config_options_set gets called by
check_config, so put it above.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-7-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith fd1b4e5d19
kconfig check: drop "necessary_"
Rename necessary_kconfig_options_* variables to kconfig_options_*. It's
clear that these are necessary to pass the check, so let's shorten them.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-6-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith a568cf4520
kconfig check: config_file -> config_path
Rename config_file to config_path for consistency with the rest of the
file. Both are used to point to the full path of the config file.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-5-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith 1a124ec2b6
kconfig check: remove redundant component lists
Components are the sets of kernel config options in
pmb/config/__init__.py, such as the ones for waydroid, nftables, etc.

Generate the list of components dynamically and refactor the code, so we
can add/remove components in just one place.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-4-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith d14dbd49de
pmb.parse.kconfig: remove config_path_pretty
Don't set the config_path_pretty variable and pass it through various
functions until it ends up being used in check_option. This was just the
directory name of the kernel config and the kernel config filename (or
in case of pmbootstrap kconfig check --file, just the same as the
config path).

Instead we can just print the filename of the kernel config, for example
"config-postmarketos-qcom-sdm845.aarch64". It is shorter and already
obvious to which package it belongs.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-3-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith dbe13fa812
pmb.parse.kconfig.check_option: refactor
Deduplicate the code for printing warnings and tweak it slightly:
* Shorten ".See <$url> for details." to ": $url".
* Include the component in the detailed output too.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-2-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:41 +01:00
Jami Kettunen 2f9bdb7a66
pmb.parse.arch: Include comma after last dictionary items
See https://lists.sr.ht/~postmarketos/pmbootstrap-devel/patches/39643#feedback-272128:1018-1194

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167883228237.9452.6945199441942129608-0@git.sr.ht%3E
2023-03-16 08:58:34 +01:00
Jami Kettunen 0230f4def2
pmb.config.apkbuild_attributes: Add _depends_dev
Fixes building of linux-edge (from "pmbootstrap aportgen --fork-alpine
linux-edge")

  ERROR: Package '$_depends_dev': Could not find aport, and could not find this package in any APKINDEX!

See also: https://gitlab.alpinelinux.org/alpine/aports/-/issues/14250

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Tested-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167858994438.8366.14589100885556224988-0@git.sr.ht%3E
2023-03-12 17:21:02 +01:00
Jami Kettunen 7186673d59
pmb/parse/arch.py: add riscv64 -> riscv to kernel carch mapping
This fixes both the default riscv64 kernel APKBUILD "_carch" selection and
"pmbootstrap kconfig edit" functionality even after manually corrected
"_carch" variable.

  (native) make menuconfig
  ../Makefile:609: ../arch/riscv64/Makefile: No such file or directory

See also https://github.com/torvalds/linux/tree/master/arch with search
for "riscv"

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167858205621.22796.14648808830656459528-0@git.sr.ht%3E
2023-03-12 17:21:02 +01:00
Jami Kettunen 90ae7f3a79
pmb.aportgen.linux: Depend on findutils
Without this on newer kernels the following is emitted during kernel
build (though it still appears to build a package fine):

  find: unrecognized: -printf
  BusyBox v1.36.0 (2023-03-03 16:36:18 UTC) multi-call binary.
  ...

See also: https://gitlab.com/postmarketOS/pmaports/-/issues/910

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167855488981.11771.14125941910883500913-0@git.sr.ht%3E
2023-03-12 17:20:55 +01:00
Oliver Smith 60901ac462
pmb.helpers.run: don't pass stdin to output=stdout
When looking at the table for possible output modes, it only makes sense
to pass stdin to "interactive" and "tui". The output mode "stdout" is
for non-interactive commands.

This fixes apk going interactive (asking for confirmation) when running
pmbootstrap with --details-to-stdout and building a package that depends
on postmarketos-base.

Fixes: issue 2208
Tested-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230301204112.4351-1-ollieparanoid@postmarketos.org%3E
2023-03-02 08:10:34 +01:00
Oliver Smith a26e203f53
partition_cgpt: use pmb.chroot.apk.install
Be consistent with the rest of the pmbootstrap codebase and install
packages through this function.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230301204311.4456-1-ollieparanoid@postmarketos.org%3E
2023-03-02 08:10:29 +01:00
Pablo Correa Gómez bf820f442d
install: make cached remote repositories available on first install
This allows the usage of APK (and apk-polkit-rs) on first boot without
the need to establish an internet connection.

Closes https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2206

Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3CPAXP192MB1293A82207F0983ED6C8665EC7AE9@PAXP192MB1293.EURP192.PROD.OUTLOOK.COM%3E
2023-02-27 08:35:20 +01:00
Pablo Correa Gómez 014509b427
install: run alpine-appstream-downloader if available
This makes sure that the AppStream data is available on the built
image. To avoid unnecessary downloads on multiple invocations of
install, create a new cache chroot.

Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3CPAXP192MB12938C6B570CA56AA162E8A6C7AE9@PAXP192MB1293.EURP192.PROD.OUTLOOK.COM%3E
2023-02-27 08:35:20 +01:00
Oliver Smith f296dc62e5
ci/pytest.sh: check if pmaports dir is clean
Verify that the pmaports dir is clean before starting the testsuite, as
some tests will fail if it is not. It's annoying when this is the reason
that you have to restart the testsuite run after it already spent some
time, so rather check for it beforehand.

Ideally the testsuite wouldn't depend on the state of pmaports, see
issue 2105 for that.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230226184731.6989-1-ollieparanoid@postmarketos.org%3E
2023-02-27 08:35:20 +01:00
Oliver Smith f208bba4f2
qemu: warn that network won't work with ui=none
The other day I spent way too long trying to find a regression that
caused network inside qemu not to work anymore, before I realized it was
caused by selecting UI=none instead of anything else.

Print a warning and link to a wiki page describing this in more detail.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230226184546.6869-1-ollieparanoid@postmarketos.org%3E
2023-02-27 08:35:14 +01:00
Newbyte 162867a08c
config: Add BINFMT_ELF and BINFMT_SCRIPT
pmOS cannot boot without these two. There was recently someone in the
unofficial postmarketOS Telegram group who used some minimal config and
couldn't figure out why the initramfs couldn't start up, and these
options being disabled turned out to be the culprit. As such, let's
ensure these are enabled to make life easier for people setting up new
kernel configurations.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230128203716.152281-1-newbyte@postmarketos.org%3E
2023-02-26 19:42:05 +01:00
Oliver Smith a0cea6255a
pmb.config: mount go caching directories
Always set GOCACHE during build and let it point to pmbootstrap's work
dir. This has a similar effect as using ccache for C/C++.

Set GOMODCACHE conditionally: this is for caching the go modules (git
repositories) that get cloned during a build if they are not bundled
with the source. Usually APKBUILDs should cache them, but when using
pmbootstrap build --src, they would get downloaded over and over again.
Set GOMODCACHE automatically for --src and allow enabling/disabling this
manually with new --go-mod-cache / --no-go-mod-cache.

This speeds up multiple iterations of building the same package
significantly. I'm using it for:

$ pmbootstrap build postmarketos-mkinitfs --src=...

I've verified that using the same GOCACHE dir for multiple architectures
doesn't result in build artifacts for the wrong arch getting used.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230223064743.1773-1-ollieparanoid@postmarketos.org%3E
2023-02-26 19:37:30 +01:00
Oliver Smith 3ad159f98d
Prepare 1.51.0 release 2023-02-19 16:13:52 +01:00
Oliver Smith 98290f9191
pmbootstrap init: don't allow 'root' as username
While going over the wiki pages recently, I've noticed that somebody
added a note to not use "root" as username. I don't think many people
will attempt to do this, but let's catch it here in pmbootstrap with a
friendly message instead of mentioning it in the wiki.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230126071154.1689-1-ollieparanoid@postmarketos.org%3E
2023-02-19 16:13:52 +01:00
Oliver Smith e8a2ab2e98
check_binfmt_misc: print more helpful error
Instead of letting the mount -t binfmt_misc … command fail, and simply
telling the user that the command failed: rely on the following check
that prints a better error message on error. If it fails at this point,
the kernel option isn't enabled. Add two comments explaining why
check=False is used in this function while at it.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230123064516.1607-1-ollieparanoid@postmarketos.org%3E
2023-02-19 16:13:52 +01:00
Oliver Smith 9975d373b0
Bump copyright to 2023 2023-01-22 19:18:06 +01:00
Oliver Smith faccbdc2eb
qemu: ssh hostfwd on 127.0.0.1 instead of 0.0.0.0
Make it accessible over localhost only. The idea is that users can login
into the VM on their own PC while developing, not that it's exposed over
the network. If somebody needs to have hostfwd on another IP, send a
patch to make this configurable via argparse.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221221220305.1809-1-ollieparanoid@postmarketos.org%3E
2023-01-20 15:28:07 +01:00
Clayton Craft 917ff92f63
pmb.qemu.run: fix 3d accel on amd64
Recent changes to qemu and Alpine packaging now require using the
virtio-vga-gl device and installing -gl packages to get virglrenderer
support.

Without this, wlroots fails to get an EGL context (among other problems
you'd expect by not having a useful GPU around...)

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221210185021.3546-1-clayton@craftyguy.net%3E
2023-01-20 15:28:07 +01:00
Jane Rachinger 42feaf0d49
pmb.build.other: do not copy leftover abuild dirs
Running abuild on the host directly creates directories in the
aport where it gets built. Interrupting abuild results in those
working-dirs not getting deleted.
We don't want to copy those entries to our builder.

It's only really noticeable if pmbootstrap tries to copy a broken
symlink in src/ and thus fails.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221207205201.22139-1-jane400@bingo-ev.de%3E
2023-01-20 15:28:02 +01:00
Oliver Smith 9d93d34b65
Prepare 1.50.1 release 2022-12-05 08:19:27 +01:00
Weijia Wang a56838fb45
pmb/parse/arch.py: add arm64 -> aarch64 to mapping
This allows pmbootstrap to work on aarch64-darwin systems.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167000939722.18650.8735853729695007724-0@git.sr.ht%3E
2022-12-05 08:19:27 +01:00
Oliver Smith bb5399d6d5
pmb.config.apk_tools_min_version: add alpine 3.17 2022-11-27 19:10:05 +01:00
Sicelo 4b339763db
pmb.parse.kconfig: update version for MEMCG_SWAP change
Without the `_rc1` tag, building kernel results in
"linux-nokia-n900/config-nokia-n900.armv7: CONFIG_MEMCG_SWAP should be set. See
<https://wiki.postmarketos.org/wiki/kconfig#CONFIG_MEMCG_SWAP> for details"
message.

See also https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2220#note_1146219177

Signed-off-by: Sicelo A. Mhlongo <absicsz@gmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
2022-11-22 07:43:35 +01:00
Oliver Smith 757d2a0bff
Prepare 1.50.0 release 2022-11-20 16:43:03 +01:00
Oliver Smith aeeeb826fb
CI: fix typo in pmOS CI url 2022-11-20 16:38:59 +01:00
Oliver Smith 147b3ce4b9
pmb ci: fix typo in URL
Fix a typo in the URL https://postmarketos.org/pmb-ci that is being used
to detect if a script is compatible with 'pmbootstrap ci' or not.
2022-11-20 16:38:37 +01:00
Luca Weiss 0f6c6238f9
pmb.qemu.run: stop forcing bios for riscv64
We decided now not to force the bios firmware being used for riscv64 but
instead just use the one built into QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221118203424.106861-1-luca@z3ntu.xyz%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 315621d5b8
pmb ci: add --fast argument
Make it easy to only run the fast tests.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-3-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 3fd22104a8
pmb ci: error on using --all with script names
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 9a84ad20b1
pmb ci: fix arg desc for --all
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-1-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:05 +01:00
vaino ed7b0273f5
flasher: heimdall: take depends from pmaports.cfg
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Co-developed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221114115705.1107886-1-vaino@vke.fi%3E
2022-11-16 09:01:32 +01:00
Caleb Connolly ab0aa7f956
pmb: sideload: wait for apk database lock
When installing the APK wait for a while for the APK database lock,
services like apk-polkit-server sometimes hog it for a while but there's
no point bailing out immediately.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111212251.1360612-1-kc@postmarketos.org%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 4a6c5657d5
pmb.parse.kconfig: don't enforce non-core checks for testing devices
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.

Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.

As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 381a1ca907
pmb.parse.kconfig: add wireguard, filesystems & more to community check
Add wireguard options, supported file systems and some extra options to
the community kconfig check.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-1-luca@z3ntu.xyz%3E
2022-11-12 14:28:40 +01:00
Oliver Smith 6f45a5d5fb
Prepare 1.49.0 release 2022-11-09 09:19:33 +01:00
Oliver Smith 30055c14d2
flasher: fastboot: take depends from pmaports.cfg
Adjust to avbtool now being part of android-tools in alpine edge.
Instead of trying to install both (which fails on edge), take the
dependencies from a new pmaports.cfg variable
supported_fastboot_depends, which only contains android-tools in
pmaports.git master branch.

Related: https://postmarketos.org/pmaports.cfg
2022-11-09 09:19:33 +01:00
Oliver Smith b64641bb1c
pmb.flasher.init.install_depends: new function
Move logic to install depends into one shared function.
2022-11-09 09:19:32 +01:00
Oliver Smith 04f8f59208
pmb.config.required_programs: add tar
Add tar as it is required for 'pmbootstrap ci' to get the source into
the pmbootstrap chroot.

Reviewed-by: Caleb Connolly <kc@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-2-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:32 +01:00