Commit Graph

358 Commits

Author SHA1 Message Date
Clayton Craft 4ff0b1f6c2
parse.apkindex: fix typo
CI apparently enforces spellcheck for comments now, so this also fixes
CI :D
2023-10-11 12:15:17 -07:00
Lauren N. Liberda 753e563ccc
parse: allow "-r" in pkgver
this seems to be parsed correctly, other than this one check

bug: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2260
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230813202045.24493-1-lauren@selfisekai.rocks%3E
2023-08-14 09:52:23 +02:00
Pablo Correa Gómez a806835441
deviceinfo: make parse_kernel_suffix private
Just a small and trivial improvement to signal method is private

Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230720164718.18964-1-pabloyoyoista@postmarketos.org%3E
2023-07-21 08:54:10 +02:00
Oliver Smith 67fe5a62fd
treewide: fix typos found with codespell
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230706211537.10438-2-ollieparanoid@postmarketos.org%3E
2023-07-13 10:07:53 +02:00
Anton Bambura 047df10d57
Support --no-cgpt
Some people may use U-Boot instead of depthcharge and they don't want cgpt

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C168861285289.22824.11703157261469263638-0@git.sr.ht%3E
2023-07-13 10:07:47 +02:00
Oliver Smith 185973fd97
Remove distcc support
With this code path, pmbootstrap would start a distccd + sshd in the
native chroot, and configure it so it runs the cross compiler. The
foreign arch chroots would then call this cross compiler from localhost
by calling the distcc client instead of gcc.

This code has been obsoleted by the much simpler crossdirect in 2019.
Let's finally remove it.

Fixes: issue 2179
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230613161437.570196-4-ollieparanoid@postmarketos.org%3E
2023-06-15 08:31:03 +02:00
Oliver Smith 25b3530c4e
Remove --no-crossdirect argument
With an upcoming patch to remove the legacy distcc code, the
--no-crossdirect argument would do the same as --no-cross: perform the
build entirely in qemu. So remove the argument. I doubt anybody was
using it anyway.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230613161437.570196-3-ollieparanoid@postmarketos.org%3E
2023-06-15 08:31:03 +02:00
Luca Weiss 97eb39f34b
flasher: remove long deprecated flash_system alias
A long long time ago we renamed flash_system to flash_rootfs, and now
it's time to finally remove this alias since nobody should be using it
anymore.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230605220329.14328-4-luca@z3ntu.xyz%3E
2023-06-12 08:46:22 +02:00
Luca Weiss b8d5b9e4e5
treewide: rename _system to _rootfs in various places
A long time ago we renamed the flash_system action into flash_rootfs.
Since we still kept some variables around, it's finally time to clean
that up.

Keep backwards compatibility for now since we cannot update pmaports at
the same time since the new deviceinfo names won't be supported in older
pmbootstrap versions.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230605220329.14328-3-luca@z3ntu.xyz%3E
2023-06-12 08:46:22 +02:00
Luca Weiss d200414d87
kconfig check: Add USB gadget check to community
Make sure that we disable legacy gadgets like USB_ETH. Our initramfs
uses configfs gadgets instead, currently RNDIS for USB networking.

In the future this can be expanded to more options like mass storage,
MIDI or whatever we're going to integrate into the OS that can be
configured by the user.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230423163438.746054-1-luca@z3ntu.xyz%3E
2023-05-02 08:38:04 +02:00
Oliver Smith 6352ab9c2d
Move version to pmb.__version__
Move the pmbootstrap version to __version__ in the pmb module, so it can
still be found automatically after setup.py is replaced.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230407233026.1712-4-ollieparanoid@postmarketos.org%3E
2023-04-15 01:44:37 +02:00
Oliver Smith 89cbae6d31
pmb.parse.kconfig.check: fix writing to list arg
Python passes all arguments by reference. For mutable objects such as
a list used here, changing the object will change the original one. If
components_list is not set, this means the default value gets modified.

This lead to kernels getting checked with the wrong required components.
For example, when checking a kernel from community first with the
default component_list, it would get extended with all options needed
for a kernel in community. When checking another kernel from the
testing category, also with the default component_list, it would now
check for the community options in the testing kernel.

Related: https://stackoverflow.com/a/986145
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230402124338.64886-1-ollieparanoid@postmarketos.org%3E
2023-04-05 08:53:05 +02:00
Oliver Smith 05675321d6
pmb.parse.arguments: don't use python 3.9 feature
Replace action=argparse.BooleanOptionalAction with action="store_true"
so it still works with Python 3.8 and 3.7. These aren't EOL yet, and
unless we have a good reason to drop support for them, we can still
support them.

The purpose of vermin ("pmbootstrap ci vermin") would be to warn about
this, filed a bug that it didn't catch it:
https://github.com/netromdk/vermin/issues/168

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230326164931.2390-1-ollieparanoid@postmarketos.org%3E
2023-04-04 09:13:09 +02:00
Clayton Craft 6c66bfb8aa
parse/arguments: add qemu --efi option
This will be used to support EFI boot in qemu

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Caleb Connolly <kc@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230316192838.21431-2-clayton@craftyguy.net%3E
2023-03-24 09:11:48 +01:00
Oliver Smith 97e21fa876
parse.arguments.add_kernel_arg: add nargs param
Prepare for a future patch that will set it to None to make sure we get
exactly one kernel argument.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-15-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 3b1ec76395
kconfig check: extract_arch: support riscv64
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-12-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 905ccbb840
kconfig check: extract_version: replace - with _
Replace the dash in rc kernels with an underscore, so the resulting
version of e.g. 6.1.0-rc3 becomes a valid alpine pkgver (6.1.0_rc3).
This is needed so pmbootstrap can properly parse and compare the
version.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-11-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith f393bbdab1
kconfig check: less errors for non-detailed mode
Don't print more than one non-detailed error per component, as it
results in the same message. Adjust the description of the details
parameter to reflect this.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-10-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith dffb8614bc
kconfig check: add --no-details argument
Provide a fast way to go through the no-details mode for kconfig check,
which is otherwise only triggered after running "kconfig edit". This is
useful for development.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-9-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 9bbb1c11fc
kconfig check: add descriptions to more functions
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-8-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith 23014ef76f
kconfig check: move check_config_options_set up
In this file and throughout most of the rest of the pmbootstrap code,
functions that get called by other functions are defined on top of them
if they are in the same file. check_config_options_set gets called by
check_config, so put it above.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-7-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith fd1b4e5d19
kconfig check: drop "necessary_"
Rename necessary_kconfig_options_* variables to kconfig_options_*. It's
clear that these are necessary to pass the check, so let's shorten them.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-6-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:47 +01:00
Oliver Smith a568cf4520
kconfig check: config_file -> config_path
Rename config_file to config_path for consistency with the rest of the
file. Both are used to point to the full path of the config file.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-5-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith 1a124ec2b6
kconfig check: remove redundant component lists
Components are the sets of kernel config options in
pmb/config/__init__.py, such as the ones for waydroid, nftables, etc.

Generate the list of components dynamically and refactor the code, so we
can add/remove components in just one place.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-4-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith d14dbd49de
pmb.parse.kconfig: remove config_path_pretty
Don't set the config_path_pretty variable and pass it through various
functions until it ends up being used in check_option. This was just the
directory name of the kernel config and the kernel config filename (or
in case of pmbootstrap kconfig check --file, just the same as the
config path).

Instead we can just print the filename of the kernel config, for example
"config-postmarketos-qcom-sdm845.aarch64". It is shorter and already
obvious to which package it belongs.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-3-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:46 +01:00
Oliver Smith dbe13fa812
pmb.parse.kconfig.check_option: refactor
Deduplicate the code for printing warnings and tweak it slightly:
* Shorten ".See <$url> for details." to ": $url".
* Include the component in the detailed output too.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230312151325.1968-2-ollieparanoid@postmarketos.org%3E
2023-03-24 09:11:41 +01:00
Jami Kettunen 2f9bdb7a66
pmb.parse.arch: Include comma after last dictionary items
See https://lists.sr.ht/~postmarketos/pmbootstrap-devel/patches/39643#feedback-272128:1018-1194

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167883228237.9452.6945199441942129608-0@git.sr.ht%3E
2023-03-16 08:58:34 +01:00
Jami Kettunen 7186673d59
pmb/parse/arch.py: add riscv64 -> riscv to kernel carch mapping
This fixes both the default riscv64 kernel APKBUILD "_carch" selection and
"pmbootstrap kconfig edit" functionality even after manually corrected
"_carch" variable.

  (native) make menuconfig
  ../Makefile:609: ../arch/riscv64/Makefile: No such file or directory

See also https://github.com/torvalds/linux/tree/master/arch with search
for "riscv"

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167858205621.22796.14648808830656459528-0@git.sr.ht%3E
2023-03-12 17:21:02 +01:00
Oliver Smith a0cea6255a
pmb.config: mount go caching directories
Always set GOCACHE during build and let it point to pmbootstrap's work
dir. This has a similar effect as using ccache for C/C++.

Set GOMODCACHE conditionally: this is for caching the go modules (git
repositories) that get cloned during a build if they are not bundled
with the source. Usually APKBUILDs should cache them, but when using
pmbootstrap build --src, they would get downloaded over and over again.
Set GOMODCACHE automatically for --src and allow enabling/disabling this
manually with new --go-mod-cache / --no-go-mod-cache.

This speeds up multiple iterations of building the same package
significantly. I'm using it for:

$ pmbootstrap build postmarketos-mkinitfs --src=...

I've verified that using the same GOCACHE dir for multiple architectures
doesn't result in build artifacts for the wrong arch getting used.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230223064743.1773-1-ollieparanoid@postmarketos.org%3E
2023-02-26 19:37:30 +01:00
Oliver Smith 9975d373b0
Bump copyright to 2023 2023-01-22 19:18:06 +01:00
Weijia Wang a56838fb45
pmb/parse/arch.py: add arm64 -> aarch64 to mapping
This allows pmbootstrap to work on aarch64-darwin systems.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167000939722.18650.8735853729695007724-0@git.sr.ht%3E
2022-12-05 08:19:27 +01:00
Oliver Smith 315621d5b8
pmb ci: add --fast argument
Make it easy to only run the fast tests.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-3-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 9a84ad20b1
pmb ci: fix arg desc for --all
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-1-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:05 +01:00
Luca Weiss 4a6c5657d5
pmb.parse.kconfig: don't enforce non-core checks for testing devices
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.

Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.

As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 381a1ca907
pmb.parse.kconfig: add wireguard, filesystems & more to community check
Add wireguard options, supported file systems and some extra options to
the community kconfig check.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-1-luca@z3ntu.xyz%3E
2022-11-12 14:28:40 +01:00
Oliver Smith 0bcd58765c
pmbootstrap ci: new command
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.

$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all

pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.

Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
2022-10-25 20:31:19 +02:00
Luca Weiss dfada2a522
pmb: enable riscv64 architecture (MR 2215)
Enable pmbootstrap to work on riscv64 packages, and at the same time
also expand the alpine_to_hostspec list to match upstream (adding
riscv32 and loongarch*).
2022-10-19 09:11:54 +02:00
Caleb Connolly 66706f896d
pmb.parse.build: mention host compiled kernels in envkenel help (MR 2175) 2022-10-18 20:08:41 +02:00
Luca Weiss f7fb1fe078
pmb.parse.kconfig: rename anbox check to waydroid (MR 2204)
Change the naming to fit what software is actually used in postmarketOS.
2022-09-13 09:30:23 +02:00
Luca Weiss e468fc518e
pmb.parse.kconfig: add 'community' option (MR 2204)
Add the new option that will be mandatory for all devices in
community/main category. This is just a combination of anbox + iwd +
nftables + containers + zram + netboot.

While the existing options could be removed we're keeping it for now
given that also some devices with downstream kernel might find some
options useful.
2022-09-13 09:30:23 +02:00
Luca Weiss 52530caaf8
pmb.parse.kconfig: remove apparmor check (MR 2200)
Remove the check as apparmor is not the way we want to go for privilege
separation anymore.
See https://gitlab.com/postmarketOS/pmaports/-/issues/1596
2022-09-04 19:10:48 +02:00
Oliver Smith c88a18d3f6
treewide: fix various lint errors (MR 2197)
Fix errors found by new flake8 version
2022-08-19 10:40:25 +02:00
Jami Kettunen 255e69be5e
pmb.parse.bootimg: Add preliminary support for header v2 (MR 2194)
This includes setting header_version="2" as well as dtb_offset
according to the input boot.img when header v2 is detected.

Also adds the following previously missed deviceinfo_attributes:

* "header_version"
* "bootimg_custom_args"

And fixes failing tests now that header_version is always parsed from
boot.img files.
2022-07-18 14:49:16 -07:00
Minecrell f1cbcb7b3b
pmb: flasher: frontend: don't fail if kernel config cannot be found (MR 2184)
When using a kernel from Alpine the kernel configuration cannot be
found in pmaports. We cannot check the kernel config for missing
options in that case, but that's no reason to break the flasher
entirely.
2022-06-06 09:52:17 +02:00
Clayton Craft 9901cb31ea
pmb.parse.depends: ignore conflict dependency that no longer exists (MR 2181)
So it looks like some packages might be mentioned in the "depends=" of
other packages and marked as conflict, for example:
    depends="!foo"

..and the conflicting package can be dropped from the repo entirely if
it is no longer needed, however the reference to it in `depends=` still
exists. This handles that situation by just ignoring the missing package
if it is only a conflicting dependency.

fixes https://gitlab.com/postmarketOS/pmaports/-/issues/1525
2022-05-12 22:26:53 -07:00
Newbyte 9a74848f19
pmb.parse.kconfig: print a helpful error message on invalid kconfig name (MR 2172)
Currently pmbootstrap prints a stacktrace if there's no . in the
kernel configuration name. Let's make it more helpful.
2022-04-02 16:36:17 +02:00
Bart Ribbers 1eef7fbdeb
pmb.parse.kconfig: implement UEFI kconfig check (MR 2169) 2022-03-16 09:25:00 +01:00
Oliver Smith 301f0995fb
pmb.parse.kconfig.check_file: fix broken f-string (MR 2169) 2022-03-16 09:24:55 +01:00
Laszlo Molnar 2e39912790
pmb.parse.arguments: add zap --all flag (MR 2117)
Fixes: #1692
2022-02-28 17:09:21 +03:00
Marian Stramm 80c988236f
pmb.install: add support for creating btrfs root filesystem (MR 2155)
in preparation for reversible upgrades, this change adds the option of
creating a btrfs root filesystem
2022-01-29 11:47:29 +03:00