Commit Graph

2350 Commits

Author SHA1 Message Date
Newbyte 162867a08c
config: Add BINFMT_ELF and BINFMT_SCRIPT
pmOS cannot boot without these two. There was recently someone in the
unofficial postmarketOS Telegram group who used some minimal config and
couldn't figure out why the initramfs couldn't start up, and these
options being disabled turned out to be the culprit. As such, let's
ensure these are enabled to make life easier for people setting up new
kernel configurations.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230128203716.152281-1-newbyte@postmarketos.org%3E
2023-02-26 19:42:05 +01:00
Oliver Smith a0cea6255a
pmb.config: mount go caching directories
Always set GOCACHE during build and let it point to pmbootstrap's work
dir. This has a similar effect as using ccache for C/C++.

Set GOMODCACHE conditionally: this is for caching the go modules (git
repositories) that get cloned during a build if they are not bundled
with the source. Usually APKBUILDs should cache them, but when using
pmbootstrap build --src, they would get downloaded over and over again.
Set GOMODCACHE automatically for --src and allow enabling/disabling this
manually with new --go-mod-cache / --no-go-mod-cache.

This speeds up multiple iterations of building the same package
significantly. I'm using it for:

$ pmbootstrap build postmarketos-mkinitfs --src=...

I've verified that using the same GOCACHE dir for multiple architectures
doesn't result in build artifacts for the wrong arch getting used.

Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230223064743.1773-1-ollieparanoid@postmarketos.org%3E
2023-02-26 19:37:30 +01:00
Oliver Smith 3ad159f98d
Prepare 1.51.0 release 2023-02-19 16:13:52 +01:00
Oliver Smith 98290f9191
pmbootstrap init: don't allow 'root' as username
While going over the wiki pages recently, I've noticed that somebody
added a note to not use "root" as username. I don't think many people
will attempt to do this, but let's catch it here in pmbootstrap with a
friendly message instead of mentioning it in the wiki.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230126071154.1689-1-ollieparanoid@postmarketos.org%3E
2023-02-19 16:13:52 +01:00
Oliver Smith e8a2ab2e98
check_binfmt_misc: print more helpful error
Instead of letting the mount -t binfmt_misc … command fail, and simply
telling the user that the command failed: rely on the following check
that prints a better error message on error. If it fails at this point,
the kernel option isn't enabled. Add two comments explaining why
check=False is used in this function while at it.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230123064516.1607-1-ollieparanoid@postmarketos.org%3E
2023-02-19 16:13:52 +01:00
Oliver Smith 9975d373b0
Bump copyright to 2023 2023-01-22 19:18:06 +01:00
Oliver Smith faccbdc2eb
qemu: ssh hostfwd on 127.0.0.1 instead of 0.0.0.0
Make it accessible over localhost only. The idea is that users can login
into the VM on their own PC while developing, not that it's exposed over
the network. If somebody needs to have hostfwd on another IP, send a
patch to make this configurable via argparse.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221221220305.1809-1-ollieparanoid@postmarketos.org%3E
2023-01-20 15:28:07 +01:00
Clayton Craft 917ff92f63
pmb.qemu.run: fix 3d accel on amd64
Recent changes to qemu and Alpine packaging now require using the
virtio-vga-gl device and installing -gl packages to get virglrenderer
support.

Without this, wlroots fails to get an EGL context (among other problems
you'd expect by not having a useful GPU around...)

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221210185021.3546-1-clayton@craftyguy.net%3E
2023-01-20 15:28:07 +01:00
Jane Rachinger 42feaf0d49
pmb.build.other: do not copy leftover abuild dirs
Running abuild on the host directly creates directories in the
aport where it gets built. Interrupting abuild results in those
working-dirs not getting deleted.
We don't want to copy those entries to our builder.

It's only really noticeable if pmbootstrap tries to copy a broken
symlink in src/ and thus fails.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221207205201.22139-1-jane400@bingo-ev.de%3E
2023-01-20 15:28:02 +01:00
Oliver Smith 9d93d34b65
Prepare 1.50.1 release 2022-12-05 08:19:27 +01:00
Weijia Wang a56838fb45
pmb/parse/arch.py: add arm64 -> aarch64 to mapping
This allows pmbootstrap to work on aarch64-darwin systems.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167000939722.18650.8735853729695007724-0@git.sr.ht%3E
2022-12-05 08:19:27 +01:00
Oliver Smith bb5399d6d5
pmb.config.apk_tools_min_version: add alpine 3.17 2022-11-27 19:10:05 +01:00
Sicelo 4b339763db
pmb.parse.kconfig: update version for MEMCG_SWAP change
Without the `_rc1` tag, building kernel results in
"linux-nokia-n900/config-nokia-n900.armv7: CONFIG_MEMCG_SWAP should be set. See
<https://wiki.postmarketos.org/wiki/kconfig#CONFIG_MEMCG_SWAP> for details"
message.

See also https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2220#note_1146219177

Signed-off-by: Sicelo A. Mhlongo <absicsz@gmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
2022-11-22 07:43:35 +01:00
Oliver Smith 757d2a0bff
Prepare 1.50.0 release 2022-11-20 16:43:03 +01:00
Oliver Smith aeeeb826fb
CI: fix typo in pmOS CI url 2022-11-20 16:38:59 +01:00
Oliver Smith 147b3ce4b9
pmb ci: fix typo in URL
Fix a typo in the URL https://postmarketos.org/pmb-ci that is being used
to detect if a script is compatible with 'pmbootstrap ci' or not.
2022-11-20 16:38:37 +01:00
Luca Weiss 0f6c6238f9
pmb.qemu.run: stop forcing bios for riscv64
We decided now not to force the bios firmware being used for riscv64 but
instead just use the one built into QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221118203424.106861-1-luca@z3ntu.xyz%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 315621d5b8
pmb ci: add --fast argument
Make it easy to only run the fast tests.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-3-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 3fd22104a8
pmb ci: error on using --all with script names
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 9a84ad20b1
pmb ci: fix arg desc for --all
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-1-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:05 +01:00
vaino ed7b0273f5
flasher: heimdall: take depends from pmaports.cfg
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Co-developed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221114115705.1107886-1-vaino@vke.fi%3E
2022-11-16 09:01:32 +01:00
Caleb Connolly ab0aa7f956
pmb: sideload: wait for apk database lock
When installing the APK wait for a while for the APK database lock,
services like apk-polkit-server sometimes hog it for a while but there's
no point bailing out immediately.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111212251.1360612-1-kc@postmarketos.org%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 4a6c5657d5
pmb.parse.kconfig: don't enforce non-core checks for testing devices
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.

Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.

As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 381a1ca907
pmb.parse.kconfig: add wireguard, filesystems & more to community check
Add wireguard options, supported file systems and some extra options to
the community kconfig check.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-1-luca@z3ntu.xyz%3E
2022-11-12 14:28:40 +01:00
Oliver Smith 6f45a5d5fb
Prepare 1.49.0 release 2022-11-09 09:19:33 +01:00
Oliver Smith 30055c14d2
flasher: fastboot: take depends from pmaports.cfg
Adjust to avbtool now being part of android-tools in alpine edge.
Instead of trying to install both (which fails on edge), take the
dependencies from a new pmaports.cfg variable
supported_fastboot_depends, which only contains android-tools in
pmaports.git master branch.

Related: https://postmarketos.org/pmaports.cfg
2022-11-09 09:19:33 +01:00
Oliver Smith b64641bb1c
pmb.flasher.init.install_depends: new function
Move logic to install depends into one shared function.
2022-11-09 09:19:32 +01:00
Oliver Smith 04f8f59208
pmb.config.required_programs: add tar
Add tar as it is required for 'pmbootstrap ci' to get the source into
the pmbootstrap chroot.

Reviewed-by: Caleb Connolly <kc@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-2-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:32 +01:00
Oliver Smith f8d186e776
pmbootstrap ci: fix error with deleted files
Check if each file listed by the two git ls-files commands still exists,
and only add the existing ones to the tarball. Otherwise it would crash
when a file has been deleted without having the change commited yet.

Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-1-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:26 +01:00
Newbyte 397225667f
helpers/envkernel.sh: rename yaml-lint to yamllint
There's no package called yaml-lint in Alpine. Presumably, the intent
was to install yamllint here.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105132937.34008-1-newbie13xd@gmail.com%3E
2022-11-05 17:55:21 +00:00
Oliver Smith e8b0b4ba78
aportgen: don't fail if binary ver < APKBUILD ver
The typical workflow for upgrading cross/gcc-* is:

$ cd pmaports/cross
$ pmbootstrap aportgen gcc-*

Currently this is failing because the APKBUILD has been updated in
Alpine for gcc, but gcc has not been built for all arches yet. This
shouldn't prevent us from generating the proper updated APKBUILDs in
pmaports so just print a note and don't fail here.

I'm committing this directly to master as this currently breaks
test/test_aportgen.py::test_aportgen.
2022-11-02 21:09:28 +01:00
Dzmitry Sankouski 6a21898ce4
envkernel: add dependencies for running 'make dtbs_check'
Mainline device-tree patches development require linting.
Install needed packages to be able to perform dtbs_check task.

Signed-off-by: Dzmitry Sankouski <dsankouski@gmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221102160913.1896756-1-dsankouski@gmail.com%3E
2022-11-02 21:09:27 +01:00
Oliver Smith 57359bfd51
CI: flake8: remove ignore for W605
This was for 'invalid escape sequence' (even though the comment has W604
instead of W605). After removing it, flake8 runs through fine. We
probably fixed whatever invalid escape squence we had earlier while
doing f-string related refactoring.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221028074119.3309-1-ollieparanoid@postmarketos.org%3E
2022-11-02 21:09:27 +01:00
Luca Weiss 4771fbac65
pmb.qemu.run: support riscv64
We're using the standard virt machine, adding virtio-gpu-pci for
graphics and are using the bios firmware that is installed from the
device package, instead of the one built-in to QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-2-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Luca Weiss 0624a1ae33
pmb.qemu.run: replace -nic option with -netdev and -device
On qemu-system-riscv64 the -nic option doesn't seem to work correctly.

  qemu-system-riscv64: warning: requested NIC (anonymous, model virtio-net-pci) was not created (not supported by this machine?)

Using -netdev and -device provides the same functionality and also works
on riscv64.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-1-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Oliver Smith 38f850161c
b4-config: linkmask: encode < and > chars
Encode < as %3C and > as %3E in the linkmask. This gets used in the
"Link: ..." line written to commit messages. Reasoning is that at least
gnome-terminal will break the link at the < otherwise, making it harder
to right click and copy the link to paste it into another VM to open it
in a browser. That's how everybody does it, right?

Use %% instead of % for proper escaping with python's %-formatting.
2022-11-02 21:09:22 +01:00
Oliver Smith 094bf38abb
b4-config: new file
Add a config, so the 'b4' tool from kernel tools can be used to get
Reviewed-By: etc. trailers into the commit messages. The tool only
supports lore.kernel.org, however I've written a proof of concept for
using this with SourceHut mailing lists too. See the wiki page for
details: https://postmarketos.org/patch-review
2022-11-01 15:14:31 +01:00
Martijn Braam 5042a947e5
README: Document git commands for setting the git-send-email defaults
It's possible to set the default To: address and subject prefix in the
git config of the local checkout. This makes the workflow a bit easier
and makes sure the subject starts with [PATCH pmbootstrap] instead of
the regular [PATCH].
2022-10-29 15:36:20 +01:00
Oliver Smith c1e4790947
build.yml: add missing sources line
Add the sources line, so CI should work for patches submitted to the
mailing list.
2022-10-28 09:06:18 +02:00
Oliver Smith 1ef933e3be
README: add ML subscribe link 2022-10-26 00:36:34 +02:00
Alexander Martinz 89c2ff0926
pmb.parse.kconfig: explicitly add dependency on SWAP
Commit d8f2f20186 removed
the requirement of selecting MEMCG_SWAP for Kernels >= 6.1.

However, it did not fully account for the behavior change,
as MEMCG_SWAP depends on both MEMCG and SWAP.

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-25 23:18:47 +02:00
Oliver Smith 3bee7222d8
README: add link to issues 2022-10-25 22:49:23 +02:00
Oliver Smith d61afe99eb
README: add new line
Just a detail, but given that the rest of this section is organized like
title: (new line) information, it looks more consistent.
2022-10-25 22:45:35 +02:00
Oliver Smith 1df2160129
README: update development section
* Mention that upstream is on SourceHut now
* Move it to the top, so people see this immediatelly
* Modernize instructions for running CI
2022-10-25 22:30:29 +02:00
Oliver Smith 39d75445b5
README: add line breaks
Get rid of very long lines, break them at 80 characters except if there
is a link or command-line output that is longer.
2022-10-25 22:30:29 +02:00
Oliver Smith 7914c11d1c
README: remove outdated information
* Links in the top row point to some very old blog posts and to the
  devices wiki page. Remove them, the first line has a link to
  postmarketos.org where the user should be able to get all relevant
  information about postmarketOS if they don't know this project.

* "Package build scripts live in the pmaports repository now.", this
  was useful shortly after we made the change, but it has been like this
  for years now. Again, the postmarketOS homepage lists where other
  related source code is, in case the user doesn't know.

* 2 GB of RAM recommened for compiling: this was a rule of thumb for
  when all postmarketOS packages had to be compiled from source. This is
  not the case anymore, we have a binary repository. Depending on which
  package you want to compile, you don't need as much RAM. And some
  users who just want to run 'pmbootstrap install' don't need to build
  anything at all. I think this is more confusing than helpful, so
  remove it.

* Kernels based on grsec patchset are not supported. I don't think
  anyone will try this, this was more relevant when the patchset was
  still distributed freely and you could actually use it in Alpine
  Linux (by default even?) and Arch Linux. An artefact of the past.
2022-10-25 22:30:28 +02:00
Oliver Smith 81f0d669c8
CI: add note about running scripts locally 2022-10-25 20:31:19 +02:00
Oliver Smith 0bcd58765c
pmbootstrap ci: new command
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.

$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all

pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.

Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
2022-10-25 20:31:19 +02:00
Oliver Smith 6ac39d17e7
pmb/helpers/frontend: order imports alphabetically 2022-10-25 20:31:19 +02:00
Oliver Smith 4b8a0db5bc
CI: replace gitlab CI with sourcehut CI
Replace .gitlab-ci.yml with a minimal .build.yml that runs a script in
the .ci dir for each task. Each of these scripts runs as root first,
installs dependencies, and then drops rights to the testuser and runs
the actual test. I went with this design, so we can add a
'pmbootstrap ci' command in following patches that run these in a
pmbootstrap chroot (see pmaports#2169).

Looking at flake8, we currently ignore W504 and W604. Would be nice to
fix these instead, but let's do that in another patch.

I've added a minver package to Alpine, so we don't need to install it
form pip anymore. Use minver's new --lint argument to simplify the
script, and as recommended by minver itself, use --no-parse-comments as
speed optimization since we don't have minver specific comments.

Using doas instead of sudo in build.yml would be nice, but this doesn't
work out of the box. Can be changed in the future.

Add a note to the pytest script, that 'pmbootstrap log' can be used to
follow the log while tests are running.
2022-10-25 20:31:19 +02:00