Commit Graph

2245 Commits

Author SHA1 Message Date
Oliver Smith 9975d373b0
Bump copyright to 2023 2023-01-22 19:18:06 +01:00
Oliver Smith faccbdc2eb
qemu: ssh hostfwd on 127.0.0.1 instead of 0.0.0.0
Make it accessible over localhost only. The idea is that users can login
into the VM on their own PC while developing, not that it's exposed over
the network. If somebody needs to have hostfwd on another IP, send a
patch to make this configurable via argparse.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221221220305.1809-1-ollieparanoid@postmarketos.org%3E
2023-01-20 15:28:07 +01:00
Clayton Craft 917ff92f63
pmb.qemu.run: fix 3d accel on amd64
Recent changes to qemu and Alpine packaging now require using the
virtio-vga-gl device and installing -gl packages to get virglrenderer
support.

Without this, wlroots fails to get an EGL context (among other problems
you'd expect by not having a useful GPU around...)

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221210185021.3546-1-clayton@craftyguy.net%3E
2023-01-20 15:28:07 +01:00
Jane Rachinger 42feaf0d49
pmb.build.other: do not copy leftover abuild dirs
Running abuild on the host directly creates directories in the
aport where it gets built. Interrupting abuild results in those
working-dirs not getting deleted.
We don't want to copy those entries to our builder.

It's only really noticeable if pmbootstrap tries to copy a broken
symlink in src/ and thus fails.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221207205201.22139-1-jane400@bingo-ev.de%3E
2023-01-20 15:28:02 +01:00
Oliver Smith 9d93d34b65
Prepare 1.50.1 release 2022-12-05 08:19:27 +01:00
Weijia Wang a56838fb45
pmb/parse/arch.py: add arm64 -> aarch64 to mapping
This allows pmbootstrap to work on aarch64-darwin systems.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C167000939722.18650.8735853729695007724-0@git.sr.ht%3E
2022-12-05 08:19:27 +01:00
Oliver Smith bb5399d6d5
pmb.config.apk_tools_min_version: add alpine 3.17 2022-11-27 19:10:05 +01:00
Sicelo 4b339763db
pmb.parse.kconfig: update version for MEMCG_SWAP change
Without the `_rc1` tag, building kernel results in
"linux-nokia-n900/config-nokia-n900.armv7: CONFIG_MEMCG_SWAP should be set. See
<https://wiki.postmarketos.org/wiki/kconfig#CONFIG_MEMCG_SWAP> for details"
message.

See also https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2220#note_1146219177

Signed-off-by: Sicelo A. Mhlongo <absicsz@gmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
2022-11-22 07:43:35 +01:00
Oliver Smith 757d2a0bff
Prepare 1.50.0 release 2022-11-20 16:43:03 +01:00
Oliver Smith aeeeb826fb
CI: fix typo in pmOS CI url 2022-11-20 16:38:59 +01:00
Oliver Smith 147b3ce4b9
pmb ci: fix typo in URL
Fix a typo in the URL https://postmarketos.org/pmb-ci that is being used
to detect if a script is compatible with 'pmbootstrap ci' or not.
2022-11-20 16:38:37 +01:00
Luca Weiss 0f6c6238f9
pmb.qemu.run: stop forcing bios for riscv64
We decided now not to force the bios firmware being used for riscv64 but
instead just use the one built into QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221118203424.106861-1-luca@z3ntu.xyz%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 315621d5b8
pmb ci: add --fast argument
Make it easy to only run the fast tests.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-3-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 3fd22104a8
pmb ci: error on using --all with script names
You can either say you want all scripts, or give a list of script names,
not both. Add it this way and not with an add_mutually_exclusive_group,
as I'll add a add_mutually_exclusive_group in the next patch to only
specify --all or --fast, but having --fast with script names is fine.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-2-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:11 +01:00
Oliver Smith 9a84ad20b1
pmb ci: fix arg desc for --all
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111072354.3431-1-ollieparanoid@postmarketos.org%3E
2022-11-20 15:35:05 +01:00
vaino ed7b0273f5
flasher: heimdall: take depends from pmaports.cfg
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Co-developed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221114115705.1107886-1-vaino@vke.fi%3E
2022-11-16 09:01:32 +01:00
Caleb Connolly ab0aa7f956
pmb: sideload: wait for apk database lock
When installing the APK wait for a while for the APK database lock,
services like apk-polkit-server sometimes hog it for a while but there's
no point bailing out immediately.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221111212251.1360612-1-kc@postmarketos.org%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 4a6c5657d5
pmb.parse.kconfig: don't enforce non-core checks for testing devices
Currently when any device does not conform to the options they declare,
we fail the whole kconfig check.

Now that we start requiring more options, especially with
pmb:kconfigcheck-community it makes sense to relax these restrictions so
we're more free to edit kconfig options and don't have to adjust all
testing devices that may or may not be properly maintained.

As a side effect this patch makes it practically impossible to make
kconfig check actually fail for any testing device which might not be
optimal. If these use cases appear in the future we will want to adjust
pmbootstrap to allow for that.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-2-luca@z3ntu.xyz%3E
2022-11-12 14:28:41 +01:00
Luca Weiss 381a1ca907
pmb.parse.kconfig: add wireguard, filesystems & more to community check
Add wireguard options, supported file systems and some extra options to
the community kconfig check.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105074432.13804-1-luca@z3ntu.xyz%3E
2022-11-12 14:28:40 +01:00
Oliver Smith 6f45a5d5fb
Prepare 1.49.0 release 2022-11-09 09:19:33 +01:00
Oliver Smith 30055c14d2
flasher: fastboot: take depends from pmaports.cfg
Adjust to avbtool now being part of android-tools in alpine edge.
Instead of trying to install both (which fails on edge), take the
dependencies from a new pmaports.cfg variable
supported_fastboot_depends, which only contains android-tools in
pmaports.git master branch.

Related: https://postmarketos.org/pmaports.cfg
2022-11-09 09:19:33 +01:00
Oliver Smith b64641bb1c
pmb.flasher.init.install_depends: new function
Move logic to install depends into one shared function.
2022-11-09 09:19:32 +01:00
Oliver Smith 04f8f59208
pmb.config.required_programs: add tar
Add tar as it is required for 'pmbootstrap ci' to get the source into
the pmbootstrap chroot.

Reviewed-by: Caleb Connolly <kc@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-2-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:32 +01:00
Oliver Smith f8d186e776
pmbootstrap ci: fix error with deleted files
Check if each file listed by the two git ls-files commands still exists,
and only add the existing ones to the tarball. Otherwise it would crash
when a file has been deleted without having the change commited yet.

Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221031111614.1377-1-ollieparanoid@postmarketos.org%3E
2022-11-09 09:19:26 +01:00
Newbyte 397225667f
helpers/envkernel.sh: rename yaml-lint to yamllint
There's no package called yaml-lint in Alpine. Presumably, the intent
was to install yamllint here.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221105132937.34008-1-newbie13xd@gmail.com%3E
2022-11-05 17:55:21 +00:00
Oliver Smith e8b0b4ba78
aportgen: don't fail if binary ver < APKBUILD ver
The typical workflow for upgrading cross/gcc-* is:

$ cd pmaports/cross
$ pmbootstrap aportgen gcc-*

Currently this is failing because the APKBUILD has been updated in
Alpine for gcc, but gcc has not been built for all arches yet. This
shouldn't prevent us from generating the proper updated APKBUILDs in
pmaports so just print a note and don't fail here.

I'm committing this directly to master as this currently breaks
test/test_aportgen.py::test_aportgen.
2022-11-02 21:09:28 +01:00
Dzmitry Sankouski 6a21898ce4
envkernel: add dependencies for running 'make dtbs_check'
Mainline device-tree patches development require linting.
Install needed packages to be able to perform dtbs_check task.

Signed-off-by: Dzmitry Sankouski <dsankouski@gmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221102160913.1896756-1-dsankouski@gmail.com%3E
2022-11-02 21:09:27 +01:00
Oliver Smith 57359bfd51
CI: flake8: remove ignore for W605
This was for 'invalid escape sequence' (even though the comment has W604
instead of W605). After removing it, flake8 runs through fine. We
probably fixed whatever invalid escape squence we had earlier while
doing f-string related refactoring.

Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221028074119.3309-1-ollieparanoid@postmarketos.org%3E
2022-11-02 21:09:27 +01:00
Luca Weiss 4771fbac65
pmb.qemu.run: support riscv64
We're using the standard virt machine, adding virtio-gpu-pci for
graphics and are using the bios firmware that is installed from the
device package, instead of the one built-in to QEMU.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-2-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Luca Weiss 0624a1ae33
pmb.qemu.run: replace -nic option with -netdev and -device
On qemu-system-riscv64 the -nic option doesn't seem to work correctly.

  qemu-system-riscv64: warning: requested NIC (anonymous, model virtio-net-pci) was not created (not supported by this machine?)

Using -netdev and -device provides the same functionality and also works
on riscv64.

Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20221029114536.100268-1-luca@z3ntu.xyz%3E
2022-11-02 21:09:27 +01:00
Oliver Smith 38f850161c
b4-config: linkmask: encode < and > chars
Encode < as %3C and > as %3E in the linkmask. This gets used in the
"Link: ..." line written to commit messages. Reasoning is that at least
gnome-terminal will break the link at the < otherwise, making it harder
to right click and copy the link to paste it into another VM to open it
in a browser. That's how everybody does it, right?

Use %% instead of % for proper escaping with python's %-formatting.
2022-11-02 21:09:22 +01:00
Oliver Smith 094bf38abb
b4-config: new file
Add a config, so the 'b4' tool from kernel tools can be used to get
Reviewed-By: etc. trailers into the commit messages. The tool only
supports lore.kernel.org, however I've written a proof of concept for
using this with SourceHut mailing lists too. See the wiki page for
details: https://postmarketos.org/patch-review
2022-11-01 15:14:31 +01:00
Martijn Braam 5042a947e5
README: Document git commands for setting the git-send-email defaults
It's possible to set the default To: address and subject prefix in the
git config of the local checkout. This makes the workflow a bit easier
and makes sure the subject starts with [PATCH pmbootstrap] instead of
the regular [PATCH].
2022-10-29 15:36:20 +01:00
Oliver Smith c1e4790947
build.yml: add missing sources line
Add the sources line, so CI should work for patches submitted to the
mailing list.
2022-10-28 09:06:18 +02:00
Oliver Smith 1ef933e3be
README: add ML subscribe link 2022-10-26 00:36:34 +02:00
Alexander Martinz 89c2ff0926
pmb.parse.kconfig: explicitly add dependency on SWAP
Commit d8f2f20186 removed
the requirement of selecting MEMCG_SWAP for Kernels >= 6.1.

However, it did not fully account for the behavior change,
as MEMCG_SWAP depends on both MEMCG and SWAP.

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-25 23:18:47 +02:00
Oliver Smith 3bee7222d8
README: add link to issues 2022-10-25 22:49:23 +02:00
Oliver Smith d61afe99eb
README: add new line
Just a detail, but given that the rest of this section is organized like
title: (new line) information, it looks more consistent.
2022-10-25 22:45:35 +02:00
Oliver Smith 1df2160129
README: update development section
* Mention that upstream is on SourceHut now
* Move it to the top, so people see this immediatelly
* Modernize instructions for running CI
2022-10-25 22:30:29 +02:00
Oliver Smith 39d75445b5
README: add line breaks
Get rid of very long lines, break them at 80 characters except if there
is a link or command-line output that is longer.
2022-10-25 22:30:29 +02:00
Oliver Smith 7914c11d1c
README: remove outdated information
* Links in the top row point to some very old blog posts and to the
  devices wiki page. Remove them, the first line has a link to
  postmarketos.org where the user should be able to get all relevant
  information about postmarketOS if they don't know this project.

* "Package build scripts live in the pmaports repository now.", this
  was useful shortly after we made the change, but it has been like this
  for years now. Again, the postmarketOS homepage lists where other
  related source code is, in case the user doesn't know.

* 2 GB of RAM recommened for compiling: this was a rule of thumb for
  when all postmarketOS packages had to be compiled from source. This is
  not the case anymore, we have a binary repository. Depending on which
  package you want to compile, you don't need as much RAM. And some
  users who just want to run 'pmbootstrap install' don't need to build
  anything at all. I think this is more confusing than helpful, so
  remove it.

* Kernels based on grsec patchset are not supported. I don't think
  anyone will try this, this was more relevant when the patchset was
  still distributed freely and you could actually use it in Alpine
  Linux (by default even?) and Arch Linux. An artefact of the past.
2022-10-25 22:30:28 +02:00
Oliver Smith 81f0d669c8
CI: add note about running scripts locally 2022-10-25 20:31:19 +02:00
Oliver Smith 0bcd58765c
pmbootstrap ci: new command
Add a new command that makes running CI scripts easy. The user goes to
the git repository of choice, which has CI scripts written in a certain
format, and then runs 'pmbootstrap ci' to get an interactive selection
of which of the available scripts to run (or "all"). Specifying one or
multiple scripts on the command-line is also possible, e.g.

$ pmbootstrap ci flake8
$ pmbootstrap ci shellcheck flake8 pytest
$ pmbootstrap ci --all

pmbootstrap then either runs the selected scripts in a chroot (and
installs dependencies as defined at the beginning of the CI scripts), or
natively (with checks inside the scripts for having dependencies
installed). Running natively is needed for .ci/pytest.sh in this
pmbootstrap.git repository, as pmbootstrap can't run inside pmbootstrap.
Running natively or in chroot is defined in an "# Options: " comment
inside the script file.

Documentation for this command and how script files look like:
https://postmarketos.org/pmb-ci
2022-10-25 20:31:19 +02:00
Oliver Smith 6ac39d17e7
pmb/helpers/frontend: order imports alphabetically 2022-10-25 20:31:19 +02:00
Oliver Smith 4b8a0db5bc
CI: replace gitlab CI with sourcehut CI
Replace .gitlab-ci.yml with a minimal .build.yml that runs a script in
the .ci dir for each task. Each of these scripts runs as root first,
installs dependencies, and then drops rights to the testuser and runs
the actual test. I went with this design, so we can add a
'pmbootstrap ci' command in following patches that run these in a
pmbootstrap chroot (see pmaports#2169).

Looking at flake8, we currently ignore W504 and W604. Would be nice to
fix these instead, but let's do that in another patch.

I've added a minver package to Alpine, so we don't need to install it
form pip anymore. Use minver's new --lint argument to simplify the
script, and as recommended by minver itself, use --no-parse-comments as
speed optimization since we don't have minver specific comments.

Using doas instead of sudo in build.yml would be nice, but this doesn't
work out of the box. Can be changed in the future.

Add a note to the pytest script, that 'pmbootstrap log' can be used to
follow the log while tests are running.
2022-10-25 20:31:19 +02:00
Oliver Smith 4428c7bcdc
pmbootstrap log: show testsuite log too
When running the testsuite, most logging gets written to a separate
log_testsuite.txt file. Check if it exists, and if so, instruct tail to
print its output as well. This allows immediatelly figuring out what the
testsuite is doing without manually attaching to log_testsuite.txt
(which I often did while running the testsuite).
2022-10-25 20:31:14 +02:00
Newbyte 8770aba287
helpers/envkernel.sh: return 0 from fish_compat if --fish is not present (MR 2221)
Right now this returns 1 when envkernel.sh is sourced for some reason. I
don't understand why exactly, but this seems like a sensible solution to
me.

Closes https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2133
2022-10-25 08:50:06 +02:00
Alexander Martinz 2ca4518799
pmb.parse.kconfig: drop remaining "_rc1" references (MR 2220)
As discussed at: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2220#note_1146631834

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-24 17:36:34 +02:00
Alexander Martinz d8f2f20186
pmb.parse.kconfig: fix container check for kernels >= 6.1 (MR 2220)
Kernel 6.1 dropped CONFIG_MEMCG_SWAP[1].

[1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e55b9f96860f6c6026cff97966a740576285e07b

Signed-off-by: Alexander Martinz <amartinz@shiftphones.com>
2022-10-24 11:14:47 +02:00
Minecrell 3b5492d91e
pmb: build: envkernel: Fix bind mount detection (MR 2217)
At the moment the "envkernel.sh hasn't run, assuming the kernel was
cross compiled on host and using current dir as source" code path
triggers even when using envkernel.sh, which works somewhat but
requires sourcing envkernel.sh again after each invocation of
"pmbootstrap build --envkernel ...".

The reason is that os.path.ismount() does not work for bind mounts
(see https://bugs.python.org/issue29707). There is a workaround for
that already in pmbootstrap but it is not used here for some reason.
2022-10-20 11:41:27 +02:00