Commit Graph

236 Commits

Author SHA1 Message Date
Oliver Smith 5ff0bbe4b2
test/static: enforce E501 length check iteratively (MR 1993)
Enforce the E501 length check of max 79 characters for all files where
this test passes already. We can add more to the list as we adjust them
and eventuelly require the check for all files.
2020-11-23 11:31:57 +01:00
Johannes Marbach 5dea31058d
Add option to specify extra free space in pmbootstrap init (MR 1989)
This adds a new commandline flag -E / --extra-space for
specifying the amount of additional space to be added to
the image size to work around cases where the automatically
determined size turns out to not actually be enough.

The value is also asked for in the "Additional options"
section of the interactive mode.

Fixes: #1904
2020-11-17 20:07:01 +01:00
timbz 5e01477d64
test: add deviceinfo kernel suffix parsing (MR 1986)
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
2020-11-10 09:15:52 +03:00
Oliver Smith 0dfe489b78
pmbootstrap aportgen: replace arch=all with native (MR 1965)
Packages like binutils-*, busybox-static-*, gcc-*, grub-efi-*, musl-*
are only needed for the native architecture during cross compilation.
Don't bother with trying to build them for other arches to save time and
to avoid getting stuck frequently at "armv7/binutils-aarch64" etc.

A few people like to use pmbootstrap on aarch64 hosts (e.g. PineBook
Pro), so let's make it available for aarch64 again when we can build
aarch64 packages natively in CI and bpo. (They do get stuck there right
now, because of qemu user emulation.)

Related: https://gitlab.com/postmarketOS/build.postmarketos.org/-/issues/75
2020-09-21 17:48:34 +02:00
Antoine Fontaine 1b5120e24f
test/testcases_fast.sh: correct pmaports location (MR 1974)
This makes the script ask pmbootstrap what is the correct directory
instead of hardcoding the default value.
2020-09-12 10:35:37 +02:00
Ralf Rachinger 72e24f7f96
Use $XDG_CONFIG_HOME instead of hardcoded path (MR 1969) 2020-08-23 13:51:42 +02:00
Luca Weiss 86d61b8012
pmb.parse.bootimg: detect mediatek header (MR 1955)
Some Mediatek devices have a special 512-byte header around the zImage
which must be generated so the device boots.

Support for that exists for a while in postmarketOS but detection was
missing. Add that.
2020-08-21 18:33:30 +02:00
Oliver Smith 7c26c1e9e5
test_crossdirect_rust: run on stable channel (MR 1966)
This test is failing with Alpine edge, because x86_64 has a different
rustc version than armv7:
	found crate `std` compiled by an incompatible version of rustc

Run the test with the latest Alpine stable version instead, where it is
more likely that rustc has the same version across all arches.
2020-08-05 10:38:23 +02:00
Oliver Smith d8615a9cae
pmbootstrap install: support _pmb_recommends (MR 1962)
Let UI meta-packages specify apps in "pmb_recommends" to be explicitly
installed by default, and not implicitly as dependency of the UI
meta-package ("depends"). Therefore make these apps uninstallable,
without removing the meta-package.

Add pmbootstrap install --no-recommends to disable this feature.
2020-07-20 14:29:06 +02:00
Pellegrino Prevete 95762a229a
pmb/config/__init__.py: add CONFIG_USER_NS (for bubblewrap/flatpak) (MR 1943)
Also fix tests
2020-07-14 23:19:50 +03:00
Oliver Smith 05c013536d
pmb.config.cipher: set default to aes-xts-plain64 (MR 1958)
Replace aes-cbc-plain64 with the stronger cipher aes-xts-plain64.
CONFIG_CRYPTO_XTS is necessary for this, so require it in
"pmbootstrap kconfig check".

Related: https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#5-security-aspects
2020-07-11 13:24:21 +02:00
Oliver Smith 357c1acc82
test/test_aportgen.py::test_aportgen: armhf->armv7 (MR 1942)
Alpine's armhf edge build bot is stuck, so musl-1.1.24-r9.*.apk is
missing and the test fails. armhf is on it's way out, so replace it with
armv7.
2020-06-21 11:50:57 +02:00
Oliver Smith ea710caafe
pmb/aportgen/binutils.py: pass apkbuild-lint (MR 1942) 2020-06-21 11:50:57 +02:00
Oliver Smith 654cc56062
pmb/aportgen/core.py: no quotes around some keys (MR 1942)
Don't print quotes for pkgname, pkgver, pkgrel. This is required to pass
apkbuild-lint.
2020-06-21 11:50:57 +02:00
Oliver Smith c0b80b26b5
pmb.aportgen.core.rewrite: no lines with empty val (MR 1942)
Avoid apkbuild-lint errors like:
	[AL5]:APKBUILD:15:variable set to empty string
2020-06-21 11:50:52 +02:00
Minecrell 3ebb994206
pmb.parse: Respect provider_priority if multiple providers exist (MR 1945)
Alpine APKBUILDs have the concept of "provider priorities" that affect
the choice of the provider to install when multiple packages provide
a virtual package.

One use case for this is to allow installation of different firmware
versions. bq-paella can run unsigned firmware, therefore you have the
choice between using the original firmware from the manufacturer, or
a slightly newer version from Qualcomm for the Dragonboard 410c.

We add provides="firmware-qcom-msm8916-wcnss" (the "virtual package")
to both firmware-qcom-db410c-wcnss and firmware-bq-picmt-wcnss.
At this point, attempting to install "firmware-qcom-msm8916-wcnss"
would still fail with apk. (Because it does not know which provider
to install.)

To pick a default we can set e.g. provider_priority=100 for
firmware-qcom-db410c-wcnss (the slightly newer version).
In that case, firmware-qcom-db410c-wcnss should be installed by default.

However, the user can choose to do "apk add firmware-bq-picmt-wcnss"
to override the default choice in case of problems. In that case,
the conflicting firmware-qcom-db410c-wcnss will be automatically removed.

At the moment, pmbootstrap does not respect the "provider_priority" at all.
In the above case, it would always install "firmware-bq-picmt-wcnss"
during "pmbootstrap install" since that has the shortest name.

Extend the pmbootstrap code to pick a provider with the highest priority
(if any of the providers has a priority set).
2020-06-16 01:31:28 +03:00
Anjandev Momi 37b4af19fc
make boot partition size user-overridable (MR 1931)
Ask for the boot partition size during "pmbootstrap init" in the
additional options with a default of 128 MB, and allow to override it
with -B.
2020-06-02 10:48:03 +02:00
Oliver Smith d623913491
pmbootstrap init: (build => additional) options (MR 1931)
Rename "build options" to "additional options", so we can put non-build
related options there too.
2020-06-02 10:47:58 +02:00
Anjandev Momi 0ff9c5d471
pmbootstrap init: only display UIs for device arch (MR 1937)
Hide user interfaces that are not available for the selected device's
architecture.

Closes #1790
2020-06-02 09:18:39 +02:00
Oliver Smith 2d2ab1df56
testdata: update lg-mako deviceinfo (MR 1937)
Copy from current pmaports master, so pmbootstrap doesn't complain about
missing deviceinfo_codename when parsing it. Right now pmbootstrap
doesn't complain about this, but with the next patch it will attempt to
parse the file.

This replaces the year from 1337 to 2012 too, but let's rather have it
slightly more maintainable than introducing a diff for a funny reference.
2020-06-02 09:18:34 +02:00
Luca Weiss 6bb7f022b1
pmb: introduce deviceinfo_chassis (MR 1933)
"Chassis" defines the what kind of device a device is. Currently the
following standardized types exist:

  "desktop", "laptop", "convertible", "server", "tablet", "handset",
  "watch", "embedded", "vm", "container"

This property is exposed by org.freedesktop.hostname1 (e.g.
openrc-settingsd) and can be used by applications for example to display
appropriate strings instead of "About this phone" for non-phone devices.
2020-05-24 08:38:10 +02:00
Antoine Fontaine c5d0f76005
pmb.parse.kconfig: add tests (MR 1916) 2020-05-24 04:42:13 +02:00
Oliver Smith 7f60a6d782
chroot: put built packages into channel subdir (MR 1912)
Migrate to workdir version 5 and move already built packages into the edge
channel subdir, for example:
	$WORK/packages/x86_64/hello-world-1-r5.apk
to:
	$WORK/packages/edge/x86_64/hello-world-1-r5.apk

The build.postmarketos.org code has already been adjusted to find built
packages in either directory structure.
2020-05-17 08:08:45 +02:00
Oliver Smith 3ff0b18a08
pmb.chroot.init: verify chroot channel (MR 1912)
Refuse to work with a chroot that was created for another release
channel.
2020-05-17 08:08:45 +02:00
Oliver Smith ce433ae012
pmb.helpers.repo.urls: use channels.cfg mirrordirs (MR 1912)
Use mirrordir_pmos and mirrordir_alpine from channels.cfg to generate
the mirror URLs for postmarketOS and Alpine, which get written to
/etc/apk/repositories and which postmarketOS uses to download the
APKINDEX files.

Remove hardcoded "master" at the end of the postmarketOS mirror and use
mirrordir_pmos instead (which is "master" for the edge channel). Let the
postmarketOS mirror end in a '/' for consistency with the Alpine mirror
in pmb/config/__init__.py.

Remove obsolete --alpine-version. To experiment with a different Alpine
version, one should pass a custom --config-channels from now on.
2020-05-17 08:08:45 +02:00
Oliver Smith 6d64586b9f
test/test_repo.py: rename to test_helpers_repo.py (MR 1912)
This file is about testing pmb/helpers/repo.py, so make the name of the
test file consistent with other test/test_helpers_*.py files.
2020-05-17 08:08:45 +02:00
Oliver Smith 17f3b3c2f0
pmbootstrap init: ask for release channel (MR 1912)
Ask for release channel and switch pmaports branch to the related branch
defined in channels.cfg.

Store in pmbootstrap.cfg whether the user chose a channel (boolean). If
the user did not choose a channel yet, suggest the recommended channel
from channels.cfg (currently "edge").
2020-05-17 08:08:45 +02:00
Oliver Smith c616874443
pmb.helpers.git: parse channels.cfg (MR 1912)
Prepare to base postmarketOS on Alpine stable by parsing the new
channels.cfg file in pmaports.git, that describes which channel
needs which branches and mirror dirs from postmarketOS and Alpine.

Use the information in pmb.helpers.git.get_branches_official() first,
more is coming in follow-up commits.

Read the file from origin/master, so we get the latest fetched version
even if the last checked out master branch is not up-to-date (think of
currently checked out release branch instead of master, master will
never be updated to point to latest origin/master). Allow to override
the file with a new --config-channels parameter.

Related: https://postmarketos.org/channels.cfg
2020-05-17 08:08:44 +02:00
Oliver Smith f3466dcd3f
pmbootstrap lint: use CUSTOM_VALID_OPTIONS (MR 1934)
Let apkbuild-lint check options again, and pass the pmbootstrap specific
options with the new CUSTOM_VALID_OPTIONS variable. Add a test case and
adjust pmb.helpers.lint.check to return the output of apkbuild-lint, so
we can properly test it.

Related: https://gitlab.alpinelinux.org/Leo/atools/-/merge_requests/28
Fixes: pmaports#553
2020-05-10 16:35:35 +02:00
Daniele Debernardi 9c2d951fc9
pmb.aportgen.device: replace mesa-dri-swrast with mesa-dri-gallium (MR 1917)
The `mesa-dri-swrast` package was merged into `mesa-dri-gallium` in Alpine.
(See 298e20d04f)

The old alias might be removed in the future, so change the generated APKBUILDs
to depend on `mesa-dri-gallium` instead.
2020-04-28 09:23:09 +02:00
Minecrell 87f81de052
test: test_questions_ui_extras: add tests for UI extras (!1911)
Ensure that it does not fail with the "none" UI (it does at the moment).
2020-04-11 20:57:01 +02:00
Minecrell 6731d02c4b
test: test_questions_ui: ensure that we can select "none" (!1911) 2020-04-11 20:57:01 +02:00
Minecrell 7f0898f1c5
test: test_questions_ui: use local aports (!1911)
This allows us to test the extras UI implementation without
modifying some packages in pmaports.
2020-04-11 20:57:01 +02:00
Minecrell 181687a2d7
test: test_qemu_running_processes: ensure "none" UI boots correctly (!1911)
No special process needs to be running, SSH is good already :)
2020-04-11 20:56:57 +02:00
Oliver Smith 8dc8daa05a
test: adjust to device-qemu-amd64 move to main (!1910)
Don't assume that the device selected in "pmbootstrap init" is in the
testing subdir anymore. The default device, qemu-amd64, was recently
moved to main, so this does not hold up anymore. Instead, use
pmb.helpers.pmaports.find to figure out the currect dir.
2020-04-11 18:56:30 +02:00
Oliver Smith 44bd235f3e
test/test_build_package.py: fix new flake8 errors (!1910)
F540 f-string is missing placeholder

Apparently I like the new f-strings so much that I used them even where
they make no sense ;)
2020-04-11 18:56:25 +02:00
Oliver Smith b4a05cbcfb
pmb.build.is_necessary: fix case with Alpine's pkg (!1904)
No build is necessary if pmaport can't be built for given arch.
pmbootstrap must use Alpine's binary package in that case, even if the
pmaport version is higher than Alpine's binary package version.

Fixes: #1897
2020-04-04 13:19:03 +02:00
Oliver Smith 44cb06d345
pmb build --no-depends: stop on outdated pkgs too (!1900)
The --no-depends option is supposed to stop pmbootstrap if it was
instructed to build a package, but a dependency must be built first. So
far, this only covers the case if there is no binary package for a dependency.
Make it stop if the binary package exists, but is outdated, too.

Fixes: #1895
2020-04-01 09:22:35 +02:00
Minecrell b4c54636b3
test/testcases_fast.sh: stop hardcoding testing/ directory (!1899)
Take the change from pmaports that expands a glob for more flexibility.
2020-04-01 02:50:09 +03:00
Oliver Smith 9d17ed29e3
pmb aportgen binutils: add texinfo to makedepends (!1901)
The current version of binutils does not build anymore without texinfo.

Related: #1894
Related: pmaports!1114
2020-03-31 23:33:41 +02:00
Oliver Smith cfd93bc6ea
pmbootstrap status: check if git repo is outdated (!1882)
Use the timestamp of .git/FETCH_HEAD in each git repository, to
determine if too much time has passed since the last fetch/pull.
Modify pmb.helpers.git.clone, so FETCH_HEAD is always created if it does
not exist (because "git clone" would not create it).

Related: #1829
2020-03-14 19:15:32 +03:00
Oliver Smith c8526f2fcb
pmbootstrap status: basic checks for git repos (!1882)
Extend "pmbootstrap status" with checks for all git repositories,
derived from relevant checks in pmb.helpers.git.pull (using shared
code):
* on official branch
* workdir is clean
* tracking proper remote
* up to date

Related: #1829
2020-03-14 19:15:32 +03:00
Oliver Smith 6977f5f3e5
test.test_helpers_git.test_pull: move prep code (!1882)
Create pmb_test.git.prepare_tmpdir() with preparation code, so we can
use it in a "pmbootstrap status" related test without duplication.
2020-03-14 19:15:32 +03:00
Minecrell fb8de5a553
pmb: Look for device/* files in device/*/... instead (!1879)
In the future, device ports will be located in a subdirectory
below device/... (e.g. device/testing/device-...).
Replace all occurrences of device/* with a glob that checks the
subdirectories instead.

Note: To ensure that this always works properly we should also add some
checks that all devices are indeed located under one of the supported
subdirectories (i.e. testing/community/main).

Change the glob for pmaports to <aports>/**/APKBUILD.
This allows using subdirectories for organization outside of device/
as well.
2020-03-14 08:44:16 +01:00
Minecrell c399ff81a1
test/test_helpers_pmaports: create empty APKBUILDs for fake packages (!1879)
In general, we should not assume that pmbootstrap considers empty
package directories (without APKBUILD in them). With upcoming changes
this is no longer the case, so lets create an empty file for them.
2020-03-14 08:39:13 +01:00
Minecrell 903491f518
test/test_qemu_running_processes: use Alpine virt kernel (!1886)
The QEMU device ports no longer make use of the linux-postmarketos-
stable/mainline kernels, causing the test to fail with

  RuntimeError: Selected kernel (stable) is not configured for device qemu-amd64.
  Please run 'pmbootstrap init' to select a valid kernel.

Let's configure the test to use the "virt" kernel.
2020-03-14 08:05:32 +01:00
Minecrell 345425ef48
test/test_qemu_running_processes: make timeout actually work (!1886)
Just counting the number of tries does not prevent the QEMU tests
from running long periods of time. In some situations, the ssh command
will keep trying to reach the destination almost indefinitely.

To make the 5 minutes timeout work correctly we need to:
  - Add an appropriate ConnectTimeout to the ssh command
  - Don't sleep for another second if ssh already took several seconds
  - Check the elapsed time instead of setting a fixed amount of tries
2020-03-14 08:05:32 +01:00
Minecrell 320b2faa4c
pmb.qemu: remove QEMU mesa driver setup question (!1886)
mesa-dri-swrast and mesa-dri-virtio are both provided by mesa-dri-gallium
now, so this option does not have much use anymore. With both selections,
exactly the same packages are installed.
2020-03-14 08:05:32 +01:00
Oliver Smith 7c4db04b9d
test_crossdirect: s/armhf/armv7 (!1888)
Work around old rust version in Alpine's armhf repository (builder
currently stuck?) compared to other arches. Armv7 is fine, so switch to
that and avoid this error:

error[E0514]: found crate `std` compiled by an incompatible version of rustc
2020-03-14 01:07:02 +01:00
Minecrell 0997a46b3a
pmb: replace deviceinfo_date with deviceinfo_year, add to wizard (!1888)
See: https://gitlab.com/postmarketOS/pmaports/issues/447
2020-03-14 00:33:22 +01:00