Do not crash in pmaports.git CI for merge requests where a different
repository was cloned as "origin". Fixes:
"ERROR: pmaports: could not find remote name for URL
'https://gitlab.com/postmarketOS/pmaports.git' in git repository:
/builds/plata-gl/pmaports"
Migrate to workdir version 5 and move already built packages into the edge
channel subdir, for example:
$WORK/packages/x86_64/hello-world-1-r5.apk
to:
$WORK/packages/edge/x86_64/hello-world-1-r5.apk
The build.postmarketos.org code has already been adjusted to find built
packages in either directory structure.
Use mirrordir_pmos and mirrordir_alpine from channels.cfg to generate
the mirror URLs for postmarketOS and Alpine, which get written to
/etc/apk/repositories and which postmarketOS uses to download the
APKINDEX files.
Remove hardcoded "master" at the end of the postmarketOS mirror and use
mirrordir_pmos instead (which is "master" for the edge channel). Let the
postmarketOS mirror end in a '/' for consistency with the Alpine mirror
in pmb/config/__init__.py.
Remove obsolete --alpine-version. To experiment with a different Alpine
version, one should pass a custom --config-channels from now on.
Ask for release channel and switch pmaports branch to the related branch
defined in channels.cfg.
Store in pmbootstrap.cfg whether the user chose a channel (boolean). If
the user did not choose a channel yet, suggest the recommended channel
from channels.cfg (currently "edge").
Prepare to base postmarketOS on Alpine stable by parsing the new
channels.cfg file in pmaports.git, that describes which channel
needs which branches and mirror dirs from postmarketOS and Alpine.
Use the information in pmb.helpers.git.get_branches_official() first,
more is coming in follow-up commits.
Read the file from origin/master, so we get the latest fetched version
even if the last checked out master branch is not up-to-date (think of
currently checked out release branch instead of master, master will
never be updated to point to latest origin/master). Allow to override
the file with a new --config-channels parameter.
Related: https://postmarketos.org/channels.cfg
Let apkbuild-lint check options again, and pass the pmbootstrap specific
options with the new CUSTOM_VALID_OPTIONS variable. Add a test case and
adjust pmb.helpers.lint.check to return the output of apkbuild-lint, so
we can properly test it.
Related: https://gitlab.alpinelinux.org/Leo/atools/-/merge_requests/28
Fixes: pmaports#553
The `mesa-dri-swrast` package was merged into `mesa-dri-gallium` in Alpine.
(See 298e20d04f)
The old alias might be removed in the future, so change the generated APKBUILDs
to depend on `mesa-dri-gallium` instead.
Don't assume that the device selected in "pmbootstrap init" is in the
testing subdir anymore. The default device, qemu-amd64, was recently
moved to main, so this does not hold up anymore. Instead, use
pmb.helpers.pmaports.find to figure out the currect dir.
No build is necessary if pmaport can't be built for given arch.
pmbootstrap must use Alpine's binary package in that case, even if the
pmaport version is higher than Alpine's binary package version.
Fixes: #1897
The --no-depends option is supposed to stop pmbootstrap if it was
instructed to build a package, but a dependency must be built first. So
far, this only covers the case if there is no binary package for a dependency.
Make it stop if the binary package exists, but is outdated, too.
Fixes: #1895
Use the timestamp of .git/FETCH_HEAD in each git repository, to
determine if too much time has passed since the last fetch/pull.
Modify pmb.helpers.git.clone, so FETCH_HEAD is always created if it does
not exist (because "git clone" would not create it).
Related: #1829
Extend "pmbootstrap status" with checks for all git repositories,
derived from relevant checks in pmb.helpers.git.pull (using shared
code):
* on official branch
* workdir is clean
* tracking proper remote
* up to date
Related: #1829
In the future, device ports will be located in a subdirectory
below device/... (e.g. device/testing/device-...).
Replace all occurrences of device/* with a glob that checks the
subdirectories instead.
Note: To ensure that this always works properly we should also add some
checks that all devices are indeed located under one of the supported
subdirectories (i.e. testing/community/main).
Change the glob for pmaports to <aports>/**/APKBUILD.
This allows using subdirectories for organization outside of device/
as well.
In general, we should not assume that pmbootstrap considers empty
package directories (without APKBUILD in them). With upcoming changes
this is no longer the case, so lets create an empty file for them.
The QEMU device ports no longer make use of the linux-postmarketos-
stable/mainline kernels, causing the test to fail with
RuntimeError: Selected kernel (stable) is not configured for device qemu-amd64.
Please run 'pmbootstrap init' to select a valid kernel.
Let's configure the test to use the "virt" kernel.
Just counting the number of tries does not prevent the QEMU tests
from running long periods of time. In some situations, the ssh command
will keep trying to reach the destination almost indefinitely.
To make the 5 minutes timeout work correctly we need to:
- Add an appropriate ConnectTimeout to the ssh command
- Don't sleep for another second if ssh already took several seconds
- Check the elapsed time instead of setting a fixed amount of tries
mesa-dri-swrast and mesa-dri-virtio are both provided by mesa-dri-gallium
now, so this option does not have much use anymore. With both selections,
exactly the same packages are installed.
Work around old rust version in Alpine's armhf repository (builder
currently stuck?) compared to other arches. Armv7 is fine, so switch to
that and avoid this error:
error[E0514]: found crate `std` compiled by an incompatible version of rustc
Add a helper class that provides readline-based tab completion and an
extra optional argument "completion_choices" with possible completion
targets.
While at this, also improve paramters documentation for ask() func.
Add new "pmbootstrap status" command, which does a quick health check
for the work dir. As first health check, verify that the chroots are not
too old. Replace the reminder text at the end of "pmbootstrap init" to
tell users to run "pmbootstrap status" instead of "pmbootstrap zap" once
a day before working with pmbootstrap.
Related: #1829
Whenever initializing new chroots, save the date in $WORK/workdir.cfg.
Add pmb.config.workdir.chroots_outdated() to check if it's time to zap
the chroots or not (since we don't update them automatically). Mark them
as outdated after two days.
This will be the first check in "pmbootstrap status" (future patches).
Related: #1829
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
Replace the "Add topdir to import path" boilerplate lines in each test
file with a simple "import pmb_test". Use the "# noqa" comment if
"pmb_test" is not used further in the test file, so flake8 does not
complain about an unused module.
Make the path to the testdata available as pmb_test.const.testdata, and
use pmb.config.pmb_src to access the topdir in all tests.
This is in preparation for new "pmbootstrap status" related tests, which
will have shared test code in test/pmb_test/. Also, this makes the
pmbootstrap codebase more consistent with the bpo codebase, which has a
similar "import bpo_test" mechanism.
At the moment we have a simple subpkgdesc() function that can only
parse "pkgdesc" from subpackages, without support for any variables.
But we have a quite nice variable parser now that can be extended
to work for subpackages.
Simply put this works by:
- Finding the lines that belong to the subpackage function
- Stripping indentation (tab)
- Parsing relevant attributes similar to the apkbuild() function
The "subpackages" in the parsed APKBUILD are replaced by a dict
of subpkgname: {"pkgdesc": "...", "depends": "..."} which are
parsed from the subpackage function (if found).
This makes it possible to get the "depends" of a subpackage.
At the moment we parse all attributes, split them, and eventually
join them back together for variable replacement.
Replacing variables immediately after parsing (before splitting)
has several advantages:
- No need to handle different value types
(e.g. lists by joining them every time they are accessed)
- Variables like depends="$depends ..." are handled directly
by the variable parser
- APKBUILDs are shell scripts, so we match abuild more closely
if variables defined later do not affect previous attributes
Add a shortcut for "git pull --ff-only" in all repositories cloned by
pmbootstrap (currently pmaports and aports_upstream, new pmdevices
repository coming soon).
'pmbootstrap pull' will only update the repositories, if:
* they are on an officially supported branch (e.g. master)
* the history is not conflicting (fast-forward is possible)
* the git workdirs are clean
Otherwise it shows the user a descriptive message about what to do. The
list of supported branches is only "master" right now, and will be
extended in later commits, so we can have a stable branch for pmaports
based on Alpine's releases. More about that in the project direction
2020 issue.
Closes: #1858
When defining a new kernel subpackage with a "-" in it
(e.g. $pkgname-kernel-mainline-modem), then pmbootstrap is unable to
find the function that builds the subpackage:
ERROR: Could not find subpackage function, no line starts with 'kernel_mainline-modem() {'
This is because it assumes that a $pkgname-kernel-<name> subpackage
is built by a kernel_<name> function, but this does not have to be the case.
We should really respect the name of the subpackage function that
is specified when defining the subpackage, but unfortunately it is
stripped away in cut_off_function_names().
For now let's fix this by replacing "-" with "_", but ideally the
APKBUILD parser API should be refactored to expose the subpackage
function in the future.
Install rust (rustc) in native chroot, if it is in the APKBUILD's
build dependencies. Add a test to verify that crossdirect + rust works
as expected.
Closes: #1861
Oliver Smith
Daniele Debernardi
Minecrell
Alexey Min
Luca Weiss