Whenever initializing new chroots, save the date in $WORK/workdir.cfg.
Add pmb.config.workdir.chroots_outdated() to check if it's time to zap
the chroots or not (since we don't update them automatically). Mark them
as outdated after two days.
This will be the first check in "pmbootstrap status" (future patches).
Related: #1829
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
Replace the "Add topdir to import path" boilerplate lines in each test
file with a simple "import pmb_test". Use the "# noqa" comment if
"pmb_test" is not used further in the test file, so flake8 does not
complain about an unused module.
Make the path to the testdata available as pmb_test.const.testdata, and
use pmb.config.pmb_src to access the topdir in all tests.
This is in preparation for new "pmbootstrap status" related tests, which
will have shared test code in test/pmb_test/. Also, this makes the
pmbootstrap codebase more consistent with the bpo codebase, which has a
similar "import bpo_test" mechanism.
At the moment we have a simple subpkgdesc() function that can only
parse "pkgdesc" from subpackages, without support for any variables.
But we have a quite nice variable parser now that can be extended
to work for subpackages.
Simply put this works by:
- Finding the lines that belong to the subpackage function
- Stripping indentation (tab)
- Parsing relevant attributes similar to the apkbuild() function
The "subpackages" in the parsed APKBUILD are replaced by a dict
of subpkgname: {"pkgdesc": "...", "depends": "..."} which are
parsed from the subpackage function (if found).
This makes it possible to get the "depends" of a subpackage.
At the moment we parse all attributes, split them, and eventually
join them back together for variable replacement.
Replacing variables immediately after parsing (before splitting)
has several advantages:
- No need to handle different value types
(e.g. lists by joining them every time they are accessed)
- Variables like depends="$depends ..." are handled directly
by the variable parser
- APKBUILDs are shell scripts, so we match abuild more closely
if variables defined later do not affect previous attributes
Add a shortcut for "git pull --ff-only" in all repositories cloned by
pmbootstrap (currently pmaports and aports_upstream, new pmdevices
repository coming soon).
'pmbootstrap pull' will only update the repositories, if:
* they are on an officially supported branch (e.g. master)
* the history is not conflicting (fast-forward is possible)
* the git workdirs are clean
Otherwise it shows the user a descriptive message about what to do. The
list of supported branches is only "master" right now, and will be
extended in later commits, so we can have a stable branch for pmaports
based on Alpine's releases. More about that in the project direction
2020 issue.
Closes: #1858
When defining a new kernel subpackage with a "-" in it
(e.g. $pkgname-kernel-mainline-modem), then pmbootstrap is unable to
find the function that builds the subpackage:
ERROR: Could not find subpackage function, no line starts with 'kernel_mainline-modem() {'
This is because it assumes that a $pkgname-kernel-<name> subpackage
is built by a kernel_<name> function, but this does not have to be the case.
We should really respect the name of the subpackage function that
is specified when defining the subpackage, but unfortunately it is
stripped away in cut_off_function_names().
For now let's fix this by replacing "-" with "_", but ideally the
APKBUILD parser API should be refactored to expose the subpackage
function in the future.
Install rust (rustc) in native chroot, if it is in the APKBUILD's
build dependencies. Add a test to verify that crossdirect + rust works
as expected.
Closes: #1861
Properly ignore comments at the end of lines, instead of assuming that
all lines below belong to the attribute:
subpackages="$pkgname-dev" # $pkgname-lang
Fixes build.postmarketos.org#61, where pmbootstrap would assume that a
random package provides "make", just because the word "make" is written
somewhere below subpackages=" in the APKBUILD and it is parsed
incorrectly.
While at it, also support the ' character for quotations and detect if
a quotation for a value was started, but there is no end quotation sign
in the rest of the file.
I've added tests, and manually checked that this did not introduce any
parsing bugs for all the APKBUILDs in pmaports.git, by running
'pmbootstrap apkbuild_parse' with the old and new code, and diffing the
result.
Make /home/pmos/build/.git point to the .git dir from pmaports.git, with
a symlink so abuild does not fail.
abuild expects the current working directory to be a subdirectory of a
cloned git repository (e.g. main/openrc from aports.git). If git is
installed, it will try to get the last git commit from that repository,
and place it in the resulting apk (.PKGINFO) as well as use the date
from that commit as SOURCE_DATE_EPOCH (for reproducible builds).
With that symlink, we actually make it use the last git commit from
pmaports.git for SOURCE_DATE_EPOCH and have that in the resulting apk's
.PKGINFO.
Fixes: #1841
When running pmbootstrap init, first select device vendor, then device
codename. Also fixed tests for new behavior and added some new ones for
new scenarios.
postmarketOS/pmaports!700 adds a new "deviceinfo_bootimg_dtb_second"
option that places the DTB in the "second" area of the Android boot
image.
Attempt to detect this automatically by checking the extracted
second binary for the FDT magic (0xd00dfeed).
Packages ending in -dev: just assume that the originating aport has the
same pkgname, except for the -dev at the end. Otherwise we may end up
with the wrong package.
For example, if something depends on plasma-framework-dev, and
plasma-framework is in Alpine, but plasma is in pmaports, then the
regular guess_main() algorithm below would pick plasma instead of
plasma-framework.
Fixes: build.postmarketos.org#52
Remove the recursive check, as it caused an infinite loop. It took a
very long time to complete anyway, even when it worked. The reasoning
for having the recursive check in the first place was, that we would
have device packages with arch=noarch set in the APKBUILD, but which
could only be built for a certain architecture (the device
architecture). Nowadays using arch=noarch in device packages is
forbidden, and we enforce that rule [1]. So the recursive arch check
isn't necessary anymore.
[1] ac6c0a2997
Make sure, that "args.fork_alpine" is always present. Otherwise,
pmbootstrap will fail if the aportgen code is called by anything but
"pmbootstrap aportgen". For example, when the user is adding a new
device during "pmbootstrap init".
Fixes: 54e51759ad ("aportgen: add feature to fork upstream packages")
The important part about this patch is the change for armhf, which
adjusts the hostspec to the one used by Alpine.
Fixes a part of postmarketOS/pmaports#363
This MR add the --fork-alpine argument to the pmboostrap aportgen
command, which downloads the APKBUILD and related files and copies them
into the pmaports/temp folder.
Properly handle the following two cases in APKBUILDs:
* depends="$depends ..."
* depends="${depends} ..."
First I've attempted to refactor the parsing code to do this in a more
generic way. But I've realized that it would make more sense to retire
the python based APKBUILD parsing approach altogether and finally use a
shell script parser. Let's discuss this in #1801.
Be more patient with waiting for processes to start up. From looking at
the logs, it seems that the only reason CI was failing is because we did
not wait long enough.
Compare a version against a check string. This will be used in
"pmbootstrap kconfig check", to only require certain options if the
pkgver is in a specified range.
Download all sources and verify their checksums.
This will be used in pmaports.git CI, if ci:skip-build is set in the
commit message (currently it just skips the build, and we don't test if
the source checksums are valid or not).
The qemu test executes a pmbootstrap instance with
pmb.helpers.run.user(). Use the --details-to-stdout flag from now on for
the started pmbootstrap process, so it is not silent when downloading
packages with apk, and will therefore not run into the timeout of the
parent process (which kills processes that are silent for more than five
minutes by default).
This fixes the test in our CI infrastructure.
Explicitly depend on mpc1. Our generated gcc aports use the !tracedeps
option, so we need to explicitly set the libraries it depends on.
This has mostly not been an issue, as we are installing our gcc
packages together with Alpine's gcc package, which causes the libraries
to get installed anyway.
Related: pmaports#236
Packages can add pmb:strict to their options to enable the --strict
mode. This can be used if packages need to get build in a clean chroot
or their make dependencies need to get removed.
Aborts the build if any dependencies would have to be build first. This
is useful for build.postmarketos.org, because we want to build exactly
one package in one build job. If dependencies would need to be built, we
made a mistake earlier, and not aborting the build makes it harder to
find that orginal mistake.
When a pmaport can not be built for the desired architecture, fall back
to the binary package (from postmarketOS or Alpine) if it exists. This
allows us to provide an updated version of mesa for arm arches, but
using Alpine's mesa package on x86* arches.
Adjust the stub function in test_generate_output_format() to accept the
replace_subpkgnames argument, which was introduced in the previous
commit ("repo_missing: return pkgnames, not subpkgnames").
For some reason, this CI test only ran after the MR was merged to master
and not when it ran in the MR's branch. I'm investigating this.
"breeze-icons" depends on "qt5-qtbase-dev", but
"pmbootstrap repo_missing" should return "qt5-qtbase" instead.
This patch fixes it, as one can see with:
$ pmbootstrap repo_missing --built breeze-icons --overview