Set CTARGET and CTARGET_ARCH on top of the generated APKBUILD and use
the existing code in the APKBUILD for build(), package(), makedepends,
subpackages. This is now possible, because the pmbootstrap parser for
APKBUILDs has been improved a lot since this was originally written and
because the Alpine APKBUILD does everything that was hacked in here,
like removing man, info directories and bfd-plugins if doing a cross
build.
This is now very similar to bootstrapping binutils for another
architecture via scripts/bootstrap.sh in aports.git.
Most importantly this sets --disable-gprofng, which fails to build
against musl and is the reason why our cross/binutils-* packages
generated from binutils 2.39.x didn't build anymore.
Related: https://git.alpinelinux.org/aports/tree/main/binutils/APKBUILD
Related: https://sourceware.org/bugzilla/show_bug.cgi?id=29477
Replace "args.arch_native" with the direct function call in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
Newly generated binutils package has file conflicsts with
non-cross binutils.
ERROR: binutils-2.37-r3: trying to overwrite usr/lib/bfd-plugins/libdep.so
owned by binutils-aarch64-2.37-r3.
Fix the generator to produce package that does not include these plugins.
Packages like binutils-*, busybox-static-*, gcc-*, grub-efi-*, musl-*
are only needed for the native architecture during cross compilation.
Don't bother with trying to build them for other arches to save time and
to avoid getting stuck frequently at "armv7/binutils-aarch64" etc.
A few people like to use pmbootstrap on aarch64 hosts (e.g. PineBook
Pro), so let's make it available for aarch64 again when we can build
aarch64 packages natively in CI and bpo. (They do get stuck there right
now, because of qemu user emulation.)
Related: https://gitlab.com/postmarketOS/build.postmarketos.org/-/issues/75
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
APKBUILD should be indented with tabs not spaces.
This also resolves a bug which the replace_functions feature was always
trimming off the beginning of a line.
For example:
replace_functions = {"build": "return 0"}
would be formatted as "rn 0".
In order to get cross-compilers, we generate a few aports (e.g.
binutils-armhf, gcc-armhf) automatically from Alpine's aports.
pmbootstrap was already able to perform a git checkout of Alpine's
aports repository. But it needed to be manually updated. Otherwise
the `pmbootstrap aportgen` command could actually downgrade the aport
instead of updating it to the current version.
After thinking about adding a dedicated pmbootstrap command for
updating git repositories, I thought it would be better to not open
that can of worms (pmbootstrap as general git wrapper? no thanks).
The solution implemented here compares the upstream aport version of
the git checkout of a certain package (e.g. gcc for gcc-armhf) with the
version in Alpine's binary package APKINDEX. When the aport version is
lower than the binary package version, it shows the user how to update
the git repository with just one command:
pmbootstrap chroot --add=git --user -- \
git -C /mnt/pmbootstrap-git/aports_upstream pull
Changes:
* `pmb.aportgen.core.get_upstream_aport()`: new function, that returns
the absolute path to the upstream aport on disk, after checking the
version of the aport against the binary package.
* Use that new function in pmb.aportgen.gcc and pmb.aportgen.binutils
* New function `pmb.helpers.repo.alpine_apkindex_path()`: updates the
APKINDEX if necessary and returns the absolute path to the APKINDEX.
This code was basically present already, but not as function, so now
we have a bit less overhead there.
* `pmbootstrap chroot`: new `--user` argument
* `pmb.parse.apkbuild`: make pkgname check optional, as it fails with
the official gcc APKBUILD before we modify it (the current APKBUILD
parser is not meant to be perfect, as this would require a full shell
parsing implementation).
* Extended `test_aportgen.py` and enabled it by default in
`testcases_fast.sh`. Previously it was disabled due to traffic
concerns (cloning the aports repo, but then again we do a full KDE
plasma mobile installation in Travis now, so that shouldn't matter
too much).
* `testcases_fast.sh`: With "test_aport_in_sync_with_git" removed
from the disabled-by-default list (left over from timestamp based
rebuilds), there were no more test cases disabled by default. I've
changed it, so now the qemu_running_processes test case is disabled,
and added an `--all` parameter to the script to disable no test
cases. Travis runs with the `--all` parameter while it's useful to
do a quick local test without `--all` in roughly 2 minutes instead of
10.
* `aports/cross/binutils-*`: Fix `_mirror` variable to point to current
default Alpine mirror (so the aportgen testcase runs through).
Example: Building gcc-armhf for armhf does not make sense, so this
commit changes arch="all" to arch="aarch64 x86_64". This helps to
simplify the repository scripts (#970).
* pmbootstrap init: Generate new port device- and linux-package
* adds `pmbootstrap aportgen device-*` and
`pmbootstrap aportgen linux-*`
* ask for confirmation when selecting a non-existing device
* generate the packages directly from init
* refactor aportgen code
* fixed some easy things in the linux- APKBUILD (more to come in
follow-up PRs!)
Testing:
* Test all questions to the user from pmb.config.init and pmb.aportgen.device
(except for the timezone question, because we would need to monkeypatch the
os.path.exists() function, which messes up pytest, so we'd need to refactor
the timezone function to be more testsuite friendly first)
* Run the device wizard in a testcase a few times and check the output, that
pmbootstrap.aportgen.device and pmbootstrap.aportgen.linux create by parsing
the resulting APKBUILDs and deviceinfo and checking its contents.
* Build the generated device package once in the same testcase
Thanks a lot to @drebrez for all the help with this one:
<https://github.com/postmarketOS/pmbootstrap/pull/821>
See also the updated porting guide:
<https://wiki.postmarketos.org/wiki/Porting_to_a_new_device>
* Fix hardcoded `armhf` in pmb/aportgen/binutils.py
* Generate aports: `binutils-aarch64`, `musl-aarch64`, `gcc-aarch64`
* Distccd: Remember the cross-compiler architecture (currently armhf
or aarch64), that the current distccd is running as, and restart
distccd with the correct architecture, in case a different arch
is needed than what it is currently running as. (Depending on the
cross-compiler arch, the PATH variable gets adjusted before
starting distccd)
* Testcases: add aport generation for aarch64, add cross-compiling
to aarch64
* pmb/parse/arch.py: Add aarch64 to the mapping
Oliver Smith
bo41
Alexey Min
Caio Fontes
Daniele Debernardi
Oliver Smith
Robert Yang
Pablo Castellano