At the moment, pmbootstrap updates the kernel and the initfs whenever
using the flasher or export. This is useful, but sometimes you just want
to boot exactly the same kernel several times. In that case, having to wait
several seconds for the (redundant) update to complete is quite annoying.
Add a --no-install option that allows skipping the kernel/initfs update.
At the moment, sparse images are generated if the device sets
deviceinfo_flash_sparse="true". But for testing purposes it can be
useful to specifically enable or disable the default behavior.
Add a --sparse and --no-sparse option that enables or disables
sparse image generation.
Adjust the mesa related packages installed for running in qemu, after
the mesa APKBUILD in Alpine was changed. Install mesa-va-gallium instead
of:
* mesa-dri-freedreno
* mesa-dri-swrast
* mesa-dri-virtio
* mesa-dri-vmwgfx
Fix error in pmbootstrap-qemu CI test:
ERROR: Could not find dependency 'mesa-dri-freedreno' in any aports folder or APKINDEX.
Related: 298e20d04f
Now that the qemu-user binary is bind-mounted, we no longer copy
the binary to the device rootfs. However, there is still the empty
stub file that we used as a destination mount point.
Let's remove it before copying it to the device rootfs.
It is automatically re-created the next time the qemu-user binary
is needed.
Right now we explicitly add ttf-droid to /etc/apk/world, and force
installation of it on every postmarketOS installation. But there really
is no reason to force this from pmbootstrap:
- If ttf-droid is really required for every postmarketOS installation,
it should be a depends of postmarketos-base.
- Even with this change it is still installed on every device,
because postmarketos-splash depends on it.
- There is no need for fonts when using the "none" UI.
For now, the only difference of this change is that we no longer add
ttf-droid explicitly to the APK world. Instead, it is installed as
dependency of postmarketos-splash.
At the moment we assume that everyone running QEMU is either using
ALSA, or has the ALSA PulseAudio plugin configured on the host system.
(Since we run QEMU outside of the chroot at the moment, configuration
files are read from the host system instead of the chroot...)
Other distributions have much wider support for PulseAudio,
so not everyone will actually have the ALSA PulseAudio plugin configured.
In that case, it is better to use QEMU's PulseAudio backend since that
does not require any configuration. It also lets us drop the alsa-lib
fork, since that was only needed for loading the ALSA PulseAudio plugin.
In general, it seems difficult to detect which audio backend the user
wants to run (if any). Selecting the wrong one results in ugly warnings
when running QEMU. So let's not assume any by default, and add a
--audio option instead which accepts one of QEMU's audio backends
(alsa, pa or sdl).
Add a shortcut for "git pull --ff-only" in all repositories cloned by
pmbootstrap (currently pmaports and aports_upstream, new pmdevices
repository coming soon).
'pmbootstrap pull' will only update the repositories, if:
* they are on an officially supported branch (e.g. master)
* the history is not conflicting (fast-forward is possible)
* the git workdirs are clean
Otherwise it shows the user a descriptive message about what to do. The
list of supported branches is only "master" right now, and will be
extended in later commits, so we can have a stable branch for pmaports
based on Alpine's releases. More about that in the project direction
2020 issue.
Closes: #1858
Remove a legacy code path from back in the day, where pmaports were in
the same repository as pmbootstrap ([1]). Nowadays, pmaports are always in a
git repository, so this is not necessary anymore. If git fails at this
point, crash hard and don't carry on with an empty string.
This is in preparation for using rev_parse in new code paths, so we
don't need to check the return value of rev_parse.
[1] 39548835 "Write custom os-release (closes#324) (#439)"
https://github.com/postmarketOS/pmbootstrap/pull/439
When defining a new kernel subpackage with a "-" in it
(e.g. $pkgname-kernel-mainline-modem), then pmbootstrap is unable to
find the function that builds the subpackage:
ERROR: Could not find subpackage function, no line starts with 'kernel_mainline-modem() {'
This is because it assumes that a $pkgname-kernel-<name> subpackage
is built by a kernel_<name> function, but this does not have to be the case.
We should really respect the name of the subpackage function that
is specified when defining the subpackage, but unfortunately it is
stripped away in cut_off_function_names().
For now let's fix this by replacing "-" with "_", but ideally the
APKBUILD parser API should be refactored to expose the subpackage
function in the future.
This adds a new deviceinfo parameter, 'boot_part_start' which accepts an
int and indicates the number of sectors from the start of the drive to
place the boot partition.
The librem5 devkit (and actual phone) u-boot has grown beyond the 2048
sector space previously before the boot partition, so this is necessary in
order to boot pmos on this device.
Install rust (rustc) in native chroot, if it is in the APKBUILD's
build dependencies. Add a test to verify that crossdirect + rust works
as expected.
Closes: #1861
Rust packaging is new and still a bit weird in Alpine and postmarketOS.
As of writing, we only have one package (squeekboard), and use cargo to
download the source of all dependencies at build time (several git
repositories!) and compile it. Usually, this is a no-go, but at least
until this is resolved properly, let's cache the downloads as suggested
in: https://doc.rust-lang.org/cargo/guide/cargo-home.html
Related: #1861
When initializing the building chroots, some symlinks are created to
/mnt/pmbootstrap* dirs. With upcoming rust-related symlinks, they will
point to subdirs in /mnt/pmbootstrap-rust, such as
/mnt/pmbootstrap-rust/registry/cache. Create this directory structure if
it does not exist.
Do not install git in the native chroot and use it from there. Remove the
chown_to_user argument from pmb.helpers.git.clone(), the resulting dir
is now always owned by the user. While at it, refactor the function and
display the clone URL.
Previously we had cloned aports_upstream (from Alpine) with
chown_to_user=False (legacy) and pmaports with chown_to_user=True.
pmb.helpers.git.rev_parse() would only work after chown_to_user=True.
Check if git is installed in "pmbootstrap init", and remove the same
check from rev_parse(). Add a new work dir version, that checks for git
and changes ownership of already checked out aports_upstream to the
host system's user.
When creating a new work dir, create cache_git instead of cache_http.
cache_http is created on demand already, with proper permissions. But
cache_git must be created, otherwise pmb.helpers.mount.bind will create it
as root.
This is in preparation for the "pmbootstrap pull" feature, as it allows
using the host system's git in all new code paths. We will be able to
handle repositories even if they were cloned outside of the work dir
(which we do in a few CI scripts for example).
Related: #1858
Remove /home/pmos/git from pmb.config.chroot_home_symlinks, so
pmb.chroot.init.init() will not change the ownership of the
$WORK/cache_git dir.
The symlink was added two years ago in [1] as compatibility to previous
behavior, because the mountpoint of cache_git was changed from
/home/user/git to /mnt/pmbootstrap-git. Forcing the ownership of that
dir to be the pmos user inside the chroot made sense, because the only
cloned repository (aports_upstream) would also be owned by the pmos
user. With the upcoming change to have all cloned repos owned by the
host-system's user, this is just confusing.
The /home/pmos/git symlink does not seem to be used by anybody, it is
not mentioned in the pmbootstrap code and in the wiki.
[1] b4dd7a89 "Close #709: Improve user creation (#725)"
https://github.com/postmarketOS/pmbootstrap/pull/725
The APKBUILD reference for "provides" [1] is not worded clearly; but
after reading it over and over again, my understanding is:
* package with provides='foo=1.2' will be automatically installed if
user requests installing "foo"
* package with provides='foo' (without version) will NOT get
automatically installed if user requests installing "foo"
For pmbootstrap, this means, that we must not attempt to build a package
where the pkgname mentioned in provides matches what we are currently
resolving, unless there is an equals sign in the provides entry.
Fixes: #1862, pmaports#404
[1] https://wiki.alpinelinux.org/wiki/APKBUILD_Reference#provides says:
"List of package names (and optionally version info) this package
provides.
If package with a version is provided (provides='foo=1.2') apk will
consider it as an alternate name and it will automatically consider the
package for installation by the alternate name, and conflict with other
packages having the same name, or provides.
If version is not provided (provides='foo'), apk will consider it as
virtual package name. Several package with same non-versioned provides
can be installed simultaneously. However, none of them will be installed
by default when requested by the virtual name - instead, error message
is given and user is asked to choose which package providing the virtual
name should be installed."
Changes:
* Added the ALSA_PLUGIN_DIRS environment variable otherwise it tries
to load the plugins of the host pc and they fails.
* Added the /usr/lib/pulseaudio path to the library-path argument
* Added the audiodev and soundhw parameters to the qemu command
* Added the required packages necessary to be installed in the chroot
Adjust to new text that "file" outputs for ARM linux kernels:
"ARM OpenFirmware FORTH Dictionary..."
Make test_bootimg_kernel pass again, which was failing with:
AssertionError: assert 'heimdall-isorec' in 'File is not an Android ...
This fixes#1845 without breaking #1717 (closed). The amount of inodes
only needs to be explicitly set when we don't know the size of the
filesystem before hand, for example when using fastboot or Heimdall. In
all other cases it's actually better to let it figure out automatically
how many inodes is set.
On my 16GB sdcard the amount of available inodes got increased by 10
times (from ~100K to ~1M).
Add default user to netdev group. Together with MR [1] in pmaports.git,
this fixes the permissions to use nmtui and nm-applet from the user.
[1] https://gitlab.com/postmarketOS/pmaports/merge_requests/777
Signed-off-by: Danct12 <danct12@disroot.org>
Properly ignore comments at the end of lines, instead of assuming that
all lines below belong to the attribute:
subpackages="$pkgname-dev" # $pkgname-lang
Fixes build.postmarketos.org#61, where pmbootstrap would assume that a
random package provides "make", just because the word "make" is written
somewhere below subpackages=" in the APKBUILD and it is parsed
incorrectly.
While at it, also support the ' character for quotations and detect if
a quotation for a value was started, but there is no end quotation sign
in the rest of the file.
I've added tests, and manually checked that this did not introduce any
parsing bugs for all the APKBUILDs in pmaports.git, by running
'pmbootstrap apkbuild_parse' with the old and new code, and diffing the
result.
Because of an APKINDEX subpkg parsing bug, pmbootstrap currently assumes
that "ktp-common-internals" provides "make". It fails with a cryptic
error when trying to build a package without an existing postmarketOS
binary packages mirror (like done in the build.postmarketos.org
testsuite):
File "/home/user/code/pmbootstrap/pmb/chroot/apk.py", line 174, in replace_aports_packages_with_path
package + "-" + data_repo["version"] + ".apk")
TypeError: 'NoneType' object is not subscriptable
Replace this error with a more meaningful message, that points to the
issue analyzing this bug.
Related: https://gitlab.com/postmarketOS/build.postmarketos.org/issues/61
Always install git when building packages, so abuild won't behave
slightly different if it is (not) installed. As nice side-effect, we
will always get the pmaports.git commit saved in the resulting .apk file
from now on (in .PKGINFO).
With the nokia N900 the user can choose between different keyboard
layouts and during the install the setup-keymap installs the choosen
layout.
This only changes the keyboard layout of the console, while installing
a UI using xorg, the layout was hardcoded to "us".
With a simple check for the existence of an xorg config containing
XkbLayout, this will run sed to replace the default with the layout
chosen by the user.
Tested with a clean install of the nokia N900 with the xfce4 UI.
elogind requires cgroup support in the kernel.
It does not start without it, resulting into errors like:
elogind-daemon[1654]: Failed to mount tmpfs at /sys/fs/cgroup: No such file or directory
elogind-daemon[1654]: Failed to mount cgroup at /sys/fs/cgroup/elogind: No such file or directory
elogind-daemon[1654]: Failed to allocate manager object: No such file or directory
dbus-daemon[1437]: [system] Activating service name='org.freedesktop.login1' requested by ':1.1' (uid=0 pid=1642 comm="/usr/bin/lightdm ") (using servicehelper)
dbus-daemon[1437]: [system] Failed to activate service 'org.freedesktop.login1': timed out (service_start_timeout=25000ms)
Most of the supported desktop environments (e.g. Weston) do not
start without elogind, so we should require CONFIG_CGROUPS=y in
all kernels to avoid this problem.
Make /home/pmos/build/.git point to the .git dir from pmaports.git, with
a symlink so abuild does not fail.
abuild expects the current working directory to be a subdirectory of a
cloned git repository (e.g. main/openrc from aports.git). If git is
installed, it will try to get the last git commit from that repository,
and place it in the resulting apk (.PKGINFO) as well as use the date
from that commit as SOURCE_DATE_EPOCH (for reproducible builds).
With that symlink, we actually make it use the last git commit from
pmaports.git for SOURCE_DATE_EPOCH and have that in the resulting apk's
.PKGINFO.
Fixes: #1841
When 'pmbootstrap build' is started with '--no-depends', we make sure
that binary packages of all dependencies exist before proceeding with a
build. Make sure that we have downloaded the APKINDEX for the given arch
first, otherwise this may complain that a binary package is missing
although it does exist.
This happened when using the "native" cross compile method, e.g. when
building kernels for different architectures. It would complain that
there is no binary package for "bash" (which is in Alpine obviously):
https://builds.sr.ht/~postmarketos/job/103882
When running pmbootstrap init, first select device vendor, then device
codename. Also fixed tests for new behavior and added some new ones for
new scenarios.
postmarketOS/pmaports!700 adds a new "deviceinfo_bootimg_dtb_second"
option that places the DTB in the "second" area of the Android boot
image.
Attempt to detect this automatically by checking the extracted
second binary for the FDT magic (0xd00dfeed).
Packages ending in -dev: just assume that the originating aport has the
same pkgname, except for the -dev at the end. Otherwise we may end up
with the wrong package.
For example, if something depends on plasma-framework-dev, and
plasma-framework is in Alpine, but plasma is in pmaports, then the
regular guess_main() algorithm below would pick plasma instead of
plasma-framework.
Fixes: build.postmarketos.org#52
mkbootimg from osm0sis changed the filename of the dtb section in [1]
which breaks the detection of qcdt in pmbootstrap.
Fix this by adjusting the filename pmbootstrap checks to match the
unpackbootimg behavior.
[1] https://github.com/osm0sis/mkbootimg/commit/d8222a4d
Currently, building with envkernel.sh entirely ignores the
makedepends listed in the kernel APKBUILD. Common dependencies
needed by most kernels are hardcoded in envkernel.sh.
However, some kernels may need extra dependencies either during
the build process or when the kernel is packaged.
Installing the makedepends when the build environment is initialized
is difficult because "source helpers/envkernel.sh" is not aware
of the exact kernel package that is going to be built later.
However, we can easily modify the packaging step
(i.e. pmbootstrap build --envkernel linux-...)
to install the required makedepends.
This fixes building "linux-postmarketos-qcom-msm8916" using envkernel,
since it requires "dtbTool" and "installkernel" to be installed when
the APKBUILD package() function is executed.
The sparse image tools (simg2img, img2simg, append2simg) are now
part of Alpine's android-tools where they are continously updated.
(See https://github.com/nmeum/android-tools/pull/8)
Therefore, "libsparse" now conflicts with "android-tools", which
causes the fastboot flasher to fail if "libsparse" is already
installed.
Install "android-tools" instead of "libsparse" before generating
a sparse image to avoid this problem.
Make it more obvious that we are generating packages during
'pmbootstrap init', when using the new device wizard:
[22:32:32] *** pmaport generated: /home/user/.local/var/pmbootstrap/cache_git/pmaports/device/device-openphoenux-neo900
[22:32:32] *** pmaport generated: /home/user/.local/var/pmbootstrap/cache_git/pmaports/device/linux-openphoenux-neo900
This file is used by the package browser frontend build by the alpine
developers. It uses the contents of the DESCRIPTION file to validate the
cache in the database.
Remove the recursive check, as it caused an infinite loop. It took a
very long time to complete anyway, even when it worked. The reasoning
for having the recursive check in the first place was, that we would
have device packages with arch=noarch set in the APKBUILD, but which
could only be built for a certain architecture (the device
architecture). Nowadays using arch=noarch in device packages is
forbidden, and we enforce that rule [1]. So the recursive arch check
isn't necessary anymore.
[1] ac6c0a2997
This reverts commit 6fb5b28e2f.
The -i option was removed from fastboot, so we can't use it anymore
unless we fork the package. There was only one device using it,
amazon-thor. I will add a note to the wiki page.
Fixes#1830.
Linking big libraries may take more than five minutes, so change the
default timeout to fifteen minutes to save users from frustration.
This is the same value that I'm using to build binary packages for the
repository.
Pushing directly to master without making a merge request, because this
is a trivial change.
Make sure, that "args.fork_alpine" is always present. Otherwise,
pmbootstrap will fail if the aportgen code is called by anything but
"pmbootstrap aportgen". For example, when the user is adding a new
device during "pmbootstrap init".
Fixes: 54e51759ad ("aportgen: add feature to fork upstream packages")
The important part about this patch is the change for armhf, which
adjusts the hostspec to the one used by Alpine.
Fixes a part of postmarketOS/pmaports#363
This MR add the --fork-alpine argument to the pmboostrap aportgen
command, which downloads the APKBUILD and related files and copies them
into the pmaports/temp folder.
At the moment, attempting to install to SD card when
flash_sparse is set to "true" will always fail because
/home/pmos/rootfs does not exist in that case.
Sparse images are only useful to speed up Fastboot flashing.
Nothing will be able to read it from the SD card.
The problem can therefore be avoided by simply skipping the
generation of the sparse image when SD card installation is used.
Remove u-boot from the native_cross_compile list as we now have
crossdirect. Compiling is not terribly slow anymore, and when not using
the "native" method, it is possible to depend on firmware packages like
arm-trusted-firmware-sun50i.
This reverts commit d7d7ccb672.
This allows for example for me to call the kconfig check function on the
.config file in my Linux tree: $ pmbootstrap kconfig check --file
.config and it reports me which kconfig options I need to enable.
From https://docs.python.org/3/library/re.html
"Backslashes are not handled in any special way
in a string literal prefixed with 'r'."
Signed-off-by: Steffen Pankratz <kratz00@gmx.de>
Properly handle the following two cases in APKBUILDs:
* depends="$depends ..."
* depends="${depends} ..."
First I've attempted to refactor the parsing code to do this in a more
generic way. But I've realized that it would make more sense to retire
the python based APKBUILD parsing approach altogether and finally use a
shell script parser. Let's discuss this in #1801.
Add check for UEVENT_HELPER for kernels >= 4.0.0. Change LBDAF check to
be only required for kernels <5.2.0. Adjust the config format and
checking code to support such range specific checks.
Compare a version against a check string. This will be used in
"pmbootstrap kconfig check", to only require certain options if the
pkgver is in a specified range.
This fixes regression from commits 0431a519 and 4daf9916.
pmbootstrap kconfig_edit raised an error while trying to checksum a package.
Error was:
(028793) [16:13:22] ERROR: 'module' object is not callable
(028793) [16:13:22] See also: <https://postmarketos.org/troubleshooting>
(028793) [16:13:22] Traceback (most recent call last):
File "/home/lexx/dev/pmos/pmbootstrap/pmb/__init__.py", line 63, in main
getattr(frontend, args.action)(args)
File "/home/lexx/dev/pmos/pmbootstrap/pmb/helpers/frontend.py", line 279, in kconfig
pmb.build.menuconfig(args, args.package)
File "/home/lexx/dev/pmos/pmbootstrap/pmb/build/menuconfig.py", line 158, in menuconfig
pmb.build.checksum(args, pkgname)
TypeError: 'module' object is not callable
The function was renamed, call new function.
(Some?) Mediatek kernels don't have the kernel in the top-level
directory but in a subdirectory called 'kernel' next to a folder
'mediatek' containing most device-specific code.
Adjust the menuconfig code to be able to work with that.
Download all sources and verify their checksums.
This will be used in pmaports.git CI, if ci:skip-build is set in the
commit message (currently it just skips the build, and we don't test if
the source checksums are valid or not).
Fix configure crash when building mozjs60 by mounting tmpfs as /dev/shm
in chroots. This is an important fix, because of this dependency chain,
that is currently broken in armhf and armv7 (see pmaports#244):
postmarketos-base -> networkmanager -> polkit -> mozjs60
Minecrell
Oliver Smith
Luca Weiss
Clayton Craft
Daniele Debernardi
Drew DeVault
Bart Ribbers
Danct12
Alexey Min
lambdadroid
Martijn Braam
Steffen Pankratz
Oliver Smith
Idan Horo
Robert Yang