Follow up to last commit, turns out we need to use it in both functions
or otherwise at least gitlab-ci can fail, as seen in this MR:
https://gitlab.com/postmarketOS/pmaports/merge_requests/270
After using 'busybox tar' instead of 'tar' in both places in the
APKBUILDs, it went through.
Use 'busybox tar' to avoid 'tar: Child returned status 141' on some
machines (builds.sr.ht, gitlab-ci).
See pmaports#26.
[skip ci]: already ran through successfully
This adds flasher support for uuu, a utility used by NXP devices for
flashing images.
The flasher expects a uuu command list script to be installed in the
device rootfs at /usr/share/uuu/flash_script.lst.
This allows building u-boot much faster, previously it used to build
whole u-boot with distcc+qemu cross-compile, now it builds u-boot on
native chroot using cross-compilers.
In pmb.helpers.repo.alpine_apkindex_path(), default to
arch = args.arch_native *before* calling pmb.helpers.repo.update().
Because otherwise update() defaults to all arches instead of
args.arch_native.
This caused the APKINDEX files for all arches to get downloaded, as
pmb.chroot.apk_static.init() calls alpine_apkindex_path() without
setting arch = args.arch_native explicitly.
Related to a proprietary "T-Mobile's WiFi-Calling" feature. It should
not affect normal calls, so it should be safe to disable it. Having it
enabled causes sudo, ip and other commands to hang.
It is dangerous to flash the boot partition on the Ouya. Doing so can
result in a bricked device. Allow deviceinfo to blacklist partitions from
ever being flashed.
Ignore APKBUILDs that have "!pmb:kconfigcheck" in their options by
default in "pmbootstrap kconfig check", but print a note that they have
been skipped. Check all kernels with "pmbootstrap kconfig check -f".
This is necessary, because the Librem 5 devboard kernel's config does
not have CONFIG_DM_CRYPT enabled in their config, and we check for
that. As the device is still under heavy development, we will make our
lives easier by just using the upstream kernel config without any
changes and ignoring it in our check by default.
"breeze-icons" depends on "qt5-qtbase-dev", but
"pmbootstrap repo_missing" should return "qt5-qtbase" instead.
This patch fixes it, as one can see with:
$ pmbootstrap repo_missing --built breeze-icons --overview
Provides a quick way to incrementally compile a kernel and push it to
device.
Example usage.
Compile the kernel:
$ cd /src/linux/
$ source /src/pmbootstrap/helpers/envkernel.sh
$ make tegra_postmarketos_defconfig
$ make -jX
Package kernel and flash to device:
$ pmbootstrap build --envkernel linux-samsung-p4wifi
$ pmbootstrap flasher flash_kernel
Modify kernel source then incremental compile, package, and flash:
$ make -jX
$ pmbootstrap build --envkernel linux-samsung-p4wifi
$ pmbootstrap flasher flash_kernel
Set the log level to DEBUG if args (and therefore logging) failed to
initialize. This causes the default logging handler to be used, and
everything that gets written to the log (error message and trace!) will
get printed to stdout instead of not being printed anywhere at all.
This can be tested by removing x86_64 from the mapping in
pmb.parse.arch.alpine_native() (assuming the host is x86_64). Without
this patch, no meaningful error appears. With the patch, the real error
message, and a stack trace gets printed.
Test with flake8: *.py
./test/check_checksums.py:13:13: E117 over-indented
./pmb/config/init.py:97:8: F632 use ==/!= to compare str, bytes, and int literals
./pmb/parse/arguments.py:229:13: E117 over-indented
Increase pakage version and minimum required apk version. The latter
should be done with every new release from now on, so we always have a
recent version there. I've updated the pmbootstrep release checklist
accordingly: https://wiki.postmarketos.org/wiki/Pmbootstrap_release
This drops the --apparent-size parameter when calculating the size
required for rootfs, which seemed to return a size that was too small
for some devices. This also includes specifying the number of inodes to
support when formatting rootfs.
Fixes#1717
Allow changing the kernel partition for fastboot and heimdall in
deviceinfo and on the fly while doing "pmbootstrap flasher
flash_kernel". Also allow changing the partition for
"... flash_rootfs" with fastboot (this was only possible with
heimdall so far).
Introduce two new deviceinfo variables:
* flash_fastboot_partition_kernel
* flash_fastboot_partition_system
This is useful for devices with dual partitioning that have boot_a
and boot_b.
This allows embedding multiple firmware binaries into SD images.
Firmware images for most devices that require this functionality consist
mainly of u-boot, but some devices (e.g. librem5) have multiple firmware
images that need to be embedded in the SD image created by pmbootstrap.
This functionality uses two new deviceinfo parameters:
- deviceinfo_sd_embed_firmware: a comma-separated list of
binary:offset (where binary is under /usr/share/firmware)
- deviceinfo_sd_embed_firmware_step_size: The number of bytes for
each increment of the offset specified in the
deviceinfo_sd_embed_firmware (typically 1024 or 2048)
deviceinfo_write_uboot_spl has been obsoleted by these new parameters.
Rename pmb.helpers.devices.list() to
pmb.helpers.devices.list_codenames(). Python already has a list()
function, so we name our function to make the calls to the codenames
listing function inside pmb/helpers/devices.py less confusing.
This fixes list_apkbuilds which is dead code currently but is useful
for external software using pmbootstrap. It also adds list_deviceinfos
which does the same thing but for the deviceinfo data.
The plan is to add an API later, where both functions can be used.
APKBUILD should be indented with tabs not spaces.
This also resolves a bug which the replace_functions feature was always
trimming off the beginning of a line.
For example:
replace_functions = {"build": "return 0"}
would be formatted as "rn 0".
Python < 3.6 randomized the order of keys in dictionaries, unless
OrderedDict was used. Use OrderedDict to store the version suffixes.
When the order was randomized, the valid version string 3.0.0_pre1 did
not always pass the validation check. The suffix "pre" should always be
detected as such, but with the random order, it was sometimes detected
as "p" suffix (see below). The following letters "re" are not a valid
suffix_no (the number expected to follow the suffix) and so it failed.
suffixes = {
"pre": ["pre", ...],
"post": ["p". ...]
}
Display the resolved deviceinfo file path in exceptions from the
deviceinfo parser. Instead of messages like these:
RuntimeError: Please add 'deviceinfo_codename="lg-mako"' to: /home/user/.local/var/pmbootstrap/cache_git/pmaports/.gitlab-ci/testcases/../../device/device-lg-mako/deviceinfo
We get the more readable version:
RuntimeError: Please add 'deviceinfo_codename="lg-mako"' to: /home/user/.local/var/pmbootstrap/cache_git/pmaports/device/device-lg-mako/deviceinfo
deviceinfo_codneame holds the device's code name, so we can easily look
it up in the finished postmarketOS installation by reading
/etc/deviceinfo.
Related: postmarketOS/pmaports#157
Properly replace ~ with $HOME in all paths. Fix tab completion when
passing a path starting with ~ to pmaports:
$ pmbootstrap --aports ~/src/pmaports/ build linux-<TAB>
Make tab completion work again for packages, if the default pmaports
dir is used. In the last "args" code refactor, the variable replacing
code was moved into its own function. We did not call it in the
packagecompleter() yet, so it could not replace the $WORK variable in
the default pmaports path.
Show all config keys that can be queried and set in the
'pmbootstrap config -h' output and make tab completion work for the
key names.
I've set "metavar" and placed the variables in the helpstring. That
way, argparse will not generate a huge "positional arguments" string
that blows up the layout of the help output:
[{ccache_size,device,extra_packages,hostname,jobs,kernel,keymap,...
Make it possible to point the pmaports dir to an existing repository
clone, while not having to give the --aports/-p parameter with every
command.
pmbootstrap --aports=/my/pmaports/path init
If the config file exists already, the dir can also be set with:
pmbootstrap config aports /my/pmaports/path
Adds an optional deviceinfo variable, `deviceinfo_rootfs_image_sector_size`,
which specifies the logical sector size of the device's storage.
Some devices made after 2016 with UFS storage uses 4096 byte sectors
instead of the normal 512 bytes. The partition table in our rootfs
must match, otherwise the root filesystem won't mount on the device.
This change passes the sector size to `losetup` when creating the image
if the deviceinfo specifies it, so the image will have the correct
sector size.
If the deviceinfo doesn't specify the new option, the behaviour
is the same as previous versions of pmbootstrap.
Note that the sector size option only works on Linux 4.14 and above,
so pmbootstrap should be run on a >4.14 computer when installing to
devices with non-standard sector size.
To find if a device needs this parameter, run `fdisk -l` on the device.
If the output shows
`Note: sector size is 4096 (not 512)`
then add `deviceinfo_rootfs_image_sector_size="4096"` to the deviceinfo.
This is needed by the Pixel 3 XL (google-crosshatch) port.
See https://gitlab.com/postmarketOS/pmbootstrap/issues/1696.
Do not fail when an APKINDEX can not be downloaded, print a WARNING
instead. This matches, what apk does. Add a new allow_404 parameter to
pmb.helpers.http.download(), and use it in pmb.helpers.repo.update()
for downloading APKINDEX files. Cache the APKINDEX URLs that gave a 404
for the session, so we do not attempt to download these again.
This is needed for the new binary repository: the initial build is done
without existing APKINDEX files, so we must not fail in that case.
Change the cache format from args.cache["offline_msg_shown"] to
args.cache["pmb.helpers.repo.update"]["offline_msg_shown"]. This is in
preparation for saving more data in the cache of
pmb.helpers.repo.update in the next commit.
Do not fail anymore when attempting to start a new binary repository
build without any existing binary packages:
pmbootstrap -mp="" repo_missing
Find subpackages defined with subpackage functions:
subpackages="dev:mydevfunc"
Find provides defined with specific versions:
provides="mkbootimg=1.0.0"
Weston and Plasma Mobile might choose to draw on this virtual frame
buffer, just like it happened for the samsung-jflte. Disabling the
USE_VFB option fixed this, so let's make sure we don't have that option
enabled for any kernel.
Multiple -mp arguments can be used to list multiple mirrors:
$ pmbootstrap -mp=first -mp=second chroot -- cat /etc/apk/repositories
This is needed for the new build infrastructure, so we can have a WIP
repository to which we push packages until all of them are up to date,
and then publish all of them at once. Software like KDE/Plasma Mobile,
which expect a lot of packages to be updated from one version to
another will not end up with a half-way through upgrade that way.
Obscure feature: it was possible to specify a local path as
--mirror-pmOS. It would then get mounted to /mnt/postmarketos-mirror
inside the chroot, and be specified as such in the generated
/etc/apk/repositories file.
I had used this once for some testing scripts, but I am sure nobody is
using this anymore. The same can be achieved with running a local http
server anyway:
<https://wiki.postmarketos.org/wiki/Installing_packages_on_a_running_phone>
Removing this makes it easier to support multiple postmarketOS mirrors
(next commit).
-m is the Alpine mirror, -mp is the postmarketOS mirror. Use "URL" as
metavar and add help text that explains how to disable the postmarketOS
mirror (so all pmaports get built locally).
In pmb.helpers.package.get(), we are differentiating between packages
that do not exist at all, and packages that do not exist for the
specified architecture. Make sure to actuallyy download the foreign arch
APKINDEX files, before trying to find the packages in there. Otherwise
the "could not find aport, and could not find this package in any
APKINDEX" package would appear.
We ran into this when testing on sr.ht, because pmbootstrap runs on a
fresh install every time, where no old APKINDEX files are present
(which would work around the bug).
This was meant to be part of the previous commit already, and was
tested, but it was not submitted correctly. So here it is as separate
commit straight to master. It is trivial anyway.
Add a new action that lists all aports, for which no binary packages
exist. Only list packages that can be built for the relevant arch
(specified with --arch). This works recursively: when a package can be
built for a certain arch, but one of its dependencies
(or their depends) can not be built for that arch, then don't list it.
This action will be used for the new sr.ht based build infrastructure,
to figure out which packages need to be built ahead of time (so we can
trigger each of them as single build job). Determining the order of the
packages to be built is not determined with pmbootstrap, the serverside
code of build.postmarketos.org takes care of that.
For testing purposes, a single package can also be specified and the
action will list if it can be built for that arch with its
dependencies, and what needs to be built exactly.
Add pmb/helpers/package.py to hold functions that work on both pmaports
and (binary package) repos - in contrary to the existing
pmb/helpers/pmaports.py (see previous commit) and pmb/helpers/repo.py,
which only work with one of those.
Refactoring:
* pmb/helpers/pmaports.py: add a get_list() function, which lists all
aports and use it instead of writing the same glob loop over and over
* add pmb.helpers.pmaports.get(), which finds an APKBUILD and parses it
in one step.
* rename pmb.build._package.check_arch to ...check_arch_abort to
distinguish it from the other check_arch function
Move find_aport() and find_aport_guess_main() from pmb/build/other.py
to the new file pmb/helpers/pmaports.py.
Finding aports is not only needed when building packages, hence it
makes sense to move it out of pmb.build. The pmb/helpers/pmaports.py
file will have more pmaports related functions in a follow up commit.
Target arch is the system the package will run on. Host arch is the
system the package is compiled on.
For example kernel packages can be compiled on host arch x86_64 and
intended to run on target arch armhf. A build is necessary check
against host arch will always return True. The correct way is to check if
the package needs to be built for target arch.
Do not try to build and install dependencies for the package's
architecture when compiling in the "native" mode. That mode is
described here in detail:
https://wiki.postmarketos.org/wiki/Build_internals#Cross-compile_types
This makes it possible to cross compile kernels again, which need to
be built with GCC6. We have switched to Alpine's GCC6 package, but it is
not available for armhf/aarch64 on edge yet, because Alpine's build bots
are stuck (right now armhf is not even listed):
http://build.alpinelinux.org/
Huge thanks to ryang2478/Decatf for posting this patch in pmaports#138!
Clearly state which version is being used, and also display the message
when using build --force.
- Old:
WARNING: Package 'ubuntu-app-launch' in your aports folder has version
0_git20180604-r0, but the binary package repositories already have version
0_p20181101174257-r0! See also: <https://postmarketos.org/warning-repo2>
- New:
WARNING: package hello-world: aport version 1-r4 is lower than 1-r5 from
the binary repository. 1-r5 will be used when installing hello-world.
See also: <https://postmarketos.org/warning-repo2>
Find the main package by assuming it is a prefix of the subpkgname. We
do that, because in some APKBUILDs the subpkgname="" variable gets
filled with a shell loop and the APKBUILD parser in pmbootstrap can't
parse this right. (Intentionally, we don't want to implement a full
shell parser.)
This covers most use cases and saves a lot of build time. Can be
changed on demand. Again, this simplifies package building as part of
the new build infrastructure effort.
Use the device's architecture instead of noarch. Because the device
packages should never be built for other architectures, even if all
depends can be built for other arches as well.
This simplifies package building as part of the new build
infrastructure effort.
All existing pmaports will be changed shortly, along with a test case
in pmaports.
Set HOME to /root for commands started with pmb.chroot.root() and to
/home/pmos for commands started with pmb.chroot.user().
POSIX requires this variable to be set, see:
<http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html>
And this prevents a crash in "adb", which takes and alternative code
path if HOME is not set, that does not work with musl (fixes#1638).
Thanks to @ryang2678 for figuring this out!
* create symlinks to the GCC8 patches (introduced in pmaports!35)
* link to the new vendorkernel reference wiki page
* use SPDX license in the license= field
* add comment above the compiler section of the APKBUILD
* remove empty line at the end of the APKBUILD
When the timeout occurs it is important to ensure clean up of child
processes. Killing only the direct process created by a command can
leave child processes running.
For example a pmbootstrap.py install will run apk add. This run command
creates multiple processes as follows:
(cmd line arguments snipped for readability)
$ ps -e -o pid,ppid,pgid,cmd
PID PPID PGID CMD
31738 23247 31738 python3 ./pmbootstrap.py -t 15 install --no-fde
31746 31738 31738 sudo env -i /bin/sh -c ... ;apk --no-progress add
31747 31746 31738 /bin/sh -c ... ;apk --no-progress add
31748 31747 31738 apk --no-progress add
The root process of the run command is PID 31746. We want to kill
the child processes too. Otherwise only running kill -9 31746 will leave
the processes 31747 and 31748 running.
* aportgen: modify code to allow generating gcc6-armhf and other gcc6
cross compiler packages
* package: when 'gcc6' is in the depends of a package, and the cross
compiling mode is "native" (as we do it with kernels), install the
gcc6 cross compiler instead of the usual one (gcc8)
Related: pmaports#103
Overview:
In order to execute foreign arch binaries on the host system, we are
using the Linux kernel's binfmt_misc feature in combination with
static builds of QEMU. Before this patch, the statically compiled
QEMU binaries were taken from Debian (mostly because I did not realize
that Alpine ships them as well). Now we can use the ones from the aport.
Benefits:
This allows us to easily update and patch the QEMU executables, we
don't need to be in sync with Debian's versions anymore.
Alpine's package is more modular, so we can save some download,
install, zap time, as well as disk space: setting up an armhf chroot
with pmbootstrap took ~102 MB before, now it's ~18 MB.
Detailed changes:
* Remove `cross/qemu-user-static-repack` aport
* Add `data/qemu-user-binfmt.txt` with the binfmt_misc flags for ELF
binaries of various arches (extracted from Debian's packaging)
* When parsing that file, don't write verbose messages to
`pmbootstrap log` anymore, only to the verbose log (can be enabled
with `pmbootstrap -v`)
* Rename `pmb.parse.arch.alpine_to_debian()` to ...`alpine_to_qemu()`
* Rename `arch_debian` to `arch_qemu`
This commit adds a test case, which makes sure that the KDE framework
and plasma framework version are always the same.
Additional changes:
* APKBUILD parser parses the URL now (that's the best way I found to
categorize the KDE aports in frameworks and other)
* Changed single quotes to double quotes in KDE APKBUILDs, so the
parser doesn't include the single quotes in the parsed result
* Added the test case to the gitlab CI config
With this patch, "pmbootstrap flasher" will fail with "the following
arguments are required: action_flasher". Without it, it just prints
"Done" and quits.
Overview:
Since Alpine updated to distcc 3.3 last week, pmbootstrap wasn't able to use
distcc for cross compilation anymore. It always falled back to running the
compiler in QEMU (which works, but is a lot slower). The reason for that is,
that distcc requires all compilers that are being used in a whitelist now.
This partially fixes CVE-2004-2687 in distccd, which allowed trivial remote
code execution by any process connecting to the distccd server. We only run
distccd on localhost, but still this can be used for privilege escalation of
sandboxed processes running on the host system (not part of pmbootstrap
chroots).
Because the CVE is only partially fixed (see the comment in
`pmb/chroot/distccd.py` for details), we make sure that only the building
chroots can talk to the distcc server by running distcc over ssh.
Details:
* Completely refactored `pmb/chroot/distccd.py` to run distcc over ssh
* Store the running distcc server's arguments as JSON now, not as INI
* Make debugging distcc issues easy:
* Set DISTCC_BACKOFF_PERIOD=0, so the distcc client will not ignore the
server after errors happened (this masks the original error!)
* New pmbootstrap parameters:
* `--distcc-nofallback`: avoids falling back to compiling with QEMU and not
throwing an error
* `--ccache-disable`: avoid ccache (when the compiler output is cached,
distcc does not get used)
* `--verbose` prints verbose output of the distcc too
* New test case, that uses the new pmbootstrap parameters to force
compilation through distcc, and shows the output of distcc and distccd in
verbose mode on error (as well as the log of sshd)
Alpine's `abuild` will uninstall all dependencies by default, when a
package build fails.
Leaving this configuration unchanged leads to unexpected behavior with
pmbootstrap: when executing `pmbootstrap build --strict` and pressing
`^C` during the build, pmbootstrap will stop, but an `apk` process
will be started in the background to remove the dependency packages.
Running `pmbootstrap shutdown` at this time will not work, because the
`apk` process is still running.
With this commit, dependencies don't get cleaned up from the chroots.
Follow-up to !1373, where `pmbootstrap flasher flash_system` was
replaced with `pmbootstrap flasher flash_rootfs`. We still had used
terms like "system partition" in a lot of places.
This commit replaces it everywhere, so it's clear that we're talking
about the pmOS rootfs (which may or may not be installed to Android's
system partition).
The test suite needed a `pmbootstrap shutdown` after running through,
before it could successfully run again.
Explanation:
This was caused by `test/test_pkgrel_bump.py`, which creates a
temporary work folder with every subfolder ("chroot_native",
"cache_apk_x86_64", ...) linked to the original work folder except for
the "packages" folder. At the end of the test case,
`pmbootstrap shutdown` gets executed and is expected to umount
everything as usual. But it does not umount anything because of the
symlinks, so `work/chroot_native/mnt/pmbootstrap-packages` points to
the fake packages folder of that test case, even after it is finished.
As a result, any test case that tries to access the packages folder in
the native chroot, will fail until `pmbootstrap shutdown` gets called.
Detailed Changes:
* Umount all folders inside the work folder, even if these are symlinks
* Remove obsolete reference to "disable timestamp based rebuilds" in a
comment in `test/test_pkgrel_bump.py`
* Run `pmbootstrap work_migrate` and `pmbootstrap shutdown` at the
beginning of `test/testcases_fast.sh`, in case the pkgrel_bump test
case was aborted before it could properly shutdown and to make it
more robust in general (user may have changed the mountpoints, work
folder may need to be migrated)
Prior to this commit, it was possible to type in packages with a
trailing comma in `pmbootstrap init` when asked for extra packages.
This leads to problems during `pmbootstrap install`, so now we disallow
it. Fixes#1540.
It is unexpected for quite a lot of people, that the chroot folders are
still mounted when a pmbootstrap command has finished. With this commit,
it will let the user know explicitly:
> NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
Close#1524
We are analyzing the `boot.img` with `file` before we send it to
`unpackbootimg`. File does not recognize all kinds of `boot.img` files,
which `unpackbootimg` can extract, so we need a way to skip this check.
Details:
* Add `-f` parameter, continues extraction with a warning if the file
seems to be invalid
* Tell the user that `-f` can be used if the `boot.img` is invalid and
it's not specified
* Consistent spelling of `boot.img` instead of `bootimg` in messages
Fixes#1608
* Travis and Coveralls badges
* aports: instead of <https://github.com/postmarketOS>, use
<https://postmarketos.org>
* References to full URLs to issues and pull requests replaced with
a hash and the number
* grsec check: simplify error message, remove link to github issue
(nobody is using that anymore anyway)
This allows the user to talk to networkmanager to manage the system
connections. As it is mentioned in networkmanager pre-install.
```
Executing networkmanager-1.10.6-r0.pre-install
*
* To setup system connections, regular users must be member of 'plugdev' group.
```
The plugdev group gets created in the post-install hook of
networkmanager. Not all UIs depend on networkmanager, which means that
the group may not exist at installation time when we try to add the
user to the group in the python code. Therefore we create the group
first.
The linux APKBUILDs write the kernel config either to `$builddir`
(default from the template) or `$srcdir/build` (legacy, and I reverted
to that in #1556, which was not the proper fix for this regression).
With this commit, `pmbootstrap kconfig edit` is able to edit both
versions, and prints a note when the APKBUILD is still using the old
style.
Alpine ships `mkbootimg` with the `android-tools` package now. This
conflicts with the `mkbootimg` fork from osm0sis (see #441).
Changes:
* Rename `mkbootimg` to `mkbootimg-osm0sis` (aport and binary name)
* `mkbootimg-osm0sis`: provides `mkbootimg` now (so we don't need to
change all the device aports), update version to 2018.05.10
* Adjust our `mkinitfs` script to call `mkbootimg-osm0sis`
* Better aport description
* pkgrel_bump testcase: don't fail on deleted aport
Some kernels have a different `KBUILD_OUTPUT` path (e.g. #1551). When
the output path is different from `$srcdir/build`, then
`pmbootstrap kconfig edit` will not work (same with the previous
`pmbootstrap menuconfig` implementation).
This commit forces the output path to be `$srcdir/build` in the template
for new kernel aports, so we won't have that issue with future ports.
It is important, that we have DEVTMPFS enabled in the kernel config. But
it does not hurt to have DEVTMPFS_MOUNT enabled as well, and some of
Alpine's kernel configs have that by default. This commit removes the
check that forbids the option in kernel configs, so we can fork the
raspberry pi kernel package from Alpine without changing unrelated
options.
* change "pmbootstrap kconfig_check" to "pmbootstrap kconfig check"
* change "pmbootstrap menuconfig" to "pmbootstrap kconfig edit [-x|-g]"
(with legacy alias, because the first syntax was referenced to a lot)
* enable X11 interfaces: -x: xconfig, -g: gconfig
* new function to copy the xauthority file:
pmb.chroot.other.copy_xauthority()
* remove menufconfig() function from the kernel template and all kernel
aports ([skip ci] because it would rebuild all kernels and run out of
time). Alpine has dropped this as well, and it wouldn't work with the
new code anyway.
This makes the flasher work when no flash_method attribute exists on
the args variable, which happens when it is invoked through the
pmbootstrap export --odin command. This is a regression introduced
in b29cc877a7
This fixes#1527
Due to changes in abuild, our `gcc-armhf` etc. packages did not build
when using strict mode (i.e. `pmbootstrap build --strict gcc-armhf`)
anymore.
Changes:
* Set `CBUILDDIR=/`, so apk can read a valid package index from there
* Directly set `_cross_configure`, so it does not use CBUILDDIR anymore
* Set `BOOTSTRAP="nobuildbase"` to prevent apk from installing
`build-base-armhf` etc. (these don't exist in pmOS)
* Remove legacy code for lazy reproducible builds that wrapped
`package()`
Prevent the "Run 'pmbootstrap log' for details" message from being
written to the log file that gets read with "pmbootstrap log". Because
when the output of "pmbootstrap log" is pasted somewhere and people
analyze it, the message sounds like this is not the output of
"pmbootstrap log" (like it happened the other day in #postmarketOS).
* Usage: pmbootstrap install --split
* Make obvious that export is the next step when split images are created
* Fix note for missing rootfs image on export
* Change wording from "system image" to "rootfs image"
* The idea was to show the note only when the rootfs image was not
generated yet. But this was broken, because the path we checked for
was missing the chroot path prefix (which is added now).
* Also don't display the message, when the split image files exist
Device nodes in the chroots get created in a tmpfs, so they can be
created even if the filesystem where the chroot resides does not
support device nodes (#1317). In "pmbootstrap shutdown" we umount the
`dev` folder, which means all device nodes that were created inside
this folder are gone. This commit changes the code to actually recreate
the device nodes when using the chroot again.
Details:
* move `pmb.chroot.init.create_device_nodes` to
`pmb.chroot.mount.crete_device_nodes`
* don't call it in `pmb.chroot.init()` anymore, but in
`pmb.chroot.mount_dev_tmpfs()`
* Create the `null` device as well (`apk --initdb` also creates it on
`init`, but we don't call it after `shutdown`)
We require binfmt_misc to run programs of foreign architectures (e.g.
armhf) with QEMU. This is set up by default in most distributions, but
in some (e.g. Alpine, Void) it needs to be configured manually (see
the troubleshooting page in the wiki).
We have a check in place, which points to that troubleshooting wiki
page. However, the check was flawed, because we assumed the binfmt_misc
folder would not exist.
Thanks to @fxkrait for making the fix and for testing it!
The `msm-fb-refresher` updates the screen for msm based devices. It is
not needed for all devices, so we had some extra code in the initramfs,
that would only add it when the `deviceinfo_msm_refresher` variable was
set. However, we are able now to add files to initramfs hooks, so this
hack can be removed and simplify everything.
Changes:
* Remove `deviceinfo_msm_refresher` from all deviceinfos
* Add sanity check for it
* Move all `deviceinfo` sanity checks to an extra function
* `postmarketos-mkinitfs`: remove code for msm refresher
* `msm-fb-refresher`: add initramfs hook
The postmarketos-base package used to make the user part of the "video"
and "audio" groups. However, this did not work reliably, and we were
adding the "wheel" group in "pmbootstrap install" anyway.
Now all groups get added in "pmbootstrap install", and the names of the
groups have been moved to `pmb.config.install_user_groups`.
Changes:
* `helpers/envkernel.sh`:
* installs everything needed for kernel compilation in the native
chroot
* mounts the kernel source to `/mnt/linux` inside the chroot
* creates `/mnt/linux/.output` and chowns it to the `pmos` user, that
folder will be used for the kernel build output
* sets up aliases for `make`, `pmbootstrap`, `pmbroot`, `kernelroot`
* new action `pmbootstrap work_migrate`: does the interactive work
folder migration if necessary, otherwise it doesn't output anything
* when calling this first, we can safely use all other commands
non-interactively without showing the output
Benefits:
* Fast setup (especially for people who are new to kernel
compilation
* No need to figure out distribution specific package names
(cross compilers!)
* No need to do a test build just to verify that the right
packages are installed
* Less error prone
* The right dependencies are always installed
* `ARCH` and `CROSS_COMPILE` variables always get set automatically
and based on `deviceinfo_arch`
* If the build environment is broken for some reason, just zap and
start over
* Easy to reproduce problems
Notes:
* `make menuconfig` works as well
* Sourcing was tested with `zsh`, `bash` and `fish`, it should be easy to
extend for other shells
Building packages without git installed caused error messages from git
appearing on the screen, and I patched it upstream (see #1209).
However, this introduced a regression: when git *is* installed and the
aport folder *is not* part of a git repository (like we do it with
pmbootstrap), the build gets aborted now with git complaining that
the folder is not a valid git folder.
But the only case where this happens is, when abuild is trying to
include the git repositories' commit hash for the APKINDEX description.
This pmbootstrap commit hardcodes "postmarketOS" as APKINDEX
description, so it does not fail anymore.
`pmbootstrap pkgrel_bump --auto` automatically increases the pkgrel for
packages linking against libraries, which don't exist anymore (because
the soname has been changed). The feature is explained in detail in
The previous implementation did not detect soname breakage, when a
subpackage linked against a certain library, but the main package
did not (e.g. `qt5-qtbase-mysql` and `qt5-qtbase`). This was, because
we iterated over the aports/* to find the packages to be checked.
To fix this, we are iterating over the packages found in the APKINDEX
files instead (of both the locally compiled packages and the downloaded
index from the pmOS mirror).
Details:
* `pmb/helpers/pkgrel_bump.py`:
* Rewrite `auto_apkindex_package()` to act upon a given parsed
`aport` and `apk` (from the index) instead of finding the `apk`
dict by itself (we need it earlier anyway).
* Rewrite `auto()` to iterate over APKINDEX files instead of aports
* Skip packages already found, so the `pkgrel` does not get
increased multiple times when the same package was found in
multipe index files.
* Put the package name at the beginning of the log messages to make
them more readable
* testdata: Create a new `testsubpkg` aport, where only the subpackage
links against `testlib`
* Adjust testing code to test everything with `testsubpkg` as well.
NOTE: This makes the command a bit slower, but we could improve
performance again by smart caching of `pmb.parse.apkindex.depends()`.
This could come in a future PR, the important part here is that the
command is bug-free again with this fix.
Oliver Smith
Clayton Craft
Bhushan Shah
Robert Yang
Daniele Debernardi
Grant Miller
Ion Agorria
Martijn Braam
Luca Weiss
Craig Comstock
Zhuowei Zhang
raingloom
NotKit
Marti Raudsepp
steamport
George Hopkins
Attila Szöllősi
Marcin Mielniczuk