Overview:
In order to execute foreign arch binaries on the host system, we are
using the Linux kernel's binfmt_misc feature in combination with
static builds of QEMU. Before this patch, the statically compiled
QEMU binaries were taken from Debian (mostly because I did not realize
that Alpine ships them as well). Now we can use the ones from the aport.
Benefits:
This allows us to easily update and patch the QEMU executables, we
don't need to be in sync with Debian's versions anymore.
Alpine's package is more modular, so we can save some download,
install, zap time, as well as disk space: setting up an armhf chroot
with pmbootstrap took ~102 MB before, now it's ~18 MB.
Detailed changes:
* Remove `cross/qemu-user-static-repack` aport
* Add `data/qemu-user-binfmt.txt` with the binfmt_misc flags for ELF
binaries of various arches (extracted from Debian's packaging)
* When parsing that file, don't write verbose messages to
`pmbootstrap log` anymore, only to the verbose log (can be enabled
with `pmbootstrap -v`)
* Rename `pmb.parse.arch.alpine_to_debian()` to ...`alpine_to_qemu()`
* Rename `arch_debian` to `arch_qemu`
This commit adds a test case, which makes sure that the KDE framework
and plasma framework version are always the same.
Additional changes:
* APKBUILD parser parses the URL now (that's the best way I found to
categorize the KDE aports in frameworks and other)
* Changed single quotes to double quotes in KDE APKBUILDs, so the
parser doesn't include the single quotes in the parsed result
* Added the test case to the gitlab CI config
With this patch, "pmbootstrap flasher" will fail with "the following
arguments are required: action_flasher". Without it, it just prints
"Done" and quits.
Overview:
Since Alpine updated to distcc 3.3 last week, pmbootstrap wasn't able to use
distcc for cross compilation anymore. It always falled back to running the
compiler in QEMU (which works, but is a lot slower). The reason for that is,
that distcc requires all compilers that are being used in a whitelist now.
This partially fixes CVE-2004-2687 in distccd, which allowed trivial remote
code execution by any process connecting to the distccd server. We only run
distccd on localhost, but still this can be used for privilege escalation of
sandboxed processes running on the host system (not part of pmbootstrap
chroots).
Because the CVE is only partially fixed (see the comment in
`pmb/chroot/distccd.py` for details), we make sure that only the building
chroots can talk to the distcc server by running distcc over ssh.
Details:
* Completely refactored `pmb/chroot/distccd.py` to run distcc over ssh
* Store the running distcc server's arguments as JSON now, not as INI
* Make debugging distcc issues easy:
* Set DISTCC_BACKOFF_PERIOD=0, so the distcc client will not ignore the
server after errors happened (this masks the original error!)
* New pmbootstrap parameters:
* `--distcc-nofallback`: avoids falling back to compiling with QEMU and not
throwing an error
* `--ccache-disable`: avoid ccache (when the compiler output is cached,
distcc does not get used)
* `--verbose` prints verbose output of the distcc too
* New test case, that uses the new pmbootstrap parameters to force
compilation through distcc, and shows the output of distcc and distccd in
verbose mode on error (as well as the log of sshd)
Alpine's `abuild` will uninstall all dependencies by default, when a
package build fails.
Leaving this configuration unchanged leads to unexpected behavior with
pmbootstrap: when executing `pmbootstrap build --strict` and pressing
`^C` during the build, pmbootstrap will stop, but an `apk` process
will be started in the background to remove the dependency packages.
Running `pmbootstrap shutdown` at this time will not work, because the
`apk` process is still running.
With this commit, dependencies don't get cleaned up from the chroots.
Follow-up to !1373, where `pmbootstrap flasher flash_system` was
replaced with `pmbootstrap flasher flash_rootfs`. We still had used
terms like "system partition" in a lot of places.
This commit replaces it everywhere, so it's clear that we're talking
about the pmOS rootfs (which may or may not be installed to Android's
system partition).
The test suite needed a `pmbootstrap shutdown` after running through,
before it could successfully run again.
Explanation:
This was caused by `test/test_pkgrel_bump.py`, which creates a
temporary work folder with every subfolder ("chroot_native",
"cache_apk_x86_64", ...) linked to the original work folder except for
the "packages" folder. At the end of the test case,
`pmbootstrap shutdown` gets executed and is expected to umount
everything as usual. But it does not umount anything because of the
symlinks, so `work/chroot_native/mnt/pmbootstrap-packages` points to
the fake packages folder of that test case, even after it is finished.
As a result, any test case that tries to access the packages folder in
the native chroot, will fail until `pmbootstrap shutdown` gets called.
Detailed Changes:
* Umount all folders inside the work folder, even if these are symlinks
* Remove obsolete reference to "disable timestamp based rebuilds" in a
comment in `test/test_pkgrel_bump.py`
* Run `pmbootstrap work_migrate` and `pmbootstrap shutdown` at the
beginning of `test/testcases_fast.sh`, in case the pkgrel_bump test
case was aborted before it could properly shutdown and to make it
more robust in general (user may have changed the mountpoints, work
folder may need to be migrated)
Prior to this commit, it was possible to type in packages with a
trailing comma in `pmbootstrap init` when asked for extra packages.
This leads to problems during `pmbootstrap install`, so now we disallow
it. Fixes#1540.
It is unexpected for quite a lot of people, that the chroot folders are
still mounted when a pmbootstrap command has finished. With this commit,
it will let the user know explicitly:
> NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
Close#1524
We are analyzing the `boot.img` with `file` before we send it to
`unpackbootimg`. File does not recognize all kinds of `boot.img` files,
which `unpackbootimg` can extract, so we need a way to skip this check.
Details:
* Add `-f` parameter, continues extraction with a warning if the file
seems to be invalid
* Tell the user that `-f` can be used if the `boot.img` is invalid and
it's not specified
* Consistent spelling of `boot.img` instead of `bootimg` in messages
Fixes#1608
* Travis and Coveralls badges
* aports: instead of <https://github.com/postmarketOS>, use
<https://postmarketos.org>
* References to full URLs to issues and pull requests replaced with
a hash and the number
* grsec check: simplify error message, remove link to github issue
(nobody is using that anymore anyway)
This allows the user to talk to networkmanager to manage the system
connections. As it is mentioned in networkmanager pre-install.
```
Executing networkmanager-1.10.6-r0.pre-install
*
* To setup system connections, regular users must be member of 'plugdev' group.
```
The plugdev group gets created in the post-install hook of
networkmanager. Not all UIs depend on networkmanager, which means that
the group may not exist at installation time when we try to add the
user to the group in the python code. Therefore we create the group
first.
The linux APKBUILDs write the kernel config either to `$builddir`
(default from the template) or `$srcdir/build` (legacy, and I reverted
to that in #1556, which was not the proper fix for this regression).
With this commit, `pmbootstrap kconfig edit` is able to edit both
versions, and prints a note when the APKBUILD is still using the old
style.
Alpine ships `mkbootimg` with the `android-tools` package now. This
conflicts with the `mkbootimg` fork from osm0sis (see #441).
Changes:
* Rename `mkbootimg` to `mkbootimg-osm0sis` (aport and binary name)
* `mkbootimg-osm0sis`: provides `mkbootimg` now (so we don't need to
change all the device aports), update version to 2018.05.10
* Adjust our `mkinitfs` script to call `mkbootimg-osm0sis`
* Better aport description
* pkgrel_bump testcase: don't fail on deleted aport
Some kernels have a different `KBUILD_OUTPUT` path (e.g. #1551). When
the output path is different from `$srcdir/build`, then
`pmbootstrap kconfig edit` will not work (same with the previous
`pmbootstrap menuconfig` implementation).
This commit forces the output path to be `$srcdir/build` in the template
for new kernel aports, so we won't have that issue with future ports.
It is important, that we have DEVTMPFS enabled in the kernel config. But
it does not hurt to have DEVTMPFS_MOUNT enabled as well, and some of
Alpine's kernel configs have that by default. This commit removes the
check that forbids the option in kernel configs, so we can fork the
raspberry pi kernel package from Alpine without changing unrelated
options.
* change "pmbootstrap kconfig_check" to "pmbootstrap kconfig check"
* change "pmbootstrap menuconfig" to "pmbootstrap kconfig edit [-x|-g]"
(with legacy alias, because the first syntax was referenced to a lot)
* enable X11 interfaces: -x: xconfig, -g: gconfig
* new function to copy the xauthority file:
pmb.chroot.other.copy_xauthority()
* remove menufconfig() function from the kernel template and all kernel
aports ([skip ci] because it would rebuild all kernels and run out of
time). Alpine has dropped this as well, and it wouldn't work with the
new code anyway.
This makes the flasher work when no flash_method attribute exists on
the args variable, which happens when it is invoked through the
pmbootstrap export --odin command. This is a regression introduced
in b29cc877a7
This fixes#1527
Due to changes in abuild, our `gcc-armhf` etc. packages did not build
when using strict mode (i.e. `pmbootstrap build --strict gcc-armhf`)
anymore.
Changes:
* Set `CBUILDDIR=/`, so apk can read a valid package index from there
* Directly set `_cross_configure`, so it does not use CBUILDDIR anymore
* Set `BOOTSTRAP="nobuildbase"` to prevent apk from installing
`build-base-armhf` etc. (these don't exist in pmOS)
* Remove legacy code for lazy reproducible builds that wrapped
`package()`
Prevent the "Run 'pmbootstrap log' for details" message from being
written to the log file that gets read with "pmbootstrap log". Because
when the output of "pmbootstrap log" is pasted somewhere and people
analyze it, the message sounds like this is not the output of
"pmbootstrap log" (like it happened the other day in #postmarketOS).
* Usage: pmbootstrap install --split
* Make obvious that export is the next step when split images are created
* Fix note for missing rootfs image on export
* Change wording from "system image" to "rootfs image"
* The idea was to show the note only when the rootfs image was not
generated yet. But this was broken, because the path we checked for
was missing the chroot path prefix (which is added now).
* Also don't display the message, when the split image files exist
Device nodes in the chroots get created in a tmpfs, so they can be
created even if the filesystem where the chroot resides does not
support device nodes (#1317). In "pmbootstrap shutdown" we umount the
`dev` folder, which means all device nodes that were created inside
this folder are gone. This commit changes the code to actually recreate
the device nodes when using the chroot again.
Details:
* move `pmb.chroot.init.create_device_nodes` to
`pmb.chroot.mount.crete_device_nodes`
* don't call it in `pmb.chroot.init()` anymore, but in
`pmb.chroot.mount_dev_tmpfs()`
* Create the `null` device as well (`apk --initdb` also creates it on
`init`, but we don't call it after `shutdown`)
raingloom
Grant Miller
Oliver Smith
NotKit
ryang
Marti Raudsepp
Daniele Debernardi
Bhushan Shah
steamport
George Hopkins
Martijn Braam
Attila Szöllősi
Marcin Mielniczuk