The test suite needed a `pmbootstrap shutdown` after running through,
before it could successfully run again.
Explanation:
This was caused by `test/test_pkgrel_bump.py`, which creates a
temporary work folder with every subfolder ("chroot_native",
"cache_apk_x86_64", ...) linked to the original work folder except for
the "packages" folder. At the end of the test case,
`pmbootstrap shutdown` gets executed and is expected to umount
everything as usual. But it does not umount anything because of the
symlinks, so `work/chroot_native/mnt/pmbootstrap-packages` points to
the fake packages folder of that test case, even after it is finished.
As a result, any test case that tries to access the packages folder in
the native chroot, will fail until `pmbootstrap shutdown` gets called.
Detailed Changes:
* Umount all folders inside the work folder, even if these are symlinks
* Remove obsolete reference to "disable timestamp based rebuilds" in a
comment in `test/test_pkgrel_bump.py`
* Run `pmbootstrap work_migrate` and `pmbootstrap shutdown` at the
beginning of `test/testcases_fast.sh`, in case the pkgrel_bump test
case was aborted before it could properly shutdown and to make it
more robust in general (user may have changed the mountpoints, work
folder may need to be migrated)
Prior to this commit, it was possible to type in packages with a
trailing comma in `pmbootstrap init` when asked for extra packages.
This leads to problems during `pmbootstrap install`, so now we disallow
it. Fixes#1540.
It is unexpected for quite a lot of people, that the chroot folders are
still mounted when a pmbootstrap command has finished. With this commit,
it will let the user know explicitly:
> NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
Close#1524
We are analyzing the `boot.img` with `file` before we send it to
`unpackbootimg`. File does not recognize all kinds of `boot.img` files,
which `unpackbootimg` can extract, so we need a way to skip this check.
Details:
* Add `-f` parameter, continues extraction with a warning if the file
seems to be invalid
* Tell the user that `-f` can be used if the `boot.img` is invalid and
it's not specified
* Consistent spelling of `boot.img` instead of `bootimg` in messages
Fixes#1608
* Travis and Coveralls badges
* aports: instead of <https://github.com/postmarketOS>, use
<https://postmarketos.org>
* References to full URLs to issues and pull requests replaced with
a hash and the number
* grsec check: simplify error message, remove link to github issue
(nobody is using that anymore anyway)
This allows the user to talk to networkmanager to manage the system
connections. As it is mentioned in networkmanager pre-install.
```
Executing networkmanager-1.10.6-r0.pre-install
*
* To setup system connections, regular users must be member of 'plugdev' group.
```
The plugdev group gets created in the post-install hook of
networkmanager. Not all UIs depend on networkmanager, which means that
the group may not exist at installation time when we try to add the
user to the group in the python code. Therefore we create the group
first.
The linux APKBUILDs write the kernel config either to `$builddir`
(default from the template) or `$srcdir/build` (legacy, and I reverted
to that in #1556, which was not the proper fix for this regression).
With this commit, `pmbootstrap kconfig edit` is able to edit both
versions, and prints a note when the APKBUILD is still using the old
style.
Alpine ships `mkbootimg` with the `android-tools` package now. This
conflicts with the `mkbootimg` fork from osm0sis (see #441).
Changes:
* Rename `mkbootimg` to `mkbootimg-osm0sis` (aport and binary name)
* `mkbootimg-osm0sis`: provides `mkbootimg` now (so we don't need to
change all the device aports), update version to 2018.05.10
* Adjust our `mkinitfs` script to call `mkbootimg-osm0sis`
* Better aport description
* pkgrel_bump testcase: don't fail on deleted aport
Some kernels have a different `KBUILD_OUTPUT` path (e.g. #1551). When
the output path is different from `$srcdir/build`, then
`pmbootstrap kconfig edit` will not work (same with the previous
`pmbootstrap menuconfig` implementation).
This commit forces the output path to be `$srcdir/build` in the template
for new kernel aports, so we won't have that issue with future ports.
It is important, that we have DEVTMPFS enabled in the kernel config. But
it does not hurt to have DEVTMPFS_MOUNT enabled as well, and some of
Alpine's kernel configs have that by default. This commit removes the
check that forbids the option in kernel configs, so we can fork the
raspberry pi kernel package from Alpine without changing unrelated
options.
* change "pmbootstrap kconfig_check" to "pmbootstrap kconfig check"
* change "pmbootstrap menuconfig" to "pmbootstrap kconfig edit [-x|-g]"
(with legacy alias, because the first syntax was referenced to a lot)
* enable X11 interfaces: -x: xconfig, -g: gconfig
* new function to copy the xauthority file:
pmb.chroot.other.copy_xauthority()
* remove menufconfig() function from the kernel template and all kernel
aports ([skip ci] because it would rebuild all kernels and run out of
time). Alpine has dropped this as well, and it wouldn't work with the
new code anyway.
This makes the flasher work when no flash_method attribute exists on
the args variable, which happens when it is invoked through the
pmbootstrap export --odin command. This is a regression introduced
in b29cc877a7
This fixes#1527
Due to changes in abuild, our `gcc-armhf` etc. packages did not build
when using strict mode (i.e. `pmbootstrap build --strict gcc-armhf`)
anymore.
Changes:
* Set `CBUILDDIR=/`, so apk can read a valid package index from there
* Directly set `_cross_configure`, so it does not use CBUILDDIR anymore
* Set `BOOTSTRAP="nobuildbase"` to prevent apk from installing
`build-base-armhf` etc. (these don't exist in pmOS)
* Remove legacy code for lazy reproducible builds that wrapped
`package()`
Prevent the "Run 'pmbootstrap log' for details" message from being
written to the log file that gets read with "pmbootstrap log". Because
when the output of "pmbootstrap log" is pasted somewhere and people
analyze it, the message sounds like this is not the output of
"pmbootstrap log" (like it happened the other day in #postmarketOS).
* Usage: pmbootstrap install --split
* Make obvious that export is the next step when split images are created
* Fix note for missing rootfs image on export
* Change wording from "system image" to "rootfs image"
* The idea was to show the note only when the rootfs image was not
generated yet. But this was broken, because the path we checked for
was missing the chroot path prefix (which is added now).
* Also don't display the message, when the split image files exist
Device nodes in the chroots get created in a tmpfs, so they can be
created even if the filesystem where the chroot resides does not
support device nodes (#1317). In "pmbootstrap shutdown" we umount the
`dev` folder, which means all device nodes that were created inside
this folder are gone. This commit changes the code to actually recreate
the device nodes when using the chroot again.
Details:
* move `pmb.chroot.init.create_device_nodes` to
`pmb.chroot.mount.crete_device_nodes`
* don't call it in `pmb.chroot.init()` anymore, but in
`pmb.chroot.mount_dev_tmpfs()`
* Create the `null` device as well (`apk --initdb` also creates it on
`init`, but we don't call it after `shutdown`)
We require binfmt_misc to run programs of foreign architectures (e.g.
armhf) with QEMU. This is set up by default in most distributions, but
in some (e.g. Alpine, Void) it needs to be configured manually (see
the troubleshooting page in the wiki).
We have a check in place, which points to that troubleshooting wiki
page. However, the check was flawed, because we assumed the binfmt_misc
folder would not exist.
Thanks to @fxkrait for making the fix and for testing it!
The `msm-fb-refresher` updates the screen for msm based devices. It is
not needed for all devices, so we had some extra code in the initramfs,
that would only add it when the `deviceinfo_msm_refresher` variable was
set. However, we are able now to add files to initramfs hooks, so this
hack can be removed and simplify everything.
Changes:
* Remove `deviceinfo_msm_refresher` from all deviceinfos
* Add sanity check for it
* Move all `deviceinfo` sanity checks to an extra function
* `postmarketos-mkinitfs`: remove code for msm refresher
* `msm-fb-refresher`: add initramfs hook
The postmarketos-base package used to make the user part of the "video"
and "audio" groups. However, this did not work reliably, and we were
adding the "wheel" group in "pmbootstrap install" anyway.
Now all groups get added in "pmbootstrap install", and the names of the
groups have been moved to `pmb.config.install_user_groups`.
Changes:
* `helpers/envkernel.sh`:
* installs everything needed for kernel compilation in the native
chroot
* mounts the kernel source to `/mnt/linux` inside the chroot
* creates `/mnt/linux/.output` and chowns it to the `pmos` user, that
folder will be used for the kernel build output
* sets up aliases for `make`, `pmbootstrap`, `pmbroot`, `kernelroot`
* new action `pmbootstrap work_migrate`: does the interactive work
folder migration if necessary, otherwise it doesn't output anything
* when calling this first, we can safely use all other commands
non-interactively without showing the output
Benefits:
* Fast setup (especially for people who are new to kernel
compilation
* No need to figure out distribution specific package names
(cross compilers!)
* No need to do a test build just to verify that the right
packages are installed
* Less error prone
* The right dependencies are always installed
* `ARCH` and `CROSS_COMPILE` variables always get set automatically
and based on `deviceinfo_arch`
* If the build environment is broken for some reason, just zap and
start over
* Easy to reproduce problems
Notes:
* `make menuconfig` works as well
* Sourcing was tested with `zsh`, `bash` and `fish`, it should be easy to
extend for other shells
Building packages without git installed caused error messages from git
appearing on the screen, and I patched it upstream (see #1209).
However, this introduced a regression: when git *is* installed and the
aport folder *is not* part of a git repository (like we do it with
pmbootstrap), the build gets aborted now with git complaining that
the folder is not a valid git folder.
But the only case where this happens is, when abuild is trying to
include the git repositories' commit hash for the APKINDEX description.
This pmbootstrap commit hardcodes "postmarketOS" as APKINDEX
description, so it does not fail anymore.
`pmbootstrap pkgrel_bump --auto` automatically increases the pkgrel for
packages linking against libraries, which don't exist anymore (because
the soname has been changed). The feature is explained in detail in
The previous implementation did not detect soname breakage, when a
subpackage linked against a certain library, but the main package
did not (e.g. `qt5-qtbase-mysql` and `qt5-qtbase`). This was, because
we iterated over the aports/* to find the packages to be checked.
To fix this, we are iterating over the packages found in the APKINDEX
files instead (of both the locally compiled packages and the downloaded
index from the pmOS mirror).
Details:
* `pmb/helpers/pkgrel_bump.py`:
* Rewrite `auto_apkindex_package()` to act upon a given parsed
`aport` and `apk` (from the index) instead of finding the `apk`
dict by itself (we need it earlier anyway).
* Rewrite `auto()` to iterate over APKINDEX files instead of aports
* Skip packages already found, so the `pkgrel` does not get
increased multiple times when the same package was found in
multipe index files.
* Put the package name at the beginning of the log messages to make
them more readable
* testdata: Create a new `testsubpkg` aport, where only the subpackage
links against `testlib`
* Adjust testing code to test everything with `testsubpkg` as well.
NOTE: This makes the command a bit slower, but we could improve
performance again by smart caching of `pmb.parse.apkindex.depends()`.
This could come in a future PR, the important part here is that the
command is bug-free again with this fix.
* As discussed in IRC/matrix, we're removing `linux-postmarketos-lts`
for now. The kernel isn't used right now, and we save lots of
maintenance effort with not updating it every week or so.
* new config option `"kernel"` with possible values:
`"downstream", "mainline", "stable"` (downstream is always
`linux-$devicename`)
* ask for the kernel during `pmbootstrap init` if the device package
has kernel subpackages and install it in `_install.py`
* postmarketos-mkinitfs: display note instead of exit with error when
the `deviceinfo_dtb` file is missing (because we expect it to be
missing for downstream kernels)
* device-sony-amami:
* add kernel subpackages for downstream, mainline
* set `deviceinfo_dtb`
* device-qemu-amd64: add kernel subpackages for stable, lts, mainline
* test cases and test data for new functions
* test case that checks all aports for right usage of the feature:
* don't mix specifying kernels in depends *and* subpackages
* 1 kernel in depends is maximum
* kernel subpackages must have a valid name
* Test if devices packages reference at least one kernel
* Remove `_build_device_depends_note()` which informs the user that
`--ignore-depends` can be used with device packages to avoid building
the kernel. The idea was to make the transition easier after a change
we did months ago, and now the kernel doesn't always get built before
building the device package so it's not relevant anymore.
* pmb/chroot/other.py:
* Add autoinstall=True to kernel_flavors_installed(). When the flag
is set, the function makes sure that at least one kernel for the
device is installed.
* Remove kernel_flavor_autodetect() function, wherever it was used,
it has been replaced with kernel_flavors_installed()[0].
* pmb.helpers.frontend.py: remove code to install at least one kernel,
kernel_flavors_installed() takes care of that now.
As noted in commit 255c715624
`/var/cache/distfiles` is writable by everyone. It is supposed to be
writable only by `root` and by the `abuild` group (in which we put the
`pmos` user already for building packages).
Changes:
* `pmb.build.init()`: make `/var/cache/distfiles` writable only by
members of the `abuild` group (and root)
* Increase workfolder version to 2
* Add migration code that fixes the permissions for existing work
folders
* Refactor the migration code a bit to make this possible
* Change `pmbootstrap flasher flash_system` command to
`pmbootstrap flasher flash_rootfs`
* The old command still works, but all references have been changed to
the new command
* Remove obsolete `pmbootstrap flasher export` (that was changed to
`pmbootstrap export` a few months ago)
* Update `README.md` and ZSH auto completion
* Change the description of the generated rootfs image (not talking
about a system image anymore, mention that it has subpartitions)
* Better description of `pmbootstrap flasher flash_rootfs --partition`