This commit adds a test case, which makes sure that the KDE framework
and plasma framework version are always the same.
Additional changes:
* APKBUILD parser parses the URL now (that's the best way I found to
categorize the KDE aports in frameworks and other)
* Changed single quotes to double quotes in KDE APKBUILDs, so the
parser doesn't include the single quotes in the parsed result
* Added the test case to the gitlab CI config
Follow-up to !1373, where `pmbootstrap flasher flash_system` was
replaced with `pmbootstrap flasher flash_rootfs`. We still had used
terms like "system partition" in a lot of places.
This commit replaces it everywhere, so it's clear that we're talking
about the pmOS rootfs (which may or may not be installed to Android's
system partition).
* Travis and Coveralls badges
* aports: instead of <https://github.com/postmarketOS>, use
<https://postmarketos.org>
* References to full URLs to issues and pull requests replaced with
a hash and the number
* grsec check: simplify error message, remove link to github issue
(nobody is using that anymore anyway)
This allows the user to talk to networkmanager to manage the system
connections. As it is mentioned in networkmanager pre-install.
```
Executing networkmanager-1.10.6-r0.pre-install
*
* To setup system connections, regular users must be member of 'plugdev' group.
```
The plugdev group gets created in the post-install hook of
networkmanager. Not all UIs depend on networkmanager, which means that
the group may not exist at installation time when we try to add the
user to the group in the python code. Therefore we create the group
first.
It is important, that we have DEVTMPFS enabled in the kernel config. But
it does not hurt to have DEVTMPFS_MOUNT enabled as well, and some of
Alpine's kernel configs have that by default. This commit removes the
check that forbids the option in kernel configs, so we can fork the
raspberry pi kernel package from Alpine without changing unrelated
options.
Device nodes in the chroots get created in a tmpfs, so they can be
created even if the filesystem where the chroot resides does not
support device nodes (#1317). In "pmbootstrap shutdown" we umount the
`dev` folder, which means all device nodes that were created inside
this folder are gone. This commit changes the code to actually recreate
the device nodes when using the chroot again.
Details:
* move `pmb.chroot.init.create_device_nodes` to
`pmb.chroot.mount.crete_device_nodes`
* don't call it in `pmb.chroot.init()` anymore, but in
`pmb.chroot.mount_dev_tmpfs()`
* Create the `null` device as well (`apk --initdb` also creates it on
`init`, but we don't call it after `shutdown`)
The postmarketos-base package used to make the user part of the "video"
and "audio" groups. However, this did not work reliably, and we were
adding the "wheel" group in "pmbootstrap install" anyway.
Now all groups get added in "pmbootstrap install", and the names of the
groups have been moved to `pmb.config.install_user_groups`.
* As discussed in IRC/matrix, we're removing `linux-postmarketos-lts`
for now. The kernel isn't used right now, and we save lots of
maintenance effort with not updating it every week or so.
* new config option `"kernel"` with possible values:
`"downstream", "mainline", "stable"` (downstream is always
`linux-$devicename`)
* ask for the kernel during `pmbootstrap init` if the device package
has kernel subpackages and install it in `_install.py`
* postmarketos-mkinitfs: display note instead of exit with error when
the `deviceinfo_dtb` file is missing (because we expect it to be
missing for downstream kernels)
* device-sony-amami:
* add kernel subpackages for downstream, mainline
* set `deviceinfo_dtb`
* device-qemu-amd64: add kernel subpackages for stable, lts, mainline
* test cases and test data for new functions
* test case that checks all aports for right usage of the feature:
* don't mix specifying kernels in depends *and* subpackages
* 1 kernel in depends is maximum
* kernel subpackages must have a valid name
* Test if devices packages reference at least one kernel
* Remove `_build_device_depends_note()` which informs the user that
`--ignore-depends` can be used with device packages to avoid building
the kernel. The idea was to make the transition easier after a change
we did months ago, and now the kernel doesn't always get built before
building the device package so it's not relevant anymore.
* pmb/chroot/other.py:
* Add autoinstall=True to kernel_flavors_installed(). When the flag
is set, the function makes sure that at least one kernel for the
device is installed.
* Remove kernel_flavor_autodetect() function, wherever it was used,
it has been replaced with kernel_flavors_installed()[0].
* pmb.helpers.frontend.py: remove code to install at least one kernel,
kernel_flavors_installed() takes care of that now.
As noted in commit 255c715624
`/var/cache/distfiles` is writable by everyone. It is supposed to be
writable only by `root` and by the `abuild` group (in which we put the
`pmos` user already for building packages).
Changes:
* `pmb.build.init()`: make `/var/cache/distfiles` writable only by
members of the `abuild` group (and root)
* Increase workfolder version to 2
* Add migration code that fixes the permissions for existing work
folders
* Refactor the migration code a bit to make this possible
* Change `pmbootstrap flasher flash_system` command to
`pmbootstrap flasher flash_rootfs`
* The old command still works, but all references have been changed to
the new command
* Remove obsolete `pmbootstrap flasher export` (that was changed to
`pmbootstrap export` a few months ago)
* Update `README.md` and ZSH auto completion
* Change the description of the generated rootfs image (not talking
about a system image anymore, mention that it has subpartitions)
* Better description of `pmbootstrap flasher flash_rootfs --partition`
* Save "" (empty string) in the user's config as hostname if the user
let it default to the name of the device. That way, when the device
gets changed, the user won't get the old device's name as hostname
by accident.
* Add a test case
* pmbootstrap newapkbuild: Properly parse arguments
The `pmbootstrap newapkbuild` action wraps Alpine's `newapkbuild`. We
used to directly pass all arguments to `newapkbuild` without verifying
in Python whether they make sense or not. However, as `newpakbuild`
doesn't do strict sanity checks on the arguments, it is easy to end up
with unexpected behavior when using the command for the first time.
For example, `newapkbuild` allows either specifying a PKGNAME or SRCURL
as last parameter, and also allows setting a PKGNAME with the `-n`
parameter. It only makes sense to use that option when passing a
SRCURL.
With this commit, we duplicate the optins that should be passed through
to `newapkbuild` and use argparse to fully sanitize the options and
display a help page (`pmbootstrap newapkbuild -h`) that is consistent
with the other help pages.
Details:
* The `-f` (force) flag does not get passed through anymore. Instead we
use it in Python to skip asking if an existing aport should be
overwritten (the aports are outside of the chroot, so `newapkbuild`
can't handle it in a way that makes sense for pmbootstrap).
* Output of `newapkbuild` gets redirected to the log file now, as we
don't need it to display a help page.
* Don't verify the pkgver while creating the new APKBUILD. When passing
a SRCURL, the pkgver gets extracted from the end of the URL and may
not have a valid format yet (but we want the APKBUILD anyway).
* Stored options passed through in `pmb/config/__init__.py` and use it
in both `pmb/parse/arguments.py` and `pmb/helpers/frontend.py`.
* Only allow `-n` with SRCURL
* The postmarketOS aports folder gets specified with `--folder` now.
That way the generated help page is much closer to the original one
from `newapkbuild`. The default is `main`.
* Made the package type flags (CMake, autotools, ...) exclusive so only
one of them can be specified
Use case: `mkbootimg` provides the `unpackbootimg` package. When
running `pmb.chroot.apk.install(args,"unpackbootimg")`, it was not
able to properly build the package.
Reproducing the error:
```
sudo rm ~/.local/var/pmbootstrap/packages/x86_64/mkbootimg*
pmbootstrap index
pmbootstrap --mirror-pmOS="" chroot --add=unpackbootimg
```
Or alternatively (simpler but less illustrative):
```
pmbootstrap build unpackbootimg --force
```
Here are the changes necessary in pmbootstrap to make proprietary
software installed onto the device (firmware and userspace drivers)
optional (#756). To full close the issue, we need to apply this concept
to all device packages we already have in a follow-up PR.
Changes:
* New config file options nonfree_firmware and nonfree_userland, which
we ask for during "pmbootstrap init" if there are non-free components
for the selected device.
* We find that out by checking the APKBUILD's subpakages: The non-free
packages are called $pkgname-nonfree-firmware and
$pkgname-nonfree-userland.
* During "pmbootstrap init" we also show the pkgdesc of these
subpackages. Parsing that is implemented in
pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of
the regular APKBUILD parsing, as this would need a change in the
output format, and it is a lot *less* code if done like in this
commit.
* pmb/parse/apkbuild.py was renamed to _apkbuild.py, and
pmb/install/install.py to _install.py: needed to call the function in
the usual way (e.g. pmb.parse.apkbuild()) but still being able to
test the individual functions from these files in the test suite.
We did the same thing for pmb/build/_package.py already.
* Install: New function get_nonfree_packages() returns the non-free
packages that will be installed, based on the user's choice in
"pmbootstrap init" and on the subpackages the device has.
* Added test cases and test data (APKBUILDs) for all new code,
refactored test/test_questions.py to have multiple functions for
testing the various questions / question types from
"pmbootstrap init" instead of having it all in one big function.
This allows to use another aport folder for testing the new
non-free related questions in init.
The 'necessary_kconfig_options' dictionary in pmb/config/__init__.py
now has the different architectures (space separated) as the keys and
the dictionary, which matches kernel config options and their
expected value, as its value.
For that purpose, the 'check' function in pmb/parse/kconfig.py was
modified, so that it takes the architecture from the kconfig filename
and uses it to find the needed kernel config options.
Closes#1218.
* Fail if mkbootimg/uboot-tools are not installed, but creating a
boot.img file / u-boot legacy image was requested via deviceinfo
(fixes#312)
* Fail if /boot/dt.img is missing, but we have a qcdt device
* Fail if the dtb file specified in deviceinfo does not exist
* Fail if mkbootimg etc. exit with error code
* Don't try to add the ext4 module into the initramfs. We always
compile it into the kernel. Instead, kconfig_check makes sure it
is enabled now. (fixes#1037)
* Add a note that modprobe warnings can be ignored mostly
If you want to build a package without changing the version number,
please use `--force` from now on. For example:
pmbootstrap build --force hello-world
Prior to this commit, changes were detected automatically (timestamp
based rebuilds). However, that feature does not work as expected with
the binary package repository we have now, and depending on how you use
git, it has never worked. Close#1167, close#1156, close#1023 and
close#985. This commit also mentions --force when a package is up to date,
but the user requested to build it.
* pmbootstrap: __config_.py - update the deviceinfo_attributes table
Add missing attributes:
* "screen_width"
* "screen_height"
* "dev_touchscreen"
* "dev_touchscreen_calibration"
* "dev_keyboard"
* "bootimg_qcdt"
Reorder the list to correspond to pmb/aportgen/device.py
Add a comment in the aforementioned file to avoid forgetting to update
this list.
Signed-off-by: Mayeul Cantan <mayeul.cantan@gmail.com>
* pmbootstrap: add qcdt generation to the linux aportgen APKBUILDs
This checks the next box in #688
When the device has bootimg_qcdt set to true, the following is done to
the linux APKBUILD:
* Add dtbtool to makedepends
* Call dtbTool during build() to generate dt.img
* Add the generated dt.img in the package's boot/dt.img
Signed-off-by: Mayeul Cantan <mayeul.cantan@gmail.com>
* Don't ask for the mesa driver when the Qemu arch is not the
native arch and always use swrast in that case
* qemu-vexpress: use LTS kernel
* qemu-aarch64: use drm-backend for weston
If pmbootstrap says your apk is outdated, just run...
pmbootstrap zap -hc
...as it advises, to clear your http cache which contains the old
apk-tools-static.
* Rename deviceinfo variable flash_methods to flash_method
* Update pmb.config.deviceinfo_attributes / add sanity check
* Add test case that parses all deviceinfo files
* ccache: Fix for distcc cross-compiling / various improvements
* Make ccache work when cross-compiling with distcc (fix#716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
alphabetically
* Improve comment in arch-bin-masquerade APKBUILD
Add qt5-qtbase with OpenGL ES2 enabled and adjust the
upstream compatibility test case.
* Test case: don't get the qt5-qtbase version from any APKINDEX, but
only from Alpine's community APKINDEX
* Test case: If the pkgver is 9999, look at _pkgver
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
'pmbootstrap init', so the binary repo gets used even if a config file
already exists (this also removes a workaround, that deletes the work
folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
attempting to build packages (so we know which ones aleady exist in
the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
This adds a new deviceinfo 'flash_fastboot_max_size' used for
preventing fastboot from flashing a system partition that is too
large. Some devices do not support flashing over a certain size
(e.g. 500MB).
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:
```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```
It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).
Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
reported that to work best with plasma mobile on his PC). `--display` and
`--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
because it has all necessary information already) and the spice port can be
specified on the commandline now (previously it was hardcoded in one place and
then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
"pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
* pmbootstrap init: Generate new port device- and linux-package
* adds `pmbootstrap aportgen device-*` and
`pmbootstrap aportgen linux-*`
* ask for confirmation when selecting a non-existing device
* generate the packages directly from init
* refactor aportgen code
* fixed some easy things in the linux- APKBUILD (more to come in
follow-up PRs!)
Testing:
* Test all questions to the user from pmb.config.init and pmb.aportgen.device
(except for the timezone question, because we would need to monkeypatch the
os.path.exists() function, which messes up pytest, so we'd need to refactor
the timezone function to be more testsuite friendly first)
* Run the device wizard in a testcase a few times and check the output, that
pmbootstrap.aportgen.device and pmbootstrap.aportgen.linux create by parsing
the resulting APKBUILDs and deviceinfo and checking its contents.
* Build the generated device package once in the same testcase
Thanks a lot to @drebrez for all the help with this one:
<https://github.com/postmarketOS/pmbootstrap/pull/821>
See also the updated porting guide:
<https://wiki.postmarketos.org/wiki/Porting_to_a_new_device>
Oliver Smith
Martijn Braam
NotKit
Bhushan Shah
George Hopkins
Daniele Debernardi
Alexander
Mayeul Cantan
Alex Roth
Attila Szöllősi
Bart Ribbers
clayton craft