* New "pmbootstrap build --src=/local/source/path hello-world" syntax
* The local source path gets mounted inside the chroot
* From there, a copy of the source code gets created with rsync (so
we can write into the source folder if necessary, for better
compatibility with all kinds of APKBUILDs)
* After the aport gets copied into the chroot before building (as
usually), we extend the APKBUILD with overrides to make it use
mountpoint's source instead of downloading the package's source
from the web as usually
* The package built with the local source gets _pYYYYMMDDHHMMSS
appended to the pkgver
* linux-postmarketos-mainline: use $builddir, fix patch checksum
The 'necessary_kconfig_options' dictionary in pmb/config/__init__.py
now has the different architectures (space separated) as the keys and
the dictionary, which matches kernel config options and their
expected value, as its value.
For that purpose, the 'check' function in pmb/parse/kconfig.py was
modified, so that it takes the architecture from the kconfig filename
and uses it to find the needed kernel config options.
Closes#1218.
Closes#441. Adjust bootimg_analyze code:
* Install mkbootimg (which now provides unpackbootimg) instead of
unpackbootimg. In theory, pmbootstrap should recognize this
automatically, however right now it does not yet handle this case.
* The file names of the extracted files have changed.
Follow up to #1162.
* `pmb.build.buildinfo()`: Used to record the build environment. It is
flawed because it scans the repo APKINDEX files instead of using the
actually installed packages list. When it was implemented we were not
able to do the latter. After this is removed, `pmb.parse.depends` can
be simplified (it needs to be rewritten for #1122).
* `pmb.helpers.repo.diff()` and `pmb.helpers.repo.files()`: These were
used exclusively by `pmb.build.buildinfo()`, to learn about which
files have been changed in the local repository folder after a
package was built. The idea was, that we could find subpackages that
way. But this information is present in the installed package list as
well, which is a much cleaner approach.
zap -m:
* APKINDEX parsing: parse the "origin" field as well, so we know
where a subpackage comes from
* pmbootstrap zap -m: properly delete all packages, that do not
have an aport or where the aport has another version. This also
works with subpackages now,
we use the origin field to resolve it.
* Only reindex when packages have been deleted in "zap -m"
zap in general:
* Show the amount of cleared up space after the deletion instead
of "Done"
* Print "Shutdown complete" to "pmbootstrap log" instead of stdout
(we need to call it twice during zap now to get the space
calculation right)
* Add `--dry` argument to `pmbootstrap zap` (this was very useful
for debugging) to list the packages/chroots that would get
deleted
* Roughly output the command that would get executed to delete
files, so it's obvious what's going on in --dry mode. (% rm ...)
If you want to build a package without changing the version number,
please use `--force` from now on. For example:
pmbootstrap build --force hello-world
Prior to this commit, changes were detected automatically (timestamp
based rebuilds). However, that feature does not work as expected with
the binary package repository we have now, and depending on how you use
git, it has never worked. Close#1167, close#1156, close#1023 and
close#985. This commit also mentions --force when a package is up to date,
but the user requested to build it.
Preparation for #1122.
* `pmb.parse.apkindex.parse()`, removed strict parameter: This used to raise
an exception when two entries in the apkindex provided the same package.
Turns out this is *not* invalid after all, two packages can provide the same
soname for example (e.g. libhybris, mesa-egl). In an APKINDEX, sonames are
listed as they were packages ("so:libjpeg.so.8" etc.).
* Remove `pmbootstrap challenge` leftover code from reproducible builds effort,
which was a dead end. This code uses the broken strict feature.
This PR makes the workflow faster and pmbootstrap will
produce less traffic. Details:
* Check if it's possible to create and read from a device
node directly when initializing a chroot (closes#472)
* Copy the Qemu binary into the forign-arch chroots
before initializing them, so the post-install script
directly work during the chroot setup and we don't need
to call apk fix afterwards
* Use pmb.helpers.repo.update(), which only updates the
APKINDEX files if they are older than 4 hours, instead
of using apk's repo update function which always
downloads the APKINDEX files
* Chroot initialization
* Getting the initial APKINDEX to download apk-tools-static
* Updating the APKINDEX at the start of pmbootstrap install
* Fixed a bug in from_chroot_suffix: the buildroot_x86_64 has
architecture x86_64, not x86.
Small improvements:
* Allow to specify multiple packages to `pmbootstrap parse_apkbuild`
* Specifying no package will parse all packages (like kconfig_check)
(also `parse_apkbuild`)
* JSON output is sorted of `parse_apkbuild`
* Make pkgver check optional, so we can disable it in the device wizard test case
* Parse_apk* -> apk*_parse
* Don't let the user mess with globs (disallow '*' in pkgname)
Fixes#893. Changes:
* New action: "pmbootstrap pkgrel_bump"
* pmbootstrap detects missing soname depends when trying to install
anyting, and suggests "pkgrel_bump --auto" to fix it
* Testcase test_soname_bump.py checks the pmOS binary package repo
for soname breakage, so we see it when CI runs for new PRs
* libsamsung-ipc: bump pkgrel because of soname bump
* Don't ask for the mesa driver when the Qemu arch is not the
native arch and always use swrast in that case
* qemu-vexpress: use LTS kernel
* qemu-aarch64: use drm-backend for weston
* Changed usb-shell behavior, it wait for some user action before continue booting
* Rename usb-shell to debug-shell and changed port to 23
* Add `20-debug-shell.sh` script to static code analysis
* Enable eth0 interface in initramfs (qemu)
* Add additional script to run a shell in order to be able to kill it from a telnet session
* Rename deviceinfo variable flash_methods to flash_method
* Update pmb.config.deviceinfo_attributes / add sanity check
* Add test case that parses all deviceinfo files
* ccache: Fix for distcc cross-compiling / various improvements
* Make ccache work when cross-compiling with distcc (fix#716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
alphabetically
* Improve comment in arch-bin-masquerade APKBUILD
Fixes#955. Previously we did not look through all APKINDEXes while
looking for the package with a given name and the highest version.
This caused pmbootstrap to build packages even if they are in the
binary repo and up to date.
This is a follow-up to #935.
* fix regression #941: pmbootstrap doesn't automatically pick the
right architecture for building when none is specified
* remove obsolete --noarch-arch parameter
* Rename pmb/build/package.py to pmb/build/_package.py, so we can
access the functions it contains in testcases, and still use
pmb.build.package()
* Refactor the entire file. Instead of one big function that does
too many things, we have many small ones now, that are tested
in the testsuite and easier to modify
* Whenever building a package, pmbootstrap does not only build and
install the "makedepends" (like we did before), now it does the
same for the "depends". That's required to be compatible with
abuild. The old behavior can still be used with 'pmbootstrap
build --ignore-depends'.
* Because of that change, noarch packages can no longer be built in
the native chroot if we need them for a foreign chroot. A device-
package depending on a kernel would pull in the same kernel for
the native architecture otherwise.
* Running 'pmbootstrap build device-...' without '--ignore-depends'
and without a matching '--arch' displays a note that explains
this change to the user and tells how to use it instead.
* Noarch packages no longer get symlinked. That was only
implemented for packages built in the native chroot, and now that
is not always the case anymore. Symlinking these packages creates
packages with broken dependencies anyway (e.g.
device-samsung-i9100 can't be installed in x86_64, because
linux-samsung-i9100 is armhf only).
* Rename "carch" to "arch" wherever used. Naming it "carch"
sometimes is confusing with no benefit.
* Add a testcase for the aarch64 qemu workaround (because it failed
first and I needed to know for sure if it is working again).
* Improved some verbose logging, which helped with development of
this feature.
* Removed the old "build" test case (which was disabled in
testcases_fast.sh) as the new "build_package" test case covers its
functionallity.
* Only build indexes if the packages folder exists for that arch (Travis
couldn't run a test case otherwise)
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
'pmbootstrap init', so the binary repo gets used even if a config file
already exists (this also removes a workaround, that deletes the work
folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
attempting to build packages (so we know which ones aleady exist in
the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:
```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```
It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).
Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
reported that to work best with plasma mobile on his PC). `--display` and
`--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
because it has all necessary information already) and the spice port can be
specified on the commandline now (previously it was hardcoded in one place and
then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
"pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
* Add architecture argument for the buildroot chroot, defaults to device architecture
* Output pmbootstrap log file after failure to debug Travis failure
* Travis: disable timestamp based rebuilds
* apkindex:
* Also parse the architecture field
* symlink_noarch_package:
* Renamed to symlink_noarch_packages
* Always work on all packages (so we don't need to guess which
subpackages have been generated after a certain build)
* Get invoked when running 'pmbootstrap index'
* Use 'apk index' to generate one index, where the architecture
does not get rewritten (abuild does that by default, due to
Alpine's repos not having a 'noarch' folder and diverging from
that doesn't make things easier for us). That goes super fast,
and then we know which packages are noarch packages and can
create the symlinks.
* Made output less verbose:
* Use -q for 'apk index' when calling it directly (when it gets
called by abuild we can't control that)
* Output that the APKINDEXes get reindexed only to the 'pmbootstrap
log'.
* pmb.helpers.run: support running processes in background
* enable QXL driver support in the linux kernel configurations so
that we can also use SPICE to connect to the VM.
QXL is a paravirtual graphics driver with 2D support
The SPICE project aims to provide a complete open source solution for remote
access to virtual machines in a seamless way.
Both DRM_QXL and DRM_BOCHS are enabled as modules.
According to [1], on Linux guests, the qxl and bochs_drm kernel modules
must be loaded in order to gain a decent performance
* qemu: add new option --spice to connect to VM using a SPICE client
If specified, 'pmbootstrap qemu' will look for some SPICE client in the
user's PATH and run qemu using the QXL driver.
Currently supported spice clients are 'spicy' and 'remote-viewer' but
adding support for more clients can be easily done.
qemu with qxl support will run on port 8077/tcp, which doesn't belong to
any well-known service and represents 'PM' in decimal.
References:
[0] https://www.linux-kvm.org/page/SPICE
[1] https://wiki.archlinux.org/index.php/QEMU#qxl
[2] https://wiki.archlinux.org/index.php/QEMU#SPICE
[3] https://github.com/postmarketOS/pmbootstrap/issues/453 (partially fixed)
This happens currently, when a makedepend is invalid:
ERROR: UnboundLocalError: local variable 'pkgname' referenced before assignment
With this patch, you get the meaning full error that should have been printed instead:
ERROR: Could not find package 'invalid-package' in the aports folder and could not find it in any APKINDEX
This is a workaround for #429, until the iteration count can be
specified directly in cryptsetup.
* Add default iter-time option, and option to override
* set SHA1 default hash for luksFormat, add option to override
* [RX51] load omap-sham in initramfs for HW accel. sha1
* Check kernel config
* Allow specifying multiple kernel packages, and also no packages
which defaults to scanning all kernel configs (it is super fast
anyway)
* Add the check to Travis CI
* Adjust existing kernel configs, so they pass the kconfig_check.
(We've had to put in a lot of defaults in the aarch64
linux-postmarketos configs, that's why the diff is a bit unclean.)
* Increase modified kernel pkgrels
Contrary to abuild, pmbootstrap only installs makedepends, and
keeps the installed packages around - both hacks save lots of time.
However, they may introduce missing makedepends in the APKBUILDs,
that the authors of the APKBUILDs do not notice because it works
for them.
This PR adss a strict mode, which will always clean the chroots
before building a package, and also remove all installed dependencies
after the package was built. You can use the following syntax to
only zap once, but build many packages at once:
`pmbootstrap build --strict hello-world 0xffff heimdall`
It also builds dependencies properly without leaving makedepends
behind.
* moved export code to pmb/export and split it up
* added deprecation notice to "pmbootstrap flasher export"
* made "pmbootstrap export" work
* adjusted the "pmbootstrap flasher export" hints in the code
This adds a new option to `zap`: `-m / --mismatch-bins`
When set, any binary apks in the work directory packages folder will be
removed if their version differs from the version in the relevant
APKBUILD in aports.