If you want to build a package without changing the version number,
please use `--force` from now on. For example:
pmbootstrap build --force hello-world
Prior to this commit, changes were detected automatically (timestamp
based rebuilds). However, that feature does not work as expected with
the binary package repository we have now, and depending on how you use
git, it has never worked. Close#1167, close#1156, close#1023 and
close#985. This commit also mentions --force when a package is up to date,
but the user requested to build it.
Preparation for #1122.
* `pmb.parse.apkindex.parse()`, removed strict parameter: This used to raise
an exception when two entries in the apkindex provided the same package.
Turns out this is *not* invalid after all, two packages can provide the same
soname for example (e.g. libhybris, mesa-egl). In an APKINDEX, sonames are
listed as they were packages ("so:libjpeg.so.8" etc.).
* Remove `pmbootstrap challenge` leftover code from reproducible builds effort,
which was a dead end. This code uses the broken strict feature.
Small improvements:
* Allow to specify multiple packages to `pmbootstrap parse_apkbuild`
* Specifying no package will parse all packages (like kconfig_check)
(also `parse_apkbuild`)
* JSON output is sorted of `parse_apkbuild`
* Make pkgver check optional, so we can disable it in the device wizard test case
* Parse_apk* -> apk*_parse
* Don't let the user mess with globs (disallow '*' in pkgname)
Fixes#893. Changes:
* New action: "pmbootstrap pkgrel_bump"
* pmbootstrap detects missing soname depends when trying to install
anyting, and suggests "pkgrel_bump --auto" to fix it
* Testcase test_soname_bump.py checks the pmOS binary package repo
for soname breakage, so we see it when CI runs for new PRs
* libsamsung-ipc: bump pkgrel because of soname bump
* Changed usb-shell behavior, it wait for some user action before continue booting
* Rename usb-shell to debug-shell and changed port to 23
* Add `20-debug-shell.sh` script to static code analysis
* Enable eth0 interface in initramfs (qemu)
* Add additional script to run a shell in order to be able to kill it from a telnet session
* ccache: Fix for distcc cross-compiling / various improvements
* Make ccache work when cross-compiling with distcc (fix#716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
alphabetically
* Improve comment in arch-bin-masquerade APKBUILD
This is a follow-up to #935.
* fix regression #941: pmbootstrap doesn't automatically pick the
right architecture for building when none is specified
* remove obsolete --noarch-arch parameter
* Rename pmb/build/package.py to pmb/build/_package.py, so we can
access the functions it contains in testcases, and still use
pmb.build.package()
* Refactor the entire file. Instead of one big function that does
too many things, we have many small ones now, that are tested
in the testsuite and easier to modify
* Whenever building a package, pmbootstrap does not only build and
install the "makedepends" (like we did before), now it does the
same for the "depends". That's required to be compatible with
abuild. The old behavior can still be used with 'pmbootstrap
build --ignore-depends'.
* Because of that change, noarch packages can no longer be built in
the native chroot if we need them for a foreign chroot. A device-
package depending on a kernel would pull in the same kernel for
the native architecture otherwise.
* Running 'pmbootstrap build device-...' without '--ignore-depends'
and without a matching '--arch' displays a note that explains
this change to the user and tells how to use it instead.
* Noarch packages no longer get symlinked. That was only
implemented for packages built in the native chroot, and now that
is not always the case anymore. Symlinking these packages creates
packages with broken dependencies anyway (e.g.
device-samsung-i9100 can't be installed in x86_64, because
linux-samsung-i9100 is armhf only).
* Rename "carch" to "arch" wherever used. Naming it "carch"
sometimes is confusing with no benefit.
* Add a testcase for the aarch64 qemu workaround (because it failed
first and I needed to know for sure if it is working again).
* Improved some verbose logging, which helped with development of
this feature.
* Removed the old "build" test case (which was disabled in
testcases_fast.sh) as the new "build_package" test case covers its
functionallity.
* Only build indexes if the packages folder exists for that arch (Travis
couldn't run a test case otherwise)
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
'pmbootstrap init', so the binary repo gets used even if a config file
already exists (this also removes a workaround, that deletes the work
folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
attempting to build packages (so we know which ones aleady exist in
the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:
```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```
It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).
Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
reported that to work best with plasma mobile on his PC). `--display` and
`--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
because it has all necessary information already) and the spice port can be
specified on the commandline now (previously it was hardcoded in one place and
then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
"pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
* Add architecture argument for the buildroot chroot, defaults to device architecture
* Output pmbootstrap log file after failure to debug Travis failure
* Travis: disable timestamp based rebuilds
* pmb.helpers.run: support running processes in background
* enable QXL driver support in the linux kernel configurations so
that we can also use SPICE to connect to the VM.
QXL is a paravirtual graphics driver with 2D support
The SPICE project aims to provide a complete open source solution for remote
access to virtual machines in a seamless way.
Both DRM_QXL and DRM_BOCHS are enabled as modules.
According to [1], on Linux guests, the qxl and bochs_drm kernel modules
must be loaded in order to gain a decent performance
* qemu: add new option --spice to connect to VM using a SPICE client
If specified, 'pmbootstrap qemu' will look for some SPICE client in the
user's PATH and run qemu using the QXL driver.
Currently supported spice clients are 'spicy' and 'remote-viewer' but
adding support for more clients can be easily done.
qemu with qxl support will run on port 8077/tcp, which doesn't belong to
any well-known service and represents 'PM' in decimal.
References:
[0] https://www.linux-kvm.org/page/SPICE
[1] https://wiki.archlinux.org/index.php/QEMU#qxl
[2] https://wiki.archlinux.org/index.php/QEMU#SPICE
[3] https://github.com/postmarketOS/pmbootstrap/issues/453 (partially fixed)
This is a workaround for #429, until the iteration count can be
specified directly in cryptsetup.
* Add default iter-time option, and option to override
* set SHA1 default hash for luksFormat, add option to override
* [RX51] load omap-sham in initramfs for HW accel. sha1
* Check kernel config
* Allow specifying multiple kernel packages, and also no packages
which defaults to scanning all kernel configs (it is super fast
anyway)
* Add the check to Travis CI
* Adjust existing kernel configs, so they pass the kconfig_check.
(We've had to put in a lot of defaults in the aarch64
linux-postmarketos configs, that's why the diff is a bit unclean.)
* Increase modified kernel pkgrels
Contrary to abuild, pmbootstrap only installs makedepends, and
keeps the installed packages around - both hacks save lots of time.
However, they may introduce missing makedepends in the APKBUILDs,
that the authors of the APKBUILDs do not notice because it works
for them.
This PR adss a strict mode, which will always clean the chroots
before building a package, and also remove all installed dependencies
after the package was built. You can use the following syntax to
only zap once, but build many packages at once:
`pmbootstrap build --strict hello-world 0xffff heimdall`
It also builds dependencies properly without leaving makedepends
behind.
* moved export code to pmb/export and split it up
* added deprecation notice to "pmbootstrap flasher export"
* made "pmbootstrap export" work
* adjusted the "pmbootstrap flasher export" hints in the code
This adds a new option to `zap`: `-m / --mismatch-bins`
When set, any binary apks in the work directory packages folder will be
removed if their version differs from the version in the relevant
APKBUILD in aports.
* The system image size is now calculated as: root size - home size.
* New function in `pmb/helpers/other.py`: `folder_size()`, with a
testcase.
* Instead of copying everything to the system image folder, and
deleting the home folder afterwards, do not copy the home folder
in the first place.
* Added `pmbootstrap -s` to skip generating the initramfs for faster
debugging.
* Set the default value in the "are you sure, that your partition has
at least..." to "y", so we can run `yes '' | pmbootstrap install`
to make it run through the whole installation process.
* Increase full size to 120%, boot partition gets 15 MB free space now
* Implement command to retrieve and set configuration values
* qemu: show advice to use "pmbootstrap config"
* Allow "pmbootstrap config" without positional arguments (prints the full config)
* Check valid variable names
Thanks to Pablo Castellano and Martijn Braam!
In postmarketOS we are now able to generate system images with the
correct configuration so that they can boot already using qemu
This commit brings the `pmbootstrap qemu` action.
This command is very handy because you don't have to set all the
qemu parameters, pmbootstrap does it for you.
* device-qemu-vexpress: Added kernel command line according to wiki
* qemu: Added workaround for image writing permissions
* qemu: Added support to launch postmarketOS in a QEMU virtual machine
- Support for emulating these architectures in QEMU: arm, aarch64, x86_84
- Generate QEMU command correctly depending no guest architecture (arm/x86)
- Run QEMU in the same architecture as the host by default
- Refactoring in pmb.parse.arch and pmb.qemu.run
- Raise exception if DTB file or system image are not present
- Display more useful information when something fails (e.g. image not found)
- Run qemu version depending on arch (host or argument), not device configured
* device-qemu-amd64: set deviceinfo_kernel_cmdline to "PMOS_NO_OUTPUT_REDIRECT"
* qemu: added --memory argument to specific guest RAM
* device-qemu-amd64: adjusted deviceinfo_kernel_cmdline (console=tty1)
* Added /etc/network/interfaces for qemu-amd64
* qemu: Added KVM support if /dev/kvm if present
* Specify separate machines for architecture
* qemu: Check if QEMU is installed instead of crashing
* Added graphics driver to qemu-aarch64
- Use arm (as used in qemu) instead of armhf (used in Alpine)
- qemu argument is -dtb
- Follow same style to build the command + arguments
* qemu: Added SSH port redirection: ./pmbootstrap.py qemu -p 2222
Changes:
* Removed the apkindex_files cache. That particular cache caused
bug #189 and didn't bring any real-world performance improvements
(tested 3x with that cache and 3x without, no significant speed
difference). I decided to remove it instead of keeping it/adding
even more code to resolve the bug.
* Fix the check for already built packages: always use the architecture,
that the package should be built for instead of the architecture of
the build environment (e.g. use armhf, even when building a noarch
package in the x86_64 chroot). This partially resolves#341.
* Make pmb.chroot.apk.install_is_necessary() more robust: If the binary
package is missing, although it should be there, print a warning and
build it with force.
When this parameter is set, `args.logfd` (the log file descriptor)
gets set to `sys.stdout`, so all logging gets directly sent to the
active pmbootstrap session, and not to a log file.
This will be used for Travis jobs, because they get killed after
10 minutes without any output (and builds may take longer).
* Quote architecture in logging message for easier reading
* Added shortcut arguments for --rootfs, --buildroot and --suffix
* Simply remove beforehand link to nowhere if exists (fix#278)
Fixed crash when symlink already existed but pointed to a non-existing location
* Various improvements for 'pmbootstrap export' (Fix#232)
* Do not try to generate the system image, but print a notice that
it was not generated yet
* Display export path
* New default path: /tmp/postmarketOS-export (instead of current
working dir)
* Create the export folder, if it does not exist yet
* Prettier output in the export message
* Adjust export message in install for consistency
* postmarketos-splash: change arch from "all" to "noarch"
* lg-mako: use .tar.gz instead of .zip (because that's the reference
APKBUILD)
* #220: Allow specifying multiple packages for checksum, build,
aportgen
* #239: Add chroot shortcuts (--rootfs and --buildroot)
* Show chroot and command before entering chroot
* Ignore `>`, `<`, `=` and `!` operators, when they are specified in
the dependencies. This was the desired behavior before, but it was
not implemented correctly (so it wouldn't ignore them everywhere).
Of course the real fix would be to honor these operators like apk
does. But this isn't feasible right now, and it should work for
most, if not all, our use-cases. I have documented this in the wiki
under build internals and if we happen to need correct operator
handling, we should do it then.
Minor other changes:
* `pmbootstrap parse_apkindex`: support optional package parameter to
only show the parsed content for one package.
* Support building most python APKBUILDs by replacing ${pkgname#py-}
properly
I've done some refactoring while debugging #209.
* Unused file `pmb/build/crosscompiler.py` removed (that was a
left over from `_pmb_build_in_native_chroot` hack
* Do verbose logging in distccd, when `pmbootstrap --verbose` is
being invoked
* Restart distccd, when the commandline has changed (e.g. when the
currently running version was not verbose, and the new one is
verbose.) Prior to this change, it only got restarted, when the
architecture changed (so it did not allow changing the job count
on the fly for example).
* Insert missing whitespace in arguments help.
This is important for the binary repository scripts, so it's feasible
to test the binary package build and challenge process locally without
setting up a new chroot whenever changing the repo URLs.
Also it behaves a bit more intuitively, because it really uses the
repo URL specified on the commandline, even when the chroot is already
set up.