This way we could give the user a rough idea what will be installed,
and also use this to display a short warning about long compile times
(e.g. until the plasma mobile stuff is upstreamed).
* Allow to specify a custom username in "pmbootstrap init"
* Build chroots have "pmos" instead of "user" as username now
* Installation user UID is 1000 now (as in all other Linux distributions)
* Adjust autologins
* postmarketos-base: enable wheel group for sudo, removed previous sudoers file
* Implement safe upgrade path:
We save the version of the work folder format now, in $WORK/version.
When this file does not exist, it defaults to 0.
In case it does not match the currently required version
(pmb.config.work_version), then ask the user if it should
automatically be upgraded.
* apkindex:
* Also parse the architecture field
* symlink_noarch_package:
* Renamed to symlink_noarch_packages
* Always work on all packages (so we don't need to guess which
subpackages have been generated after a certain build)
* Get invoked when running 'pmbootstrap index'
* Use 'apk index' to generate one index, where the architecture
does not get rewritten (abuild does that by default, due to
Alpine's repos not having a 'noarch' folder and diverging from
that doesn't make things easier for us). That goes super fast,
and then we know which packages are noarch packages and can
create the symlinks.
* Made output less verbose:
* Use -q for 'apk index' when calling it directly (when it gets
called by abuild we can't control that)
* Output that the APKINDEXes get reindexed only to the 'pmbootstrap
log'.
This checks for /sys/modules/loop before modprobing the loop module. My
understanding is that if this module is built into the kernel, that this
directory is still created. For a kernel without loop built in,
losetup.py will try to load the module using modprobe.
* pmb.helpers.run: support running processes in background
* enable QXL driver support in the linux kernel configurations so
that we can also use SPICE to connect to the VM.
QXL is a paravirtual graphics driver with 2D support
The SPICE project aims to provide a complete open source solution for remote
access to virtual machines in a seamless way.
Both DRM_QXL and DRM_BOCHS are enabled as modules.
According to [1], on Linux guests, the qxl and bochs_drm kernel modules
must be loaded in order to gain a decent performance
* qemu: add new option --spice to connect to VM using a SPICE client
If specified, 'pmbootstrap qemu' will look for some SPICE client in the
user's PATH and run qemu using the QXL driver.
Currently supported spice clients are 'spicy' and 'remote-viewer' but
adding support for more clients can be easily done.
qemu with qxl support will run on port 8077/tcp, which doesn't belong to
any well-known service and represents 'PM' in decimal.
References:
[0] https://www.linux-kvm.org/page/SPICE
[1] https://wiki.archlinux.org/index.php/QEMU#qxl
[2] https://wiki.archlinux.org/index.php/QEMU#SPICE
[3] https://github.com/postmarketOS/pmbootstrap/issues/453 (partially fixed)
We check if origin/HEAD is present. In case that reference is
missing, we show a meaningful error message now, with an explanation
on how to add it. Also moved find_out_of_sync_files_tracked() to
pmb.helpers.git
This only displays the ssh key note if no keys are found, since it's
confusing if a user has 1 key but not the other. In that case, the
message would be displayed despite copying one of the keys over.
This happens currently, when a makedepend is invalid:
ERROR: UnboundLocalError: local variable 'pkgname' referenced before assignment
With this patch, you get the meaning full error that should have been printed instead:
ERROR: Could not find package 'invalid-package' in the aports folder and could not find it in any APKINDEX
This is a workaround for #429, until the iteration count can be
specified directly in cryptsetup.
* Add default iter-time option, and option to override
* set SHA1 default hash for luksFormat, add option to override
* [RX51] load omap-sham in initramfs for HW accel. sha1
* Check kernel config
* Allow specifying multiple kernel packages, and also no packages
which defaults to scanning all kernel configs (it is super fast
anyway)
* Add the check to Travis CI
* Adjust existing kernel configs, so they pass the kconfig_check.
(We've had to put in a lot of defaults in the aarch64
linux-postmarketos configs, that's why the diff is a bit unclean.)
* Increase modified kernel pkgrels
Contrary to abuild, pmbootstrap only installs makedepends, and
keeps the installed packages around - both hacks save lots of time.
However, they may introduce missing makedepends in the APKBUILDs,
that the authors of the APKBUILDs do not notice because it works
for them.
This PR adss a strict mode, which will always clean the chroots
before building a package, and also remove all installed dependencies
after the package was built. You can use the following syntax to
only zap once, but build many packages at once:
`pmbootstrap build --strict hello-world 0xffff heimdall`
It also builds dependencies properly without leaving makedepends
behind.
* moved export code to pmb/export and split it up
* added deprecation notice to "pmbootstrap flasher export"
* made "pmbootstrap export" work
* adjusted the "pmbootstrap flasher export" hints in the code
This adds a new option to `zap`: `-m / --mismatch-bins`
When set, any binary apks in the work directory packages folder will be
removed if their version differs from the version in the relevant
APKBUILD in aports.
Parted often succeeds, but then returns a non-zero exit code, because
it can not inform the kernel of the changes.
In most cases this is not even necessary, so it really should not
fail there.
When the error was fatal, pmbootstrap will crash shortly afterwards
when it tried to mount or run mkfs on the partition anyway.
* Write custom os-release (closes#324)
* Return empty string instead of crashing when the directory is not a git repository
* Use https in homepage URL
The device chroot shouldn't exist yet anyways. Also add a 'device_exists'
variable that can be used to determine if the device entered earlier in
the init has a deviceinfo under aports/device/device-*
* Don't specify pkgnames from "provides" as dependencies
Always use the regular pkgname. That way, we avoid listing all
kinds of so: files as dependencies (because Alpine automatically
adds them as depends= to the package database). This fixes building
weston, and reproducing the build with `pmbootstrap challenge`.
Additional changes.
* Clear the parsed APKINDEX cache for the current pmbootstrap
session after building a package
* Avoid rebuilding a package, in case it was already built due to
circular dependencies
* The system image size is now calculated as: root size - home size.
* New function in `pmb/helpers/other.py`: `folder_size()`, with a
testcase.
* Instead of copying everything to the system image folder, and
deleting the home folder afterwards, do not copy the home folder
in the first place.
* Added `pmbootstrap -s` to skip generating the initramfs for faster
debugging.
* Set the default value in the "are you sure, that your partition has
at least..." to "y", so we can run `yes '' | pmbootstrap install`
to make it run through the whole installation process.
* Increase full size to 120%, boot partition gets 15 MB free space now
This extends zap() to add a 'no_confirm' option (False by default), and
zap() is now called by init with no_confirm=True to automatically zap
any existing chroots after the user runs init. This helps insure that
what is installed in the chroots is exactly what the user expects after
setting options in init.
Additionally, we create `cache_http` to verify write access to the work
folder instead of `chroot_native`. So we can ask for zapping only if
no chroot folder exists.
I've replaced all instances in the code of `os.path.abspath`
with `os.path.realpath`, as this does the same as `abspath`
plus resolving symlinks.
See also: https://stackoverflow.com/a/40311142
* Implement command to retrieve and set configuration values
* qemu: show advice to use "pmbootstrap config"
* Allow "pmbootstrap config" without positional arguments (prints the full config)
* Check valid variable names
This fixes https://github.com/postmarketOS/binary-package-repo/issues/1
GCC generates hardlinks between files `A` and `B` in its `make install` step. The problem is, that `tar` randomly packages `A` as full binary, and links `B` to `A`, or the other way around! I was able to reproduce this issue consistently when re-building `gcc-aarch64` on Travis CI (interestingly, this did not appear for `gcc-armhf`).
The fix is, to delete `B` and create a symlink `B` that points to `A` instead.
Previously, distutils.version.LooseVersion was used, because it was
sort of close enough to how Alpine parses versions.
This new version uses the exact same algorithm, as `apk` does, and
it passes *all* of `apk`'s testcases for version checking (previously
we simply skipped the ones, that did not pass).
* Remove pmb/helpers/version.py left-over (it is in parse now)
* Make asserts consistent, do not use unnecessary parenthesis
* aports: Use $install variable inside $source
So these files also get fingerprinted
* qemu: Use shutil.which instead of pmb.helpers.run.user
* Fix typo in comment
Thanks to Pablo Castellano and Martijn Braam!
In postmarketOS we are now able to generate system images with the
correct configuration so that they can boot already using qemu
This commit brings the `pmbootstrap qemu` action.
This command is very handy because you don't have to set all the
qemu parameters, pmbootstrap does it for you.
* device-qemu-vexpress: Added kernel command line according to wiki
* qemu: Added workaround for image writing permissions
* qemu: Added support to launch postmarketOS in a QEMU virtual machine
- Support for emulating these architectures in QEMU: arm, aarch64, x86_84
- Generate QEMU command correctly depending no guest architecture (arm/x86)
- Run QEMU in the same architecture as the host by default
- Refactoring in pmb.parse.arch and pmb.qemu.run
- Raise exception if DTB file or system image are not present
- Display more useful information when something fails (e.g. image not found)
- Run qemu version depending on arch (host or argument), not device configured
* device-qemu-amd64: set deviceinfo_kernel_cmdline to "PMOS_NO_OUTPUT_REDIRECT"
* qemu: added --memory argument to specific guest RAM
* device-qemu-amd64: adjusted deviceinfo_kernel_cmdline (console=tty1)
* Added /etc/network/interfaces for qemu-amd64
* qemu: Added KVM support if /dev/kvm if present
* Specify separate machines for architecture
* qemu: Check if QEMU is installed instead of crashing
* Added graphics driver to qemu-aarch64
- Use arm (as used in qemu) instead of armhf (used in Alpine)
- qemu argument is -dtb
- Follow same style to build the command + arguments
* qemu: Added SSH port redirection: ./pmbootstrap.py qemu -p 2222
Changes:
* Removed the apkindex_files cache. That particular cache caused
bug #189 and didn't bring any real-world performance improvements
(tested 3x with that cache and 3x without, no significant speed
difference). I decided to remove it instead of keeping it/adding
even more code to resolve the bug.
* Fix the check for already built packages: always use the architecture,
that the package should be built for instead of the architecture of
the build environment (e.g. use armhf, even when building a noarch
package in the x86_64 chroot). This partially resolves#341.
* Make pmb.chroot.apk.install_is_necessary() more robust: If the binary
package is missing, although it should be there, print a warning and
build it with force.
libstdc++.a from gcc-armhf was not reproducible on Travis (it was, when built locally!). These .a files are just archives of object files .o, and in this case it was caused by a random order of the .o files in the archive.
This PR patches the package() function of the APKBUILD when running pmbootstrap aportgen gcc-armhf (same for aarch64 of course), to extract all .a files, and repack them to be reproducible (by sorting the files before packing them).
As usually, we can still inherit everything from the upstream gcc aport from Alpine, and apply our changes on top of that.
Travis without the patch:
https://api.travis-ci.org/jobs/260402679/log.txt?deansi=true
> CHALLENGE FAILED for usr/armv6-alpine-linux-muslgnueabihf/lib/libstdc++.a:File 'usr/armv6-alpine-linux-muslgnueabihf/lib/libstdc++.a' is different!
Travis with the patch (I've instructed Travis to run off this branch to test it):
https://api.travis-ci.org/jobs/260806203/log.txt?deansi=true
> Done. Your build exited with 0.
* Automatically compute the minimum size for the partitions
* Automatically resize the pmOS_root partition during the boot process
* Resize root partition only if there is unallocated space at the end of the device.
* Added more echos to make debugging easier while looking at the pmOS_init.log.
* Updated static_code_analysis.sh script to run shellcheck with `-x` option.
When this parameter is set, `args.logfd` (the log file descriptor)
gets set to `sys.stdout`, so all logging gets directly sent to the
active pmbootstrap session, and not to a log file.
This will be used for Travis jobs, because they get killed after
10 minutes without any output (and builds may take longer).
* Packaged libsparse
libsparse from the Android project provides multiple tools like img2simg
and simg2img.
These are used to split a large image for the system partition into
separate smaller chunks with sparse headers
This is required for several devices (at least bullhead, fp2 and titan)
because it fixes the "Invalid sparse file format at header magi" error
https://github.com/postmarketOS/pmbootstrap/issues/299
* Added new variable deviceinfo_flash_sparse (fixes#299)
Right after the system image is generated, pmbootstrap checks this
variable. In case it is true, run img2simg on it
* motorola-titan: enable deviceinfo_flash_sparse
* libsparse: use source from github: anestisb/android-simg2img
It is not that easy to use the upstream archive because everytime
you download it, the files have the current date as creation date
and that makes the file have a different checksum every download
https://github.com/postmarketOS/pmbootstrap/pull/303#issuecomment-319017197
We can allow both ways:
* pmbootstrap menuconfig linux-motorola-titan
* pmbootstrap menuconfig motorola-titan
The former will show a tip about the second
This also prevents users from running menuconfig on aports like
'device-motorola-titan' or 'mkbootimg'
* Quote architecture in logging message for easier reading
* Added shortcut arguments for --rootfs, --buildroot and --suffix
* Simply remove beforehand link to nowhere if exists (fix#278)
Fixed crash when symlink already existed but pointed to a non-existing location
This was a regression from the improved input validation PR, which
created the chroot_native folder with the wrong permissions (to test
if we have write access to the work folder).
sjamaan found out, that this was the cause of the issue, and also
explained how to fix it, thanks a lot!
Huge thank you to @drebrez for his amazing work on this PR!
* Add generation of initramfs-extra with additional binaries
Extract both initramfs with `pmbootstrap initfs extract`
Add new splashscreens for missing partitions/files errors
Changes in init script:
- use busybox findfs applet to find boot partition
- mount boot partion
- extract initramfs-extra
- show error splashscreens accordingly and loop forever
- start usb unlock directly from unlock_root_partition (hook removed)
* Print out a text message for serial debugging in case of errors
Add initramfs-extra files to `pmbootstrap initfs ls` output
* Fix trailing whitespace in comment...
* ls: Indicate which initramfs we're looking at / add wiki link
I've rewritten the initramfs-development article to reflect the
changes made in this PR. It will be a good read for someone who
extracted the initramfs and wants to know why we have two files.
I've realized that truncate is provided by busybox anyway, so no
need to install coreutils for this.
This should fix creating system image files on ZFS, where fallocate
is not implemented.
If you really want to build busybox, I recommend turning the
"timestamp based rebuilds" feature off, otherwise it will build
for all architectures all the time whenever you change something,
even if you do not increase the version number (that's the idea
of that feature). This is, because busybox is a dependency for
basiscally everything, so it must get updated whenever you install
something, in case it was out of date.
It is easier to simply rename the package.
* Refactored `umount_all` to get the list of folders to be umounted from
`umount_all_list`, so we can test that function in a test case.
* Adjusted `umount_all_list` to return the deep folder levels first
* Added a testcase for that
* Remove redundant calls to `umount_all()` (which were from a time before
`pmbootstrap` was released, in which failing commands did not cause
`pmbootstrap` to raise an exception)
* Various improvements for 'pmbootstrap export' (Fix#232)
* Do not try to generate the system image, but print a notice that
it was not generated yet
* Display export path
* New default path: /tmp/postmarketOS-export (instead of current
working dir)
* Create the export folder, if it does not exist yet
* Prettier output in the export message
* Adjust export message in install for consistency
* postmarketos-splash: change arch from "all" to "noarch"
* lg-mako: use .tar.gz instead of .zip (because that's the reference
APKBUILD)
* #220: Allow specifying multiple packages for checksum, build,
aportgen
* #239: Add chroot shortcuts (--rootfs and --buildroot)
* Show chroot and command before entering chroot
* Validate all inputs from `pmbootstrap init`
* Add a new `confirm()` function, that validates input of yes/no
questions properly
* All questions loop until they have a valid answer now
The creates the linux-postmarketos package which for now supports the n900 (and probably the other maemo devices) and qemu with the vexpress-a9 machine simulation.
I've put the generated dtbs in /usr/share/dtb for now and set the dtb field in the deviceinfo for the n900 and my new qemu device.
* Introduced linux-postmarketos-omap
* Unified kernel progress
* Created kernel bootable with qemu
* Updated n900 deviceinfo for generic kernel
* Changed qemu device to vexpress
* Updated APK comments and added linux-postmarketos-dev package
* Append dtb in mkinitfs
* Fixed bootscript on n900 for the generic kernel
* Don't detect double flavors with -dtb appending
* Added graphics drivers for vexpress (qemu)
* Added more drivers for qemu
This adds virtio network support in qemu for the vexpress-a9 machine.
The keyboard and mouse don't work yet.
You can boot into weston if --no-fde is specified
Thanks, craftyguy!
- UI is selectable with `pmbootstrap init`. Currently only 'weston' and 'none' are options. It'll automatically pick up any new `postmarketos-ui-*` packages added at later dates.
- splits off weston packages install from postmarketos-base and puts
them into postmarketos-ui-weston. Also note that NO weston packages are
installed by "pmbootstrap install" by default unless the user selects a ui in the `init`
- configuration of weston is now in postmarketos-ui-weston.
- the demos have been spun off to `postmarketos-demos`, and `postmarketos-ui-weston` lists this package as a dependency.
This addresses #246 by not storing the aports directory in the config
file. The default location is still available (from config/__init__.py),
and can be overridden.
* Update runtime error message thrown for grsec/hardened kernels, saying that they are not supported right now, #107
* Remove unnecessary check for grsec enable/disable status - the build will not work in either case