Commit Graph

263 Commits

Author SHA1 Message Date
Oliver Smith 7750c1dd40
Happy new year! (update copyright to 2018) 2018-01-04 04:53:35 +01:00
Oliver Smith 3198d51854
Update min. apk-tools-static to 2.9.0-r0
If pmbootstrap says your apk is outdated, just run...
	pmbootstrap zap -hc
...as it advises, to clear your http cache which contains the old
apk-tools-static.
2018-01-04 04:49:39 +01:00
Oliver Smith c403edefe4
pmbootstrap init: ask for work folder before asking for device (#1068)
This fixes #1066, where pmbootstrap crashes with a permission error
when you run it for the first time (no work folder exists) and you
run the boot.img analyzer because you want to start a new device
port.

It also prints a more helpful message if pmbootstrap crashes before
the log file was generated (suggests to use --details-to-stdout).
2018-01-02 04:38:28 +00:00
Alex Roth 36a8971426 Add Teclast X80 Pro (EFI/x86_64 tablet) (#1050) 2017-12-29 02:02:29 +00:00
drebrez c0c197f57b Feature debug-shell (#1028)
* Changed usb-shell behavior, it wait for some user action before continue booting
* Rename usb-shell to debug-shell and changed port to 23
* Add `20-debug-shell.sh` script to static code analysis
* Enable eth0 interface in initramfs (qemu)
* Add additional script to run a shell in order to be able to kill it from a telnet session
2017-12-23 19:22:28 +00:00
Oliver Smith f87a21a8a7
Fix broken UI and timezone selection
One comma was missing after the last `git rebase` I did before merging code.
2017-12-22 16:46:09 +00:00
Attila Szöllősi c6eb56c200 Rename deviceinfo variable flash_methods to flash_method (#1030)
* Rename deviceinfo variable flash_methods to flash_method
* Update pmb.config.deviceinfo_attributes / add sanity check
* Add test case that parses all deviceinfo files
2017-12-21 22:12:51 +00:00
Oliver Smith 567ac64e26
ccache: Fix for distcc cross-compiling / various improvements (#1026)
* ccache: Fix for distcc cross-compiling / various improvements

* Make ccache work when cross-compiling with distcc (fix #716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
  pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
  together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
  alphabetically

* Improve comment in arch-bin-masquerade APKBUILD
2017-12-21 16:42:29 +00:00
Oliver Smith 071ec4c214
build necessary logic: look at the package with the highest version (#1004)
Fixes #955. Previously we did not look through all APKINDEXes while
looking for the package with a given name and the highest version.
This caused pmbootstrap to build packages even if they are in the
binary repo and up to date.
2017-12-20 16:04:28 +00:00
lawl 66bd1fb8e6 Slight logging improvement when menuconfig fails (#1018) 2017-12-17 19:17:53 +00:00
drebrez 254150567e Add `devicepkg-dev` which generate the touchscreen udev rule based on the deviceinfo (#995)
* Use devicepkg-dev by default in new device wizard
* Add link to reference wiki page
2017-12-14 21:17:18 +00:00
Bart Ribbers fbb4704bdb Add qt5-qtbase with OpenGL ES2 enabled (#996)
Add qt5-qtbase with OpenGL ES2 enabled and adjust the
upstream compatibility test case.
* Test case: don't get the qt5-qtbase version from any APKINDEX, but
  only from Alpine's community APKINDEX
* Test case: If the pkgver is 9999, look at _pkgver
2017-12-11 20:24:30 +00:00
zhuowei 5e7de79b90 pmbootstrap: fix listing of mounts if the file was deleted (#964)
On my system, /proc/mounts sometimes contains a line like

```
udev /media/zhuowei/redhd/pmbootstrap/chroot_native/dev/loop0p2\040(deleted)
 devtmpfs rw,relatime,size=1959476k,nr_inodes=489869,mode=755 0 0
```

The "\040(deleted)" text confuses `pmbootstrap shutdown`. Remove the suffix
if we find it in a /proc/mounts entry. This fixes #545.
2017-12-11 19:56:46 +00:00
Oliver Smith 5526c5b8f5
Add 50 MB of free space to the root partition (#986)
Closes #928.
2017-12-08 22:48:41 +00:00
Oliver Smith 96eaf481d1
Cache which packages have been built (#972)
Fixes #968: Dependency calculation took a long time for `postmarketos-ui-plasma-mobile`
2017-12-05 23:17:39 +00:00
Oliver Smith 77a90b9e01
pmbootstrap: Check if work folder exists before doing anything (#966)
Fixes #949.
2017-12-02 16:40:55 +00:00
Oliver Smith ae97f9d738
Fix #948: a package depending on itself recursed forever (#963) 2017-12-02 11:51:43 +00:00
Oliver Smith 30d1a5cada
Fix 960: Disable root login (sudo still works) (#962) 2017-12-02 08:19:11 +00:00
shwsh 0e7203fbc6 Fix #947: Fix "pmbootstrap initfs ls" (#958) 2017-11-30 20:15:47 +00:00
Oliver Smith e9ca68dc05
Fix #941: Use the right arch for foreign-arch-only packages (#943)
This is a follow-up to #935.

* fix regression #941: pmbootstrap doesn't automatically pick the
  right architecture for building when none is specified
* remove obsolete --noarch-arch parameter
2017-11-28 19:12:16 +00:00
Oliver Smith 74344a36c2
Bump version to 0.4.0 2017-11-26 19:01:36 +01:00
Oliver Smith d3c77c39ac
Fix #824: Refactor pmb/build/package.py (make depends work like in abuild) (#935)
* Rename pmb/build/package.py to pmb/build/_package.py, so we can
  access the functions it contains in testcases, and still use
  pmb.build.package()
* Refactor the entire file. Instead of one big function that does
  too many things, we have many small ones now, that are tested
  in the testsuite and easier to modify
* Whenever building a package, pmbootstrap does not only build and
  install the "makedepends" (like we did before), now it does the
  same for the "depends". That's required to be compatible with
  abuild. The old behavior can still be used with 'pmbootstrap
  build --ignore-depends'.
* Because of that change, noarch packages can no longer be built in
  the native chroot if we need them for a foreign chroot. A device-
  package depending on a kernel would pull in the same kernel for
  the native architecture otherwise.
* Running 'pmbootstrap build device-...' without '--ignore-depends'
  and without a matching '--arch' displays a note that explains
  this change to the user and tells how to use it instead.
* Noarch packages no longer get symlinked. That was only
  implemented for packages built in the native chroot, and now that
  is not always the case anymore. Symlinking these packages creates
  packages with broken dependencies anyway (e.g.
  device-samsung-i9100 can't be installed in x86_64, because
  linux-samsung-i9100 is armhf only).
* Rename "carch" to "arch" wherever used. Naming it "carch"
  sometimes is confusing with no benefit.
* Add a testcase for the aarch64 qemu workaround (because it failed
  first and I needed to know for sure if it is working again).
* Improved some verbose logging, which helped with development of
  this feature.
* Removed the old "build" test case (which was disabled in
  testcases_fast.sh) as the new "build_package" test case covers its
  functionallity.
* Only build indexes if the packages folder exists for that arch (Travis
  couldn't run a test case otherwise)
2017-11-26 14:32:02 +00:00
drebrez 948f4ef224 Close #929: Add flash method "none" (#934) 2017-11-25 02:57:58 +00:00
drebrez 0cca286aba Move flash_methods variable to __init__.py (#919) 2017-11-20 16:40:16 +00:00
Oliver Smith a7b881e4cc
Close #871: Enable binary repository (#887)
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
  'pmbootstrap init', so the binary repo gets used even if a config file
  already exists (this also removes a workaround, that deletes the work
  folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
  attempting to build packages (so we know which ones aleady exist in
  the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
  them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
2017-11-19 15:04:08 +00:00
drebrez 94e2387af5 Add `pmbootstrap bootimg_analyze` / prompt during new device wizard (#905) 2017-11-19 14:35:23 +00:00
Pablo Castellano 99d7b58ee5 init: Add manufacturer to pkgdesc when creating new device (#913) 2017-11-18 11:02:19 +00:00
Attila Szöllősi d9e7a3d7de Run recovery installer in chroot (#901)
The recovery installer now has as few dependencies on the
Android recovery system as possible.
2017-11-16 22:20:57 +00:00
Oliver Smith 4c7793e766
Workaround for Qemu aarch64 abuild-tar bug (#907)
See also: <https://github.com/postmarketOS/pmbootstrap/issues/546>
2017-11-15 22:28:10 +00:00
Jochen Sprickerhof c50ecd7c5a Unset restrictive umask before creating files (#908)
If a user has a restrictive umask (for example 0077) set, it will be
inherited to sudo and thus files created with sudo where not readable by
the normal user. In that case, when setting up a new chroot, the
etc/apk/repositories file would have umask 600 and a os.path.exists()
(in update_repository_list) would return false. Setting the umask to 022
(octal) first, results in world readable files and directories, so the
user running `./pmbootstrap.py install` can read them.
2017-11-14 21:02:56 +00:00
Oliver Smith 90f89ef18c
Fix #722: don't use a HTTPS package mirror by default (#896)
* Fix #722: don't use a HTTPS package mirror by default
* Use load balancing mirror http://dl-cdn.alpinelinux.org/alpine/
2017-11-14 19:02:27 +00:00
clayton craft 0eb856a2f6 Close #897: Add max size option for fastboot (#899)
This adds a new deviceinfo 'flash_fastboot_max_size' used for
preventing fastboot from flashing a system partition that is too
large. Some devices do not support flashing over a certain size
(e.g. 500MB).
2017-11-11 22:00:22 +00:00
Oliver Smith c96da8cd37
Fix: package gets rebuilt, even if it exists, when using default arch (#881)
When not specifying an architecture for `pmbootstrap build`, and the `APKBUILD`
says that it can't be built for the native arch, it gets built for the right
foreign arch. `pmbootstrap` did not properly detect if packages were already
built in that case, and tried to build them again.

(I've noticed that while building packages for the binary repo #871.)

Use any `linux-` package, that is not available for `x86_64` on a `x86_64` PC
and build it twice. It should get properly detected now:

```shell
pmbootstrap build linux-amazon-thor
pmbootstrap build linux-amazon-thor # should not get built again
```
2017-11-07 23:23:55 +00:00
Oliver Smith 1bf892f5eb
Close #453: Support mesa-dri-virtio in Qemu (#861)
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:

```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```

It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).

Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
  directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
  reported that to work best with plasma mobile on his PC). `--display` and
  `--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
  since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
  because it has all necessary information already) and the spice port can be
  specified on the commandline now (previously it was hardcoded in one place and
  then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
  "pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
2017-11-05 13:48:49 +00:00
Oliver Smith 50faf5fe22
Close #865: Check line endings in APKBUILDs (#868) 2017-11-04 16:12:33 +00:00
clayton craft cacb05e7e6 Fix #818: Clean up updated repo list after zapping chroots (#867) 2017-11-04 02:43:06 +00:00
clayton craft 2987294ea6 Remove existing apk cache symlink before creating link (#866)
This removes any existing symlinks (which always seem to be broken when
this is encountered) to <workdir>/chroot_native/etc/apk/cache before
creating the symlink.
2017-11-04 02:19:23 +00:00
Oliver Smith 6962803fbb
Fix #831: Disallow work dir being inside the pmbootstrap dir (#852) 2017-11-04 02:04:55 +00:00
drebrez 6fb5b28e2f Add "flash_fastboot_vendor_id" deviceinfo variable for fastboot flash method (#857) 2017-11-03 23:20:55 +00:00
drebrez ba3e2f6d26 Add argument to zap (-o) older versions of packages from all chroots (#841)
- also moved the mismatch clean (-m) before zapping the chroots
- added confirmation questions to -o and -m
2017-11-02 17:33:14 +00:00
Oliver Smith 6627599cf0
pmbootstrap init: Wizard for new port device- and linux-packages (#821)
* pmbootstrap init: Generate new port device- and linux-package
* adds `pmbootstrap aportgen device-*` and
  `pmbootstrap aportgen linux-*`
* ask for confirmation when selecting a non-existing device
* generate the packages directly from init
* refactor aportgen code
* fixed some easy things in the linux- APKBUILD (more to come in
  follow-up PRs!)

Testing:
* Test all questions to the user from pmb.config.init and pmb.aportgen.device
  (except for the timezone question, because we would need to monkeypatch the
  os.path.exists() function, which messes up pytest, so we'd need to refactor
  the timezone function to be more testsuite friendly first)
* Run the device wizard in a testcase a few times and check the output, that
  pmbootstrap.aportgen.device and pmbootstrap.aportgen.linux create by parsing
  the resulting APKBUILDs and deviceinfo and checking its contents.
* Build the generated device package once in the same testcase

Thanks a lot to @drebrez for all the help with this one:
<https://github.com/postmarketOS/pmbootstrap/pull/821>

See also the updated porting guide:
<https://wiki.postmarketos.org/wiki/Porting_to_a_new_device>
2017-10-30 19:56:38 +00:00
drebrez 9c1ef821b3 Add architecture argument for the buildroot chroot (#832)
* Add architecture argument for the buildroot chroot, defaults to device architecture
* Output pmbootstrap log file after failure to debug Travis failure
* Travis: disable timestamp based rebuilds
2017-10-28 00:45:15 +00:00
Oliver Smith a3c1e22dd9
kconfig_check: disable CONFIG_PFT (#830)
This caused builds to fail with a modern GCC at least once, and while it originally was meant as
security feature by Qualcomm, "it is unsupported by Qualcomm, and opens up to a wide range
of potential attack surfaces that has not been audited by anyone."
2017-10-27 20:41:44 +00:00
Attila Szöllősi ee2bbf8385 Don't hardcode kernel flavor when launching qemu (#827)
This makes qemu work again with linux-postmarketos-stable (we renamed it recently).
2017-10-26 19:19:42 +00:00
drebrez 732a94fd45 Unregister all binfmt architectures during `pmbootstrap shutdown` (#822)
* Unregister all binfmt architectures during `pmbootstrap shutdown`
* Remove `-r, --rootfs` argument from `pmbootstrap build_init` command
2017-10-25 22:54:17 +00:00
drebrez 459f97ef1d Support multiple architectures in `pmbootstrap menuconfig` command (#814) 2017-10-24 16:18:42 +00:00
Oliver Smith ed94ab7449 Run fast testcases in Travis CI, remove obsolete testcase (#760)
* Removed obsolete apkindex_files cache testcase (the corresponding
function has been removed in #345 already).
* Fix test_challenge_apk: It failed on Travis, because we're accessing
/etc/abuild.conf, which only exists after initializing the build environment.
It's a random dummy file anyway, so I've replaced it with another file.
* Fix test_folder_size: accept a tolerance in the result
2017-10-23 19:44:08 +00:00
drebrez b1c86d4586 Fix #808: Add SYSVIPC kernel config check (#809)
* Add SYSVIPC kernel config check
* Add SYSVIPC kernel config option to all linux-device packages
2017-10-23 19:25:40 +00:00
Oliver Smith 8ba2964b64 Fix #800: Menuconfig didn't build dependencies (#801) 2017-10-21 14:34:43 +00:00
clayton craft 05b271d2e3 Fix #708: Detect existing pmOS install on sd card, show different warning(#766)
The method of 'install detection' used here is to look for a partition with pmOS_boot in the partition label. It's not a guarantee, but it works when FDE is enabled without having to unlock the partition to read files within it.
2017-10-21 00:33:58 +00:00