Commit Graph

301 Commits

Author SHA1 Message Date
Oliver Smith ef68ed99ad
bump version to 1.1.0 2018-12-26 22:07:30 +01:00
Luca Weiss 747ccfdd75
deviceinfo: add and require _codename (!1732)
deviceinfo_codneame holds the device's code name, so we can easily look
it up in the finished postmarketOS installation by reading
/etc/deviceinfo.

Related: postmarketOS/pmaports#157
2018-12-26 21:43:38 +01:00
Craig Comstock c4b4aa7252
Allow overriding pmaports path in pmbootstrap.cfg (!1728)
Make it possible to point the pmaports dir to an existing repository
clone, while not having to give the --aports/-p parameter with every
command.

  pmbootstrap --aports=/my/pmaports/path init

If the config file exists already, the dir can also be set with:

  pmbootstrap config aports /my/pmaports/path
2018-12-17 08:38:36 +01:00
Grant Miller dfaca1e06f Enable armv7 (!1730)
I need to do some more work on pmaports in order for this to actually
be useful.
2018-12-17 08:06:10 +01:00
Zhuowei Zhang 1efe288fe2
losetup: support building rootfs images with 4096 byte sector sizes (!1725)
Adds an optional deviceinfo variable, `deviceinfo_rootfs_image_sector_size`,
which specifies the logical sector size of the device's storage.

Some devices made after 2016 with UFS storage uses 4096 byte sectors
instead of the normal 512 bytes. The partition table in our rootfs
must match, otherwise the root filesystem won't mount on the device.

This change passes the sector size to `losetup` when creating the image
if the deviceinfo specifies it, so the image will have the correct
sector size.

If the deviceinfo doesn't specify the new option, the behaviour
is the same as previous versions of pmbootstrap.

Note that the sector size option only works on Linux 4.14 and above,
so pmbootstrap should be run on a >4.14 computer when installing to
devices with non-standard sector size.

To find if a device needs this parameter, run `fdisk -l` on the device.
If the output shows

`Note: sector size is 4096 (not 512)`

then add `deviceinfo_rootfs_image_sector_size="4096"` to the deviceinfo.

This is needed by the Pixel 3 XL (google-crosshatch) port.

See https://gitlab.com/postmarketOS/pmbootstrap/issues/1696.
2018-12-11 09:08:12 +01:00
Oliver Smith 9cb27e5d22
kconfig check: disallow USE_VFB (!1723)
Weston and Plasma Mobile might choose to draw on this virtual frame
buffer, just like it happened for the samsung-jflte. Disabling the
USE_VFB option fixed this, so let's make sure we don't have that option
enabled for any kernel.
2018-12-06 07:42:17 +01:00
Oliver Smith a5a64158e9
allow specifying multiple postmarketOS mirrors (!1718)
Multiple -mp arguments can be used to list multiple mirrors:
$ pmbootstrap -mp=first -mp=second chroot -- cat /etc/apk/repositories

This is needed for the new build infrastructure, so we can have a WIP
repository to which we push packages until all of them are up to date,
and then publish all of them at once. Software like KDE/Plasma Mobile,
which expect a lot of packages to be updated from one version to
another will not end up with a half-way through upgrade that way.
2018-12-06 07:31:43 +01:00
Oliver Smith d7b888907f
bump version to 1.0.4 2018-12-01 02:37:44 +01:00
Oliver Smith dab053e96c
bump version to 1.0.3 2018-11-07 08:08:53 +01:00
Oliver Smith b8dc816af0
bump version to 1.0.2 2018-10-12 09:09:23 +02:00
Oliver Smith 1e495baa3c bump version to 1.0.1
I'll tag a new version, because installing with pip is fixed now.
2018-10-09 04:35:03 +00:00
Oliver Smith 87384aace6 increase min. required pmaports version
After all the GCC6-as-extra-package related changes.
2018-10-08 05:06:59 +00:00
Oliver Smith 8eb3b5d5c6
bump version to 1.0.0
(imagine epic music playing in background)
2018-09-19 09:51:01 +02:00
Oliver Smith 1f8832c4cc install apk keys and binfmt data with setup.py 2018-09-17 10:06:57 +00:00
Oliver Smith cff9185401
Update minimum required apk version to 2.10.1-r0
Max Justicz found a clever way to exploit apk, which is fixed in the
latest version:

<https://justi.cz/security/2018/09/13/alpine-apk-rce.html>
2018-09-14 06:26:46 +02:00
Martijn Braam d88a09e6bd Added write_uboot_spl deviceinfo variable 2018-09-11 07:05:04 +00:00
Oliver Smith 3ce00de710 Move aports into own repository (pmaports) 2018-09-05 05:57:38 +00:00
Oliver Smith 7cb471cb42 aports/kde: test case for framework versions
This commit adds a test case, which makes sure that the KDE framework
and plasma framework version are always the same.

Additional changes:
* APKBUILD parser parses the URL now (that's the best way I found to
  categorize the KDE aports in frameworks and other)
* Changed single quotes to double quotes in KDE APKBUILDs, so the
  parser doesn't include the single quotes in the parsed result
* Added the test case to the gitlab CI config
2018-08-02 20:33:20 +00:00
NotKit e22ba8caea Change default user uid to 10000 2018-08-02 20:10:56 +00:00
Oliver Smith 0d7802d7ef
s/system partition/rootfs: fix remaining mentions
Follow-up to !1373, where `pmbootstrap flasher flash_system` was
replaced with `pmbootstrap flasher flash_rootfs`. We still had used
terms like "system partition" in a lot of places.

This commit replaces it everywhere, so it's clear that we're talking
about the pmOS rootfs (which may or may not be installed to Android's
system partition).
2018-07-15 23:41:31 +02:00
Oliver Smith f121dce59b pmb: bump version to 0.8.0 2018-07-14 01:16:04 +00:00
Oliver Smith 0854aa1dc4
Remove github.com references
* Travis and Coveralls badges
* aports: instead of <https://github.com/postmarketOS>, use
  <https://postmarketos.org>
* References to full URLs to issues and pull requests replaced with
  a hash and the number
* grsec check: simplify error message, remove link to github issue
  (nobody is using that anymore anyway)
2018-06-30 09:00:48 +02:00
Bhushan Shah 26808e4d6a
pmb: Add user to plugdev group and create it
This allows the user to talk to networkmanager to manage the system
connections. As it is mentioned in networkmanager pre-install.

```
Executing networkmanager-1.10.6-r0.pre-install
  *
  * To setup system connections, regular users must be member of 'plugdev' group.
```

The plugdev group gets created in the post-install hook of
networkmanager. Not all UIs depend on networkmanager, which means that
the group may not exist at installation time when we try to add the
user to the group in the python code. Therefore we create the group
first.
2018-06-25 22:53:29 +02:00
Oliver Smith cd181c541a
Allow DEVTMPFS_MOUNT in kernel config (#1548)
It is important, that we have DEVTMPFS enabled in the kernel config. But
it  does not hurt to have DEVTMPFS_MOUNT enabled as well, and some of
Alpine's kernel configs have that by default. This commit removes the
check that forbids the option in kernel configs, so we can fork the
raspberry pi kernel package from Alpine without changing unrelated
options.
2018-06-10 13:20:27 +00:00
George Hopkins 5edabd4d17 pmbootstrap: handle checkdepends (#1533) 2018-06-04 19:07:52 +00:00
Daniele Debernardi dfde37c132 Add Raspberry Pi device (#1501) 2018-05-29 19:33:40 +00:00
Oliver Smith 4e665a2190
Recreate device nodes in chroot after shutdown (#1447)
Device nodes in the chroots get created in a tmpfs, so they can be
created even if the filesystem where the chroot resides does not
support device nodes (#1317). In "pmbootstrap shutdown" we umount the
`dev` folder, which means all device nodes that were created inside
this folder are gone. This commit changes the code to actually recreate
the device nodes when using the chroot again.

Details:
* move `pmb.chroot.init.create_device_nodes` to
  `pmb.chroot.mount.crete_device_nodes`
* don't call it in `pmb.chroot.init()` anymore, but in
  `pmb.chroot.mount_dev_tmpfs()`
* Create the `null` device as well (`apk --initdb` also creates it on
  `init`, but we don't call it after `shutdown`)
2018-04-29 22:17:57 +00:00
Oliver Smith d6391791f7
Set all of user's groups in "pmbootstrap install" (#1432)
The postmarketos-base package used to make the user part of the "video"
and "audio" groups. However, this did not work reliably, and we were
adding the "wheel" group in "pmbootstrap install" anyway.

Now all groups get added in "pmbootstrap install", and the names of the
groups have been moved to `pmb.config.install_user_groups`.
2018-04-23 20:52:03 +00:00
Daniele Debernardi 400562a753 Ask to copy SSH public keys during init (#1394) 2018-04-08 14:12:01 +00:00
Oliver Smith b66b5dcc34
pmbootstrap init: kernel selection / remove linux-pmos-lts (#1363)
* As discussed in IRC/matrix, we're removing `linux-postmarketos-lts`
  for now. The kernel isn't used right now, and we save lots of
  maintenance effort with not updating it every week or so.
* new config option `"kernel"` with possible values:
  `"downstream", "mainline", "stable"` (downstream is always
  `linux-$devicename`)
* ask for the kernel during `pmbootstrap init` if the device package
  has kernel subpackages and install it in `_install.py`
* postmarketos-mkinitfs: display note instead of exit with error when
  the `deviceinfo_dtb` file is missing (because we expect it to be
  missing for downstream kernels)
* device-sony-amami:
  * add kernel subpackages for downstream, mainline
  * set `deviceinfo_dtb`
* device-qemu-amd64: add kernel subpackages for stable, lts, mainline
* test cases and test data for new functions
* test case that checks all aports for right usage of the feature:
  * don't mix specifying kernels in depends *and* subpackages
  * 1 kernel in depends is maximum
  * kernel subpackages must have a valid name
  * Test if devices packages reference at least one kernel
* Remove `_build_device_depends_note()` which informs the user that
  `--ignore-depends` can be used with device packages to avoid building
  the kernel. The idea was to make the transition easier after a change
  we did months ago, and now the kernel doesn't always get built before
  building the device package so it's not relevant anymore.
* pmb/chroot/other.py:
  * Add autoinstall=True to kernel_flavors_installed(). When the flag
    is set, the function makes sure that at least one kernel for the
    device is installed.
  * Remove kernel_flavor_autodetect() function, wherever it was used,
    it has been replaced with kernel_flavors_installed()[0].
* pmb.helpers.frontend.py: remove code to install at least one kernel,
  kernel_flavors_installed() takes care of that now.
2018-04-03 23:50:09 +00:00
Oliver Smith 4d8afc4aa5
Fix: /var/cache/distfiles writable by everyone (#1329)
As noted in commit 255c715624
`/var/cache/distfiles` is writable by everyone. It is supposed to be
writable only by `root` and by the `abuild` group (in which we put the
`pmos` user already for building packages).

Changes:
* `pmb.build.init()`: make `/var/cache/distfiles` writable only by
  members of the `abuild` group (and root)
* Increase workfolder version to 2
* Add migration code that fixes the permissions for existing work
  folders
* Refactor the migration code a bit to make this possible
2018-03-30 21:46:31 +00:00
Oliver Smith 6bb8444fef
"...flasher flash_rootfs" instead of "..._system" (#1373)
* Change `pmbootstrap flasher flash_system` command to
  `pmbootstrap flasher flash_rootfs`
* The old command still works, but all references have been changed to
  the new command
* Remove obsolete `pmbootstrap flasher export` (that was changed to
  `pmbootstrap export` a few months ago)
* Update `README.md` and ZSH auto completion
* Change the description of the generated rootfs image (not talking
  about a system image anymore, mention that it has subpartitions)
* Better description of `pmbootstrap flasher flash_rootfs --partition`
2018-03-30 01:11:20 +00:00
Daniele Debernardi 147082ec58 pmbootstrap init: Ask for hostname, default: device name (#1327)
* Save "" (empty string) in the user's config as hostname if the user
  let it default to the name of the device. That way, when the device
  gets changed, the user won't get the old device's name as hostname
  by accident.
* Add a test case
2018-03-17 18:41:41 +00:00
Oliver Smith 5ea00e0862
pmbootstrap newapkbuild: Properly parse arguments (#1320)
* pmbootstrap newapkbuild: Properly parse arguments

The `pmbootstrap newapkbuild` action wraps Alpine's `newapkbuild`. We
used to directly pass all arguments to `newapkbuild` without verifying
in Python whether they make sense or not. However, as `newpakbuild`
doesn't do strict sanity checks on the arguments, it is easy to end up
with unexpected behavior when using the command for the first time.

For example, `newapkbuild` allows either specifying a PKGNAME or SRCURL
as last parameter, and also allows setting a PKGNAME with the `-n`
parameter. It only makes sense to use that option when passing a
SRCURL.

With this commit, we duplicate the optins that should be passed through
to `newapkbuild` and use argparse to fully sanitize the options and
display a help page (`pmbootstrap newapkbuild -h`) that is consistent
with the other help pages.

Details:
* The `-f` (force) flag does not get passed through anymore. Instead we
  use it in Python to skip asking if an existing aport should be
  overwritten (the aports are outside of the chroot, so `newapkbuild`
  can't handle it in a way that makes sense for pmbootstrap).
* Output of `newapkbuild` gets redirected to the log file now, as we
  don't need it to display a help page.
* Don't verify the pkgver while creating the new APKBUILD. When passing
  a SRCURL, the pkgver gets extracted from the end of the URL and may
  not have a valid format yet (but we want the APKBUILD anyway).
* Stored options passed through in `pmb/config/__init__.py` and use it
  in both `pmb/parse/arguments.py` and `pmb/helpers/frontend.py`.
* Only allow `-n` with SRCURL
* The postmarketOS aports folder gets specified with `--folder` now.
  That way the generated help page is much closer to the original one
  from `newapkbuild`. The default is `main`.
* Made the package type flags (CMake, autotools, ...) exclusive so only
  one of them can be specified
2018-03-15 21:42:34 +00:00
Oliver Smith 3510a4868f
Fix building packages by provides name (#1303)
Use case: `mkbootimg` provides the `unpackbootimg` package. When
running `pmb.chroot.apk.install(args,"unpackbootimg")`, it was not
able to properly build the package.

Reproducing the error:
```
sudo rm ~/.local/var/pmbootstrap/packages/x86_64/mkbootimg*
pmbootstrap index
pmbootstrap --mirror-pmOS="" chroot --add=unpackbootimg
```

Or alternatively (simpler but less illustrative):
```
pmbootstrap build unpackbootimg --force
```
2018-03-08 21:30:55 +00:00
Daniele Debernardi 94306b51ee deviceinfo: remove external_disk_install and external_disk, use external_storage instead (#1301)
* deviceinfo: remove external_disk_install and external_disk, use external_storage instead
* Complain when external_disk* is used
2018-03-07 22:35:02 +00:00
Oliver Smith ad5a0d4294
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254)
Here are the changes necessary in pmbootstrap to make proprietary
software installed onto the device (firmware and userspace drivers)
optional (#756). To full close the issue, we need to apply this concept
to all device packages we already have in a follow-up PR.

Changes:
* New config file options nonfree_firmware and nonfree_userland, which
  we ask for during "pmbootstrap init" if there are non-free components
  for the selected device.
* We find that out by checking the APKBUILD's subpakages: The non-free
  packages are called $pkgname-nonfree-firmware and
  $pkgname-nonfree-userland.
* During "pmbootstrap init" we also show the pkgdesc of these
  subpackages. Parsing that is implemented in
  pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of
  the regular APKBUILD parsing, as this would need a change in the
  output format, and it is a lot *less* code if done like in this
  commit.
* pmb/parse/apkbuild.py was renamed to _apkbuild.py, and
  pmb/install/install.py to _install.py: needed to call the function in
  the usual way (e.g. pmb.parse.apkbuild()) but still being able to
  test the individual functions from these files in the test suite.
  We did the same thing for pmb/build/_package.py already.
* Install: New function get_nonfree_packages() returns the non-free
  packages that will be installed, based on the user's choice in
  "pmbootstrap init" and on the subpackages the device has.
* Added test cases and test data (APKBUILDs) for all new code,
  refactored test/test_questions.py to have multiple functions for
  testing the various questions / question types from
  "pmbootstrap init" instead of having it all in one big function.
  This allows to use another aport folder for testing the new
  non-free related questions in init.
2018-02-24 21:49:10 +00:00
Oliver Smith 2a9c1da46c
Bump version to 0.7.0 (previous one got tagged) 2018-02-20 19:53:50 +00:00
Alexander 3d35b45835 Check kernel config options based on the architecture (#1233)
The 'necessary_kconfig_options' dictionary in pmb/config/__init__.py
now has the different architectures (space separated) as the keys and
the dictionary, which matches kernel config options and their
expected value, as its value.
For that purpose, the 'check' function in pmb/parse/kconfig.py was
modified, so that it takes the architecture from the kconfig filename
and uses it to find the needed kernel config options.
Closes #1218.
2018-02-17 13:37:35 +00:00
Oliver Smith 57afbda3b9
Bump version to 0.6.0 2018-02-11 12:36:47 +01:00
Oliver Smith 77701ac484
mkinitfs: Fail on missing depends / better usability in general (#1133)
* Fail if mkbootimg/uboot-tools are not installed, but creating a
  boot.img file / u-boot legacy image was requested via deviceinfo
  (fixes #312)
* Fail if /boot/dt.img is missing, but we have a qcdt device
* Fail if the dtb file specified in deviceinfo does not exist
* Fail if mkbootimg etc. exit with error code
* Don't try to add the ext4 module into the initramfs. We always
  compile it into the kernel. Instead, kconfig_check makes sure it
  is enabled now. (fixes #1037)
* Add a note that modprobe warnings can be ignored mostly
2018-01-31 19:39:09 +00:00
Oliver Smith 3c59126bc1
Remove timestamp based rebuilds (#1174)
If you want to build a package without changing the version number,
please use `--force` from now on. For example:

    pmbootstrap build --force hello-world

Prior to this commit, changes were detected automatically (timestamp
based rebuilds). However, that feature does not work as expected with
the binary package repository we have now, and depending on how you use
git, it has never worked. Close #1167, close #1156, close #1023 and
close #985. This commit also mentions --force when a package is up to date,
but the user requested to build it.
2018-01-28 23:27:33 +00:00
Oliver Smith cd7280e1ee
Bump version (last version has been tagged) 2018-01-27 18:53:27 +01:00
Daniele Debernardi 8ec9eec080 Allow to build musl/binutils/gcc for x86 architecture (#1150) 2018-01-23 00:26:33 +00:00
Daniele Debernardi 0bc60138f3 Add musl/binutils/gcc for cross-compiling to x86_64 (#1130) 2018-01-18 21:48:04 +00:00
Mayeul Cantan 00aa65f2d0 pmbootstrap: add qcdt generation to the linux aportgen APKBUILDs (#1125)
* pmbootstrap: __config_.py - update the deviceinfo_attributes table

Add missing attributes:
 * "screen_width"
 * "screen_height"
 * "dev_touchscreen"
 * "dev_touchscreen_calibration"
 * "dev_keyboard"
 * "bootimg_qcdt"

Reorder the list to correspond to pmb/aportgen/device.py

Add a comment in the aforementioned file to avoid forgetting to update
this list.

Signed-off-by: Mayeul Cantan <mayeul.cantan@gmail.com>

* pmbootstrap: add qcdt generation to the linux aportgen APKBUILDs

This checks the next box in #688
When the device has bootimg_qcdt set to true, the following is done to
the linux APKBUILD:

 * Add dtbtool to makedepends
 * Call dtbTool during build() to generate dt.img
 * Add the generated dt.img in the package's boot/dt.img

Signed-off-by: Mayeul Cantan <mayeul.cantan@gmail.com>
2018-01-17 17:53:58 +00:00
Oliver Smith 12340fe5f6
Fix qemu-vexpress and qemu-aarch64 (#1029)
* Don't ask for the mesa driver when the Qemu arch is not the
  native arch and always use swrast in that case
* qemu-vexpress: use LTS kernel
* qemu-aarch64: use drm-backend for weston
2018-01-08 15:18:37 +00:00
Oliver Smith 7750c1dd40
Happy new year! (update copyright to 2018) 2018-01-04 04:53:35 +01:00
Oliver Smith 3198d51854
Update min. apk-tools-static to 2.9.0-r0
If pmbootstrap says your apk is outdated, just run...
	pmbootstrap zap -hc
...as it advises, to clear your http cache which contains the old
apk-tools-static.
2018-01-04 04:49:39 +01:00
Alex Roth 36a8971426 Add Teclast X80 Pro (EFI/x86_64 tablet) (#1050) 2017-12-29 02:02:29 +00:00
Oliver Smith f87a21a8a7
Fix broken UI and timezone selection
One comma was missing after the last `git rebase` I did before merging code.
2017-12-22 16:46:09 +00:00
Attila Szöllősi c6eb56c200 Rename deviceinfo variable flash_methods to flash_method (#1030)
* Rename deviceinfo variable flash_methods to flash_method
* Update pmb.config.deviceinfo_attributes / add sanity check
* Add test case that parses all deviceinfo files
2017-12-21 22:12:51 +00:00
Oliver Smith 567ac64e26
ccache: Fix for distcc cross-compiling / various improvements (#1026)
* ccache: Fix for distcc cross-compiling / various improvements

* Make ccache work when cross-compiling with distcc (fix #716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
  pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
  together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
  alphabetically

* Improve comment in arch-bin-masquerade APKBUILD
2017-12-21 16:42:29 +00:00
Bart Ribbers fbb4704bdb Add qt5-qtbase with OpenGL ES2 enabled (#996)
Add qt5-qtbase with OpenGL ES2 enabled and adjust the
upstream compatibility test case.
* Test case: don't get the qt5-qtbase version from any APKINDEX, but
  only from Alpine's community APKINDEX
* Test case: If the pkgver is 9999, look at _pkgver
2017-12-11 20:24:30 +00:00
Oliver Smith 74344a36c2
Bump version to 0.4.0 2017-11-26 19:01:36 +01:00
drebrez 948f4ef224 Close #929: Add flash method "none" (#934) 2017-11-25 02:57:58 +00:00
drebrez 0cca286aba Move flash_methods variable to __init__.py (#919) 2017-11-20 16:40:16 +00:00
Oliver Smith a7b881e4cc
Close #871: Enable binary repository (#887)
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
  'pmbootstrap init', so the binary repo gets used even if a config file
  already exists (this also removes a workaround, that deletes the work
  folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
  attempting to build packages (so we know which ones aleady exist in
  the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
  them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
2017-11-19 15:04:08 +00:00
Oliver Smith 90f89ef18c
Fix #722: don't use a HTTPS package mirror by default (#896)
* Fix #722: don't use a HTTPS package mirror by default
* Use load balancing mirror http://dl-cdn.alpinelinux.org/alpine/
2017-11-14 19:02:27 +00:00
clayton craft 0eb856a2f6 Close #897: Add max size option for fastboot (#899)
This adds a new deviceinfo 'flash_fastboot_max_size' used for
preventing fastboot from flashing a system partition that is too
large. Some devices do not support flashing over a certain size
(e.g. 500MB).
2017-11-11 22:00:22 +00:00
Oliver Smith 1bf892f5eb
Close #453: Support mesa-dri-virtio in Qemu (#861)
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:

```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```

It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).

Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
  directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
  reported that to work best with plasma mobile on his PC). `--display` and
  `--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
  since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
  because it has all necessary information already) and the spice port can be
  specified on the commandline now (previously it was hardcoded in one place and
  then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
  "pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
2017-11-05 13:48:49 +00:00
drebrez 6fb5b28e2f Add "flash_fastboot_vendor_id" deviceinfo variable for fastboot flash method (#857) 2017-11-03 23:20:55 +00:00
Oliver Smith 6627599cf0
pmbootstrap init: Wizard for new port device- and linux-packages (#821)
* pmbootstrap init: Generate new port device- and linux-package
* adds `pmbootstrap aportgen device-*` and
  `pmbootstrap aportgen linux-*`
* ask for confirmation when selecting a non-existing device
* generate the packages directly from init
* refactor aportgen code
* fixed some easy things in the linux- APKBUILD (more to come in
  follow-up PRs!)

Testing:
* Test all questions to the user from pmb.config.init and pmb.aportgen.device
  (except for the timezone question, because we would need to monkeypatch the
  os.path.exists() function, which messes up pytest, so we'd need to refactor
  the timezone function to be more testsuite friendly first)
* Run the device wizard in a testcase a few times and check the output, that
  pmbootstrap.aportgen.device and pmbootstrap.aportgen.linux create by parsing
  the resulting APKBUILDs and deviceinfo and checking its contents.
* Build the generated device package once in the same testcase

Thanks a lot to @drebrez for all the help with this one:
<https://github.com/postmarketOS/pmbootstrap/pull/821>

See also the updated porting guide:
<https://wiki.postmarketos.org/wiki/Porting_to_a_new_device>
2017-10-30 19:56:38 +00:00
Oliver Smith a3c1e22dd9
kconfig_check: disable CONFIG_PFT (#830)
This caused builds to fail with a modern GCC at least once, and while it originally was meant as
security feature by Qualcomm, "it is unsupported by Qualcomm, and opens up to a wide range
of potential attack surfaces that has not been audited by anyone."
2017-10-27 20:41:44 +00:00
drebrez b1c86d4586 Fix #808: Add SYSVIPC kernel config check (#809)
* Add SYSVIPC kernel config check
* Add SYSVIPC kernel config option to all linux-device packages
2017-10-23 19:25:40 +00:00
Oliver Smith 92e55ae1cd pmbootstrap init: Show pkgdesc for each postmarketos-ui package (#763)
This way we could give the user a rough idea what will be installed,
and also use this to display a short warning about long compile times
(e.g. until the plasma mobile stuff is upstreamed).
2017-10-16 20:01:21 +00:00
clayton craft 816cc9f625 Fix #588: Add chrony ntp client & timezone config (#674)
This adds chrony as the ntp client (starting on boot), and also adds a
prompt in the init step for configuring the timezone.
2017-10-13 19:18:24 +00:00
Pablo Castellano b4dd7a89d2 Close #709: Improve user creation (#725)
* Allow to specify a custom username in "pmbootstrap init"
* Build chroots have "pmos" instead of "user" as username now
* Installation user UID is 1000 now (as in all other Linux distributions)
* Adjust autologins
* postmarketos-base: enable wheel group for sudo, removed previous sudoers file
* Implement safe upgrade path:
We save the version of the work folder format now, in $WORK/version.
When this file does not exist, it defaults to 0.
In case it does not match the currently required version
(pmb.config.work_version), then ask the user if it should
automatically be upgraded.
2017-10-12 20:08:10 +00:00
Oliver Smith 75210b5cb5
Version bump to 0.3.0 2017-10-12 22:02:13 +02:00
Luca Weiss 0cc7869576 Fix #737: Disable ANDROID_PARANOID_NETWORK for all devices (#742)
* Disable ANDROID_PARANOID_NETWORK for all devices
* Add APKBUILD comments
* Bump pkgrels

This was done with a script, see:
<https://github.com/postmarketOS/pmbootstrap/pull/742#issuecomment-335608932>
2017-10-11 15:11:05 +00:00
clayton craft 558cd40fbf Adjust luksFormat options to reduce time to open luks device on boot (#430)
This is a workaround for #429, until the iteration count can be
specified directly in cryptsetup.

* Add default iter-time option, and option to override
* set SHA1 default hash for luksFormat, add option to override
* [RX51] load omap-sham in initramfs for HW accel. sha1
2017-09-19 19:46:18 +00:00
Attila Szöllősi ae6a58b6ed Close #554: kernel config checking (#589)
* Check kernel config
* Allow specifying multiple kernel packages, and also no packages
  which defaults to scanning all kernel configs (it is super fast
  anyway)
* Add the check to Travis CI
* Adjust existing kernel configs, so they pass the kconfig_check.
(We've had to put in a lot of defaults in the aarch64
linux-postmarketos configs, that's why the diff is a bit unclean.)
* Increase modified kernel pkgrels
2017-09-18 21:36:54 +00:00
drebrez a31cd0897b flasher flash_system: add possibility to select partition (#565)
Usage example:
pmbootstrap flasher flash_system --partition=userdata
2017-09-13 17:50:06 +00:00
drebrez e421313632 Add possibility to specify the system partition name for heimdall flashing method (#537)
Also added default values for the others heimdall partitions:
 kernel => KERNEL
 initfs => RECOVERY
 system => SYSTEM
2017-09-09 13:53:06 +00:00
Attila Szöllősi fbe968f1ab Recovery installer zip (#404)
Supports flashing with TWRP and other Android recovery OS through adb sideload,
as well as exporting a generated recovery zip file.
See also:
https://github.com/postmarketOS/pmbootstrap/wiki/deviceinfo_flash_methods#recovery-mode-adb
2017-08-24 21:07:36 +00:00
Martijn Braam c536e4ea58 Add setting for device keymap for devices with keymaps (#379)
* Added rx51_us keymap to nokia n900
* Added keymap option to init
* Made install command run setup-keymap when neccesary
* Validate keymap on install
2017-08-19 21:40:20 +00:00
drebrez a1370bafe1 Always use generated boot.img for `fastboot boot` command (#370) 2017-08-14 23:03:27 +00:00
Pablo Castellano b41e75e662 Minor fixes (#316)
* Fixed typo and specify list order

* Simplify code for jobs default value
2017-08-06 12:50:38 +00:00
drebrez 147863d98a Automatically compute the minimum size for the partition and resize it during the boot process (#127)
* Automatically compute the minimum size for the partitions
* Automatically resize the pmOS_root partition during the boot process
* Resize root partition only if there is unallocated space at the end of the device.
* Added more echos to make debugging easier while looking at the pmOS_init.log.
* Updated static_code_analysis.sh script to run shellcheck with `-x` option.
2017-08-03 16:01:00 +00:00
Pablo Castellano 0e4017105d Implement sparse system images (fix #299) (#303)
* Packaged libsparse

libsparse from the Android project provides multiple tools like img2simg
and simg2img.
These are used to split a large image for the system partition into
separate smaller chunks with sparse headers

This is required for several devices (at least bullhead, fp2 and titan)
because it fixes the "Invalid sparse file format at header magi" error

https://github.com/postmarketOS/pmbootstrap/issues/299

* Added new variable deviceinfo_flash_sparse (fixes #299)

Right after the system image is generated, pmbootstrap checks this
variable. In case it is true, run img2simg on it

* motorola-titan: enable deviceinfo_flash_sparse

* libsparse: use source from github: anestisb/android-simg2img

It is not that easy to use the upstream archive because everytime
you download it, the files have the current date as creation date
and that makes the file have a different checksum every download
https://github.com/postmarketOS/pmbootstrap/pull/303#issuecomment-319017197
2017-08-02 16:21:50 +00:00
Pablo Castellano 8389448bfd init: Allow to specify extra packages that will be always installed to rootfs (#304) 2017-08-01 15:31:33 +00:00
Oliver Smith af007250d4
Bump pmbootstrap version to 0.2.0 2017-07-29 01:09:07 +02:00
Oliver Smith e94ed7b03d Close #212: Split heimdall in heimdall-isorec and heimdall-bootimg (#254) 2017-07-25 18:56:10 +00:00
clayton craft 924546135c De-couple weston from postmarketos-base (#233)
Thanks, craftyguy!

- UI is selectable with `pmbootstrap init`. Currently only 'weston' and 'none' are options. It'll automatically pick up any new `postmarketos-ui-*` packages added at later dates.
- splits off weston packages install from postmarketos-base and puts
them into postmarketos-ui-weston. Also note that NO weston packages are
installed by "pmbootstrap install" by default unless the user selects a ui in the `init`
- configuration of weston is now in postmarketos-ui-weston.
- the demos have been spun off to `postmarketos-demos`, and `postmarketos-ui-weston` lists this package as a dependency.
2017-07-24 20:55:55 +00:00
Oliver Smith 9677d89f84 Fix #145: add deviceinfo_flash_offset_base again (#175)
* Fix #145: add deviceinfo_flash_offset_base again

* Add default OFFSET_BASE value in flasher if deviceinfo_flash_offset_base variable is not set
...and set correct default base value in create_bootimg function
2017-07-19 20:44:12 +00:00
Oliver Smith ca5b8e3367
Fix #200: Parse depends_dev in APKBUILDs 2017-07-19 21:24:03 +02:00
Pablo Castellano 6be7d656ee Two trivial changes (typo + info message) (#179)
Thanks, Pablo Castellano!

* Fixed typo

* Guide user about what are the next steps after flashing kernel and password

* Increased postmarketos-mkinitfs' pkgver and rewording

Changes requested:
https://github.com/postmarketOS/pmbootstrap/pull/179
2017-07-19 18:05:34 +00:00
Oliver Smith efab009a1f
deviceinfo: add heimdall partitions / make flasher variables optional 2017-07-10 20:27:06 +02:00
Oliver Smith 51bdc24315 Properly rebuild/install packages when something changed (Fix #120, #108, #131) (#129)
TLDR: Always rebuild/install packages when something changed when executing "pmbootstrap install/initfs/flash", more speed in dependency resolution.
---
pmbootstrap has already gotten some support for "timestamp based rebuilds", which modifies the logic for when packages should be rebuilt. It doesn't only consider packages outdated with old pkgver/pkgrel combinations, but also packages, where a source file has a newer timestamp, than the built package has.

I've found out, that this can lead to more rebuilds than expected. For example, when you check out the pmbootstrap git repository again into another folder, although you have already built packages. Then all files have the timestamp of the checkout, and the packages will appear to be outdated. While this is not largely a concern now, this will become a problem once we have a binary package repository, because then the packages from the binary repo will always seem to be outdated, if you just freshly checked out the repository.

To combat this, git gets asked if the files from the aport we're looking at are in sync with upstream, or not. Only when the files are not in sync with upstream and the timestamps of the sources are newer, a rebuild gets triggered from now on.

In case this logic should fail, I've added an option during "pmbootstrap init" where you can enable or disable the "timestamp based rebuilds" option.

In addition to that, this commit also works on fixing #120: packages do not get updated in "pmbootstrap install" after they have been rebuilt. For this to work, we specify all packages explicitly for abuild, instead of letting abuild do the resolving. This feature will also work with the "timestamp based rebuilds".

This commit also fixes the working_dir argument in pmb.helpers.run.user, which was simply ignored before.

Finally, the performance of the dependency resolution is faster again (when compared to the current version in master), because the parsed apkbuilds and finding the aport by pkgname gets cached during one pmbootstrap call (in args.cache, which also makes it easy to put fake data there in testcases).

The new dependency resolution code can output lots of verbose messages for debugging by specifying the `-v` parameter. The meaning of that changed, it used to output the file names where log messages come from, but no one seemed to use that anyway.
2017-07-10 15:23:43 +00:00
Oliver Smith dede11657d
Get rid of the _pmb_build_in_native_chroot hack 2017-07-05 18:40:00 +02:00
Martijn Braam 2518627579 Fixed typo in new boot.img flasher 2017-07-05 00:48:19 +02:00
Oliver Smith 615880e161
pmbootstrap flasher: use boot.img file with fastboot 2017-07-04 22:47:03 +02:00
Oliver Smith fb1e8ec73b
Update min apk version/add more apk version checks
* Minimum version: 2.7.2 (which fixes two CVEs)
* Check the minimum apk version before doing something with apk and
  before entering the chroot manually (previously, it has just checked
  the apk-tools-static version, which gets used to set up the chroot)
* Reword the message for an outdated APK version. Most likely it is
  just the outdated http cache, instead of a man-in-the-middle attack.

See also:
b849b481a0
2017-06-23 17:04:49 +02:00
Oliver Smith ed4275dd9b
Add support for the binary repository, inactive by default (#64)
* New commandline parameter --mirror-pmOS, where the binary repository
  URL for postmarketOS can be specified (empty by default as of now,
  this will be filled with the real URL once the repo works)
* Do not build packages, when they are in the binary repository and
  the version of the package in the binary repository is up-to-date.
* Add a testcase for pmb.build.is_necessary().
2017-06-20 20:13:05 +02:00
Oliver Smith 1adeee70b6
Fix #90: noarch: create symlinks for all supported architectures
* The supported architectures are inside the config now
* Symlinks get created for that list of supported architectures now.
* During initialization, the architecture from the selected device
  gets checked against the list of supported architectures. When
  it is not included, a meaningful exception gets raised.
* the aportgen and (cross-compiler) build tests make use of the
  new variable now (they had armhf and aarch64 hardcoded previously).
2017-06-15 02:26:32 +02:00
Oliver Smith 18339d0a14
Close #69: add 'pmbootstrap initfs' and improve initfs workflow
* allows to build/extract/list initramfs, add/del hook
* rebuild the initfs whenever running install or trying to flash/boot it
* flasher flash/boot: automatically set up a minimal rootfs with kernel and initfs,
  if it does not exist yet
2017-06-09 19:22:25 +02:00
Oliver Smith 5016c196ba
Fix two typos in the config 2017-06-08 18:02:00 +02:00
Oliver Smith 9d4bc61545
Fix #47: Support /dev/sd* as sdcard device
* The check for valid sdcard paths has been removed (because it
  doesn't make sense for /dev/sd*, this might as well be a real
  harddrive)
* New check: abort if any partition of the sdcard is mounted
* Automatically detect if the partitions are called "p1", "p2" etc.
  or just "1", "2" etc. (/dev/sda1 vs. /dev/mmcblk0p1)
* pmb.helpers.mount.ismount(): check the source (new) and the target
  path in /proc/mounts
2017-06-04 02:57:37 +02:00
Oliver Smith 3b5d5d8086
Debian Jessie/Python 3.4 support for the most part (#6)
* automatically find the chroot binary on Debian, even if it is not
  in the user's PATH
* don't use subprocess.run anymore (remove related testcase, that explicitly
  checked for subprocess.run usage, and used recursive globbing, another
  post 3.4 Python feature, for the checks. A similar case can be added in the
  future, but right now it's more important to get Debian 3.4 working and all
  PRs are reviewed anyway.)
* pytest fixtures: don't use the newer "yield" feature, as this is only
  supported in a newer version of pytest, than provided on Debian Jessie

From manually testing, most stuff works in Debian Jessie. However, the
testsuite does not run through - creating an empty .tar.gz with Python
fails for some reason (this is done in test_apk_static.py).
2017-05-29 20:38:11 +02:00
Oliver Smith ed9fc7f8b2
Put postmarketOS aports inside pmbootstrap repo
Later, the aports folder will probably get split up in its own repository.
But right now this is simply convenient.
2017-05-26 22:26:25 +02:00
Oliver Smith ae950fb9f7
Hello, there! 2017-05-26 22:08:45 +02:00