Commit Graph

285 Commits

Author SHA1 Message Date
Alexey Min 5761a29f07
pmb.aportgen.binutils: fix for new binutils file conflict (MR 2116)
Newly generated binutils package has file conflicsts with
non-cross binutils.

  ERROR: binutils-2.37-r3: trying to overwrite usr/lib/bfd-plugins/libdep.so
         owned by binutils-aarch64-2.37-r3.

Fix the generator to produce package that does not include these plugins.
2021-09-29 23:41:58 +02:00
Newbyte 2d23849aa3
pmb, test: remove redundant commas (MR 2115) 2021-09-26 17:58:40 +02:00
Bart Ribbers 4675def8e6
pmb/config: add required option for Waydroid support (MR 2109) 2021-09-13 21:10:05 +03:00
Oliver Smith bf02c72da7
test: add test_bootimg_mtk_mkimage_recovery (MR 2105)
Verify that using a Mediatek recovery boot.img file works with
bootimg_analyze. The mtk_mkimage-boot.img file is the twrp boot.img file
posted in pmbootstrap issue 2066.
2021-09-13 14:40:15 +03:00
Oliver Smith cd366859de
testdata: replace mtk_mtkimage-boot.img (MR 2105)
Replace the previous mtkimage-boot.img file, which were only the first 4
KiB of a mediatek boot.img file, with a full one. I made this full image
with pmbootstrap install for fairphone-fp1, it contains the kernel
packaged in pmaports.git master.

Reason for the replacement is, that the previous image did not contain
the label for the ramdisk. The current code ignored this, but upcoming
refactoring will test this more strictly and the dummy image will not
pass the test anymore.
2021-09-12 22:02:47 +02:00
BO41 77d39ecdf1
lint: enforce max line length (MR 2097) 2021-09-10 03:18:27 +02:00
Johannes Marbach ac4c967e18
pmbootstrap lint: avoid looping and copying files (MR 2100)
Before this commit, package folders were copied into the chroot one by
one in order to run apkbuild-lint on them. This logic is replaced by
mounting pmaports.git into the chroot and using a single apkbuild-lint
invocation to lint the supplied packages.

Both of these changes result in a performance improvement, especially
when linting multiple packages at once.

Before this change:

    $ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
        > /dev/null

    real    0m5,261s
    user    0m7,046s
    sys     0m1,842s

Using the pmaports.git mount but calling apkbuild-lint in a loop:

    $ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
        > /dev/null

    real    0m4,089s
    user    0m6,418s
    sys     0m1,219s

After this change:

    $ time ./pmbootstrap.py -q lint $(cd ../pmaports/cross; echo *) \
        > /dev/null

    real    0m3,518s
    user    0m5,968s
    sys     0m0,959s

Additionally, running apkbuild-lint from the pmaports.git mount point
has the benefit that every printed violation contains a nice source
identifier à la "./cross/grub-x86/APKBUILD". This makes it possible to
differentiate between different packages even though only a single
apkbuild-lint invocation is used.

Relates: postmarketOS/pmaports#564
2021-09-03 10:52:50 -07:00
afeuerstein 83c4f142e2
pmb.aportgen.gcc: sync makedepends of gcc aportgen to upstream (MR 2092)
Remove paxmark dep. Follow up to
1c6ba9f907,
which happened 5 months ago.

Fixup testdata too.
2021-08-10 01:16:54 +03:00
Newbyte 3c98cae230
kconfig check: require CONFIG_SWAP for zram (MR 2085)
While zram has uses outside of swap-on-zram, I would believe that the
main use of zram in postmarketOS is swap-on-zram. In addition to that,
I imagine many probably expect swap-on-zram to work if the zram kconfig
succeeds.
2021-07-17 18:11:50 -07:00
Anri Dellal d764b0de58
bootimg_analyze: Add support for boot header version 3 (MR 2073) 2021-07-07 04:43:24 +03:00
Clayton Craft 9d22989a88
kconfig: add missing options for nftables support to kconfig check (MR 2076)
I missed these when collecting the required kconfig options for
nftables, and it was a happy coincidence that the kernels I enabled the
nftables stuff on already had these enabled so that the initial kconfig
check passsed

test/kconfig: add missing nftables options
2021-07-07 04:24:07 +03:00
Oliver Smith d38bd6be2b
CI: drop support for cached venv + ci runner (MR 2080)
CI started failing with:
/builds/ollieparanoid/pmbootstrap/venv/bin/python3: No module named pytest

I've briefly tried to fix this with the existing scripts. However,
instead of investing more time into that, do the long overdue
refactoring of the scripts that involve dropping the venv logic and
support for a custon gitlab-ci-runner using some python docker image as
base. This configuration hasn't been used for a long time and is
probably broken anyway.

Refactor the logic to skip the qemu test case in gitlab CI by using
pytest markers. The new script is now similar to bpo's .ci/pytest.py.
2021-07-04 22:51:53 +02:00
Maxim Karasev 1bb15765ed
pmb.helpers.run_core.kill_commands: use minimal subset of ps parameters (MR 2074)
again, busybox ps supports only -o option (-e is ignored, because
busybox always shows all processes).
2021-06-28 02:07:55 +03:00
Maxim Karasev 31e7a0006d
pmb.helpers.other.folder_size: use kilobytes (MR 2074)
busybox du doesn't support bytes or custom block sizes.
2021-06-28 02:07:55 +03:00
Anri Dellal 5c1da79634
pmbootstrap init: Add mirror choice (MR 2035)
Allows user to choose one of the mirrors from
https://mirrors.postmarketos.org.

Example:
[1] Mirror 1 (Location 1)
[2] Mirror 2 (Location 2)
[3] Mirror 3 (Location 3)
Select a mirror [1]: 2

Co-Authored-By: Alexey Min <alexey.min@gmail.com>
2021-06-27 18:38:01 +02:00
Dylan Van Assche 834cc7f877
install: allow to use last block for embedding (MR 2069)
Devices such as ODROIDs have binaries use which every single block for
embedding. Do not raise an error when binaries are touching, but not
overlapping, each other when embedding these binaries during installation.

Add a test for this scenario, which fails when reverting the change.

Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
2021-06-26 20:11:42 +02:00
Dylan Van Assche e900ee169a
install: refactor embed_firmware and add test (MR 2069) 2021-06-26 20:11:36 +02:00
Clayton Craft 6afd35eb11
kconfig check: add support for checking zram options (MR 2075)
Device kernels that enable zram support can use `pmb:kconfigcheck-zram`
to protect against kconfig regressions that disable support for zram.
2021-06-23 21:17:41 -07:00
Alexey Min fbffee3645
test: fix crossdirect rust test (MR 2071)
Currently running test_crossdirect locally may fail if selected device
does not exist on stable branch. Fix it by switching to qemu-amd64 before
running this test.

While at it, switch to more recent stable branch, v20.05 is now
unmaintained.
2021-06-14 23:07:03 +02:00
Clayton Craft 599d74ec25
kconfig check: add support for nftables (MR 2042) 2021-06-14 21:50:45 +02:00
Alexey Min 21b9449813
test/test_helpers_package: use armv7 instead of armhf (MR 2054) 2021-06-13 15:05:27 +03:00
Clayton Craft 80cc09a84d
test/aportgen_device_wizard: use armv7 in test instead of armhf (MR 2054)
armhf isn't valid for new devices anymore
2021-06-13 15:05:24 +03:00
Alexey Min 4a8fac6c6e
pmb.aportgen: use correct arch for gcc aportgen (MR 2068)
Previously it always used native (x84_64) arch for apkindex
version check.

As pmb.aportgen.core.alpine_apkindex_path() already has arch
parameter, use it to look up correct APKINDEX.

Also fixup test.
2021-06-13 02:17:28 +03:00
Caio Fontes 1704851468
test/static_code_analysis.sh: update E501 check (MR 2058)
Change from checking some files for E501 to excluding some files from
checking.
2021-06-06 19:21:31 +02:00
Caio Fontes 27746db188
enforce E501 in test/ (MR 2058) 2021-06-06 19:21:31 +02:00
Oliver Smith 1481e69981
pmb.aportgen.binutils: adjust to subpackages in if (MR 2048)
Adjust the code so subpackages="" stays empty even with the latest
version of Alpine's APKBUILD.
2021-05-19 22:15:36 +02:00
Oliver Smith 2a0128d48a
test/testdata/aportgen: update binutils APKBUILD (MR 2048)
Put Alpine's current binutils APKBUILD there, with the subpackages line
that currently breaks the generated pmaport.
2021-05-19 22:15:31 +02:00
Caio Fontes 5144195091
fix long lines (E501) in pmb/build (MR 2050) 2021-04-26 23:56:42 -03:00
Anri Dellal 88ec1d1106
pmb.chroot: fix E501, switch to f-strings (MR 2040) 2021-03-30 15:13:26 +02:00
Shubham Naik 87c7859b40
Enforce E501: Limit the line length to 79 for files in pmb/install (MR 2031)
Made changes to limit the line length in following files for #1986,
 - pmb/install/_install.py
 - pmb/install/blockdevice.py
 - pmb/install/losetup.py
 - pmb/install/partition.py

Added the above files in E501 flake8 command list.
Substitute f-string for string concatenation.
2021-03-30 14:51:53 +02:00
Oliver Smith 278dfced61
Replace pmaports channels stable, stable-next (MR 2032)
Translate the pmaports channels "stable" to "v20.05" and "stable-next"
to "v21.03", so these have the same channel name as the pmaports.git
branch name.

The original plan was to switch the "stable" channel from the "v20.05"
branch to the "v21.03" branch when the release is done. However, now
that we are close to that, I'm realizing that this would not be useful.
It would lead to conflicts in the dir with locally built packages
(default: ~/.local/var/pmbootstrap/packages/$CHANNEL). And it would make
it awkward to go back to a previous branch (we may name it old-stable
for the time being, but what after that, old-old-stable?).
2021-03-18 18:58:13 +01:00
Oliver Smith 7320caead0
test_parse_depends: adjust to hello-world bump (MR 2028)
Adjust to pkgrel bump of the hello-world package in pmaports. Of course
it would be best if the tests did not depend on specific versions in
pmaports.git, but as long as they do, we need to keep them in sync.
This fixes the currently failing CI.
2021-02-11 23:36:46 +01:00
nybbled 5f485b8ad3
pmb.aportgen: put dependencies on separate lines (MR 2024)
Generate APKBUILDs with dependencies sorted and placed one per line.
The dependency lists changed are
 - depends in pmb/aportgen/device.py
 - makedepends in pmb/aportgen/linux.py

Sort dependency lists in test/test_aportgen_device_wizard.py
assertions.
2021-02-10 00:08:22 +01:00
Luca Weiss df58d35e27
kconfig check: require TMPFS_POSIX_ACL (MR 2025)
See pmaports!1939 and pmaports#904.
2021-02-09 23:11:31 +01:00
Minecrell 7dc2e197d3
pmb: Introduce support for "unmaintained" devices (MR 2018)
Unmaintained devices are device packages that:
  - Are known to be broken in some way without an active maintainer
    who can investigate how to fix it, or
  - Have not received any updates for a very long time, or
  - Are discouraged from using because they are just intended for testing.
    An example for this are ports using the downstream kernel for devices
    which have a mainline port that is working quite well.

Unmaintained devices are still built by bpo (otherwise it would not make
sense to keep them), but they do not show up in "pmbootstrap init".
However, it is possible to manually select them by entering the name.
pmbootstrap will warn in that case.

Unmaintained packages should have a # Unmaintained: <reason> comment
in the APKBUILD, this comment is displayed in "pmbootstrap init"
so that the user knows why the device should not be used unless they
know what they are doing.
2021-02-05 00:39:16 +01:00
Minecrell 03b3b250a5
pmb.parse._apkbuild: Add function to parse maintainers of APKBUILD (MR 2018)
This can be used for CI checks in pmaports to ensure that:
  - Devices in main have >= 2 maintainers
  - Devices in community have at least one maintainer
2021-02-05 00:39:12 +01:00
Shubham Naik 684cb3e1fb
Enforce E501: Limit the line length to 79 for files in pmb/parse - part 2 (MR 2020)
Made changes to limit the line length in following files,
 - pmb/parse/bootimg.py
 - pmb/parse/depends.py
 - pmb/parse/kconfig.py
 - test/test_parse_depends.py

Added the above files in E501 flake8 command list.
Substitute f-string for string concatenation.
2021-02-04 22:54:00 +01:00
Shubham Naik d1fadba5b4
Enforce E501: Limit the line length to 79 for files in pmb/parse - part 1 (MR 2019)
Made changes to limit the line length in following files,
 - pmb/parse/_apkbuild.py
 - pmb/parse/apkindex.py
 - pmb/parse/binfmt_info.py
 - pmb/parse/deviceinfo.py
 - test/test_parse_apkbuild.py

Added the above files in E501 flake8 command list.
Substitute f-string for string concatenation.
2021-02-04 22:32:35 +01:00
Oliver Smith 69cd7895e2
pmb.config.apk_tools_min_version: support branches (MR 2015)
Support branches, so pmbootstrap won't fail if v20.05 is selected:
  ERROR: You have an outdated version of the 'apk' package manager installed
  (your version: 2.10.5-r1, expected at least: 2.12.1-r0).

Move the logic for this check to pmb.helpers.apk.check_outdated and
adjust the test.

This fixes the CI failure in test_crossdirect_rust, which uses the
stable channel. (My bad for not creating this patch earlier, while at
the same time explaining in the creating pmbootstrap release instructions,
that this minimum apk version should be adjusted.)
2021-01-27 22:05:25 +01:00
Oliver Smith e9947f42de
pmb.config.apk_tools_static_min_version: rename (MR 2015)
Remove "_static" from the variable name, as this version isn't just
used to compare apk-tools-static's version (used to set up chroot), but
also for regular apk-tools before entering chroots.
2021-01-27 22:05:20 +01:00
Oliver Smith 51c01f384f
pmb/install/format.py: fix too long line (MR 2008) 2021-01-27 15:01:57 +01:00
Dylan Van Assche dd61d57b0e
pmbootstrap install: support _pmb_groups (MR 2010) 2021-01-27 08:38:53 +03:00
Dylan Van Assche 8455c82eee
install.ui: move get_recommends to new file (MR 2010) 2021-01-24 16:18:33 +01:00
Oliver Smith 1c791da482
treewide: bump copyright to 2021 2021-01-07 23:30:47 +01:00
Johannes Marbach 1eac61bcf7
Add option to run sudo -v in a loop to prevent repeated password entries (MR 1997)
Many of pmbootstrap's actions require root rights. When after requesting
sudo access pmbootstrap takes longer than the sudo timeout interval to finish
execution, the password will have to be entered again on the next sudo
action.

This change adds an opt-in feature to run sudo -v in a background loop
in order to prevent having to enter the password more than once for a single
pmbootstrap run. The loop runs as a daemon timer which automatically gets
canceled when pmbootstrap exits.

Closes: #1677
2020-12-14 19:14:14 +01:00
Johannes Marbach 8842a7d5c0
pmb.helpers.run_core: change kill_as_root to sudo (MR 1997)
Replace the "kill_as_root" argument with a much simpler "sudo" argument
and remove the now obsolete check for the output mode of "kill_as_root".

"kill_as_root" would only get set to True if both conditions are met:
a) command is running with sudo
b) command is running with an output mode ("log" or "stdout") where
   pmb.helpers.run_core would kill it if it does not output anything
   before a timeout is reached

The new "sudo" argument just indicates if the command is running with
sudo (a), regardless of the output mode (b).
2020-12-14 19:08:07 +01:00
Martijn Braam 1921fc5f27
Add sideload command
The sideload command runs the supplied names through the pmbootstrap
buildsystem to make sure they're up-to-date, then uses scp from the host
to copy the built apks to /tmp on the phone and installs them through
ssh.

If the --install-key option is set then it will also copy over the apk
key that's used for signing the packages built by pmbootstrap in case
the postmarketOS install on the device isn't build by the same machine
as you're sideloading from.
2020-12-08 17:42:30 +01:00
Johannes Marbach bbf0a70e5b
Add progress bar when running apk commands (MR 1996)
This adds a progress bar when running apk commands both inside and
outside of the chroot.

Closes: #1700
2020-12-07 12:41:28 +01:00
Johannes Marbach 45dbeca587
pmb.helpers.run_core: add output=pipe (MR 1996)
This adds a new output mode "pipe" that is identical to the existing
"background" mode except for that its stdout is redirected into a
pipe so that it can be retrieved.
2020-12-07 12:41:28 +01:00
Oliver Smith 5ff0bbe4b2
test/static: enforce E501 length check iteratively (MR 1993)
Enforce the E501 length check of max 79 characters for all files where
this test passes already. We can add more to the list as we adjust them
and eventuelly require the check for all files.
2020-11-23 11:31:57 +01:00