pmbootstrap/test/test_parse_apkbuild.py

154 lines
5.3 KiB
Python
Raw Normal View History

2021-01-07 22:30:30 +00:00
# Copyright 2021 Oliver Smith
# SPDX-License-Identifier: GPL-3.0-or-later
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254) Here are the changes necessary in pmbootstrap to make proprietary software installed onto the device (firmware and userspace drivers) optional (#756). To full close the issue, we need to apply this concept to all device packages we already have in a follow-up PR. Changes: * New config file options nonfree_firmware and nonfree_userland, which we ask for during "pmbootstrap init" if there are non-free components for the selected device. * We find that out by checking the APKBUILD's subpakages: The non-free packages are called $pkgname-nonfree-firmware and $pkgname-nonfree-userland. * During "pmbootstrap init" we also show the pkgdesc of these subpackages. Parsing that is implemented in pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of the regular APKBUILD parsing, as this would need a change in the output format, and it is a lot *less* code if done like in this commit. * pmb/parse/apkbuild.py was renamed to _apkbuild.py, and pmb/install/install.py to _install.py: needed to call the function in the usual way (e.g. pmb.parse.apkbuild()) but still being able to test the individual functions from these files in the test suite. We did the same thing for pmb/build/_package.py already. * Install: New function get_nonfree_packages() returns the non-free packages that will be installed, based on the user's choice in "pmbootstrap init" and on the subpackages the device has. * Added test cases and test data (APKBUILDs) for all new code, refactored test/test_questions.py to have multiple functions for testing the various questions / question types from "pmbootstrap init" instead of having it all in one big function. This allows to use another aport folder for testing the new non-free related questions in init.
2018-02-24 21:49:10 +00:00
import pytest
import sys
import pmb_test
import pmb_test.const
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254) Here are the changes necessary in pmbootstrap to make proprietary software installed onto the device (firmware and userspace drivers) optional (#756). To full close the issue, we need to apply this concept to all device packages we already have in a follow-up PR. Changes: * New config file options nonfree_firmware and nonfree_userland, which we ask for during "pmbootstrap init" if there are non-free components for the selected device. * We find that out by checking the APKBUILD's subpakages: The non-free packages are called $pkgname-nonfree-firmware and $pkgname-nonfree-userland. * During "pmbootstrap init" we also show the pkgdesc of these subpackages. Parsing that is implemented in pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of the regular APKBUILD parsing, as this would need a change in the output format, and it is a lot *less* code if done like in this commit. * pmb/parse/apkbuild.py was renamed to _apkbuild.py, and pmb/install/install.py to _install.py: needed to call the function in the usual way (e.g. pmb.parse.apkbuild()) but still being able to test the individual functions from these files in the test suite. We did the same thing for pmb/build/_package.py already. * Install: New function get_nonfree_packages() returns the non-free packages that will be installed, based on the user's choice in "pmbootstrap init" and on the subpackages the device has. * Added test cases and test data (APKBUILDs) for all new code, refactored test/test_questions.py to have multiple functions for testing the various questions / question types from "pmbootstrap init" instead of having it all in one big function. This allows to use another aport folder for testing the new non-free related questions in init.
2018-02-24 21:49:10 +00:00
import pmb.parse._apkbuild
pmbootstrap init: kernel selection / remove linux-pmos-lts (#1363) * As discussed in IRC/matrix, we're removing `linux-postmarketos-lts` for now. The kernel isn't used right now, and we save lots of maintenance effort with not updating it every week or so. * new config option `"kernel"` with possible values: `"downstream", "mainline", "stable"` (downstream is always `linux-$devicename`) * ask for the kernel during `pmbootstrap init` if the device package has kernel subpackages and install it in `_install.py` * postmarketos-mkinitfs: display note instead of exit with error when the `deviceinfo_dtb` file is missing (because we expect it to be missing for downstream kernels) * device-sony-amami: * add kernel subpackages for downstream, mainline * set `deviceinfo_dtb` * device-qemu-amd64: add kernel subpackages for stable, lts, mainline * test cases and test data for new functions * test case that checks all aports for right usage of the feature: * don't mix specifying kernels in depends *and* subpackages * 1 kernel in depends is maximum * kernel subpackages must have a valid name * Test if devices packages reference at least one kernel * Remove `_build_device_depends_note()` which informs the user that `--ignore-depends` can be used with device packages to avoid building the kernel. The idea was to make the transition easier after a change we did months ago, and now the kernel doesn't always get built before building the device package so it's not relevant anymore. * pmb/chroot/other.py: * Add autoinstall=True to kernel_flavors_installed(). When the flag is set, the function makes sure that at least one kernel for the device is installed. * Remove kernel_flavor_autodetect() function, wherever it was used, it has been replaced with kernel_flavors_installed()[0]. * pmb.helpers.frontend.py: remove code to install at least one kernel, kernel_flavors_installed() takes care of that now.
2018-04-03 23:50:09 +00:00
@pytest.fixture
def args(tmpdir, request):
import pmb.parse
sys.argv = ["pmbootstrap.py", "init"]
args = pmb.parse.arguments()
args.log = args.work + "/log_testsuite.txt"
pmb.helpers.logging.init(args)
request.addfinalizer(pmb.helpers.logging.logfd.close)
pmbootstrap init: kernel selection / remove linux-pmos-lts (#1363) * As discussed in IRC/matrix, we're removing `linux-postmarketos-lts` for now. The kernel isn't used right now, and we save lots of maintenance effort with not updating it every week or so. * new config option `"kernel"` with possible values: `"downstream", "mainline", "stable"` (downstream is always `linux-$devicename`) * ask for the kernel during `pmbootstrap init` if the device package has kernel subpackages and install it in `_install.py` * postmarketos-mkinitfs: display note instead of exit with error when the `deviceinfo_dtb` file is missing (because we expect it to be missing for downstream kernels) * device-sony-amami: * add kernel subpackages for downstream, mainline * set `deviceinfo_dtb` * device-qemu-amd64: add kernel subpackages for stable, lts, mainline * test cases and test data for new functions * test case that checks all aports for right usage of the feature: * don't mix specifying kernels in depends *and* subpackages * 1 kernel in depends is maximum * kernel subpackages must have a valid name * Test if devices packages reference at least one kernel * Remove `_build_device_depends_note()` which informs the user that `--ignore-depends` can be used with device packages to avoid building the kernel. The idea was to make the transition easier after a change we did months ago, and now the kernel doesn't always get built before building the device package so it's not relevant anymore. * pmb/chroot/other.py: * Add autoinstall=True to kernel_flavors_installed(). When the flag is set, the function makes sure that at least one kernel for the device is installed. * Remove kernel_flavor_autodetect() function, wherever it was used, it has been replaced with kernel_flavors_installed()[0]. * pmb.helpers.frontend.py: remove code to install at least one kernel, kernel_flavors_installed() takes care of that now.
2018-04-03 23:50:09 +00:00
return args
def test_subpackages(args):
testdata = pmb_test.const.testdata
path = testdata + "/apkbuild/APKBUILD.subpackages"
apkbuild = pmb.parse.apkbuild(args, path, check_pkgname=False)
subpkg = apkbuild["subpackages"]["simple"]
assert subpkg["pkgdesc"] == ""
# Inherited from parent package
assert subpkg["depends"] == ["postmarketos-base"]
subpkg = apkbuild["subpackages"]["custom"]
assert subpkg["pkgdesc"] == "This is one of the custom subpackages"
assert subpkg["depends"] == ["postmarketos-base", "glibc"]
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254) Here are the changes necessary in pmbootstrap to make proprietary software installed onto the device (firmware and userspace drivers) optional (#756). To full close the issue, we need to apply this concept to all device packages we already have in a follow-up PR. Changes: * New config file options nonfree_firmware and nonfree_userland, which we ask for during "pmbootstrap init" if there are non-free components for the selected device. * We find that out by checking the APKBUILD's subpakages: The non-free packages are called $pkgname-nonfree-firmware and $pkgname-nonfree-userland. * During "pmbootstrap init" we also show the pkgdesc of these subpackages. Parsing that is implemented in pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of the regular APKBUILD parsing, as this would need a change in the output format, and it is a lot *less* code if done like in this commit. * pmb/parse/apkbuild.py was renamed to _apkbuild.py, and pmb/install/install.py to _install.py: needed to call the function in the usual way (e.g. pmb.parse.apkbuild()) but still being able to test the individual functions from these files in the test suite. We did the same thing for pmb/build/_package.py already. * Install: New function get_nonfree_packages() returns the non-free packages that will be installed, based on the user's choice in "pmbootstrap init" and on the subpackages the device has. * Added test cases and test data (APKBUILDs) for all new code, refactored test/test_questions.py to have multiple functions for testing the various questions / question types from "pmbootstrap init" instead of having it all in one big function. This allows to use another aport folder for testing the new non-free related questions in init.
2018-02-24 21:49:10 +00:00
# Successful extraction
path = (testdata + "/init_questions_device/aports/device/testing/"
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254) Here are the changes necessary in pmbootstrap to make proprietary software installed onto the device (firmware and userspace drivers) optional (#756). To full close the issue, we need to apply this concept to all device packages we already have in a follow-up PR. Changes: * New config file options nonfree_firmware and nonfree_userland, which we ask for during "pmbootstrap init" if there are non-free components for the selected device. * We find that out by checking the APKBUILD's subpakages: The non-free packages are called $pkgname-nonfree-firmware and $pkgname-nonfree-userland. * During "pmbootstrap init" we also show the pkgdesc of these subpackages. Parsing that is implemented in pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of the regular APKBUILD parsing, as this would need a change in the output format, and it is a lot *less* code if done like in this commit. * pmb/parse/apkbuild.py was renamed to _apkbuild.py, and pmb/install/install.py to _install.py: needed to call the function in the usual way (e.g. pmb.parse.apkbuild()) but still being able to test the individual functions from these files in the test suite. We did the same thing for pmb/build/_package.py already. * Install: New function get_nonfree_packages() returns the non-free packages that will be installed, based on the user's choice in "pmbootstrap init" and on the subpackages the device has. * Added test cases and test data (APKBUILDs) for all new code, refactored test/test_questions.py to have multiple functions for testing the various questions / question types from "pmbootstrap init" instead of having it all in one big function. This allows to use another aport folder for testing the new non-free related questions in init.
2018-02-24 21:49:10 +00:00
"device-nonfree-firmware/APKBUILD")
apkbuild = pmb.parse.apkbuild(args, path)
subpkg = (apkbuild["subpackages"]
["device-nonfree-firmware-nonfree-firmware"])
assert subpkg["pkgdesc"] == "firmware description"
Make proprietary drivers optional (1/2): pmbootstrap changes (#1254) Here are the changes necessary in pmbootstrap to make proprietary software installed onto the device (firmware and userspace drivers) optional (#756). To full close the issue, we need to apply this concept to all device packages we already have in a follow-up PR. Changes: * New config file options nonfree_firmware and nonfree_userland, which we ask for during "pmbootstrap init" if there are non-free components for the selected device. * We find that out by checking the APKBUILD's subpakages: The non-free packages are called $pkgname-nonfree-firmware and $pkgname-nonfree-userland. * During "pmbootstrap init" we also show the pkgdesc of these subpackages. Parsing that is implemented in pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of the regular APKBUILD parsing, as this would need a change in the output format, and it is a lot *less* code if done like in this commit. * pmb/parse/apkbuild.py was renamed to _apkbuild.py, and pmb/install/install.py to _install.py: needed to call the function in the usual way (e.g. pmb.parse.apkbuild()) but still being able to test the individual functions from these files in the test suite. We did the same thing for pmb/build/_package.py already. * Install: New function get_nonfree_packages() returns the non-free packages that will be installed, based on the user's choice in "pmbootstrap init" and on the subpackages the device has. * Added test cases and test data (APKBUILDs) for all new code, refactored test/test_questions.py to have multiple functions for testing the various questions / question types from "pmbootstrap init" instead of having it all in one big function. This allows to use another aport folder for testing the new non-free related questions in init.
2018-02-24 21:49:10 +00:00
# Can't find the pkgdesc in the function
path = testdata + "/apkbuild/APKBUILD.missing-pkgdesc-in-subpackage"
apkbuild = pmb.parse.apkbuild(args, path, check_pkgname=False)
subpkg = (apkbuild["subpackages"]
["missing-pkgdesc-in-subpackage-subpackage"])
assert subpkg["pkgdesc"] == ""
# Can't find the function
assert apkbuild["subpackages"]["invalid-function"] is None
pmbootstrap init: kernel selection / remove linux-pmos-lts (#1363) * As discussed in IRC/matrix, we're removing `linux-postmarketos-lts` for now. The kernel isn't used right now, and we save lots of maintenance effort with not updating it every week or so. * new config option `"kernel"` with possible values: `"downstream", "mainline", "stable"` (downstream is always `linux-$devicename`) * ask for the kernel during `pmbootstrap init` if the device package has kernel subpackages and install it in `_install.py` * postmarketos-mkinitfs: display note instead of exit with error when the `deviceinfo_dtb` file is missing (because we expect it to be missing for downstream kernels) * device-sony-amami: * add kernel subpackages for downstream, mainline * set `deviceinfo_dtb` * device-qemu-amd64: add kernel subpackages for stable, lts, mainline * test cases and test data for new functions * test case that checks all aports for right usage of the feature: * don't mix specifying kernels in depends *and* subpackages * 1 kernel in depends is maximum * kernel subpackages must have a valid name * Test if devices packages reference at least one kernel * Remove `_build_device_depends_note()` which informs the user that `--ignore-depends` can be used with device packages to avoid building the kernel. The idea was to make the transition easier after a change we did months ago, and now the kernel doesn't always get built before building the device package so it's not relevant anymore. * pmb/chroot/other.py: * Add autoinstall=True to kernel_flavors_installed(). When the flag is set, the function makes sure that at least one kernel for the device is installed. * Remove kernel_flavor_autodetect() function, wherever it was used, it has been replaced with kernel_flavors_installed()[0]. * pmb.helpers.frontend.py: remove code to install at least one kernel, kernel_flavors_installed() takes care of that now.
2018-04-03 23:50:09 +00:00
def test_kernels(args):
# Kernel hardcoded in depends
args.aports = pmb_test.const.testdata + "/init_questions_device/aports"
pmbootstrap init: kernel selection / remove linux-pmos-lts (#1363) * As discussed in IRC/matrix, we're removing `linux-postmarketos-lts` for now. The kernel isn't used right now, and we save lots of maintenance effort with not updating it every week or so. * new config option `"kernel"` with possible values: `"downstream", "mainline", "stable"` (downstream is always `linux-$devicename`) * ask for the kernel during `pmbootstrap init` if the device package has kernel subpackages and install it in `_install.py` * postmarketos-mkinitfs: display note instead of exit with error when the `deviceinfo_dtb` file is missing (because we expect it to be missing for downstream kernels) * device-sony-amami: * add kernel subpackages for downstream, mainline * set `deviceinfo_dtb` * device-qemu-amd64: add kernel subpackages for stable, lts, mainline * test cases and test data for new functions * test case that checks all aports for right usage of the feature: * don't mix specifying kernels in depends *and* subpackages * 1 kernel in depends is maximum * kernel subpackages must have a valid name * Test if devices packages reference at least one kernel * Remove `_build_device_depends_note()` which informs the user that `--ignore-depends` can be used with device packages to avoid building the kernel. The idea was to make the transition easier after a change we did months ago, and now the kernel doesn't always get built before building the device package so it's not relevant anymore. * pmb/chroot/other.py: * Add autoinstall=True to kernel_flavors_installed(). When the flag is set, the function makes sure that at least one kernel for the device is installed. * Remove kernel_flavor_autodetect() function, wherever it was used, it has been replaced with kernel_flavors_installed()[0]. * pmb.helpers.frontend.py: remove code to install at least one kernel, kernel_flavors_installed() takes care of that now.
2018-04-03 23:50:09 +00:00
func = pmb.parse._apkbuild.kernels
device = "lg-mako"
assert func(args, device) is None
# Upstream and downstream kernel
device = "sony-amami"
ret = {"downstream": "Downstream description",
"mainline": "Mainline description"}
assert func(args, device) == ret
# Long kernel name (e.g. two different mainline kernels)
device = "wileyfox-crackling"
ret = {"mainline": "Mainline kernel (no modem)",
"mainline-modem": "Mainline kernel (with modem)",
"downstream": "Downstream kernel"}
assert func(args, device) == ret
def test_depends_in_depends(args):
path = pmb_test.const.testdata + "/apkbuild/APKBUILD.depends-in-depends"
apkbuild = pmb.parse.apkbuild(args, path, check_pkgname=False)
assert apkbuild["depends"] == ["first", "second", "third"]
def test_parse_attributes():
# Convenience function for calling the function with a block of text
def func(attribute, block):
lines = block.split("\n")
for i in range(0, len(lines)):
lines[i] += "\n"
i = 0
path = "(testcase in " + __file__ + ")"
print("=== parsing attribute '" + attribute + "' in test block:")
print(block)
print("===")
return pmb.parse._apkbuild.parse_attribute(attribute, lines, i, path)
assert func("depends", "pkgname='test'") == (False, None, 0)
assert func("pkgname", 'pkgname="test"') == (True, "test", 0)
assert func("pkgname", "pkgname='test'") == (True, "test", 0)
assert func("pkgname", "pkgname=test") == (True, "test", 0)
assert func("pkgname", 'pkgname="test\n"') == (True, "test", 1)
assert func("pkgname", 'pkgname="\ntest\n"') == (True, "test", 2)
assert func("pkgname", 'pkgname="test" # random comment\npkgrel=3') == \
(True, "test", 0)
assert func("depends", "depends='\nfirst\nsecond\nthird\n'#") == \
(True, "first second third", 4)
assert func("depends", 'depends="\nfirst\n\tsecond third"') == \
(True, "first second third", 2)
assert func("depends", 'depends=') == (True, "", 0)
with pytest.raises(RuntimeError) as e:
2020-01-06 16:53:23 +00:00
func("depends", 'depends="\nmissing\nend\nquote\nsign')
assert str(e.value).startswith("Can't find closing")
with pytest.raises(RuntimeError) as e:
func("depends", 'depends="')
assert str(e.value).startswith("Can't find closing")
def test_variable_replacements(args):
path = pmb_test.const.testdata + "/apkbuild/APKBUILD.variable-replacements"
apkbuild = pmb.parse.apkbuild(args, path, check_pkgname=False)
assert apkbuild["pkgdesc"] == "this should not affect variable replacement"
assert apkbuild["url"] == "replacements variable string-replacements"
assert list(apkbuild["subpackages"].keys()) == ["replacements", "test"]
assert apkbuild["subpackages"]["replacements"] is None
test_subpkg = apkbuild["subpackages"]["test"]
assert test_subpkg["pkgdesc"] == ("this should not affect variable "
"replacement")
def test_parse_maintainers(args):
path = pmb_test.const.testdata + "/apkbuild/APKBUILD.lint"
maintainers = [
"Oliver Smith <ollieparanoid@postmarketos.org>",
"Hello World <hello@world>"
]
assert pmb.parse._apkbuild.maintainers(path) == maintainers
def test_parse_unmaintained(args):
path = (f"{pmb_test.const.testdata}/apkbuild"
"/APKBUILD.missing-pkgdesc-in-subpackage")
assert pmb.parse._apkbuild.unmaintained(path) == "This is broken!"