Close #453: Support mesa-dri-virtio in Qemu (#861)

The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:

```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```

It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).

Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
  directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
  reported that to work best with plasma mobile on his PC). `--display` and
  `--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
  since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
  because it has all necessary information already) and the spice port can be
  specified on the commandline now (previously it was hardcoded in one place and
  then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
  "pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
This commit is contained in:
Oliver Smith 2017-11-05 13:48:49 +00:00 committed by GitHub
parent 6089b037dd
commit 1bf892f5eb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 135 additions and 114 deletions

View File

@ -1,13 +1,13 @@
pkgname=device-qemu-amd64
pkgver=1
pkgrel=10
pkgrel=11
pkgdesc="Simulated device in qemu with an x86 platform"
url="https://github.com/postmarketOS"
arch="noarch"
license="MIT"
depends="linux-postmarketos-stable"
makedepends=""
subpackages="$pkgname-mesa $pkgname-x11 $pkgname-weston"
subpackages="$pkgname-x11 $pkgname-weston"
source="deviceinfo interfaces weston.ini"
options="!check"
@ -18,15 +18,9 @@ package() {
"$pkgdir"/etc/network/interfaces
}
mesa() {
install_if="$pkgname mesa-glapi"
depends="mesa-dri-swrast"
mkdir "$subpkgdir"
}
x11() {
install_if="$pkgname xorg-server"
depends="xf86-video-qxl mesa-egl"
depends="xf86-video-qxl"
mkdir "$subpkgdir"
}

View File

@ -9,7 +9,7 @@ case $pkgver in
*.*.*) _kernver=${pkgver%.*};;
*.*) _kernver=$pkgver;;
esac
pkgrel=0
pkgrel=1
arch="all"
pkgdesc="Linux for pmOS supported chipsets (stable)"
@ -179,8 +179,8 @@ dev() {
}
sha512sums="a557c2f0303ae618910b7106ff63d9978afddf470f03cb72aa748213e099a0ecd5f3119aea6cbd7b61df30ca6ef3ec57044d524b7babbaabddf8b08b8bafa7d2 linux-4.13.tar.xz
2dbcf4eae8297b8a3bf2e886bcdba350918b2f7575a3ff8d7789f2729784c2002273acc559adff7f0577c18d8470ed1a11f67ea8a134971bc9cd63e312d9866f config-postmarketos-stable.armhf
d49d5d01adb7f314d5649cf8d7e917eeca9ec286eff2687c6865a7fac46e18227e278ca8965ad29bd99320a676e167256dc1b4212c5702f819e4cd2cf40c33c9 config-postmarketos-stable.aarch64
c1dc9a02bbe5bb047051ee4cda2454fab3c4c06a05abc86107948dd63da7b6c90dc35f48c27f04338673a49b246246f67c7b19ac942354a73058cc4088744120 config-postmarketos-stable.x86_64
e5d49d01241ff535e23e69b57dfaf76e5efdd1abb47e549ef136bd2ef4739f45d9ac132d519747bb628a5be2ce9b94adde4cfc2c0efe8dbfcadba7560b556970 config-postmarketos-stable.armhf
be4737b72bdf1f79f1713f465275bc8465b08fac351a4f3e92e05605fa280b0b41a1449defec07653e11837df82a17b1e8b93a279a7c8c03412817fd6ac1c7d7 config-postmarketos-stable.aarch64
3cc4620dee3fe91214e2bcb24b2df69b6fba187b6574b06365f59de1af6a2b45ac4d4978413d38bc82a74fa31422096687b9288c226ebb7433c2ff13866213b6 config-postmarketos-stable.x86_64
17c48bb7b4218297bd2be6faa5b6570ce1560a33385237a9962c0884d782c9a722a25a30077b6721d2943a9b98c29dcad6adfef718b0163c559c19a79519319b 0001-rx51-drm-regression-workaround.patch
ad38845a4c05fcaace68563ffa005cf537d3564448b28750b2c872788cbc0c2495dbc9fdf98817d21aef41863614d8b707acdfb05d8f07845d921c909b5f1d22 patch-4.13.11.xz"

View File

@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm64 4.13.2 Kernel Configuration
# Linux/arm64 4.13.11 Kernel Configuration
#
CONFIG_ARM64=y
CONFIG_64BIT=y

View File

@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/arm 4.13.6 Kernel Configuration
# Linux/arm 4.13.11 Kernel Configuration
#
CONFIG_ARM=y
CONFIG_ARM_HAS_SG_CHAIN=y
@ -611,6 +611,7 @@ CONFIG_HAVE_MEMBLOCK=y
CONFIG_NO_BOOTMEM=y
# CONFIG_HAVE_BOOTMEM_INFO_NODE is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_MEMORY_BALLOON=y
# CONFIG_COMPACTION is not set
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_BOUNCE=y
@ -3039,7 +3040,7 @@ CONFIG_DRM_OMAP_PANEL_SONY_ACX565AKM=y
# CONFIG_DRM_TILCDC is not set
CONFIG_DRM_QXL=m
CONFIG_DRM_BOCHS=m
# CONFIG_DRM_VIRTIO_GPU is not set
CONFIG_DRM_VIRTIO_GPU=m
# CONFIG_DRM_FSL_DCU is not set
# CONFIG_DRM_STM is not set
CONFIG_DRM_PANEL=y
@ -3950,8 +3951,9 @@ CONFIG_VIRTIO=m
#
# Virtio drivers
#
# CONFIG_VIRTIO_PCI is not set
# CONFIG_VIRTIO_BALLOON is not set
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_PCI_LEGACY=y
CONFIG_VIRTIO_BALLOON=m
CONFIG_VIRTIO_INPUT=m
CONFIG_VIRTIO_MMIO=m
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y

View File

@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 4.13.6 Kernel Configuration
# Linux/x86_64 4.13.11 Kernel Configuration
#
CONFIG_64BIT=y
CONFIG_X86_64=y
@ -488,6 +488,7 @@ CONFIG_ARCH_DISCARD_MEMBLOCK=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
CONFIG_MEMORY_BALLOON=y
# CONFIG_COMPACTION is not set
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_BOUNCE=y
@ -3189,7 +3190,7 @@ CONFIG_DRM_CIRRUS_QEMU=y
# CONFIG_DRM_RCAR_DW_HDMI is not set
CONFIG_DRM_QXL=m
CONFIG_DRM_BOCHS=m
# CONFIG_DRM_VIRTIO_GPU is not set
CONFIG_DRM_VIRTIO_GPU=m
CONFIG_DRM_PANEL=y
#
@ -4110,8 +4111,9 @@ CONFIG_VIRTIO=m
#
# Virtio drivers
#
# CONFIG_VIRTIO_PCI is not set
# CONFIG_VIRTIO_BALLOON is not set
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_PCI_LEGACY=y
CONFIG_VIRTIO_BALLOON=m
CONFIG_VIRTIO_INPUT=m
CONFIG_VIRTIO_MMIO=m
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y

View File

@ -59,6 +59,7 @@ defaults = {
"mirror_postmarketos": "",
"work": os.path.expanduser("~") + "/.local/var/pmbootstrap",
"port_distccd": "33632",
"qemu_mesa_driver": "dri-virtio",
"ui": "weston",
"user": "user",
"keymap": "",
@ -363,3 +364,8 @@ aportgen = {
"confirm_overwrite": True,
}
}
#
# QEMU
#
qemu_mesa_drivers = ["dri-swrast", "dri-virtio"]

View File

@ -146,12 +146,30 @@ def ask_for_device(args):
return (device, device_exists)
def ask_for_qemu_mesa_driver(args):
drivers = pmb.config.qemu_mesa_drivers
logging.info("Which mesa driver do you prefer for your Qemu device? Only"
" select something other than the default if you are having"
" graphical problems (such as glitches).")
while True:
ret = pmb.helpers.cli.ask(args, "Mesa driver", drivers,
args.qemu_mesa_driver)
if ret in drivers:
return ret
logging.fatal("ERROR: Please specify a driver from the list. To change"
" it, see qemu_mesa_drivers in pmb/config/__init__.py.")
def frontend(args):
cfg = pmb.config.load(args)
# Device
cfg["pmbootstrap"]["device"], device_exists = ask_for_device(args)
# Qemu mesa driver
if cfg["pmbootstrap"]["device"].startswith("qemu-"):
cfg["pmbootstrap"]["qemu_mesa_driver"] = ask_for_qemu_mesa_driver(args)
# Device keymap
if device_exists:
cfg["pmbootstrap"]["keymap"] = ask_for_keymaps(args, device=cfg["pmbootstrap"]["device"])

View File

@ -301,6 +301,8 @@ def install(args):
install_packages += args.extra_packages.split(",")
if args.add:
install_packages += args.add.split(",")
if args.device.startswith("qemu-"):
install_packages += ["mesa-" + args.qemu_mesa_driver]
for pkgname in install_packages:
pmb.build.package(args, pkgname, args.deviceinfo["arch"])

View File

@ -51,7 +51,8 @@ def arguments_flasher(subparser):
action.add_argument("--flavor", default=None)
# Flash system
flash_system = sub.add_parser("flash_system", help="flash the system partition")
flash_system = sub.add_parser(
"flash_system", help="flash the system partition")
flash_system.add_argument("--partition", default=None, help="partition to flash"
" the system image")
@ -99,6 +100,29 @@ def arguments_initfs(subparser):
return ret
def arguments_qemu(subparser):
ret = subparser.add_parser("qemu")
ret.add_argument("--arch", choices=["aarch64", "arm", "x86_64"],
help="emulate a different architecture")
ret.add_argument("--cmdline", help="override kernel commandline")
ret.add_argument(
"--image-size", help="set system image size (e.g. 2048M or 2G)")
ret.add_argument("-m", "--memory", type=int, default=1024,
help="guest RAM (default: 1024)")
ret.add_argument("-p", "--port", type=int, default=2222,
help="SSH port (default: 2222)")
display = ret.add_mutually_exclusive_group()
display.add_argument("--spice", dest="spice_port", const="8077",
action="store", nargs="?", default=None,
help="use SPICE for 2D acceleration (default port:"
" 8077)")
display.add_argument("--display", dest="qemu_display", const="sdl,gl=on",
help="Qemu's display parameter (default: sdl,gl=on)",
default="sdl,gl=on", nargs="?")
return ret
def arguments():
parser = argparse.ArgumentParser(prog="pmbootstrap")
arch_native = pmb.parse.arch.alpine_native()
@ -153,6 +177,7 @@ def arguments():
arguments_export(sub)
arguments_flasher(sub)
arguments_initfs(sub)
arguments_qemu(sub)
# Action: log
log = sub.add_parser("log", help="follow the pmbootstrap logfile")
@ -291,22 +316,6 @@ def arguments():
config.add_argument("name", nargs="?", help="variable name")
config.add_argument("value", nargs="?", help="set variable to value")
# Action: qemu
qemu = sub.add_parser("qemu")
qemu.add_argument("--arch", choices=["aarch64", "arm", "x86_64"],
help="emulate a different architecture")
qemu.add_argument("--cmdline", help="override kernel commandline")
qemu.add_argument("--image-size", help="set system image size (e.g. 2048M or 2G)")
qemu.add_argument("-m", "--memory", type=int, default=1024,
help="guest RAM (default: 1024)")
qemu.add_argument("-p", "--port", type=int, default=2222,
help="ssh port (default: 2222)")
qemu.add_argument("--spice", dest="use_spice",
default=False, action="store_true",
help="connect to the VM using SPICE (NOTE: you need to"
" have a SPICE client installed in your host"
" machine)")
# Use defaults from the user's config file
args = parser.parse_args()
cfg = pmb.config.load(args)

View File

@ -66,7 +66,7 @@ def which_qemu(args, arch):
def which_spice(args):
"""
Finds some SPICE executable or raises an exception otherwise
:returns: tuple (spice_was_found, path_to_spice_executable)
:returns: path_to_spice_executable or None
"""
executables = ["remote-viewer", "spicy"]
for executable in executables:
@ -75,37 +75,29 @@ def which_spice(args):
return None
def spice_command(args):
def command_spice(args):
"""
Generate the full SPICE command with arguments connect to
the virtual machine
:returns: tuple (dict, list), configuration parameters and spice command
Generate the full SPICE command with arguments connect to the virtual
machine
:returns: None or list with the spice command, e.g.:
["spicy", "-h", "127.0.0.1", "-p", "8077"]
"""
parameters = {
"spice_addr": "127.0.0.1",
"spice_port": "8077"
}
if not args.use_spice:
parameters["enable_spice"] = False
return parameters, []
if not args.spice_port:
return None
spice_binary = which_spice(args)
if not spice_binary:
parameters["enable_spice"] = False
return parameters, []
spice_addr = parameters["spice_addr"]
spice_port = parameters["spice_port"]
commands = {
"spicy": ["spicy", "-h", spice_addr, "-p", spice_port],
"remote-viewer": [
"remote-viewer",
"spice://" + spice_addr + "?port=" + spice_port
]
}
parameters["enable_spice"] = True
return parameters, commands[spice_binary]
logging.warning("WARNING: Could not find any SPICE client (spicy,"
" remote-viewer) in your PATH, starting without"
" SPICE support!")
return None
if spice_binary == "spicy":
return ["spicy", "-h", "127.0.0.1", "-p", args.spice_port]
return ["remote-viewer", "spice://127.0.0.1?port=" + args.spice_port]
def qemu_command(args, arch, device, img_path, config):
def command_qemu(args, arch, device, img_path, spice_enabled):
"""
Generate the full qemu command with arguments to run postmarketOS
"""
@ -114,11 +106,10 @@ def qemu_command(args, arch, device, img_path, config):
cmdline = deviceinfo["kernel_cmdline"]
if args.cmdline:
cmdline = args.cmdline
logging.info("cmdline: " + cmdline)
logging.debug("Kernel cmdline: " + cmdline)
ssh_port = str(args.port)
telnet_port = str(args.port + 1)
telnet_debug_port = str(args.port + 2)
port_ssh = str(args.port)
port_telnet = str(args.port + 2)
suffix = "rootfs_" + device
rootfs = args.work + "/chroot_" + suffix
@ -130,11 +121,11 @@ def qemu_command(args, arch, device, img_path, config):
command += ["-m", str(args.memory)]
command += ["-netdev",
"user,id=net0,"
"hostfwd=tcp::" + ssh_port + "-:22,"
"hostfwd=tcp::" + telnet_port + "-:23,"
"hostfwd=tcp::" + telnet_debug_port + "-:24"
"hostfwd=tcp::" + port_ssh + "-:22,"
"hostfwd=tcp::" + port_telnet + "-:24"
",net=172.16.42.0/24,dhcpstart=" + pmb.config.default_ip
]
command += ["-show-cursor"]
if deviceinfo["dtb"] != "":
dtb_image = rootfs + "/usr/share/dtb/" + deviceinfo["dtb"] + ".dtb"
@ -158,7 +149,7 @@ def qemu_command(args, arch, device, img_path, config):
command += ["-device", "virtio-gpu-pci"]
command += ["-device", "virtio-net-device,netdev=net0"]
# add storage
# Add storage
command += ["-device", "virtio-blk-device,drive=system"]
command += ["-drive", "if=none,id=system,file={},id=hd0".format(img_path)]
@ -174,14 +165,18 @@ def qemu_command(args, arch, device, img_path, config):
if enable_kvm and os.path.exists("/dev/kvm"):
command += ["-enable-kvm"]
else:
logging.info("Warning: qemu is not using KVM and will run slower!")
logging.info("WARNING: Qemu is not using KVM and will run slower!")
# QXL / SPICE (2D acceleration support)
if config["enable_spice"]:
# 2D acceleration support via QXL/SPICE or virtio
if spice_enabled:
command += ["-vga", "qxl"]
command += ["-spice",
"port={spice_port},addr={spice_addr}".format(**config) +
"port=" + args.spice_port + ",addr=127.0.0.1" +
",disable-ticketing"]
else:
if args.qemu_mesa_driver == "dri-virtio":
command += ["-vga", "virtio"]
command += ["-display", args.qemu_display]
return command
@ -194,18 +189,18 @@ def resize_image(args, img_size_new, img_path):
:param img_size_new: new image size in M or G
:param img_path: the path to the system image
"""
# current image size in bytes
# Current image size in bytes
img_size = os.path.getsize(img_path)
# make sure we have at least 1 integer followed by either M or G
# Make sure we have at least 1 integer followed by either M or G
pattern = re.compile("^[0-9]+[M|G]$")
if not pattern.match(img_size_new):
raise RuntimeError("You must specify the system image size in [M]iB or [G]iB, e.g. 2048M or 2G")
# remove M or G and convert to bytes
# Remove M or G and convert to bytes
img_size_new_bytes = int(img_size_new[:-1]) * 1024 * 1024
# convert further for G
# Convert further for G
if (img_size_new[-1] == "G"):
img_size_new_bytes = img_size_new_bytes * 1024
@ -213,8 +208,8 @@ def resize_image(args, img_size_new, img_path):
logging.info("Setting the system image size to " + img_size_new)
pmb.helpers.run.root(args, ["truncate", "-s", img_size_new, img_path])
else:
# convert to human-readable format
# Note: We convert to M here, and not G, so that we don't have to display
# Convert to human-readable format
# NOTE: We convert to M here, and not G, so that we don't have to display
# a size like 1.25G, since decimal places are not allowed by truncate.
# We don't want users thinking they can use decimal numbers, and so in
# this example, they would need to use a size greater then 1280M instead.
@ -227,55 +222,43 @@ def run(args):
"""
Run a postmarketOS image in qemu
"""
# Get arch, device, img_path
arch = pmb.parse.arch.uname_to_qemu(args.arch_native)
if args.arch:
arch = pmb.parse.arch.uname_to_qemu(args.arch)
device = pmb.parse.arch.qemu_to_pmos_device(arch)
img_path = system_image(args, device)
spice_parameters, command_spice = spice_command(args)
logging.info("Running postmarketOS in QEMU VM (" + arch + ")")
# Workaround: qemu runs as local user and needs write permissions in the
# Get the Qemu and spice commands
spice = command_spice(args)
spice_enabled = True if spice else False
qemu = command_qemu(args, arch, device, img_path, spice_enabled)
# Workaround: Qemu runs as local user and needs write permissions in the
# system image, which is owned by root
if not os.access(img_path, os.W_OK):
pmb.helpers.run.root(args, ["chmod", "666", img_path])
run_spice = spice_parameters["enable_spice"]
command = qemu_command(args, arch, device, img_path, spice_parameters)
logging.info("Running postmarketOS in QEMU VM (" + arch + ")")
logging.info("Command: " + " ".join(command))
# Resize the system image (or show hint)
if args.image_size:
resize_image(args, args.image_size, img_path)
else:
logging.info("NOTE: Run 'pmbootstrap qemu --image-size 2G' to set"
" the system image size when you run out of space!")
print()
logging.info("You can connect to the virtual machine using the"
" following services:")
logging.info("(ssh) ssh -p {port} {user}@localhost".format(**vars(args)))
logging.info("(telnet) telnet localhost " + str(args.port + 1))
logging.info("(telnet debug) telnet localhost " + str(args.port + 2))
# SPICE related messages
if not run_spice:
if args.use_spice:
logging.warning("WARNING: Could not find any SPICE client (spicy,"
" remote-viewer) in your PATH, starting without"
" SPICE support!")
else:
logging.info("NOTE: Consider using --spice for potential"
" performance improvements (2d acceleration)")
# SSH/telnet hints
logging.info("Connect to the VM (telnet requires 'pmbootstrap initfs"
" hook_add usb-shell'):")
logging.info("* (ssh) ssh -p {port} {user}@localhost".format(**vars(args)))
logging.info("* (telnet) telnet localhost " + str(args.port + 2))
# Run Qemu (or Qemu + SPICE)
process = None
try:
process = pmb.helpers.run.user(args, command, background=run_spice)
# Launch SPICE client
if run_spice:
logging.info("Command: " + " ".join(command_spice))
pmb.helpers.run.user(args, command_spice)
process = pmb.helpers.run.user(args, qemu, background=spice_enabled)
if spice:
pmb.helpers.run.user(args, spice)
except KeyboardInterrupt:
pass
finally:

View File

@ -123,6 +123,11 @@ def test_questions(args, monkeypatch, tmpdir):
assert func(args, "nokia-rx51") == "us/rx51_us"
assert func(args, "lg-mako") == ""
# Qemu mesa driver
func = pmb.config.init.ask_for_qemu_mesa_driver
answers = ["invalid_driver", "dri-swrast"]
assert func(args) == "dri-swrast"
# UI
answers = ["invalid_UI", "weston"]
assert pmb.config.init.ask_for_ui(args) == "weston"