Emulating MIPS guests in Proxmox 8

I wanted to emulate MIPS guests on my Proxmox hypervisor so that I could do some security research on router firmware. Unfortunately, Proxmox has customised some of the QEMU packages and their dependencies, which makes it impossible to install the standard Debian qemu-system-mips package.

To solve this, we need to build a modified version of Proxmox’s QEMU package from source.

Building qemu-system-mips for Proxmox 8

We’ll build this package inside a Debian 12 Bookworm container, rather than building it on the Proxmox host directly. This will allow us to avoid conflicts with Proxmox packages during apt-get build-dep.

In Proxmox’s web interface, select the storage you use to hold container templates, then on the Content page, hit the Templates button. Download the Debian 12 template.

Create a new container that uses this Debian 12 template. You’ll need to give it at least 4GB of total memory (RAM + swap) for the build to complete successfully.

Start up the new container and SSH into it. If you are unable to connect to the container by SSH, you may need to run pct enter YOUR-CONTAINER-ID from Proxmox and run service ssh restart to kick it into life.

Edit /etc/apt/sources.list to add deb-src entries for each of the Debian repositories, and add the Proxmox package repository:

deb-src http://deb.debian.org/debian bookworm main contrib
deb-src http://deb.debian.org/debian bookworm-updates main contrib
deb-src http://security.debian.org bookworm-security main contrib

deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

Now fetch those new repositories and get the container up to date:

# Add the Proxmox repo signing key:
wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg

apt update && apt dist-upgrade -y

Grab the build dependencies for qemu:

apt build-dep qemu -y
apt install git libpci-dev libproxmox-backup-qemu0-dev libsdl1.2-dev libsystemd-dev python3-venv quilt xfslibs-dev lintian check -y

Now use these commands to fetch and build my fork of pve-qemu:

git clone -b alien-architectures-8 https://github.com/thenickdude/pve-qemu.git
cd pve-qemu
make

This will build pve-qemu-kvm_9.0.2-3_amd64.deb and pve-qemu-kvm-dbgsym_9.0.2-3_amd64.deb files in the current directory for you, copy those to Proxmox and install them using:

dpkg -i pve-qemu-kvm_9.0.2-3_amd64.deb pve-qemu-kvm-dbgsym_9.0.2-3_amd64.deb

Note that as well as adding qemu-system-* binaries for MIPS (mips-softmmu, mips64-softmmu, mips64el-softmmu, mipsel-softmmu), this also adds support for ppc-softmmu, ppc64-softmmu, riscv32-softmmu, and riscv64-softmmu.

Creating a Debian MIPS64 virtual machine

We’ll create a Debian MIPS64 virtual machine that we can run with qemu-system-mips64el. First, let’s set up networking. We’ll use qemu-bridge-helper to bridge the VM’s network to Proxmox’s. To enable support for that, run these commands in Proxmox:

# Allow all users to start up the bridge as root:
chmod u+s /usr/lib/kvm/qemu-bridge-helper

# Allow us to connect to Proxmox's default bridge:
mkdir -p /etc/kvm
echo "allow vmbr0" > /etc/kvm/bridge.conf

Grab the image “mips64el-malta” from this website (which are weekly builds produced by the Debian Quick Image Baker Project), rename it to add a “.zip” file extension, unpack it, and copy it to Proxmox where you want your VM to live:

https://people.debian.org/~gio/dqib/

Archive: mips.zip
creating: dqib_mips64el-malta/
inflating: dqib_mips64el-malta/image.qcow2
inflating: dqib_mips64el-malta/initrd
inflating: dqib_mips64el-malta/kernel
inflating: dqib_mips64el-malta/readme.txt
inflating: dqib_mips64el-malta/ssh_user_ecdsa_key
inflating: dqib_mips64el-malta/ssh_user_ed25519_key
inflating: dqib_mips64el-malta/ssh_user_rsa_key

Create a script called “startvm” in the dqib_mips64el-malta/ directory with this content:

#!/usr/bin/env bash

qemu-system-mips64el -m 1G -machine malta -cpu 5KEc -kernel kernel -initrd initrd  -drive file=image.qcow2 -nographic -append "root=LABEL=rootfs console=ttyS0" -serial mon:stdio -net nic -net bridge,br=vmbr0

# To emulate a CPU compatible with Ubiquiti's Nanobeam AC (and lots of their other products), use qemu-system-mips, and "-cpu 34Kf"

Make that script executable:

chmod +x startvm

Now you can start up your new Debian VM by just running that script on Proxmox:

./startvm

The default password for the root account is “root”. There is also an account called “debian” with password “debian”.

If the VM gets stuck, then to terminate the VM press ctrl+A, then press X.

32-bit MIPS guests should use “qemu-system-mips” (or mipsel) as the executable name in the startvm script instead.

Setting up networking

The Debian VM uses static IP addresses and has no DHCP client installed. Run nano /etc/network/interfaces to configure your static IP address and gateway, e.g.

iface eth0 inet static
      address 192.168.0.15/24
      gateway 192.168.0.1

Run nano /etc/resolv.conf to set up your DNS server:

nameserver 192.168.0.1

Then reload that config with:

ifdown eth0
ifup eth0

Afterwards, if you want to switch to using DHCP you can add DHCP support to the VM like so:

apt install isc-dhcp-client -y
echo "auto eth0
iface eth0 inet dhcp" > /etc/network/interfaces
ifdown eth0
ifup eth0