Configuring VMs and Virt Manager
The next step in the VFIO passthrough process is to properly set up your KVM. With this tutorial we will be using the Virt-Manager software created by RedHat.
This is the Virtual Maching Manager (Virt-Manager). Here you will have a list of KVMs you have built and registered in the Virt-Manager software. The + symbol is to create new KVMs within Virt-Manager. The open button will open a new window with the KVM's console, as well as a details page about your KVM; this button is clickable when you have a KVM selected. The play button will start a selected KVM. The pause button will freeze your KVM in it's current state, if the KVM is running (WARNING: use with caution with KVMs that have hardware passthroughs). The last button is the power button; this button can be used to shutdown a KVM, restart, or force shutdown/reset (NOTE: with hardware being passed to the KVMs were are building in this tutorial, it's best to use the native OS shutdown functions, if possible).
Click the + button and you will be presented with this window. In this tutorial we will be using a local Windows 10 ISO file (Microsoft Download Link). Select "Local install media (ISO image or CDROM)" and click the Forward button.
Next step is to provide an ISO to your OS installation. Click the Browse... button and you will be presented with a different window.
Here you can create a Storage volume, or you can simply click Browse Local and search for you ISO file. I recommend keeping your ISO in a designated location on your system. This is a good practice so you can easily re-use ISO, or in case a KVM still requires that ISO file in order to boot properly (an example would be a KVM that wasn't fully installed or a KVM have holds reference to the ISO in a virtual CDROM).
Once you have your ISO selected, click Forward on the "Create a new virtual machine" page.
In the next window, you will configure your CPUs, and RAM. Base off your hardware, select a number of CPUs, and amount of memory for your computer. For gaming purposes, I recommend 16gbs of memory and 8 CPUs; however, you can get by with 8gbs of RAM and 4 CPUs. Any other purposes will depend on your hardware and level of performance desired (NOTE: CPU performance and lack of RAM can have negative side effects to the overall performance of a GPU, if the GPU is fast enough).
PLEASE READ THE FOLLOWING WARNING BEFORE MOVING ON!
WARNING - Do NOT give your KVM ALL of your available memory or CPUs indicated on the "New VM" window. THIS WILL BREAK YOUR SYSTEM! Your host system will need to meet the distro's minimum requirements in order to function!
In this next step, you will configure your storage. You can create a disk image (essentially create a file on your host that contains your KVM) or you can passthrough a hardware storage device.
To create a disk image, set the amount of storage you wish to give your KVM in the "Create a disk image for virtual machine" field and click Forward.
To passthrough hardware storage, you must have an extra disk storage that is not being used by your host. This can be a HDD, SSD, or even a USB flash drive. To set this up, you will need to select "Select or create custom storage", then click Manage.... From there you will need to set your physical disk as a Storage Volume by clicking the + button to create a "pool". From the "Add a New Storage Pool" window, change your Type to disk: Pysical Disk Device. You will need to set the Source Path to your device and click Finish.
Make sure to look over your KVM's details before clicking Finish. Make sure "Customize configuration before install" is checked. Take this time to name your install as well.
Once you have clicked Finish fromt he previous window, you will be presented with the Overview of your KVM. Make sure the Chipset is set to Q35 and the Firmware is set to a UEFI profile.
Allocation basically means how many are taken from the host's usage and given to the KVM. In this case we are allocating CPUs (taking CPUs from the host system and giving them to the KVM).
Make sure to set the Current allocation to the number of CPUs you wish for you KVM to use. This will designate a number of CPUs on the start of the KVM and remove them from use of your host system until the KVM shuts down. Setting your Current allocation to your max is benfitial for performace, as it tells the system to statically assign CPUs to your KVM instead of dynamically allocating new CPUs to your KVM.
inside of Configuration you can copy your host's CPU configuration or you can manually set it. For the cpu architecture you can copy the host's configurations by checking "Copy host CPU configuration" if you are unsure what you want. If want a particular profile, uncheck "Copy host CPU configuration" and select your profile.
I unchecked "Copy host CPU configuration" and set my profile to "host-passthrough". I had to manually type this out as the profile was not in my list of available profiles (not sure why it wasn't listed). What this does is it will passthrough your host's CPU architecture to the KVM. In my scenario, my Windows KVM detects the hardware as my host's Threadripper 2970wx, however, Windows only sees 16 threads (a Threadripper 2970wx CPU comes with 48 threads, and I gave my KVM 16 of those threads). The reason I did this was so any software on my KVM that requires a desktop CPU architecture to run (in my case Nvidia Geforce Experience would optimize my games without this architecutre).
This is very important for you to match the CPU topology of your host's system, as this is where you define what a CPU meant in your allocation section. Make sure you check "Manually set CPU topology" and assign your Sockets, cores and threads. This will depend on the architecture of your CPU.
-
Sockets- how many sockets are available for your CPU architecture (in my case 1). -
Cores- based from the CPU allocation set above, how many cores are there? -
Threads- how many threads per CPU are there?
In case you are unfamiliar with CPU architectures: a Socket connects the CPU to the motherboard, and a core carries thread(s). Your CPU will most likely carry multiple cores and those cores will carry at least one thread (in most cases there are two threads per core.)
The math you want to follow in this section:
Sockets * Cores * Threads = Max allocation
In my case...
1 Socket * 8 Cores * 2 Threads = 16 allocations
This means I have 1 socket connection for my CPU, I am giving my KVM 8 cores, and all my cores carry 2 threads; this adds up to 16 allocated threads given to my KVM. And because we set Current allocation to 16, my KVM will have 16 threads allocated on start.
RedHat provides drivers for basic KVM functionality. These drivers will help the KVM understand how to simulator different hardware components that aren't really there and translate operations to your host's kernel.
To get started, download the drivers here, and select the stable virtio-win-iso.
Then on your Virt-Manager window, click Add Hardware.
Select Storage and configure it to a Device type of CDROM device. Make sure the bus type is SATA. And select the ISO from your local system by clicking Manage... under "Select or create custom storage". Then Finish.
Now you want to set the boot order of you KVM. To do this, select Boot Options.
Next, make sure Enable boot menu is clicked. Make sure all storage devices are checked (Device storage, CDROM with Windows 10 ISO, and CDROM with Virtio drivers). Make sure the CDROM with the Windows 10 ISO is at the top of the list. To change this, click the CDROM that has the Windows 10 ISO and highlight it, then click the up arrow button until the CDROM is at the top (In my case SATA CDROM1).
This is the final step. You're going to add any PCI devices to be passed through to the KVM (this includes your GPU, GPU audio controller). Click Add Hardware and Select PCI Host Device
Select your GPU you wish to passthrough. Then do this again for your GPU's audio controller.
If you are attaching a PCIe USB hub to the KVM, makse sure to repeat this step for that device as well.
Continue to Windows 10 Setup and Install