Setting up a Windows 10 Virtual Machine on unRAID with an NVIDIA Graphics Card

Apr 7, 2020

I recently decided to add a Windows 10 virtual machine to my unRAID server so I could potentially start gaming with it, and I ran into all kinds of strange issues. And you might be thinking “gaming on a virtual machine? that sounds terrible!” but in reality it runs with close to bare-metal performance. Here I am going to try and lay out all of the specific steps I went through to get my machine up and running so hopefully you don’t run into the same issues I did!

Step 1: Install the Graphics Card

Before we even start downloading anything, we’re going to need to install our graphics card. I’m not going to go into this, as I’m sure anyone who’s trying to go through this process either already knows how to do this or can figure it out on their own. The one step that is unique to this process is making sure the card is not set as our systems default bios from the startup screen. The steps to do this may vary between different brands of motherboards, but in my case (with the MSI bios tools) I had to navigate to the startup section, find the “Initiate Graphic Adapter” section, and change it from PEG (PCI Express Graphics) to IGD (Integrated Graphics Device). Once thats all set, you can boot up your server and continue with the next steps.

NOTE — If at any point things just aren’t working, try switching your graphics card to a different PCIe port on your motherboard. I have read many reports that this has solved peoples problems.

Step 2: Download a Windows Disk Image

The first step is downloading the most recent disk image for windows 10 from Microsoft. You can get this from the official Microsoft website.

windows 10

If you are on a Mac or Linux machine, this will prompt you to download the ISO file which is what you’re looking for, just select your language and the 64-bit download and you’re all set. If you’re on Windows however, it will prompt you to download the media creation tool. In this case you can download the media creation tool and select “Create installation media” and then “ISO file”.

media creation

Alternatively, you could follow these steps to spoof your browser to think you’re on a mac so it will give you a direct download link.

Next you will need to copy this iso file over to your unraid server, and we’ll want to put this in the isos share. By default this share is not public so to access it from your other computer, click on the Shares tab, click on the isos share, then under “SMB Security Settings” change the Export option to Yes and click apply. Then from your computer where you downloaded the iso, navigate to the share and copy your fresh new iso over to it.

Step 3: Download the Virtio Drivers

Next we’ll need the virtio (virtual input output) drivers for our new VM. Navigate to the Settings tab, and click on VM Manager. In the simple view, all you will see is “Enable VMs” and “Default Windows VirtIO driver ISO”. Click on the dropdown next to this second option and make sure the newest version is selected and then click on the download button next to it.

virtio

This will be downloaded to your iso share as well.

Step 4: Download the vBIOS for your Graphics Card

The last thing we’re gonna need before we create our VM is a vBIOS file for our graphics card. This is basically required for NVIDIA graphics card but apparently it can help with performance for AMD as well, so as long as this doesn’t give you any issues you might as well do it. TechPowerUp has a great library of vBIOS files for pretty much all graphics cards which you can find here.

vbios db

If you don’t know it, the easiest way to find the exact name of your video card in unRAID is to navigate to the Tools tab and click on System Devices. Under “PCI Devices and IOMMU Groups” find your graphics card. It should look something like this:

IOMMU group 14:     [10de:1187] 03:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 760] (rev a1)
                    [10de:0e0a] 03:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)

The part we’re interested in is NVIDIA Corporation GK104 [GeForce GTX 760]. Now, to find the vBIOS for this card, the easiest way would just be to google [card name] vbios +site:techpowerup.com/vgabios/. Once you find the page with the vbios for your card, you’ll need to ensure that where it says “UEFI Supported” it says yes, as we will be setting up the VM with UEFI and in order to do that, the vBIOS must support UEFI as well. If this is not the case, click “Find Compatible Bios” at the bottom and look for the most recent compatible bios for your card, and hopefully it will support UEFI. If it does not, we can boot our VM in legacy mode which I will explain later but I have had issues trying to get this to work. When we have found the correct bios, we can download it by scrolling to the bottom where it will say “Download Now”.

vbios page

There is one issue with all of the more recent version of these bios roms, and by more recent I mean most of them (I had this issue with my GTX 760 I got 9 years ago). The nvidia vbios files will have an nvidia header on them which prevents them from working in a VM. To fix this, we will need to download a hex editor and manually remove this header. If you’re on Windows I recommend HxD and if you’re on a Mac, the first one I found is Hex Fiend. Download one of these and load up your vbios file. Start scrolling down until you find something similar to UªzëK7400éLwÌVI which follows a bunch of lines with just periods.

hex editor

We will need to delete everything above this line, and save the new version of the rom. Once done copy this file over to somewhere on your server, I personally just put it with the Windows iso and the virio drivers. Now we’re finally ready to set up our virtual machine!

Step 5: Set Up the Virtual Machine

Finally we’re ready to set up our machine. However, before you begin the process make sure to plug in a USB mouse and keyboard that we will use to setup the Windows 10 OS.

Head over to the VMS tab of your unRAID server, click Add VM and then click Windows 10. Give it whatever name and description you want. For the rest of the fields, you can leave them as default unless I say otherwise.

Leave CPU Mode on Host Passthrough, and for now Leave the Logical CPUs on the default. Later you can add more but there have been some reported issues getting windows installed with more than one. Something to note is that when you do add more CPUs, you’ll want to make sure you only add CPUs that are paired with each other. However this is only an issue if your CPU has multiple cores which mine does not. To figure out which threads are paired with each other, go to Tools > System Devices > CPU Thread Pairings and remember the numbers that are paired with each other.

threads

You can find more information about this here.

For Initial memory, I’d say give the VM at least 8192 MB, but you can always change this later once you know how it performs. You can leave the Machine field on the default i440fx but if you are trying to troubleshoot issues, one thing you could try is changing this to the most recent Q35. I am not sure what these mean but thats just what I read when looking through forum posts.

For BIOS, I believe its best to use OVMF unless you run into issues with windows installation, in which case I would try SeaBIOS. However if you do have to switch, you will have to recreate the VM entirely. Also, I think the difference between the two is that OVMF is for a UEFI boot, so you should use it if your GPU has UEFI support where as you should use SeaBIOS if it does not.

For OS Install ISO, click the field and the ISO you downloaded earlier should pop up, so select that. The VirtIO Drivers ISO should already be selected in the next field, but if its not, select it.

Next we will need to allocate the Primary vDisk size. We can leave the location as auto, but for the size, decide how big you would like your C drive to be. You don’t need to make this huge as you can map the shares in your array as network drives later. I decided to go with 70GB. At this point, I also passed through a dedicated solid state drive to my VM as a secondary vDisk which I will explain later.

For the Graphics Card, make sure to select your card from the dropdown. Then select the path to the bios rom file we modified earlier. Additionally, we have to select the Sound Card built in to our graphics card as the primary Sound Card. This is because they are in the same IOMMU group and your VM will not start properly otherwise. However, this does not mean you can’t use the audio port built into your case, or USB headphones. To use them, you can just click the plus under sound card and add your whichever audio device you’d like to use.

Finally, we will have to select the mouse and keyboard under USB Devices at the bottom. Here is an example of what your completed config should look like:

config

Step 6: Installing Windows

Before you click create, make sure you have a monitor plugged in to your graphics card. Once that’s all set, click create and you monitor should light up with your freshly created VM.

On the screen, you should see “Press any key to begin installation”, so press a key…

Work in progess…

Step 7: Setup the Rest of your Drivers

Step 8: Fix Demonic Sound

Work in progess…

Optional Steps

There are a couple of things I like to do to setup a new windows installation. The first step is getting the machine up to date with all of the latest updates from microsoft. To do this, just search for updates and install everything there until it says up to date. This will involve restarting your machine probably a few times.

Next I like, to run this script Windows 10 Debloater to get rid of all of the crapware that comes with Windows. There is a GUI version so if you don’t want to deal with the command line, you can just use this.

To run this program, download the zip from this link and extract it. Right click on the file…

Work in progess…

Videos