===== notes =====

<code>
### 1
# Run an intel system with "intel_iommu=on iommu=pt".
# The latter option is optional on older RHEL versions (i.e. 6),
# disables checks for better performance.
$ cat /proc/cmdline 
[..] intel_iommu=on iommu=pt

### 2
# verify that cgroups are created (only applicable to RHEL7)
$ find /sys/kernel/iommu_groups/ -type l

### 3
# inspect the systems pci devices
$ lspci -nvv | less
# For example, I want to export 03:00.1
03:00.1 0200: 14e4:165f
        Subsystem: 1028:0639
[..]
        Kernel driver in use: tg3
        Kernel modules: tg3
# => we want to pass through here device 03:00.1

### 4
# find out the devices in the same cgroup
$ ll /sys/bus/pci/devices/0000\:03\:00.1/iommu_group/devices/
total 0
[..] 0000:03:00.0 -> ../../../../devices/pci0000:00/0000:00:1c.3/0000:03:00.0
[..] 0000:03:00.1 -> ../../../../devices/pci0000:00/0000:00:1c.3/0000:03:00.1
# or via virsh:
$ virsh nodedev-list --tree
computer
  |
  [..]
  +- pci_0000_00_1c_3
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_em3_18_66_da_5a_ee_95
  |   |     
  |   +- pci_0000_03_00_1
# => So we will need to detach these 2 devices,
     pci_0000_03_00_0 and pci_0000_03_00_1, from the host

### 5
# get the virsh name for the device
$ virsh nodedev-list --cap pci
[..]
pci_0000_03_00_1
[..]

### 6
# detach the devices seen in 4)
$ virsh nodedev-dettach pci_0000_03_00_0
Device pci_0000_03_00_0 detached
$ virsh nodedev-dettach pci_0000_03_00_1
Device pci_0000_03_00_1 detached

### 5
# start the VM with the device passthrough
$ virt-install --name=guest1-rhel6-64 \
    --disk path=/var/lib/libvirt/images/guest1-rhel6-64.img,size=8 
    --vcpus=2 --ram=2048 \
    --location=http://ip/6.8/Server/x86_64/os/ 
    --network bridge=virbr0,model=virtio \
    --os-type=linux --os-variant=rhel6 \
    --host-device=pci_0000_03_00_1
</code>

===== errors and hints =====
== error ==
<code>
$ virt-install [..] --host-device=pci_0000_03_00_1
Starting install...
Retrieving file vmlinuz...                                                                                                                               | 4.1 MB  00:00:10     
Retrieving file initrd.img...                                                                                                                            |  39 MB  00:00:12     
Allocating 'guest1-rhel6-64.img'                                                                                                                         | 8.0 GB  00:00:00     
ERROR    internal error: qemu unexpectedly closed the monitor: 2017-05-11T08:25:06.847580Z qemu-kvm: -device vfio-pci,host=03:00.1,id=hostdev0,bus=pci.0,addr=0x8: vfio: error, group 18 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
2017-05-11T08:25:06.847605Z qemu-kvm: -device vfio-pci,host=03:00.1,id=hostdev0,bus=pci.0,addr=0x8: vfio: failed to get group 18
2017-05-11T08:25:06.847629Z qemu-kvm: -device vfio-pci,host=03:00.1,id=hostdev0,bus=pci.0,addr=0x8: Device initialization failed.
2017-05-11T08:25:06.847641Z qemu-kvm: -device vfio-pci,host=03:00.1,id=hostdev0,bus=pci.0,addr=0x8: Device 'vfio-pci' could not be initialized
Removing disk 'guest1-rhel6-64.img'                                                                                                                      |    0 B  00:00:00     
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start guest1-rhel6-64
otherwise, please restart your installation.
</code>
== hint ==
Although just this device is intended to be reached through, all devices in the group have to be in the cgroup, and to be detached from the host. Perform steps 4 and 6 from above.