= Device Specification for Inter-VM shared memory device = The Inter-VM shared memory device (ivshmem) is designed to share a memory region between multiple QEMU processes running different guests and the host. In order for all guests to be able to pick up the shared memory area, it is modeled by QEMU as a PCI device exposing said memory to the guest as a PCI BAR. The device can use a shared memory object on the host directly, or it can obtain one from an ivshmem server. In the latter case, the device can additionally interrupt its peers, and get interrupted by its peers. == Configuring the ivshmem PCI device == There are two basic configurations: - Just shared memory: -device ivshmem-plain,memdev=HMB,... This uses host memory backend HMB. It should have option "share" set. - Shared memory plus interrupts: -device ivshmem,chardev=CHR,vectors=N,... An ivshmem server must already be running on the host. The device connects to the server's UNIX domain socket via character device CHR. Each peer gets assigned a unique ID by the server. IDs must be between 0 and 65535. Interrupts are message-signaled (MSI-X). vectors=N configures the number of vectors to use. For more details on ivshmem device properties, see The QEMU Emulator User Documentation (qemu-doc.*). == The ivshmem PCI device's guest interface == The device has vendor ID 1af4, device ID 1110, revision 1. Before QEMU 2.6.0, it had revision 0. === PCI BARs === The ivshmem PCI device has two or three BARs: - BAR0 holds device registers (256 Byte MMIO) - BAR1 holds MSI-X table and PBA (only ivshmem-doorbell) - BAR2 maps the shared memory object There are two ways to use this device: - If you only need the shared memory part, BAR2 suffices. This way, you have access to the shared memory in the guest and can use it as you see fit. Memnic, for example, uses ivshmem this way from guest user space (see http://dpdk.org/browse/memnic). - If you additionally need the capability for peers to interrupt each other, you need BAR0 and BAR1. You will most likely want to write a kernel driver to handle interrupts. Requires the device to be configured for interrupts, obviously. Before QEMU 2.6.0, BAR2 can initially be invalid if the device is configured for interrupts. It becomes safely accessible only after the ivshmem server provided the shared memory. These devices have PCI revision 0 rather than 1. Guest software should wait for the IVPosition register (described below) to become non-negative before accessing BAR2. Revision 0 of the device is not capable to tell guest software whether it is configured for interrupts. === PCI device registers === BAR 0 contains the following registers: Offset Size Access On reset Function 0 4 read/write 0 Interrupt Mask bit 0: peer interrupt (rev 0) reserved (rev 1) bit 1..31: reserved 4 4 read/write 0 Interrupt Status bit 0: peer interrupt (rev 0) reserved (rev 1) bit 1..31: reserved 8 4 read-only 0 or ID IVPosition 12 4 write-only N/A Doorbell bit 0..15: vector bit 16..31: peer ID 16 240 none N/A reserved Software should only access the registers as specified in column "Access". Reserved bits should be ignored on read, and preserved on write. In revision 0 of the device, Interrupt Status and Mask Register together control the legacy INTx interrupt when the device has no MSI-X capability: INTx is asserted when the bit-wise AND of Status and Mask is non-zero and the device has no MSI-X capability. Interrupt Status Register bit 0 becomes 1 when an interrupt request from a peer is received. Reading the register clears it. IVPosition Register: if the device is not configured for interrupts, this is zero. Else, it is the device's ID (between 0 and 65535). Before QEMU 2.6.0, the register may read -1 for a short while after reset. These devices have PCI revision 0 rather than 1. There i