1# RDMA (ibverb) Ethernet driver {#rdma_doc}
2
3This driver relies on Linux rdma-core (libibverb) userspace poll-mode driver
4to rx/tx Ethernet packets. Despite using the RDMA APIs, this is **not** about
5RDMA (no Infiniband, no RoCE, no iWARP), just pure traditional Ethernet
6packets.
7
8## Maturity level
9Under development: it should work, but has not been thoroughly tested.
10
11## Supported Hardware
12 - Mellanox ConnectX-4
13 - Mellanox ConnectX-5
14
15## Features
16 - bifurcation: MAC based flow steering for transparent sharing of a single
17physical port between multiple virtual interfaces including Linux netdev
18 - multiqueue
19
20## Security considerations
21When creating a rdma interface, it will receive all packets to the MAC address
22attributed to the interface plus a copy of all broadcast and multicast
23traffic.
24The MAC address is under the control of VPP: **the user controlling VPP can
25divert all traffic of any MAC address to the VPP process, including the Linux
26netdev MAC address as long as it can create a rdma interface**.
27The rights to create a rdma interface are controlled by the access rights of
28the `/dev/infiniband/uverbs[0-9]+`device nodes.
29
30## Quickstart
311. Make sure the `ib_uverbs` module is loaded:
32```
33~# modprobe ib_uverbs
34```
352. In VPP, create a new rdma virtual interface tied to the Linux netdev of the
36physical port you want to use (`enp94s0f0` in this example):
37```
38vpp# create int rdma host-if enp94s0f0 name rdma-0
39```
403. Use the interface as usual, eg.:
41```
42vpp# set int ip addr rdma-0 1.1.1.1/24
43vpp# set int st rdma-0 up
44vpp# ping 1.1.1.100`
45```
46
47## Containers support
48It should work in containers as long as:
49 - the `ib_uverbs` module is loaded
50 - the device nodes `/dev/infiniband/uverbs[0-9]+` are usable from the
51   container (but see [security considerations](#Security considerations))
52
53## SR-IOV VFs support
54It should work on SR-IOV VFs the same way it does with PFs. Because of VFs
55security containment features, make sure the MAC address of the rdma VPP
56interface matches the MAC address assigned to the underlying VF.
57For example:
58```
59host# echo 1 > /sys/class/infiniband/mlx5_0/device/sriov_numvfs
60host# ip l set dev enp94s0f0 vf 0 mac 92:5d:f5:df:b1:6f spoof on trust off
61host# ip l set dev enp94s0f2 up
62vpp# create int rdma host-if enp94s0f2 name rdma-0
63vpp# set int mac address rdma-0 92:5d:f5:df:b1:6f
64```
65If you plan to use L2 features such as switching, make sure the underlying
66VF is configured in trusted mode and spoof-checking is disabled (of course, be
67aware of the [security considerations](#Security considerations)):
68```
69host# ip l set dev enp94s0f0 vf 0 spoof off trust on
70```
71
72## Direct Verb mode
73Direct Verb allows the driver to access the NIC HW RX/TX rings directly
74instead of having to go through libibverb and suffering associated overhead.
75It will be automatically selected if the adapter supports it.
76