i40e.rst revision 32e04ea0
1..  BSD LICENSE
2    Copyright(c) 2016 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31I40E Poll Mode Driver
32======================
33
34The I40E PMD (librte_pmd_i40e) provides poll mode driver support
35for the Intel X710/XL710/X722 10/40 Gbps family of adapters.
36
37
38Features
39--------
40
41Features of the I40E PMD are:
42
43- Multiple queues for TX and RX
44- Receiver Side Scaling (RSS)
45- MAC/VLAN filtering
46- Packet type information
47- Flow director
48- Cloud filter
49- Checksum offload
50- VLAN/QinQ stripping and inserting
51- TSO offload
52- Promiscuous mode
53- Multicast mode
54- Port hardware statistics
55- Jumbo frames
56- Link state information
57- Link flow control
58- Mirror on port, VLAN and VSI
59- Interrupt mode for RX
60- Scattered and gather for TX and RX
61- Vector Poll mode driver
62- DCB
63- VMDQ
64- SR-IOV VF
65- Hot plug
66- IEEE1588/802.1AS timestamping
67
68
69Prerequisites
70-------------
71
72- Identifying your adapter using `Intel Support
73  <http://www.intel.com/support>`_ and get the latest NVM/FW images.
74
75- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
76
77- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
78  section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
79
80
81Pre-Installation Configuration
82------------------------------
83
84Config File Options
85~~~~~~~~~~~~~~~~~~~
86
87The following options can be modified in the ``config`` file.
88Please note that enabling debugging options may affect system performance.
89
90- ``CONFIG_RTE_LIBRTE_I40E_PMD`` (default ``y``)
91
92  Toggle compilation of the ``librte_pmd_i40e`` driver.
93
94- ``CONFIG_RTE_LIBRTE_I40E_DEBUG_*`` (default ``n``)
95
96  Toggle display of generic debugging messages.
97
98- ``CONFIG_RTE_LIBRTE_I40E_RX_ALLOW_BULK_ALLOC`` (default ``y``)
99
100  Toggle bulk allocation for RX.
101
102- ``CONFIG_RTE_LIBRTE_I40E_INC_VECTOR`` (default ``n``)
103
104  Toggle the use of Vector PMD instead of normal RX/TX path.
105  To enable vPMD for RX, bulk allocation for Rx must be allowed.
106
107- ``CONFIG_RTE_LIBRTE_I40E_RX_OLFLAGS_ENABLE`` (default ``y``)
108
109  Toggle to enable RX ``olflags``.
110  This is only meaningful when Vector PMD is used.
111
112- ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` (default ``n``)
113
114  Toggle to use a 16-byte RX descriptor, by default the RX descriptor is 32 byte.
115
116- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF`` (default ``64``)
117
118  Number of queues reserved for PF.
119
120- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF`` (default ``4``)
121
122  Number of queues reserved for each SR-IOV VF.
123
124- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM`` (default ``4``)
125
126  Number of queues reserved for each VMDQ Pool.
127
128- ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` (default ``-1``)
129
130  Interrupt Throttling interval.
131
132
133Driver Compilation
134~~~~~~~~~~~~~~~~~~
135
136To compile the I40E PMD see :ref:`Getting Started Guide for Linux <linux_gsg>` or
137:ref:`Getting Started Guide for FreeBSD <freebsd_gsg>` depending on your platform.
138
139
140Linux
141-----
142
143
144Running testpmd
145~~~~~~~~~~~~~~~
146
147This section demonstrates how to launch ``testpmd`` with Intel XL710/X710
148devices managed by ``librte_pmd_i40e`` in the Linux operating system.
149
150#. Load ``igb_uio`` or ``vfio-pci`` driver:
151
152   .. code-block:: console
153
154      modprobe uio
155      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
156
157   or
158
159   .. code-block:: console
160
161      modprobe vfio-pci
162
163#. Bind the XL710/X710 adapters to ``igb_uio`` or ``vfio-pci`` loaded in the previous step:
164
165   .. code-block:: console
166
167      ./tools/dpdk-devbind.py --bind igb_uio 0000:83:00.0
168
169   Or setup VFIO permissions for regular users and then bind to ``vfio-pci``:
170
171   .. code-block:: console
172
173      ./tools/dpdk-devbind.py --bind vfio-pci 0000:83:00.0
174
175#. Start ``testpmd`` with basic parameters:
176
177   .. code-block:: console
178
179      ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 83:00.0 -- -i
180
181   Example output:
182
183   .. code-block:: console
184
185      ...
186      EAL: PCI device 0000:83:00.0 on NUMA socket 1
187      EAL: probe driver: 8086:1572 rte_i40e_pmd
188      EAL: PCI memory mapped at 0x7f7f80000000
189      EAL: PCI memory mapped at 0x7f7f80800000
190      PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 8000208a
191      Interactive-mode selected
192      Configuring Port 0 (socket 0)
193      ...
194
195      PMD: i40e_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are
196      satisfied.Rx Burst Bulk Alloc function will be used on port=0, queue=0.
197
198      ...
199      Port 0: 68:05:CA:26:85:84
200      Checking link statuses...
201      Port 0 Link Up - speed 10000 Mbps - full-duplex
202      Done
203
204      testpmd>
205
206
207SR-IOV: Prerequisites and sample Application Notes
208~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
209
210#. Load the kernel module:
211
212   .. code-block:: console
213
214      modprobe i40e
215
216   Check the output in dmesg:
217
218   .. code-block:: console
219
220      i40e 0000:83:00.1 ens802f0: renamed from eth0
221
222#. Bring up the PF ports:
223
224   .. code-block:: console
225
226      ifconfig ens802f0 up
227
228#. Create VF device(s):
229
230   Echo the number of VFs to be created into the ``sriov_numvfs`` sysfs entry
231   of the parent PF.
232
233   Example:
234
235   .. code-block:: console
236
237      echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs
238
239
240#. Assign VF MAC address:
241
242   Assign MAC address to the VF using iproute2 utility. The syntax is:
243
244   .. code-block:: console
245
246      ip link set <PF netdev id> vf <VF id> mac <macaddr>
247
248   Example:
249
250   .. code-block:: console
251
252      ip link set ens802f0 vf 0 mac a0:b0:c0:d0:e0:f0
253
254#. Assign VF to VM, and bring up the VM.
255   Please see the documentation for the *I40E/IXGBE/IGB Virtual Function Driver*.
256
257
258Sample Application Notes
259------------------------
260
261Vlan filter
262~~~~~~~~~~~
263
264Vlan filter only works when Promiscuous mode is off.
265
266To start ``testpmd``, and add vlan 10 to port 0:
267
268.. code-block:: console
269
270    ./app/testpmd -c ffff -n 4 -- -i --forward-mode=mac
271    ...
272
273    testpmd> set promisc 0 off
274    testpmd> rx_vlan add 10 0
275
276
277Flow Director
278~~~~~~~~~~~~~
279
280The Flow Director works in receive mode to identify specific flows or sets of flows and route them to specific queues.
281The Flow Director filters can match the different fields for different type of packet: flow type, specific input set per flow type and the flexible payload.
282
283The default input set of each flow type is::
284
285   ipv4-other : src_ip_address, dst_ip_address
286   ipv4-frag  : src_ip_address, dst_ip_address
287   ipv4-tcp   : src_ip_address, dst_ip_address, src_port, dst_port
288   ipv4-udp   : src_ip_address, dst_ip_address, src_port, dst_port
289   ipv4-sctp  : src_ip_address, dst_ip_address, src_port, dst_port,
290                verification_tag
291   ipv6-other : src_ip_address, dst_ip_address
292   ipv6-frag  : src_ip_address, dst_ip_address
293   ipv6-tcp   : src_ip_address, dst_ip_address, src_port, dst_port
294   ipv6-udp   : src_ip_address, dst_ip_address, src_port, dst_port
295   ipv6-sctp  : src_ip_address, dst_ip_address, src_port, dst_port,
296                verification_tag
297   l2_payload : ether_type
298
299The flex payload is selected from offset 0 to 15 of packet's payload by default, while it is masked out from matching.
300
301Start ``testpmd`` with ``--disable-rss`` and ``--pkt-filter-mode=perfect``:
302
303.. code-block:: console
304
305   ./app/testpmd -c ffff -n 4 -- -i --disable-rss --pkt-filter-mode=perfect \
306                 --rxq=8 --txq=8 --nb-cores=8 --nb-ports=1
307
308Add a rule to direct ``ipv4-udp`` packet whose ``dst_ip=2.2.2.5, src_ip=2.2.2.3, src_port=32, dst_port=32`` to queue 1:
309
310.. code-block:: console
311
312   testpmd> flow_director_filter 0 mode IP add flow ipv4-udp  \
313            src 2.2.2.3 32 dst 2.2.2.5 32 vlan 0 flexbytes () \
314	    fwd pf queue 1 fd_id 1
315
316Check the flow director status:
317
318.. code-block:: console
319
320   testpmd> show port fdir 0
321
322   ######################## FDIR infos for port 0      ####################
323     MODE:   PERFECT
324     SUPPORTED FLOW TYPE:  ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other
325                           ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp ipv6-other
326			   l2_payload
327     FLEX PAYLOAD INFO:
328     max_len:	    16	        payload_limit: 480
329     payload_unit:  2	        payload_seg:   3
330     bitmask_unit:  2	        bitmask_num:   2
331     MASK:
332       vlan_tci: 0x0000,
333       src_ipv4: 0x00000000,
334       dst_ipv4: 0x00000000,
335       src_port: 0x0000,
336       dst_port: 0x0000
337       src_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000,
338       dst_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000
339     FLEX PAYLOAD SRC OFFSET:
340       L2_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
341       L3_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
342       L4_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
343     FLEX MASK CFG:
344       ipv4-udp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
345       ipv4-tcp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
346       ipv4-sctp:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
347       ipv4-other:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
348       ipv4-frag:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
349       ipv6-udp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
350       ipv6-tcp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
351       ipv6-sctp:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
352       ipv6-other:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
353       ipv6-frag:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
354       l2_payload:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
355     guarant_count: 1	        best_count:    0
356     guarant_space: 512         best_space:    7168
357     collision:     0	        free:	       0
358     maxhash:	    0	        maxlen:        0
359     add:	    0	        remove:        0
360     f_add:	    0	        f_remove:      0
361
362
363Delete all flow director rules on a port:
364
365.. code-block:: console
366
367   testpmd> flush_flow_director 0
368
369Floating VEB
370~~~~~~~~~~~~~
371
372The Intel® Ethernet Controller X710 and XL710 Family support a feature called
373"Floating VEB".
374
375A Virtual Ethernet Bridge (VEB) is an IEEE Edge Virtual Bridging (EVB) term
376for functionality that allows local switching between virtual endpoints within
377a physical endpoint and also with an external bridge/network.
378
379A "Floating" VEB doesn't have an uplink connection to the outside world so all
380switching is done internally and remains within the host. As such, this
381feature provides security benefits.
382
383In addition, a Floating VEB overcomes a limitation of normal VEBs where they
384cannot forward packets when the physical link is down. Floating VEBs don't need
385to connect to the NIC port so they can still forward traffic from VF to VF
386even when the physical link is down.
387
388Therefore, with this feature enabled VFs can be limited to communicating with
389each other but not an outside network, and they can do so even when there is
390no physical uplink on the associated NIC port.
391
392To enable this feature, the user should pass a ``devargs`` parameter to the
393EAL, for example::
394
395    -w 84:00.0,enable_floating_veb=1
396
397In this configuration the PMD will use the floating VEB feature for all the
398VFs created by this PF device.
399
400Alternatively, the user can specify which VFs need to connect to this floating
401VEB using the ``floating_veb_list`` argument::
402
403    -w 84:00.0,enable_floating_veb=1,floating_veb_list=1;3-4
404
405In this example ``VF1``, ``VF3`` and ``VF4`` connect to the floating VEB,
406while other VFs connect to the normal VEB.
407
408The current implementation only supports one floating VEB and one regular
409VEB. VFs can connect to a floating VEB or a regular VEB according to the
410configuration passed on the EAL command line.
411
412The floating VEB functionality requires a NIC firmware version of 5.0
413or greater.
414
415
416Limitations or Known issues
417---------------------------
418
419MPLS packet classification on X710/XL710
420~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
421
422For firmware versions prior to 5.0, MPLS packets are not recognized by the NIC.
423The L2 Payload flow type in flow director can be used to classify MPLS packet
424by using a command in testpmd like:
425
426   testpmd> flow_director_filter 0 mode IP add flow l2_payload ether \
427            0x8847 flexbytes () fwd pf queue <N> fd_id <M>
428
429With the NIC firmware version 5.0 or greater, some limited MPLS support
430is added: Native MPLS (MPLS in Ethernet) skip is implemented, while no
431new packet type, no classification or offload are possible. With this change,
432L2 Payload flow type in flow director cannot be used to classify MPLS packet
433as with previous firmware versions. Meanwhile, the Ethertype filter can be
434used to classify MPLS packet by using a command in testpmd like:
435
436   testpmd> ethertype_filter 0 add mac_ignr 00:00:00:00:00:00 ethertype \
437            0x8847 fwd queue <M>
438
43916 Byte Descriptor cannot be used on DPDK VF
440~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
441
442If the Linux i40e kernel driver is used as host driver, while DPDK i40e PMD
443is used as the VF driver, DPDK cannot choose 16 byte receive descriptor. That
444is to say, user should keep ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in
445config file.
446
447Link down with i40e kernel driver after DPDK application exist
448~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
449
450After DPDK application quit, and the device is bound back to Linux i40e
451kernel driver, the link cannot be up after ``ifconfig <dev> up``.
452To work around this issue, ``ethtool -s <dev> autoneg on`` should be
453set first and then the link can be brought up through ``ifconfig <dev> up``.
454
455NOTE: requires Linux kernel i40e driver version >= 1.4.X
456
457Receive packets with Ethertype 0x88A8
458~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
459
460Due to the FW limitation, PF can receive packets with Ethertype 0x88A8
461only when floating VEB is disabled.
462