i40e.rst revision 39157ec0
1..  BSD LICENSE
2    Copyright(c) 2016 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31I40E Poll Mode Driver
32======================
33
34The I40E PMD (librte_pmd_i40e) provides poll mode driver support
35for the Intel X710/XL710/X722 10/40 Gbps family of adapters.
36
37
38Features
39--------
40
41Features of the I40E PMD are:
42
43- Multiple queues for TX and RX
44- Receiver Side Scaling (RSS)
45- MAC/VLAN filtering
46- Packet type information
47- Flow director
48- Cloud filter
49- Checksum offload
50- VLAN/QinQ stripping and inserting
51- TSO offload
52- Promiscuous mode
53- Multicast mode
54- Port hardware statistics
55- Jumbo frames
56- Link state information
57- Link flow control
58- Mirror on port, VLAN and VSI
59- Interrupt mode for RX
60- Scattered and gather for TX and RX
61- Vector Poll mode driver
62- DCB
63- VMDQ
64- SR-IOV VF
65- Hot plug
66- IEEE1588/802.1AS timestamping
67
68
69Prerequisites
70-------------
71
72- Identifying your adapter using `Intel Support
73  <http://www.intel.com/support>`_ and get the latest NVM/FW images.
74
75- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment.
76
77- To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms"
78  section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
79
80
81Pre-Installation Configuration
82------------------------------
83
84Config File Options
85~~~~~~~~~~~~~~~~~~~
86
87The following options can be modified in the ``config`` file.
88Please note that enabling debugging options may affect system performance.
89
90- ``CONFIG_RTE_LIBRTE_I40E_PMD`` (default ``y``)
91
92  Toggle compilation of the ``librte_pmd_i40e`` driver.
93
94- ``CONFIG_RTE_LIBRTE_I40E_DEBUG_*`` (default ``n``)
95
96  Toggle display of generic debugging messages.
97
98- ``CONFIG_RTE_LIBRTE_I40E_RX_ALLOW_BULK_ALLOC`` (default ``y``)
99
100  Toggle bulk allocation for RX.
101
102- ``CONFIG_RTE_LIBRTE_I40E_INC_VECTOR`` (default ``n``)
103
104  Toggle the use of Vector PMD instead of normal RX/TX path.
105  To enable vPMD for RX, bulk allocation for Rx must be allowed.
106
107- ``CONFIG_RTE_LIBRTE_I40E_RX_OLFLAGS_ENABLE`` (default ``y``)
108
109  Toggle to enable RX ``olflags``.
110  This is only meaningful when Vector PMD is used.
111
112- ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` (default ``n``)
113
114  Toggle to use a 16-byte RX descriptor, by default the RX descriptor is 32 byte.
115
116- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF`` (default ``64``)
117
118  Number of queues reserved for PF.
119
120- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF`` (default ``4``)
121
122  Number of queues reserved for each SR-IOV VF.
123
124- ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM`` (default ``4``)
125
126  Number of queues reserved for each VMDQ Pool.
127
128- ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` (default ``-1``)
129
130  Interrupt Throttling interval.
131
132
133Runtime Config Options
134~~~~~~~~~~~~~~~~~~~~~~
135
136- ``Support multiple driver`` (default ``disable``)
137
138  There was a multiple driver support issue during use of 700 series Ethernet
139  Adapter with both Linux kernel and DPDK PMD. To fix this issue, ``devargs``
140  parameter ``support-multi-driver`` is introduced, for example::
141
142    -w 84:00.0,support-multi-driver=1
143
144  With the above configuration, DPDK PMD will not change global registers, and
145  will switch PF interrupt from IntN to Int0 to avoid interrupt conflict between
146  DPDK and Linux Kernel.
147
148Driver Compilation
149~~~~~~~~~~~~~~~~~~
150
151To compile the I40E PMD see :ref:`Getting Started Guide for Linux <linux_gsg>` or
152:ref:`Getting Started Guide for FreeBSD <freebsd_gsg>` depending on your platform.
153
154
155Linux
156-----
157
158
159Running testpmd
160~~~~~~~~~~~~~~~
161
162This section demonstrates how to launch ``testpmd`` with Intel XL710/X710
163devices managed by ``librte_pmd_i40e`` in the Linux operating system.
164
165#. Load ``igb_uio`` or ``vfio-pci`` driver:
166
167   .. code-block:: console
168
169      modprobe uio
170      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
171
172   or
173
174   .. code-block:: console
175
176      modprobe vfio-pci
177
178#. Bind the XL710/X710 adapters to ``igb_uio`` or ``vfio-pci`` loaded in the previous step:
179
180   .. code-block:: console
181
182      ./tools/dpdk-devbind.py --bind igb_uio 0000:83:00.0
183
184   Or setup VFIO permissions for regular users and then bind to ``vfio-pci``:
185
186   .. code-block:: console
187
188      ./tools/dpdk-devbind.py --bind vfio-pci 0000:83:00.0
189
190#. Start ``testpmd`` with basic parameters:
191
192   .. code-block:: console
193
194      ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 83:00.0 -- -i
195
196   Example output:
197
198   .. code-block:: console
199
200      ...
201      EAL: PCI device 0000:83:00.0 on NUMA socket 1
202      EAL: probe driver: 8086:1572 rte_i40e_pmd
203      EAL: PCI memory mapped at 0x7f7f80000000
204      EAL: PCI memory mapped at 0x7f7f80800000
205      PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 8000208a
206      Interactive-mode selected
207      Configuring Port 0 (socket 0)
208      ...
209
210      PMD: i40e_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are
211      satisfied.Rx Burst Bulk Alloc function will be used on port=0, queue=0.
212
213      ...
214      Port 0: 68:05:CA:26:85:84
215      Checking link statuses...
216      Port 0 Link Up - speed 10000 Mbps - full-duplex
217      Done
218
219      testpmd>
220
221
222SR-IOV: Prerequisites and sample Application Notes
223~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
224
225#. Load the kernel module:
226
227   .. code-block:: console
228
229      modprobe i40e
230
231   Check the output in dmesg:
232
233   .. code-block:: console
234
235      i40e 0000:83:00.1 ens802f0: renamed from eth0
236
237#. Bring up the PF ports:
238
239   .. code-block:: console
240
241      ifconfig ens802f0 up
242
243#. Create VF device(s):
244
245   Echo the number of VFs to be created into the ``sriov_numvfs`` sysfs entry
246   of the parent PF.
247
248   Example:
249
250   .. code-block:: console
251
252      echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs
253
254
255#. Assign VF MAC address:
256
257   Assign MAC address to the VF using iproute2 utility. The syntax is:
258
259   .. code-block:: console
260
261      ip link set <PF netdev id> vf <VF id> mac <macaddr>
262
263   Example:
264
265   .. code-block:: console
266
267      ip link set ens802f0 vf 0 mac a0:b0:c0:d0:e0:f0
268
269#. Assign VF to VM, and bring up the VM.
270   Please see the documentation for the *I40E/IXGBE/IGB Virtual Function Driver*.
271
272
273Sample Application Notes
274------------------------
275
276Vlan filter
277~~~~~~~~~~~
278
279Vlan filter only works when Promiscuous mode is off.
280
281To start ``testpmd``, and add vlan 10 to port 0:
282
283.. code-block:: console
284
285    ./app/testpmd -c ffff -n 4 -- -i --forward-mode=mac
286    ...
287
288    testpmd> set promisc 0 off
289    testpmd> rx_vlan add 10 0
290
291
292Flow Director
293~~~~~~~~~~~~~
294
295The Flow Director works in receive mode to identify specific flows or sets of flows and route them to specific queues.
296The Flow Director filters can match the different fields for different type of packet: flow type, specific input set per flow type and the flexible payload.
297
298The default input set of each flow type is::
299
300   ipv4-other : src_ip_address, dst_ip_address
301   ipv4-frag  : src_ip_address, dst_ip_address
302   ipv4-tcp   : src_ip_address, dst_ip_address, src_port, dst_port
303   ipv4-udp   : src_ip_address, dst_ip_address, src_port, dst_port
304   ipv4-sctp  : src_ip_address, dst_ip_address, src_port, dst_port,
305                verification_tag
306   ipv6-other : src_ip_address, dst_ip_address
307   ipv6-frag  : src_ip_address, dst_ip_address
308   ipv6-tcp   : src_ip_address, dst_ip_address, src_port, dst_port
309   ipv6-udp   : src_ip_address, dst_ip_address, src_port, dst_port
310   ipv6-sctp  : src_ip_address, dst_ip_address, src_port, dst_port,
311                verification_tag
312   l2_payload : ether_type
313
314The flex payload is selected from offset 0 to 15 of packet's payload by default, while it is masked out from matching.
315
316Start ``testpmd`` with ``--disable-rss`` and ``--pkt-filter-mode=perfect``:
317
318.. code-block:: console
319
320   ./app/testpmd -c ffff -n 4 -- -i --disable-rss --pkt-filter-mode=perfect \
321                 --rxq=8 --txq=8 --nb-cores=8 --nb-ports=1
322
323Add a rule to direct ``ipv4-udp`` packet whose ``dst_ip=2.2.2.5, src_ip=2.2.2.3, src_port=32, dst_port=32`` to queue 1:
324
325.. code-block:: console
326
327   testpmd> flow_director_filter 0 mode IP add flow ipv4-udp  \
328            src 2.2.2.3 32 dst 2.2.2.5 32 vlan 0 flexbytes () \
329	    fwd pf queue 1 fd_id 1
330
331Check the flow director status:
332
333.. code-block:: console
334
335   testpmd> show port fdir 0
336
337   ######################## FDIR infos for port 0      ####################
338     MODE:   PERFECT
339     SUPPORTED FLOW TYPE:  ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other
340                           ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp ipv6-other
341			   l2_payload
342     FLEX PAYLOAD INFO:
343     max_len:	    16	        payload_limit: 480
344     payload_unit:  2	        payload_seg:   3
345     bitmask_unit:  2	        bitmask_num:   2
346     MASK:
347       vlan_tci: 0x0000,
348       src_ipv4: 0x00000000,
349       dst_ipv4: 0x00000000,
350       src_port: 0x0000,
351       dst_port: 0x0000
352       src_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000,
353       dst_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000
354     FLEX PAYLOAD SRC OFFSET:
355       L2_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
356       L3_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
357       L4_PAYLOAD:    0      1	    2	   3	  4	 5	6  ...
358     FLEX MASK CFG:
359       ipv4-udp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
360       ipv4-tcp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
361       ipv4-sctp:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
362       ipv4-other:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
363       ipv4-frag:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
364       ipv6-udp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
365       ipv6-tcp:    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
366       ipv6-sctp:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
367       ipv6-other:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
368       ipv6-frag:   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
369       l2_payload:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
370     guarant_count: 1	        best_count:    0
371     guarant_space: 512         best_space:    7168
372     collision:     0	        free:	       0
373     maxhash:	    0	        maxlen:        0
374     add:	    0	        remove:        0
375     f_add:	    0	        f_remove:      0
376
377
378Delete all flow director rules on a port:
379
380.. code-block:: console
381
382   testpmd> flush_flow_director 0
383
384Floating VEB
385~~~~~~~~~~~~~
386
387The Intel® Ethernet Controller X710 and XL710 Family support a feature called
388"Floating VEB".
389
390A Virtual Ethernet Bridge (VEB) is an IEEE Edge Virtual Bridging (EVB) term
391for functionality that allows local switching between virtual endpoints within
392a physical endpoint and also with an external bridge/network.
393
394A "Floating" VEB doesn't have an uplink connection to the outside world so all
395switching is done internally and remains within the host. As such, this
396feature provides security benefits.
397
398In addition, a Floating VEB overcomes a limitation of normal VEBs where they
399cannot forward packets when the physical link is down. Floating VEBs don't need
400to connect to the NIC port so they can still forward traffic from VF to VF
401even when the physical link is down.
402
403Therefore, with this feature enabled VFs can be limited to communicating with
404each other but not an outside network, and they can do so even when there is
405no physical uplink on the associated NIC port.
406
407To enable this feature, the user should pass a ``devargs`` parameter to the
408EAL, for example::
409
410    -w 84:00.0,enable_floating_veb=1
411
412In this configuration the PMD will use the floating VEB feature for all the
413VFs created by this PF device.
414
415Alternatively, the user can specify which VFs need to connect to this floating
416VEB using the ``floating_veb_list`` argument::
417
418    -w 84:00.0,enable_floating_veb=1,floating_veb_list=1;3-4
419
420In this example ``VF1``, ``VF3`` and ``VF4`` connect to the floating VEB,
421while other VFs connect to the normal VEB.
422
423The current implementation only supports one floating VEB and one regular
424VEB. VFs can connect to a floating VEB or a regular VEB according to the
425configuration passed on the EAL command line.
426
427The floating VEB functionality requires a NIC firmware version of 5.0
428or greater.
429
430
431Limitations or Known issues
432---------------------------
433
434MPLS packet classification on X710/XL710
435~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
436
437For firmware versions prior to 5.0, MPLS packets are not recognized by the NIC.
438The L2 Payload flow type in flow director can be used to classify MPLS packet
439by using a command in testpmd like:
440
441   testpmd> flow_director_filter 0 mode IP add flow l2_payload ether \
442            0x8847 flexbytes () fwd pf queue <N> fd_id <M>
443
444With the NIC firmware version 5.0 or greater, some limited MPLS support
445is added: Native MPLS (MPLS in Ethernet) skip is implemented, while no
446new packet type, no classification or offload are possible. With this change,
447L2 Payload flow type in flow director cannot be used to classify MPLS packet
448as with previous firmware versions. Meanwhile, the Ethertype filter can be
449used to classify MPLS packet by using a command in testpmd like:
450
451   testpmd> ethertype_filter 0 add mac_ignr 00:00:00:00:00:00 ethertype \
452            0x8847 fwd queue <M>
453
45416 Byte Descriptor cannot be used on DPDK VF
455~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
456
457If the Linux i40e kernel driver is used as host driver, while DPDK i40e PMD
458is used as the VF driver, DPDK cannot choose 16 byte receive descriptor. That
459is to say, user should keep ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in
460config file.
461
462Link down with i40e kernel driver after DPDK application exist
463~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
464
465After DPDK application quit, and the device is bound back to Linux i40e
466kernel driver, the link cannot be up after ``ifconfig <dev> up``.
467To work around this issue, ``ethtool -s <dev> autoneg on`` should be
468set first and then the link can be brought up through ``ifconfig <dev> up``.
469
470NOTE: requires Linux kernel i40e driver version >= 1.4.X
471
472Receive packets with Ethertype 0x88A8
473~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
474
475Due to the FW limitation, PF can receive packets with Ethertype 0x88A8
476only when floating VEB is disabled.
477
478Global configuration warning
479~~~~~~~~~~~~~~~~~~~~~~~~~~~~
480
481I40E PMD will set some global registers to enable some function or set some
482configure. Then when using different ports of the same NIC with Linux kernel
483and DPDK, the port with Linux kernel will be impacted by the port with DPDK.
484For example, register I40E_GL_SWT_L2TAGCTRL is used to control L2 tag, i40e
485PMD uses I40E_GL_SWT_L2TAGCTRL to set vlan TPID. If setting TPID in port A
486with DPDK, then the configuration will also impact port B in the NIC with
487kernel driver, which don't want to use the TPID.
488So PMD reports warning to clarify what is changed by writing global register.
489