1809f0800SChristian Ehrhardt..  BSD LICENSE
2809f0800SChristian Ehrhardt    Copyright (c) 2016 QLogic Corporation
3809f0800SChristian Ehrhardt    All rights reserved.
4809f0800SChristian Ehrhardt
5809f0800SChristian Ehrhardt    Redistribution and use in source and binary forms, with or without
6809f0800SChristian Ehrhardt    modification, are permitted provided that the following conditions
7809f0800SChristian Ehrhardt    are met:
8809f0800SChristian Ehrhardt
9809f0800SChristian Ehrhardt    * Redistributions of source code must retain the above copyright
10809f0800SChristian Ehrhardt    notice, this list of conditions and the following disclaimer.
11809f0800SChristian Ehrhardt    * Redistributions in binary form must reproduce the above copyright
12809f0800SChristian Ehrhardt    notice, this list of conditions and the following disclaimer in
13809f0800SChristian Ehrhardt    the documentation and/or other materials provided with the
14809f0800SChristian Ehrhardt    distribution.
15809f0800SChristian Ehrhardt    * Neither the name of QLogic Corporation nor the names of its
16809f0800SChristian Ehrhardt    contributors may be used to endorse or promote products derived
17809f0800SChristian Ehrhardt    from this software without specific prior written permission.
18809f0800SChristian Ehrhardt
19809f0800SChristian Ehrhardt    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20809f0800SChristian Ehrhardt    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21809f0800SChristian Ehrhardt    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22809f0800SChristian Ehrhardt    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23809f0800SChristian Ehrhardt    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24809f0800SChristian Ehrhardt    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25809f0800SChristian Ehrhardt    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26809f0800SChristian Ehrhardt    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27809f0800SChristian Ehrhardt    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28809f0800SChristian Ehrhardt    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29809f0800SChristian Ehrhardt    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30809f0800SChristian Ehrhardt
31809f0800SChristian EhrhardtQEDE Poll Mode Driver
32809f0800SChristian Ehrhardt======================
33809f0800SChristian Ehrhardt
34809f0800SChristian EhrhardtThe QEDE poll mode driver library (**librte_pmd_qede**) implements support
353d9b7210SChristian Ehrhardtfor **QLogic FastLinQ QL4xxxx 25G/40G/100G CNA** family of adapters as well
36809f0800SChristian Ehrhardtas their virtual functions (VF) in SR-IOV context. It is supported on
37809f0800SChristian Ehrhardtseveral standard Linux distros like RHEL7.x, SLES12.x and Ubuntu.
38809f0800SChristian EhrhardtIt is compile-tested under FreeBSD OS.
39809f0800SChristian Ehrhardt
40809f0800SChristian EhrhardtMore information can be found at `QLogic Corporation's Website
41809f0800SChristian Ehrhardt<http://www.qlogic.com>`_.
42809f0800SChristian Ehrhardt
43809f0800SChristian EhrhardtSupported Features
44809f0800SChristian Ehrhardt------------------
45809f0800SChristian Ehrhardt
46809f0800SChristian Ehrhardt- Unicast/Multicast filtering
47809f0800SChristian Ehrhardt- Promiscuous mode
48809f0800SChristian Ehrhardt- Allmulti mode
49809f0800SChristian Ehrhardt- Port hardware statistics
503d9b7210SChristian Ehrhardt- Jumbo frames
51809f0800SChristian Ehrhardt- VLAN offload - Filtering and stripping
52809f0800SChristian Ehrhardt- Stateless checksum offloads (IPv4/TCP/UDP)
533d9b7210SChristian Ehrhardt- Multiple Rx/Tx queues
543d9b7210SChristian Ehrhardt- RSS (with RETA/hash table/key)
55809f0800SChristian Ehrhardt- TSS
56809f0800SChristian Ehrhardt- Multiple MAC address
57809f0800SChristian Ehrhardt- Default pause flow control
583d9b7210SChristian Ehrhardt- SR-IOV VF
593d9b7210SChristian Ehrhardt- MTU change
603d9b7210SChristian Ehrhardt- Multiprocess aware
613d9b7210SChristian Ehrhardt- Scatter-Gather
62809f0800SChristian Ehrhardt
63809f0800SChristian EhrhardtNon-supported Features
64809f0800SChristian Ehrhardt----------------------
65809f0800SChristian Ehrhardt
66809f0800SChristian Ehrhardt- SR-IOV PF
67809f0800SChristian Ehrhardt- Tunneling offloads
683d9b7210SChristian Ehrhardt- LRO/TSO
693d9b7210SChristian Ehrhardt- NPAR
70809f0800SChristian Ehrhardt
71809f0800SChristian EhrhardtSupported QLogic Adapters
72809f0800SChristian Ehrhardt-------------------------
73809f0800SChristian Ehrhardt
743d9b7210SChristian Ehrhardt- QLogic FastLinQ QL4xxxx 10G/25G/40G/100G CNAs.
75809f0800SChristian Ehrhardt
76809f0800SChristian EhrhardtPrerequisites
77809f0800SChristian Ehrhardt-------------
78809f0800SChristian Ehrhardt
793d9b7210SChristian Ehrhardt- Requires firmware version **8.10.x.** and management firmware
803d9b7210SChristian Ehrhardt  version **8.10.x or higher**. Firmware may be available
81809f0800SChristian Ehrhardt  inbox in certain newer Linux distros under the standard directory
823d9b7210SChristian Ehrhardt  ``E.g. /lib/firmware/qed/qed_init_values-8.10.9.0.bin``
83809f0800SChristian Ehrhardt
84809f0800SChristian Ehrhardt- If the required firmware files are not available then visit
85809f0800SChristian Ehrhardt  `QLogic Driver Download Center <http://driverdownloads.qlogic.com>`_.
86809f0800SChristian Ehrhardt
87809f0800SChristian EhrhardtPerformance note
88809f0800SChristian Ehrhardt~~~~~~~~~~~~~~~~
89809f0800SChristian Ehrhardt
90809f0800SChristian Ehrhardt- For better performance, it is recommended to use 4K or higher RX/TX rings.
91809f0800SChristian Ehrhardt
92809f0800SChristian EhrhardtConfig File Options
93809f0800SChristian Ehrhardt~~~~~~~~~~~~~~~~~~~
94809f0800SChristian Ehrhardt
95809f0800SChristian EhrhardtThe following options can be modified in the ``.config`` file. Please note that
96809f0800SChristian Ehrhardtenabling debugging options may affect system performance.
97809f0800SChristian Ehrhardt
98809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_PMD`` (default **y**)
99809f0800SChristian Ehrhardt
100809f0800SChristian Ehrhardt  Toggle compilation of QEDE PMD driver.
101809f0800SChristian Ehrhardt
102809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO`` (default **n**)
103809f0800SChristian Ehrhardt
104809f0800SChristian Ehrhardt  Toggle display of generic debugging messages.
105809f0800SChristian Ehrhardt
106809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_DRIVER`` (default **n**)
107809f0800SChristian Ehrhardt
108809f0800SChristian Ehrhardt  Toggle display of ecore related messages.
109809f0800SChristian Ehrhardt
110809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **n**)
111809f0800SChristian Ehrhardt
112809f0800SChristian Ehrhardt  Toggle display of transmit fast path run-time messages.
113809f0800SChristian Ehrhardt
114809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **n**)
115809f0800SChristian Ehrhardt
116809f0800SChristian Ehrhardt  Toggle display of receive fast path run-time messages.
117809f0800SChristian Ehrhardt
118809f0800SChristian Ehrhardt- ``CONFIG_RTE_LIBRTE_QEDE_FW`` (default **""**)
119809f0800SChristian Ehrhardt
120809f0800SChristian Ehrhardt  Gives absolute path of firmware file.
1213d9b7210SChristian Ehrhardt  ``Eg: "/lib/firmware/qed/qed_init_values_zipped-8.10.9.0.bin"``
122809f0800SChristian Ehrhardt  Empty string indicates driver will pick up the firmware file
123809f0800SChristian Ehrhardt  from the default location.
124809f0800SChristian Ehrhardt
125809f0800SChristian EhrhardtDriver Compilation
126809f0800SChristian Ehrhardt~~~~~~~~~~~~~~~~~~
127809f0800SChristian Ehrhardt
128809f0800SChristian EhrhardtTo compile QEDE PMD for Linux x86_64 gcc target, run the following ``make``
129809f0800SChristian Ehrhardtcommand::
130809f0800SChristian Ehrhardt
131809f0800SChristian Ehrhardt   cd <DPDK-source-directory>
132809f0800SChristian Ehrhardt   make config T=x86_64-native-linuxapp-gcc install
133809f0800SChristian Ehrhardt
134809f0800SChristian EhrhardtTo compile QEDE PMD for Linux x86_64 clang target, run the following ``make``
135809f0800SChristian Ehrhardtcommand::
136809f0800SChristian Ehrhardt
137809f0800SChristian Ehrhardt   cd <DPDK-source-directory>
138809f0800SChristian Ehrhardt   make config T=x86_64-native-linuxapp-clang install
139809f0800SChristian Ehrhardt
140809f0800SChristian EhrhardtTo compile QEDE PMD for FreeBSD x86_64 clang target, run the following ``gmake``
141809f0800SChristian Ehrhardtcommand::
142809f0800SChristian Ehrhardt
143809f0800SChristian Ehrhardt   cd <DPDK-source-directory>
144809f0800SChristian Ehrhardt   gmake config T=x86_64-native-bsdapp-clang install
145809f0800SChristian Ehrhardt
146809f0800SChristian EhrhardtTo compile QEDE PMD for FreeBSD x86_64 gcc target, run the following ``gmake``
147809f0800SChristian Ehrhardtcommand::
148809f0800SChristian Ehrhardt
149809f0800SChristian Ehrhardt   cd <DPDK-source-directory>
150809f0800SChristian Ehrhardt   gmake config T=x86_64-native-bsdapp-gcc install -Wl,-rpath=\
1513d9b7210SChristian Ehrhardt                                        /usr/local/lib/gcc49 CC=gcc49
152809f0800SChristian Ehrhardt
153809f0800SChristian Ehrhardt
154809f0800SChristian EhrhardtSample Application Notes
155809f0800SChristian Ehrhardt~~~~~~~~~~~~~~~~~~~~~~~~
156809f0800SChristian Ehrhardt
157809f0800SChristian EhrhardtThis section demonstrates how to launch ``testpmd`` with QLogic 4xxxx
158809f0800SChristian Ehrhardtdevices managed by ``librte_pmd_qede`` in Linux operating system.
159809f0800SChristian Ehrhardt
160809f0800SChristian Ehrhardt#. Request huge pages:
161809f0800SChristian Ehrhardt
162809f0800SChristian Ehrhardt   .. code-block:: console
163809f0800SChristian Ehrhardt
164809f0800SChristian Ehrhardt      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/ \
165809f0800SChristian Ehrhardt                                                                nr_hugepages
166809f0800SChristian Ehrhardt
167809f0800SChristian Ehrhardt#. Load ``igb_uio`` driver:
168809f0800SChristian Ehrhardt
169809f0800SChristian Ehrhardt   .. code-block:: console
170809f0800SChristian Ehrhardt
171809f0800SChristian Ehrhardt      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
172809f0800SChristian Ehrhardt
173809f0800SChristian Ehrhardt#. Bind the QLogic 4xxxx adapters to ``igb_uio`` loaded in the
174809f0800SChristian Ehrhardt   previous step:
175809f0800SChristian Ehrhardt
176809f0800SChristian Ehrhardt   .. code-block:: console
177809f0800SChristian Ehrhardt
1785b1ff351SRicardo Salveti      ./tools/dpdk-devbind.py --bind igb_uio 0000:84:00.0 0000:84:00.1 \
179809f0800SChristian Ehrhardt                                              0000:84:00.2 0000:84:00.3
180809f0800SChristian Ehrhardt
181809f0800SChristian Ehrhardt#. Start ``testpmd`` with basic parameters:
182809f0800SChristian Ehrhardt   (Enable QEDE_DEBUG_INFO=y to view informational messages)
183809f0800SChristian Ehrhardt
184809f0800SChristian Ehrhardt   .. code-block:: console
185809f0800SChristian Ehrhardt
186809f0800SChristian Ehrhardt      testpmd -c 0xff1 -n 4 -- -i --nb-cores=8 --portmask=0xf --rxd=4096 \
187809f0800SChristian Ehrhardt      --txd=4096 --txfreet=4068 --enable-rx-cksum --rxq=4 --txq=4 \
188809f0800SChristian Ehrhardt      --rss-ip --rss-udp
189809f0800SChristian Ehrhardt
190809f0800SChristian Ehrhardt      [...]
191809f0800SChristian Ehrhardt
192809f0800SChristian Ehrhardt    EAL: PCI device 0000:84:00.0 on NUMA socket 1
193809f0800SChristian Ehrhardt    EAL:   probe driver: 1077:1634 rte_qede_pmd
194809f0800SChristian Ehrhardt    EAL:   Not managed by a supported kernel driver, skipped
195809f0800SChristian Ehrhardt    EAL: PCI device 0000:84:00.1 on NUMA socket 1
196809f0800SChristian Ehrhardt    EAL:   probe driver: 1077:1634 rte_qede_pmd
197809f0800SChristian Ehrhardt    EAL:   Not managed by a supported kernel driver, skipped
198809f0800SChristian Ehrhardt    EAL: PCI device 0000:88:00.0 on NUMA socket 1
199809f0800SChristian Ehrhardt    EAL:   probe driver: 1077:1656 rte_qede_pmd
200809f0800SChristian Ehrhardt    EAL:   PCI memory mapped at 0x7f738b200000
201809f0800SChristian Ehrhardt    EAL:   PCI memory mapped at 0x7f738b280000
202809f0800SChristian Ehrhardt    EAL:   PCI memory mapped at 0x7f738b300000
203809f0800SChristian Ehrhardt    PMD: Chip details : BB1
204809f0800SChristian Ehrhardt    PMD: Driver version : QEDE PMD 8.7.9.0_1.0.0
205809f0800SChristian Ehrhardt    PMD: Firmware version : 8.7.7.0
206809f0800SChristian Ehrhardt    PMD: Management firmware version : 8.7.8.0
207809f0800SChristian Ehrhardt    PMD: Firmware file : /lib/firmware/qed/qed_init_values_zipped-8.7.7.0.bin
208809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_common_dev_init:macaddr \
209809f0800SChristian Ehrhardt                                                        00:0e:1e:d2:09:9c
210809f0800SChristian Ehrhardt      [...]
211809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 0 num_desc 4096 \
212809f0800SChristian Ehrhardt                                                tx_free_thresh 4068 socket 0
213809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 1 num_desc 4096 \
214809f0800SChristian Ehrhardt                                                tx_free_thresh 4068 socket 0
215809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 2 num_desc 4096 \
216809f0800SChristian Ehrhardt                                                 tx_free_thresh 4068 socket 0
217809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 3 num_desc 4096 \
218809f0800SChristian Ehrhardt                                                 tx_free_thresh 4068 socket 0
219809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 0 num_desc 4096 \
220809f0800SChristian Ehrhardt                                                rx_buf_size=2148 socket 0
221809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 1 num_desc 4096 \
222809f0800SChristian Ehrhardt                                                rx_buf_size=2148 socket 0
223809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 2 num_desc 4096 \
224809f0800SChristian Ehrhardt                                                rx_buf_size=2148 socket 0
225809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 3 num_desc 4096 \
226809f0800SChristian Ehrhardt                                                rx_buf_size=2148 socket 0
227809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:port 0
228809f0800SChristian Ehrhardt    [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:link status: down
229809f0800SChristian Ehrhardt      [...]
230809f0800SChristian Ehrhardt    Checking link statuses...
231809f0800SChristian Ehrhardt    Port 0 Link Up - speed 25000 Mbps - full-duplex
232809f0800SChristian Ehrhardt    Port 1 Link Up - speed 25000 Mbps - full-duplex
233809f0800SChristian Ehrhardt    Port 2 Link Up - speed 25000 Mbps - full-duplex
234809f0800SChristian Ehrhardt    Port 3 Link Up - speed 25000 Mbps - full-duplex
235809f0800SChristian Ehrhardt    Done
236809f0800SChristian Ehrhardt    testpmd>
237809f0800SChristian Ehrhardt
238809f0800SChristian Ehrhardt
239809f0800SChristian EhrhardtSR-IOV: Prerequisites and Sample Application Notes
240809f0800SChristian Ehrhardt~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
241809f0800SChristian Ehrhardt
242809f0800SChristian EhrhardtThis section provides instructions to configure SR-IOV with Linux OS.
243809f0800SChristian Ehrhardt
2443d9b7210SChristian Ehrhardt**Note**: librte_pmd_qede will be used to bind to SR-IOV VF device and Linux native kernel driver (QEDE) will function as SR-IOV PF driver. Requires PF driver to be 8.10.x.x or higher.
245809f0800SChristian Ehrhardt
246809f0800SChristian Ehrhardt#. Verify SR-IOV and ARI capability is enabled on the adapter using ``lspci``:
247809f0800SChristian Ehrhardt
248809f0800SChristian Ehrhardt   .. code-block:: console
249809f0800SChristian Ehrhardt
250809f0800SChristian Ehrhardt      lspci -s <slot> -vvv
251809f0800SChristian Ehrhardt
252809f0800SChristian Ehrhardt   Example output:
253809f0800SChristian Ehrhardt
254809f0800SChristian Ehrhardt   .. code-block:: console
255809f0800SChristian Ehrhardt
256809f0800SChristian Ehrhardt      [...]
257809f0800SChristian Ehrhardt      Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI)
258809f0800SChristian Ehrhardt      [...]
259809f0800SChristian Ehrhardt      Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV)
260809f0800SChristian Ehrhardt      [...]
261809f0800SChristian Ehrhardt      Kernel driver in use: igb_uio
262809f0800SChristian Ehrhardt
263809f0800SChristian Ehrhardt#. Load the kernel module:
264809f0800SChristian Ehrhardt
265809f0800SChristian Ehrhardt   .. code-block:: console
266809f0800SChristian Ehrhardt
267809f0800SChristian Ehrhardt      modprobe qede
268809f0800SChristian Ehrhardt
269809f0800SChristian Ehrhardt   Example output:
270809f0800SChristian Ehrhardt
271809f0800SChristian Ehrhardt   .. code-block:: console
272809f0800SChristian Ehrhardt
273809f0800SChristian Ehrhardt      systemd-udevd[4848]: renamed network interface eth0 to ens5f0
274809f0800SChristian Ehrhardt      systemd-udevd[4848]: renamed network interface eth1 to ens5f1
275809f0800SChristian Ehrhardt
276809f0800SChristian Ehrhardt#. Bring up the PF ports:
277809f0800SChristian Ehrhardt
278809f0800SChristian Ehrhardt   .. code-block:: console
279809f0800SChristian Ehrhardt
280809f0800SChristian Ehrhardt      ifconfig ens5f0 up
281809f0800SChristian Ehrhardt      ifconfig ens5f1 up
282809f0800SChristian Ehrhardt
283809f0800SChristian Ehrhardt#. Create VF device(s):
284809f0800SChristian Ehrhardt
285809f0800SChristian Ehrhardt   Echo the number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
286809f0800SChristian Ehrhardt   of the parent PF.
287809f0800SChristian Ehrhardt
288809f0800SChristian Ehrhardt   Example output:
289809f0800SChristian Ehrhardt
290809f0800SChristian Ehrhardt   .. code-block:: console
291809f0800SChristian Ehrhardt
292809f0800SChristian Ehrhardt      echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs
293809f0800SChristian Ehrhardt
294809f0800SChristian Ehrhardt
295809f0800SChristian Ehrhardt#. Assign VF MAC address:
296809f0800SChristian Ehrhardt
297809f0800SChristian Ehrhardt   Assign MAC address to the VF using iproute2 utility. The syntax is::
298809f0800SChristian Ehrhardt
299809f0800SChristian Ehrhardt      ip link set <PF iface> vf <VF id> mac <macaddr>
300809f0800SChristian Ehrhardt
301809f0800SChristian Ehrhardt   Example output:
302809f0800SChristian Ehrhardt
303809f0800SChristian Ehrhardt   .. code-block:: console
304809f0800SChristian Ehrhardt
305809f0800SChristian Ehrhardt      ip link set ens5f0 vf 0 mac 52:54:00:2f:9d:e8
306809f0800SChristian Ehrhardt
307809f0800SChristian Ehrhardt
308809f0800SChristian Ehrhardt#. PCI Passthrough:
309809f0800SChristian Ehrhardt
310809f0800SChristian Ehrhardt   The VF devices may be passed through to the guest VM using ``virt-manager`` or
311809f0800SChristian Ehrhardt   ``virsh``. QEDE PMD should be used to bind the VF devices in the guest VM
312809f0800SChristian Ehrhardt   using the instructions outlined in the Application notes above.
313