ena.rst revision 7b53c036
1.. BSD LICENSE
2
3    Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates.
4    All rights reserved.
5
6    Redistribution and use in source and binary forms, with or without
7    modification, are permitted provided that the following conditions
8    are met:
9
10    * Redistributions of source code must retain the above copyright
11    notice, this list of conditions and the following disclaimer.
12    * Redistributions in binary form must reproduce the above copyright
13    notice, this list of conditions and the following disclaimer in
14    the documentation and/or other materials provided with the
15    distribution.
16    * Neither the name of Amazon.com, Inc. nor the names of its
17    contributors may be used to endorse or promote products derived
18    from this software without specific prior written permission.
19
20    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31
32ENA Poll Mode Driver
33====================
34
35The ENA PMD is a DPDK poll-mode driver for the Amazon Elastic
36Network Adapter (ENA) family.
37
38Overview
39--------
40
41The ENA driver exposes a lightweight management interface with a
42minimal set of memory mapped registers and an extendable command set
43through an Admin Queue.
44
45The driver supports a wide range of ENA adapters, is link-speed
46independent (i.e., the same driver is used for 10GbE, 25GbE, 40GbE,
47etc.), and it negotiates and supports an extendable feature set.
48
49ENA adapters allow high speed and low overhead Ethernet traffic
50processing by providing a dedicated Tx/Rx queue pair per CPU core.
51
52The ENA driver supports industry standard TCP/IP offload features such
53as checksum offload and TCP transmit segmentation offload (TSO).
54
55Receive-side scaling (RSS) is supported for multi-core scaling.
56
57Some of the ENA devices support a working mode called Low-latency
58Queue (LLQ), which saves several more microseconds.
59
60Management Interface
61--------------------
62
63ENA management interface is exposed by means of:
64
65* Device Registers
66* Admin Queue (AQ) and Admin Completion Queue (ACQ)
67
68ENA device memory-mapped PCIe space for registers (MMIO registers)
69are accessed only during driver initialization and are not involved
70in further normal device operation.
71
72AQ is used for submitting management commands, and the
73results/responses are reported asynchronously through ACQ.
74
75ENA introduces a very small set of management commands with room for
76vendor-specific extensions. Most of the management operations are
77framed in a generic Get/Set feature command.
78
79The following admin queue commands are supported:
80
81* Create I/O submission queue
82* Create I/O completion queue
83* Destroy I/O submission queue
84* Destroy I/O completion queue
85* Get feature
86* Set feature
87* Get statistics
88
89Refer to ``ena_admin_defs.h`` for the list of supported Get/Set Feature
90properties.
91
92Data Path Interface
93-------------------
94
95I/O operations are based on Tx and Rx Submission Queues (Tx SQ and Rx
96SQ correspondingly). Each SQ has a completion queue (CQ) associated
97with it.
98
99The SQs and CQs are implemented as descriptor rings in contiguous
100physical memory.
101
102Refer to ``ena_eth_io_defs.h`` for the detailed structure of the descriptor
103
104The driver supports multi-queue for both Tx and Rx.
105
106Configuration information
107-------------------------
108
109**DPDK Configuration Parameters**
110
111  The following configuration options are available for the ENA PMD:
112
113   * **CONFIG_RTE_LIBRTE_ENA_PMD** (default y): Enables or disables inclusion
114     of the ENA PMD driver in the DPDK compilation.
115
116
117   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_INIT** (default y): Enables or disables debug
118     logging of device initialization within the ENA PMD driver.
119
120   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_RX** (default n): Enables or disables debug
121     logging of RX logic within the ENA PMD driver.
122
123   * **CONFIG_RTE_LIBRTE_ENA_DEBUG_TX** (default n): Enables or disables debug
124     logging of TX logic within the ENA PMD driver.
125
126   * **CONFIG_RTE_LIBRTE_ENA_COM_DEBUG** (default n): Enables or disables debug
127     logging of low level tx/rx logic in ena_com(base) within the ENA PMD driver.
128
129**ENA Configuration Parameters**
130
131   * **Number of Queues**
132
133     This is the requested number of queues upon initialization, however, the actual
134     number of receive and transmit queues to be created will be the minimum between
135     the maximal number supported by the device and number of queues requested.
136
137   * **Size of Queues**
138
139     This is the requested size of receive/transmit queues, while the actual size
140     will be the minimum between the requested size and the maximal receive/transmit
141     supported by the device.
142
143Building DPDK
144-------------
145
146See the :ref:`DPDK Getting Started Guide for Linux <linux_gsg>` for
147instructions on how to build DPDK.
148
149By default the ENA PMD library will be built into the DPDK library.
150
151For configuring and using UIO and VFIO frameworks, please also refer :ref:`the
152documentation that comes with DPDK suite <linux_gsg>`.
153
154Supported ENA adapters
155----------------------
156
157Current ENA PMD supports the following ENA adapters including:
158
159* ``1d0f:ec20`` - ENA VF
160* ``1d0f:ec21`` - ENA VF with LLQ support
161
162Supported Operating Systems
163---------------------------
164
165Any Linux distribution fulfilling the conditions described in ``System Requirements``
166section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK Release Notes*.
167
168Supported features
169------------------
170
171* Jumbo frames up to 9K
172* Port Hardware Statistics
173* IPv4/TCP/UDP checksum offload
174* TSO offload
175* Multiple receive and transmit queues
176* RSS
177* Low Latency Queue for Tx
178
179Unsupported features
180--------------------
181
182The features supported by the device and not yet supported by this PMD include:
183
184* Asynchronous Event Notification Queue (AENQ)
185
186Prerequisites
187-------------
188
189#. Prepare the system as recommended by DPDK suite.  This includes environment
190   variables, hugepages configuration, tool-chains and configuration
191
192#. Insert igb_uio kernel module using the command 'modprobe igb_uio'
193
194#. Bind the intended ENA device to igb_uio module
195
196
197At this point the system should be ready to run DPDK applications. Once the
198application runs to completion, the ENA can be detached from igb_uio if necessary.
199
200Usage example
201-------------
202
203This section demonstrates how to launch **testpmd** with Amazon ENA
204devices managed by librte_pmd_ena.
205
206#. Load the kernel modules:
207
208   .. code-block:: console
209
210      modprobe uio
211      insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
212
213   .. note::
214
215      Currently Amazon ENA PMD driver depends on igb_uio user space I/O kernel module
216
217#. Mount and request huge pages:
218
219   .. code-block:: console
220
221      mount -t hugetlbfs nodev /mnt/hugepages
222      echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
223
224#. Bind UIO driver to ENA device (using provided by DPDK binding tool):
225
226   .. code-block:: console
227
228      ./tools/dpdk-devbind.py --bind=igb_uio 0000:02:00.1
229
230#. Start testpmd with basic parameters:
231
232   .. code-block:: console
233
234      ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -- -i
235
236   Example output:
237
238   .. code-block:: console
239
240      [...]
241      EAL: PCI device 0000:02:00.1 on NUMA socket -1
242      EAL:   probe driver: 1d0f:ec20 rte_ena_pmd
243      EAL:   PCI memory mapped at 0x7f9b6c400000
244      PMD: eth_ena_dev_init(): Initializing 0:2:0.1
245      Interactive-mode selected
246      Configuring Port 0 (socket 0)
247      Port 0: 00:00:00:11:00:01
248      Checking link statuses...
249      Port 0 Link Up - speed 10000 Mbps - full-duplex
250      Done
251      testpmd>
252