vmdq_dcb_forwarding.rst revision 97f17497
1..  BSD LICENSE
2    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31VMDQ and DCB Forwarding Sample Application
32==========================================
33
34The VMDQ and DCB Forwarding sample application is a simple example of packet processing using the DPDK.
35The application performs L2 forwarding using VMDQ and DCB to divide the incoming traffic into queues.
36The traffic splitting is performed in hardware by the VMDQ and DCB features of the Intel® 82599 and X710/XL710 Ethernet Controllers.
37
38Overview
39--------
40
41This sample application can be used as a starting point for developing a new application that is based on the DPDK and
42uses VMDQ and DCB for traffic partitioning.
43
44The VMDQ and DCB filters work on MAC and VLAN traffic to divide the traffic into input queues on the basis of the Destination MAC
45address, VLAN ID and VLAN user priority fields.
46VMDQ filters split the traffic into 16 or 32 groups based on the Destination MAC and VLAN ID.
47Then, DCB places each packet into one of queues within that group, based upon the VLAN user priority field.
48
49All traffic is read from a single incoming port (port 0) and output on port 1, without any processing being performed.
50With Intel® 82599 NIC, for example, the traffic is split into 128 queues on input, where each thread of the application reads from
51multiple queues. When run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues.
52
53As supplied, the sample application configures the VMDQ feature to have 32 pools with 4 queues each as indicated in :numref:`figure_vmdq_dcb_example`.
54The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 16 pools of 8 queues. While the
55Intel® X710 or XL710 Ethernet Controller NICs support many configurations of VMDQ pools of 4 or 8 queues each. For simplicity, only 16
56or 32 pools is supported in this sample. And queues numbers for each VMDQ pool can be changed by setting CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM
57in config/common_* file.
58The nb-pools, nb-tcs and enable-rss parameters can be passed on the command line, after the EAL parameters:
59
60.. code-block:: console
61
62    ./build/vmdq_dcb [EAL options] -- -p PORTMASK --nb-pools NP --nb-tcs TC --enable-rss
63
64where, NP can be 16 or 32, TC can be 4 or 8, rss is disabled by default.
65
66.. _figure_vmdq_dcb_example:
67
68.. figure:: img/vmdq_dcb_example.*
69
70   Packet Flow Through the VMDQ and DCB Sample Application
71
72
73In Linux* user space, the application can display statistics with the number of packets received on each queue.
74To have the application display the statistics, send a SIGHUP signal to the running application process.
75
76The VMDQ and DCB Forwarding sample application is in many ways simpler than the L2 Forwarding application
77(see :doc:`l2_forward_real_virtual`)
78as it performs unidirectional L2 forwarding of packets from one port to a second port.
79No command-line options are taken by this application apart from the standard EAL command-line options.
80
81.. note::
82
83    Since VMD queues are being used for VMM, this application works correctly
84    when VTd is disabled in the BIOS or Linux* kernel (intel_iommu=off).
85
86Compiling the Application
87-------------------------
88
89#.  Go to the examples directory:
90
91    .. code-block:: console
92
93        export RTE_SDK=/path/to/rte_sdk
94        cd ${RTE_SDK}/examples/vmdq_dcb
95
96#.  Set the target (a default target is used if not specified). For example:
97
98    .. code-block:: console
99
100        export RTE_TARGET=x86_64-native-linuxapp-gcc
101
102    See the *DPDK Getting Started Guide* for possible RTE_TARGET values.
103
104#.  Build the application:
105
106    .. code-block:: console
107
108        make
109
110Running the Application
111-----------------------
112
113To run the example in a linuxapp environment:
114
115.. code-block:: console
116
117    user@target:~$ ./build/vmdq_dcb -c f -n 4 -- -p 0x3 --nb-pools 32 --nb-tcs 4
118
119Refer to the *DPDK Getting Started Guide* for general information on running applications and
120the Environment Abstraction Layer (EAL) options.
121
122Explanation
123-----------
124
125The following sections provide some explanation of the code.
126
127Initialization
128~~~~~~~~~~~~~~
129
130The EAL, driver and PCI configuration is performed largely as in the L2 Forwarding sample application,
131as is the creation of the mbuf pool.
132See :doc:`l2_forward_real_virtual`.
133Where this example application differs is in the configuration of the NIC port for RX.
134
135The VMDQ and DCB hardware feature is configured at port initialization time by setting the appropriate values in the
136rte_eth_conf structure passed to the rte_eth_dev_configure() API.
137Initially in the application,
138a default structure is provided for VMDQ and DCB configuration to be filled in later by the application.
139
140.. code-block:: c
141
142    /* empty vmdq+dcb configuration structure. Filled in programmatically */
143    static const struct rte_eth_conf vmdq_dcb_conf_default = {
144        .rxmode = {
145            .mq_mode        = ETH_MQ_RX_VMDQ_DCB,
146            .split_hdr_size = 0,
147            .header_split   = 0, /**< Header Split disabled */
148            .hw_ip_checksum = 0, /**< IP checksum offload disabled */
149            .hw_vlan_filter = 0, /**< VLAN filtering disabled */
150            .jumbo_frame    = 0, /**< Jumbo Frame Support disabled */
151        },
152        .txmode = {
153            .mq_mode = ETH_MQ_TX_VMDQ_DCB,
154        },
155        /*
156         * should be overridden separately in code with
157         * appropriate values
158         */
159        .rx_adv_conf = {
160            .vmdq_dcb_conf = {
161                .nb_queue_pools = ETH_32_POOLS,
162                .enable_default_pool = 0,
163                .default_pool = 0,
164                .nb_pool_maps = 0,
165                .pool_map = {{0, 0},},
166                .dcb_tc = {0},
167            },
168            .dcb_rx_conf = {
169                .nb_tcs = ETH_4_TCS,
170                /** Traffic class each UP mapped to. */
171                .dcb_tc = {0},
172            },
173            .vmdq_rx_conf = {
174                .nb_queue_pools = ETH_32_POOLS,
175                .enable_default_pool = 0,
176                .default_pool = 0,
177                .nb_pool_maps = 0,
178                .pool_map = {{0, 0},},
179            },
180        },
181        .tx_adv_conf = {
182            .vmdq_dcb_tx_conf = {
183                .nb_queue_pools = ETH_32_POOLS,
184                .dcb_tc = {0},
185            },
186        },
187    };
188
189The get_eth_conf() function fills in an rte_eth_conf structure with the appropriate values,
190based on the global vlan_tags array,
191and dividing up the possible user priority values equally among the individual queues
192(also referred to as traffic classes) within each pool. With Intel® 82599 NIC,
193if the number of pools is 32, then the user priority fields are allocated 2 to a queue.
194If 16 pools are used, then each of the 8 user priority fields is allocated to its own queue within the pool.
195With Intel® X710/XL710 NICs, if number of tcs is 4, and number of queues in pool is 8,
196then the user priority fields are allocated 2 to one tc, and a tc has 2 queues mapping to it, then
197RSS will determine the destination queue in 2.
198For the VLAN IDs, each one can be allocated to possibly multiple pools of queues,
199so the pools parameter in the rte_eth_vmdq_dcb_conf structure is specified as a bitmask value.
200For destination MAC, each VMDQ pool will be assigned with a MAC address. In this sample, each VMDQ pool
201is assigned to the MAC like 52:54:00:12:<port_id>:<pool_id>, that is,
202the MAC of VMDQ pool 2 on port 1 is 52:54:00:12:01:02.
203
204.. code-block:: c
205
206    const uint16_t vlan_tags[] = {
207        0, 1, 2, 3, 4, 5, 6, 7,
208        8, 9, 10, 11, 12, 13, 14, 15,
209        16, 17, 18, 19, 20, 21, 22, 23,
210        24, 25, 26, 27, 28, 29, 30, 31
211    };
212
213    /* pool mac addr template, pool mac addr is like: 52 54 00 12 port# pool# */
214    static struct ether_addr pool_addr_template = {
215        .addr_bytes = {0x52, 0x54, 0x00, 0x12, 0x00, 0x00}
216    };
217
218    /* Builds up the correct configuration for vmdq+dcb based on the vlan tags array
219     * given above, and the number of traffic classes available for use. */
220    static inline int
221    get_eth_conf(struct rte_eth_conf *eth_conf)
222    {
223        struct rte_eth_vmdq_dcb_conf conf;
224        struct rte_eth_vmdq_rx_conf  vmdq_conf;
225        struct rte_eth_dcb_rx_conf   dcb_conf;
226        struct rte_eth_vmdq_dcb_tx_conf tx_conf;
227        uint8_t i;
228
229        conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools;
230        vmdq_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools;
231        tx_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools;
232        conf.nb_pool_maps = num_pools;
233        vmdq_conf.nb_pool_maps = num_pools;
234        conf.enable_default_pool = 0;
235        vmdq_conf.enable_default_pool = 0;
236        conf.default_pool = 0; /* set explicit value, even if not used */
237        vmdq_conf.default_pool = 0;
238
239        for (i = 0; i < conf.nb_pool_maps; i++) {
240            conf.pool_map[i].vlan_id = vlan_tags[i];
241            vmdq_conf.pool_map[i].vlan_id = vlan_tags[i];
242            conf.pool_map[i].pools = 1UL << i ;
243            vmdq_conf.pool_map[i].pools = 1UL << i;
244        }
245        for (i = 0; i < ETH_DCB_NUM_USER_PRIORITIES; i++){
246            conf.dcb_tc[i] = i % num_tcs;
247            dcb_conf.dcb_tc[i] = i % num_tcs;
248            tx_conf.dcb_tc[i] = i % num_tcs;
249        }
250        dcb_conf.nb_tcs = (enum rte_eth_nb_tcs)num_tcs;
251        (void)(rte_memcpy(eth_conf, &vmdq_dcb_conf_default, sizeof(*eth_conf)));
252        (void)(rte_memcpy(&eth_conf->rx_adv_conf.vmdq_dcb_conf, &conf,
253                  sizeof(conf)));
254        (void)(rte_memcpy(&eth_conf->rx_adv_conf.dcb_rx_conf, &dcb_conf,
255                  sizeof(dcb_conf)));
256        (void)(rte_memcpy(&eth_conf->rx_adv_conf.vmdq_rx_conf, &vmdq_conf,
257                  sizeof(vmdq_conf)));
258        (void)(rte_memcpy(&eth_conf->tx_adv_conf.vmdq_dcb_tx_conf, &tx_conf,
259                  sizeof(tx_conf)));
260        if (rss_enable) {
261            eth_conf->rxmode.mq_mode= ETH_MQ_RX_VMDQ_DCB_RSS;
262            eth_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP |
263                                ETH_RSS_UDP |
264                                ETH_RSS_TCP |
265                                ETH_RSS_SCTP;
266        }
267        return 0;
268    }
269
270    ......
271
272    /* Set mac for each pool.*/
273    for (q = 0; q < num_pools; q++) {
274        struct ether_addr mac;
275        mac = pool_addr_template;
276        mac.addr_bytes[4] = port;
277        mac.addr_bytes[5] = q;
278        printf("Port %u vmdq pool %u set mac %02x:%02x:%02x:%02x:%02x:%02x\n",
279            port, q,
280            mac.addr_bytes[0], mac.addr_bytes[1],
281            mac.addr_bytes[2], mac.addr_bytes[3],
282            mac.addr_bytes[4], mac.addr_bytes[5]);
283        retval = rte_eth_dev_mac_addr_add(port, &mac,
284                q + vmdq_pool_base);
285        if (retval) {
286            printf("mac addr add failed at pool %d\n", q);
287            return retval;
288        }
289    }
290
291Once the network port has been initialized using the correct VMDQ and DCB values,
292the initialization of the port's RX and TX hardware rings is performed similarly to that
293in the L2 Forwarding sample application.
294See :doc:`l2_forward_real_virtual` for more information.
295
296Statistics Display
297~~~~~~~~~~~~~~~~~~
298
299When run in a linuxapp environment,
300the VMDQ and DCB Forwarding sample application can display statistics showing the number of packets read from each RX queue.
301This is provided by way of a signal handler for the SIGHUP signal,
302which simply prints to standard output the packet counts in grid form.
303Each row of the output is a single pool with the columns being the queue number within that pool.
304
305To generate the statistics output, use the following command:
306
307.. code-block:: console
308
309    user@host$ sudo killall -HUP vmdq_dcb_app
310
311Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running,
312rather than the terminal from which the HUP signal was sent.
313