NameDateSize

..13-Mar-201835

be.cfgH A D13-Mar-2018154

common.hH A D13-Mar-201818.4 KiB

dpdk_legacy.hH A D13-Mar-2018958

fe.cfgH A D13-Mar-20181.1 KiB

fwdtbl.hH A D13-Mar-20182.7 KiB

gen_fe_cfg.pyH A D13-Mar-20186.2 KiB

lcore.hH A D13-Mar-201811.4 KiB

main.cH A D03-Mar-20207.4 KiB

MakefileH A D31-Dec-20191.2 KiB

netbe.hH A D03-Mar-20207 KiB

parse.cH A D03-Mar-202022.5 KiB

parse.hH A D13-Mar-20181.7 KiB

pkt.cH A D14-Jun-201924.9 KiB

port.hH A D03-Mar-202011.6 KiB

READMEH A D03-Mar-202015.2 KiB

tcp.hH A D13-Mar-201816.3 KiB

test/H04-Mar-202083

udp.hH A D13-Mar-201813.2 KiB

README

11. INTRODUCTION
2
3   l4fwd is a sample application to demonstrate and test TLDK TCP/UDP
4   functionalities. Depending on configuration it can do simple send, recv or
5   both over opened TCP/UDP streams. Also it implements ability to do TCP/UDP
6   packet forwarding between different streams, so it is possible to use the
7   l4fwd application as a TCP/UDP proxy.
8
9   The l4fwd application logically is divided into two parts, Back End (BE) and
10   Front End (FE).
11
121.1 Back End (BE)
13
14   BE is responsible for:
15   - RX over DPDK ports and feed them into TCP/UDP TLDK context(s)
16     (via tle_*_rx_bulk).
17
18   - retrieve packets ready to be send out from TCP/UDP TLDK context(s) and TX
19     them over destined DPDK port.
20
21   - Multiple RX/TX queues per port are supported by RSS. Right now the number
22     of TX is same as the number of RX queue.
23
24- Each BE lcore can serve multiple DPDK ports, TLDK TCP/UDP contexts.
25
26   BE configuration record format:
27
28   port=<uint>,addr=<ipv4/ipv6>,masklen=<uint>,mac=<ether><mtu>
29
30   port -    DPDK port id to be used to send packets to the destination.
31             It is an mandatory option.
32   addr -    destination network address. It is an mandatory option.
33   masklen - destination network prefix length. It is an mandatory option.
34   mac -     destination Ethernet address. It is an mandatory option.
35   mtu -     MTU to be used on that port (= application data size + L2/L3/L4
36             headers sizes, default=1514). It is an optional option.
37
38   Below are some example of BE entries
39
40   port=0,masklen=16,addr=192.168.0.0,mac=01:de:ad:be:ef:01
41   port=0,addr=2001:4860:b002::,masklen=64,mac=01:de:ad:be:ef:01
42
43   These examples are also available in be.cfg file.
44
451.2 Front End (FE)
46
47   FE is responsible for:
48   - to open configured TCP/UDP streams and perform send/recv over them.
49     These streams can belong to different TCP/UDP contexts.
50
51   Each lcore can act as BE and/or FE.
52
53   In UDP mode the application can reassemble input fragmented IP packets, and
54   fragment outgoing IP packets (if destination MTU is less then packet size).
55
56   FE configuration record format:
57
58   lcore=<uint>,op=<"rx|tx|echo|fwd|rxtx">,\
59   laddr=<ip>,lport=<uint16>,raddr=<ip>,rport=<uint16>,\
60   [txlen=<uint>,fwladdr=<ip>,fwlport=<uint16>,fwraddr=<ip>,fwrport=<uint16>,\
61   belcore=<uint>,rxlen=<uint>]
62
63   lcore -   EAL lcore to manage that stream(s) in the FE. It is an mandatory
64             option.
65   belcore - EAL lcore to manage that stream(s) in the BE. It is an optional
66             option. lcore and belcore can specify the same cpu core.
67   op -      operation to perform on that stream:
68             "rx" - do receive only on that stream.
69             "tx" - do send only on that stream.
70             "echo" - mimic recvfrom(..., &addr);sendto(..., &addr);
71             on that stream.
72             "fwd" - forward packets between streams.
73             "rxtx" - Receive/reply transactions on stream.
74             It is an mandatory option.
75   laddr -   local address for the stream to open. It is an mandatory option.
76   lport -   local port for the stream to open. It is an mandatory option.
77   raddr -   remote address for the stream to open. It is an mandatory option.
78   rport -   remote port for the stream to open. It is an mandatory option.
79   txlen -   data length sending in each packet.
80             (mandatory for "tx" & "rxtx" modes only).
81   rxlen -   Expected response length (mandatory for "rxtx" mode only).
82   fwladdr - local address for the forwarding stream(s) to open
83             (mandatory for "fwd" mode only).
84   fwlport - local port for the forwarding stream(s) to open
85             (mandatory for "fwd" mode only).
86   fwraddr - remote address for the forwarding stream(s) to open
87             (mandatory for "fwd" mode only).
88   fwrport - remote port for the forwarding stream(s) to open
89             (mandatory for "fwd" mode only).
90
91   Below are some example of FE entries
92
93   lcore=3,op=echo,laddr=192.168.1.233,lport=0x8000,raddr=0.0.0.0,rport=0
94
95   lcore=3,op=tx,laddr=192.168.1.233,lport=0x8001,raddr=192.168.1.56,\
96   rport=0x200,txlen=72
97
98   lcore=3,op=rx,laddr=::,lport=0x200,raddr=::,rport=0,txlen=72
99
100   lcore=3,op=fwd,laddr=0.0.0.0,lport=11211,raddr=0.0.0.0,rport=0,\
101   fwladdr=::,fwlport=0,fwraddr=2001:4860:b002::56,fwrport=11211
102
103   These examples are also available in fe.cfg file with some more explanation.
104
1051.3 Configuration files format
106
107   - each record on a separate line.
108   - lines started with '#' are treated as comments.
109   - empty lines (containing whitespace chars only) are ignored.
110   - kvargs style format for each record.
111   - each FE record correspond to at least one stream to be opened
112     (could be multiple streams in case of op="fwd").
113   - each BE record define a ipv4/ipv6 destination.
114
1152. REQUIREMENTS
116
117   DPDK libraries (16.11 or higher)
118   TLDK libraries (1.0)
119   Back-End (BE) configuration file
120   Front-End(FE) configuration file
121
1223. USAGE
123
124   l4fwd <DPDK EAL parameters> -- \
125      -P | --promisc          /* promiscuous mode enabled. */    \
126      -R | --rbufs <num>      /* max recv buffers per stream. */ \
127      -S | --sbufs <num>      /* max send buffers per stream. */ \
128      -s | --streams <num>    /* streams to open per context. */ \
129      -b | --becfg <filename> /* backend configuration file. */  \
130      -f | --fecfg <filename> /* frontend configuration file. */ \
131      -c | --txcnt <filename> /* file with TX payload (used by rxtx mode). */  \
132      -U | --udp /* run the app to handle UDP streams only. */ \
133      -T | --tcp /* run the app to handle TCP streams only. */ \
134      -L | --listen /* open TCP streams in server mode (listen). */ \
135      -a | --enable-arp /* enable arp responses (request not supported) */ \
136      -v | --verbose /* different level of verbose mode */ \
137      -H | --hash <string> /* hash algorithm i.e. siphash or jhash to be */ \
138                           /* used to generate the sequence number. */ \
139      -K | --seckey <string> /* 16 character long secret key used by */ \
140                             /* hash algorithms to generate the */ \
141                             /* sequence number. */ \
142      -M | --mbuf-num <num> /* other than default number of mbufs per pool. */ \
143      <port0_params> <port1_params> ... <portN_params>
144
145   Note that: options -U and -T cannot be used together.
146   Option -L can be used only with option -T.
147
148   portX_params: port=<uint>,lcore=<uint>[-<uint>],[lcore=<uint>[-<uint>],]\
149   [rx_offload=<uint>,tx_offload=<uint>,mtu=<uint>,ipv4=<ipv4>,ipv6=<ipv6>]
150
151   portX_params are used to configure the particular DPDK device
152   (rte_ethdev port), and specify BE lcore that will handle RX/TX from/to the
153   device and manage BE part of corresponding TCP/UDP context.
154   Multiple BE lcore can be specified.
155
156   port -       DPDK port id (RSS are supported when multiple lcores are
157                specified for a port). It is an mandatory option.
158   lcore -      EAL lcore id to handle IO over that port (rx_burst/tx_burst).
159                several ports can be managed by the same lcore, and same port
160                can be managed by more than one lcore.
161                It is an mandatory option. At least one lcore option has to be
162                specified. lcore range can be specified in one lcore option.
163                e.g. lcore=2-3,lcore=6 will enable lcores 2, 3, and 6 to
164                handle BE.
165   rx_offload - RX HW offload capabilities to enable/use on this port.
166                (bitmask of DEV_RX_OFFLOAD_* values). It is an optional option.
167   tx_offload - TX HW offload capabilities to enable/use on this port.
168                (bitmask of DEV_TX_OFFLOAD_* values).
169   mtu -        MTU to be used on that port (= application data size + L2/L3/L4
170                headers sizes, default=1514).
171   ipv4 -       ipv4 address to assign to that port.
172   ipv6 -       ipv6 address to assign to that port.
173
174   At least one of ipv4/ipv6 values have to be specified for each port.
175
1763.1 RSS
177
178   If multiple lcore is specified per DPDK port, the following RSS hash will
179   be enabled on that port:
180      ETH_RSS_UDP, or ETH_RSS_TCP
181
182   The RSS queue qid will handle the stream according to the TCP/UDP source
183   ports of the stream. The qid can be calculated as below
184   
185   qid = (src_port % power_of_2(n)) % n
186   
187   where n is number of lcore used to mane the DPDK port.
188
1894. EXAMPLES
190
1914.1 Sample testbed
192
193+----------------------------+                +-------------------------------+
194|                   TLDK Box |                | Linux Box                     |
195|                            |                |                               |
196|                     port 0 +----------------+ port 0                        |
197|                192.168.1.1 |                | 192.168.1.2                   |
198|          2001:4860:b002::1 |                | 2001:4860:b002::2             |
199|          AA:BB:CC:DD:EE:F1 |                | AA:BB:CC:DD:EE:F2             |
200+----------------------------+                +-------------------------------+
201
2024.2 UDP, "rx" mode, IPv4-only, Single core
203
204   This example shows receiving data from a IPv4 stream. The TLDK UDP server
205   runs on single core where both BE and FE run on cpu core 3.
206
207   be.cfg file contains:
208
209   port=0,masklen=24,addr=192.168.1.0,mac=AA:BB:CC:DD:EE:F2
210
211   fe.cfg file contains (udp server listening to port 6000):
212
213   lcore=3,op=rx,laddr=192.168.1.1,lport=6000,raddr=0.0.0.0,rport=0
214
215   run the l4fwd application as below (DPDK port 0 (pci 01:00.0)):
216
217   l4fwd --lcores='3' -w 01:00.0 -- \
218   --promisc --rbufs 0x100 --sbufs 0x100 --streams 0x100 --fecfg fe.cfg \
219   --becfg be.cfg -U port=0,lcore=3,ipv4=192.168.1.1
220
221   This will create TLDK UDP context on lcore=3 (BE lcore) to manage
222   DPDK port 0. The port 0 will have IPv4 address 192.168.1.1.
223   All the streams will be in server mode and also managed by lcore 3.
224
2254.3 UDP, "echo" mode, IPv6-only, Multicore
226
227   This example shows receiving data from a IPv6 stream and sending the data
228   back through the same IPv6 stream. The TLDK UDP server runs on multicore
229   where BE runs on cpu core 2 and FE runs on cpu core 3.
230
231   be.cfg file contains:
232
233   port=0,masklen=64,addr=2001:4860:b002::,mac=AA:BB:CC:DD:EE:F2
234
235   fe.cfg file contains (udp server listening to port 6000):
236
237   lcore=3,op=rx,laddr=2001:4860:b002::1,lport=6000,raddr=::,rport=0
238
239   run the l4fwd application as below (DPDK port 0 (pci 01:00.0)):
240
241   l4fwd --lcores='2,3' -w 01:00.0 -- \
242   --promisc --rbufs 0x100 --sbufs 0x100 --streams 0x100 --fecfg fe.cfg \
243   --becfg be.cfg -U port=0,lcore=2,ipv6=2001:4860:b002::1
244
245   This will create TLDK UDP context on lcore=2 (BE lcore) to manage
246   DPDK port 0. The port 0 will have IPv4 address 2001:4860:b002::1.
247   All the streams will be in server mode and managed by lcore 3 (FE lcore).
248   In this case, the UDP server will send the incoming data back to the sender.
249
2504.4 TCP, "echo" mode, IPv4-only, Multicore, RX-Offload
251
252   This example shows receiving data from a IPv4 stream and sending the data
253   back through the same IPv4 stream. The TLDK TCP server runs on multicore
254   where BE runs on cpu core 2 and FE runs on cpu core 3. The BE also uses
255   receive offload features of the NIC.
256
257   be.cfg file contains:
258
259   port=0,masklen=24,addr=192.168.1.0,mac=AA:BB:CC:DD:EE:F2
260
261   fe.cfg file contains (tcp server listening to port 6000):
262
263   lcore=3,op=echo,laddr=192.168.1.1,lport=6000,raddr=0.0.0.0,rport=0
264
265   run the l4fwd application as below (DPDK port 0 (pci 01:00.0)):
266
267   l4fwd --lcores='2,3' -w 01:00.0 -- \
268   --promisc --rbufs 0x100 --sbufs 0x100 --streams 0x100 --fecfg fe.cfg \
269   --becfg be.cfg -T -L port=0,lcore=2,rx_offload=0xf,tx_offload=0,\
270   ipv4=192.168.1.1
271
272   This will create TLDK TCP context on lcore=2 (BE lcore) to manage
273   DPDK port 0. The port 0 will have IPv4 address 192.168.1.1. The following
274   DPDK RX HW offloads will be enabled on that port.
275      DEV_RX_OFFLOAD_VLAN_STRIP,
276      DEV_RX_OFFLOAD_IPV4_CKSUM,
277      DEV_RX_OFFLOAD_UDP_CKSUM,
278      DEV_RX_OFFLOAD_TCP_CKSUM
279   No HW TX offloads will be enabled.
280   All the streams will be in server mode and managed by lcore 3 (FE core).
281   In this case, the TCP server will send the incoming data back to the sender.
282
2834.5 TCP, "fwd" (proxy) mode, IPv4-to-IPv6, Multi-core, RX-Offload
284
285   This example shows receiving data from a IPv4 stream and forwarding the
286   data to a IPv6 stream. The TLDK TCP server runs on multicore
287   where BE runs on cpu core 2 and FE runs on cpu core 3. The BE also uses
288   receive offload features of the NIC.
289
290   be.cfg file contains:
291
292   port=0,masklen=24,addr=192.168.1.0,mac=AA:BB:CC:DD:EE:F2
293
294   fe.cfg file contains (tcp server listening to port 6000):
295
296   lcore=3,op=fwd,laddr=192.168.1.1,lport=6000,raddr=0.0.0.0,rport=0,\
297      rladdr=::,lport=0,raddr=2001:4860:b002::2,rport=7000
298
299   run the l4fwd application as below (DPDK port 0 (pci 01:00.0)):
300
301   l4fwd --lcores='2,3' -w 01:00.0 -- \
302   --promisc --rbufs 0x100 --sbufs 0x100 --streams 0x100 --fecfg fe.cfg \
303   --becfg be.cfg -T -L port=0,lcore=2,rx_offload=0xf,tx_offload=0,\
304   ipv4=192.168.1.1,ipv6=2001:4860:b002::1
305
306   This will create TLDK TCP context on lcore=2 (BE lcore) to manage
307   DPDK port 0. The port 0 will have IPv4 address 192.168.1.1. The following
308   DPDK RX HW offloads will be enabled on that port.
309      DEV_RX_OFFLOAD_VLAN_STRIP,
310      DEV_RX_OFFLOAD_IPV4_CKSUM,
311      DEV_RX_OFFLOAD_UDP_CKSUM,
312      DEV_RX_OFFLOAD_TCP_CKSUM
313   No HW TX offloads will be enabled.
314   All the streams will be in server mode and managed by lcore 3 (FE core).
315   In this case, the IPv4 TCP server will forward the incoming data to the IPv6
316   TCP server 2001:4860:b002::2 listening to port 7000.
317
3184.6 TCP, "echo" mode, RSS, IPv4-only, Multicore, RX-Offload
319
320   This example shows receiving data from a IPv4 stream and sending the data
321   back through the same IPv4 stream. The TLDK TCP server runs on multicore
322   where BE runs on cpu cores 1-2 and FE runs on cpu core 3. As BE runs on
323   multicore, Receive Side Scaling (RSS) feature will be automatically enabled.
324   The BE also uses receive offload features of the NIC.
325
326   be.cfg file contains:
327
328   port=0,masklen=24,addr=192.168.1.0,mac=AA:BB:CC:DD:EE:F2
329
330   fe.cfg file contains (tcp server listening to port 6000):
331
332   lcore=3,op=echo,laddr=192.168.1.1,lport=6000,raddr=0.0.0.0,rport=0
333   lcore=3,op=echo,laddr=192.168.1.1,lport=6001,raddr=0.0.0.0,rport=0
334
335   run the l4fwd application as below (DPDK port 0 (pci 01:00.0)):
336
337   l4fwd --lcores='1,2,3' -w 01:00.0 -- \
338   --promisc --rbufs 0x100 --sbufs 0x100 --streams 0x100 --fecfg fe.cfg \
339   --becfg be.cfg -T -L port=0,lcore="1-2",rx_offload=0xf,tx_offload=0,\
340   ipv4=192.168.1.1
341
342   This will create TLDK TCP context on lcore=1-2 (BE lcore) to manage
343   DPDK port 0. The port 0 will have IPv4 address 192.168.1.1. The following
344   DPDK RX HW offloads will be enabled on that port.
345      DEV_RX_OFFLOAD_VLAN_STRIP,
346      DEV_RX_OFFLOAD_IPV4_CKSUM,
347      DEV_RX_OFFLOAD_UDP_CKSUM,
348      DEV_RX_OFFLOAD_TCP_CKSUM
349   No HW TX offloads will be enabled.
350   All the streams will be in server mode and managed by lcore 3 (FE core).
351   In this case, the TCP server will send the incoming data back to the sender.
352
353   As RSS is enabled, all the packets with destination port 6000 and 6001 will
354   be managed by HW queue 0 and queue 1 respectively. Please note that RSS
355   is not supported on the interface when both IPv4 and IPv6 are enabled.
356   Only one of IPv4 or IPv6 has to be enabled in the port.
357