index.rst revision 80e55855
1
2.. raw:: latex
3
4    \clearpage
5
6.. _VPP_Packet_Throughput:
7
8Packet Throughput
9=================
10
11Throughput graphs are generated based on the results data obtained from
12the |csit-release| test jobs. In order to verify benchmark results
13repeatibility selected, CSIT performance tests are executed multiple
14times (target: 10 times) on each physical testbed type. Box-and-Whisker
15plots are used to display variations in measured throughput values.
16
17Lists of tests selected for multiple execution and graphing are captured
18per testbed type in `test_select_list_{testbed_type}.md
19<https://git.fd.io/csit/tree/docs/job_specs>`_ files.
20
21Graphs are split into sections as follows:
22
23#. **Header 1**: VPP packet path and lookup types
24
25   - **L2 Ethernet Switching**: L2 bridge-doman, L2 cross-connect and L2 patch
26   - **IPv4 Routing**: IPv4 routing with /32 prefixes
27   - **IPv6 Routing**: IPv6 routing with /128 prefixes
28   - **SRv6 Routing**: SRv6 with IPv6 routing
29   - **IPv4 Tunnels**: IPv4 overlay tunnels
30   - **KVM VMs vhost-user**: KVM VMs connected over virtio and vhost-user interfaces
31   - **LXC/DRC Container Memif**: Linux containers and Docker containers connected over Memif interfaces
32   - **IPsec IPv4 Routing**: IPsec encryption/decryption with IPv4 routing
33   - **Virtual Topology System**: VXLAN configurations with L2 bridge-domains
34
35#. **Header 2**: testbeds and NIC models
36
37   - section name format:
38
39     - {**testbed_type**}-{**nic_model**}
40
41   - **testbed_type**:
42
43     - 2n-skx: 2-node Xeon Skylake
44     - 3n-skx: 3-node Xeon Skylake
45     - 2n-clx: 2-node Xeon Cascade Lake
46     - 3n-hsw: 3-node Xeon Haswell
47     - 3n-tsh: 3-node Arm TaiShan
48     - 2n-dnv: 2-node Atom Denverton
49     - 3n-dnv: 3-node Atom Denverton
50
51   - **nic_model**:
52
53     - xxv710: xxv710 2p25GE Intel (Fortville)
54     - x710: x710 4p10GE Intel (Fortville)
55     - xl710: xl710 2p40GE Intel (Fortville)
56     - x520: x520 2p10GE Intel (Niantic)
57     - x553: x553 2p10GE Intel (Niantic)
58
59#. **Header 3**: test group names
60
61   - section name format:
62
63     - {**frame_size**}-{**worker_thread_core_cfg**}-{**vpp_functionality**}-{**vpp_lookup_type**}-{**baseline_scale**}-{**nic_driver**}
64
65   - **frame_size**:
66
67     - 64b: 64 byte frames, smallest frame size for untagged IPv4 packets
68     - 78b: 78 byte frames, smallest frame size for untagged IPv6 packets
69     - 114b: VXLAN encapsulated L2 frames
70     - imix: a sequence of (7x64B, 4x570, 1x1518) byte frames
71
72   - **worker_thread_core_cfg**:
73
74     - 1t1c: 1 worker thread on 1 core, hyper-threading not used
75     - 2t1c: 2 worker threads on 1 core, hyper-threading used
76
77   - **vpp_functionality** (optional):
78
79     - features: including input-acl, output-acl, macip-iacl, nat44
80     - srv6: srv6 encap/decap, proxy
81     - link-bonding: L2 link aggregation with 1 or 2 bonded links
82     - ipsec: IPsec encryption/decryption with different ciphers
83     - vts: Virtual Topology System specific tests
84
85   - **vpp_lookup_type**:
86
87     - l2switching, ip4routing, ip6routing, ip4tunnel, vhost, memif
88
89   - **baseline_scale**:
90
91     - base: baseline tests with less than 10 forwarding entries
92     - scale: scale tests with up to 2 million forwarding entries
93     - base-scale: both baseline and scale tests grouped together
94
95   - **nic_driver**:
96
97     - avf: VPP native avf driver for Intel Fortville NICs
98     - i40e: dpdk poll mode driver for Intel Fortville NICs
99     - ixgbe: dpdk poll mode driver for Intel Niantic NICs
100
101For each test case, Box-and-Whisker plots show the quartiles (Min, 1st
102quartile / 25th percentile, 2nd quartile / 50th percentile / mean, 3rd
103quartile / 75th percentile, Max) across collected data set. Outliers are
104plotted as individual points.
105
106Additional information about graph data:
107
108#. **Graph Title**: describes tested packet path, testbed topology,
109   processor model, NIC model, packet size, number of cores and
110   threads used by data plane workers and indication of VPP DUT
111   configuration.
112
113#. **X-axis Labels**: indices of individual test suites as listed in
114   Graph Legend.
115
116#. **Y-axis Labels**: measured Packets Per Second [pps] throughput
117   values.
118
119#. **Graph Legend**: lists X-axis indices with associated CSIT test
120   suites executed to generate graphed test results.
121
122#. **Hover Information**: lists minimum, first quartile, median,
123   third quartile, and maximum. If either type of outlier is present the
124   whisker on the appropriate side is taken to 1.5×IQR from the quartile
125   (the "inner fence") rather than the max or min, and individual outlying
126   data points are displayed as unfilled circles (for suspected outliers)
127   or filled circles (for outliers). (The "outer fence" is 3×IQR from the
128   quartile.)
129
130.. note::
131
132    Test results have been generated by
133    `FD.io test executor vpp performance job 2n-skx`_,
134    `FD.io test executor vpp performance job 3n-skx`_,
135    `FD.io test executor vpp performance job 2n-clx`_,
136    `FD.io test executor vpp performance job 3n-hsw`_,
137    `FD.io test executor vpp performance job 3n-tsh`_,
138    `FD.io test executor vpp performance job 2n-dnv`_ and
139    `FD.io test executor vpp performance job 3n-dnv`_ with RF
140    result files csit-vpp-perf-|srelease|-\*.zip
141    `archived here <../../_static/archive/>`_.
142    Required per test case data set size is **10**, but for VPP tests
143    the actual size varies per test case and is <=10.
144
145.. toctree::
146    :maxdepth: 3
147
148    l2
149    ip4
150    ip6
151    srv6
152    ip4_tunnels
153    vm_vhost
154    container_memif
155    ipsec
156    vts
157