tag_documentation.rst revision ac89ba8a
1CSIT Tags
2=========
3
4All CSIT test cases are labelled with Robot Framework tags used to allow for
5easy test case type identification, test case grouping and selection for
6execution. Following sections list currently used CSIT tags and their
7descriptions.
8
9Testbed Topology Tags
10---------------------
11
12.. topic:: 2_NODE_DOUBLE_LINK_TOPO
13
14    2 nodes connected in a circular topology with two links interconnecting
15    the devices.
16
17.. topic:: 2_NODE_SINGLE_LINK_TOPO
18
19    2 nodes connected in a circular topology with at least one link
20    interconnecting devices.
21
22.. topic:: 3_NODE_DOUBLE_LINK_TOPO
23
24    3 nodes connected in a circular topology with two links interconnecting
25    the devices.
26
27.. topic:: 3_NODE_SINGLE_LINK_TOPO
28
29    3 nodes connected in a circular topology with at least one link
30    interconnecting devices.
31
32Objective Tags
33--------------
34
35.. topic:: SKIP_PATCH
36
37    Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch)
38    and csit-vpp-verify jobs (i.e. CSIT patch).
39
40.. topic:: SKIP_VPP_PATCH
41
42    Test case(s) marked to not run in case of vpp-csit-verify (i.e. VPP patch).
43
44Environment Tags
45----------------
46
47.. topic:: HW_ENV
48
49    DUTs and TGs are running on bare metal.
50
51.. topic:: VM_ENV
52
53    DUTs and TGs are running in virtual environment.
54
55.. topic:: VPP_VM_ENV
56
57    DUTs with VPP and capable of running Virtual Machine.
58
59NIC Model Tags
60--------------
61
62.. topic:: NIC_Intel-X520-DA2
63
64    Intel X520-DA2 NIC.
65
66.. topic:: NIC_Intel-XL710
67
68    Intel XL710 NIC.
69
70.. topic:: NIC_Intel-X710
71
72    Intel X710 NIC.
73
74.. topic:: NIC_Intel-XXV710
75
76    Intel XXV710 NIC.
77
78.. topic:: NIC_Cisco-VIC-1227
79
80    VIC-1227 by Cisco.
81
82.. topic:: NIC_Cisco-VIC-1385
83
84    VIC-1385 by Cisco.
85
86Scaling Tags
87------------
88
89.. topic:: FIB_20K
90
91    2x10,000 entries in single fib table
92
93.. topic:: FIB_200K
94
95    2x100,000 entries in single fib table
96
97.. topic:: FIB_2M
98
99    2x1,000,000 entries in single fib table
100
101.. topic:: L2BD_1
102
103    Test with 1 L2 bridge domain.
104
105.. topic:: L2BD_10
106
107    Test with 10 L2 bridge domains.
108
109.. topic:: L2BD_100
110
111    Test with 100 L2 bridge domains.
112
113.. topic:: L2BD_1K
114
115    Test with 1000 L2 bridge domains.
116
117.. topic:: VLAN_1
118
119    Test with 1 VLAN sub-interface.
120
121.. topic:: VLAN_10
122
123    Test with 10 VLAN sub-interfaces.
124
125.. topic:: VLAN_100
126
127    Test with 100 VLAN sub-interfaces.
128
129.. topic:: VLAN_1K
130
131    Test with 1000 VLAN sub-interfaces.
132
133.. topic:: VXLAN_1
134
135    Test with 1 VXLAN tunnel.
136
137.. topic:: VXLAN_10
138
139    Test with 10 VXLAN tunnels.
140
141.. topic:: VXLAN_100
142
143    Test with 100 VXLAN tunnels.
144
145.. topic:: VXLAN_1K
146
147    Test with 1000 VXLAN tunnels.
148
149.. topic:: TNL_{t}
150
151    IPSec in tunnel mode - {t} tunnels.
152
153.. topic:: SRC_USER_1
154
155    Traffic flow with 1 unique IP (users) in one direction.
156
157.. topic:: SRC_USER_10
158
159    Traffic flow with 10 unique IPs (users) in one direction.
160
161.. topic:: SRC_USER_100
162
163    Traffic flow with 100 unique IPs (users) in one direction.
164
165.. topic:: SRC_USER_1000
166
167    Traffic flow with 1000 unique IPs (users) in one direction.
168
169.. topic:: SRC_USER_2000
170
171    Traffic flow with 2000 unique IPs (users) in one direction.
172
173.. topic:: SRC_USER_4000
174
175    Traffic flow with 4000 unique IPs (users) in one direction.
176
177.. topic:: 100_FLOWS
178
179    Traffic stream with 100 unique flows (10 IPs/users x 10 UDP ports) in one
180    direction.
181
182.. topic:: 10k_FLOWS
183
184    Traffic stream with 10 000 unique flows (10 IPs/users x 1000 UDP ports) in
185    one direction.
186
187.. topic:: 100k_FLOWS
188
189    Traffic stream with 100 000 unique flows (100 IPs/users x 1000 UDP ports) in
190    one direction.
191
192Test Category Tags
193------------------
194
195.. topic:: FUNCTEST
196
197    All functional test cases.
198
199.. topic:: PERFTEST
200
201    All performance test cases.
202
203Performance Type Tags
204---------------------
205
206.. topic:: NDRPDR
207
208    Single test finding both No Drop Rate and Partial Drop Rate simultaneously.
209    The search is done by optimized algorithm which performs
210    multiple trial runs at different durations and transmit rates.
211    The results come from the final trials, which have duration of 30 seconds.
212
213.. topic:: MRR
214
215    Performance tests where TG sends the traffic at maximum rate (line rate)
216    and reports total sent/received packets over trial duration.
217    The result is an average of 10 trials of 1 second duration.
218
219.. topic:: SOAK
220
221    Performance tests using PLRsearch to find the critical load.
222
223.. topic:: RECONF
224
225    Performance tests aimed to measure lost packets (time) when performing
226    reconfiguration while full throughput offered load is applied.
227
228Ethernet Frame Size Tags
229------------------------
230
231These are describing the traffic offered by Traffic Generator,
232"primary" traffic in case of asymmetric load.
233For traffic between DUTs, or for "secondary" traffic, see ${overhead} value.
234
235.. topic:: 64B
236
237    64B frames used for test. Generic ethernet or IPv4.
238
239.. topic:: 78B
240
241    78B frames used for test. Ipv6.
242
243.. topic:: 114B
244
245    114B frames used for test. IPv4+vxlan.
246
247.. topic:: 118B
248
249    118B frames used for test. Dot1q+IPv4+vxlan.
250
251.. topic:: IMIX
252
253    IMIX frame sequence (28x 64B, 16x 570B, 4x 1518B) used for test.
254
255.. topic:: 1460B
256
257    1460B frames used for test.
258
259.. topic:: 1480B
260
261    1480B frames used for test.
262
263.. topic:: 1514B
264
265    1514B frames used for test.
266
267.. topic:: 1518B
268
269    1518B frames used for test.
270
271.. topic:: 9000B
272
273    9000B frames used for test.
274
275Test Type Tags
276--------------
277
278.. topic:: BASE
279
280    Baseline test cases, no encapsulation, no feature(s) configured in tests.
281
282.. topic:: IP4BASE
283
284    IPv4 baseline test cases, no encapsulation, no feature(s) configured in
285    tests.
286
287.. topic:: IP6BASE
288
289    IPv6 baseline test cases, no encapsulation, no feature(s) configured in
290    tests.
291
292.. topic:: L2XCBASE
293
294    L2XC baseline test cases, no encapsulation, no feature(s) configured in
295    tests.
296
297.. topic:: L2BDBASE
298
299    L2BD baseline test cases, no encapsulation, no feature(s) configured in
300    tests.
301
302.. topic:: L2PATCH
303
304    L2PATCH baseline test cases, no encapsulation, no feature(s) configured in
305    tests.
306
307.. topic:: SCALE
308
309    Scale test cases.
310
311.. topic:: ENCAP
312
313    Test cases where encapsulation is used. Use also encapsulation tag(s).
314
315.. topic:: FEATURE
316
317    At least one feature is configured in test cases. Use also feature tag(s).
318
319.. topic:: TLDK
320
321    Functional test cases for TLDK.
322
323.. topic:: DMM
324
325    Functional test cases for DMM.
326
327.. topic:: TCP
328
329    Tests which use TCP.
330
331.. topic:: TCP_CPS
332
333    Performance tests which measure connections per second using http requests.
334
335.. topic:: TCP_RPS
336
337    Performance tests which measure requests per second using http requests.
338
339.. topic:: HTTP
340
341    Tests which use HTTP.
342
343.. topic:: NF_DENSITY
344
345    Performance tests that measure throughput of multiple VNF and CNF
346    service topologies at different service densities.
347
348NF Service Density Tags
349-----------------------
350
351.. topic:: CHAIN
352
353    NF service density tests with VNF or CNF service chain topology(ies).
354
355.. topic:: PIPE
356
357    NF service density tests with CNF service pipeline topology(ies).
358
359.. topic:: NF_L3FWDIP4
360
361    NF service density tests with DPDK l3fwd IPv4 routing as NF workload.
362
363.. topic:: NF_VPPIP4
364
365    NF service density tests with VPP IPv4 routing as NF workload.
366
367.. topic:: {r}R{c}C
368
369    Service density matrix locator {r}R{c}C, {r}Row denoting number of
370    service instances, {c}Column denoting number of NFs per service
371    instance. {r}=(1,2,4,6,8,10), {c}=(1,2,4,6,8,10).
372
373.. topic:: {n}VM{t}T
374
375    Service density {n}VM{t}T, {n}Number of NF Qemu VMs, {t}Number of threads
376    per NF.
377
378.. topic:: {n}DCRt}T
379
380    Service density {n}DCR{t}T, {n}Number of NF Docker containers, {t}Number of
381    threads per NF.
382
383.. topic:: {n}_ADDED_CHAINS
384
385    {n}Number of chains (or pipelines) added (and/or removed)
386    during RECONF test.
387
388Forwarding Mode Tags
389--------------------
390
391.. topic:: L2BDMACSTAT
392
393    VPP L2 bridge-domain, L2 MAC static.
394
395.. topic:: L2BDMACLRN
396
397    VPP L2 bridge-domain, L2 MAC learning.
398
399.. topic:: L2XCFWD
400
401    VPP L2 point-to-point cross-connect.
402
403.. topic:: IP4FWD
404
405    VPP IPv4 routed forwarding.
406
407.. topic:: IP6FWD
408
409    VPP IPv6 routed forwarding.
410
411.. topic:: LOADBALANCER
412
413    VPP Load balancer.
414
415Underlay Tags
416-------------
417
418.. topic:: IP4UNRLAY
419
420    IPv4 underlay.
421
422.. topic:: IP6UNRLAY
423
424    IPv6 underlay.
425
426.. topic:: MPLSUNRLAY
427
428    MPLS underlay.
429
430Overlay Tags
431------------
432
433.. topic:: L2OVRLAY
434
435    L2 overlay.
436
437.. topic:: IP4OVRLAY
438
439    IPv4 overlay (IPv4 payload).
440
441.. topic:: IP6OVRLAY
442
443    IPv6 overlay (IPv6 payload).
444
445Tagging Tags
446------------
447
448.. topic:: DOT1Q
449
450    All test cases with dot1q.
451
452.. topic:: DOT1AD
453
454    All test cases with dot1ad.
455
456Encapsulation Tags
457------------------
458
459.. topic:: ETH
460
461    All test cases with base Ethernet (no encapsulation).
462
463.. topic:: LISP
464
465    All test cases with LISP.
466
467.. topic:: LISPGPE
468
469    All test cases with LISP-GPE.
470
471.. topic:: LISP_IP4o4
472
473    All test cases with LISP_IP4o4.
474
475.. topic:: LISPGPE_IP4o4
476
477    All test cases with LISPGPE_IP4o4.
478
479.. topic:: LISPGPE_IP6o4
480
481    All test cases with LISPGPE_IP6o4.
482
483.. topic:: LISPGPE_IP4o6
484
485    All test cases with LISPGPE_IP4o6.
486
487.. topic:: LISPGPE_IP6o6
488
489    All test cases with LISPGPE_IP6o6.
490
491.. topic:: VXLAN
492
493    All test cases with Vxlan.
494
495.. topic:: VXLANGPE
496
497    All test cases with VXLAN-GPE.
498
499.. topic:: GRE
500
501    All test cases with GRE.
502
503.. topic:: IPSEC
504
505    All test cases with IPSEC.
506
507.. topic:: SRv6
508
509    All test cases with Segment routing over IPv6 dataplane.
510
511.. topic:: SRv6_1SID
512
513    All SRv6 test cases with single SID.
514
515.. topic:: SRv6_2SID_DECAP
516
517    All SRv6 test cases with two SIDs and with decapsulation.
518
519.. topic:: SRv6_2SID_NODECAP
520
521    All SRv6 test cases with two SIDs and without decapsulation.
522
523Interface Tags
524--------------
525
526.. topic:: PHY
527
528    All test cases which use physical interface(s).
529
530.. topic:: VHOST
531
532    All test cases which uses VHOST.
533
534.. topic:: VHOST_256
535
536    All test cases which uses VHOST with qemu queue size set to 256.
537
538.. topic:: VHOST_1024
539
540    All test cases which uses VHOST with qemu queue size set to 1024.
541
542.. topic:: CFS_OPT
543
544    All test cases which uses VM with optimised scheduler policy.
545
546.. topic:: TUNTAP
547
548    All test cases which uses TUN and TAP.
549
550.. topic:: AFPKT
551
552    All test cases which uses AFPKT.
553
554.. topic:: NETMAP
555
556    All test cases which uses Netmap.
557
558.. topic:: MEMIF
559
560    All test cases which uses Memif.
561
562.. topic:: SINGLE_MEMIF
563
564    All test cases which uses only single Memif connection per DUT. One DUT
565    instance is running in container having one physical interface exposed to
566    container.
567
568.. topic:: LBOND
569
570    All test cases which uses link bonding (BondEthernet interface).
571
572.. topic:: LBOND_DPDK
573
574    All test cases which uses DPDK link bonding.
575
576.. topic:: LBOND_VPP
577
578    All test cases which uses VPP link bonding.
579
580.. topic:: LBOND_MODE_XOR
581
582    All test cases which uses link bonding with mode XOR.
583
584.. topic:: LBOND_MODE_LACP
585
586    All test cases which uses link bonding with mode LACP.
587
588.. topic:: LBOND_LB_L34
589
590    All test cases which uses link bonding with load-balance mode l34.
591
592.. topic:: LBOND_1L
593
594    All test cases which uses one link for link bonding.
595
596.. topic:: LBOND_2L
597
598    All test cases which uses two links for link bonding.
599
600.. topic:: DRV_AVF
601
602    All test cases which uses Intel Adaptive Virtual Function (AVF) device
603    plugin for VPP. This plugins provides native device support for Intel AVF.
604    AVF is driver specification for current and future Intel Virtual Function
605    devices. In essence, today this driver can be used only with Intel
606    XL710 / X710 / XXV710 adapters.
607
608.. topic:: DRV_VFIO_PCI
609
610    All test cases which uses vfio-pci device driver. It supports variety of NIC
611    adapters.
612
613.. topic:: DRV_RDMA_CORE
614
615    All test cases which uses rdma-core device driver. It supports Mellanox
616    NIC adapters.
617
618Feature Tags
619------------
620
621.. topic:: IACLDST
622
623    iACL destination.
624
625.. topic:: COPWHLIST
626
627    COP whitelist.
628
629.. topic:: NAT44
630
631    NAT44 configured and tested.
632
633.. topic:: NAT64
634
635    NAT44 configured and tested.
636
637.. topic:: ACL
638
639    ACL plugin configured and tested.
640
641.. topic:: IACL
642
643    ACL plugin configured and tested on input path.
644
645.. topic:: OACL
646
647    ACL plugin configured and tested on output path.
648
649.. topic:: ACL_STATELESS
650
651    ACL plugin configured and tested in stateless mode (permit action).
652
653.. topic:: ACL_STATEFUL
654
655    ACL plugin configured and tested in stateful mode (permit+reflect action).
656
657.. topic:: ACL1
658
659    ACL plugin configured and tested with 1 not-hitting ACE.
660
661.. topic:: ACL10
662
663    ACL plugin configured and tested with 10 not-hitting ACEs.
664
665.. topic:: ACL50
666
667    ACL plugin configured and tested with 50 not-hitting ACEs.
668
669.. topic:: SRv6_PROXY
670
671    SRv6 endpoint to SR-unaware appliance via proxy.
672
673.. topic:: SRv6_PROXY_STAT
674
675    SRv6 endpoint to SR-unaware appliance via static proxy.
676
677.. topic:: SRv6_PROXY_DYN
678
679    SRv6 endpoint to SR-unaware appliance via dynamic proxy.
680
681.. topic:: SRv6_PROXY_MASQ
682
683    SRv6 endpoint to SR-unaware appliance via masquerading proxy.
684
685Encryption Tags
686---------------
687
688.. topic:: IPSECSW
689
690    Crypto in software.
691
692.. topic:: IPSECHW
693
694    Crypto in hardware.
695
696.. topic:: IPSECTRAN
697
698    IPSec in transport mode.
699
700.. topic:: IPSECTUN
701
702    IPSec in tunnel mode.
703
704.. topic:: IPSECINT
705
706    IPSec in interface mode.
707
708.. topic:: AES
709
710    IPSec using AES algorithms.
711
712.. topic:: AES_128_CBC
713
714    IPSec using AES 128 CBC algorithms.
715
716.. topic:: AES_128_GCM
717
718    IPSec using AES 128 GCM algorithms.
719
720.. topic:: AES_256_GCM
721
722    IPSec using AES 256 GCM algorithms.
723
724.. topic:: HMAC
725
726    IPSec using HMAC integrity algorithms.
727
728.. topic:: HMAC_SHA_256
729
730    IPSec using HMAC SHA 256 integrity algorithms.
731
732.. topic:: HMAC_SHA_512
733
734    IPSec using HMAC SHA 512 integrity algorithms.
735
736Client-Workload Tags
737--------------------
738
739.. topic:: VM
740
741    All test cases which use at least one virtual machine.
742
743.. topic:: LXC
744
745    All test cases which use Linux container and LXC utils.
746
747.. topic:: DRC
748
749    All test cases which use at least one Docker container.
750
751.. topic:: DOCKER
752
753    All test cases which use Docker as container manager.
754
755.. topic:: APP
756
757    All test cases with specific APP use.
758
759Container Orchestration Tags
760----------------------------
761
762.. topic:: K8S
763
764    All test cases which use Kubernetes for orchestration.
765
766.. topic:: SFC_CONTROLLER
767
768    All test cases which use ligato/sfc_controller for driving configuration
769    of vpp inside container.
770
771.. topic:: VPP_AGENT
772
773    All test cases which use Golang implementation of a control/management plane
774    for VPP
775
776.. topic:: 1VSWITCH
777
778    VPP running in Docker container acting as VSWITCH.
779
780.. topic:: 1VNF
781
782    1 VPP running in Docker container acting as VNF work load.
783
784.. topic:: 2VNF
785
786    2 VPP running in 2 Docker containers acting as VNF work load.
787
788.. topic:: 4VNF
789
790    4 VPP running in 4 Docker containers acting as VNF work load.
791
792Multi-Threading Tags
793--------------------
794
795.. topic:: STHREAD
796
797   *Dynamic tag*.
798   All test cases using single poll mode thread.
799
800.. topic:: MTHREAD
801
802   *Dynamic tag*.
803    All test cases using more then one poll mode driver thread.
804
805.. topic:: 1NUMA
806
807    All test cases with packet processing on single socket.
808
809.. topic:: 2NUMA
810
811    All test cases with packet processing on two sockets.
812
813.. topic:: 1C
814
815    1 worker thread pinned to 1 dedicated physical core; or if HyperThreading is
816    enabled, 2 worker threads each pinned to a separate logical core within 1
817    dedicated physical core. Main thread pinned to core 1.
818
819.. topic:: 2C
820
821    2 worker threads pinned to 2 dedicated physical cores; or if HyperThreading
822    is enabled, 4 worker threads each pinned to a separate logical core within 2
823    dedicated physical cores. Main thread pinned to core 1.
824
825.. topic:: 4C
826
827    4 worker threads pinned to 4 dedicated physical cores; or if HyperThreading
828    is enabled, 8 worker threads each pinned to a separate logical core within 4
829    dedicated physical cores. Main thread pinned to core 1.
830
831.. topic:: 1T1C
832
833   *Dynamic tag*.
834    1 worker thread pinned to 1 dedicated physical core. 1 receive queue per
835    interface. Main thread pinned to core 1.
836
837.. topic:: 2T2C
838
839   *Dynamic tag*.
840    2 worker threads pinned to 2 dedicated physical cores. 1 receive queue per
841    interface. Main thread pinned to core 1.
842
843.. topic:: 4T4C
844
845   *Dynamic tag*.
846    4 worker threads pinned to 4 dedicated physical cores. 2 receive queues per
847    interface. Main thread pinned to core 1.
848
849.. topic:: 2T1C
850
851   *Dynamic tag*.
852    2 worker threads each pinned to a separate logical core within 1 dedicated
853    physical core. 1 receive queue per interface. Main thread pinned to core 1.
854
855.. topic:: 4T2C
856
857   *Dynamic tag*.
858    4 worker threads each pinned to a separate logical core within 2 dedicated
859    physical cores. 2 receive queues per interface. Main thread pinned to core
860    1.
861
862.. topic:: 8T4C
863
864   *Dynamic tag*.
865    8 worker threads each pinned to a separate logical core within 4 dedicated
866    physical cores. 4 receive queues per interface. Main thread pinned to core
867    1.
868
869Honeycomb Tags
870--------------
871
872.. topic:: HC_FUNC
873
874    Honeycomb functional test cases.
875
876.. topic:: HC_NSH
877
878    Honeycomb NSH test cases.
879
880.. topic:: HC_PERSIST
881
882    Honeycomb persistence test cases.
883
884.. topic:: HC_REST_ONLY
885
886    (Exclusion tag) Honeycomb test cases that cannot be run in Netconf mode
887    using ODL client for Restfconf -> Netconf translation.
888