thread_safety_dpdk_functions.rst revision 97f17497
1..  BSD LICENSE
2    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31Thread Safety of DPDK Functions
32===============================
33
34The DPDK is comprised of several libraries.
35Some of the functions in these libraries can be safely called from multiple threads simultaneously, while others cannot.
36This section allows the developer to take these issues into account when building their own application.
37
38The run-time environment of the DPDK is typically a single thread per logical core.
39In some cases, it is not only multi-threaded, but multi-process.
40Typically, it is best to avoid sharing data structures between threads and/or processes where possible.
41Where this is not possible, then the execution blocks must access the data in a thread- safe manner.
42Mechanisms such as atomics or locking can be used that will allow execution blocks to operate serially.
43However, this can have an effect on the performance of the application.
44
45Fast-Path APIs
46--------------
47
48Applications operating in the data plane are performance sensitive but
49certain functions within those libraries may not be safe to call from multiple threads simultaneously.
50The hash, LPM and mempool libraries and RX/TX in the PMD are examples of this.
51
52The hash and LPM libraries are, by design, thread unsafe in order to maintain performance.
53However, if required the developer can add layers on top of these libraries to provide thread safety.
54Locking is not needed in all situations, and in both the hash and LPM libraries,
55lookups of values can be performed in parallel in multiple threads.
56Adding, removing or modifying values, however,
57cannot be done in multiple threads without using locking when a single hash or LPM table is accessed.
58Another alternative to locking would be to create multiple instances of these tables allowing each thread its own copy.
59
60The RX and TX of the PMD are the most critical aspects of a DPDK application
61and it is recommended that no locking be used as it will impact performance.
62Note, however, that these functions can safely be used from multiple threads
63when each thread is performing I/O on a different NIC queue.
64If multiple threads are to use the same hardware queue on the same NIC port,
65then locking, or some other form of mutual exclusion, is necessary.
66
67The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety.
68Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations.
69The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe.
70
71Performance Insensitive API
72---------------------------
73
74Outside of the performance sensitive areas described in Section 25.1,
75the DPDK provides a thread-safe API for most other libraries.
76For example, malloc and memzone functions are safe for use in multi-threaded and multi-process environments.
77
78The setup and configuration of the PMD is not performance sensitive, but is not thread safe either.
79It is possible that the multiple read/writes during PMD setup and configuration could be corrupted in a multi-thread environment.
80Since this is not performance sensitive, the developer can choose to add their own layer to provide thread-safe setup and configuration.
81It is expected that, in most applications, the initial configuration of the network ports would be done by a single thread at startup.
82
83Library Initialization
84----------------------
85
86It is recommended that DPDK libraries are initialized in the main thread at application startup
87rather than subsequently in the forwarding threads.
88However, the DPDK performs checks to ensure that libraries are only initialized once.
89If initialization is attempted more than once, an error is returned.
90
91In the multi-process case, the configuration information of shared memory will only be initialized by the master process.
92Thereafter, both master and secondary processes can allocate/release any objects of memory that finally rely on rte_malloc or memzones.
93
94Interrupt Thread
95----------------
96
97The DPDK works almost entirely in Linux user space in polling mode.
98For certain infrequent operations, such as receiving a PMD link status change notification,
99callbacks may be called in an additional thread outside the main DPDK processing threads.
100These function callbacks should avoid manipulating DPDK objects that are also managed by the normal DPDK threads,
101and if they need to do so,
102it is up to the application to provide the appropriate locking or mutual exclusion restrictions around those objects.
103