ring_lib.rst revision 5d4e5dcd
1..  BSD LICENSE
2    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31.. _Ring_Library:
32
33Ring Library
34============
35
36The ring allows the management of queues.
37Instead of having a linked list of infinite size, the rte_ring has the following properties:
38
39*   FIFO
40
41*   Maximum size is fixed, the pointers are stored in a table
42
43*   Lockless implementation
44
45*   Multi-consumer or single-consumer dequeue
46
47*   Multi-producer or single-producer enqueue
48
49*   Bulk dequeue - Dequeues the specified count of objects if successful; otherwise fails
50
51*   Bulk enqueue - Enqueues the specified count of objects if successful; otherwise fails
52
53*   Burst dequeue - Dequeue the maximum available objects if the specified count cannot be fulfilled
54
55*   Burst enqueue - Enqueue the maximum available objects if the specified count cannot be fulfilled
56
57The advantages of this data structure over a linked list queue are as follows:
58
59*   Faster; only requires a single Compare-And-Swap instruction of sizeof(void \*) instead of several double-Compare-And-Swap instructions.
60
61*   Simpler than a full lockless queue.
62
63*   Adapted to bulk enqueue/dequeue operations.
64    As pointers are stored in a table, a dequeue of several objects will not produce as many cache misses as in a linked queue.
65    Also, a bulk dequeue of many objects does not cost more than a dequeue of a simple object.
66
67The disadvantages:
68
69*   Size is fixed
70
71*   Having many rings costs more in terms of memory than a linked list queue. An empty ring contains at least N pointers.
72
73A simplified representation of a Ring is shown in with consumer and producer head and tail pointers to objects stored in the data structure.
74
75.. _figure_ring1:
76
77.. figure:: img/ring1.*
78
79   Ring Structure
80
81
82References for Ring Implementation in FreeBSD*
83----------------------------------------------
84
85The following code was added in FreeBSD 8.0, and is used in some network device drivers (at least in Intel drivers):
86
87    * `bufring.h in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/sys/buf_ring.h?revision=199625&view=markup>`_
88
89    * `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_
90
91Lockless Ring Buffer in Linux*
92------------------------------
93
94The following is a link describing the `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_.
95
96Additional Features
97-------------------
98
99Name
100~~~~
101
102A ring is identified by a unique name.
103It is not possible to create two rings with the same name (rte_ring_create() returns NULL if this is attempted).
104
105Water Marking
106~~~~~~~~~~~~~
107
108The ring can have a high water mark (threshold).
109Once an enqueue operation reaches the high water mark, the producer is notified, if the water mark is configured.
110
111This mechanism can be used, for example, to exert a back pressure on I/O to inform the LAN to PAUSE.
112
113Debug
114~~~~~
115
116When debug is enabled (CONFIG_RTE_LIBRTE_RING_DEBUG is set),
117the library stores some per-ring statistic counters about the number of enqueues/dequeues.
118These statistics are per-core to avoid concurrent accesses or atomic operations.
119
120Use Cases
121---------
122
123Use cases for the Ring library include:
124
125    *  Communication between applications in the DPDK
126
127    *  Used by memory pool allocator
128
129Anatomy of a Ring Buffer
130------------------------
131
132This section explains how a ring buffer operates.
133The ring structure is composed of two head and tail couples; one is used by producers and one is used by the consumers.
134The figures of the following sections refer to them as prod_head, prod_tail, cons_head and cons_tail.
135
136Each figure represents a simplified state of the ring, which is a circular buffer.
137The content of the function local variables is represented on the top of the figure,
138and the content of ring structure is represented on the bottom of the figure.
139
140Single Producer Enqueue
141~~~~~~~~~~~~~~~~~~~~~~~
142
143This section explains what occurs when a producer adds an object to the ring.
144In this example, only the producer head and tail (prod_head and prod_tail) are modified,
145and there is only one producer.
146
147The initial state is to have a prod_head and prod_tail pointing at the same location.
148
149Enqueue First Step
150^^^^^^^^^^^^^^^^^^
151
152First, *ring->prod_head* and ring->cons_tail are copied in local variables.
153The prod_next local variable points to the next element of the table, or several elements after in case of bulk enqueue.
154
155If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error.
156
157
158.. _figure_ring-enqueue1:
159
160.. figure:: img/ring-enqueue1.*
161
162   Enqueue first step
163
164
165Enqueue Second Step
166^^^^^^^^^^^^^^^^^^^
167
168The second step is to modify *ring->prod_head* in ring structure to point to the same location as prod_next.
169
170A pointer to the added object is copied in the ring (obj4).
171
172
173.. _figure_ring-enqueue2:
174
175.. figure:: img/ring-enqueue2.*
176
177   Enqueue second step
178
179
180Enqueue Last Step
181^^^^^^^^^^^^^^^^^
182
183Once the object is added in the ring, ring->prod_tail in the ring structure is modified to point to the same location as *ring->prod_head*.
184The enqueue operation is finished.
185
186
187.. _figure_ring-enqueue3:
188
189.. figure:: img/ring-enqueue3.*
190
191   Enqueue last step
192
193
194Single Consumer Dequeue
195~~~~~~~~~~~~~~~~~~~~~~~
196
197This section explains what occurs when a consumer dequeues an object from the ring.
198In this example, only the consumer head and tail (cons_head and cons_tail) are modified and there is only one consumer.
199
200The initial state is to have a cons_head and cons_tail pointing at the same location.
201
202Dequeue First Step
203^^^^^^^^^^^^^^^^^^
204
205First, ring->cons_head and ring->prod_tail are copied in local variables.
206The cons_next local variable points to the next element of the table, or several elements after in the case of bulk dequeue.
207
208If there are not enough objects in the ring (this is detected by checking prod_tail), it returns an error.
209
210
211.. _figure_ring-dequeue1:
212
213.. figure:: img/ring-dequeue1.*
214
215   Dequeue last step
216
217
218Dequeue Second Step
219^^^^^^^^^^^^^^^^^^^
220
221The second step is to modify ring->cons_head in the ring structure to point to the same location as cons_next.
222
223The pointer to the dequeued object (obj1) is copied in the pointer given by the user.
224
225
226.. _figure_ring-dequeue2:
227
228.. figure:: img/ring-dequeue2.*
229
230   Dequeue second step
231
232
233Dequeue Last Step
234^^^^^^^^^^^^^^^^^
235
236Finally, ring->cons_tail in the ring structure is modified to point to the same location as ring->cons_head.
237The dequeue operation is finished.
238
239
240.. _figure_ring-dequeue3:
241
242.. figure:: img/ring-dequeue3.*
243
244   Dequeue last step
245
246
247Multiple Producers Enqueue
248~~~~~~~~~~~~~~~~~~~~~~~~~~
249
250This section explains what occurs when two producers concurrently add an object to the ring.
251In this example, only the producer head and tail (prod_head and prod_tail) are modified.
252
253The initial state is to have a prod_head and prod_tail pointing at the same location.
254
255Multiple Producers Enqueue First Step
256^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
257
258On both cores, *ring->prod_head* and ring->cons_tail are copied in local variables.
259The prod_next local variable points to the next element of the table,
260or several elements after in the case of bulk enqueue.
261
262If there is not enough room in the ring (this is detected by checking cons_tail), it returns an error.
263
264
265.. _figure_ring-mp-enqueue1:
266
267.. figure:: img/ring-mp-enqueue1.*
268
269   Multiple producer enqueue first step
270
271
272Multiple Producers Enqueue Second Step
273^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
274
275The second step is to modify ring->prod_head in the ring structure to point to the same location as prod_next.
276This operation is done using a Compare And Swap (CAS) instruction, which does the following operations atomically:
277
278*   If ring->prod_head is different to local variable prod_head,
279    the CAS operation fails, and the code restarts at first step.
280
281*   Otherwise, ring->prod_head is set to local prod_next,
282    the CAS operation is successful, and processing continues.
283
284In the figure, the operation succeeded on core 1, and step one restarted on core 2.
285
286
287.. _figure_ring-mp-enqueue2:
288
289.. figure:: img/ring-mp-enqueue2.*
290
291   Multiple producer enqueue second step
292
293
294Multiple Producers Enqueue Third Step
295^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
296
297The CAS operation is retried on core 2 with success.
298
299The core 1 updates one element of the ring(obj4), and the core 2 updates another one (obj5).
300
301
302.. _figure_ring-mp-enqueue3:
303
304.. figure:: img/ring-mp-enqueue3.*
305
306   Multiple producer enqueue third step
307
308
309Multiple Producers Enqueue Fourth Step
310^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
311
312Each core now wants to update ring->prod_tail.
313A core can only update it if ring->prod_tail is equal to the prod_head local variable.
314This is only true on core 1. The operation is finished on core 1.
315
316
317.. _figure_ring-mp-enqueue4:
318
319.. figure:: img/ring-mp-enqueue4.*
320
321   Multiple producer enqueue fourth step
322
323
324Multiple Producers Enqueue Last Step
325^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
326
327Once ring->prod_tail is updated by core 1, core 2 is allowed to update it too.
328The operation is also finished on core 2.
329
330
331.. _figure_ring-mp-enqueue5:
332
333.. figure:: img/ring-mp-enqueue5.*
334
335   Multiple producer enqueue last step
336
337
338Modulo 32-bit Indexes
339~~~~~~~~~~~~~~~~~~~~~
340
341In the preceding figures, the prod_head, prod_tail, cons_head and cons_tail indexes are represented by arrows.
342In the actual implementation, these values are not between 0 and size(ring)-1 as would be assumed.
343The indexes are between 0 and 2^32 -1, and we mask their value when we access the pointer table (the ring itself).
34432-bit modulo also implies that operations on indexes (such as, add/subtract) will automatically do 2^32 modulo
345if the result overflows the 32-bit number range.
346
347The following are two examples that help to explain how indexes are used in a ring.
348
349.. note::
350
351    To simplify the explanation, operations with modulo 16-bit are used instead of modulo 32-bit.
352    In addition, the four indexes are defined as unsigned 16-bit integers,
353    as opposed to unsigned 32-bit integers in the more realistic case.
354
355
356.. _figure_ring-modulo1:
357
358.. figure:: img/ring-modulo1.*
359
360   Modulo 32-bit indexes - Example 1
361
362
363This ring contains 11000 entries.
364
365
366.. _figure_ring-modulo2:
367
368.. figure:: img/ring-modulo2.*
369
370      Modulo 32-bit indexes - Example 2
371
372
373This ring contains 12536 entries.
374
375.. note::
376
377    For ease of understanding, we use modulo 65536 operations in the above examples.
378    In real execution cases, this is redundant for low efficiency, but is done automatically when the result overflows.
379
380The code always maintains a distance between producer and consumer between 0 and size(ring)-1.
381Thanks to this property, we can do subtractions between 2 index values in a modulo-32bit base:
382that's why the overflow of the indexes is not a problem.
383
384At any time, entries and free_entries are between 0 and size(ring)-1,
385even if only the first term of subtraction has overflowed:
386
387.. code-block:: c
388
389    uint32_t entries = (prod_tail - cons_head);
390    uint32_t free_entries = (mask + cons_tail -prod_head);
391
392References
393----------
394
395    *   `bufring.h in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/sys/buf_ring.h?revision=199625&view=markup>`_ (version 8)
396
397    *   `bufring.c in FreeBSD <http://svn.freebsd.org/viewvc/base/release/8.0.0/sys/kern/subr_bufring.c?revision=199625&view=markup>`_ (version 8)
398
399    *   `Linux Lockless Ring Buffer Design <http://lwn.net/Articles/340400/>`_
400