documentation.rst revision f7a9461e
1.. _doc_guidelines:
2
3DPDK Documentation Guidelines
4=============================
5
6This document outlines the guidelines for writing the DPDK Guides and API documentation in RST and Doxygen format.
7
8It also explains the structure of the DPDK documentation and shows how to build the Html and PDF versions of the documents.
9
10
11Structure of the Documentation
12------------------------------
13
14The DPDK source code repository contains input files to build the API documentation and User Guides.
15
16The main directories that contain files related to documentation are shown below::
17
18   lib
19   |-- librte_acl
20   |-- librte_cfgfile
21   |-- librte_cmdline
22   |-- librte_compat
23   |-- librte_eal
24   |   |-- ...
25   ...
26   doc
27   |-- api
28   +-- guides
29       |-- freebsd_gsg
30       |-- linux_gsg
31       |-- prog_guide
32       |-- sample_app_ug
33       |-- guidelines
34       |-- testpmd_app_ug
35       |-- rel_notes
36       |-- nics
37       |-- xen
38       |-- ...
39
40
41The API documentation is built from `Doxygen <http://www.stack.nl/~dimitri/doxygen/>`_ comments in the header files.
42These files are mainly in the ``lib/librte_*`` directories although some of the Poll Mode Drivers in ``drivers/net``
43are also documented with Doxygen.
44
45The configuration files that are used to control the Doxygen output are in the ``doc/api`` directory.
46
47The user guides such as *The Programmers Guide* and the *FreeBSD* and *Linux Getting Started* Guides are generated
48from RST markup text files using the `Sphinx <http://sphinx-doc.org/index.html>`_ Documentation Generator.
49
50These files are included in the ``doc/guides/`` directory.
51The output is controlled by the ``doc/guides/conf.py`` file.
52
53
54Role of the Documentation
55-------------------------
56
57The following items outline the roles of the different parts of the documentation and when they need to be updated or
58added to by the developer.
59
60* **Release Notes**
61
62  The Release Notes document which features have been added in the current and previous releases of DPDK and highlight
63  any known issues.
64  The Releases Notes also contain notifications of features that will change ABI compatibility in the next major release.
65
66  Developers should include updates to the Release Notes with patch sets that relate to any of the following sections:
67
68  * New Features
69  * Resolved Issues (see below)
70  * Known Issues
71  * API Changes
72  * ABI Changes
73  * Shared Library Versions
74
75  Resolved Issues should only include issues from previous releases that have been resolved in the current release.
76  Issues that are introduced and then fixed within a release cycle do not have to be included here.
77
78  Refer to the Release Notes from the previous DPDK release for the correct format of each section.
79
80
81* **API documentation**
82
83  The API documentation explains how to use the public DPDK functions.
84  The `API index page <http://doc.dpdk.org/api/>`_ shows the generated API documentation with related groups of functions.
85
86  The API documentation should be updated via Doxygen comments when new functions are added.
87
88* **Getting Started Guides**
89
90  The Getting Started Guides show how to install and configure DPDK and how to run DPDK based applications on different OSes.
91
92  A Getting Started Guide should be added when DPDK is ported to a new OS.
93
94* **The Programmers Guide**
95
96  The Programmers Guide explains how the API components of DPDK such as the EAL, Memzone, Rings and the Hash Library work.
97  It also explains how some higher level functionality such as Packet Distributor, Packet Framework and KNI work.
98  It also shows the build system and explains how to add applications.
99
100  The Programmers Guide should be expanded when new functionality is added to DPDK.
101
102* **App Guides**
103
104  The app guides document the DPDK applications in the ``app`` directory such as ``testpmd``.
105
106  The app guides should be updated if functionality is changed or added.
107
108* **Sample App Guides**
109
110  The sample app guides document the DPDK example applications in the examples directory.
111  Generally they demonstrate a major feature such as L2 or L3 Forwarding, Multi Process or Power Management.
112  They explain the purpose of the sample application, how to run it and step through some of the code to explain the
113  major functionality.
114
115  A new sample application should be accompanied by a new sample app guide.
116  The guide for the Skeleton Forwarding app is a good starting reference.
117
118* **Network Interface Controller Drivers**
119
120  The NIC Drivers document explains the features of the individual Poll Mode Drivers, such as software requirements,
121  configuration and initialization.
122
123  New documentation should be added for new Poll Mode Drivers.
124
125* **Guidelines**
126
127  The guideline documents record community process, expectations and design directions.
128
129  They can be extended, amended or discussed by submitting a patch and getting community approval.
130
131
132Building the Documentation
133--------------------------
134
135Dependencies
136~~~~~~~~~~~~
137
138
139The following dependencies must be installed to build the documentation:
140
141* Doxygen.
142
143* Sphinx (also called python-sphinx).
144
145* TexLive (at least TexLive-core and the extra Latex support).
146
147* Inkscape.
148
149`Doxygen`_ generates documentation from commented source code.
150It can be installed as follows:
151
152.. code-block:: console
153
154   # Ubuntu/Debian.
155   sudo apt-get -y install doxygen
156
157   # Red Hat/Fedora.
158   sudo dnf     -y install doxygen
159
160`Sphinx`_ is a Python documentation tool for converting RST files to Html or to PDF (via LaTeX).
161For full support with figure and table captioning the latest version of Sphinx can be installed as follows:
162
163.. code-block:: console
164
165   # Ubuntu/Debian.
166   sudo apt-get -y install python-pip
167   sudo pip install --upgrade sphinx
168   sudo pip install --upgrade sphinx_rtd_theme
169
170   # Red Hat/Fedora.
171   sudo dnf     -y install python-pip
172   sudo pip install --upgrade sphinx
173   sudo pip install --upgrade sphinx_rtd_theme
174
175For further information on getting started with Sphinx see the `Sphinx Tutorial <http://sphinx-doc.org/tutorial.html>`_.
176
177.. Note::
178
179   To get full support for Figure and Table numbering it is best to install Sphinx 1.3.1 or later.
180
181
182`Inkscape`_ is a vector based graphics program which is used to create SVG images and also to convert SVG images to PDF images.
183It can be installed as follows:
184
185.. code-block:: console
186
187   # Ubuntu/Debian.
188   sudo apt-get -y install inkscape
189
190   # Red Hat/Fedora.
191   sudo dnf     -y install inkscape
192
193`TexLive <http://www.tug.org/texlive/>`_ is an installation package for Tex/LaTeX.
194It is used to generate the PDF versions of the documentation.
195The main required packages can be installed as follows:
196
197.. code-block:: console
198
199   # Ubuntu/Debian.
200   sudo apt-get -y install texlive-latex-extra
201
202   # Red Hat/Fedora, selective install.
203   sudo dnf     -y install texlive-collection-latexextra
204
205
206Build commands
207~~~~~~~~~~~~~~
208
209The documentation is built using the standard DPDK build system.
210Some examples are shown below:
211
212* Generate all the documentation targets::
213
214     make doc
215
216* Generate the Doxygen API documentation in Html::
217
218     make doc-api-html
219
220* Generate the guides documentation in Html::
221
222     make doc-guides-html
223
224* Generate the guides documentation in Pdf::
225
226     make doc-guides-pdf
227
228The output of these commands is generated in the ``build`` directory::
229
230   build/doc
231         |-- html
232         |   |-- api
233         |   +-- guides
234         |
235         +-- pdf
236             +-- guides
237
238
239.. Note::
240
241   Make sure to fix any Sphinx or Doxygen warnings when adding or updating documentation.
242
243The documentation output files can be removed as follows::
244
245   make doc-clean
246
247
248Document Guidelines
249-------------------
250
251Here are some guidelines in relation to the style of the documentation:
252
253* Document the obvious as well as the obscure since it won't always be obvious to the reader.
254  For example an instruction like "Set up 64 2MB Hugepages" is better when followed by a sample commandline or a link to
255  the appropriate section of the documentation.
256
257* Use American English spellings throughout.
258  This can be checked using the ``aspell`` utility::
259
260       aspell --lang=en_US --check doc/guides/sample_app_ug/mydoc.rst
261
262
263RST Guidelines
264--------------
265
266The RST (reStructuredText) format is a plain text markup format that can be converted to Html, PDF or other formats.
267It is most closely associated with Python but it can be used to document any language.
268It is used in DPDK to document everything apart from the API.
269
270The Sphinx documentation contains a very useful `RST Primer <http://sphinx-doc.org/rest.html#rst-primer>`_ which is a
271good place to learn the minimal set of syntax required to format a document.
272
273The official `reStructuredText <http://docutils.sourceforge.net/rst.html>`_ website contains the specification for the
274RST format and also examples of how to use it.
275However, for most developers the RST Primer is a better resource.
276
277The most common guidelines for writing RST text are detailed in the
278`Documenting Python <https://docs.python.org/devguide/documenting.html>`_ guidelines.
279The additional guidelines below reiterate or expand upon those guidelines.
280
281
282Line Length
283~~~~~~~~~~~
284
285* The recommended style for the DPDK documentation is to put sentences on separate lines.
286  This allows for easier reviewing of patches.
287  Multiple sentences which are not separated by a blank line are joined automatically into paragraphs, for example::
288
289     Here is an example sentence.
290     Long sentences over the limit shown below can be wrapped onto
291     a new line.
292     These three sentences will be joined into the same paragraph.
293
294     This is a new paragraph, since it is separated from the
295     previous paragraph by a blank line.
296
297  This would be rendered as follows:
298
299     *Here is an example sentence.
300     Long sentences over the limit shown below can be wrapped onto
301     a new line.
302     These three sentences will be joined into the same paragraph.*
303
304     *This is a new paragraph, since it is separated from the
305     previous paragraph by a blank line.*
306
307
308* Long sentences should be wrapped at 120 characters +/- 10 characters. They should be wrapped at words.
309
310* Lines in literal blocks must by less than 80 characters since they aren't wrapped by the document formatters
311  and can exceed the page width in PDF documents.
312
313
314Whitespace
315~~~~~~~~~~
316
317* Standard RST indentation is 3 spaces.
318  Code can be indented 4 spaces, especially if it is copied from source files.
319
320* No tabs.
321  Convert tabs in embedded code to 4 or 8 spaces.
322
323* No trailing whitespace.
324
325* Add 2 blank lines before each section header.
326
327* Add 1 blank line after each section header.
328
329* Add 1 blank line between each line of a list.
330
331
332Section Headers
333~~~~~~~~~~~~~~~
334
335* Section headers should use the use the following underline formats::
336
337   Level 1 Heading
338   ===============
339
340
341   Level 2 Heading
342   ---------------
343
344
345   Level 3 Heading
346   ~~~~~~~~~~~~~~~
347
348
349   Level 4 Heading
350   ^^^^^^^^^^^^^^^
351
352
353* Level 4 headings should be used sparingly.
354
355* The underlines should match the length of the text.
356
357* In general, the heading should be less than 80 characters, for conciseness.
358
359* As noted above:
360
361   * Add 2 blank lines before each section header.
362
363   * Add 1 blank line after each section header.
364
365
366Lists
367~~~~~
368
369* Bullet lists should be formatted with a leading ``*`` as follows::
370
371     * Item one.
372
373     * Item two is a long line that is wrapped and then indented to match
374       the start of the previous line.
375
376     * One space character between the bullet and the text is preferred.
377
378* Numbered lists can be formatted with a leading number but the preference is to use ``#.`` which will give automatic numbering.
379  This is more convenient when adding or removing items::
380
381     #. Item one.
382
383     #. Item two is a long line that is wrapped and then indented
384        to match the start of the e first line.
385
386     #. Item two is a long line that is wrapped and then indented to match
387        the start of the previous line.
388
389* Definition lists can be written with or without a bullet::
390
391     * Item one.
392
393       Some text about item one.
394
395     * Item two.
396
397       Some text about item two.
398
399* All lists, and sub-lists, must be separated from the preceding text by a blank line.
400  This is a syntax requirement.
401
402* All list items should be separated by a blank line for readability.
403
404
405Code and Literal block sections
406~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
407
408* Inline text that is required to be rendered with a fixed width font should be enclosed in backquotes like this:
409  \`\`text\`\`, so that it appears like this: ``text``.
410
411* Fixed width, literal blocks of texts should be indented at least 3 spaces and prefixed with ``::`` like this::
412
413     Here is some fixed width text::
414
415        0x0001 0x0001 0x00FF 0x00FF
416
417* It is also possible to specify an encoding for a literal block using the ``.. code-block::`` directive so that syntax
418  highlighting can be applied.
419  Examples of supported highlighting are::
420
421     .. code-block:: console
422     .. code-block:: c
423     .. code-block:: python
424     .. code-block:: diff
425     .. code-block:: none
426
427  That can be applied as follows::
428
429      .. code-block:: c
430
431         #include<stdio.h>
432
433         int main() {
434
435            printf("Hello World\n");
436
437            return 0;
438         }
439
440  Which would be rendered as:
441
442  .. code-block:: c
443
444      #include<stdio.h>
445
446      int main() {
447
448         printf("Hello World\n");
449
450         return 0;
451      }
452
453
454* The default encoding for a literal block using the simplified ``::``
455  directive is ``none``.
456
457* Lines in literal blocks must be less than 80 characters since they can exceed the page width when converted to PDF documentation.
458  For long literal lines that exceed that limit try to wrap the text at sensible locations.
459  For example a long command line could be documented like this and still work if copied directly from the docs::
460
461     build/app/testpmd -c7 -n3 --vdev=net_pcap0,iface=eth0     \
462                               --vdev=net_pcap1,iface=eth1     \
463                               -- -i --nb-cores=2 --nb-ports=2 \
464                                  --total-num-mbufs=2048
465
466* Long lines that cannot be wrapped, such as application output, should be truncated to be less than 80 characters.
467
468
469Images
470~~~~~~
471
472* All images should be in SVG scalar graphics format.
473  They should be true SVG XML files and should not include binary formats embedded in a SVG wrapper.
474
475* The DPDK documentation contains some legacy images in PNG format.
476  These will be converted to SVG in time.
477
478* `Inkscape <http://inkscape.org>`_ is the recommended graphics editor for creating the images.
479  Use some of the older images in ``doc/guides/prog_guide/img/`` as a template, for example ``mbuf1.svg``
480  or ``ring-enqueue.svg``.
481
482* The SVG images should include a copyright notice, as an XML comment.
483
484* Images in the documentation should be formatted as follows:
485
486   * The image should be preceded by a label in the format ``.. _figure_XXXX:`` with a leading underscore and
487     where ``XXXX`` is a unique descriptive name.
488
489   * Images should be included using the ``.. figure::`` directive and the file type should be set to ``*`` (not ``.svg``).
490     This allows the format of the image to be changed if required, without updating the documentation.
491
492   * Images must have a caption as part of the ``.. figure::`` directive.
493
494* Here is an example of the previous three guidelines::
495
496     .. _figure_mempool:
497
498     .. figure:: img/mempool.*
499
500        A mempool in memory with its associated ring.
501
502.. _mock_label:
503
504* Images can then be linked to using the ``:numref:`` directive::
505
506     The mempool layout is shown in :numref:`figure_mempool`.
507
508  This would be rendered as: *The mempool layout is shown in* :ref:`Fig 6.3 <mock_label>`.
509
510  **Note**: The ``:numref:`` directive requires Sphinx 1.3.1 or later.
511  With earlier versions it will still be rendered as a link but won't have an automatically generated number.
512
513* The caption of the image can be generated, with a link, using the ``:ref:`` directive::
514
515     :ref:`figure_mempool`
516
517  This would be rendered as: *A mempool in memory with its associated ring.*
518
519Tables
520~~~~~~
521
522* RST tables should be used sparingly.
523  They are hard to format and to edit, they are often rendered incorrectly in PDF format, and the same information
524  can usually be shown just as clearly with a definition or bullet list.
525
526* Tables in the documentation should be formatted as follows:
527
528   * The table should be preceded by a label in the format ``.. _table_XXXX:`` with a leading underscore and where
529     ``XXXX`` is a unique descriptive name.
530
531   * Tables should be included using the ``.. table::`` directive and must have a caption.
532
533* Here is an example of the previous two guidelines::
534
535     .. _table_qos_pipes:
536
537     .. table:: Sample configuration for QOS pipes.
538
539        +----------+----------+----------+
540        | Header 1 | Header 2 | Header 3 |
541        |          |          |          |
542        +==========+==========+==========+
543        | Text     | Text     | Text     |
544        +----------+----------+----------+
545        | ...      | ...      | ...      |
546        +----------+----------+----------+
547
548* Tables can be linked to using the ``:numref:`` and ``:ref:`` directives, as shown in the previous section for images.
549  For example::
550
551     The QOS configuration is shown in :numref:`table_qos_pipes`.
552
553* Tables should not include merged cells since they are not supported by the PDF renderer.
554
555
556.. _links:
557
558Hyperlinks
559~~~~~~~~~~
560
561* Links to external websites can be plain URLs.
562  The following is rendered as http://dpdk.org::
563
564     http://dpdk.org
565
566* They can contain alternative text.
567  The following is rendered as `Check out DPDK <http://dpdk.org>`_::
568
569     `Check out DPDK <http://dpdk.org>`_
570
571* An internal link can be generated by placing labels in the document with the format ``.. _label_name``.
572
573* The following links to the top of this section: :ref:`links`::
574
575     .. _links:
576
577     Hyperlinks
578     ~~~~~~~~~~
579
580     * The following links to the top of this section: :ref:`links`:
581
582.. Note::
583
584   The label must have a leading underscore but the reference to it must omit it.
585   This is a frequent cause of errors and warnings.
586
587* The use of a label is preferred since it works across files and will still work if the header text changes.
588
589
590.. _doxygen_guidelines:
591
592Doxygen Guidelines
593------------------
594
595The DPDK API is documented using Doxygen comment annotations in the header files.
596Doxygen is a very powerful tool, it is extremely configurable and with a little effort can be used to create expressive documents.
597See the `Doxygen website <http://www.stack.nl/~dimitri/doxygen/>`_ for full details on how to use it.
598
599The following are some guidelines for use of Doxygen in the DPDK API documentation:
600
601* New libraries that are documented with Doxygen should be added to the Doxygen configuration file: ``doc/api/doxy-api.conf``.
602  It is only required to add the directory that contains the files.
603  It isn't necessary to explicitly name each file since the configuration matches all ``rte_*.h`` files in the directory.
604
605* Use proper capitalization and punctuation in the Doxygen comments since they will become sentences in the documentation.
606  This in particular applies to single line comments, which is the case the is most often forgotten.
607
608* Use ``@`` style Doxygen commands instead of ``\`` style commands.
609
610* Add a general description of each library at the head of the main header files:
611
612  .. code-block:: c
613
614      /**
615       * @file
616       * RTE Mempool.
617       *
618       * A memory pool is an allocator of fixed-size object. It is
619       * identified by its name, and uses a ring to store free objects.
620       * ...
621       */
622
623* Document the purpose of a function, the parameters used and the return
624  value:
625
626  .. code-block:: c
627
628     /**
629      * Attach a new Ethernet device specified by arguments.
630      *
631      * @param devargs
632      *  A pointer to a strings array describing the new device
633      *  to be attached. The strings should be a pci address like
634      *  `0000:01:00.0` or **virtual** device name like `net_pcap0`.
635      * @param port_id
636      *  A pointer to a port identifier actually attached.
637      *
638      * @return
639      *  0 on success and port_id is filled, negative on error.
640      */
641     int rte_eth_dev_attach(const char *devargs, uint8_t *port_id);
642
643* Doxygen supports Markdown style syntax such as bold, italics, fixed width text and lists.
644  For example the second line in the ``devargs`` parameter in the previous example will be rendered as:
645
646     The strings should be a pci address like ``0000:01:00.0`` or **virtual** device name like ``net_pcap0``.
647
648* Use ``-`` instead of ``*`` for lists within the Doxygen comment since the latter can get confused with the comment delimiter.
649
650* Add an empty line between the function description, the ``@params`` and ``@return`` for readability.
651
652* Place the ``@params`` description on separate line and indent it by 2 spaces.
653  (It would be better to use no indentation since this is more common and also because checkpatch complains about leading
654  whitespace in comments.
655  However this is the convention used in the existing DPDK code.)
656
657* Documented functions can be linked to simply by adding ``()`` to the function name:
658
659  .. code-block:: c
660
661      /**
662       * The functions exported by the application Ethernet API to setup
663       * a device designated by its port identifier must be invoked in
664       * the following order:
665       *     - rte_eth_dev_configure()
666       *     - rte_eth_tx_queue_setup()
667       *     - rte_eth_rx_queue_setup()
668       *     - rte_eth_dev_start()
669       */
670
671  In the API documentation the functions will be rendered as links, see the
672  `online section of the rte_ethdev.h docs <http://doc.dpdk.org/api/rte__ethdev_8h.html>`_ that contains the above text.
673
674* The ``@see`` keyword can be used to create a *see also* link to another file or library.
675  This directive should be placed on one line at the bottom of the documentation section.
676
677  .. code-block:: c
678
679     /**
680      * ...
681      *
682      * Some text that references mempools.
683      *
684      * @see eal_memzone.c
685      */
686
687* Doxygen supports two types of comments for documenting variables, constants and members: prefix and postfix:
688
689  .. code-block:: c
690
691     /** This is a prefix comment. */
692     #define RTE_FOO_ERROR  0x023.
693
694     #define RTE_BAR_ERROR  0x024. /**< This is a postfix comment. */
695
696* Postfix comments are preferred for struct members and constants if they can be documented in the same way:
697
698  .. code-block:: c
699
700     struct rte_eth_stats {
701         uint64_t ipackets; /**< Total number of received packets. */
702         uint64_t opackets; /**< Total number of transmitted packets.*/
703         uint64_t ibytes;   /**< Total number of received bytes. */
704         uint64_t obytes;   /**< Total number of transmitted bytes. */
705         uint64_t imissed;  /**< Total of RX missed packets. */
706         uint64_t ibadcrc;  /**< Total of RX packets with CRC error. */
707         uint64_t ibadlen;  /**< Total of RX packets with bad length. */
708     }
709
710  Note: postfix comments should be aligned with spaces not tabs in accordance
711  with the :ref:`coding_style`.
712
713* If a single comment type can't be used, due to line length limitations then
714  prefix comments should be preferred.
715  For example this section of the code contains prefix comments, postfix comments on the same line and postfix
716  comments on a separate line:
717
718  .. code-block:: c
719
720     /** Number of elements in the elt_pa array. */
721     uint32_t    pg_num __rte_cache_aligned;
722     uint32_t    pg_shift;     /**< LOG2 of the physical pages. */
723     uintptr_t   pg_mask;      /**< Physical page mask value. */
724     uintptr_t   elt_va_start;
725     /**< Virtual address of the first mempool object. */
726     uintptr_t   elt_va_end;
727     /**< Virtual address of the <size + 1> mempool object. */
728     phys_addr_t elt_pa[MEMPOOL_PG_NUM_DEFAULT];
729     /**< Array of physical page addresses for the mempool buffer. */
730
731  This doesn't have an effect on the rendered documentation but it is confusing for the developer reading the code.
732  It this case it would be clearer to use prefix comments throughout:
733
734  .. code-block:: c
735
736     /** Number of elements in the elt_pa array. */
737     uint32_t    pg_num __rte_cache_aligned;
738     /** LOG2 of the physical pages. */
739     uint32_t    pg_shift;
740     /** Physical page mask value. */
741     uintptr_t   pg_mask;
742     /** Virtual address of the first mempool object. */
743     uintptr_t   elt_va_start;
744     /** Virtual address of the <size + 1> mempool object. */
745     uintptr_t   elt_va_end;
746     /** Array of physical page addresses for the mempool buffer. */
747     phys_addr_t elt_pa[MEMPOOL_PG_NUM_DEFAULT];
748
749* Check for Doxygen warnings in new code by checking the API documentation build::
750
751     make doc-api-html >/dev/null
752
753* Read the rendered section of the documentation that you have added for correctness, clarity and consistency
754  with the surrounding text.
755