dev_kit_build_system.rst revision a41e6ff1
1..  BSD LICENSE
2    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31.. _Development_Kit_Build_System:
32
33Development Kit Build System
34============================
35
36The DPDK requires a build system for compilation activities and so on.
37This section describes the constraints and the mechanisms used in the DPDK framework.
38
39There are two use-cases for the framework:
40
41*   Compilation of the DPDK libraries and sample applications;
42    the framework generates specific binary libraries,
43    include files and sample applications
44
45*   Compilation of an external application or library, using an installed binary DPDK
46
47Building the Development Kit Binary
48-----------------------------------
49
50The following provides details on how to build the DPDK binary.
51
52Build Directory Concept
53~~~~~~~~~~~~~~~~~~~~~~~
54
55After installation, a build directory structure is created.
56Each build directory contains include files, libraries, and applications:
57
58.. code-block:: console
59
60    ~/DPDK$ ls
61    app                               MAINTAINERS
62    config                            Makefile
63    COPYRIGHT                         mk
64    doc                               scripts
65    examples                          lib
66    tools                             x86_64-native-linuxapp-gcc
67    x86_64-native-linuxapp-icc        i686-native-linuxapp-gcc
68    i686-native-linuxapp-icc
69
70    ...
71    ~/DEV/DPDK$ ls i686-native-linuxapp-gcc
72
73    app build buildtools include kmod lib Makefile
74
75
76    ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/app/
77    cmdline_test   dump_cfg     test     testpmd
78    cmdline_test.map      dump_cfg.map   test.map
79	    testpmd.map
80
81
82    ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/lib/
83
84    libethdev.a  librte_hash.a  librte_mbuf.a librte_pmd_ixgbe.a
85
86    librte_cmdline.a librte_lpm.a librte_mempool.a librte_ring.a
87
88    librte_eal.a librte_pmd_e1000.a librte_timer.a
89
90
91    ~/DEV/DPDK$ ls i686-native-linuxapp-gcc/include/
92    arch                       rte_cpuflags.h       rte_memcpy.h
93    cmdline_cirbuf.h           rte_cycles.h         rte_memory.h
94    cmdline.h                  rte_debug.h          rte_mempool.h
95    cmdline_parse_etheraddr.h  rte_eal.h            rte_memzone.h
96    cmdline_parse.h            rte_errno.h          rte_pci_dev_ids.h
97    cmdline_parse_ipaddr.h     rte_ethdev.h         rte_pci.h
98    cmdline_parse_num.h        rte_ether.h          rte_per_lcore.h
99    cmdline_parse_portlist.h   rte_fbk_hash.h       rte_prefetch.h
100    cmdline_parse_string.h     rte_hash_crc.h       rte_random.h
101    cmdline_rdline.h           rte_hash.h           rte_ring.h
102    cmdline_socket.h           rte_interrupts.h     rte_rwlock.h
103    cmdline_vt100.h            rte_ip.h             rte_sctp.h
104    exec-env                   rte_jhash.h          rte_spinlock.h
105    rte_alarm.h                rte_launch.h         rte_string_fns.h
106    rte_atomic.h               rte_lcore.h          rte_tailq.h
107    rte_branch_prediction.h    rte_log.h            rte_tcp.h
108    rte_byteorder.h            rte_lpm.h            rte_timer.h
109    rte_common.h               rte_malloc.h         rte_udp.h
110    rte_config.h               rte_mbuf.h
111
112
113A build directory is specific to a configuration that includes architecture + execution environment + toolchain.
114It is possible to have several build directories sharing the same sources with different configurations.
115
116For instance, to create a new build directory called my_sdk_build_dir using the default configuration template config/defconfig_x86_64-linuxapp,
117we use:
118
119.. code-block:: console
120
121    cd ${RTE_SDK}
122    make config T=x86_64-native-linuxapp-gcc O=my_sdk_build_dir
123
124This creates a new my_sdk_build_dir directory. After that, we can compile by doing:
125
126.. code-block:: console
127
128    cd my_sdk_build_dir
129    make
130
131which is equivalent to:
132
133.. code-block:: console
134
135    make O=my_sdk_build_dir
136
137The content of the my_sdk_build_dir is then:
138
139::
140
141    -- .config                         # used configuration
142
143    -- Makefile                        # wrapper that calls head Makefile
144                                       # with $PWD as build directory
145
146
147        -- build                              #All temporary files used during build
148        +--app                                # process, including . o, .d, and .cmd files.
149            |  +-- test                       # For libraries, we have the .a file.
150            |  +-- test.o                     # For applications, we have the elf file.
151            |  `-- ...
152            +-- lib
153                +-- librte_eal
154                |   `-- ...
155                +-- librte_mempool
156                |  +--  mempool-file1.o
157                |  +--  .mempool-file1.o.cmd
158                |  +--  .mempool-file1.o.d
159                |  +--   mempool-file2.o
160                |  +--  .mempool-file2.o.cmd
161                |  +--  .mempool-file2.o.d
162                |  `--  mempool.a
163                `-- ...
164
165    -- include                # All include files installed by libraries
166        +-- librte_mempool.h  # and applications are located in this
167        +-- rte_eal.h         # directory. The installed files can depend
168        +-- rte_spinlock.h    # on configuration if needed (environment,
169        +-- rte_atomic.h      # architecture, ..)
170        `-- \*.h ...
171
172    -- lib                    # all compiled libraries are copied in this
173        +-- librte_eal.a      # directory
174        +-- librte_mempool.a
175        `-- \*.a ...
176
177    -- app                    # All compiled applications are installed
178    + --test                  # here. It includes the binary in elf format
179
180Refer to
181:ref:`Development Kit Root Makefile Help <Development_Kit_Root_Makefile_Help>`
182for details about make commands that can be used from the root of DPDK.
183
184Building External Applications
185------------------------------
186
187Since DPDK is in essence a development kit, the first objective of end users will be to create an application using this SDK.
188To compile an application, the user must set the RTE_SDK and RTE_TARGET environment variables.
189
190.. code-block:: console
191
192    export RTE_SDK=/opt/DPDK
193    export RTE_TARGET=x86_64-native-linuxapp-gcc
194    cd /path/to/my_app
195
196For a new application, the user must create their own Makefile that includes some .mk files, such as
197${RTE_SDK}/mk/rte.vars.mk, and ${RTE_SDK}/mk/ rte.app.mk.
198This is described in
199:ref:`Building Your Own Application <Building_Your_Own_Application>`.
200
201Depending on the chosen target (architecture, machine, executive environment, toolchain) defined in the Makefile or as an environment variable,
202the applications and libraries will compile using the appropriate .h files and will link with the appropriate .a files.
203These files are located in ${RTE_SDK}/arch-machine-execenv-toolchain, which is referenced internally by ${RTE_BIN_SDK}.
204
205To compile their application, the user just has to call make.
206The compilation result will be located in /path/to/my_app/build directory.
207
208Sample applications are provided in the examples directory.
209
210.. _Makefile_Description:
211
212Makefile Description
213--------------------
214
215General Rules For DPDK Makefiles
216~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
217
218In the DPDK, Makefiles always follow the same scheme:
219
220#. Include $(RTE_SDK)/mk/rte.vars.mk at the beginning.
221
222#. Define specific variables for RTE build system.
223
224#. Include a specific $(RTE_SDK)/mk/rte.XYZ.mk, where XYZ can be app, lib, extapp, extlib, obj, gnuconfigure,
225   and so on, depending on what kind of object you want to build.
226   :ref:`See Makefile Types <Makefile_Types>` below.
227
228#. Include user-defined rules and variables.
229
230   The following is a very simple example of an external application Makefile:
231
232   ..  code-block:: make
233
234        include $(RTE_SDK)/mk/rte.vars.mk
235
236        # binary name
237        APP = helloworld
238
239        # all source are stored in SRCS-y
240        SRCS-y := main.c
241
242        CFLAGS += -O3
243        CFLAGS += $(WERROR_FLAGS)
244
245        include $(RTE_SDK)/mk/rte.extapp.mk
246
247.. _Makefile_Types:
248
249Makefile Types
250~~~~~~~~~~~~~~
251
252Depending on the .mk file which is included at the end of the user Makefile, the Makefile will have a different role.
253Note that it is not possible to build a library and an application in the same Makefile.
254For that, the user must create two separate Makefiles, possibly in two different directories.
255
256In any case, the rte.vars.mk file must be included in the user Makefile as soon as possible.
257
258Application
259^^^^^^^^^^^
260
261These Makefiles generate a binary application.
262
263*   rte.app.mk: Application in the development kit framework
264
265*   rte.extapp.mk: External application
266
267*   rte.hostapp.mk: prerequisite tool to build dpdk
268
269Library
270^^^^^^^
271
272Generate a .a library.
273
274*   rte.lib.mk: Library in the development kit framework
275
276*   rte.extlib.mk: external library
277
278*   rte.hostlib.mk: host library in the development kit framework
279
280Install
281^^^^^^^
282
283*   rte.install.mk: Does not build anything, it is only used to create links or copy files to the installation directory.
284    This is useful for including files in the development kit framework.
285
286Kernel Module
287^^^^^^^^^^^^^
288
289*   rte.module.mk: Build a kernel module in the development kit framework.
290
291Objects
292^^^^^^^
293
294*   rte.obj.mk: Object aggregation (merge several .o in one) in the development kit framework.
295
296*   rte.extobj.mk: Object aggregation (merge several .o in one) outside the development kit framework.
297
298Misc
299^^^^
300
301*   rte.doc.mk: Documentation in the development kit framework
302
303*   rte.gnuconfigure.mk: Build an application that is configure-based.
304
305*   rte.subdir.mk: Build several directories in the development kit framework.
306
307.. _Internally_Generated_Build_Tools:
308
309Internally Generated Build Tools
310~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
311
312``app/pmdinfogen``
313
314
315``pmdinfogen`` scans an object (.o) file for various well known symbol names.  These
316well known symbol names are defined by various macros and used to export
317important information about hardware support and usage for pmd files.  For
318instance the macro:
319
320.. code-block:: c
321
322   PMD_REGISTER_DRIVER(drv, name)
323
324Creates the following symbol:
325
326.. code-block:: c
327
328   static char this_pmd_name0[] __attribute__((used)) = "<name>";
329
330
331Which pmdinfogen scans for.  Using this information other relevant bits of data
332can be exported from the object file and used to produce a hardware support
333description, that pmdinfogen then encodes into a json formatted string in the
334following format:
335
336.. code-block:: c
337
338   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
339
340
341These strings can then be searched for by external tools to determine the
342hardware support of a given library or application.
343
344
345.. _Useful_Variables_Provided_by_the_Build_System:
346
347Useful Variables Provided by the Build System
348~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
349
350*   RTE_SDK: The absolute path to the DPDK sources.
351    When compiling the development kit, this variable is automatically set by the framework.
352    It has to be defined by the user as an environment variable if compiling an external application.
353
354*   RTE_SRCDIR: The path to the root of the sources. When compiling the development kit, RTE_SRCDIR = RTE_SDK.
355    When compiling an external application, the variable points to the root of external application sources.
356
357*   RTE_OUTPUT: The path to which output files are written.
358    Typically, it is $(RTE_SRCDIR)/build, but it can be overridden by the O= option in the make command line.
359
360*   RTE_TARGET: A string identifying the target for which we are building.
361    The format is arch-machine-execenv-toolchain.
362    When compiling the SDK, the target is deduced by the build system from the configuration (.config).
363    When building an external application, it must be specified by the user in the Makefile or as an environment variable.
364
365*   RTE_SDK_BIN: References $(RTE_SDK)/$(RTE_TARGET).
366
367*   RTE_ARCH: Defines the architecture (i686, x86_64).
368    It is the same value as CONFIG_RTE_ARCH  but without the double-quotes around the string.
369
370*   RTE_MACHINE: Defines the machine.
371    It is the same value as CONFIG_RTE_MACHINE but without the double-quotes around the string.
372
373*   RTE_TOOLCHAIN: Defines the toolchain (gcc , icc).
374    It is the same value as CONFIG_RTE_TOOLCHAIN but without the double-quotes around the string.
375
376*   RTE_EXEC_ENV: Defines the executive environment (linuxapp).
377    It is the same value as CONFIG_RTE_EXEC_ENV but without the double-quotes around the string.
378
379*   RTE_KERNELDIR: This variable contains the absolute path to the kernel sources that will be used to compile the kernel modules.
380    The kernel headers must be the same as the ones that will be used on the target machine (the machine that will run the application).
381    By default, the variable is set to /lib/modules/$(shell uname -r)/build,
382    which is correct when the target machine is also the build machine.
383
384*   RTE_DEVEL_BUILD: Stricter options (stop on warning). It defaults to y in a git tree.
385
386Variables that Can be Set/Overridden in a Makefile Only
387~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
388
389*   VPATH: The path list that the build system will search for sources. By default, RTE_SRCDIR will be included in VPATH.
390
391*   CFLAGS: Flags to use for C compilation. The user should use +=  to append data in this variable.
392
393*   LDFLAGS: Flags to use for linking. The user should use +=  to append data in this variable.
394
395*   ASFLAGS: Flags to use for assembly. The user should use +=  to append data in this variable.
396
397*   CPPFLAGS: Flags to use to give flags to C preprocessor (only useful when assembling .S files).
398    The user should use += to append data in this variable.
399
400*   LDLIBS: In an application, the list of libraries to link with (for example, -L  /path/to/libfoo -lfoo ).
401    The user should use  +=  to append data in this variable.
402
403*   SRC-y: A list of source files (.c, .S, or .o  if the source is a binary) in case of application, library or object Makefiles.
404    The sources must be available from VPATH.
405
406*   INSTALL-y-$(INSTPATH): A list of files to be installed in  $(INSTPATH).
407    The files must be available from VPATH and will be copied in $(RTE_OUTPUT)/$(INSTPATH). Can be used in almost any RTE Makefile.
408
409*   SYMLINK-y-$(INSTPATH): A list of files to be installed in $(INSTPATH).
410    The files must be available from VPATH and will be linked (symbolically) in  $(RTE_OUTPUT)/$(INSTPATH).
411    This variable can be used in almost any DPDK Makefile.
412
413*   PREBUILD: A list of prerequisite actions to be taken before building. The user should use +=  to append data in this variable.
414
415*   POSTBUILD: A list of actions to be taken after the main build. The user should use += to append data in this variable.
416
417*   PREINSTALL: A list of prerequisite actions to be taken before installing. The user should use += to append data in this variable.
418
419*   POSTINSTALL: A list of actions to be taken after installing. The user should use += to append data in this variable.
420
421*   PRECLEAN: A list of prerequisite actions to be taken before cleaning. The user should use += to append data in this variable.
422
423*   POSTCLEAN: A list of actions to be taken after cleaning. The user should use += to append data in this variable.
424
425*   DEPDIR-y: Only used in the development kit framework to specify if the build of the current directory depends on build of another one.
426    This is needed to support parallel builds correctly.
427
428Variables that can be Set/Overridden by the User on the Command Line Only
429~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
430
431Some variables can be used to configure the build system behavior. They are documented in
432:ref:`Development Kit Root Makefile Help <Development_Kit_Root_Makefile_Help>` and
433:ref:`External Application/Library Makefile Help <External_Application/Library_Makefile_Help>`
434
435    *   WERROR_CFLAGS: By default, this is set to a specific value that depends on the compiler.
436        Users are encouraged to use this variable as follows:
437
438            CFLAGS += $(WERROR_CFLAGS)
439
440This avoids the use of different cases depending on the compiler (icc or gcc).
441Also, this variable can be overridden from the command line, which allows bypassing of the flags for testing purposes.
442
443Variables that Can be Set/Overridden by the User in a Makefile or Command Line
444~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
445
446*   CFLAGS_my_file.o: Specific flags to add for C compilation of my_file.c.
447
448*   LDFLAGS_my_app: Specific flags to add when linking my_app.
449
450*   EXTRA_CFLAGS: The content of this variable is appended after CFLAGS when compiling.
451
452*   EXTRA_LDFLAGS: The content of this variable is appended after LDFLAGS when linking.
453
454*   EXTRA_LDLIBS: The content of this variable is appended after LDLIBS when linking.
455
456*   EXTRA_ASFLAGS: The content of this variable is appended after ASFLAGS when assembling.
457
458*   EXTRA_CPPFLAGS: The content of this variable is appended after CPPFLAGS when using a C preprocessor on assembly files.
459