dev_kit_build_system.rst revision 6b3e017e
1..  BSD LICENSE
2    Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
3    All rights reserved.
4
5    Redistribution and use in source and binary forms, with or without
6    modification, are permitted provided that the following conditions
7    are met:
8
9    * Redistributions of source code must retain the above copyright
10    notice, this list of conditions and the following disclaimer.
11    * Redistributions in binary form must reproduce the above copyright
12    notice, this list of conditions and the following disclaimer in
13    the documentation and/or other materials provided with the
14    distribution.
15    * Neither the name of Intel Corporation nor the names of its
16    contributors may be used to endorse or promote products derived
17    from this software without specific prior written permission.
18
19    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
20    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
21    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
22    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
23    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
24    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
25    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
26    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
27    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
28    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30
31.. _Development_Kit_Build_System:
32
33Development Kit Build System
34============================
35
36The DPDK requires a build system for compilation activities and so on.
37This section describes the constraints and the mechanisms used in the DPDK framework.
38
39There are two use-cases for the framework:
40
41*   Compilation of the DPDK libraries and sample applications;
42    the framework generates specific binary libraries,
43    include files and sample applications
44
45*   Compilation of an external application or library, using an installed binary DPDK
46
47Building the Development Kit Binary
48-----------------------------------
49
50The following provides details on how to build the DPDK binary.
51
52Build Directory Concept
53~~~~~~~~~~~~~~~~~~~~~~~
54
55After installation, a build directory structure is created.
56Each build directory contains include files, libraries, and applications.
57
58A build directory is specific to a configuration that includes architecture + execution environment + toolchain.
59It is possible to have several build directories sharing the same sources with different configurations.
60
61For instance, to create a new build directory called my_sdk_build_dir using the default configuration template config/defconfig_x86_64-linuxapp,
62we use:
63
64.. code-block:: console
65
66    cd ${RTE_SDK}
67    make config T=x86_64-native-linuxapp-gcc O=my_sdk_build_dir
68
69This creates a new my_sdk_build_dir directory. After that, we can compile by doing:
70
71.. code-block:: console
72
73    cd my_sdk_build_dir
74    make
75
76which is equivalent to:
77
78.. code-block:: console
79
80    make O=my_sdk_build_dir
81
82The content of the my_sdk_build_dir is then:
83
84::
85
86    -- .config                         # used configuration
87
88    -- Makefile                        # wrapper that calls head Makefile
89                                       # with $PWD as build directory
90
91
92        -- build                              #All temporary files used during build
93        +--app                                # process, including . o, .d, and .cmd files.
94            |  +-- test                       # For libraries, we have the .a file.
95            |  +-- test.o                     # For applications, we have the elf file.
96            |  `-- ...
97            +-- lib
98                +-- librte_eal
99                |   `-- ...
100                +-- librte_mempool
101                |  +--  mempool-file1.o
102                |  +--  .mempool-file1.o.cmd
103                |  +--  .mempool-file1.o.d
104                |  +--   mempool-file2.o
105                |  +--  .mempool-file2.o.cmd
106                |  +--  .mempool-file2.o.d
107                |  `--  mempool.a
108                `-- ...
109
110    -- include                # All include files installed by libraries
111        +-- librte_mempool.h  # and applications are located in this
112        +-- rte_eal.h         # directory. The installed files can depend
113        +-- rte_spinlock.h    # on configuration if needed (environment,
114        +-- rte_atomic.h      # architecture, ..)
115        `-- \*.h ...
116
117    -- lib                    # all compiled libraries are copied in this
118        +-- librte_eal.a      # directory
119        +-- librte_mempool.a
120        `-- \*.a ...
121
122    -- app                    # All compiled applications are installed
123    + --test                  # here. It includes the binary in elf format
124
125Refer to
126:ref:`Development Kit Root Makefile Help <Development_Kit_Root_Makefile_Help>`
127for details about make commands that can be used from the root of DPDK.
128
129Building External Applications
130------------------------------
131
132Since DPDK is in essence a development kit, the first objective of end users will be to create an application using this SDK.
133To compile an application, the user must set the RTE_SDK and RTE_TARGET environment variables.
134
135.. code-block:: console
136
137    export RTE_SDK=/opt/DPDK
138    export RTE_TARGET=x86_64-native-linuxapp-gcc
139    cd /path/to/my_app
140
141For a new application, the user must create their own Makefile that includes some .mk files, such as
142${RTE_SDK}/mk/rte.vars.mk, and ${RTE_SDK}/mk/ rte.app.mk.
143This is described in
144:ref:`Building Your Own Application <Building_Your_Own_Application>`.
145
146Depending on the chosen target (architecture, machine, executive environment, toolchain) defined in the Makefile or as an environment variable,
147the applications and libraries will compile using the appropriate .h files and will link with the appropriate .a files.
148These files are located in ${RTE_SDK}/arch-machine-execenv-toolchain, which is referenced internally by ${RTE_BIN_SDK}.
149
150To compile their application, the user just has to call make.
151The compilation result will be located in /path/to/my_app/build directory.
152
153Sample applications are provided in the examples directory.
154
155.. _Makefile_Description:
156
157Makefile Description
158--------------------
159
160General Rules For DPDK Makefiles
161~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
162
163In the DPDK, Makefiles always follow the same scheme:
164
165#. Include $(RTE_SDK)/mk/rte.vars.mk at the beginning.
166
167#. Define specific variables for RTE build system.
168
169#. Include a specific $(RTE_SDK)/mk/rte.XYZ.mk, where XYZ can be app, lib, extapp, extlib, obj, gnuconfigure,
170   and so on, depending on what kind of object you want to build.
171   :ref:`See Makefile Types <Makefile_Types>` below.
172
173#. Include user-defined rules and variables.
174
175   The following is a very simple example of an external application Makefile:
176
177   ..  code-block:: make
178
179        include $(RTE_SDK)/mk/rte.vars.mk
180
181        # binary name
182        APP = helloworld
183
184        # all source are stored in SRCS-y
185        SRCS-y := main.c
186
187        CFLAGS += -O3
188        CFLAGS += $(WERROR_FLAGS)
189
190        include $(RTE_SDK)/mk/rte.extapp.mk
191
192.. _Makefile_Types:
193
194Makefile Types
195~~~~~~~~~~~~~~
196
197Depending on the .mk file which is included at the end of the user Makefile, the Makefile will have a different role.
198Note that it is not possible to build a library and an application in the same Makefile.
199For that, the user must create two separate Makefiles, possibly in two different directories.
200
201In any case, the rte.vars.mk file must be included in the user Makefile as soon as possible.
202
203Application
204^^^^^^^^^^^
205
206These Makefiles generate a binary application.
207
208*   rte.app.mk: Application in the development kit framework
209
210*   rte.extapp.mk: External application
211
212*   rte.hostapp.mk: prerequisite tool to build dpdk
213
214Library
215^^^^^^^
216
217Generate a .a library.
218
219*   rte.lib.mk: Library in the development kit framework
220
221*   rte.extlib.mk: external library
222
223*   rte.hostlib.mk: host library in the development kit framework
224
225Install
226^^^^^^^
227
228*   rte.install.mk: Does not build anything, it is only used to create links or copy files to the installation directory.
229    This is useful for including files in the development kit framework.
230
231Kernel Module
232^^^^^^^^^^^^^
233
234*   rte.module.mk: Build a kernel module in the development kit framework.
235
236Objects
237^^^^^^^
238
239*   rte.obj.mk: Object aggregation (merge several .o in one) in the development kit framework.
240
241*   rte.extobj.mk: Object aggregation (merge several .o in one) outside the development kit framework.
242
243Misc
244^^^^
245
246*   rte.doc.mk: Documentation in the development kit framework
247
248*   rte.gnuconfigure.mk: Build an application that is configure-based.
249
250*   rte.subdir.mk: Build several directories in the development kit framework.
251
252.. _Internally_Generated_Build_Tools:
253
254Internally Generated Build Tools
255~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
256
257``app/dpdk-pmdinfogen``
258
259
260``dpdk-pmdinfogen`` scans an object (.o) file for various well known symbol names.
261These well known symbol names are defined by various macros and used to export
262important information about hardware support and usage for pmd files.  For
263instance the macro:
264
265.. code-block:: c
266
267   RTE_PMD_REGISTER_PCI(name, drv)
268
269Creates the following symbol:
270
271.. code-block:: c
272
273   static char this_pmd_name0[] __attribute__((used)) = "<name>";
274
275
276Which ``dpdk-pmdinfogen`` scans for.  Using this information other relevant
277bits of data can be exported from the object file and used to produce a
278hardware support description, that ``dpdk-pmdinfogen`` then encodes into a
279json formatted string in the following format:
280
281.. code-block:: c
282
283   static char <name_pmd_string>="PMD_INFO_STRING=\"{'name' : '<name>', ...}\"";
284
285
286These strings can then be searched for by external tools to determine the
287hardware support of a given library or application.
288
289
290.. _Useful_Variables_Provided_by_the_Build_System:
291
292Useful Variables Provided by the Build System
293~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
294
295*   RTE_SDK: The absolute path to the DPDK sources.
296    When compiling the development kit, this variable is automatically set by the framework.
297    It has to be defined by the user as an environment variable if compiling an external application.
298
299*   RTE_SRCDIR: The path to the root of the sources. When compiling the development kit, RTE_SRCDIR = RTE_SDK.
300    When compiling an external application, the variable points to the root of external application sources.
301
302*   RTE_OUTPUT: The path to which output files are written.
303    Typically, it is $(RTE_SRCDIR)/build, but it can be overridden by the O= option in the make command line.
304
305*   RTE_TARGET: A string identifying the target for which we are building.
306    The format is arch-machine-execenv-toolchain.
307    When compiling the SDK, the target is deduced by the build system from the configuration (.config).
308    When building an external application, it must be specified by the user in the Makefile or as an environment variable.
309
310*   RTE_SDK_BIN: References $(RTE_SDK)/$(RTE_TARGET).
311
312*   RTE_ARCH: Defines the architecture (i686, x86_64).
313    It is the same value as CONFIG_RTE_ARCH  but without the double-quotes around the string.
314
315*   RTE_MACHINE: Defines the machine.
316    It is the same value as CONFIG_RTE_MACHINE but without the double-quotes around the string.
317
318*   RTE_TOOLCHAIN: Defines the toolchain (gcc , icc).
319    It is the same value as CONFIG_RTE_TOOLCHAIN but without the double-quotes around the string.
320
321*   RTE_EXEC_ENV: Defines the executive environment (linuxapp).
322    It is the same value as CONFIG_RTE_EXEC_ENV but without the double-quotes around the string.
323
324*   RTE_KERNELDIR: This variable contains the absolute path to the kernel sources that will be used to compile the kernel modules.
325    The kernel headers must be the same as the ones that will be used on the target machine (the machine that will run the application).
326    By default, the variable is set to /lib/modules/$(shell uname -r)/build,
327    which is correct when the target machine is also the build machine.
328
329*   RTE_DEVEL_BUILD: Stricter options (stop on warning). It defaults to y in a git tree.
330
331Variables that Can be Set/Overridden in a Makefile Only
332~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
333
334*   VPATH: The path list that the build system will search for sources. By default, RTE_SRCDIR will be included in VPATH.
335
336*   CFLAGS: Flags to use for C compilation. The user should use +=  to append data in this variable.
337
338*   LDFLAGS: Flags to use for linking. The user should use +=  to append data in this variable.
339
340*   ASFLAGS: Flags to use for assembly. The user should use +=  to append data in this variable.
341
342*   CPPFLAGS: Flags to use to give flags to C preprocessor (only useful when assembling .S files).
343    The user should use += to append data in this variable.
344
345*   LDLIBS: In an application, the list of libraries to link with (for example, -L  /path/to/libfoo -lfoo ).
346    The user should use  +=  to append data in this variable.
347
348*   SRC-y: A list of source files (.c, .S, or .o  if the source is a binary) in case of application, library or object Makefiles.
349    The sources must be available from VPATH.
350
351*   INSTALL-y-$(INSTPATH): A list of files to be installed in  $(INSTPATH).
352    The files must be available from VPATH and will be copied in $(RTE_OUTPUT)/$(INSTPATH). Can be used in almost any RTE Makefile.
353
354*   SYMLINK-y-$(INSTPATH): A list of files to be installed in $(INSTPATH).
355    The files must be available from VPATH and will be linked (symbolically) in  $(RTE_OUTPUT)/$(INSTPATH).
356    This variable can be used in almost any DPDK Makefile.
357
358*   PREBUILD: A list of prerequisite actions to be taken before building. The user should use +=  to append data in this variable.
359
360*   POSTBUILD: A list of actions to be taken after the main build. The user should use += to append data in this variable.
361
362*   PREINSTALL: A list of prerequisite actions to be taken before installing. The user should use += to append data in this variable.
363
364*   POSTINSTALL: A list of actions to be taken after installing. The user should use += to append data in this variable.
365
366*   PRECLEAN: A list of prerequisite actions to be taken before cleaning. The user should use += to append data in this variable.
367
368*   POSTCLEAN: A list of actions to be taken after cleaning. The user should use += to append data in this variable.
369
370*   DEPDIR-y: Only used in the development kit framework to specify if the build of the current directory depends on build of another one.
371    This is needed to support parallel builds correctly.
372
373Variables that can be Set/Overridden by the User on the Command Line Only
374~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
375
376Some variables can be used to configure the build system behavior. They are documented in
377:ref:`Development Kit Root Makefile Help <Development_Kit_Root_Makefile_Help>` and
378:ref:`External Application/Library Makefile Help <External_Application/Library_Makefile_Help>`
379
380    *   WERROR_CFLAGS: By default, this is set to a specific value that depends on the compiler.
381        Users are encouraged to use this variable as follows:
382
383            CFLAGS += $(WERROR_CFLAGS)
384
385This avoids the use of different cases depending on the compiler (icc or gcc).
386Also, this variable can be overridden from the command line, which allows bypassing of the flags for testing purposes.
387
388Variables that Can be Set/Overridden by the User in a Makefile or Command Line
389~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
390
391*   CFLAGS_my_file.o: Specific flags to add for C compilation of my_file.c.
392
393*   LDFLAGS_my_app: Specific flags to add when linking my_app.
394
395*   EXTRA_CFLAGS: The content of this variable is appended after CFLAGS when compiling.
396
397*   EXTRA_LDFLAGS: The content of this variable is appended after LDFLAGS when linking.
398
399*   EXTRA_LDLIBS: The content of this variable is appended after LDLIBS when linking.
400
401*   EXTRA_ASFLAGS: The content of this variable is appended after ASFLAGS when assembling.
402
403*   EXTRA_CPPFLAGS: The content of this variable is appended after CPPFLAGS when using a C preprocessor on assembly files.
404