diff options
Diffstat (limited to 'doc/bitbake-user-manual/bitbake-user-manual-metadata.xml')
| -rw-r--r-- | doc/bitbake-user-manual/bitbake-user-manual-metadata.xml | 1638 |
1 files changed, 1638 insertions, 0 deletions
diff --git a/doc/bitbake-user-manual/bitbake-user-manual-metadata.xml b/doc/bitbake-user-manual/bitbake-user-manual-metadata.xml new file mode 100644 index 000000000..41ae3b8c0 --- /dev/null +++ b/doc/bitbake-user-manual/bitbake-user-manual-metadata.xml @@ -0,0 +1,1638 @@ +<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + +<chapter id="bitbake-user-manual-metadata"> + <title>Syntax and Operators</title> + + <para> + Bitbake files have their own syntax. + The syntax has similarities to several + other languages but also has some unique features. + This section describes the available syntax and operators + as well as provides examples. + </para> + + <section id='basic-syntax'> + <title>Basic Syntax</title> + + <para> + This section provides some basic syntax examples. + </para> + + <section id='basic-variable-setting'> + <title>Basic Variable Setting</title> + + <para> + The following example sets <filename>VARIABLE</filename> to + "value". + This assignment occurs immediately as the statement is parsed. + It is a "hard" assignment. + <literallayout class='monospaced'> + VARIABLE = "value" + </literallayout> + As expected, if you include leading or trailing spaces as part of + an assignment, the spaces are retained: + <literallayout class='monospaced'> + VARIABLE = " value" + VARIABLE = "value " + </literallayout> + Setting <filename>VARIABLE</filename> to "" sets it to an empty string, + while setting the variable to " " sets it to a blank space + (i.e. these are not the same values). + <literallayout class='monospaced'> + VARIABLE = "" + VARIABLE = " " + </literallayout> + </para> + </section> + + <section id='variable-expansion'> + <title>Variable Expansion</title> + + <para> + BitBake supports variables referencing one another's + contents using a syntax that is similar to shell scripting. + Following is an example that results in <filename>A</filename> + containing "aval" and <filename>B</filename> evaluating to + "preavalpost" based on that current value of + <filename>A</filename>. + <literallayout class='monospaced'> + A = "aval" + B = "pre${A}post" + </literallayout> + You should realize that whenever <filename>B</filename> is + referenced, its evaluation will depend on the state of + <filename>A</filename> at that time. + Thus, later evaluations of <filename>B</filename> in the + previous example could result in different values + depending on the value of <filename>A</filename>. + </para> + </section> + + <section id='setting-a-default-value'> + <title>Setting a default value (?=)</title> + + <para> + You can use the "?=" operator to achieve a "softer" assignment + for a variable. + This type of assignment allows you to define a variable if it + is undefined when the statement is parsed, but to leave the + value alone if the variable has a value. + Here is an example: + <literallayout class='monospaced'> + A ?= "aval" + </literallayout> + If <filename>A</filename> is set at the time this statement is parsed, + the variable retains its value. + However, if <filename>A</filename> is not set, + the variable is set to "aval". + <note> + This assignment is immediate. + Consequently, if multiple "?=" assignments + to a single variable exist, the first of those ends up getting + used. + </note> + </para> + </section> + + <section id='setting-a-weak-default-value'> + <title>Setting a weak default value (??=)</title> + + <para> + It is possible to use a "weaker" assignment than in the + previous section by using the "??=" operator. + This assignment behaves identical to "?=" except that the + assignment is made at the end of the parsing process rather + than immediately. + Consequently, when multiple "??=" assignments exist, the last + one is used. + Also, any "=" or "?=" assignment will override the value set with + "??=". + Here is an example: + <literallayout class='monospaced'> + A ??= "somevalue" + A ??= "someothervalue" + </literallayout> + If <filename>A</filename> is set before the above statements are parsed, + the variable retains its value. + If <filename>A</filename> is not set, + the variable is set to "someothervalue". + </para> + + <para> + Again, this assignment is a "lazy" or "weak" assignment + because it does not occur until the end + of the parsing process. + </para> + </section> + + <section id='immediate-variable-expansion'> + <title>Immediate variable expansion (:=)</title> + + <para> + The ":=" operator results in a variable's + contents being expanded immediately, + rather than when the variable is actually used: + <literallayout class='monospaced'> + T = "123" + A := "${B} ${A} test ${T}" + T = "456" + B = "${T} bval" + C = "cval" + C := "${C}append" + </literallayout> + In this example, <filename>A</filename> contains + "test 123" because <filename>${B}</filename> and + <filename>${A}</filename> at the time of parsing are undefined, + which leaves "test 123". + And, the variable <filename>C</filename> + contains "cvalappend" since <filename>${C}</filename> immediately + expands to "cval". + </para> + </section> + + <section id='appending-and-prepending'> + <title>Appending (+=) and prepending (=+) With Spaces</title> + + <para> + Appending and prepending values is common and can be accomplished + using the "+=" and "=+" operators. + These operators insert a space between the current + value and prepended or appended value. + Here are some examples: + <literallayout class='monospaced'> + B = "bval" + B += "additionaldata" + C = "cval" + C =+ "test" + </literallayout> + The variable <filename>B</filename> contains + "bval additionaldata" and <filename>C</filename> + contains "test cval". + </para> + </section> + + <section id='appending-and-prepending-without-spaces'> + <title>Appending (.=) and Prepending (=.) Without Spaces</title> + + <para> + If you want to append or prepend values without an + inserted space, use the ".=" and "=." operators. + Here are some examples: + <literallayout class='monospaced'> + B = "bval" + B .= "additionaldata" + C = "cval" + C =. "test" + </literallayout> + The variable <filename>B</filename> contains + "bvaladditionaldata" and + <filename>C</filename> contains "testcval". + </para> + </section> + + <section id='appending-and-prepending-override-style-syntax'> + <title>Appending and Prepending (Override Style Syntax)</title> + + <para> + You can also append and prepend a variable's value + using an override style syntax. + When you use this syntax, no spaces are inserted. + Here are some examples: + <literallayout class='monospaced'> + B = "bval" + B_append = " additional data" + C = "cval" + C_prepend = "additional data " + D = "dval" + D_append = "additional data" + </literallayout> + The variable <filename>B</filename> becomes + "bval additional data" and <filename>C</filename> becomes + "additional data cval". + The variable <filename>D</filename> becomes + "dvaladditional data". + <note> + You must control all spacing when you use the + override syntax. + </note> + </para> + + <para> + The operators "_append" and "_prepend" differ from + the operators ".=" and "=." in that they are deferred + until after parsing completes rather than being immediately + applied. + </para> + </section> + + <section id='removing-override-style-syntax'> + <title>Removal (Override Style Syntax)</title> + + <para> + You can remove values from lists using the removal + override style syntax. + Specifying a value for removal causes all occurrences of that + value to be removed from the variable. + </para> + + <para> + When you use this syntax, BitBake expects one or more strings. + Surrounding spaces are removed as well. + Here is an example: + <literallayout class='monospaced'> + FOO = "123 456 789 123456 123 456 123 456" + FOO_remove = "123" + FOO_remove = "456" + FOO2 = "abc def ghi abcdef abc def abc def" + FOO2_remove = "abc def" + </literallayout> + The variable <filename>FOO</filename> becomes + "789 123456" and <filename>FOO2</filename> becomes + "ghi abcdef". + </para> + </section> + + <section id='variable-flag-syntax'> + <title>Variable Flag Syntax</title> + + <para> + Variable flags are BitBake's implementation of variable properties + or attributes. + It is a way of tagging extra information onto a variable. + You can find more out about variable flags in general in the + "<link linkend='variable-flags'>Variable Flags</link>" + section. + </para> + + <para> + You can define, append, and prepend values to variable flags. + All the standard syntax operations previously mentioned work + for variable flags except for override style syntax + (i.e. <filename>_prepend</filename>, <filename>_append</filename>, + and <filename>_remove</filename>). + </para> + + <para> + Here are some examples showing how to set variable flags: + <literallayout class='monospaced'> + FOO[a] = "abc" + FOO[b] = "123" + FOO[a] += "456" + </literallayout> + The variable <filename>FOO</filename> has two flags: + <filename>a</filename> and <filename>b</filename>. + The flags are immediately set to "abc" and "123", respectively. + The <filename>a</filename> flag becomes "abc456". + </para> + </section> + + <section id='inline-python-variable-expansion'> + <title>Inline Python Variable Expansion</title> + + <para> + You can use inline Python variable expansion to + set variables. + Here is an example: + <literallayout class='monospaced'> + DATE = "${@time.strftime('%Y%m%d',time.gmtime())}" + </literallayout> + This example results in the <filename>DATE</filename> + variable becoming the current date. + </para> + </section> + </section> + + <section id='conditional-syntax-overrides'> + <title>Conditional Syntax (Overrides)</title> + + <para> + BitBake uses + <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link> + to control what variables are overridden after BitBake + parses recipes and configuration files. + This section describes how you can use + <filename>OVERRIDES</filename> as conditional metadata, + talks about key expansion in relationship to + <filename>OVERRIDES</filename>, and provides some examples + to help with understanding. + </para> + + <section id='conditional-metadata'> + <title>Conditional Metadata</title> + + <para> + You can use <filename>OVERRIDES</filename> to conditionally select + a specific version of a variable and to conditionally + append or prepend the value of a variable. + <itemizedlist> + <listitem><para><emphasis>Selecting a Variable:</emphasis> + The <filename>OVERRIDES</filename> variable is + a colon-character-separated list that contains items + for which you want to satisfy conditions. + Thus, if you have a variable that is conditional on “arm”, and “arm” + is in <filename>OVERRIDES</filename>, then the “arm”-specific + version of the variable is used rather than the non-conditional + version. + Here is an example: + <literallayout class='monospaced'> + OVERRIDES = "architecture:os:machine" + TEST = "default" + TEST_os = "osspecific" + TEST_nooverride = "othercondvalue" + </literallayout> + In this example, the <filename>OVERRIDES</filename> + variable lists three overrides: + "architecture", "os", and "machine". + The variable <filename>TEST</filename> by itself has a default + value of "default". + You select the os-specific version of the <filename>TEST</filename> + variable by appending the "os" override to the variable + (i.e.<filename>TEST_os</filename>). + </para></listitem> + <listitem><para><emphasis>Appending and Prepending:</emphasis> + BitBake also supports append and prepend operations to + variable values based on whether a specific item is + listed in <filename>OVERRIDES</filename>. + Here is an example: + <literallayout class='monospaced'> + DEPENDS = "glibc ncurses" + OVERRIDES = "machine:local" + DEPENDS_append_machine = "libmad" + </literallayout> + In this example, <filename>DEPENDS</filename> becomes + "glibc ncurses libmad". + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='key-expansion'> + <title>Key Expansion</title> + + <para> + Key expansion happens when the BitBake datastore is finalized + just before BitBake expands overrides. + To better understand this, consider the following example: + <literallayout class='monospaced'> + A${B} = "X" + B = "2" + A2 = "Y" + </literallayout> + In this case, after all the parsing is complete, and + before any overrides are handled, BitBake expands + <filename>${B}</filename> into "2". + This expansion causes <filename>A2</filename>, which was + set to "Y" before the expansion, to become "X". + </para> + </section> + + <section id='variable-interaction-worked-examples'> + <title>Examples</title> + + <para> + Despite the previous explanations that show the different forms of + variable definitions, it can be hard to work + out exactly what happens when variable operators, conditional + overrides, and unconditional overrides are combined. + This section presents some common scenarios along + with explanations for variable interactions that + typically confuse users. + </para> + + <para> + There is often confusion concerning the order in which + overrides and various "append" operators take effect. + Recall that an append or prepend operation using "_append" + and "_prepend" does not result in an immediate assignment + as would "+=", ".=", "=+", or "=.". + Consider the following example: + <literallayout class='monospaced'> + OVERRIDES = "foo" + A = "Z" + A_foo_append = "X" + </literallayout> + For this case, <filename>A</filename> is + unconditionally set to "Z" and "X" is + unconditionally and immediately appended to the variable + <filename>A_foo</filename>. + Because overrides have not been applied yet, + <filename>A_foo</filename> is set to "X" due to the append + and <filename>A</filename> simply equals "Z". + </para> + + <para> + Applying overrides, however, changes things. + Since "foo" is listed in <filename>OVERRIDES</filename>, + the conditional variable <filename>A</filename> is replaced + with the "foo" version, which is equal to "X". + So effectively, <filename>A_foo</filename> replaces <filename>A</filename>. + </para> + + <para> + This next example changes the order of the override and + the append: + <literallayout class='monospaced'> + OVERRIDES = "foo" + A = "Z" + A_append_foo = "X" + </literallayout> + For this case, before overrides are handled, + <filename>A</filename> is set to "Z" and <filename>A_append_foo</filename> + is set to "X". + Once the override for "foo" is applied, however, + <filename>A</filename> gets appended with "X". + Consequently, <filename>A</filename> becomes "ZX". + Notice that spaces are not appended. + </para> + + <para> + This next example has the order of the appends and overrides reversed + back as in the first example: + <literallayout class='monospaced'> + OVERRIDES = "foo" + A = "Y" + A_foo_append = "Z" + A_foo_append += "X" + </literallayout> + For this case, before any overrides are resolved, + <filename>A</filename> is set to "Y" using an immediate assignment. + After this immediate assignment, <filename>A_foo</filename> is set + to "Z", and then further appended with + "X" leaving the variable set to "Z X". + Finally, applying the override for "foo" results in the conditional + variable <filename>A</filename> becoming "Z X" (i.e. + <filename>A</filename> is replaced with <filename>A_foo</filename>). + </para> + + <para> + This final example mixes in some varying operators: + <literallayout class='monospaced'> + A = "1" + A_append = "2" + A_append = "3" + A += "4" + A .= "5" + </literallayout> + For this case, the type of append operators are affecting the + order of assignments as BitBake passes through the code + multiple times. + Initially, <filename>A</filename> is set to "1 45" because + of the three statements that use immediate operators. + After these assignments are made, BitBake applies the + <filename>_append</filename> operations. + Those operations result in <filename>A</filename> becoming "1 4523". + </para> + </section> + </section> + + <section id='sharing-functionality'> + <title>Sharing Functionality</title> + + <para> + BitBake allows for metadata sharing through include files + (<filename>.inc</filename>) and class files + (<filename>.bbclass</filename>). + For example, suppose you have a piece of common functionality + such as a task definition that you want to share between + more than one recipe. + In this case, creating a <filename>.bbclass</filename> + file that contains the common functionality and then using + the <filename>inherit</filename> directive in your recipes to + inherit the class would be a common way to share the task. + </para> + + <para> + This section presents the mechanisms BitBake provides to + allow you to share functionality between recipes. + Specifically, the mechanisms include <filename>include</filename>, + <filename>inherit</filename>, <filename>INHERIT</filename>, and + <filename>require</filename> directives. + </para> + + <section id='locating-include-and-class-files'> + <title>Locating Include and Class Files</title> + + <para> + BitBake uses the + <link linkend='var-BBPATH'><filename>BBPATH</filename></link> + variable to locate needed include and class files. + The <filename>BBPATH</filename> variable is analogous to + the environment variable <filename>PATH</filename>. + </para> + + <para> + In order for include and class files to be found by BitBake, + they need to be located in a "classes" subdirectory that can + be found in <filename>BBPATH</filename>. + </para> + </section> + + <section id='inherit-directive'> + <title><filename>inherit</filename> Directive</title> + + <para> + When writing a recipe or class file, you can use the + <filename>inherit</filename> directive to inherit the + functionality of a class (<filename>.bbclass</filename>). + BitBake only supports this directive when used within recipe + and class files (i.e. <filename>.bb</filename> and + <filename>.bbclass</filename>). + </para> + + <para> + The <filename>inherit</filename> directive is a rudimentary + means of specifying what classes of functionality your + recipes require. + For example, you can easily abstract out the tasks involved in + building a package that uses Autoconf and Automake and put + those tasks into a class file that can be used by your recipe. + </para> + + <para> + As an example, your recipes could use the following directive + to inherit an <filename>autotools.bbclass</filename> file. + The class file would contain common functionality for using + Autotools that could be shared across recipes: + <literallayout class='monospaced'> + inherit autotools + </literallayout> + In this case, BitBake would search for the directory + <filename>classes/autotools.bbclass</filename> + in <filename>BBPATH</filename>. + <note> + You can override any values and functions of the + inherited class within your recipe by doing so + after the "inherit" statement. + </note> + </para> + </section> + + <section id='include-directive'> + <title><filename>include</filename> Directive</title> + + <para> + BitBake understands the <filename>include</filename> + directive. + This directive causes BitBake to parse whatever file you specify, + and to insert that file at that location. + The directive is much like its equivalent in Make except + that if the path specified on the include line is a relative + path, BitBake locates the first file it can find + within <filename>BBPATH</filename>. + </para> + + <para> + As an example, suppose you needed a recipe to include some + self-test definitions: + <literallayout class='monospaced'> + include test_defs.inc + </literallayout> + <note> + The <filename>include</filename> directive does not + produce an error when the file cannot be found. + Consequently, it is recommended that if the file you + are including is expected to exist, you should use + <link linkend='require-inclusion'><filename>require</filename></link> + instead of <filename>include</filename>. + Doing so makes sure that an error is produced if the + file cannot be found. + </note> + </para> + </section> + + <section id='require-inclusion'> + <title><filename>require</filename> Directive</title> + + <para> + BitBake understands the <filename>require</filename> + directive. + This directive behaves just like the + <filename>include</filename> directive with the exception that + BitBake raises a parsing error if the file to be included cannot + be found. + Thus, any file you require is inserted into the file that is + being parsed at the location of the directive. + </para> + + <para> + Similar to how BitBake handles + <link linkend='include-directive'><filename>include</filename></link>, + if the path specified + on the require line is a relative path, BitBake locates + the first file it can find within <filename>BBPATH</filename>. + </para> + + <para> + As an example, suppose you have two versions of a recipe + (e.g. <filename>foo_1.2.2.bb</filename> and + <filename>foo_2.0.0.bb</filename>) where + each version contains some identical functionality that could be + shared. + You could create an include file named <filename>foo.inc</filename> + that contains the common definitions needed to build "foo". + You need to be sure <filename>foo.inc</filename> is located in the + same directory as your two recipe files as well. + Once these conditions are set up, you can share the functionality + using a <filename>require</filename> directive from within each + recipe: + <literallayout class='monospaced'> + require foo.inc + </literallayout> + </para> + </section> + + <section id='inherit-configuration-directive'> + <title><filename>INHERIT</filename> Configuration Directive</title> + + <para> + When creating a configuration file (<filename>.conf</filename>), + you can use the <filename>INHERIT</filename> directive to + inherit a class. + BitBake only supports this directive when used within + a configuration file. + </para> + + <para> + As an example, suppose you needed to inherit a class + file called <filename>abc.bbclass</filename> from a + configuration file as follows: + <literallayout class='monospaced'> + INHERIT += "abc" + </literallayout> + This configuration directive causes the named + class to be inherited at the point of the directive + during parsing. + As with the <filename>inherit</filename> directive, the + <filename>.bbclass</filename> file must be located in a + "classes" subdirectory in one of the directories specified + in <filename>BBPATH</filename>. + <note> + Because <filename>.conf</filename> files are parsed + first during BitBake's execution, using + <filename>INHERIT</filename> to inherit a class effectively + inherits the class globally (i.e. for all recipes). + </note> + </para> + </section> + </section> + + <section id='functions'> + <title>Functions</title> + + <para> + As with most languages, functions are the building blocks that + are used to build up operations into tasks. + BitBake supports three types of functions: + <itemizedlist> + <listitem><para><emphasis>Shell Functions:</emphasis> + Functions written in shell script and executed either + directly as functions, tasks, or both. + They can also be called by other shell functions. + </para></listitem> + <listitem><para><emphasis>BitBake Style Python Functions:</emphasis> + Functions written in Python and executed by BitBake or other + Python functions using <filename>bb.build.exec_func()</filename>. + </para></listitem> + <listitem><para><emphasis>Python Functions:</emphasis> + Functions written in Python and executed by Python. + </para></listitem> + </itemizedlist> + Regardless of the type of function, you can only + define them in class (<filename>.bbclass</filename>) + and recipe (<filename>.bb</filename> or <filename>.inc</filename>) + files. + </para> + + <section id='shell-functions'> + <title>Shell Functions</title> + + <para> + Functions written in shell script and executed either + directly as functions, tasks, or both. + They can also be called by other shell functions. + Here is an example shell function definition: + <literallayout class='monospaced'> + some_function () { + echo "Hello World" + } + </literallayout> + When you create these types of functions in your recipe + or class files, you need to follow the shell programming + rules. + The scripts are executed by <filename>/bin/sh</filename>, + which may not be a bash shell but might be something + such as <filename>dash</filename>. + You should not use Bash-specific script (bashisms). + </para> + </section> + + <section id='bitbake-style-python-functions'> + <title>BitBake Style Python Functions</title> + + <para> + These functions are written in Python and executed by + BitBake or other Python functions using + <filename>bb.build.exec_func()</filename>. + </para> + + <para> + An example BitBake function is: + <literallayout class='monospaced'> + python some_python_function () { + d.setVar("TEXT", "Hello World") + print d.getVar("TEXT", True) + } + </literallayout> + Because the Python "bb" and "os" modules are already + imported, you do not need to import these modules. + Also in these types of functions, the datastore ("d") + is a global variable and is always automatically + available. + </para> + </section> + + <section id='python-functions'> + <title>Python Functions</title> + + <para> + These functions are written in Python and are executed by + other Python code. + Examples of Python functions are utility functions + that you intend to call from in-line Python or + from within other Python functions. + Here is an example: + <literallayout class='monospaced'> + def get_depends(d): + if d.getVar('SOMECONDITION', True): + return "dependencywithcond" + else: + return "dependency" + SOMECONDITION = "1" + DEPENDS = "${@get_depends(d)}" + </literallayout> + This would result in <filename>DEPENDS</filename> + containing <filename>dependencywithcond</filename>. + </para> + + <para> + Here are some things to know about Python functions: + <itemizedlist> + <listitem><para>Python functions can take parameters. + </para></listitem> + <listitem><para>The BitBake datastore is not + automatically available. + Consequently, you must pass it in as a + parameter to the function. + </para></listitem> + <listitem><para>The "bb" and "os" Python modules are + automatically available. + You do not need to import them. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='automatically-mapping-functions-within-the-context-of-a-class'> + <title>Automatically Mapping Functions Within the Context of a Class</title> + + <para> + Through coding techniques and the use of + <filename>EXPORT_FUNCTIONS</filename>, BitBake supports + automatic mapping for functions within the context of + a class. + </para> + + <para> + To understand the benefits of this feature, consider the basic scenario + where a class defines a function and your recipe inherits the class. + In this basic scenario, your recipe has access to the function in the + class by way of inheritance and can freely call and use the function + as defined in the class. + However, if you need to have a modified version of that function + in your recipe you are limited to using either your modified version + of the function or using "prepend_" or "_append" operators to add + code to be executed before or after the original function in the + class. + Your recipe cannot use both versions of the fucntion. + </para> + + <para> + Function mapping allows you to access both your custom function + function that is defined in the recipe and the original function that + is defined in the class. + You have this access all from within your recipe. + To accomplish this, you need some things in place: + <itemizedlist> + <listitem><para> + The class needs to define the function as follows: + <literallayout class='monospaced'> + <classname>_<functionname> + </literallayout> + For example, if you have a class file + <filename>bar.bbclass</filename> and a function named + <filename>do_foo</filename>, the class must define the function + as follows: + <literallayout class='monospaced'> + bar_do_foo + </literallayout> + </para></listitem> + <listitem><para> + The class needs to contain the <filename>EXPORT_FUNCTIONS</filename> + statement as follows: + <literallayout class='monospaced'> + EXPORT_FUNCTIONS <functionname> + </literallayout> + For example, continuing with the same example, the + statement in the <filename>bar.bbclass</filename> would be + as follows: + <literallayout class='monospaced'> + EXPORT_FUNCTIONS do_foo + </literallayout> + </para></listitem> + <listitem><para> + You need to call the function appropriately from within your + recipe. + Continuing with the same example, + your recipe would call the <filename>do_foo</filename> function + from the recipe by referring to it as + <filename>bar_do_foo</filename>. + To call your modified version of the function as defined in your + recipe, call it as <filename>do_foo</filename>. + </para></listitem> + </itemizedlist> + With these conditions met, your single recipe + can freely choose between the original function + as defined in the class file and the modified function in your recipe. + If you do not set up these conditions, you are limited to using one function + or the other. + </para> + </section> + </section> + + <section id='tasks'> + <title>Tasks</title> + + <para> + Tasks are BitBake execution units that originate as + functions and make up the steps that BitBake needs to run + for given recipe. + Tasks are only supported in recipe (<filename>.bb</filename> + or <filename>.inc</filename>) and class + (<filename>.bbclass</filename>) files. + By convention, task names begin with the string "do_". + </para> + + <para> + Here is an example of a task that prints out the date: + <literallayout class='monospaced'> + python do_printdate () { + import time + print time.strftime('%Y%m%d', time.gmtime()) + } + addtask printdate after do_fetch before do_build + </literallayout> + </para> + + <section id='promoting-a-function-to-a-task'> + <title>Promoting a Function to a Task</title> + + <para> + Any function can be promoted to a task by applying the + <filename>addtask</filename> command. + The <filename>addtask</filename> command also describes + inter-task dependencies. + Here is the function from the previous section but with the + <filename>addtask</filename> command promoting it to a task + and defining some dependencies: + <literallayout class='monospaced'> + python do_printdate () { + import time + print time.strftime('%Y%m%d', time.gmtime()) + } + addtask printdate after do_fetch before do_build + </literallayout> + In the example, the function is defined and then promoted + as a task. + The <filename>do_printdate</filename> task becomes a dependency of + the <filename>do_build</filename> task, which is the default + task. + And, the <filename>do_printdate</filename> task is dependent upon + the <filename>do_fetch</filename> task. + Execution of the <filename>do_build</filename> task results + in the <filename>do_printdate</filename> task running first. + </para> + </section> + + <section id='deleting-a-task'> + <title>Deleting a Task</title> + + <para> + As well as being able to add tasks, tasks can also be deleted. + This is done simply with <filename>deltask</filename> command. + For example, to delete the example task used in the previous + sections, you would use: + <literallayout class='monospaced'> + deltask printdate + </literallayout> + </para> + </section> + + <section id='passing-information-into-the-build-task-environment'> + <title>Passing Information Into the Build Task Environment</title> + + <para> + When running a task, BitBake tightly controls the execution + environment of the build tasks to make + sure unwanted contamination from the build machine cannot + influence the build. + Consequently, if you do want something to get passed into the + build task environment, you must take these two steps: + <orderedlist> + <listitem><para> + Tell BitBake to load what you want from the environment + into the datastore. + You can do so through the + <link linkend='var-BB_ENV_EXTRAWHITE'><filename>BB_ENV_EXTRAWHITE</filename></link> + variable. + For example, assume you want to prevent the build system from + accessing your <filename>$HOME/.ccache</filename> + directory. + The following command tells BitBake to load + <filename>CCACHE_DIR</filename> from the environment into + the datastore: + <literallayout class='monospaced'> + export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE CCACHE_DIR" + </literallayout></para></listitem> + <listitem><para> + Tell BitBake to export what you have loaded into the + datastore to the task environment of every running task. + Loading something from the environment into the datastore + (previous step) only makes it available in the datastore. + To export it to the task environment of every running task, + use a command similar to the following in your local configuration + file <filename>local.conf</filename> or your + distribution configuration file: + <literallayout class='monospaced'> + export CCACHE_DIR + </literallayout> + <note> + A side effect of the previous steps is that BitBake + records the variable as a dependency of the build process + in things like the setscene checksums. + If doing so results in unnecessary rebuilds of tasks, you can + whitelist the variable so that the setscene code + ignores the dependency when it creates checksums. + </note></para></listitem> + </orderedlist> + </para> + + <para> + Sometimes, it is useful to be able to obtain information + from the original execution environment. + Bitbake saves a copy of the original environment into + a special variable named + <link linkend='var-BB_ORIGENV'><filename>BB_ORIGENV</filename></link>. + </para> + + <para> + The <filename>BB_ORIGENV</filename> variable returns a datastore + object that can be queried using the standard datastore operators + such as <filename>getVar()</filename>. + The datastore object is useful, for example, to find the original + <filename>DISPLAY</filename> variable. + Here is an example: + <literallayout class='monospaced'> + BB_ORIGENV - add example? + + origenv = d.getVar("BB_ORIGENV", False) + bar = origenv.getVar("BAR", False) + </literallayout> + The previous example returns <filename>BAR</filename> from the original + execution environment. + </para> + + <para> + By default, BitBake cleans the environment to include only those + things exported or listed in its whitelist to ensure that the build + environment is reproducible and consistent. + </para> + </section> + </section> + + <section id='variable-flags'> + <title>Variable Flags</title> + + <para> + Variable flags (varflags) help control a task's functionality + and dependencies. + BitBake reads and writes varflags to the datastore using the following + command forms: + <literallayout class='monospaced'> + <variable> = d.getVarFlags("<variable>") + self.d.setVarFlags("FOO", {"func": True}) + </literallayout> + </para> + + <para> + When working with varflags, the same syntax, with the exception of + overrides, applies. + In other words, you can set, append, and prepend varflags just like + variables. + See the + "<link linkend='variable-flag-syntax'>Variable Flag Syntax</link>" + section for details. + </para> + + <para> + BitBake has a defined set of varflags available for recipes and + classes. + Tasks support a number of these flags which control various + functionality of the task: + <itemizedlist> + <listitem><para><emphasis>dirs:</emphasis> + Directories that should be created before the task runs. + </para></listitem> + <listitem><para><emphasis>cleandirs:</emphasis> + Empty directories that should created before the task runs. + </para></listitem> + <listitem><para><emphasis>noexec:</emphasis> + Marks the tasks as being empty and no execution required. + The <filename>noexec</filename> flag can be used to set up + tasks as dependency placeholders, or to disable tasks defined + elsewhere that are not needed in a particular recipe. + </para></listitem> + <listitem><para><emphasis>nostamp:</emphasis> + Tells BitBake to not generate a stamp file for a task, + which implies the task should always be executed. + </para></listitem> + <listitem><para><emphasis>fakeroot:</emphasis> + Causes a task to be run in a fakeroot environment, + obtained by adding the variables in + <link linkend='var-FAKEROOTENV'><filename>FAKEROOTENV</filename></link> + to the environment. + </para></listitem> + <listitem><para><emphasis>umask:</emphasis> + The umask to run the task under. + </para></listitem> + <listitem><para><emphasis>deptask:</emphasis> + Controls task build-time dependencies. + See the + <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> + variable and the + "<link linkend='build-dependencies'>Build Dependencies</link>" + section for more information. + </para></listitem> + <listitem><para><emphasis>rdeptask:</emphasis> + Controls task runtime dependencies. + See the + <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link> + variable, the + <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link> + variable, and the + "<link linkend='runtime-dependencies'>Runtime Dependencies</link>" + section for more information. + </para></listitem> + <listitem><para><emphasis>recrdeptask:</emphasis> + Controls task recursive runtime dependencies. + See the + <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link> + variable, the + <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link> + variable, and the + "<link linkend='recursive-dependencies'>Recursive Dependencies</link>" + section for more information. + </para></listitem> + <listitem><para><emphasis>depends:</emphasis> + Controls inter-task dependencies. + See the + <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> + variable and the + "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>" + section for more information. + </para></listitem> + <listitem><para><emphasis>rdepends:</emphasis> + Controls inter-task runtime dependencies. + See the + <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link> + variable, the + <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link> + variable, and the + "<link linkend='inter-task-dependencies'>Inter-Task Dependencies</link>" + section for more information. + </para></listitem> + <listitem><para><emphasis>postfuncs:</emphasis> + List of functions to call after the completion of the task. + </para></listitem> + <listitem><para><emphasis>prefuncs:</emphasis> + List of functions to call before the task executes. + </para></listitem> + <listitem><para><emphasis>stamp-extra-info:</emphasis> + Extra stamp information to append to the task's stamp. + As an example, OpenEmbedded uses this flag to allow + machine-specific tasks. + </para></listitem> + </itemizedlist> + </para> + + <para> + Several varflags are useful for controlling how signatures are + calculated for variables. + For more information on this process, see the + "<link linkend='checksums'>Checksums (Signatures)</link>" + section. + <itemizedlist> + <listitem><para><emphasis>vardeps:</emphasis> + Specifies a space-separated list of additional + variables to add to a variable's dependencies + for the purposes of calculating its signature. + Adding variables to this list is useful, for example, when + a function refers to a variable in a manner that + does not allow BitBake to automatically determine + that the variable is referred to. + </para></listitem> + <listitem><para><emphasis>vardepvalue:</emphasis> + If set, instructs BitBake to ignore the actual + value of the variable and instead use the specified + value when calculating the variable's signature. + </para></listitem> + <listitem><para><emphasis>vardepsexclude:</emphasis> + Specifies a space-separated list of variables + that should be excluded from a variable's dependencies + for the purposes of calculating its signature. + </para></listitem> + <listitem><para><emphasis>vardepvalueexclude:</emphasis> + Specifies a pipe-separated list of strings to exclude + from the variable's value when calculating the + variable's signature. + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='events'> + <title>Events</title> + + <para> + BitBake allows installation of event handlers within + recipe and class files. + Events are triggered at certain points during operation, + such as the beginning of operation against a given + <filename>.bb</filename>, the start of a given task, + task failure, task success, and so forth. + The intent is to make it easy to do things like email + notification on build failure. + </para> + + <para> + Following is an example event handler that + prints the name of the event and the content of + the <filename>FILE</filename> variable: + <literallayout class='monospaced'> + addhandler myclass_eventhandler + python myclass_eventhandler() { + from bb.event import getName + from bb import data + print("The name of the Event is %s" % getName(e)) + print("The file we run for is %s" % data.getVar('FILE', e.data, True)) + } + </literallayout> + This event handler gets called every time an event is + triggered. + A global variable "<filename>e</filename>" is defined and + "<filename>e.data</filename>" contains an instance of + "<filename>bb.data</filename>". + With the <filename>getName(e)</filename> method, one can get + the name of the triggered event. + </para> + + <para> + During a standard build, the following common events might occur: + <itemizedlist> + <listitem><para> + <filename>bb.event.ConfigParsed()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.ParseStarted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.ParseProgress()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.ParseCompleted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.BuildStarted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.build.TaskStarted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.build.TaskInvalid()</filename> + </para></listitem> + <listitem><para> + <filename>bb.build.TaskFailedSilent()</filename> + </para></listitem> + <listitem><para> + <filename>bb.build.TaskFailed()</filename> + </para></listitem> + <listitem><para> + <filename>bb.build.TaskSucceeded()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.BuildCompleted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.cooker.CookerExit()</filename> + </para></listitem> + </itemizedlist> + Here is a list of other events that occur based on specific requests + to the server: + <itemizedlist> + <listitem><para> + <filename>bb.event.TreeDataPreparationStarted()</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.TreeDataPreparationProgress</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.TreeDataPreparationCompleted</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.DepTreeGenerated</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.CoreBaseFilesFound</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.ConfigFilePathFound</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.FilesMatchingFound</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.ConfigFilesFound</filename> + </para></listitem> + <listitem><para> + <filename>bb.event.TargetsTreeGenerated</filename> + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='variants-class-extension-mechanism'> + <title>Variants - Class Extension Mechanism</title> + + <para> + BitBake supports two features that facilitate creating + from a single recipe file multiple incarnations of that + recipe file where all incarnations are buildable. + These features are enabled through the + <link linkend='var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></link> + and + <link linkend='var-BBVERSIONS'><filename>BBVERSIONS</filename></link> + variables. + <note> + The mechanism for this class extension is extremely + specific to the implementation. + Usually, the recipe's + <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>, + <link linkend='var-PN'><filename>PN</filename></link>, and + <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> + variables would need to be modified by the extension class. + For specific examples, see the OE-Core + <filename>native</filename>, <filename>nativesdk</filename>, + and <filename>multilib</filename> classes. + </note> + <itemizedlist> + <listitem><para><emphasis><filename>BBCLASSEXTEND</filename>:</emphasis> + This variable is a space separated list of classes used to "extend" the + recipe for each variant. + Here is an example that results in a second incarnation of the current + recipe being available. + This second incarnation will have the "native" class inherited. + <literallayout class='monospaced'> + BBCLASSEXTEND = "native" + </literallayout></para></listitem> + <listitem><para><emphasis><filename>BBVERSIONS</filename>:</emphasis> + This variable allows a single recipe to build multiple versions of a + project from a single recipe file. + You can also specify conditional metadata + (using the + <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link> + mechanism) for a single version, or an optionally named range of versions. + Here is an example: + <literallayout class='monospaced'> + BBVERSIONS = "1.0 2.0 git" + SRC_URI_git = "git://someurl/somepath.git" + + BBVERSIONS = "1.0.[0-6]:1.0.0+ \ 1.0.[7-9]:1.0.7+" + SRC_URI_append_1.0.7+ = "file://some_patch_which_the_new_versions_need.patch;patch=1" + </literallayout> + The name of the range defaults to the original version of the + recipe. + For example, in OpenEmbedded, the recipe file + <filename>foo_1.0.0+.bb</filename> creates a default name range + of <filename>1.0.0+</filename>. + This is useful because the range name is not only placed + into overrides, but it is also made available for the metadata to use + in the variable that defines the base recipe versions for use in + <filename>file://</filename> search paths + (<link linkend='var-FILESPATH'><filename>FILESPATH</filename></link>). + </para></listitem> + </itemizedlist> + </para> + </section> + + <section id='dependencies'> + <title>Dependencies</title> + + <para> + To allow for efficient operation given multiple processes + executing in parallel, BitBake handles dependencies at + the task level. + BitBake supports a robust method to handle these dependencies. + </para> + + <para> + This section describes several types of dependency mechanisms. + </para> + + <section id='dependencies-internal-to-the-bb-file'> + <title>Dependencies Internal to the <filename>.bb</filename> File</title> + + <para> + BitBake uses the <filename>addtask</filename> directive + to manage dependencies that are internal to a given recipe + file. + You can use the <filename>addtask</filename> directive to + indicate when a task is dependent on other tasks or when + other tasks depend on that recipe. + Here is an example: + <literallayout class='monospaced'> + addtask printdate after do_fetch before do_build + </literallayout> + In this example, the <filename>printdate</filename> task is + depends on the completion of the <filename>do_fetch</filename> + task. + And, the <filename>do_build</filename> depends on the completion + of the <filename>printdate</filename> task. + </para> + </section> + + <section id='build-dependencies'> + <title>Build Dependencies</title> + + <para> + BitBake uses the + <link linkend='var-DEPENDS'><filename>DEPENDS</filename></link> + variable to manage build time dependencies. + The "deptask" varflag for tasks signifies the task of each + item listed in <filename>DEPENDS</filename> that must + complete before that task can be executed. + Here is an example: + <literallayout class='monospaced'> + do_configure[deptask] = "do_populate_staging" + </literallayout> + In this example, the <filename>do_populate_staging</filename> + task of each item in <filename>DEPENDS</filename> must complete before + <filename>do_configure</filename> can execute. + </para> + </section> + + <section id='runtime-dependencies'> + <title>Runtime Dependencies</title> + + <para> + BitBake uses the + <link linkend='var-PACKAGES'><filename>PACKAGES</filename></link>, + <link linkend='var-RDEPENDS'><filename>RDEPENDS</filename></link>, and + <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link> + variables to manage runtime dependencies. + </para> + + <para> + The <filename>PACKAGES</filename> variable lists runtime + packages. + Each of those packages can have <filename>RDEPENDS</filename> and + <filename>RRECOMMENDS</filename> runtime dependencies. + The "rdeptask" flag for tasks is used to signify the task of each + item runtime dependency which must have completed before that + task can be executed. + <literallayout class='monospaced'> + do_package_write[rdeptask] = "do_package" + </literallayout> + In the previous example, the <filename>do_package</filename> + task of each item in <filename>RDEPENDS</filename> must have + completed before <filename>do_package_write</filename> can execute. + </para> + </section> + + <section id='recursive-dependencies'> + <title>Recursive Dependencies</title> + + <para> + BitBake uses the "recrdeptask" flag to manage + recursive task dependencies. + BitBake looks through the build-time and runtime + dependencies of the current recipe, looks through + the task's inter-task + dependencies, and then adds dependencies for the + listed task. + Once BitBake has accomplished this, it recursively works through + the dependencies of those tasks. + Iterative passes continue until all dependencies are discovered + and added. + </para> + + <para> + You might want to not only have BitBake look for + dependencies of those tasks, but also have BitBake look + for build-time and runtime dependencies of the dependent + tasks as well. + If that is the case, you need to reference the task name + itself in the task list: + <literallayout class='monospaced'> + do_a[recrdeptask] = "do_a do_b" + </literallayout> + </para> + </section> + + <section id='inter-task-dependencies'> + <title>Inter-Task Dependencies</title> + + <para> + BitBake uses the "depends" flag in a more generic form + to manage inter-task dependencies. + This more generic form allows for inter-dependency + checks for specific tasks rather than checks for + the data in <filename>DEPENDS</filename>. + Here is an example: + <literallayout class='monospaced'> + do_patch[depends] = "quilt-native:do_populate_staging" + </literallayout> + In this example, the <filename>do_populate_staging</filename> + task of the target <filename>quilt-native</filename> + must have completed before the + <filename>do_patch</filename> task can execute. + </para> + + <para> + The "rdepends" flag works in a similar way but takes targets + in the runtime namespace instead of the build-time dependency + namespace. + </para> + </section> + </section> + + <section id='accessing-datastore-variables-using-python'> + <title>Accessing Datastore Variables Using Python</title> + + <para> + It is often necessary to access variables in the + BitBake datastore using Python functions. + The Bitbake datastore has an API that allows you this + access. + Here is a list of available operations: + </para> + + <para> + <informaltable frame='none'> + <tgroup cols='2' align='left' colsep='1' rowsep='1'> + <colspec colname='c1' colwidth='1*'/> + <colspec colname='c2' colwidth='1*'/> + <thead> + <row> + <entry align="left"><emphasis>Operation</emphasis></entry> + <entry align="left"><emphasis>Description</emphasis></entry> + </row> + </thead> + <tbody> + <row> + <entry align="left"><filename>d.getVar("X", expand=False)</filename></entry> + <entry align="left">Returns the value of variable "X". + Using "expand=True" expands the value.</entry> + </row> + <row> + <entry align="left"><filename>d.setVar("X", "value")</filename></entry> + <entry align="left">Sets the variable "X" to "value".</entry> + </row> + <row> + <entry align="left"><filename>d.appendVar("X", "value")</filename></entry> + <entry align="left">Adds "value" to the end of the variable "X".</entry> + </row> + <row> + <entry align="left"><filename>d.prependVar("X", "value")</filename></entry> + <entry align="left">Adds "value" to the start of the variable "X".</entry> + </row> + <row> + <entry align="left"><filename>d.delVar("X")</filename></entry> + <entry align="left">Deletes the variable "X" from the datastore.</entry> + </row> + <row> + <entry align="left"><filename>d.renameVar("X", "Y")</filename></entry> + <entry align="left">Renames the variable "X" to "Y".</entry> + </row> + <row> + <entry align="left"><filename>d.getVarFlag("X", flag, expand=False)</filename></entry> + <entry align="left">Gets then named flag from the variable "X". + Using "expand=True" expands the named flag.</entry> + </row> + <row> + <entry align="left"><filename>d.setVarFlag("X", flag, "value")</filename></entry> + <entry align="left">Sets the named flag for variable "X" to "value".</entry> + </row> + <row> + <entry align="left"><filename>d.appendVarFlag("X", flag, "value")</filename></entry> + <entry align="left">Appends "value" to the named flag on the + variable "X".</entry> + </row> + <row> + <entry align="left"><filename>d.prependVarFlag("X", flag, "value")</filename></entry> + <entry align="left">Prepends "value" to the named flag on + the variable "X".</entry> + </row> + <row> + <entry align="left"><filename>d.delVarFlag("X", flag)</filename></entry> + <entry align="left">Deletes the named flag on the variable + "X" from the datastore.</entry> + </row> + <row> + <entry align="left"><filename>d.setVarFlags("X", flagsdict)</filename></entry> + <entry align="left">Sets the flags specified in + the <filename>flagsdict()</filename> parameter. + <filename>setVarFlags</filename> does not clear previous flags. + Think of this operation as <filename>addVarFlags</filename>.</entry> + </row> + <row> + <entry align="left"><filename>d.getVarFlags("X")</filename></entry> + <entry align="left">Returns a <filename>flagsdict</filename> of the flags for + the variable "X".</entry> + </row> + <row> + <entry align="left"><filename>d.delVarFlags("X")</filename></entry> + <entry align="left">Deletes all the flags for the variable "X".</entry> + </row> + </tbody> + </tgroup> + </informaltable> + </para> + </section> + + <section id='task-checksums-and-setscene'> + <title>Task Checksums and Setscene</title> + + <para> + BitBake uses checksums (or signatures) along with the setscene + to determine if a task needs to be run. + This section describes the process. + To help understand how BitBake does this, the section assumes an + OpenEmbedded metadata-based example. + </para> + + <para> + This list is a place holder of content existed from previous work + on the manual. + Some or all of it probably needs integrated into the subsections + that make up this section. + For now, I have just provided a short glossary-like description + for each variable. + Ultimately, this list goes away. + <itemizedlist> + <listitem><para><filename>STAMP</filename>: + The base path to create stamp files.</para></listitem> + <listitem><para><filename>STAMPCLEAN</filename> + Again, the base path to create stamp files but can use wildcards + for matching a range of files for clean operations. + </para></listitem> + <listitem><para><filename>BB_STAMP_WHITELIST</filename> + Lists stamp files that are looked at when the stamp policy + is "whitelist". + </para></listitem> + <listitem><para><filename>BB_STAMP_POLICY</filename> + Defines the mode for comparing timestamps of stamp files. + </para></listitem> + <listitem><para><filename>BB_HASHCHECK_FUNCTION</filename> + Specifies the name of the function to call during + the "setscene" part of the task's execution in order + to validate the list of task hashes. + </para></listitem> + <listitem><para><filename>BB_SETSCENE_VERIFY_FUNCTION</filename> + Specifies a function to call that verifies the list of + planned task execution before the main task execution + happens. + </para></listitem> + <listitem><para><filename>BB_SETSCENE_DEPVALID</filename> + Specifies a function BitBake calls that determines + whether BitBake requires a setscene dependency to + be met. + </para></listitem> + <listitem><para><filename>BB_TASKHASH</filename> + Within an executing task, this variable holds the hash + of the task as returned by the currently enabled + signature generator. + </para></listitem> + </itemizedlist> + </para> + </section> +</chapter> |