diff options
Diffstat (limited to 'handbook/ref-bitbake.xml')
| -rw-r--r-- | handbook/ref-bitbake.xml | 340 |
1 files changed, 340 insertions, 0 deletions
diff --git a/handbook/ref-bitbake.xml b/handbook/ref-bitbake.xml new file mode 100644 index 0000000000..8652424466 --- /dev/null +++ b/handbook/ref-bitbake.xml @@ -0,0 +1,340 @@ +<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" +"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> + +<appendix id='ref-bitbake'> + + <title>Reference: Bitbake</title> + + <para> + Bitbake a program written in Python which interprets the metadata + that makes up Poky. At some point, people wonder what actually happens + when you type <command>bitbake poky-image-sato</command>. This section + aims to give an overview of what happens behind the scenes from a + BitBake perspective. + </para> + + <para> + It is worth noting that bitbake aims to be a generic "task" executor + capable of handling complex dependency relationships. As such it has no + real knowledge of what the tasks its executing actually do. It just + considers a list of tasks with dependencies and handles metadata + consisting of variables in a certain format which get passed to the + tasks. + </para> + + <section id='ref-bitbake-parsing'> + <title>Parsing</title> + + <para> + The first thing BitBake does is work out its configuration by + looking for a file called <filename>bitbake.conf</filename>. + Bitbake searches through the <varname>BBPATH</varname> environment + variable looking for a <filename class="directory">conf/</filename> + directory containing a <filename>bitbake.conf</filename> file and + adds the first <filename>bitbake.conf</filename> file found in + <varname>BBPATH</varname> (similar to the PATH environment variable). + For Poky, <filename>bitbake.conf</filename> is found in <filename + class="directory">meta/conf/</filename>. + </para> + + <para> + In Poky, <filename>bitbake.conf</filename> lists other configuration + files to include from a <filename class="directory">conf/</filename> + directory below the directories listed in <varname>BBPATH</varname>. + In general the most important configuration file from a user's perspective + is <filename>local.conf</filename>, which contains a users customized + settings for Poky. Other notable configuration files are the distribution + configuration file (set by the <glossterm><link linkend='var-DISTRO'> + DISTRO</link></glossterm> variable) and the machine configuration file + (set by the <glossterm><link linkend='var-MACHINE'>MACHINE</link> + </glossterm> variable). The <glossterm><link linkend='var-DISTRO'> + DISTRO</link></glossterm> and <glossterm><link linkend='var-MACHINE'> + MACHINE</link></glossterm> environment variables are both usually set in + the <filename>local.conf</filename> file. Valid distribution + configuration files are available in the <filename class="directory"> + meta/conf/distro/</filename> directory and valid machine configuration + files in the <filename class="directory">meta/conf/machine/</filename> + directory. Within the <filename class="directory"> + meta/conf/machine/include/</filename> directory are various <filename> + tune-*.inc</filename> configuration files which provide common + "tuning" settings specific to and shared between particular + architectures and machines. + </para> + + <para> + After the parsing of the configuration files some standard classes + are included. In particular, <filename>base.bbclass</filename> is + always included, as will any other classes + specified in the configuration using the <glossterm><link + linkend='var-INHERIT'>INHERIT</link></glossterm> + variable. Class files are searched for in a classes subdirectory + under the paths in <varname>BBPATH</varname> in the same way as + configuration files. + </para> + + <para> + After the parsing of the configuration files is complete, the + variable <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> + is set, usually in + <filename>local.conf</filename>, and defines the list of places to search for + <filename class="extension">.bb</filename> files. By + default this specifies the <filename class="directory">meta/packages/ + </filename> directory within Poky, but other directories such as + <filename class="directory">meta-extras/</filename> can be included + too. If multiple directories are specified a system referred to as + <link linkend='usingpoky-changes-collections'>"collections"</link> is used to + determine which files have priority. + </para> + + <para> + Bitbake parses each <filename class="extension">.bb</filename> file in + <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> and + stores the values of various variables. In summary, for each + <filename class="extension">.bb</filename> + file the configuration + base class of variables are set, followed + by the data in the <filename class="extension">.bb</filename> file + itself, followed by any inherit commands that + <filename class="extension">.bb</filename> file might contain. + </para> + + <para> + Parsing <filename class="extension">.bb</filename> files is a time + consuming process, so a cache is kept to speed up subsequent parsing. + This cache is invalid if the timestamp of the <filename class="extension">.bb</filename> + file itself has changed, or if the timestamps of any of the include, + configuration or class files the <filename class="extension">.bb</filename> + file depends on have changed. + </para> + </section> + + <section id='ref-bitbake-providers'> + <title>Preferences and Providers</title> + + <para> + Once all the <filename class="extension">.bb</filename> files have been + parsed, BitBake will proceed to build "poky-image-sato" (or whatever was + specified on the commandline) and looks for providers of that target. + Once a provider is selected, BitBake resolves all the dependencies for + the target. In the case of "poky-image-sato", it would lead to + <filename>task-oh.bb</filename> and <filename>task-base.bb</filename> + which in turn would lead to packages like <application>Contacts</application>, + <application>Dates</application>, <application>BusyBox</application> + and these in turn depend on glibc and the toolchain. + </para> + + <para> + Sometimes a target might have multiple providers and a common example + is "virtual/kernel" that is provided by each kernel package. Each machine + will often elect the best provider of its kernel with a line like the + following in the machine configuration file: + </para> + <programlisting><glossterm><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></glossterm>_virtual/kernel = "linux-rp"</programlisting> + <para> + The default <glossterm><link linkend='var-PREFERRED_PROVIDER'> + PREFERRED_PROVIDER</link></glossterm> is the provider with the same name as + the target. + </para> + + <para> + Understanding how providers are chosen is complicated by the fact + multiple versions might be present. Bitbake defaults to the highest + version of a provider by default. Version comparisons are made using + the same method as Debian. The <glossterm><link + linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm> + variable can be used to specify a particular version + (usually in the distro configuration) but the order can + also be influenced by the <glossterm><link + linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> + variable. By default files + have a preference of "0". Setting the + <glossterm><link + linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> to "-1" will + make a package unlikely to be used unless it was explicitly referenced and + "1" makes it likely the package will be used. + <glossterm><link + linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm> overrides + any default preference. <glossterm><link + linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> + is often used to mark more + experimental new versions of packages until they've undergone sufficient + testing to be considered stable. + </para> + + <para> + The end result is that internally, BitBake has now built a list of + providers for each target it needs in order of priority. + </para> + </section> + + <section id='ref-bitbake-dependencies'> + <title>Dependencies</title> + + <para> + Each target BitBake builds consists of multiple tasks (e.g. fetch, + unpack, patch, configure, compile etc.). For best performance on + multi-core systems, BitBake considers each task as an independent + entity with a set of dependencies. There are many variables that + are used to signify these dependencies and more information can be found + found about these in the <ulink url='http://bitbake.berlios.de/manual/'> + BitBake manual</ulink>. At a basic level it is sufficient to know + that BitBake uses the <glossterm><link + linkend='var-DEPENDS'>DEPENDS</link></glossterm> and + <glossterm><link linkend='var-RDEPENDS'>RDEPENDS</link></glossterm> variables when + calculating dependencies and descriptions of these variables are + available through the links. + </para> + + </section> + + <section id='ref-bitbake-tasklist'> + <title>The Task List</title> + + <para> + Based on the generated list of providers and the dependency information, + BitBake can now calculate exactly which tasks it needs to run and in what + order. The build now starts with BitBake forking off threads up to + the limit set in the <glossterm><link + linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></glossterm> variable + as long there are tasks ready to run, i.e. tasks with all their + dependencies met. + </para> + + <para> + As each task completes, a timestamp is written to the directory + specified by the <glossterm><link + linkend='var-STAMPS'>STAMPS</link></glossterm> variable (usually + <filename class="directory">build/tmp/stamps/*/</filename>). On + subsequent runs, BitBake looks at the <glossterm><link + linkend='var-STAMPS'>STAMPS</link></glossterm> + directory and will not rerun + tasks its already completed unless a timestamp is found to be invalid. + Currently, invalid timestamps are only considered on a per <filename + class="extension">.bb</filename> file basis so if for example the configure stamp has a timestamp greater than the + compile timestamp for a given target the compile task would rerun but this + has no effect on other providers depending on that target. This could + change or become configurable in future versions of BitBake. Some tasks + are marked as "nostamp" tasks which means no timestamp file will be written + and the task will always rerun. + </para> + + <para>Once all the tasks have been completed BitBake exits.</para> + + </section> + + <section id='ref-bitbake-runtask'> + <title>Running a Task</title> + + <para> + It's worth noting what BitBake does to run a task. A task can either + be a shell task or a python task. For shell tasks, BitBake writes a + shell script to <filename>${WORKDIR}/temp/run.do_taskname.pid</filename> + and then executes the script. The generated + shell script contains all the exported variables, and the shell functions + with all variables expanded. Output from the shell script is + sent to the file <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>. + Looking at the + expanded shell functions in the run file and the output in the log files + is a useful debugging technique. + </para> + + <para> + Python functions are executed internally to BitBake itself and + logging goes to the controlling terminal. Future versions of BitBake will + write the functions to files in a similar way to shell functions and + logging will also go to the log files in a similar way. + </para> + </section> + + + <section id='ref-bitbake-commandline'> + <title>Commandline</title> + + <para> + To quote from "bitbake --help": + </para> + + <screen>Usage: bitbake [options] [package ...] + +Executes the specified task (default is 'build') for a given set of BitBake files. +It expects that BBFILES is defined, which is a space separated list of files to +be executed. BBFILES does support wildcards. +Default BBFILES are the .bb files in the current directory. + +Options: + --version show program's version number and exit + -h, --help show this help message and exit + -b BUILDFILE, --buildfile=BUILDFILE + execute the task against this .bb file, rather than a + package from BBFILES. + -k, --continue continue as much as possible after an error. While the + target that failed, and those that depend on it, + cannot be remade, the other dependencies of these + targets can be processed all the same. + -f, --force force run of specified cmd, regardless of stamp status + -i, --interactive drop into the interactive mode also called the BitBake + shell. + -c CMD, --cmd=CMD Specify task to execute. Note that this only executes + the specified task for the providee and the packages + it depends on, i.e. 'compile' does not implicitly call + stage for the dependencies (IOW: use only if you know + what you are doing). Depending on the base.bbclass a + listtasks tasks is defined and will show available + tasks + -r FILE, --read=FILE read the specified file before bitbake.conf + -v, --verbose output more chit-chat to the terminal + -D, --debug Increase the debug level. You can specify this more + than once. + -n, --dry-run don't execute, just go through the motions + -p, --parse-only quit after parsing the BB files (developers only) + -d, --disable-psyco disable using the psyco just-in-time compiler (not + recommended) + -s, --show-versions show current and preferred versions of all packages + -e, --environment show the global or per-package environment (this is + what used to be bbread) + -g, --graphviz emit the dependency trees of the specified packages in + the dot syntax + -I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS + Stop processing at the given list of dependencies when + generating dependency graphs. This can help to make + the graph more appealing + -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS + Show debug logging for the specified logging domains + -P, --profile profile the command and print a report</screen> + + </section> + + <section id='ref-bitbake-fetchers'> + <title>Fetchers</title> + + <para> + As well as the containing the parsing and task/dependency handling + code, bitbake also contains a set of "fetcher" modules which allow + fetching of source code from various types of sources. Example + sources might be from disk with the metadata, from websites, from + remote shell accounts or from SCM systems like cvs/subversion/git. + </para> + + <para> + The fetchers are usually triggered by entries in + <glossterm><link linkend='var-SRC_URI'>SRC_URI</link></glossterm>. Information about the + options and formats of entries for specific fetchers can be found in the + <ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>. + </para> + + <para> + One useful feature for certain SCM fetchers is the ability to + "auto-update" when the upstream SCM changes version. Since this + requires certain functionality from the SCM only certain systems + support it, currently Subversion, Bazaar and to a limited extent, Git. It + works using the <glossterm><link linkend='var-SRCREV'>SRCREV</link> + </glossterm> variable. See the <link linkend='platdev-appdev-srcrev'> + developing with an external SCM based project</link> section for more + information. + </para> + + </section> + +</appendix> +<!-- +vim: expandtab tw=80 ts=4 spell spelllang=en_gb +--> |