At it's core, BitBake is a tool generic task execution engine allowing shell and python tasks to be executed efficiently and in parallel whilst working within complex inter-task dependency constraints. One of it's main users, OpenEmbedded, takes this core and uses it to build embedded Linux software stacks using a task oriented approach. Conceptually, BitBake is similar to GNU Make in some regards but also has significant differences. BitBake executes tasks based upon the metadata provided to it that build up tasks. It also includes a fetcher library for obtaining source code from various places such as source control systems or websites. Metadata is stored in recipe (.bb), configuration (.conf), and class (.bbclass) files and provide to BitBake the instructions about which tasks to run and the dependencies between them. The instructions for each unit to be built (e.g. a piece of software) are known as recipe files and contain all the information about the unit (dependencies, source file locations, checksums, description and so on). BitBake includes a client/server abstraction and can be used from a commandline or as a service over XMLRPC and has several different user interfaces.

BitBake was originally a part of the OpenEmbedded project. It was inspired by the Portage package management system used by the Gentoo Linux distribution. On December 7, 2004, OpenEmbedded project team member, Chris Larson split the project into two distinct pieces: BitBake, a generic task executor OpenEmbedded, a metadata set utilized by BitBake. Today, BitBake is the primary basis of the OpenEmbedded project, which is being used to build and maintain a number of projects and embedded Linux distributions such as the Angstrom Distribution and the Yocto Project. Prior to BitBake, no other build tool adequately met the needs of an aspiring embedded Linux distribution. All of the buildsystems used by traditional desktop Linux distributions lacked important functionality, and none of the ad-hoc buildroot systems, prevalent in the embedded space, were scalable or maintainable. Some important original goals for BitBake were: Handle crosscompilation. Handle interpackage dependencies (build time on target architecture, build time on native architecture, and runtime). Support running any number of tasks within a given package, including, but not limited to, fetching upstream sources, unpacking them, patching them, configuring them, etc. Must be Linux distribution agnostic (both build and target). Must be architecture agnostic Must support multiple build and target operating systems (including Cygwin, the BSDs, etc). Must be able to be self contained, rather than tightly integrated into the build machine's root filesystem. There must be a way to handle conditional metadata (on target architecture, operating system, distribution, machine). It must be easy for the person using the tools to supply their own local metadata and packages to operate against. Must make it easy to collaborate between multiple projects using BitBake for their builds. Should provide an inheritance mechanism to share common metadata between many packages. Over time it has become apparent that some further requirements were necessary: Handle variants of a base recipe (native, sdk, multilib). Able to split metadata into layers and allow layers to override each other. Allow representation of a given set of input variables to a task as a checksum. Based on that checksum, allow accelleration of builds with prebuilt components. BitBake satisfies all the original requirements and many more with extensions being made to the basic functionality to reflect the additionl requirements. Flexibility and power have always been the priorities. It is highly extensible, supporting embedded Python code and execution of any arbitrary tasks.

BitBake is a program written in the Python language. At the highest level, BitBake interprets metadata, decides what tasks are required to run, and executes those tasks. Similar to GNU Make, BitBake controls how software is built. GNU Make does this using "makefiles". BitBake uses "recipes". BitBake extends the capabilities of a simple tool like GNU Make by allowing for much more complex tasks to be completed, such as assembling entire embedded Linux distributions. Several concepts must be understood to be able to leverage the power of the tool.

A BitBake Recipe, denoted by the file extension .bb is the most basic metadata file. For example, in OpenEmbedded recipes tell BitBake: Descriptive information about the package. The version of the recipe. If there are any dependencies. Where the source code resides. Whether the source code requires any patches. How to compile the source code. Where to install the package being compiled on the target machine. Within the context of BitBake, or any project utilizing BitBake as it's build system, files with the .bb extension are referred to as recipes. The term "package" is also commonly used to describe recipes. However, since the same word is used to describe packaged output from a project, it is best to maintain a single descriptive term, "recipes".

Configuration Files, denoted by the .conf extension define various configuration variables that govern the project build process. These files fall into several areas that define machine configuration options, distribution configuration options, compiler tuning options, general common configuration options and user configuration options. The main configuration file is the sample bitbake.conf file, located within the bitbake source tree /conf directory.

Class files, denoted by the .bbclass extension contain information that is useful to share between metadata files. The BitBake source tree comes with one class metadata file currently, called base.bbclass and it is found in the /classes directory. The base.bbclass is special in that any new classes that a developer adds to a project is required to inherit it automatically. This class contains definitions for standard basic tasks such as fetching, unpacking, configuring (empty by default), compiling (runs any Makefile present), installing (empty by default) and packaging (empty by default). These classes are often overridden or extended by other classes added during the project development process.

There are several ways to obtain BitBake. These include installing using your Linux distribution's package management systemi (not recommended), downloading a snapshot from the BitBake source code repository, or using Git to clone the BitBake source code repository. The recommended method for daily BitBake use is to download a stable release from the BitBake source code repository. Using your distribution's version as provided inthe package management system is generally not recommended as in most cases, such as with the Ubuntu and Fedora distributions, the version provided is several releases behind the repository snapshot version and is missing important bug fixes and enhancements. Similarly, daily use of the latest clone of the Git repository is not recommended as it can be unstable. However, the Git repository clone will provide the User with the absolute latest version of BitBake.

The recommended method for obtaining and using BitBake on a daily basis is to download the most recent stable snapshot from the Git source code repository as follows: $ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz $ tar zxpvf bitbake-1.17.0.tar.gz After extraction of the tarball using the tar utility, you will have a directory entitled bitbake-1.17.0 .

To obtain the latest BitBake source code from the BitBake Git repository: $ git clone git://git.openembedded.org/bitbake This will clone the BitBake Git repository into a directory called bitbake. Alternatively, you can designate a directory after the git clone command if you'd prefer to call the new directory something other than bitbake. For example: $ git clone git://git.openembedded.org/bitbake bbdev This would clone the Git repository into a local directory called bbdev. Please note that although this method of obtaining the source code will provide the absolute latest version, it is under active development and may not be as stable as a released snapshot.

At this point you should have a general idea of the concepts that BitBake was built on and how the source code is organized. You should have a working version of BitBake installed and understand how to setup your environment. In the next chapter, a simple Hello World example is used to demonstrate the BitBake concepts.