aboutsummaryrefslogtreecommitdiffstats
path: root/doc/user-manual/user-manual-intro.xml
blob: f349e67856030c8adb0679f62fba9e270a37ca59 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
    "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
<chapter id="user-manual-intro">
  <title>BitBake User Manual</title>
  <section id="intro">
    <title>Introduction</title>
    <para>At it's core, BitBake is a tool generic task execution engine
	    allowing shell and python tasks to be executed efficiently and in
	    parallel whilst working within complex inter-task dependency
	    constraints.
	    One of it's main users, OpenEmbedded, takes this core and uses it
	    to build embedded Linux software stacks using a task oriented
	    approach.
	    Conceptually, BitBake is similar to GNU Make in some regards but
	    also has significant differences.
	    BitBake executes tasks based upon the metadata provided to it that
	    build up tasks.
	    It also includes a fetcher library for obtaining source code from
	    various places such as source control systems or websites.
	    Metadata is stored in recipe (.bb), configuration (.conf), and
	    class (.bbclass) files and provide to BitBake the instructions
	    about which tasks to run and the dependencies between them.
	    The instructions for each unit to be built (e.g. a piece of
	    software) are known as recipe files and contain all the
	    information about the unit (dependencies, source file locations,
	    checksums, description and so on).
	    BitBake includes a client/server abstraction and can be used from
	    a commandline or as a service over XMLRPC and has several 
	    different user interfaces.
    </para>
  </section>
  
  <section id="history-and-goals">
    <title>History and Goals</title>
    <para>BitBake was originally a part of the OpenEmbedded project.
	    It was inspired by the Portage package management system used by
	    the Gentoo Linux distribution.
	    On December 7, 2004, OpenEmbedded project team member, Chris 
	    Larson split the project into two distinct pieces: 
    <itemizedlist>
      <listitem>BitBake, a generic task executor</listitem>
      <listitem>OpenEmbedded, a metadata set utilized by BitBake.</listitem>
    </itemizedlist>
    Today, BitBake is the primary basis of the 
    <ulink url="http://www.openembedded.org/"> OpenEmbedded</ulink>
    project, which is being used to build and
    maintain a number of projects and embedded Linux distributions such as the
    Angstrom Distribution and the Yocto Project.
    </para>
    
    <para>Prior to BitBake, no other build tool adequately met the needs of an
	    aspiring embedded Linux distribution.
	    All of the buildsystems used by traditional desktop Linux
	    distributions lacked important functionality, and none of the
	    ad-hoc <emphasis>buildroot</emphasis> systems, prevalent in the
	    embedded space, were scalable or maintainable.
    </para>
    
    <para>Some important original goals for BitBake were: 
    <itemizedlist>
      <listitem>
        <para>Handle crosscompilation.</para>
      </listitem>
      <listitem>
	      <para>Handle interpackage dependencies (build time on target 
		      architecture, build time on native architecture, and
		      runtime).
	      </para>
      </listitem>
      <listitem>
	      <para>Support running any number of tasks within a given package,
		      including, but not limited to, fetching upstream sources,
		      unpacking them, patching them, configuring them, etc.
	      </para>
      </listitem>
      <listitem>
	      <para>Must be Linux distribution agnostic (both build and target).
	      </para>
      </listitem>
      <listitem>
	      <para>Must be architecture agnostic
	      </para>
      </listitem>
      <listitem>
	      <para>Must support multiple build and target operating systems 
		      (including Cygwin, the BSDs, etc).
	      </para>
      </listitem>
      <listitem>
	      <para>Must be able to be self contained, rather than tightly 
		      integrated into the build machine's root filesystem.
	      </para>
      </listitem>
      <listitem>
	      <para>There must be a way to handle conditional metadata (on
		      target architecture, operating system, distribution,
		      machine).
	      </para>
      </listitem>
      <listitem>
	      <para>It must be easy for the person using the tools to supply 
		      their own local metadata and packages to operate against.
	      </para>
      </listitem>
      <listitem>
	      <para>Must make it easy to collaborate between multiple projects
		      using BitBake for their builds.
	      </para>
      </listitem>
      <listitem>
	      <para>Should provide an inheritance mechanism to share common 
		      metadata between many packages.
	      </para>
      </listitem>
    </itemizedlist></para>
    <para>Over time it has become apparent that some further 
	    requirements were necessary: 
    </para>
    <itemizedlist>
	    <listitem>
		    <para>Handle variants of a base recipe (native, sdk,
			    multilib).
		    </para>
	    </listitem>
	    <listitem>
		    <para>Able to split metadata into layers and allow
			    layers to override each other.
		    </para>
	    </listitem>
	    <listitem>
		    <para>Allow representation of a given set of input
			    variables to a task as a checksum.
			    Based on that checksum, allow accelleration
			    of builds with prebuilt components.
		    </para>
	    </listitem>
     </itemizedlist>
     <para>BitBake satisfies all the original requirements and many
	     more with extensions being made to the basic functionality to
	     reflect the additionl requirements.
	     Flexibility and power have always been the priorities.
	     It is highly extensible, supporting embedded Python code and
	     execution of any arbitrary tasks.
     </para>
     </section>
     
     <section id="Concepts">
	     <title>Concepts</title>
	     <para>BitBake is a program written in the Python language.
		     At the highest level, BitBake interprets metadata,
		     decides what tasks are required to run, and executes
		     those tasks.
		     Similar to GNU Make, BitBake controls how software is
		     built.
		     GNU Make does this using "makefiles".
		     BitBake uses "recipes".
		     BitBake extends the capabilities of a simple tool like
		     GNU Make by allowing for much more complex tasks to be
		     completed, such as assembling entire embedded Linux
		     distributions.
		     Several concepts must be understood to be able to leverage
		     the power of the tool.
	     </para>
    </section>
    
    <section>
	    <title>Recipes</title>
	    <para>A BitBake Recipe, denoted by the file extension .bb is
		    the most basic metadata file.
		    For example, in OpenEmbedded recipes tell BitBake: 
		    <itemizedlist>
			    <listitem>Descriptive information about the
				    package.
			    </listitem>
			    <listitem>The version of the recipe.
			    </listitem>
			    <listitem>If there are any dependencies.
			    </listitem>
			    <listitem>Where the source code resides.
				    </listitem>
				    <listitem>Whether the source code requires
					    any patches.</listitem>
				    <listitem>How to compile the source code.
				    </listitem>
				    <listitem>Where to install the package
					    being compiled on the target
					    machine.</listitem>
		    </itemizedlist>
	    </para>
	    <para>Within the context of BitBake, or any project utilizing
		    BitBake as it's build system, files with the .bb extension
		    are referred to as recipes.
		    The term "package" is also commonly used to describe
		    recipes. 
		    However, since the same word is used to describe packaged
		    output from a project, it is best to maintain a single
		    descriptive term, "recipes".
	    </para>
    </section>
    <section>
	    <title>Configuration Files</title>
	    <para>Configuration Files, denoted by the .conf extension
		    define various configuration variables that govern the
		    project build process.
		    These files fall into several areas that define machine
		    configuration options, distribution configuration options,
		    compiler tuning options, general common configuration
		    options and user configuration options.
		    The main configuration file is the sample bitbake.conf 
		    file, located within the bitbake source tree /conf 
		    directory.
	    </para>
    </section>
    <section>
	    <title>Classes</title>
	    <para>Class files, denoted by the .bbclass extension contain
		    information that is useful to share between metadata files.
		    The BitBake source tree comes with one class metadata file
		    currently, called base.bbclass and it is found in the 
		    /classes directory.
		    The base.bbclass is special in that any new classes that
		    a developer adds to a project is required to inherit it
		    automatically.
		    This class contains definitions for standard basic tasks
		    such as fetching, unpacking, configuring (empty by 
		    default), compiling (runs any Makefile present), installing
		    (empty by default) and packaging (empty by default).
		    These classes are often overridden or extended by other
		    classes added during the project development process.
	    </para>
    </section>
    <section>
	    <title>Obtaining BitBake</title>
	    <para>There are several ways to obtain BitBake.
		    These include installing using your Linux distribution's
		    package management systemi (not recommended), downloading
		    a snapshot from the BitBake source code repository, or 
		    using Git to clone the BitBake source code repository.
		    The recommended method for daily BitBake use is to
		    download a stable release from the BitBake source code
		    repository.
		    Using your distribution's version as provided inthe package
		    management system is generally not recommended as in most
		    cases, such as with the Ubuntu and Fedora distributions,
		    the version provided is several releases behind the 
		    repository snapshot version and is missing important bug
		    fixes and enhancements.
		    Similarly, daily use of the latest clone of the Git 
		    repository is not recommended as it can be unstable.
		    However, the Git repository clone will provide the
		    User with the absolute latest version of BitBake.
	    </para>
    </section>
    <section>
	    <title>Downloading a Snapshot from the BitBake Source Tree</title>
	    <para>The recommended method for obtaining and using BitBake on a 
		    daily basis is to download the most recent stable snapshot
		    from the Git source code repository as follows: 
		    <screen>$ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz
			    $ tar zxpvf bitbake-1.17.0.tar.gz
		    </screen>
		    After extraction of the tarball using the tar utility, you
		    will have a directory entitled 
		    <emphasis>bitbake-1.17.0
	    </emphasis>.
    </para>
    </section>
    <section>
	    <title>Cloning the BitBake Git Repository</title>
	    <para>To obtain the latest BitBake source code from the BitBake
		    Git repository: 
		    <screen>$ git clone git://git.openembedded.org/bitbake
		    </screen>
		    This will clone the BitBake Git repository into a directory
		    called <emphasis>bitbake</emphasis>. 
		    Alternatively, you can designate a directory after the git 
		    clone command if you'd prefer to call the new directory
		    something other than <emphasis>bitbake</emphasis>.
		    For example:
		    <screen>$ git clone git://git.openembedded.org/bitbake
			    bbdev
		    </screen>
		    This would clone the Git repository into a local directory 
		    called <emphasis>bbdev</emphasis>. 
		    Please note that although this method of obtaining the
		    source code will provide the absolute latest version, it is
		    under active development and may not be as stable as a 
		    released snapshot.
	    </para>
    </section>
    
    <section>
    <title>Summary</title>
    <para>At this point you should have a general idea of the concepts that
	    BitBake was built on and how the source code is organized.
	    You should have a working version of BitBake installed and 
	    understand how to setup your environment.
    </para>

    <para>In the next chapter, a simple Hello World example is used to 
	    demonstrate the BitBake concepts.</para>
  </section>
</chapter>