diff options
author | Jamie Lenehan <lenehan@twibble.org> | 2007-05-04 07:54:00 +0000 |
---|---|---|
committer | Jamie Lenehan <lenehan@twibble.org> | 2007-05-04 07:54:00 +0000 |
commit | 67619252389aa0b7937185dba5320bc21c5112a0 (patch) | |
tree | 2ef07de799ff77a8c7e4c070b3cac3c2c727b217 /usermanual/chapters | |
parent | 6c78836b83609a6be795c52d07c44ea27a54f6b7 (diff) | |
download | openembedded-67619252389aa0b7937185dba5320bc21c5112a0.tar.gz |
usermanual: Work some more on the using bitbake/OE chapter. Enable it -
while it's not great but I think there is probably a few useful items in
there for new users now.
Diffstat (limited to 'usermanual/chapters')
-rw-r--r-- | usermanual/chapters/recipes.xml | 2 | ||||
-rw-r--r-- | usermanual/chapters/usage.xml | 613 |
2 files changed, 404 insertions, 211 deletions
diff --git a/usermanual/chapters/recipes.xml b/usermanual/chapters/recipes.xml index a7904f2b58..4234a5c4cd 100644 --- a/usermanual/chapters/recipes.xml +++ b/usermanual/chapters/recipes.xml @@ -1,5 +1,5 @@ <?xml version="1.0" encoding="UTF-8"?> -<chapter id="chapter_recipes"> +<chapter id="chapter_recipes" xreflabel="Recipes chapter"> <title>Recipes</title> <section id="recipes_introduction" xreflabel="introduction"> diff --git a/usermanual/chapters/usage.xml b/usermanual/chapters/usage.xml index f2031b3346..e4f056a764 100644 --- a/usermanual/chapters/usage.xml +++ b/usermanual/chapters/usage.xml @@ -7,16 +7,17 @@ <para>If your reading this manual you probably already have some idea of what OpenEmbedded is all about, which is taking a lot of software and - creating something that you can run on another device. Its all about + creating something that you can run on another device. This involves downloading some source code, compiling it, creating packages (like .deb or .rpm) and/or creating boot images that can written to flash on the - device. The complexities of cross-compiling and the variety of devices to - be supported lead to a lot more complexity in on OpenEmbedded based - distribution than you'd find in a typical desktop distribution (which - doesn't get cross-compiled).</para> + device. The difficulties of cross-compiling and the variety of devices + which can be supported lead to a lot more complexity in an OpenEmbedded + based distribution than you'd find in a typical desktop distribution + (which cross-compiling isn't needed).</para> <para>A major part of OpenEmbedded deals with compiling source code for - various projects. For each project OpenEmbedded has to:</para> + various projects. For each project this generally requires the same basic + set of tasks:</para> <orderedlist> <listitem> @@ -44,43 +45,57 @@ </listitem> </orderedlist> - <para>As mentioned this is made for more complex than normal due - to:</para> + <para>There's nothing particular unusual about this process when building + on the machine the package is to be installed on. What makes this + difficult is:</para> <orderedlist> <listitem> <para>Cross-compiling: cross-compiling is difficult, and lots of - software has no support for cross-compiling. All packages included in + software has no support for cross-compiling - all packages included in OE are cross-compiled;</para> </listitem> <listitem> - <para>Due to cross-compiling the executable's that are created cannot - normally be run, so any software that tries to run things as part of - it's build process need to have changes made to handle this some other - way</para> + <para>Target and host are different: This means you can't compile up a + program and then run it - it's compiled to run on the target system, + not on the system compiling it. Lots of software tries to build and + run little helper and/or test applications and this won't work when + cross-compiling.</para> + </listitem> + + <listitem> + <para>Tool chains (compiler, linker etc) are often difficult to + compile. Cross tool chains are even more difficult. Typically you'd go + out and download a tool chain made by someone else - but not when your + using OE. In OE the entire toolchain is built as part of the process. + This may make things take longer initially and may make it more + difficult to get started but makes it easier to apply patches and test + out changes to the tool chain.</para> </listitem> </orderedlist> <para>Of course there's a lot more to OE then just compiling packages - though. Just some of the things that OE can handle:</para> + though. Some of the features that OE supports includes:</para> <itemizedlist> <listitem> - <para>Support for glibc and uclibc;</para> + <para>Support for both glibc and uclibc;</para> </listitem> <listitem> - <para>Support for building for multiple targets;</para> + <para>Support for building for multiple target devices from the one + code base;</para> </listitem> <listitem> - <para>Dependencies - automatically building anything that is required - for the package your really want;</para> + <para>Automatically building anything that is required for the package + to compile and/or run (build and run time dependencies);</para> </listitem> <listitem> - <para>Creation of flash and disk images for booting directly on the + <para>Creation of flash and disk images of any one of a number of + types (jffs2, ext2.gz, squashfs etc) for booting directly on the target device;</para> </listitem> @@ -89,29 +104,36 @@ </listitem> <listitem> - <para>Automatically creating all of the cross-compiling tools you'll + <para>Automatic building all of the cross-compiling tools you'll need;</para> </listitem> <listitem> <para>Support for "native" packages that are built for the host - computer and not for the target;</para> + computer and not for the target and used to help during the build + process;</para> </listitem> </itemizedlist> - <para>This chapter assumes you have master the Getting Start guides to - OpenEmbedded (see the OpenEmbedded web site for details), and therefore - have an appropriately configured setup and have managed to build the - cross-compilers for your target. This section talks you through some of - the background on what is happening with the aim of helping you understand - how to debug and develop within OpenEmbedded.</para> + <para>The rest of this chapter assumes you have mastered the Getting Start + guides to OpenEmbedded (see the OpenEmbedded web site for details), and + therefore have an appropriately configured setup and that you have managed + to actually build the cross-compilers for your target. This section talks + you through some of the background on what is happening with the aim of + helping you understand how to debug and develop within + OpenEmbedded.</para> + + <para>You'll also not a lot of reference to variables that define specific + directories or change the behaviour of some part of the build process. You + should refer to <xref linkend="chapter_recipes" /> for full details on + these variables.</para> </section> <section id="usage_workspace" xreflabel="workspace"> <title>Work space</title> - <para>Let's start out by taking a look at a typically working area OE. - Note that this isn't exactly what you'll see - there are a lot of options that + <para>Let's start out by taking a look at a typically working area. Note + that this may not be exactly what see - there are a lot of options that can effect exactly how things are done, but it gives us a pretty good idea of whats going on. What we are looking at here is the tmp directory (as specified by TMPDIR in your local.conf):<screen>%> find tmp -maxdepth 2 -type d @@ -240,43 +262,59 @@ tmp/deploy/images</screen></para> </varlistentry> </variablelist> - <para></para> + <para>When people refer to the <emphasis>"tmp directory"</emphasis> this + is the directory them are talking about.</para> + + <para>To perform a complete rebuild from script you would usually rename + or delete tmp and then restart your build. I recommend keeping one old + version of tmp around to use for comparison if something goes wrong with + your new build. For example:<screen>%> rm -fr tmp.OLD +$> mv tmp tmp.OLD +%> bitbake bootstrap-image</screen></para> <section id="usage_workdir" xreflabel="work directory"> - <title>work directory</title> + <title>work directory (tmp/work)</title> - <para>The working directory is where all files are extracted and - everything is configured, compiled and packaged. In other words this is - where all the action happens. Each bitbake recipe will produce a - corresponding directory in the working area, with the name containing - the recipe name, version and the release number (as defined by the PR - variable in the recipe).</para> + <para>The work directory is where all source code is unpacked into, + where source is configured, compiled and packaged. In other words this + is where all the action happens. Each bitbake recipe will produce a + corresponding sub directory in the work directory. The sub directory + name will contain the recipe name, version and the release number (as + defined by the PR variable within the recipe).</para> - <para>In the following example we show some of the directories that are - created and in this case they are created directly in the work + <para>Here's an example of a few of the subdirectories under the work directory:<screen>~%> find tmp/work -maxdepth 1 -type d | head -4 tmp/work tmp/work/busybox-1.2.1-r13 tmp/work/libice-1_1.0.3-r0 -tmp/work/arpwatch-2.1a15-r2</screen>Depending on your distribution settings - you may have an additional subdirectory present:<screen>~%> find tmp/work -maxdepth 2 -type d | head -4 +tmp/work/arpwatch-2.1a15-r2</screen>You can see that the first three (of + several hundred) recipes here and they are for release 13 of busybox + 1.2.1, release 0 or libice 1.1.0.3 and release 2 of arpwatch 2.1a15. + It's also possible that you may just have a sub directory for your + targets architecture and operating system in which case these + directories will be in that additional subdirectory, as shown + here:<screen>~%> find tmp/work -maxdepth 2 -type d | head -4 tmp/work tmp/work/sh4-linux tmp/work/sh4-linux/busybox-1.2.1-r13 tmp/work/sh4-linux/libice-1_1.0.3-r0 tmp/work/sh4-linux/arpwatch-2.1a15-r2</screen></para> - <para>where are additional directory corresponding to the target - architecture and OS has been inserted. This is added by the use of the + <para>The <emphasis role="bold">sh4-linux</emphasis> directory in the + above example is a combination of the target architecture (sh4) and + operating system (linux). This subdirectory has been added by the use of + one of OpenEmbedded's many features. In this case it's the <emphasis>multimachine</emphasis> feature which is used to allow builds - for multiple targets within the one work directory (which in turn - enables the sharing of native functionality and a reduction in the time - taken to build for multiple machines). We'll assume multimachine is not - being used for the rest of this chapter, just remember to add the extra - directory if your distribution is using it.</para> - - <para>Using lzo 1.08 as an example we'll examine the working directory - and see what it contain for a typical recipe:<screen>%> find tmp/work/lzo-1.08-r14 -maxdepth 1 + for multiple targets within the one work directory and can be enabled on + a per distribution basis. This feature enables the sharing of native and + architecture neutral packages and building for multiple targets that + support the same architecture but require different linux kernels (for + example). We'll assume multimachine isn't being used for the rest of + this chapter, just remember to add the extra directory if your + distribution is using it.</para> + + <para>Using lzo 1.08 as an example we'll examine the contents of the + working directory for a typical recipe:<screen>%> find tmp/work/lzo-1.08-r14 -maxdepth 1 tmp/work/lzo-1.08-r14 tmp/work/lzo-1.08-r14/temp tmp/work/lzo-1.08-r14/lzo-1.08 @@ -284,17 +322,18 @@ tmp/work/lzo-1.08-r14/install tmp/work/lzo-1.08-r14/image</screen></para> <para>The directory, <emphasis - role="bold">tmp/work/lzo-1.08-r14</emphasis>, is know as the working - directory for the recipe is held in the <emphasis - role="bold">WORKDIR</emphasis> variable in bitbake. You'll sometimes see - recipes refer directly to <emphasis role="bold">WORKDIR</emphasis> and - if so this is the directory they are referring to. The <emphasis - role="bold">1.08</emphasis> is the version of lzo and the <emphasis - role="bold">r14</emphasis> is the release number, as defined by the - <emphasis role="bold">PR</emphasis> variable in the recipe.</para> - - <para>Under <emphasis role="bold">WORKDIR</emphasis> there are four - directories:</para> + role="bold">tmp/work/lzo-1.08-r14</emphasis>, is know as the + <emphasis>"working directory"</emphasis> for the recipe and is specified + via the <emphasis role="bold">WORKDIR</emphasis> variable in bitbake. + You'll sometimes see recipes refer directly to <emphasis + role="bold">WORKDIR</emphasis> and this is the directory they are + referencing. The <emphasis role="bold">1.08</emphasis> is the version of + lzo and <emphasis role="bold">r14</emphasis> is the release number, as + defined by the <emphasis role="bold">PR</emphasis> variable within the + recipe.</para> + + <para>Under the working directory (<emphasis + role="bold">WORKDIR</emphasis>) there are four subdirectories:</para> <variablelist> <varlistentry> @@ -302,7 +341,8 @@ tmp/work/lzo-1.08-r14/image</screen></para> <listitem> <para>The temp directories contains logs and in some cases scripts - that actually implement specific tasks.</para> + that actually implement specific tasks (such as a script to + configure or compile the source).</para> <para>You can look at the logs in this directory to get more information into what happened (or didn't happen). This is usually @@ -324,50 +364,60 @@ tmp/work/lzo-1.08-r14/image</screen></para> the actual source code packaging. Within recipes this directory is referred to as <emphasis role="bold">S</emphasis> and is usually expected to be named like this, that is - <emphasis><name>-<version></emphasis>. If the source - code extracts to somewhere else you would need to manually specify - the value of <emphasis role="bold">S</emphasis> in your - recipe.</para> + <emphasis>"<name>-<version>"</emphasis>. If the source + code extracts to somewhere else then that would need to be + declared in the recipe by explicitly setting the value of the + variable <emphasis role="bold">S</emphasis> to the appropriate + directory.</para> </listitem> </varlistentry> <varlistentry> - <term>install</term> + <term>image</term> <listitem> - <para>The installation directory (or destination directory) is - where the software needs to be installed into in order to be - packaged. This directory is referred to as <emphasis - role="bold">D</emphasis> in recipes. So instead of installing into - <emphasis role="bold">/usr/bin</emphasis> and <emphasis + <para>The image directory (or destination directory) is where the + software needs to be installed into in order to be packaged. This + directory is referred to as <emphasis role="bold">D</emphasis> in + recipes. So instead of installing binaries into <emphasis + role="bold">/usr/bin</emphasis> and libraries into <emphasis role="bold">/usr/lib</emphasis> for example you would need to install into <emphasis role="bold">${D}/usr/bin</emphasis> and - <emphasis role="bold">${D}/usr/lib</emphasis> in order for the - packaging system to work.</para> + <emphasis role="bold">${D}/usr/lib</emphasis> instead. When + installed on the target the ${D} will be not be included so + they'll end up in the correct place. You definitely don't wont + files on your host system being replaced by cross-compiled + binaries for your target!</para> </listitem> </varlistentry> <varlistentry> - <term>image</term> + <term>install</term> <listitem> - <para>The image directory is used to split the installed files + <para>The install directory is used to split the installed files into separate packages. One subdirectory is created per package to - be generated and the files and moved from the install directory - (<emphasis role="bold">D</emphasis>) over to this directory as - each packaging instruction is processed. Typically there will be - separate documentation (<emphasis>-doc</emphasis>), debugging + be generated and the files are moved from the image directory + (<emphasis role="bold">D</emphasis>) over to this directory, and + into the appropriate package subdirectory, as each packaging + instruction is processed. Typically there will be separate + documentation (<emphasis>-doc</emphasis>), debugging (<emphasis>-dbg</emphasis>) and development - (<emphasis>-dev</emphasis>) packages automatically created.</para> + (<emphasis>-dev</emphasis>) packages automatically created. There + are variables such as <emphasis role="bold">FILES_</emphasis> and + <emphasis role="bold">PACKAGES</emphasis> used in recipes which + control the separation of various files into individual + packages.</para> </listitem> </varlistentry> </variablelist> - <para>The various variables that can be used in a recipe are described - in detail in the recipes chapter of this manual.</para> + <para>So lets show some examples of the useful information you now have + access to. </para> - <para>What can you do with this information? How about checking out what - happened during the configuration of lzo:<screen>~%> less tmp/work/lzo-1.08-r14/temp/log.do_configure.* + <para>How about checking out what happened during the configuration of + lzo? Well that requires checking the log file for configure that is + generated in the temp directory:<screen>~%> less tmp/work/lzo-1.08-r14/temp/log.do_configure.* ... checking whether ccache sh4-linux-gcc -ml -m4 suffers the -fschedule-insns bug... unknown checking whether ccache sh4-linux-gcc -ml -m4 suffers the -fstrength-reduce bug... unknown @@ -385,8 +435,10 @@ config.status: creating tests/Makefile config.status: creating config.h config.status: executing depfiles commands</screen></para> - <para>Or perhaps you want to see which files were included in each of - the generated packages:<screen>~%> find tmp/work/lzo-1.08-r14/install + <para>Or perhaps you want to see how the files were distributed into + individual packages prior to packaging? The install directory is where + the files are split into separate packages and so that shows us which + files end up where:<screen>~%> find tmp/work/lzo-1.08-r14/install tmp/work/lzo-1.08-r14/install tmp/work/lzo-1.08-r14/install/lzo-doc tmp/work/lzo-1.08-r14/install/lzo-dbg @@ -426,25 +478,154 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <section id="usage_tasks" xreflabel="tasks"> <title>Tasks</title> - <para>When you build a software package you generally perform a number of - steps or tasks. These would include things like "download the source - code", "unpack the source code", "run the configure script", "build the - software", "install the software" etc. OpenEmbedded builds software in a - similar way - by performing a set of tasks. Understanding these tasks is - critical to understanding how things get done in OpenEmbedded.</para> - - <para></para> - - <para>The following is a list of the most commonly seen tasks:</para> + <para>When you go about building and installing a software package there + are a number of tasks that you generally follow with most software + packages. You probably need to start out by downloading the source code, + then unpacking the source code. Maye you need to apply some patches for + some reason. Then you might run the configure script of the package, + perhaps passing it some options to configure it to your liking. The you + might run "make install" to install the software. If your actually going + to make some packages, such as .deb or .rpm, then you'd have additional + tasks you'd perform to make them.</para> + + <para>You find that building things in OpenEmbedded works in a similar way + - there are a number of tasks that are executed in a predefined order for + each recipe. Any many of the tasks correspond to those listed above like + <emphasis>"download the source"</emphasis>. In fact you've probably + already seen some of the names of these tasks - bitbake displays them as + they are processed:<screen>~%> bitbake lzo +NOTE: Psyco JIT Compiler (http://psyco.sf.net) not available. Install it to increase performance. +NOTE: Handling BitBake files: \ (4541/4541) [100 %] +NOTE: Parsing finished. 4325 cached, 0 parsed, 216 skipped, 0 masked. +NOTE: build 200705041709: started + +OE Build Configuration: +BB_VERSION = "1.8.2" +OE_REVISION = "<unknown>" +TARGET_ARCH = "sh4" +TARGET_OS = "linux" +MACHINE = "titan" +DISTRO = "erouter" +DISTRO_VERSION = "0.1-20070504" +TARGET_FPU = "" + +NOTE: Resolving missing task queue dependencies +NOTE: preferred version 2.5 of glibc not available (for item virtual/sh4-linux-libc-for-gcc) +NOTE: Preparing Runqueue +NOTE: Executing runqueue +NOTE: Running task 208 of 226 (ID: 11, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_fetch</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_fetch</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_fetch</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 209 of 226 (ID: 2, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_unpack</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_unpack</emphasis>: started +NOTE: Unpacking /home/lenehan/devel/oe/sources/lzo-1.08.tar.gz to /home/lenehan/devel/oe/build/titan-glibc-25/tmp/work/lzo-1.08-r14/ +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_unpack</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 216 of 226 (ID: 3, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_patch</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_patch</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_patch</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 217 of 226 (ID: 4, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_configure</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_configure</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_configure</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 218 of 226 (ID: 12, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_qa_configure</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_qa_configure</emphasis>: started +NOTE: Checking sanity of the config.log file +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_qa_configure</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 219 of 226 (ID: 0, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_compile</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_compile</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_compile</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 220 of 226 (ID: 1, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_install</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_install</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_install</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 221 of 226 (ID: 5, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_package</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_package</emphasis>: started +NOTE: DO PACKAGE QA +NOTE: Checking Package: lzo-dbg +NOTE: Checking Package: lzo +NOTE: Checking Package: lzo-doc +NOTE: Checking Package: lzo-dev +NOTE: Checking Package: lzo-locale +NOTE: DONE with PACKAGE QA +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_package</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 222 of 226 (ID: 8, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, <emphasis + role="bold">do_package_write</emphasis>) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_package_write</emphasis>: started +Packaged contents of lzo-dbg into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo-dbg_1.08-r14_sh4.ipk +Packaged contents of lzo into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo1_1.08-r14_sh4.ipk +NOTE: Not creating empty archive for lzo-doc-1.08-r14 +Packaged contents of lzo-dev into /home/lenehan/devel/oe/build/titan-glibc-25/tmp/deploy/ipk/sh4/liblzo-dev_1.08-r14_sh4.ipk +NOTE: Not creating empty archive for lzo-locale-1.08-r14 +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_package_write</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 223 of 226 (ID: 6, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_populate_staging) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_populate_staging</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_populate_staging</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 224 of 226 (ID: 9, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_qa_staging) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_qa_staging</emphasis>: started +NOTE: QA checking staging +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_qa_staging</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 225 of 226 (ID: 7, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_distribute_sources) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_distribute_sources</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_distribute_sources</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Running task 226 of 226 (ID: 10, /home/lenehan/devel/oe/build/titan-glibc-25/packages/lzo/lzo_1.08.bb, do_build) +NOTE: package lzo-1.08: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_build</emphasis>: started +NOTE: package lzo-1.08-r14: task <emphasis role="bold">do_build</emphasis>: completed +NOTE: package lzo-1.08: completed +NOTE: Tasks Summary: Attempted 226 tasks of which 213 didn't need to be rerun and 0 failed. +NOTE: build 200705041709: completed</screen><note> + <para>The output may look different depending on the version of + bitbake being used, and some tasks are only run when specific options + are enabled in your distribution. The important point to note is that + the various tasks are being run and bitbake shows you each time it + starts and completes a task.</para> + </note></para> + + <para>So there's a set of tasks here which are being run to generate the + final packages. And if you'll notice that every recipe runs through the + same set of tasks (ok I'll admit that it is possible that some additional + tasks could be run for some recipes, but we'll talk about that later). The + tasks that you'll need to be most familiar with are:</para> <variablelist> <varlistentry> <term>fetch</term> <listitem> - <para>The fetch task is responsible for fetching any source code - that is required. This means things such as downloading files and - checking out from svn or git repositories for example.</para> + <para>The <emphasis>fetch</emphasis> task is responsible for + fetching any source code that is required. This means things such as + downloading files and checking out from source control repositories + such as git or svn.</para> </listitem> </varlistentry> @@ -452,9 +633,11 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>unpack</term> <listitem> - <para>The unpack task is responsible for extracting files from - archives, such as .tar.gz, into the working area and copying any - additional files into the working area.</para> + <para>The <emphasis>unpack</emphasis> task is responsible for + extracting files from archives, such as <emphasis + role="bold">.tar.gz</emphasis>, into the working area and copying + any additional files, such as init scripts, into the working + area.</para> </listitem> </varlistentry> @@ -462,8 +645,8 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>patch</term> <listitem> - <para>The patch task is responsible for applying any patches to the - unpacked source code</para> + <para>The <emphasis>patch</emphasis> task is responsible for + applying any patches to the unpacked source code</para> </listitem> </varlistentry> @@ -471,10 +654,11 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>configure</term> <listitem> - <para>The configure task takes care of the configuration of the - package. Running a configure script ("./configure <options>") - is probably the form of configuration that is most recognised but - it's not the only configuration system that exists.</para> + <para>The <emphasis>configure</emphasis> task takes care of the + configuration of the package. Running a configure script + (<emphasis>"./configure <options>"</emphasis>) is probably the + form of configuration that is most recognised but it's not the only + configuration system that exists.</para> </listitem> </varlistentry> @@ -482,8 +666,9 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>compile</term> <listitem> - <para>The compile task actually compiles the software. This could be - as simple as running "make".</para> + <para>The <emphasis>compile</emphasis> task actually compiles the + software. This could be as simple as running <emphasis + role="bold">make</emphasis>.</para> </listitem> </varlistentry> @@ -491,25 +676,23 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>populate_staging (stage)</term> <listitem> - <para>The populate_staging task (stage is an alternate, easier to - type name, that can be used to refer to this task) is responsible - for making available libraries and headers (if any) that may be - required by other packages to build. For example if you compile zlib - then it's headers and the library need to be made available for - other applications to include and link against.</para> - - <para>NOTE that this is different to the install (and packaging) - related tasks in that this is making available things for use during - build on the development host while the installed files are being - made available for use on the target device.</para> - </listitem> - </varlistentry> - - <varlistentry> - <term></term> - - <listitem> - <para></para> + <para>The <emphasis>populate_staging</emphasis> task (stage is an + alternate, easier to type name, that can be used to refer to this + task) is responsible for making available libraries and headers (if + any) that may be required by other packages to build. For example if + you compile zlib then it's headers and the library need to be made + available for other applications to include and link against.</para> + + <note> + <para>This is different to the <emphasis>install</emphasis> task + in that this is responsible for making available libraries and + headers for use during build on the development host. Therefore + it's libraries which normal have to stage things while + applications normally don't need to. The + <emphasis>install</emphasis> task on the other hand is making + files available for packaging and ultimately installation on the + target.</para> + </note> </listitem> </varlistentry> @@ -517,12 +700,14 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>install</term> <listitem> - <para>The install task is responsible for actually installing the - software. Now this needs to install the software into the - destination directory (${D}) but once package the destination - directory will be removed from all the files. In other words if you - install something into ${D}/bin then it will end up in the /bin - directory in the package and therefore on the target.</para> + <para>The <emphasis>install</emphasis> task is responsible for + actually installing everything. Now this needs to install the + software into the destination directory, <emphasis + role="bold">D</emphasis>. This directory won't actually be a part of + the final package though. In other words if you install something + into <emphasis role="bold">${D}/bin</emphasis> then it will end up + in the <emphasis role="bold">/bin</emphasis> directory in the + package and therefore on the target.</para> </listitem> </varlistentry> @@ -530,13 +715,14 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>package</term> <listitem> - <para>The package task takes the installed files and splits them - into separate directories under the ${WORKDIR}/install directory, - one per package. It moves the files for the destination directory, - ${D}, that they were installed in into the appropriate packages - subdirectory. Usually there will be a main package a separate - documentation (-doc), development (-dev) and debugging packages - (-dbg) for example.</para> + <para>The <emphasis>package</emphasis> task takes the installed + files and splits them into separate directories under the <emphasis + role="bold">${WORKDIR}/install</emphasis> directory, one per + package. It moves the files for the destination directory, <emphasis + role="bold">${D}</emphasis>, that they were installed in into the + appropriate packages subdirectory. Usually there will be a main + package a separate documentation (-doc), development (-dev) and + debugging packages (-dbg) for example.</para> </listitem> </varlistentry> @@ -544,38 +730,43 @@ tmp/work/lzo-1.08-r14/install/lzo/usr/lib/liblzo.so.1.0.0</screen></para> <term>package_write</term> <listitem> - <para>The package_write task is responsible for taking each packages - subdirectory and creating any actual installation package, such as - .ipk, .deb or .rpm.</para> - </listitem> - </varlistentry> - - <varlistentry> - <term></term> - - <listitem> - <para></para> + <para>The <emphasis>package_write</emphasis> task is responsible for + taking each packages subdirectory and creating any actual + installation package, such as .ipk, .deb or .rpm. Currently .ipk is + the only fully supported packing format although .deb packages are + being actively worked on. It should be reasonably easy for an + experienced OpenEmbedded developer to add support for any other + packaging formats they might required.</para> </listitem> </varlistentry> </variablelist> - <para></para> - - <para>Note that these are not the only possible set of tasks. There are - various methods available to insert additional tasks in between existing - tasks if needed. As an example the insane.bbclass, which performs various - QA checks, does these checks by inserting a new task, qa_configure, - between the configure and compile tasks to check on the result of - configuration and another new tasks, qa_staging, between populate_staging - and build to validate the files that have been staged.</para> - - <para></para> - - <para>You can see a list of all the tasks available for a specific recipe - by explicitly calling bitbake on the recipe and asking it for a list of - tasks:</para> - - <para><screen>~%> bitbake -b packages/perl/perl_5.8.8.bb -c listtasks + <note> + <para>You'll notice that the bitbake output had tasks prefixed with + <emphasis>do_</emphasis>, as in <emphasis>do_install</emphasis> vs + <emphasis>install</emphasis>. This is slightly confusing but any task + <emphasis>x</emphasis> is implemented via a function called + <emphasis>do_x</emphasis> in the class or recipe where it is defined. + See places refer to the tasks via their name only and some with the + <emphasis>do</emphasis> prefix. </para> + </note> + + <para>You will almost certainly notice tasks beyond these ones - there are + various methods available to insert additional tasks into the tasks + sequence. As an example the <emphasis + role="bold">insane.bbclass</emphasis>, which performs various QA checks, + does these checks by inserting a new task called + <emphasis>qa_configure</emphasis> between the + <emphasis>configure</emphasis> and <emphasis>compile</emphasis> tasks and + another new task called <emphasis>qa_staging</emphasis> between + <emphasis>populate_staging</emphasis> and <emphasis>build</emphasis> + tasks. The format validates the result of the + <emphasis>configure</emphasis> task and the late the results of the + <emphasis>populate_staging</emphasis> task.</para> + + <para>To determine the full list of tasks available for a specific recipe + you can run bitbake on the recipe and asking it for the full list of + available tasks:<screen>~%> bitbake -b packages/perl/perl_5.8.8.bb -c listtasks NOTE: package perl-5.8.8: started NOTE: package perl-5.8.8-r11: task do_listtasks: started do_fetchall @@ -601,40 +792,42 @@ NOTE: package perl-5.8.8-r11: task do_listtasks: completed NOTE: package perl-5.8.8: completed ~%> </screen></para> - <para>Note that <emphasis>do_</emphasis> prefixed to the tasks - the - <emphasis>do_<task></emphasis> is the name of the method that - implements the required functionality for - <emphasis><task></emphasis>, so the above is actually listing the - methods that implement the available tasks. It's sometimes a but confusing - but just remember that the do_ is the method name (with a definition - usually found in the of the .bbclass files in the classes - directory.)</para> + <para>If your being observant you'll note that + <emphasis>listtasks</emphasis> is in fact a task itself, and that the + <emphasis role="bold">-c</emphasis> option to bitbake allows you to + explicitly run specific tasks. We'll make use of this in the next section + when we discuss working with a recipe.</para> </section> - <section id="usage_workwithsinglepackage" - xreflabel="working with a single package"> - <title>Working with a single package</title> - - <para>When working on fixing and/or creating a single recipe you can ask - bitbake to deal directly with a single .bb file only. The <emphasis>-b - <bb-file></emphasis> option asks bitbake to only process the named - file. Note that this ignores any dependencies that are in the recipe, so - these must have already been handled.</para> - - <para>A typically example of this is to clean and then rebuild a package - with some debugging information:</para> - - <para><screen>%> bitbake -b <bb-file> -c clean + <section id="usage_workwithsinglerecipe" + xreflabel="working with a single recipe"> + <title>Working with a single recipe</title> + + <para>During development you're likely to often find yourself working on a + single bitbake recipe - maybe trying to fix something or add a new version + or perhaps working on a totally new recipe. Now that you know all about + tasks you can use that knowledge to help speed up the development and + debugging process.</para> + + <para>Bitbake can be instructed to deal directly with a single recipe file + by passing it via the <emphasis role="bold">-b</emphasis> parameter. This + option takes the recipe as a parameter and instructs bitbake to process + the named recipe only. Note that this ignores any dependencies that are in + the recipe, so these must have already been built previously. </para> + + <para>Here's a typically example that cleans up the package (using the + <emphasis>clean</emphasis> task) and the rebuilds it with debugging output + from bitbake enabled:<screen>%> bitbake -b <bb-file> -c clean %> bitbake -b <bb-file> -D</screen></para> - <para>The various options to bitbake that are useful here are:</para> + <para>The options to bitbake that are most useful here are:</para> <variablelist> <varlistentry> <term>-b <bb-file></term> <listitem> - <para>Specify which recipe to process;</para> + <para>The recipe to process;</para> </listitem> </varlistentry> @@ -642,8 +835,8 @@ NOTE: package perl-5.8.8: completed <term>-c <action></term> <listitem> - <para>Specify which action to perform, typically the name of one of - the tasks supported by the recipe;</para> + <para>The action to perform, typically the name of one of the tasks + supported by the recipe;</para> </listitem> </varlistentry> @@ -652,7 +845,7 @@ NOTE: package perl-5.8.8: completed <listitem> <para>Display debugging information, use two <emphasis - role="bold">-D</emphasis>'s for additional debugging.</para> + role="bold">-D</emphasis>'s for additional debugging;</para> </listitem> </varlistentry> @@ -662,11 +855,11 @@ NOTE: package perl-5.8.8: completed <listitem> <para>Force an operation. This is useful in getting bitbake to perform some operation it normally wouldn't do. For example, if you - try and compile twice in a row then bitbake will not do anything on - the second attempt since it has already performed the compile task. - By adding <emphasis role="bold">-f</emphasis> it will force it to - perform the action regardless of if it thinks it's been done - previously.</para> + try and call the <emphasis>compile</emphasis> task twice in a row + then bitbake will not do anything on the second attempt since it has + already performed the task. By adding <emphasis + role="bold">-f</emphasis> it will force it to perform the action + regardless of if it thinks it's been done previously.</para> </listitem> </varlistentry> </variablelist> @@ -765,7 +958,7 @@ NOTE: package perl-5.8.8: completed actions.</para> <para>A typically development session might involve editing files in the - working directory and recompiling until it all works:<screen>[... test ...] + working directory and then recompiling until it all works:<screen>[... test ...] %> bitbake -b packages/testapp/testapp_4.3.bb -c compile -D [... save a copy of main.c and make some changes ...] @@ -793,11 +986,11 @@ NOTE: package perl-5.8.8: completed <para>Note how we install and then stage. This is one of those things where understanding the tasks helps a lot! Remember that stage moves the - files from where they were installed (${D}) into the various - subdirectories (under <emphasis role="bold">${WORKDIR}/instal</emphasis>l) - for each package. So if you try and run a stage task without a prior - install there won't be any files there to stage! Note also that the stage - tasks clears all the subdirectories in <emphasis + files from where they were installed into the various subdirectories + (under <emphasis role="bold">${WORKDIR}/instal</emphasis>l) for each + package. So if you try and run a stage task without a prior install there + won't be any files there to stage! Note also that the stage tasks clears + all the subdirectories in <emphasis role="bold">${WORKDIR}/install</emphasis> so you won't get any left over files. But beware, the install task doesn't clear <emphasis role="bold">${D}</emphasis> directory, so any left over files from a |