diff options
author | Scott Rifenbark <scott.m.rifenbark@intel.com> | 2014-02-11 20:00:02 -0600 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2014-03-09 18:57:14 -0700 |
commit | 0f82dc4c22ce015bef87fd5aae0b6fa3e429016e (patch) | |
tree | 985aa698b7f78ae2579c932dc7363e6000b5a2e8 /doc | |
parent | f3b21d1fb711f9625d2ac92d4f4fe0f269242bd7 (diff) | |
download | bitbake-0f82dc4c22ce015bef87fd5aae0b6fa3e429016e.tar.gz |
user-manual-fetching.xml: Re-write of "File Download Support" chapter.
Basic re-write to clean up text and flow.
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Diffstat (limited to 'doc')
-rw-r--r-- | doc/user-manual/user-manual-fetching.xml | 432 |
1 files changed, 271 insertions, 161 deletions
diff --git a/doc/user-manual/user-manual-fetching.xml b/doc/user-manual/user-manual-fetching.xml index cdca64c80..499d4f519 100644 --- a/doc/user-manual/user-manual-fetching.xml +++ b/doc/user-manual/user-manual-fetching.xml @@ -2,84 +2,110 @@ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> <chapter> -<title>File download support</title> +<title>File Download Support</title> + + <para> + BitBake's <filename>fetch</filename> and + <filename>fetch2</filename> modules support downloading + files. + This chapter provides an overview of the fetching process + and also presents sections on each of the fetchers BitBake + supports. + <note> + The original <filename>fetch</filename> code, for all + practical purposes, has been replaced by + <filename>fetch2</filename> code. + Consequently, the information in this chapter does not + apply to <filename>fetch</filename>. + </note> + </para> <section id='file-download-overview'> <title>Overview</title> <para> - BitBake provides support to download files. - This procedure is called fetching and is handled by the - fetch and fetch2 modules. - At this point, the original fetch code is considered to - be replaced by fetch2 and this manual is only related - to the fetch2 codebase. + When BitBake starts to execute, the very first thing + it does is to fetch the source files needed. + This section overviews the process. + For some additional information on fetching source files, see the + "<link linkend='source-fetching-dev-environment'>Source Fetching</link>" + section. </para> <para> - The <filename>SRC_URI</filename> is normally used to - tell BitBake which files to fetch. - The next sections describe the available fetchers and - their options. - Each fetcher honors a set of variables and per - URI parameters separated by a “;” consisting of a key and - a value. - The semantics of the variables and parameters are - defined by the fetcher. - BitBake tries to have consistent semantics between the - different fetchers. + When BitBake goes looking for source files, it follows a search + order: + <orderedlist> + <listitem><para><emphasis>Pre-mirror Sites:</emphasis> + BitBake first uses pre-mirrors to try and find source + files. + These locations are defined using the + <link linkend='var-PREMIRRORS'><filename>PREMIRRORS</filename></link> + variable. + </para></listitem> + <listitem><para><emphasis>Source URI:</emphasis> + If pre-mirrors fail, BitBake uses + <link linkend='var-SRC_URI'><filename>SRC_URI</filename></link>. + </para></listitem> + <listitem><para><emphasis>Mirror Sites:</emphasis> + If fetch failures occur using <filename>SRC_URI</filename>, + BitBake next uses mirror location as defined by the + <link linkend='var-MIRRORS'><filename>MIRRORS</filename></link> + variable. + </para></listitem> + </orderedlist> </para> <para> - The overall fetch process first attempts to fetch from - <filename>PREMIRRORS</filename>. - If these fail, the original <filename>SRC_URI</filename> - is attempted. - If that fails, BitBake falls back to - <filename>MIRRORS</filename>. Because cross-URLs are supported, it is possible to mirror a Git repository on an HTTP server as a tarball. Here are some examples that show commonly used mirror definitions: <literallayout class='monospaced'> -PREMIRRORS ?= "\ - bzr://.*/.* http://somemirror.org/sources/ \n \ - cvs://.*/.* http://somemirror.org/sources/ \n \ - git://.*/.* http://somemirror.org/sources/ \n \ - hg://.*/.* http://somemirror.org/sources/ \n \ - osc://.*/.* http://somemirror.org/sources/ \n \ - p4://.*/.* http://somemirror.org/sources/ \n \ - svk://.*/.* http://somemirror.org/sources/ \n \ - svn://.*/.* http://somemirror.org/sources/ \n" - -MIRRORS =+ "\ - ftp://.*/.* http://somemirror.org/sources/ \n \ - http://.*/.* http://somemirror.org/sources/ \n \ - https://.*/.* http://somemirror.org/sources/ \n" + PREMIRRORS ?= "\ + bzr://.*/.* http://somemirror.org/sources/ \n \ + cvs://.*/.* http://somemirror.org/sources/ \n \ + git://.*/.* http://somemirror.org/sources/ \n \ + hg://.*/.* http://somemirror.org/sources/ \n \ + osc://.*/.* http://somemirror.org/sources/ \n \ + p4://.*/.* http://somemirror.org/sources/ \n \ + svk://.*/.* http://somemirror.org/sources/ \n \ + svn://.*/.* http://somemirror.org/sources/ \n" + + MIRRORS =+ "\ + ftp://.*/.* http://somemirror.org/sources/ \n \ + http://.*/.* http://somemirror.org/sources/ \n \ + https://.*/.* http://somemirror.org/sources/ \n" </literallayout> </para> <para> - Non-local downloaded output is placed - into the directory specified by the - <filename>DL_DIR</filename> variable. - For non local archive downloads, the code can verify - sha256 and md5 checksums for the download to ensure - the file has been downloaded correctly. - These can be specified in the following forms - for md5 and sha256 checksums, respectively: + Any source files that are not local (i.e. downloaded from + the Internet) are placed into the download directory, + which is specified by + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>. + </para> + + <para> + For non-local archive downloads, the fetcher code can verify + sha256 and md5 checksums to ensure + the archives have been downloaded correctly. + You can specify these checksums by using the + <filename>SRC_URI</filename> variable with the appropriate + varflags as follows: <literallayout class='monospaced'> SRC_URI[md5sum] SRC_URI[sha256sum] </literallayout> - You can also specify them as parameters on the - <filename>SRC_URI</filename>: + You can also specify the checksums as parameters on the + <filename>SRC_URI</filename> as shown below: <literallayout class='monospaced'> SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" </literallayout> - If <filename>BB_STRICT_CHECKSUM</filename> is set, any download - without a checksum will trigger an error message. - In cases where multiple files are listed in + If + <link linkend='var-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link> + is set, any download without a checksum triggers an error message. + In cases where multiple files are listed using <filename>SRC_URI</filename>, the name parameter is used assign names to the URLs and these are then specified in the checksums using the following form: @@ -89,142 +115,226 @@ MIRRORS =+ "\ </para> </section> - <section id='local-file-fetcher'> - <title>Local file fetcher</title> + <section id='bb-fetchers'> + <title>Fetchers</title> <para> - The URN for the local file fetcher is file. - The filename can be either absolute or relative. - If the filename is relative, - <filename>FILESPATH</filename> and failing that - <filename>FILESDIR</filename> will be used to find the - appropriate relative file. - The metadata usually extend these variables to include - variations of the values in <filename>OVERRIDES</filename>. - Single files and complete directories can be specified. - <literallayout class='monospaced'> + As mentioned in the previous section, the + <filename>SRC_URI</filename> is normally used to + tell BitBake which files to fetch. + And, the fetcher BitBake uses depends on the how + <filename>SRC_URI</filename> is set. + </para> + + <para> + These next few sections describe the available fetchers and + their options. + Each fetcher honors a set of variables URI parameters, + which are separated by semi-colon characters and consist + of a key and a value. + The semantics of the variables and parameters are + defined by the fetcher. + BitBake tries to have consistent semantics between the + different fetchers. + </para> + + <section id='local-file-fetcher'> + <title>Local file fetcher</title> + + <para> + The URN for the local file fetcher is file. + </para> + + <para> + The filename can be either absolute or relative. + If the filename is relative, + <link linkend='var-FILESPATH'><filename>FILESPATH</filename></link> + is used. + Failing that, + <link linkend='var-FILESDIR'><filename>FILESDIR</filename></link> + is used to find the appropriate relative file. + </para> + + <para> + The metadata usually extend these variables to include + variations of the values in + <link linkend='var-OVERRIDES'><filename>OVERRIDES</filename></link>. + Single files and complete directories can be specified. + <literallayout class='monospaced'> SRC_URI = "file://relativefile.patch" SRC_URI = "file://relativefile.patch;this=ignored" SRC_URI = "file:///Users/ich/very_important_software" - </literallayout> - </para> - </section> + </literallayout> + </para> + </section> - <section id='cvs-fetcher'> - <title>CVS fetcher</title> + <section id='cvs-fetcher'> + <title>CVS fetcher</title> - <para> - The URN for the CVS fetcher is cvs. - This fetcher honors the variables <filename>CVSDIR</filename>, - <filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>, - <filename>UPDATECOMMAND_cvs</filename>. - <filename>DL_DIR</filename> specifies where a - temporary checkout is saved. - <filename>SRCDATE</filename> specifies which date to - use when doing the fetching (the special value of "now" - will cause the checkout to be updated on every build). - <filename>FETCHCOMMAND</filename> and - <filename>UPDATECOMMAND</filename> specify which executables - to use for the CVS checkout or update. - </para> + <para> + The URN for the CVS fetcher is cvs. + </para> - <para> - The supported parameters are module, tag, date, - method, localdir, rshand scmdata. - The module specifies which module to check out, - the tag describes which CVS TAG should be used for - the checkout. - By default, the TAG is empty. - A date can be specified to override the - <filename>SRCDATE</filename> of the - configuration to checkout a specific date. - The special value of "now" will cause the checkout to be - updated on every build. - method is by default pserver. - If ext is used the rsh parameter will be evaluated - and <filename>CVS_RSH</filename> will be set. - Finally, localdir is used to checkout into a special - directory relative to <filename>CVSDIR</filename>. - <literallayout class='monospaced'> + <para> + This fetcher honors the variables <filename>CVSDIR</filename>, + <filename>SRCDATE</filename>, <filename>FETCHCOMMAND_cvs</filename>, + <filename>UPDATECOMMAND_cvs</filename>. + The + <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link> + variable specifies where a + temporary checkout is saved. + The + <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link> + variable specifies which date to + use when doing the fetching. + The special value of "now" causes the checkout to be + updated on every build. + The <filename>FETCHCOMMAND</filename> and + <filename>UPDATECOMMAND</filename> variables specify the executables + to use for the CVS checkout or update. + </para> + + <para> + The supported parameters are as follows: + <itemizedlist> + <listitem><para><emphasis>"module":</emphasis> + Specifies the module to check out. + </para></listitem> + <listitem><para><emphasis>"tag":</emphasis> + Describes which CVS TAG should be used for + the checkout. + By default, the TAG is empty. + </para></listitem> + <listitem><para><emphasis>"date":</emphasis> + Specifies a date. + If no "date" is specified, the + <link linkend='var-SRCDATE'><filename>SRCDATE</filename></link> + of the configuration is used to checkout a specific date. + The special value of "now" causes the checkout to be + updated on every build. + </para></listitem> + <listitem><para><emphasis>"method":</emphasis> + By default <filename>pserver</filename>. + If "method" is set to "ext", BitBake examines the "rsh" + parameter and sets <filename>CVS_RSH</filename>. + </para></listitem> + <listitem><para><emphasis>"localdir":</emphasis> + Used to checkout force into a special + directory relative to <filename>CVSDIR</filename>. + </para></listitem> + <listitem><para><emphasis>"rsh"</emphasis> + Used in conjunction with the "method" parameter. + </para></listitem> + <listitem><para><emphasis>"scmdata":</emphasis> + I need a description for this. + </para></listitem> + </itemizedlist> + Following are two examples using cvs: + <literallayout class='monospaced'> SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat" - </literallayout> - </para> - </section> + </literallayout> + </para> + </section> - <section id='http-ftp-fetcher'> - <title>HTTP/FTP fetcher</title> + <section id='http-ftp-fetcher'> + <title>HTTP/FTP fetcher</title> - <para> - The URNs for the HTTP/FTP fetcher are http, https, and ftp. - This fetcher honors the variables - <filename>FETCHCOMMAND_wget</filename>. - <filename>FETCHCOMMAND</filename> contains the command used - for fetching. - “${URI}” and “${FILES}” will be replaced by the URI and - basename of the file to be fetched. - <literallayout class='monospaced'> + <para> + The URNs for the HTTP/FTP fetcher are http, https, and ftp. + </para> + + <para> + This fetcher honors the variables + <filename>FETCHCOMMAND_wget</filename>. + The <filename>FETCHCOMMAND</filename> variable + contains the command used for fetching. + “${URI}” and “${FILES}” are replaced by the URI and + the base name of the file to be fetched. + <literallayout class='monospaced'> SRC_URI = "http://oe.handhelds.org/not_there.aac" SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac" SRC_URI = "ftp://you@oe.handheld.sorg/home/you/secret.plan" - </literallayout> - </para> - </section> + </literallayout> + </para> + </section> - <section id='svn-fetcher'> - <title>SVN Fetcher</title> + <section id='svn-fetcher'> + <title>SVN Fetcher</title> - <para> - The URN for the SVN fetcher is svn. - </para> + <para> + The URN for the SVN fetcher is svn. + </para> - <para> - This fetcher honors the variables - <filename>FETCHCOMMAND_svn</filename>, - <filename>SVNDIR</filename>, - and <filename>SRCREV</filename>. - <filename>FETCHCOMMAND</filename> contains the - subversion command. - <filename>SRCREV</filename> specifies which revision - to use when doing the fetching. - </para> + <para> + This fetcher honors the variables + <filename>FETCHCOMMAND_svn</filename>, + <filename>SVNDIR</filename>, + and + <link linkend='var-SRCREV'><filename>SRCREV</filename></link>. + The <filename>FETCHCOMMAND</filename> variable contains the + <filename>subversion</filename> command. + The <filename>SRCREV</filename> variable specifies which revision + to use when doing the fetching. + </para> - <para> - The supported parameters are proto, rev and scmdata. - proto is the Subversion protocol, rev is the - Subversion revision. - If scmdata is set to “keep”, the “.svn” directories will - be available during compile-time. - <literallayout class='monospaced'> + <para> + The supported parameters are as follows: + <itemizedlist> + <listitem><para><emphasis>"proto":</emphasis> + The Subversion protocol. + </para></listitem> + <listitem><para><emphasis>"rev":</emphasis> + The Subversion revision. + </para></listitem> + <listitem><para><emphasis>"scmdata":</emphasis> + Set to "keep" causes the “.svn” directories + to be available during compile-time. + </para></listitem> + </itemizedlist> + Following are two examples using svn: + <literallayout class='monospaced'> SRC_URI = "svn://svn.oe.handhelds.org/svn;module=vip;proto=http;rev=667" SRC_URI = "svn://svn.oe.handhelds.org/svn/;module=opie;proto=svn+ssh;date=20060126" - </literallayout> - </para> - </section> + </literallayout> + </para> + </section> - <section id='git-fetcher'> - <title>GIT Fetcher</title> + <section id='git-fetcher'> + <title>GIT Fetcher</title> - <para> - The URN for the GIT Fetcher is git. - </para> + <para> + The URN for the Git Fetcher is git. + </para> - <para> - The variable <filename>GITDIR</filename> will be used as the - base directory where the Git tree is cloned to. - </para> + <para> + The variable <filename>GITDIR</filename> is used as the + base directory in which the Git tree is cloned. + </para> - <para> - The parameters are tag, protocol, and scmdata. - The tag parameter is a Git tag, the default is “master”. - The protocol tag is the Git protocol to use and defaults to “git” - if a hostname is set, otherwise it is “file”. - If scmdata is set to “keep”, the “.git” directory will be available - during compile-time. - <literallayout class='monospaced'> + <para> + The supported parameters are as follows: + <itemizedlist> + <listitem><para><emphasis>"tag":</emphasis> + The Git tag. + The default is "master". + </para></listitem> + <listitem><para><emphasis>"protocol":</emphasis> + The Git protocol. + The default is "git" when a hostname is set. + If a hostname is not set, the Git protocol is "file". + </para></listitem> + <listitem><para><emphasis>"scmdata":</emphasis> + When set to “keep”, the “.git” directory is available + during compile-time. + </para></listitem> + </itemizedlist> + Following are two examples using git: + <literallayout class='monospaced'> SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1" SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http" - </literallayout> - </para> + </literallayout> + </para> + </section> </section> </chapter> |