diff options
Diffstat (limited to 'doc/bitbake-user-manual/bitbake-user-manual-fetching.xml')
| -rw-r--r-- | doc/bitbake-user-manual/bitbake-user-manual-fetching.xml | 928 |
1 files changed, 0 insertions, 928 deletions
diff --git a/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml b/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml deleted file mode 100644 index fe4372ade..000000000 --- a/doc/bitbake-user-manual/bitbake-user-manual-fetching.xml +++ /dev/null @@ -1,928 +0,0 @@ -<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" -"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"> - -<chapter> -<title>File Download Support</title> - - <para> - BitBake's fetch module is a standalone piece of library code - that deals with the intricacies of downloading source code - and files from remote systems. - Fetching source code is one of the cornerstones of building software. - As such, this module forms an important part of BitBake. - </para> - - <para> - The current fetch module is called "fetch2" and refers to the - fact that it is the second major version of the API. - The original version is obsolete and has been removed from the codebase. - Thus, in all cases, "fetch" refers to "fetch2" in this - manual. - </para> - - <section id='the-download-fetch'> - <title>The Download (Fetch)</title> - - <para> - BitBake takes several steps when fetching source code or files. - The fetcher codebase deals with two distinct processes in order: - obtaining the files from somewhere (cached or otherwise) - and then unpacking those files into a specific location and - perhaps in a specific way. - Getting and unpacking the files is often optionally followed - by patching. - Patching, however, is not covered by this module. - </para> - - <para> - The code to execute the first part of this process, a fetch, - looks something like the following: - <literallayout class='monospaced'> - src_uri = (d.getVar('SRC_URI') or "").split() - fetcher = bb.fetch2.Fetch(src_uri, d) - fetcher.download() - </literallayout> - This code sets up an instance of the fetch class. - The instance uses a space-separated list of URLs from the - <link linkend='var-bb-SRC_URI'><filename>SRC_URI</filename></link> - variable and then calls the <filename>download</filename> - method to download the files. - </para> - - <para> - The instantiation of the fetch class is usually followed by: - <literallayout class='monospaced'> - rootdir = l.getVar('WORKDIR') - fetcher.unpack(rootdir) - </literallayout> - This code unpacks the downloaded files to the - specified by <filename>WORKDIR</filename>. - <note> - For convenience, the naming in these examples matches - the variables used by OpenEmbedded. - If you want to see the above code in action, examine - the OpenEmbedded class file <filename>base.bbclass</filename>. - </note> - The <filename>SRC_URI</filename> and <filename>WORKDIR</filename> - variables are not hardcoded into the fetcher, since those fetcher - methods can be (and are) called with different variable names. - In OpenEmbedded for example, the shared state (sstate) code uses - the fetch module to fetch the sstate files. - </para> - - <para> - When the <filename>download()</filename> method is called, - BitBake tries to resolve the URLs by looking for source files - in a specific search order: - <itemizedlist> - <listitem><para><emphasis>Pre-mirror Sites:</emphasis> - BitBake first uses pre-mirrors to try and find source files. - These locations are defined using the - <link linkend='var-bb-PREMIRRORS'><filename>PREMIRRORS</filename></link> - variable. - </para></listitem> - <listitem><para><emphasis>Source URI:</emphasis> - If pre-mirrors fail, BitBake uses the original URL (e.g from - <filename>SRC_URI</filename>). - </para></listitem> - <listitem><para><emphasis>Mirror Sites:</emphasis> - If fetch failures occur, BitBake next uses mirror locations as - defined by the - <link linkend='var-bb-MIRRORS'><filename>MIRRORS</filename></link> - variable. - </para></listitem> - </itemizedlist> - </para> - - <para> - For each URL passed to the fetcher, the fetcher - calls the submodule that handles that particular URL type. - This behavior can be the source of some confusion when you - are providing URLs for the <filename>SRC_URI</filename> - variable. - Consider the following two URLs: - <literallayout class='monospaced'> - http://git.yoctoproject.org/git/poky;protocol=git - git://git.yoctoproject.org/git/poky;protocol=http - </literallayout> - In the former case, the URL is passed to the - <filename>wget</filename> fetcher, which does not - understand "git". - Therefore, the latter case is the correct form since the - Git fetcher does know how to use HTTP as a transport. - </para> - - <para> - Here are some examples that show commonly used mirror - definitions: - <literallayout class='monospaced'> - PREMIRRORS ?= "\ - bzr://.*/.* http://somemirror.org/sources/ \n \ - cvs://.*/.* http://somemirror.org/sources/ \n \ - git://.*/.* http://somemirror.org/sources/ \n \ - hg://.*/.* http://somemirror.org/sources/ \n \ - osc://.*/.* http://somemirror.org/sources/ \n \ - p4://.*/.* http://somemirror.org/sources/ \n \ - svn://.*/.* http://somemirror.org/sources/ \n" - - MIRRORS =+ "\ - ftp://.*/.* http://somemirror.org/sources/ \n \ - http://.*/.* http://somemirror.org/sources/ \n \ - https://.*/.* http://somemirror.org/sources/ \n" - </literallayout> - It is useful to note that BitBake supports - cross-URLs. - It is possible to mirror a Git repository on an HTTP - server as a tarball. - This is what the <filename>git://</filename> mapping in - the previous example does. - </para> - - <para> - Since network accesses are slow, BitBake maintains a - cache of files downloaded from the network. - Any source files that are not local (i.e. - downloaded from the Internet) are placed into the download - directory, which is specified by the - <link linkend='var-bb-DL_DIR'><filename>DL_DIR</filename></link> - variable. - </para> - - <para> - File integrity is of key importance for reproducing builds. - For non-local archive downloads, the fetcher code can verify - SHA-256 and MD5 checksums to ensure the archives have been - downloaded correctly. - You can specify these checksums by using the - <filename>SRC_URI</filename> variable with the appropriate - varflags as follows: - <literallayout class='monospaced'> - SRC_URI[md5sum] = "<replaceable>value</replaceable>" - SRC_URI[sha256sum] = "<replaceable>value</replaceable>" - </literallayout> - You can also specify the checksums as parameters on the - <filename>SRC_URI</filename> as shown below: - <literallayout class='monospaced'> - SRC_URI = "http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d" - </literallayout> - If multiple URIs exist, you can specify the checksums either - directly as in the previous example, or you can name the URLs. - The following syntax shows how you name the URIs: - <literallayout class='monospaced'> - SRC_URI = "http://example.com/foobar.tar.bz2;name=foo" - SRC_URI[foo.md5sum] = 4a8e0f237e961fd7785d19d07fdb994d - </literallayout> - After a file has been downloaded and has had its checksum checked, - a ".done" stamp is placed in <filename>DL_DIR</filename>. - BitBake uses this stamp during subsequent builds to avoid - downloading or comparing a checksum for the file again. - <note> - It is assumed that local storage is safe from data corruption. - If this were not the case, there would be bigger issues to worry about. - </note> - </para> - - <para> - If - <link linkend='var-bb-BB_STRICT_CHECKSUM'><filename>BB_STRICT_CHECKSUM</filename></link> - is set, any download without a checksum triggers an - error message. - The - <link linkend='var-bb-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></link> - variable can be used to make any attempted network access a fatal - error, which is useful for checking that mirrors are complete - as well as other things. - </para> - </section> - - <section id='bb-the-unpack'> - <title>The Unpack</title> - - <para> - The unpack process usually immediately follows the download. - For all URLs except Git URLs, BitBake uses the common - <filename>unpack</filename> method. - </para> - - <para> - A number of parameters exist that you can specify within the - URL to govern the behavior of the unpack stage: - <itemizedlist> - <listitem><para><emphasis>unpack:</emphasis> - Controls whether the URL components are unpacked. - If set to "1", which is the default, the components - are unpacked. - If set to "0", the unpack stage leaves the file alone. - This parameter is useful when you want an archive to be - copied in and not be unpacked. - </para></listitem> - <listitem><para><emphasis>dos:</emphasis> - Applies to <filename>.zip</filename> and - <filename>.jar</filename> files and specifies whether to - use DOS line ending conversion on text files. - </para></listitem> - <listitem><para><emphasis>basepath:</emphasis> - Instructs the unpack stage to strip the specified - directories from the source path when unpacking. - </para></listitem> - <listitem><para><emphasis>subdir:</emphasis> - Unpacks the specific URL to the specified subdirectory - within the root directory. - </para></listitem> - </itemizedlist> - The unpack call automatically decompresses and extracts files - with ".Z", ".z", ".gz", ".xz", ".zip", ".jar", ".ipk", ".rpm". - ".srpm", ".deb" and ".bz2" extensions as well as various combinations - of tarball extensions. - </para> - - <para> - As mentioned, the Git fetcher has its own unpack method that - is optimized to work with Git trees. - Basically, this method works by cloning the tree into the final - directory. - The process is completed using references so that there is - only one central copy of the Git metadata needed. - </para> - </section> - - <section id='bb-fetchers'> - <title>Fetchers</title> - - <para> - As mentioned earlier, the URL prefix determines which - fetcher submodule BitBake uses. - Each submodule can support different URL parameters, - which are described in the following sections. - </para> - - <section id='local-file-fetcher'> - <title>Local file fetcher (<filename>file://</filename>)</title> - - <para> - This submodule handles URLs that begin with - <filename>file://</filename>. - The filename you specify within the URL can be - either an absolute or relative path to a file. - If the filename is relative, the contents of the - <link linkend='var-bb-FILESPATH'><filename>FILESPATH</filename></link> - variable is used in the same way - <filename>PATH</filename> is used to find executables. - If the file cannot be found, it is assumed that it is available in - <link linkend='var-bb-DL_DIR'><filename>DL_DIR</filename></link> - by the time the <filename>download()</filename> method is called. - </para> - - <para> - If you specify a directory, the entire directory is - unpacked. - </para> - - <para> - Here are a couple of example URLs, the first relative and - the second absolute: - <literallayout class='monospaced'> - SRC_URI = "file://relativefile.patch" - SRC_URI = "file:///Users/ich/very_important_software" - </literallayout> - </para> - </section> - - <section id='http-ftp-fetcher'> - <title>HTTP/FTP wget fetcher (<filename>http://</filename>, <filename>ftp://</filename>, <filename>https://</filename>)</title> - - <para> - This fetcher obtains files from web and FTP servers. - Internally, the fetcher uses the wget utility. - </para> - - <para> - The executable and parameters used are specified by the - <filename>FETCHCMD_wget</filename> variable, which defaults - to sensible values. - The fetcher supports a parameter "downloadfilename" that - allows the name of the downloaded file to be specified. - Specifying the name of the downloaded file is useful - for avoiding collisions in - <link linkend='var-bb-DL_DIR'><filename>DL_DIR</filename></link> - when dealing with multiple files that have the same name. - </para> - - <para> - Some example URLs are as follows: - <literallayout class='monospaced'> - SRC_URI = "http://oe.handhelds.org/not_there.aac" - SRC_URI = "ftp://oe.handhelds.org/not_there_as_well.aac" - SRC_URI = "ftp://you@oe.handhelds.org/home/you/secret.plan" - </literallayout> - </para> - <note> - Because URL parameters are delimited by semi-colons, this can - introduce ambiguity when parsing URLs that also contain semi-colons, - for example: - <literallayout class='monospaced'> - SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git;a=snapshot;h=a5dd47" - </literallayout> - Such URLs should should be modified by replacing semi-colons with '&' characters: - <literallayout class='monospaced'> - SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47" - </literallayout> - In most cases this should work. Treating semi-colons and '&' in queries - identically is recommended by the World Wide Web Consortium (W3C). - Note that due to the nature of the URL, you may have to specify the name - of the downloaded file as well: - <literallayout class='monospaced'> - SRC_URI = "http://abc123.org/git/?p=gcc/gcc.git&a=snapshot&h=a5dd47;downloadfilename=myfile.bz2" - </literallayout> - </note> - </section> - - <section id='cvs-fetcher'> - <title>CVS fetcher (<filename>(cvs://</filename>)</title> - - <para> - This submodule handles checking out files from the - CVS version control system. - You can configure it using a number of different variables: - <itemizedlist> - <listitem><para><emphasis><filename>FETCHCMD_cvs</filename>:</emphasis> - The name of the executable to use when running - the <filename>cvs</filename> command. - This name is usually "cvs". - </para></listitem> - <listitem><para><emphasis><filename>SRCDATE</filename>:</emphasis> - The date to use when fetching the CVS source code. - A special value of "now" causes the checkout to - be updated on every build. - </para></listitem> - <listitem><para><emphasis><link linkend='var-bb-CVSDIR'><filename>CVSDIR</filename></link>:</emphasis> - Specifies where a temporary checkout is saved. - The location is often <filename>DL_DIR/cvs</filename>. - </para></listitem> - <listitem><para><emphasis><filename>CVS_PROXY_HOST</filename>:</emphasis> - The name to use as a "proxy=" parameter to the - <filename>cvs</filename> command. - </para></listitem> - <listitem><para><emphasis><filename>CVS_PROXY_PORT</filename>:</emphasis> - The port number to use as a "proxyport=" parameter to - the <filename>cvs</filename> command. - </para></listitem> - </itemizedlist> - As well as the standard username and password URL syntax, - you can also configure the fetcher with various URL parameters: - </para> - - <para> - The supported parameters are as follows: - <itemizedlist> - <listitem><para><emphasis>"method":</emphasis> - The protocol over which to communicate with the CVS - server. - By default, this protocol is "pserver". - If "method" is set to "ext", BitBake examines the - "rsh" parameter and sets <filename>CVS_RSH</filename>. - You can use "dir" for local directories. - </para></listitem> - <listitem><para><emphasis>"module":</emphasis> - Specifies the module to check out. - You must supply this parameter. - </para></listitem> - <listitem><para><emphasis>"tag":</emphasis> - Describes which CVS TAG should be used for - the checkout. - By default, the TAG is empty. - </para></listitem> - <listitem><para><emphasis>"date":</emphasis> - Specifies a date. - If no "date" is specified, the - <link linkend='var-bb-SRCDATE'><filename>SRCDATE</filename></link> - of the configuration is used to checkout a specific date. - The special value of "now" causes the checkout to be - updated on every build. - </para></listitem> - <listitem><para><emphasis>"localdir":</emphasis> - Used to rename the module. - Effectively, you are renaming the output directory - to which the module is unpacked. - You are forcing the module into a special - directory relative to - <link linkend='var-bb-CVSDIR'><filename>CVSDIR</filename></link>. - </para></listitem> - <listitem><para><emphasis>"rsh"</emphasis> - Used in conjunction with the "method" parameter. - </para></listitem> - <listitem><para><emphasis>"scmdata":</emphasis> - Causes the CVS metadata to be maintained in the tarball - the fetcher creates when set to "keep". - The tarball is expanded into the work directory. - By default, the CVS metadata is removed. - </para></listitem> - <listitem><para><emphasis>"fullpath":</emphasis> - Controls whether the resulting checkout is at the - module level, which is the default, or is at deeper - paths. - </para></listitem> - <listitem><para><emphasis>"norecurse":</emphasis> - Causes the fetcher to only checkout the specified - directory with no recurse into any subdirectories. - </para></listitem> - <listitem><para><emphasis>"port":</emphasis> - The port to which the CVS server connects. - </para></listitem> - </itemizedlist> - Some example URLs are as follows: - <literallayout class='monospaced'> - SRC_URI = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext" - SRC_URI = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat" - </literallayout> - </para> - </section> - - <section id='svn-fetcher'> - <title>Subversion (SVN) Fetcher (<filename>svn://</filename>)</title> - - <para> - This fetcher submodule fetches code from the - Subversion source control system. - The executable used is specified by - <filename>FETCHCMD_svn</filename>, which defaults - to "svn". - The fetcher's temporary working directory is set by - <link linkend='var-bb-SVNDIR'><filename>SVNDIR</filename></link>, - which is usually <filename>DL_DIR/svn</filename>. - </para> - - <para> - The supported parameters are as follows: - <itemizedlist> - <listitem><para><emphasis>"module":</emphasis> - The name of the svn module to checkout. - You must provide this parameter. - You can think of this parameter as the top-level - directory of the repository data you want. - </para></listitem> - <listitem><para><emphasis>"path_spec":</emphasis> - A specific directory in which to checkout the - specified svn module. - </para></listitem> - <listitem><para><emphasis>"protocol":</emphasis> - The protocol to use, which defaults to "svn". - If "protocol" is set to "svn+ssh", the "ssh" - parameter is also used. - </para></listitem> - <listitem><para><emphasis>"rev":</emphasis> - The revision of the source code to checkout. - </para></listitem> - <listitem><para><emphasis>"scmdata":</emphasis> - Causes the “.svn” directories to be available during - compile-time when set to "keep". - By default, these directories are removed. - </para></listitem> - <listitem><para><emphasis>"ssh":</emphasis> - An optional parameter used when "protocol" is set - to "svn+ssh". - You can use this parameter to specify the ssh - program used by svn. - </para></listitem> - <listitem><para><emphasis>"transportuser":</emphasis> - When required, sets the username for the transport. - By default, this parameter is empty. - The transport username is different than the username - used in the main URL, which is passed to the subversion - command. - </para></listitem> - </itemizedlist> - Following are three examples using svn: - <literallayout class='monospaced'> - SRC_URI = "svn://myrepos/proj1;module=vip;protocol=http;rev=667" - SRC_URI = "svn://myrepos/proj1;module=opie;protocol=svn+ssh" - SRC_URI = "svn://myrepos/proj1;module=trunk;protocol=http;path_spec=${MY_DIR}/proj1" - </literallayout> - </para> - </section> - - <section id='git-fetcher'> - <title>Git Fetcher (<filename>git://</filename>)</title> - - <para> - This fetcher submodule fetches code from the Git - source control system. - The fetcher works by creating a bare clone of the - remote into - <link linkend='var-bb-GITDIR'><filename>GITDIR</filename></link>, - which is usually <filename>DL_DIR/git2</filename>. - This bare clone is then cloned into the work directory during the - unpack stage when a specific tree is checked out. - This is done using alternates and by reference to - minimize the amount of duplicate data on the disk and - make the unpack process fast. - The executable used can be set with - <filename>FETCHCMD_git</filename>. - </para> - - <para> - This fetcher supports the following parameters: - <itemizedlist> - <listitem><para><emphasis>"protocol":</emphasis> - The protocol used to fetch the files. - The default is "git" when a hostname is set. - If a hostname is not set, the Git protocol is "file". - You can also use "http", "https", "ssh" and "rsync". - </para></listitem> - <listitem><para><emphasis>"nocheckout":</emphasis> - Tells the fetcher to not checkout source code when - unpacking when set to "1". - Set this option for the URL where there is a custom - routine to checkout code. - The default is "0". - </para></listitem> - <listitem><para><emphasis>"rebaseable":</emphasis> - Indicates that the upstream Git repository can be rebased. - You should set this parameter to "1" if - revisions can become detached from branches. - In this case, the source mirror tarball is done per - revision, which has a loss of efficiency. - Rebasing the upstream Git repository could cause the - current revision to disappear from the upstream repository. - This option reminds the fetcher to preserve the local cache - carefully for future use. - The default value for this parameter is "0". - </para></listitem> - <listitem><para><emphasis>"nobranch":</emphasis> - Tells the fetcher to not check the SHA validation - for the branch when set to "1". - The default is "0". - Set this option for the recipe that refers to - the commit that is valid for a tag instead of - the branch. - </para></listitem> - <listitem><para><emphasis>"bareclone":</emphasis> - Tells the fetcher to clone a bare clone into the - destination directory without checking out a working tree. - Only the raw Git metadata is provided. - This parameter implies the "nocheckout" parameter as well. - </para></listitem> - <listitem><para><emphasis>"branch":</emphasis> - The branch(es) of the Git tree to clone. - If unset, this is assumed to be "master". - The number of branch parameters much match the number of - name parameters. - </para></listitem> - <listitem><para><emphasis>"rev":</emphasis> - The revision to use for the checkout. - The default is "master". - </para></listitem> - <listitem><para><emphasis>"tag":</emphasis> - Specifies a tag to use for the checkout. - To correctly resolve tags, BitBake must access the - network. - For that reason, tags are often not used. - As far as Git is concerned, the "tag" parameter behaves - effectively the same as the "rev" parameter. - </para></listitem> - <listitem><para><emphasis>"subpath":</emphasis> - Limits the checkout to a specific subpath of the tree. - By default, the whole tree is checked out. - </para></listitem> - <listitem><para><emphasis>"destsuffix":</emphasis> - The name of the path in which to place the checkout. - By default, the path is <filename>git/</filename>. - </para></listitem> - <listitem><para><emphasis>"usehead":</emphasis> - Enables local <filename>git://</filename> URLs to use the - current branch HEAD as the revision for use with - <filename>AUTOREV</filename>. - The "usehead" parameter implies no branch and only works - when the transfer protocol is - <filename>file://</filename>. - </para></listitem> - </itemizedlist> - Here are some example URLs: - <literallayout class='monospaced'> - SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1" - SRC_URI = "git://git.oe.handhelds.org/git/vip.git;protocol=http" - </literallayout> - </para> - </section> - - <section id='gitsm-fetcher'> - <title>Git Submodule Fetcher (<filename>gitsm://</filename>)</title> - - <para> - This fetcher submodule inherits from the - <link linkend='git-fetcher'>Git fetcher</link> and extends - that fetcher's behavior by fetching a repository's submodules. - <link linkend='var-bb-SRC_URI'><filename>SRC_URI</filename></link> - is passed to the Git fetcher as described in the - "<link linkend='git-fetcher'>Git Fetcher (<filename>git://</filename>)</link>" - section. - <note> - <title>Notes and Warnings</title> - <para> - You must clean a recipe when switching between - '<filename>git://</filename>' and - '<filename>gitsm://</filename>' URLs. - </para> - - <para> - The Git Submodules fetcher is not a complete fetcher - implementation. - The fetcher has known issues where it does not use the - normal source mirroring infrastructure properly. Further, - the submodule sources it fetches are not visible to the - licensing and source archiving infrastructures. - </para> - </note> - </para> - </section> - - <section id='clearcase-fetcher'> - <title>ClearCase Fetcher (<filename>ccrc://</filename>)</title> - - <para> - This fetcher submodule fetches code from a - <ulink url='http://en.wikipedia.org/wiki/Rational_ClearCase'>ClearCase</ulink> - repository. - </para> - - <para> - To use this fetcher, make sure your recipe has proper - <link linkend='var-bb-SRC_URI'><filename>SRC_URI</filename></link>, - <link linkend='var-bb-SRCREV'><filename>SRCREV</filename></link>, and - <link linkend='var-bb-PV'><filename>PV</filename></link> settings. - Here is an example: - <literallayout class='monospaced'> - SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module" - SRCREV = "EXAMPLE_CLEARCASE_TAG" - PV = "${@d.getVar("SRCREV", False).replace("/", "+")}" - </literallayout> - The fetcher uses the <filename>rcleartool</filename> or - <filename>cleartool</filename> remote client, depending on - which one is available. - </para> - - <para> - Following are options for the <filename>SRC_URI</filename> - statement: - <itemizedlist> - <listitem><para><emphasis><filename>vob</filename></emphasis>: - The name, which must include the - prepending "/" character, of the ClearCase VOB. - This option is required. - </para></listitem> - <listitem><para><emphasis><filename>module</filename></emphasis>: - The module, which must include the - prepending "/" character, in the selected VOB. - <note> - The <filename>module</filename> and <filename>vob</filename> - options are combined to create the <filename>load</filename> rule in - the view config spec. - As an example, consider the <filename>vob</filename> and - <filename>module</filename> values from the - <filename>SRC_URI</filename> statement at the start of this section. - Combining those values results in the following: - <literallayout class='monospaced'> - load /example_vob/example_module - </literallayout> - </note> - </para></listitem> - <listitem><para><emphasis><filename>proto</filename></emphasis>: - The protocol, which can be either <filename>http</filename> or - <filename>https</filename>. - </para></listitem> - </itemizedlist> - </para> - - <para> - By default, the fetcher creates a configuration specification. - If you want this specification written to an area other than the default, - use the <filename>CCASE_CUSTOM_CONFIG_SPEC</filename> variable - in your recipe to define where the specification is written. - <note> - the <filename>SRCREV</filename> loses its functionality if you - specify this variable. - However, <filename>SRCREV</filename> is still used to label the - archive after a fetch even though it does not define what is - fetched. - </note> - </para> - - <para> - Here are a couple of other behaviors worth mentioning: - <itemizedlist> - <listitem><para> - When using <filename>cleartool</filename>, the login of - <filename>cleartool</filename> is handled by the system. - The login require no special steps. - </para></listitem> - <listitem><para> - In order to use <filename>rcleartool</filename> with authenticated - users, an "rcleartool login" is necessary before using the fetcher. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='perforce-fetcher'> - <title>Perforce Fetcher (<filename>p4://</filename>)</title> - - <para> - This fetcher submodule fetches code from the - <ulink url='https://www.perforce.com/'>Perforce</ulink> - source control system. - The executable used is specified by - <filename>FETCHCMD_p4</filename>, which defaults - to "p4". - The fetcher's temporary working directory is set by - <link linkend='var-bb-P4DIR'><filename>P4DIR</filename></link>, - which defaults to "DL_DIR/p4". - The fetcher does not make use of a perforce client, instead it - relies on <filename>p4 files</filename> to retrieve a list of - files and <filename>p4 print</filename> to transfer the content - of those files locally. - </para> - - <para> - To use this fetcher, make sure your recipe has proper - <link linkend='var-bb-SRC_URI'><filename>SRC_URI</filename></link>, - <link linkend='var-bb-SRCREV'><filename>SRCREV</filename></link>, and - <link linkend='var-bb-PV'><filename>PV</filename></link> values. - The p4 executable is able to use the config file defined by your - system's <filename>P4CONFIG</filename> environment variable in - order to define the Perforce server URL and port, username, and - password if you do not wish to keep those values in a recipe - itself. - If you choose not to use <filename>P4CONFIG</filename>, - or to explicitly set variables that <filename>P4CONFIG</filename> - can contain, you can specify the <filename>P4PORT</filename> value, - which is the server's URL and port number, and you can - specify a username and password directly in your recipe within - <filename>SRC_URI</filename>. - </para> - - <para> - Here is an example that relies on <filename>P4CONFIG</filename> - to specify the server URL and port, username, and password, and - fetches the Head Revision: - <literallayout class='monospaced'> - SRC_URI = "p4://example-depot/main/source/..." - SRCREV = "${AUTOREV}" - PV = "p4-${SRCPV}" - S = "${WORKDIR}/p4" - </literallayout> - </para> - - <para> - Here is an example that specifies the server URL and port, - username, and password, and fetches a Revision based on a Label: - <literallayout class='monospaced'> - P4PORT = "tcp:p4server.example.net:1666" - SRC_URI = "p4://user:passwd@example-depot/main/source/..." - SRCREV = "release-1.0" - PV = "p4-${SRCPV}" - S = "${WORKDIR}/p4" - </literallayout> - <note> - You should always set <filename>S</filename> - to <filename>"${WORKDIR}/p4"</filename> in your recipe. - </note> - </para> - - <para> - By default, the fetcher strips the depot location from the - local file paths. In the above example, the content of - <filename>example-depot/main/source/</filename> - will be placed in <filename>${WORKDIR}/p4</filename>. - For situations where preserving parts of the remote depot paths - locally is desirable, the fetcher supports two parameters: - - <itemizedlist> - <listitem><para> - <emphasis>"module":</emphasis> - The top-level depot location or directory to fetch. The - value of this parameter can also point to a single file - within the depot, in which case the local file path will - include the module path. - </para></listitem> - <listitem><para> - <emphasis>"remotepath":</emphasis> - When used with the value "<filename>keep</filename>", - the fetcher will mirror the full depot paths locally - for the specified location, even in combination with - the <filename>module</filename> parameter. - </para></listitem> - </itemizedlist> - </para> - - <para> - Here is an example use of the the <filename>module</filename> - parameter: - - <literallayout class='monospaced'> - SRC_URI = "p4://user:passwd@example-depot/main;module=source/..." - </literallayout> - - In this case, the content of the top-level directory - <filename>source/</filename> will be fetched to - <filename>${P4DIR}</filename>, including the directory itself. - The top-level directory will be accesible at - <filename>${P4DIR}/source/</filename>. - </para> - - <para> - Here is an example use of the the <filename>remotepath</filename> - parameter: - - <literallayout class='monospaced'> - SRC_URI = "p4://user:passwd@example-depot/main;module=source/...;remotepath=keep" - </literallayout> - - In this case, the content of the top-level directory - <filename>source/</filename> will be fetched to - <filename>${P4DIR}</filename>, but the complete depot paths will - be mirrored locally. The top-level directory will be accessible - at <filename>${P4DIR}/example-depot/main/source/</filename>. - </para> - </section> - - <section id='repo-fetcher'> - <title>Repo Fetcher (<filename>repo://</filename>)</title> - - <para> - This fetcher submodule fetches code from - <filename>google-repo</filename> source control system. - The fetcher works by initiating and syncing sources of the - repository into - <link linkend='var-bb-REPODIR'><filename>REPODIR</filename></link>, - which is usually - <link linkend='var-bb-DL_DIR'><filename>DL_DIR</filename></link><filename>/repo</filename>. - </para> - - <para> - This fetcher supports the following parameters: - <itemizedlist> - <listitem><para> - <emphasis>"protocol":</emphasis> - Protocol to fetch the repository manifest (default: git). - </para></listitem> - <listitem><para> - <emphasis>"branch":</emphasis> - Branch or tag of repository to get (default: master). - </para></listitem> - <listitem><para> - <emphasis>"manifest":</emphasis> - Name of the manifest file (default: <filename>default.xml</filename>). - </para></listitem> - </itemizedlist> - Here are some example URLs: - <literallayout class='monospaced'> - SRC_URI = "repo://REPOROOT;protocol=git;branch=some_branch;manifest=my_manifest.xml" - SRC_URI = "repo://REPOROOT;protocol=file;branch=some_branch;manifest=my_manifest.xml" - </literallayout> - </para> - </section> - - <section id='other-fetchers'> - <title>Other Fetchers</title> - - <para> - Fetch submodules also exist for the following: - <itemizedlist> - <listitem><para> - Bazaar (<filename>bzr://</filename>) - </para></listitem> - <listitem><para> - Mercurial (<filename>hg://</filename>) - </para></listitem> - <listitem><para> - npm (<filename>npm://</filename>) - </para></listitem> - <listitem><para> - OSC (<filename>osc://</filename>) - </para></listitem> - <listitem><para> - Secure FTP (<filename>sftp://</filename>) - </para></listitem> - <listitem><para> - Secure Shell (<filename>ssh://</filename>) - </para></listitem> - <listitem><para> - Trees using Git Annex (<filename>gitannex://</filename>) - </para></listitem> - </itemizedlist> - No documentation currently exists for these lesser used - fetcher submodules. - However, you might find the code helpful and readable. - </para> - </section> - </section> - - <section id='auto-revisions'> - <title>Auto Revisions</title> - - <para> - We need to document <filename>AUTOREV</filename> and - <filename>SRCREV_FORMAT</filename> here. - </para> - </section> -</chapter> |