summaryrefslogtreecommitdiffstats
path: root/lib/bb/fetch
AgeCommit message (Collapse)Author
2011-02-11persist_data: drop SQLData indirectionChris Larson
The common case (if not the only case) is to only use a single domain. The only reason SQLData exists is to make it easier to delete a domain. Yet, there's no need for us to delete a domain if SQLTable knows how to clear itself out. So, add clear() to the table and pass the domain to persist(). Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-02-10bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2Yu Ke
there is case that we need to distingush bb.fetch and bb.fetch2, and use different API for bb.fetch and bb.fetch2. so it is necessary to add version info for distinguish purpose (From Poky rev: f12e71484593039cd58b6e7fadd038b28b05d849) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10Fetcher: break the "SRCREVINACTION" deadlockYu Ke
Current fetcher has annoying "SRCREVINACTION" deadlock, which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev(): get_srcrev()->setup_localpath()->srcrev_internal_helper() ->evaluate SRCREV->get_srcrev() current fetcher resolve the deadlock by introducing a "SRCREVINACTION" condition check. Althoguh it works, it is indeed not clean. This patch use antoehr idea to break the deadlock: break the dependency among SRCREV and get_srcrev(), i.e. assign a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet this keyword, it will check and set the latest revision to urldata.revision. get_srcrev later can use the urldata.revision for value evaluation(SRCPV etc). In this case, SRCREV no longer depends on get_srcrev, and there is not deadlock anymore. (From Poky rev: 1589a1172f9432aed1cc9ce006f68cddf3073774) Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-09Export KRB5CCNAME variableJavier Martin
This allows fetching git repositories using Kerberos authentication. Signed-off-by: Javier Martin <javier.martin@vista-silicon.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-02-09Fix comparison with SRCREVINACTION constantJavier Martin
Use '==' instead of 'is', otherwise it will always return true since 'rev' and "SRCREVINACTION" are not the same object. Signed-off-by: Javier Martin <javier.martin@vista-silicon.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-01-11*: use utils.remove() some moreBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2011-01-10fetch/__init__.py: Add missing debug level argumentRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2010-12-29fetch: add SRC_URI checksumYu Ke
This patch adds per-recipe SRC_URI checksum verification. - SRC_URI format The format of SRC_URI checksum follow OE definition: 1. SRC_URI has single src SRC_URI = "http://some.domain/file.tar.gz" SRC_URI[md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" 2. SRC_URI has multiple src, every src need specify name SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \ http://some.domain/file2.tar.gz;name=name2 " SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx" SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx" - SRC_URI checking invocation: the checksum checking is invoked in do_fetch phase, so it can be invoked manually by if recipes has no SRC_URI checksum item, bitbake will show warning: " WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072" SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768" " thus recipe author can add it to recpie file after SRC_URI - control variable BB_STRICT_CHECKSUM when SRC_URI checksum is missing, this variable decide pass or not if BB_STRICT_CHECKSUM = "1", missing checksum is fatal Signed-off-by: Yu Ke <ke.yu@intel.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-29fetch: handle no sortable_buildindex for gitChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-29fetch: fix git rev key compat for sortableChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-29fetch: resurrect branch inclusion in git revision keyChris Larson
This is similar to Richard Purdie's commit, but falls back to checking the persist_data database for the old key if the new does not exist, so as to retain compatibility for at least one bitbake minor version. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-29fetch: Ensure SRCREV is still set correctly if the pn-X override has been ↵Richard Purdie
expanded Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-29fetch: Fix named SRCREVS variable name orderingRichard Purdie
Retains backward compatibility. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-17Update persist_data usage to the new APIChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-17bzr: use utils.remove instead of os.systemBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-12-15fetchers: Add parameter scmdata=keep to include .git/ and others in ↵Andreas Oberritter
generated tarballs. * Allows generating version information from SCMs during build. * Note that tar doesn't need to use --exclude '.git', because git checkout-index doesn't clone the repository. Signed-off-by: Andreas Oberritter <obi@opendreambox.org> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-10Rename command events, adjust compareRevisionsChris Larson
- Moved the logic for comparing revisions from cooker into command - Removed 'Cooker' from the event names - Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to be a subclass of CommandExit Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-10Revert "bitbake/git.py: Make sure different branches can have different ↵Chris Larson
revisions without triggering build count increases" Per Martin Jansa, this breaks compatibility. We can revisit in the future, likely during the fetcher rewrite. This reverts commit f3292fa11723c748ef1b4270384abf6d586a822e.
2010-12-06Merged the following poky fetcher fixes:Chris Larson
- bitbake/fetch: Allow checking of a single url at a time (massive sstate speed improvement) - bitbake/fetch: Fix handling of mirrors when checking for url validity - bitbake/fetch: When fetchers return errors, ensure any partial download is cleared - bitbake/fetch: Make URL checking slightly less verbose (distracting with the sstate code) - bitbake/fetch/git: Ensure fullclone repositories are fully fetched - bitbake/fetcher: Deal with a ton of different bugs - bitbake/fetch/git.py: Fix git fetcher to correctly use mirror tarballs - bitbake/fetch/__init__.py: Abstract mirror variable handling and enhance to accept \n delimitation in variables - bitbake/fetch: fix logic to prevent fetches when the file already exists - bitbake/fetch: ensure the go() method completes when not using premirrors - Export ALL_PROXY for bitbake commands - bitbake: Add proxy variables to standard export list - bitbake/fetch/git: fix try_premirror() definition .Method takes 4 parameters. - bitbake/fetch: fix by moving try_premirror() to the right place - bitbake/fetch: add try_premirror method and special case git fetcher - fetch: fix setting of localpath in SRC_URI parameters - bitbake/fetch: ensure the mirrored repository is updated as required - bitbake/git: define a forcefetch method - bitbake/fetch: Respect forcefetch even when pulling from a mirror - bitbake/git.py: Make sure different branches can have different revisions without triggering build count increases - bitbake/git.py: Allow tracking of branches in SRC_URI without cloning for use with fullclone - bitbake/git.py: Make sure a full clone checkout always updates Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-20Revert "svn fetcher: warn people to switch to SRCREV"Chris Larson
1) too spammy 2) can be implemented in the metadata instead This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b.
2010-11-19Revert "persist_data: cache connection and use cursor"Chris Larson
Caching the database connection can cause serious issues if it results in multiple processes (e.g. multiple tasks) simultaneously using the same connection. This reverts commit 8a6876752b90efd81d92f0947bfc9527d8260969. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-19svn fetcher: warn people to switch to SRCREVBernhard Reutner-Fischer
as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644 Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: be more pythonicBernhard Reutner-Fischer
no functional changes Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: use os.path.joinBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: add common helper _strip_leading_slashes()Bernhard Reutner-Fischer
Several fetcher need a way to strip leading slashes off a local path. This helper-function consolidates all such occurances. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: rename suppports_srcrev to supports_srcrevBernhard Reutner-Fischer
osc had it already spelled correctly?! Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-08Allow mercurial fetcher to follow tipPeter Chubb
There are occasions when developing when I want a package always to grab the latest copy of a package. Witht eh CVS fetcher you can do this by setting the `date' tag to `now'. This patch adds similar functionality to the mercurial fetcher: if the revision to fetch is `tip' then always grab from the server, and don't use the cached tarball. Oh, and I fixed a typo in the Class comment. Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-21bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repositoryEric BENARD
* without this fix, we get : updating working directory 74 files updated, 0 files merged, 0 files removed, 0 files unresolved abort: There is no Mercurial repository here (.hg not found)! Signed-off-by: Eric Bénard <eric@eukrea.com> Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-06fetch: if mirror fetching fails, ensure exception is raisedJoshua Lock
We catch any exception raised by the fetchers go() method and attempt to work around it by trying any (post) mirrors which are configured. However, should the mirrors fail the exception is lost and the fetch is assumed to have completed successfully. Instead, save the exception and if the local file does not exist after trying the mirrors re-raise the exception. Fixes poky [BUGID #362] Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-28fetchers: Use tar --exclude pattern to remove SCM filesKhem Raj
This option will exclude the SCM metadata from tar files. Tested with gcc where svn tar which used to be 156M for gcc 4.5 is now 77M Signed-off-by: Khem Raj <raj.khem@gmail.com>
2010-09-03Switch bitbake internals to use logging directly rather than bb.msgChris Larson
We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-08-18Allow %20 in a file name in the SRC_URIC Michael Sundius
Signed-off-by: C Michael Sundius <msundius@sundius.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-08-03fetch: Only mark srcrev recipes as uncached when outside of SRCREV caching modeRichard Purdie
(From Poky rev: 36cfff8ed5b8a777a09cc8abcd8b01a1f297e404) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-26*: cosmetic whitespace removalBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-07-01bitbake decodeurl: fix the file:// url handlingNitin A Kamble
Without this patch decoding a url of this kind file://dir/filename gives path=/filename host=dir. With the patch it decodes as path=/dir/filename host="" Probably nobody stumbled on this issue yet because nobody used file:// urls with directory names in the path. (From Poky rev: c3682bf89bdf236a4b4d834069d947b6a1456791) Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-06-213.0 prepChris Larson
2010-06-21Apply some 2to3 refactoringsChris Larson
2010-06-10Start ditching fatal() calls in favor of raising appropriate exceptionsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-09Change wget fetcher to use the runfetchcmdRyan Phillips
* Fixes proxy support to honor standard proxy environment variables. * Quote environment variables Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04persist_data: cache connection and use cursorBernhard Reutner-Fischer
Store database connection to persistent database in fetcher. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04fetch: commentary typo fix in fetcher_init()Bernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-06-04fetch: documentation typo fixBernhard Reutner-Fischer
s/initilize/initialize/ Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-05-22fetch: Check for existence of tar file along with md5 signatureKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-12Kill unnecessary usages of the types moduleChris Larson
types.IntType -> int types.StringType -> basestring ... Also moves our ImmutableTypes tuple into our own namespace. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-11Apply some 2to3 transforms that don't cause issues in 2.6Chris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-10Switch some references to moved functionsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-10Add a missing import of print_functionChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Apply the 2to3 print function transformChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Import fixupsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Formatting cleanupsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-01 14:54:05 +0000