summaryrefslogtreecommitdiffstats
path: root/lib/bb/cache.py
AgeCommit message (Collapse)Author
2011-01-20cache: fix docstring usageChris Larson
Use comments instead of docstrings where appropriate in CacheData. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-01-07Merge branch 'poky-sync'Chris Larson
* poky-sync: build: kill unneeded environment setup bits parse: save python functions into the metadata Include exported variables in task process env runqueue: start implementing quieterrors Overhaul environment handling runqueue: resurrect use of file objects for pipein/pipeout runqueue: clean up message log levels runqueue: use correct task ID when checking validity of setscene tasks Add support for 'noexec' tasks runqueue: implement cache checking from metadata runqueue: pass task hash information to tasks runqueue: avoid unnecessary delays Implement task signatures data: Add emit_func and generate_dependencies codeparser: Implement persistent cache data_smart: track variable references codeparser: add module for parsing shell/python runqueue: implement scenequeue fetch: add SRC_URI checksum Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-01-07cache: don't expand variables for skipped recipesChris Larson
Errors can result from these expansions, but for skipped recipes, we shouldn't care about those failures. This fixes the same issue which Richard Purdie fixed in poky, commit 847b717. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2011-01-04cache: defer marking fn as cleanBernhard Reutner-Fischer
Only mark fn as clean if it is clean. This saves us from removing (prematurely added) fn from our clean set and saves me a few percent of runtime (and misleading debugging output from remove()). Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-29runqueue: implement cache checking from metadataRichard Purdie
This lets us speed up setscene processing, by letting the function from the metadata inform us about the necessity of running the setscene functions. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-29Implement task signaturesRichard Purdie
Includes functionality to find out what changes between two different signature data dumps. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-12-16Show the user progress when loading the cacheBob Foerster
Signed-off-by: Bob Foerster <robert@erafx.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-29cache: change to more incremental formatChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-23cache: bump cachever per masterChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-23cache: ensure 'pn' is included in the pkgvarsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cooker: stop loading the cache for -bChris Larson
Previously, the cache was actually being loaded from disk twice whenever using -b or -e -b. This also moves the bb_cache instance into the CookerParser, as it's not needed by the cooker itself at all. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cache: don't add info to cache if cache is disabledChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21Simplify cache syncingChris Larson
Rather than adding nocache items to the cache, then copying the cache and removing them to sync it, don't add them in the first place. Also use 'with' for the cachefile. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21Implement parallel parsing supportChris Larson
This utilizes python's multiprocessing module. The default number of threads to be used is the same as the number of available processor cores, however, you can manually set this with the BB_NUMBER_PARSE_THREADS variable. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cache: don't write out the cache unnecessarilyChris Larson
If the only recipes's we reparsed this run were those flagged as not to be cached, there's no point in re-saving the cache, as those items won't be included anyway. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cache: create and use a RecipeInfo classChris Larson
This class holds the particular pieces of information about a recipe which are needed for runqueue to do its job. By using it, I think we improve code clarity, reduce method sizes, reduce overuse of primitive types, and prepare for parallel parsing. In addition, this ditches the leaky abstraction whereby bb.cache attempted to hide the difference between cached data and a full recipe parse. This was a remnant from the way things used to be done, and the code using it had to know the difference anyway. If we choose to reimplement caching of the full recipes, we can do it in bb.parse, in a completely transparent way. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-19cache: make loadDataFull a classmethodChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: Add debug msg for a nonexistant dep fileChris Larson
If a recipe depends on a file, and that file is out of date, we show a message, but if that file was removed, we do not, until now. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: use new style classesChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: pyflakes/pep8/pylint cleanupChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: use set for clean, checkedChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: make use of defaultdictChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18Pre-explode rundeps/runrecs in CacheDataChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-15bb.cache: only log if the respective action was takenBernhard Reutner-Fischer
This avoids alot of misleading log-messages like "Removing FOO from cache" if FOO was not in the cache and as such is not a removal candidate. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-10-27Make 'cache is clean' message debugChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-24Fix another little bug with undefined var from cacheChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-24bb.cache: fix bug from last commit with loading from cacheChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-23Reduce bb.cache memory usage a bitChris Larson
- Don't store key/value pairs when the value is None - Delete the depends_cache when we're done with it This reduces the memory usage after sync on initial parse by roughly 11.5% on this machine. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-03Switch bitbake internals to use logging directly rather than bb.msgChris Larson
We use a custom Logger subclass for our loggers This logger provides: - 'debug' method which accepts a debug level - 'plain' method which bypasses log formatting - 'verbose' method which is more detail than info, but less than debug Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-08-03Add support for .bbappend files (see mailing lists for detais)Richard Purdie
(From Poky rev: 63e6ba85677b8aa9f4cf9942a1fccbb8a8c72660) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-03cache: If one virtual of a recipe is uncached, remove all versions from the ↵Richard Purdie
cache (From Poky rev: 7edde59de4fd8b19af55e8377138db6ee3dbb3d8) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-06-10Use bb.utils.mkdirhier in bb.cache to shut up the pending deprecation warningChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04cache: do not chdir unnecessarilyBernhard Reutner-Fischer
previously we called chdir() twice for every target, this patch reduces the amount of chdir() calls via openembedded master from some 16000 to 4. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04cache: use a set() for __dependsBernhard Reutner-Fischer
to make updating depends easier/more intuitive/eventually faster Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04cache: use max() to determine newest_mtimeBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04cache: use os.path ops instead of statBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-06-04cache: fix typo in error message of cacheValidUpdate()Bernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-04-20Don't try to expand non-string valuesChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-11Apply some 2to3 transforms that don't cause issues in 2.6Chris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Import fixupsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Formatting cleanupsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Ensure we always utilize the correct messaging domainsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-04-09Made '-b' work with BBCLASSEXTENDEnrico Scholz
When BBCLASSEXTEND is set, '-b' builds usually failed with messages like | ERROR: Parsing error data_fn virtual:native:<recipe>.bb and fn <recipe>.bb don't match | | File ".../bb/providers.py", line 47, in sortPriorities | priority = dataCache.bbfile_priority[f] | KeyError: 'virtual:native:<recipe>.bb' This patch fixes it and allows to specify the alternative class in a way like | ./bitbake -b virtual:native:<recipe>.bb This patch was written to be so minimal as possible; variables should be probably renamed to reflect their new meaning. Signed-off-by: Enrico Scholz <enrico.scholz@informatik.tu-chemnitz.de>
2010-04-06Pass the config metadata into the Cache constructor rather than the cookerChris Larson
Cache only uses the config metadata from the cooker, no need for the cooker itself. Signed-off-by: Chris Larson <clarson@kergoth.com>
2010-03-25cache.py: Bump the cache version after the BBVERSIONS updateRichard Purdie
(From Poky rev: fb245be03d8e3327165033045245f5f853c0b0d4) Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-23Implement BBVERSIONSChris Larson
This implements a feature similar to BBCLASSEXTEND, but for generating multiple versions of a given recipe. For example: BBVERSIONS = "1.0 2.0 git". In addition to the above, one can utilize [a-b] style patterns, and can have a :<basever> postfix, which allows you to essentially name the range of versions. Both the current version and the basever end up in OVERRIDES, and the basever gets placed into the BPV variable. The default BPV, if none is specified, is the original PV of the recipe, before bbversions processing. In this way, you can do things like: BBVERSIONS = "1.0.[0-6]:1.0.0+ 1.0.[7-9]:1.0.7+" SRC_URI_append_1.0.7+ = "file://some_extra_patch.patch;patch=1" Or you can create a recipe per range, and name the recipe file as such: nano_1.0.7+.bb. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-02-12[cache] copy instead of deepcopy in syncHolger Freyther
We only intend to modify the dict, no need to copy all elements.
2010-01-20Improve BBCLASSEXTEND handling for skipped packages and report better ↵Richard Purdie
parsing statistics Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-10-17cache.py: Add check for leakage of SRCREVINACTION variable (from Poky)Richard Purdie
Signed-off-by: Richard Purdie <rpurdie@rpsys.net>
2009-09-14cache: only invalidate for non-existent depends if they used to exist.Chris Larson
Previously, if a file listed in the cached __depends did not exist, it would always invalidate the cache. Now it only does so if the cached version existed, and the current does not. In this way, we can support storing entries in __depends for files which may or may not exist, and ensure that creating such a file results in cache invalidation. Specifically, this is for the amend.inc implementation. Signed-off-by: Chris Larson <clarson@mvista.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-09-20 12:55:15 +0000