aboutsummaryrefslogtreecommitdiffstats
path: root/lib
AgeCommit message (Collapse)Author
2010-11-21Experimental usage of the 'progressbar' moduleChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21Implement parallel parsing supportChris Larson
This utilizes python's multiprocessing module. The default number of threads to be used is the same as the number of available processor cores, however, you can manually set this with the BB_NUMBER_PARSE_THREADS variable. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cache: don't write out the cache unnecessarilyChris Larson
If the only recipes's we reparsed this run were those flagged as not to be cached, there's no point in re-saving the cache, as those items won't be included anyway. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21cache: create and use a RecipeInfo classChris Larson
This class holds the particular pieces of information about a recipe which are needed for runqueue to do its job. By using it, I think we improve code clarity, reduce method sizes, reduce overuse of primitive types, and prepare for parallel parsing. In addition, this ditches the leaky abstraction whereby bb.cache attempted to hide the difference between cached data and a full recipe parse. This was a remnant from the way things used to be done, and the code using it had to know the difference anyway. If we choose to reimplement caching of the full recipes, we can do it in bb.parse, in a completely transparent way. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-21Prefer xrange over range for small performance gain.Bob Foerster
range() allocates an actual list when called. xrange() is just an iterator and creates the next range item on demand. This provides a slight performance increase. In python 3, range will do what xrange does currently, but the upgrade will be handled by the 2to3 tool. Signed-off-by: Bob Foerster <robert@erafx.com>
2010-11-20Revert "svn fetcher: warn people to switch to SRCREV"Chris Larson
1) too spammy 2) can be implemented in the metadata instead This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b.
2010-11-19runqueue: speed up bitbake -nChris Larson
From scratch, bitbake -n task-boot times: Before: 4:39.73 After: 1:52.58 Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-19cache: make loadDataFull a classmethodChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-19Revert "persist_data: cache connection and use cursor"Chris Larson
Caching the database connection can cause serious issues if it results in multiple processes (e.g. multiple tasks) simultaneously using the same connection. This reverts commit 8a6876752b90efd81d92f0947bfc9527d8260969. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-19svn fetcher: warn people to switch to SRCREVBernhard Reutner-Fischer
as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644 Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: be more pythonicBernhard Reutner-Fischer
no functional changes Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: use os.path.joinBernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: add common helper _strip_leading_slashes()Bernhard Reutner-Fischer
Several fetcher need a way to strip leading slashes off a local path. This helper-function consolidates all such occurances. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-19fetch: rename suppports_srcrev to supports_srcrevBernhard Reutner-Fischer
osc had it already spelled correctly?! Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-18cache: Add debug msg for a nonexistant dep fileChris Larson
If a recipe depends on a file, and that file is out of date, we show a message, but if that file was removed, we do not, until now. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: use new style classesChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: pyflakes/pep8/pylint cleanupChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: use set for clean, checkedChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18cache: make use of defaultdictChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18Pre-explode rundeps/runrecs in CacheDataChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-18persist_data: handle locked db for SELECTChris Larson
Parallel processes interacting with the persist_data db can quite easily explode without this. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-15BBHandler: use os.path in inherit()Bernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-15BBHandler: simplify supports()Bernhard Reutner-Fischer
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-15bb.cache: only log if the respective action was takenBernhard Reutner-Fischer
This avoids alot of misleading log-messages like "Removing FOO from cache" if FOO was not in the cache and as such is not a removal candidate. Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
2010-11-11bb.pysh: add Case support to format_commandsChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-11bb.pysh: fix writing pyshtables all over the placeChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-11Move the pysh package into the bb packageChris Larson
The pysh we're using is modified, and we don't want to risk it conflicting with one from elsewhere. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-08Allow mercurial fetcher to follow tipPeter Chubb
There are occasions when developing when I want a package always to grab the latest copy of a package. Witht eh CVS fetcher you can do this by setting the `date' tag to `now'. This patch adds similar functionality to the mercurial fetcher: if the revision to fetch is `tip' then always grab from the server, and don't use the cached tarball. Oh, and I fixed a typo in the Class comment. Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-11-04Fix GraphViz .dot output for rdepends and rrecsChris Larson
Use bb.utils.explode_deps to break up the rdepends and rrecommends strings. This fixes the same issue which was fixed by a number of patches floating around, but uses explode_deps rather than regular expressions. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-27Make 'cache is clean' message debugChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-21bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repositoryEric BENARD
* without this fix, we get : updating working directory 74 files updated, 0 files merged, 0 files removed, 0 files unresolved abort: There is no Mercurial repository here (.hg not found)! Signed-off-by: Eric Bénard <eric@eukrea.com> Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-21Resurrect display of failed filesChris Larson
This was inadvertantly removed when trying to reduce the amount of duplicated information the user sees when a failure occurs. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-06fetch: if mirror fetching fails, ensure exception is raisedJoshua Lock
We catch any exception raised by the fetchers go() method and attempt to work around it by trying any (post) mirrors which are configured. However, should the mirrors fail the exception is lost and the fetch is assumed to have completed successfully. Instead, save the exception and if the local file does not exist after trying the mirrors re-raise the exception. Fixes poky [BUGID #362] Signed-off-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-05Access metadata vars as locals in python snippetsChris Larson
Example: FOO = "bar" BAR = "${@FOO + '/baz'}" ${BAR} == "bar/baz" Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-10-05Fix __getitem__ for DataSmartChris Larson
Ensure it raises KeyError for a missing key, this is required to use this as a mapping in various places, e.g. as locals in an eval. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-28fetchers: Use tar --exclude pattern to remove SCM filesKhem Raj
This option will exclude the SCM metadata from tar files. Tested with gcc where svn tar which used to be 156M for gcc 4.5 is now 77M Signed-off-by: Khem Raj <raj.khem@gmail.com>
2010-09-28Fix syntax issue and don't include 'd' in ui_queueChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-28Queue up events before the UI is spawnedChris Larson
- Queue up any events fired to the UI before the UI exists - At exit, check if UIs exist, and if not, flush the queue of LogRecords to the console directly. - When establishing a connection from the UI to the server, flush the queue of events to the queue in the server connection, so the UI will receive them when it begins its event loop. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-24Fix another little bug with undefined var from cacheChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-24bb.cache: fix bug from last commit with loading from cacheChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-23Reduce bb.cache memory usage a bitChris Larson
- Don't store key/value pairs when the value is None - Delete the depends_cache when we're done with it This reduces the memory usage after sync on initial parse by roughly 11.5% on this machine. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-12Fix FuncFailed constructionChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Restore FuncFailed argument compatibilityChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Only reference logfiles that existChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Fix bitbake -k issue introduced by build exception cleanupChris Larson
A SystemExit from a python function wasn't being raised as a FuncFailed, which resulted in it not being caught by the exception handlers in the runqueue for the worker process, which resulted in a SystemExit exit, rather than os._exit, which causes all manner of problems when used in a forked process. This fixes it by ensuring we raise a FuncFailed when seeing exceptions which aren't instances of Exception. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Move display of FuncFailed exception info into the runqueueChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Don't show tracebacks for SystemExit or KeyboardInterruptChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-10Fix exit code display for task failureChris Larson
Per the python documentation, os.waitpid returns the exitcode shifted up by 8 bits, and we weren't compensating, resulting in a display of 'failed with 256' when a worker process exits with a code of 1. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-09Simplify build exception handlingChris Larson
- Drop EventException - Use FuncFailed as the primary function failure exception, using TaskFailed for the event (leaving it up to the process running exec_{func,task} to display the more detailed information available in the exception). - Switch InvalidTask to an exception rather than an event, as that's a critical issue. - Reduce the number of messages shown to the user when a task fails -- they don't need to be told it fails 12 times. Work remains in this area though. Signed-off-by: Chris Larson <chris_larson@mentor.com>
2010-09-09Turn Event into a new style classChris Larson
Signed-off-by: Chris Larson <chris_larson@mentor.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-09 08:02:50 +0000