aboutsummaryrefslogtreecommitdiffstats
path: root/lib
AgeCommit message (Collapse)Author
2009-07-23runqueue: Improve recursive task dependency calculation speed (from Poky)Richard Purdie
At present there is a bottleneck in runqueue in the get_recursive_tdepends() function which bothers me as we never used to have it. It appeared when we fixed some correctness issues with the dependency tree and the code in this area has grown adhoc for too long. As an example the above function was getting called 500,000 times in my main test case of building an image. Its particularly problematic in builds with many recursive dependencies such as 'bitbake world'. This commit rewrites the problematic function entirely with the following benefits: * Replaces the most illegible code in that function with code thats easier to understand * Builds the dependency tree per filename, not per task since we don't need it per task which is a performance win * Improves the documentation in places * Much faster execution * Reuses the main dependency tree data, doesn't make its own. The code functions very differently to the original. Previously the recursive dependency tree and the main dependency tree were separate. In this implementation we use the main tree to build the recursive tree after the main tree has been completed, then inject the dependencies. Compared with the original this actually inserts small numbers (4 in my test cases) of additional dependencies into the task graph such as image_recipe:do_rootfs -> image_recipe:do_package_write_ipk which is arguably an bug in the existing implementation. I've checked into this, understand why its happening and believe none of the additional dependencies should cause any complications. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-23cooker.py: Improve server idle handling (from Poky)Richard Purdie
This patch: * Uses the xmlrpc timeout controls added in python 2.6 removing the need to hack the class instead. The hacking didn't work for 2.6. * Converts the parsing function to parse a file per call to the function * Allows checking for xmlrpc commands between parsing files allowing the user to interrupt parsing. * Starts to teach the command queue about command properties such as whether they require the cache. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-23providers.py: Add caching for the dynamic package regexps (from Poky)Richard Purdie
Add caching for the dyanmic package regexps which improves task data performance significantly. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-23utils.py: Add build_environment function (from Poky)Richard Purdie
Add a function to bring the command environment in sync with the data dictonary. Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-23ui/puccho: Improve path specification to glade files (from Poky)Richard Purdie
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-21taskdata: fix a possible infinite loop when multiple eligible providers ↵Chris Larson
aren't buildable The code which removes providers which aren't buildable from the eligible list modifies the list while iterating it, resulting in skipping some entries. If the list contained two failed providers in sequence, it left the second behind in the eligible list. Fixed by replacing the block with a list comprehension that constructs a new eligible list without the failed entries. Signed-off-by: Chris Larson <clarson@mvista.com>
2009-06-19Merge the BBCLASSEXTEND code from Poky. This allows once recipe to provide ↵Richard Purdie
mutliple targets (e.g. -native and -sdk) and has been discussed on the OE mailing list
2009-05-12providers.py: Fix + character escaping from PACKAGES_DYNAMIC (thanks Otavio ↵Richard Purdie
Salvador)
2009-05-11data.py: Make sure allowed variables inherited from the environment are ↵Richard Purdie
exported again (from Poky) shell.py: When running a stage task in bbshell, run populate_staging, not the stage task (from Poky)
2009-05-11fetch/__init__.py: Set HOME environmental variable when running fetcher ↵Richard Purdie
commands (from Poky)
2009-05-11fetch/git.py: Add username handling to git fetcher (from Poky)Richard Purdie
2009-05-11fetch/wget.py: Fix wget fetching of urls with parameters specified (from Poky)Richard Purdie
2009-05-11fetch/hg.py: Fix hg checkouts of specific revisions (from Poky)Richard Purdie
2009-05-11[bb] Remove unused imports reported by pyflakesHolger Hans Peter Freyther
2009-05-11[ui] Fix import statements reported by pyflakesHolger Hans Peter Freyther
We ignore the following kind of warnings import gtk import gtk.glade... as glade is not imported by default and need that extra line, pyflakes does not know...
2009-05-11[svk] pyflakes things the go method is broken...Holger Hans Peter Freyther
Is anyone still using svk? I don't want to mess with this code when it might work (due a global) but I might break it... my guess is that is supposed to be ud.date.
2009-05-11[fetchers] Make pyflakes happy by removing these import statementsHolger Hans Peter Freyther
We don't use the modules, no need to import them here.
2009-05-11[bb] Fix the tests by setting a debug level at startHolger Hans Peter Freyther
In preparation for python3.0 make sure our test cases are working again.
2009-04-21lib: remove use of 'sets' to make Python 2.6 happyMarcin Juszkiewicz
Acked-by: Chris Larson <clarson@kergoth.com>
2009-04-17bb.fetch.go: iterate over SRC_URI, not urldata.Chris Larson
urldata can contain urls other than just what's in SRC_URI. The urls of mirrors, premirrors, stashes, etc can end up in there as well, and we don't want bb.fetch.go to be fetching those things directly. Signed-off-by: Chris Larson <clarson@mvista.com>
2009-04-17bb.fetch.git: make empty host pass a file uri to git.Chris Larson
Signed-off-by: Tom Rini <trini@embeddedalley.com> Signed-off-by: Chris Larson <clarson@mvista.com>
2009-04-17bb.fetch: make try_mirror go through the dispatcher rather than using wget ↵Chris Larson
directly. Signed-off-by: Chris Larson <clarson@mvista.com>
2009-04-08Make the bbpath change for includes temporary rather than permanent.Chris Larson
We currently modify BBPATH so that an include/require looks in the current working directory before it looks elsewhere. Modifying it permanently doesn't gain anything, just polluting it with locations that don't need to stay there. Signed-off-by: Chris Larson <clarson@mvista.com>
2009-04-08BBMASK: survive a non-existant path and don't keep the dir after globbing.Chris Larson
Signed-off-by: Chris Larson <clarson@mvista.com>
2009-04-08BBPKGS: act as a default that can be overridden via the cmdline.Chris Larson
Signed-off-by: Chris Larson <clarson@mvista.com>
2009-02-26Clean up by Otavio SalvadorHolger Hans Peter Freyther
2009-01-03cache.py: Simplfy logic (from Poky)Richard Purdie
2009-01-02BBHandler: Move handler finalisation code into a separate functionRichard Purdie
2009-01-02utils.py: Add bb.utils.prune_suffix functionRichard Purdie
2008-12-06Add puccho, goggle and crumbs metadata from Poky (by Robert Bradford)Richard Purdie
2008-12-06cooker.py: Rearrange the code to allow init of new UIsRichard Purdie
2008-12-06shell.py: Catch up with core changesRichard Purdie
2008-12-06Update the UIs against the core changes and allow dynamic loading of the UI ↵Richard Purdie
so UIs become truly plugable.
2008-12-06Rename depexplorer -> depexp for consistencyRichard Purdie
2008-12-06command.py: Standardise to getVariable and setVariable functions for ↵Richard Purdie
consistent naming
2008-12-06Pass around the tasks to execute rather than using magic variables. Also use ↵Richard Purdie
the empty_environment() function to clear the environment
2008-12-06fetch/__init__.py: export more variables to the fetcher commands to allow ↵Richard Purdie
ssh checkouts and checkouts through proxies to work better. Also allow user and pswd options in urls globally and improve proxy handling when using mirrors (from Poky)
2008-12-06utils.py: Add clean_environment() function and call where appropriate (from ↵Richard Purdie
Poky)
2008-12-06git.py: Add username handling to the git fetcher (Robert Bragg from Poky)Richard Purdie
2008-12-06utils.py: Improve lock file function error handling (from Poky)Richard Purdie
2008-12-06providers.py: Fix typos and add some further debug which can occassionaly be ↵Richard Purdie
useful for debugging problems
2008-12-06event.py: Remove the Pkg* events, the make no sense in the task based model ↵Richard Purdie
bitbake now uses (From Rob Bradford in Poky)
2008-12-06cooker.py: Pass the task to run to tryBuild directly, don't use ↵Richard Purdie
self.configuration.cmd (From Rob Bradford in Poky)
2008-12-06runqueue.py: Fire the task started event from the worker process, not the ↵Richard Purdie
server so the pid field is useful (from Rob Bradford in Poky)
2008-12-06Add the pid to the event messages so we can identify which worker events ↵Richard Purdie
come from (From Rob Bradford in Poky)
2008-12-06Rename online/offline to sync/async for less confusion (from Rob Bradford in ↵Richard Purdie
Poky)
2008-12-06build.py: Add better debug to exec_func_python when python function ↵Richard Purdie
execution fails from Poky)
2008-12-06Add tryaltconfigs option to control whether bitbake trys using alternative ↵Richard Purdie
providers to fulfil failed dependencies. It defaults to off, changing the default since this behaviour confuses many users and isn't often useful (from Poky).
2008-12-06When SRCREV autorevisioning for a recipe is in use, don't cache the recipe ↵Richard Purdie
(from Poky)
2008-12-06Add osc fetcher (from Joshua Lock in Poky)Richard Purdie