aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* tests/event: Include tests for event filtering featuresJair Gonzalez2018-01-071-9/+111
| | | | | | | | | | | | | | | | | | This change includes unit tests for the following functions, helper class and methods in bitbake.event: - set_eventfilter(func) - set_UIHmask(handlerNum, level, debug_domains, mask) - getName(e) - class UIEventFilter(object) def __init__(self, level, debug_domains) def update(self, eventmask, level, debug_domains) def filter(self, event) [YOCTO #10773] Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: corrupted patch for bldcollector URL fileDavid Reyna2018-01-021-1/+1
| | | | | | | | | | Fix a fatal error introduced by a corrupted patch file submission. [YOCTO #12459] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* build.py: Don't return dependencies for tasks which don't existRichard Purdie2017-12-191-0/+6
| | | | | | | | If for example you deltask do_build but it has recrdeps set, its confusing to have that list returned when the task no longer exists (same would apply to deps too if it was set after the deltask). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: add 'nobuild' option to ToasterDavid Reyna2017-12-185-28/+58
| | | | | | | | | | | | Add a 'nobuild' option for starting Toaster without the project and hosted builds support. This allows a Toaster host to provide local build statistics without opening the host to external users building projects. [YOCTO #12315] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: landing page show cmdline captureDavid Reyna2017-12-181-1/+7
| | | | | | | | | | | The Toaster initial landing page needs to show that it still has the command line build capture support in addition to the new project support. [YOCTO #12316] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: enable custom env support for shell callsDavid Reyna2017-12-181-11/+15
| | | | | | | | | | Allow for custom environment additions for git cloning, for example for anspass support. [YOCTO #12193] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: some recipe events do not include packagesDavid Reyna2017-12-181-1/+5
| | | | | | | | | | Allow for "SinglePackageInfo" events that do not include package data, for example OPKGN equal 'lib32-*' or 'lib64-*'. [YOCTO #12204] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: update Toaster for Django 1.11David Reyna2017-12-186-6/+6
| | | | | | | | | Toaster needs to accomodate API changes in Django 1.11. [YOCTO #12192] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: unset BBPATH before starting bbserver for backwards compatibilityDavid Reyna2017-12-181-6/+12
| | | | | | | | | | | | | | | The latest build enviroment script is now setting BBPATH. This breaks building previous releases from a master-based Toaster, because with this newly inherited BBPATH value the 'bitbake.lock' file ends up in the Toaster build directory instead of the project's build directory. Toaster should always clear BBPATH so that the bbserver's environment is clean (enough). [YOCTO #12363] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* providers: Fix determinism issueRichard Purdie2017-12-101-3/+3
| | | | | | | | | | | | | | | | We saw builds where runtime providers were sometimes changing order and the build result was therefore non-deterministic. For example it could show: DEBUG: providers for lib32-initd-functions are: ['lib32-lsbinitscripts', 'lib32-initscripts'] or DEBUG: providers for lib32-initd-functions are: ['lib32-initscripts', 'lib32-lsbinitscripts'] which could cause a test to pass or fail. This change ensures we don't rely on the random order of dictonaries in memory and act deterministically. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: fix unpack of .txz filesAndré Draszik2017-11-291-1/+1
| | | | | | | | .txz is the same as .tar.xz, and can be found in the wild. Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch/wget: use with to ensure the response is closedRoss Burton2017-11-211-1/+2
| | | | | Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch/wget: improve proxy objectRoss Burton2017-11-211-0/+1
| | | | | | | | The connection cache class uses a dummy file object but it doesn't have a closed attribute, so we can't use it in a context manager. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker: fix typo in bitbake -g messagePaul Eggleton2017-11-111-1/+1
| | | | | | | Flatened -> flattened. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tinfoil: ensure get_recipe_info() returns Null if recipe not foundPaul Eggleton2017-11-111-7/+10
| | | | | | | | If a matching recipe is not found then return Null instead of raising KeyError because we were blindly using None as a key for pkg_fn. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: show-recipes: fix help to mention -i supports multiple classesPaul Eggleton2017-11-111-1/+1
| | | | | | | | The -i option supports more than one class, but the help didn't mention that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: remove-layer: support removing multiple layers at a timePaul Eggleton2017-11-111-9/+12
| | | | | | | | If you can add multiple layers at once, it stands to reason that you should also be able to remove more than one at a time. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: add-layer: enable adding multiple layers at oncePaul Eggleton2017-11-111-11/+13
| | | | | | | | | | | | | Allow specifying multiple layers with bitbake-layers add-layer so that you can add more than one in a single command. This is not just useful, it's actually pretty important if you need to add a layer and its dependencies at the same time - since we now go through a parse process when the layer is added, without this you have to add them all in just the right order and wait for the parse each time which is somewhat painful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Fix typo builable -> buildableRichard Purdie2017-11-111-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tinfoil: Ensure we clean up loggersRichard Purdie2017-11-081-2/+8
| | | | | | | | This is primarily paranoid but ensure we remove any loggers we setup either directly or indirectly so the initial state is restored after we exit. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* event: Ensure we clean up loggersRichard Purdie2017-11-081-0/+4
| | | | | | | | Whilst we're likely exiting in this case, clean up the loggers we add so that in the case of certain server retries there is no possibility multiple loggers stack up. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tests/fetch: Add ftp test urlRichard Purdie2017-11-071-0/+1
| | | | | | | Add in a tets ftp url so we ensure ftp urls contnue to work after the loss of the ftp.gnu.org ones. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tests/fetch: use subtests in the wget testsRoss Burton2017-11-071-8/+10
| | | | | | | | | As we test multiple URLs in this these tests and one failing abandons the test, use subtests so all URLs are tested. This should help us identify patterns in the failing URLs. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tests/fetch: Switch gnu.org urls from ftp -> http/httpsRichard Purdie2017-11-071-3/+3
| | | | | | | | | | The ftp server at ftp.gnu.org is likely to be retired at some point soon so siwtch over to the http/https services. This means bitbake-selftest doesn't have ftp test urls, however finding stable ftp test servers is proving increasingly hard. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2: Fix missing logger import in repo fetcherOleksandr Andrushchenko2017-11-071-0/+1
| | | | | | | | | | | After cleaning deprecated API usage repo fetcher is missing logger as it was indirectly imported via deprecated bb.data. Fix this by importing logger directly. Fixes: 9752fd1c10b8 ("fetch2: don't use deprecated bb.data APIs") Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: Add support for multiple recipes at onceMark Hatle2017-11-071-5/+16
| | | | | | | | | | | | bitbake-layers show-recipes and show-appends supported listing all recipes or one recipe. Adjust the system to permit specifying more then one recipe. Also update show-appends to match the --help description and support file style wildcards for selecting the recipe to display. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: be more explicit when warning about locale choiceRoss Burton2017-11-072-3/+2
| | | | | Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tests/fetch: skip network tests the idiomatic wayRoss Burton2017-11-051-212/+222
| | | | | | | | Instead of not even having the test functions if network tests are disabled, use a custom decorator to mark the network tests and skip them. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* main: Give a user readable error if we can't locate topdirRichard Purdie2017-10-311-0/+8
| | | | | | | | | | | | | | | | | Currently if you run bitbake in an invalid directory, the user experience is poor: birbake/lib/bb/main.py", line 427, in setup_bitbake topdir, lock = lockBitbake() File "./bitbake/lib/bb/main.py", line 494, in lockBitbake lockfile = topdir + "/bitbake.lock" TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' This ensures we exit straight away with a better error message. [YOCTO #12163] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker: fix watching directories with Python 3.6+Paul Eggleton2017-10-311-1/+11
| | | | | | | | | | | | | | | | | | | | | | In Python 3.6, glob.glob() was reimplemented to use os.scandir() (which itself appeared in Python 3.5), thus our monkey patching of os.listdir() here was no longer effective. The end result was not only that bitbake wouldn't notice added recipes or bbappends with BB_SERVER_TIMEOUT set when being run with Python 3.6 (the shipped Python version on Fedora 26 and some other distribution versions), it also broke devtool modify, devtool upgrade and devtool extract since they rely on the ability to create a bbappend on the fly and have bitbake pick it up. To fix it, do the same monkey patching for os.scandir(), which needs to be conditional upon that actually existing since we have to support Python 3.4 that doesn't have it. Long term we should probably look for a better way to handle this that doesn't involve monkey patching Python library code. Fixes [YOCTO #12185]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update version to 1.37.0 for developmentKhem Raj2017-10-312-2/+2
| | | | | Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Update version to 1.36.0 for stable releaseKhem Raj2017-10-312-2/+2
| | | | | Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/highlight.pack.js: Fix corrupted fileRichard Purdie2017-10-101-1/+1
| | | | | | The newly added file in the last commit was corrupted, fix it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: Remove prettifyDavid Reyna2017-10-106-35/+6
| | | | | | | | | | | | Remove "prettify.js" and "prettify.css" due to license issues with Apache2. Replace with "highlight.pack.js" with its BSD3 License. [YOCTO #12206] Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Brian Avery <brian.avery@intel.com> Signed-off-by: David Reyna <david.reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* README: new readme file including main aspects of the projectLeonardo Sandoval2017-10-071-0/+35
| | | | | | | | Includes brief description of the project, pointers to website, documentation, mailing list and source code. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual-metadata: include a space on a append exampleLeonardo Sandoval2017-10-071-1/+1
| | | | | | | | By definition, the override operator "_append" does not include a space, so include it. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual/bitbake-user-manual: remove YP as build systemLeonardo Sandoval2017-10-071-1/+1
| | | | | | | YP is not a build system, so it can not be used as build system example. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Add args.force to localargs before do_add_layerJeremy A. Puhlman2017-10-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | Adding layer "meta-signing-key" to conf/bblayers.conf Traceback (most recent call last): File "/local/build/project/build/poky/bitbake/bin/bitbake-layers", line 103, in <module> ret = main() File "/local/build/project/build/poky/bitbake/bin/bitbake-layers", line 96, in main return args.func(args) File "/local/build/project/build/poky/bitbake/lib/bblayers/layerindex.py", line 250, in do_layerindex_fetch self.do_add_layer(localargs) File "/local/build/project/build/poky/bitbake/lib/bblayers/action.py", line 44, in do_add_layer if not (args.force or notadded): AttributeError: 'Namespace' object has no attribute 'force' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual: add more gitsm fetcher limitationsMikko Ylinen2017-10-061-1/+3
| | | | | | | | | | | | | The submodule sources fetched by the gitsm fetcher are not automatically tracked by the licensing and archiver infrastructures. Add these limitations to the existing 'warnings' sections for gitsm. [YOCTO #11594] Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: build missing toaster.conf settingsDavid Reyna2017-10-061-2/+3
| | | | | | | | | | | | | | | The bitbake server changed such that the Toaster custom settings from 'toaster.conf' and 'toaster-bblayers.conf' that were set when the '--server-only' mode was started were lost when the subsequent build request happened, resulting in builds missing all custom changes. This patch asserts those environment settings in both server calls. [YOCTO #12194] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual: Updated kernel version in PREFERRED_VERSION exampleScott Rifenbark2017-09-261-1/+1
| | | | | Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: lib/bb/utils: fix movefile() copy to dir fallbackMardegan, Alberto2017-09-221-9/+10
| | | | | | | | | When the destination is a directory, building the the destination file path is always needed. That's because even if the copy fallback is taken, it's always followed by a rename. Signed-off-by: Alberto Mardegan <amardegan@luxoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* server/process: Increase runCommand timeoutRichard Purdie2017-09-221-1/+1
| | | | | | | | | | We were seeing cases where we could hit the 5s timeout on large/fast machines running many different tasks at once. Increase this to 30s since the main connection timeout path should no longer hit this slow path. [YOCTO #12116] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: allow dots in user path namesDavid Reyna2017-09-173-6/+6
| | | | | | | | | | The dot '.' character should be allowed in the user paths for local non-git layers, DL_DIR, and SSTATE_DIR. [YOCTO #10650] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: reserve HEAD from imported layersDavid Reyna2017-09-173-7/+18
| | | | | | | | | | | | | | | | | | | The HEAD reference in Toaster layers are reserved for the "Local Yocto Project" layers, stored at the top directory. Imported layers are not allowed to use this since they are managed differently - for example the 'remotes' will collide. Fix the add layer handler to not drop the data fields when it is a git repo. Explicitly inform the user when an internal Toaster error is returned via AJAX, so that they know why clicking the layer add button did not do anything. [YOCTO #9924] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker: ensure monkey-patching in collect_bbfiles() gets undone on errorPaul Eggleton2017-09-141-19/+19
| | | | | | | | | | | | In collect_bbfiles() we're monkey-patching os.listdir in order to find which directories to watch, and then undoing that when we're finished - however if an exception occurred for any reason there was nothing to ensure the latter occurred. This may not have caused any issues, but as this kind of thing really ought to be secured using try...finally just in case, so do that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker: fix watching empty directoriesPaul Eggleton2017-09-141-3/+6
| | | | | | | | | | | | | | | | | The code that was supposed to watch directories along BBFILES for creation of new files wasn't working in the case where the directory did not initially contain any matching files - since in updateCache() we are passing the directory path to add_filewatch() and the latter function calls os.path.dirname() on the path on the assumption that it is a file path, and thus the parent of the directory got watched but not the directory itself. (If the directory wasn't empty everything worked fine since add_filewatch() was called elsewhere with the path to one of the files in that directory, and thus the directory got watched). Add a parameter to add_filewatch() to tell it we are passing it directory path(s) rather than file path(s). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual: Edits to "inherit" section.Scott Rifenbark2017-09-131-3/+3
| | | | | | | | Fixes [YOCTO #12031] Applied minor wording changes based on review feedback. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
* bitbake-user-manual: Removed bad linksScott Rifenbark2017-09-131-12/+2
| | | | | | | | | | Fixes [YOCTO #12029] There were four links to an old blog post that was leveraged for the hello world example. The post has evidently been removed and the links no longer resolved. I have taken them out. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
* bitbake-user-manual: Updated inherit and INHERIT sectionsScott Rifenbark2017-09-131-5/+13
| | | | | | | | | | | | | Fixes [YOCTO #12031] Moved an inappropriate example from the bottom of the INHERIT Configuration Directive section up into the inherit directive section. Updated the example at the end of the INHERIT Configuration Directive section to apply to INHERIT. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>