aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2017-11-08cooker: fix typo in bitbake -g messagepaule/bb-fixes2Paul Eggleton
Flatened -> flattened. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-11-08tinfoil: ensure get_recipe_info() returns Null if recipe not foundPaul Eggleton
If a matching recipe is not found then return Null instead of raising KeyError because we were blindly using None as a key for pkg_fn. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-11-07tests/fetch: Add ftp test urlRichard Purdie
Add in a tets ftp url so we ensure ftp urls contnue to work after the loss of the ftp.gnu.org ones. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-07tests/fetch: use subtests in the wget testsRoss Burton
As we test multiple URLs in this these tests and one failing abandons the test, use subtests so all URLs are tested. This should help us identify patterns in the failing URLs. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-07tests/fetch: Switch gnu.org urls from ftp -> http/httpsRichard Purdie
The ftp server at ftp.gnu.org is likely to be retired at some point soon so siwtch over to the http/https services. This means bitbake-selftest doesn't have ftp test urls, however finding stable ftp test servers is proving increasingly hard. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-07fetch2: Fix missing logger import in repo fetcherOleksandr Andrushchenko
After cleaning deprecated API usage repo fetcher is missing logger as it was indirectly imported via deprecated bb.data. Fix this by importing logger directly. Fixes: 9752fd1c10b8 ("fetch2: don't use deprecated bb.data APIs") Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-07bitbake-layers: Add support for multiple recipes at onceMark Hatle
bitbake-layers show-recipes and show-appends supported listing all recipes or one recipe. Adjust the system to permit specifying more then one recipe. Also update show-appends to match the --help description and support file style wildcards for selecting the recipe to display. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-07bitbake: be more explicit when warning about locale choiceRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05tests/fetch: skip network tests the idiomatic wayRoss Burton
Instead of not even having the test functions if network tests are disabled, use a custom decorator to mark the network tests and skip them. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-31main: Give a user readable error if we can't locate topdirRichard Purdie
Currently if you run bitbake in an invalid directory, the user experience is poor: birbake/lib/bb/main.py", line 427, in setup_bitbake topdir, lock = lockBitbake() File "./bitbake/lib/bb/main.py", line 494, in lockBitbake lockfile = topdir + "/bitbake.lock" TypeError: unsupported operand type(s) for +: 'NoneType' and 'str' This ensures we exit straight away with a better error message. [YOCTO #12163] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-31cooker: fix watching directories with Python 3.6+Paul Eggleton
In Python 3.6, glob.glob() was reimplemented to use os.scandir() (which itself appeared in Python 3.5), thus our monkey patching of os.listdir() here was no longer effective. The end result was not only that bitbake wouldn't notice added recipes or bbappends with BB_SERVER_TIMEOUT set when being run with Python 3.6 (the shipped Python version on Fedora 26 and some other distribution versions), it also broke devtool modify, devtool upgrade and devtool extract since they rely on the ability to create a bbappend on the fly and have bitbake pick it up. To fix it, do the same monkey patching for os.scandir(), which needs to be conditional upon that actually existing since we have to support Python 3.4 that doesn't have it. Long term we should probably look for a better way to handle this that doesn't involve monkey patching Python library code. Fixes [YOCTO #12185]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-31bitbake: Update version to 1.37.0 for developmentKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-31bitbake: Update version to 1.36.0 for stable releaseKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-10toaster/highlight.pack.js: Fix corrupted fileRichard Purdie
The newly added file in the last commit was corrupted, fix it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-10toaster: Remove prettifyDavid Reyna
Remove "prettify.js" and "prettify.css" due to license issues with Apache2. Replace with "highlight.pack.js" with its BSD3 License. [YOCTO #12206] Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Brian Avery <brian.avery@intel.com> Signed-off-by: David Reyna <david.reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07README: new readme file including main aspects of the projectLeonardo Sandoval
Includes brief description of the project, pointers to website, documentation, mailing list and source code. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07bitbake-user-manual-metadata: include a space on a append exampleLeonardo Sandoval
By definition, the override operator "_append" does not include a space, so include it. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07bitbake-user-manual/bitbake-user-manual: remove YP as build systemLeonardo Sandoval
YP is not a build system, so it can not be used as build system example. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-06Add args.force to localargs before do_add_layerJeremy A. Puhlman
Adding layer "meta-signing-key" to conf/bblayers.conf Traceback (most recent call last): File "/local/build/project/build/poky/bitbake/bin/bitbake-layers", line 103, in <module> ret = main() File "/local/build/project/build/poky/bitbake/bin/bitbake-layers", line 96, in main return args.func(args) File "/local/build/project/build/poky/bitbake/lib/bblayers/layerindex.py", line 250, in do_layerindex_fetch self.do_add_layer(localargs) File "/local/build/project/build/poky/bitbake/lib/bblayers/action.py", line 44, in do_add_layer if not (args.force or notadded): AttributeError: 'Namespace' object has no attribute 'force' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-06bitbake-user-manual: add more gitsm fetcher limitationsMikko Ylinen
The submodule sources fetched by the gitsm fetcher are not automatically tracked by the licensing and archiver infrastructures. Add these limitations to the existing 'warnings' sections for gitsm. [YOCTO #11594] Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-06toaster: build missing toaster.conf settingsDavid Reyna
The bitbake server changed such that the Toaster custom settings from 'toaster.conf' and 'toaster-bblayers.conf' that were set when the '--server-only' mode was started were lost when the subsequent build request happened, resulting in builds missing all custom changes. This patch asserts those environment settings in both server calls. [YOCTO #12194] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-26bitbake-user-manual: Updated kernel version in PREFERRED_VERSION exampleScott Rifenbark
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-22bitbake: lib/bb/utils: fix movefile() copy to dir fallbackMardegan, Alberto
When the destination is a directory, building the the destination file path is always needed. That's because even if the copy fallback is taken, it's always followed by a rename. Signed-off-by: Alberto Mardegan <amardegan@luxoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-22server/process: Increase runCommand timeoutRichard Purdie
We were seeing cases where we could hit the 5s timeout on large/fast machines running many different tasks at once. Increase this to 30s since the main connection timeout path should no longer hit this slow path. [YOCTO #12116] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-17toaster: allow dots in user path namesDavid Reyna
The dot '.' character should be allowed in the user paths for local non-git layers, DL_DIR, and SSTATE_DIR. [YOCTO #10650] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-17toaster: reserve HEAD from imported layersDavid Reyna
The HEAD reference in Toaster layers are reserved for the "Local Yocto Project" layers, stored at the top directory. Imported layers are not allowed to use this since they are managed differently - for example the 'remotes' will collide. Fix the add layer handler to not drop the data fields when it is a git repo. Explicitly inform the user when an internal Toaster error is returned via AJAX, so that they know why clicking the layer add button did not do anything. [YOCTO #9924] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-14cooker: ensure monkey-patching in collect_bbfiles() gets undone on errorPaul Eggleton
In collect_bbfiles() we're monkey-patching os.listdir in order to find which directories to watch, and then undoing that when we're finished - however if an exception occurred for any reason there was nothing to ensure the latter occurred. This may not have caused any issues, but as this kind of thing really ought to be secured using try...finally just in case, so do that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-14cooker: fix watching empty directoriesPaul Eggleton
The code that was supposed to watch directories along BBFILES for creation of new files wasn't working in the case where the directory did not initially contain any matching files - since in updateCache() we are passing the directory path to add_filewatch() and the latter function calls os.path.dirname() on the path on the assumption that it is a file path, and thus the parent of the directory got watched but not the directory itself. (If the directory wasn't empty everything worked fine since add_filewatch() was called elsewhere with the path to one of the files in that directory, and thus the directory got watched). Add a parameter to add_filewatch() to tell it we are passing it directory path(s) rather than file path(s). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-13bitbake-user-manual: Edits to "inherit" section.Scott Rifenbark
Fixes [YOCTO #12031] Applied minor wording changes based on review feedback. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2017-09-13bitbake-user-manual: Removed bad linksScott Rifenbark
Fixes [YOCTO #12029] There were four links to an old blog post that was leveraged for the hello world example. The post has evidently been removed and the links no longer resolved. I have taken them out. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2017-09-13bitbake-user-manual: Updated inherit and INHERIT sectionsScott Rifenbark
Fixes [YOCTO #12031] Moved an inappropriate example from the bottom of the INHERIT Configuration Directive section up into the inherit directive section. Updated the example at the end of the INHERIT Configuration Directive section to apply to INHERIT. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2017-09-13bitbake-user-manual: Clarified inherit, include, require relationshipsScott Rifenbark
Fixes [YOCTO #12032] I applied some re-writing to help clarify the relationships between the inherit, include, and require directives. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
2017-09-12server/process: ensure server failure log is limited to current sessionPaul Eggleton
Printing the last 10 lines of bitbake-cookerdaemon.log when the server fails to start can sometimes result in printing the output from a previous run, which could lead the user completely down the wrong path in terms of the cause of the failure. Use a known start text containing the time which we can then look for when scanning through the log, and then grab the last 10 lines of that part instead. Fixes [YOCTO #11903]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-12runqueue: fix BB_SETSCENE_ENFORCE with empty BB_SETSCENE_ENFORCE_WHITELISTPaul Eggleton
The whitelist shouldn't have to be populated in order for the enforcement to work properly - check if the list is not None in order to determine whether the functionality is enabled or not since that is how the function that sets up the list behaves. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11process/cooker: Improve readypipe handlingRichard Purdie
Issues in start are not being correctly detected by the current readypipe code. Change it to use specific "ready" or "fail" messages to correctly determine the correct failure mode and avoid bitbake seeming to hang (it does currently timeout eventually). [YOCTO #12062] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11toaster: update pyro to rocko as stable releaseDavid Reyna
Update the Toaster stable release selection to 'Rocko' and the bitbake version to '1.36'. [YOCTO #12037] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11toaster: handle early exceptionsDavid Reyna
Stop the pending build and report to the user if there is an internal exception due to a git error, a bitbake server error, or if the server-only mode halts due to for example a user syntax error in a layer or recipe. These exceptions were not caught because they occure before the normal toastergui processing was started. [YOCTO #12056] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-07tests/fetch: handle network failures gracefullyRoss Burton
If there is a network failure the return value from latest_versionstring() is ('','') which later causes an exception when comparing versions. Improve this by checking the return value and failing the test early. [ YOCTO #12053 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-07fetch2: add stub latest_versionstring to FetchMethodRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-07Replace deprecated git branch parameter "--set-upstream"Andre Rosa
Since 2017-08-17 (git version 2.14.1.473.g3ec7d702a) using deprecated git branch parameter "--set-upstream" causes a fetcher error. Replace it by "--set-upstream-to". https://git.kernel.org/pub/scm/git/git.git/commit/?id=52668846ea2d41ffbd87cda7cb8e492dea9f2c4d says, it's deprecated since 2012-08-30 so hopefully all still supported host distributions have new enough git to support "--set-upstream-to". ERROR: PACKAGE do_unpack: Fetcher failure: ...; git -c core.fsyncobjectfiles=0 branch --set-upstream master origin/master failed with exit code 128, output: fatal: the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead. ERROR: PACKAGE do_unpack: Function failed: base_do_unpack Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-05cooker: clear extra config settings and remote datastores on client disconnectPaul Eggleton
When the UI disconnects, we can throw away any server-side remote datastores we created in response to calls from the UI, and we *must* drop everything in extraconfigdata or it will taint any future operations. Dropping extraconfigdata upon disconnect fixes taskhash mismatch errors when running devtool.DevtoolTests.test_devtool_update_recipe_local_files within oe-selftest with BB_SERVER_TIMEOUT=100 in OpenEmbedded. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-05cooker.py: Fix layer priority processingMark Hatle
If you have a layer with a blank BBFILE_PATTERN the layer was ignored when processing the list of layers with priorities. This list is not only used for processing recipes, but also by additional programs such as bitbake-layers show-layers. Without this change, a layer that provides configuration or classes only does now show up in show-layers, which is used by the yocto-compat-layer.py script. This causes a failures in the compatibility check. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-05siggen: move reset() definition to base SignatureGenerator classPaul Eggleton
If we're implementing reset() in SignatureGenerator at all (and we need to for a basic non-OE BitBake setup where that is the default signature generator), then we need it to be clearing out the internal values properly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: edit column list not sortedDavid Reyna
The list of columns in the many 'Edit Columns' pop-ups became unsorted with the 'Toaster Table' implementation. These entries need to be gathered and sorted in the column processing. [YOCTO #12004] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: recipe links broken for default layersDavid Reyna
The default layers are missing the recipe link definitions in the fixture files, and because they are predefined they do not get the updated information from the Layer Index. [YOCTO #12006] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: missing ToasterSetting importDavid Reyna
The file "lsupdates.py" is missing the import of 'ToasterSetting', and that breaks setting up and updating Toaster and its database. [YOCTO #12036] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: set default pokydirname if no external layersOlaf Mandel
If no external layers are defined, pokydirname is not set. Rectify this by taking the 'be.sourcedir' as the pokydirname. [YOCTO #12015] Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: debug message for lists layers missing separatorsOlaf Mandel
One of the debug messages during build contains a list of all layers but without spaces or other separators between them. Use pformat instead. [YOCTO #12014] Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: Order column in Tasks selectableAwais Belal
The build page provides tables related to performance that cover build time, CPU time and disk IO. The "Edit columns" drop down does not allow selection of the Order column and makes it hidden as well which is not accurate from user interaction point of view. This patch enables the hideable property for the Order column so it is hidden by default but the user can enable it through the drop down if need be. [YOCTO #11040] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-04toaster: display error when the fstype select is emptyDavid Reyna
There must be at least one FSTYPE selected in the Toaster bitbake variable editor page. When the user deselects all the "Save" button gets disabled, but the error message is missing. [YOCTO #8126] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>