summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2021-05-06runqueue: Handle deferred task rehashing in multiconfig buildsyocto-3.3.12021-04.1-hardknott1.50.1Richard Purdie
If the hash of a task changes and that hash is a deferred task (e.g. a multiconfig build), we need to ensure that the hash change propagates through to all the tasks else the build will run multiple copies of the task, sometimes with oddly differing results as the outhashes of native tasks built in differing locations can confuse things. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 2db571324f755edc4981deecbcfdf0aaa5a97627) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06runqueue: Fix multiconfig deferred task sstate validity caching issueRichard Purdie
We were testing the validity of deferred tasks setscene status "up front" which is very unlikely to succeed and leads to cache invalidation issues. With the change to rebuild the deferred task list, this status becomes out of sync. The result was tasks being executed when they should not have been leading to extra work for the build unnecessarily. Instead, don't process validity status for deferred tasks and assume their data will become available. If it doesn't, this will now result in a build error as the setscene task will fail and the main task will run instead. In theory we could try and track the state changes in the deferred list and re-test validity then but I'm not sure it is worth the effort when the other code path and errors in setscene tasks will give a pretty good idea of what is happening anyway. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit edcafac13b3b241b6687419e59018d21811507a1) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27bitbake: tests/fetch: remove write protected files tooMikko Rapeli
For some reason several git-annex files in Debian 10 buster are read-only and removing them with "rm -rf" fails. Fixes test failures like: $ bitbake-selftest ... rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied EE..................................ssss.sssssssssssssss.sssss....................................................................................................... ====================================================================== ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex fetcher, ud = self.fetch_shallow(uri) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow bb.utils.remove(ud.clonedir, recurse=True) File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path)) File "/usr/lib/python3.7/subprocess.py", line 347, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1. Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod the whole temporary directory which should cover any directory name differences between different git-annex versions. Fixes tests failing after chmod call: Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/ Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7729ef2983c72867e99fad82d671069ba5cb32b2) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27fetch/gitsm: Fix crash when using git LFS and submodulesNiels Avonds
Gitsm fetcher crashes when cloning a repository that contains LFS files. This happens because the unpack method is called during download, but the submodules have not been downloaded yet at this point. This issue was introduced in this commit: 977b7268bf4fd425cb86d4a57500350c9b829162 [YOCTO #14283] Signed-off-by: Niels Avonds <niels@codebits.be> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27bitbake-server: ensure server timeout is a floatRoss Burton
bitbake-server is spawned by process.py and passes the arguments it is given to ProcessServer. There's some type confusion here: bitbake-server is called with a string representation of the timeout, which may be None. If the timeout is not set, pass 0 instead of None. Inside bitbake-server a ProcessServer is created which expects the timeout to be a float not a string, so always float() the value. [ YOCTO #14350 ] Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c93ae1f861208f6d39fd15c84fbcd0e2b54331f5) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27bitbake: tests/fetch: fix test execution without .gitconfigMikko Rapeli
A CI user validating changes does not have any git push rights or even a .gitconfig file so fix tests so that they run by setting the user.name and user.email for the repo before committing changes. Fixes errors like: ERROR: test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist (bb.tests.fetch.GitShallowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 2055, in test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist self.add_empty_file('a') File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1562, in add_empty_file self.git(['commit', '-m', msg, path], cwd) File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1553, in git return bb.process.run(cmd, cwd=cwd)[0] File "/home/builder/src/base/poky/bitbake/lib/bb/process.py", line 184, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git commit -m a a' failed with exit code 128: *** Please tell me who you are. Run git config --global user.email "you@example.com" git config --global user.name "Your Name" to set your account's default identity. Omit --global to set the identity only in this repository. Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 57c0811f1ee19b6619f4840a39e01e3cb98c34c4) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27runqueue: Fix deferred task issuesRichard Purdie
In a multiconfig situation there are circumstances where firstly, tasks are deferred when they shouldn't be, then later, tasks can end up as both covered and not covered. This patch fixes two related issues. Firstly, the stamp validity checking is done up front in the build and not reevaulated. When rebuilding the deferred task list after scenequeue hash change updates, we need therefore need to check if a task was in notcovered *or* covered when deciding to defer it. This avoids strange logs like: NOTE: Running setscene task X of Y (mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch_setscene) NOTE: Deferring mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch after mc:host:/A/alsa-state.bb:do_deploy_source_date_epoch where tasks have run but are then deferred. Since we're recalculating the whole list, we also need to clear it before iterating to rebuild it. By ensuring covered tasks aren't added to the deferred queue, the covered + notcovered issue should also be avoided. in the task deadlock forcing code. [YOCTO #14342] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3c8717fb9ee1114dd80fc1ad22ee6c9e312bdac7) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-15bitbake-user-manual: add REQUIRED_VERSION and adjust PREFERRED_VERSION entryPaul Eggleton
Add REQUIRED_VERSION, add a reference to it in PREFERRED_VERSION and adjust the opening statement to read slightly better. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-15bitbake-user-manual: document no support for using passwords in git URLsPaul Eggleton
This is based on the comment added in revision aded964eed4ce5a725ed1ab477efabc86b1aa481. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-09bitbake: Update version to 1.50.0 stable release seriesRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06runqueue: Further fixes for confused setscene tasksyocto-3.32021-04-hardknott1.50.0Richard Purdie
There is further evidence of tasks ending up being "covered" and "notcovered" which shouldn't happen and is bad. The code that caused this problem last time appears to have issues where stamps for tasks already exist. Split out the setscene stamp checking code to a separate function and use this when checking "hard dependencies" (like pseudo-native) so that if the stamps exist and it will be "covered", it is not put on the notcovered list. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-23doc: Update links to documentationMichael Opdenacker
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-23runqueue/event: Add an event for notifying of stale setscene tasksRichard Purdie
Use the new functionality in build.py to identify stale setscene tasks and send an event to the metadata listing them. The metadata then has the option of performing cleanup operations if it thinks that appropriate. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-23build: Add find_stale_stamps functionRichard Purdie
Add a new function which compares the stamp filename we want (including taskhash) with what is in the stamp directory (using the clean mask). This tells us which stamp files are stale and are due to be rerun. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-22doc: fix syntax error in layer.conf exampleRobert P. J. Day
While this example really needs to be rewritten to not define multiple patterns in the same layer.conf, as long as it's there, it might as well be syntactically correct. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-22doc: fix glossary link for BB_INVALIDCONF variableRobert P. J. Day
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-22doc: mention that addtask handles multiple dependenciesRobert P. J. Day
Add a note explaining that "addtask" can accept multiple dependencies, just in case someone runs across such an example and is confused. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15doc: move BBFILES_DYNAMIC for alphabetical orderRobert P. J. Day
Since BBFILES_DYNAMIC does not have a "BB_" prefix, it belongs further down in the variable glossary. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15codeparser: Fix TypeError in bitbake debug modeTomasz Dziendzielski
Commit 75f87db413 fixed the confusion between bitbake and python logger but in codeparser still old method of setting debug level was used causing TypeError, because debug level value was incorrectly returned and assigned to event.msg. | File "./bitbake/lib/bb/ui/knotty.py", line 660, in main | event.msg = event.fn + ': ' + event.msg | TypeError: can only concatenate str (not "int") to str [YOCTO #14298] Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15bblayers/query: Update to handle REQUIRED_VERSIONRichard Purdie
Ii is unclear whether any changes are needed to bblayers are needed to handle the extra data from REQUIRED_VERSION. Update to at match the new API, at present it doesn't look necessary to handle the required version data. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: cooker: Add REQUIRED_VERSION checksCharlie Davies
Add logic to check that if REQUIRED_VERSION has been set that the cooker class method findBestProviders properly handles the case where the REQUIRED_VERSION has not been found. Fixes [YOCTO #10096] Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: lib: add Required Version to Bitbake --show-versions commandCharlie Davies
This commit adds a new column to the Bitbake --show-versions command called Required Version. This column will display any packages which have a REQUIRED_VERSION successfully set. Fixes [YOCTO #10096] Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: providers: check for REQUIRED_VERSION in _filterProvidersCharlie Davies
Before the REQUIRED_VERSION variable was introduced the PREFERRED_VERSION variable allowed for a fallback to the next most suitable version. Since REQUIRED_VERSION does not allow a fallback to a different version implement a check in the _filterProviders function to make sure that if a requested REQUIRED_VERSION is not found then the function returns no eligible providers have been found. Fixes [YOCTO #10096] Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: providers: introduce logic for REQUIRED_VERSION variableCharlie Davies
This commit adds checks during the findPreferredProvider function for a new variable REQUIRED_VERSION. This can be set, in exactly the same manner as PREFERRED_VERSION, on a per package basis to enforce the use of a particular version of a package. REQUIRED_VERSION is similar in behaviour to PREFERRED_VERSION except if the version specified by REQUIRED_VERSION is not found an error occurs and the execution of Bitbake stops. Fixes [YOCTO #10096] Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: providers: consistent single line formatting between functionsCharlie Davies
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12bitbake: providers: remove unneeded logging callCharlie Davies
The logging line for when no eligible providers are found only adds unnecessary noise and is not required. As the stack unwinds better logging occurs which satisfactorily describes the no eligible providers error condition. Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12gitignore: ignore runqueue-tests/bitbake-cookerdaemon.logPaul Gortmaker
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12tests/color: Fix event register to pass the datastoreRichard Purdie
Adding the "if d is None" to the event handling code meant some of these tests stopped working. The reason is that len(d) was zero but not equal to None. Passing the data object to the register() function in the test correctly registers the event handler and avoids the problem, it just happened to work previously, incorrectly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11event: Fix another 'if d' test to test for None explictlyRichard Purdie
This should have been fixed in the previous patch, well spotted Chris! Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11event: Fix multiconfig event handler change performance regressionsRichard Purdie
There were two issues in this code, firstly the code could stack duplicates in the variable, secondly, calling "if data" caused the datastore to compute len(data) which is comparitively expensive. Checking "if data is not None" is much much faster/cheaper. The issue was clear from "bitbake -p -P" output where the time in register() showed large amounts of time in the __len__ function of the datastore. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11tinfoil: Honor quiet when parsing recipesManuel Leonhardt
When using parse_recipes, honor quiet so that scripts and custom plugins for recipetool are able to mute progress bars from bitbake that would otherwise print to STDOUT. Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11runqueue: Print pseudo.log if fakeroot task failedTomasz Dziendzielski
Currently if pseudo fails we can only see the path to pseudo.log. If we have no access to server and can only rely on bitbake log then debugging becomes impossible. This printing needs to be added in runqueue level, not inside task execution, because in some cases task fails with pseudo abort really early and we don't even see any log. In this change I'm adding pseudo log printing in every fakeroot task failure that logged `mismatch`, `error` or `fatal` to logfile, because we have no other way to communicate with pseudo if it failed or not. Only lines from last pseudo server execution will be printed. Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com> Signed-off-by: Adrian Walag Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Mikolaj Lasota <mikolaj.lasota@protonmail.com> Signed-off-by: Wiktor Baura <wbaura@gmail.com> Signed-off-by: Kamil Kwiek <kamil.kwiek@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11event.py: fix regression about INVALIDCONFChen Qi
A previous commit uses __BBHANDLERS_MC to record all handlers, but it does not take into consideration of INVALIDCONF, thus causing regression. We need to record the name before returned AlreadyRegistered, otherwise, when reparsing due to INVALIDCONF, bascially all handlers are not called. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11docs: Add AZ_SAS definition to glossaryAlejandro Hernandez Samaniego
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11docs: Add Az fetcher documentationAlejandro Hernandez Samaniego
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11bitbake: Add Azure Storage fetcher implementationAlejandro Hernandez Samaniego
Allows bitbake to fetch from an Azure Storage account. The fetcher submodule is compatible with the az:// URI protocol, its functionality is based on bitbakes wget fetcher, superior in performance to using a propietary tool like azcopy which can handle cloud storage account operations with more functionality (that we dont need in a fetcher) but less compatibility. A sample URI uses can be defined in the following way: SRC_URI = "az://<azure-storage-account>.blob.core.windows.net/<container>/foo.tar.xz" This fetcher can easily be used with PREMIRRORS and SSTATE_MIRRORS, e.g.: SSTATE_MIRRORS = "file://.* az://<azure-storage-account>.blob.core.windows.net/sstate-cache/PATH;downloadfilename=PATH \n" PREMIRRORS_prepend = "\ git://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ ftp://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ http://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ https://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \ " Can also be used with non-public access Azure Storage accounts/containers via a Shared Access Signature by declaring the AZ_SAS variable which will be automatically used by the fetcher: AZ_SAS="?sv=2000-01-01&ss=...&sig=somesignature" Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-09__init__.py: Fix bitbake debug log handlingRichard Purdie
For a while I've been puzzled as to why debug logging from runqueue wouldn't appear on the console with -DD. The logic in the bbdebug handling is inverted so fix it and now we see the expected messages from runqueue with -D and -DD. This should then let us debug other issues using those log messages. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-09runqueue: Add setscene task overlap sanity checkRichard Purdie
We've seen hard to debug issues where a task ends up in both the covered and notcovered list. Add a sanity check to ensure if this happens in future, we see it in the logs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-09runqueue: Fix task execution corruption issueRichard Purdie
We've seen occasional issues where linux-yocto:do_compile_kernelmodules would run without do_shared_workdir running before it. do_shared_workdir is an setscene task but never has an sstate object generated so it will always rerun. This should not happen since compile_kernemodules should only execute if a setscene that depends on it didn't run and that should trigger do_shared_workdir not to be marked as covered. The issue is that build-appliance-image:do_package is one of the tasks which covers linux-yocto:do_compile_kernelmodules but it is also a noexec task and has a dependecy on pseudo-native:do_populate_sysroot. In the problem case, pseudo-native:do_populate_sysroot is unavailable but marked as covered since it is noexec. The "harddeps" code then also marks it as notcovered. No task should ever be both covered and notcovered and this is where the problems come from. The solution is for the harddeps code only to to fail tasks if they've not already been handled in some way. The code is assuming code couldn't have handled revdeps at this point but we now have clear evidence they can. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21fetch2/wget: Avoid crashing when connection drops mid checkstatusAlejandro Hernandez Samaniego
If an exception is raised when running host python code, the fetcher immediately crashes, this might be temporary depending on the servers reliability. Catch the exception when the connection was reset and try once again to fetch the data. File: '/usr/lib/python3.8/socket.py', lineno: 669, function: readinto 0665: if self._timeout_occurred: 0666: raise OSError("cannot read from timed out object") 0667: while True: 0668: try: *** 0669: return self._sock.recv_into(b) 0670: except timeout: 0671: self._timeout_occurred = True 0672: raise 0673: except error as e: Exception: ConnectionResetError: [Errno 104] Connection reset by peer Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-20event: Fix broken builds when multiconfig has a hyphen in the nameJoshua Watt
5f7fdf7b2d ("bitbake: event: Prevent bitbake from executing event handler for wrong multiconfig target") broke multiconfig builds contain a hyphen, since it's attempt to use the multiconfig as part of a function name and python functions are not allowed to contain a hyphen. Rework the bitbake multiconfig test to test a multiconfig with a hyphen and one with an underscore to validate this doesn't break in the future. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-20Force parser shutdown after catching an exceptionJan Brzezanski
Commit bebef58b21bdff7a3ee1fa2449b7df19144f26fd introduced forcing parser shutdown as default in case of build abort. In this case bitbake sometimes hangs after facing error during parsing, waiting for child processes to finish. Killing it then will spawn zombie processes. Thus we force the shutdown after catching an exception. Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-20bitbake: providers: use pythonic empty list checkCharlie Davies
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-20bitbake: providers: fix incorrect return type bugCharlie Davies
If no eligible providers are found then an integer zero is returned. This causes the following error, in two possible places in taskdata.py, when the return value is used in a list comprehension: [snip] eligible = [p for p in eligible if not p in self.failed_fns] TypeError: 'int' object is not iterable [\snip] Fix by returning the variable eligible itself, of type list. Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-16contrib: Add Dockerfile for building hash serverJoshua Watt
Adds a Dockerfile for build the reference hash equivalence server in a container Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-16bitbake: Bump version to 1.49.2Richard Purdie
This allows metadata to rely upon BB_DEFAULT_UMASK. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-16bitbake-worker/runqueue: Add support for BB_DEFAULT_UMASKRichard Purdie
Currently each task has to have a umask specified individually. This is leading to determinism issues since it is easy to miss specifying this for an extra task. Add support for specifing the default task umask globally which simplifies the problem. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-11event: Prevent bitbake from executing event handler for wrong multiconfig targetTomasz Dziendzielski
When multiconfig is used bitbake might try to run events that don't exist for specific mc target. In cooker.py we pass `self.databuilder.mcdata[mc]` data that contains names of events' handlers per mc target, but fire_class_handlers uses global _handlers variable that is created during parsing of all the targets. This leads to a problem where bitbake runs event handler that don't exist for a target or even overrides them - if multiple targets use event handler with the same name but different code then only one version will be executed for all targets. See [YOCTO #13071] for detailed bug information. Add mc target name as a prefix to event handler name so there won't be two different handlers with the same name. Add internal __BBHANDLERS_MC variable to have the handlers lists per machine. Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10bitbake: Bump release to 1.49.1Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10logging: Make bitbake logger compatible with python loggerJoshua Watt
The bitbake logger overrode the definition of the debug() logging call to include a debug level, but this causes problems with code that may be using standard python logging, since the extra argument is interpreted differently. Instead, change the bitbake loggers debug() call to match the python logger call and add a debug2() and debug3() API to replace calls that were logging to a different debug level. [RP: Small fix to ensure bb.debug calls bbdebug()] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>