| Age | Commit message (Collapse) | Author |
|---|
|
If the client process never sends cooker data, the server timeout will
be 0.0, not None. This will prevent the server from exiting, as it is
waiting for a new client. In particular, the client will disconnect with
a bad "INHERIT" line, such as:
INHERIT += "this-class-does-not-exist"
Instead of checking explicitly for None, check for a false value, which
means either 0.0 or None.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that ":" is a valid character in variable key names, it needs to be
allowed by the variable expansion code too, to match.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is becomming increasingly clear we need to find a way to show what
is/is not an override in our syntax. We need to do this in a way which
is clear to users, readable and in a way we can transition to.
The most effective way I've found to this is to use the ":" charater
to directly replace "_" where an override is being specified. This
includes "append", "prepend" and "remove" which are effectively special
override directives.
This patch simply adds the character to the parser so bitbake accepts
the value but maps it back to "_" internally so there is no behaviour
change.
This change is simple enough it could potentially be backported to older
version of bitbake meaning layers using the new syntax/markup could
work with older releases. Even if other no other changes are accepted
at this time and we don't backport, it does set us on a path where at
some point in future we could
require a more explict syntax.
I've tested this patch by converting oe-core/meta-yocto to the new
syntax for overrides (9000+ changes) and then seeing that builds
continue to work with this patch.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
No error was being reported when the hash string was set to empty.
For example: SRC_URI[md5sum] = ""
On a related note (not a bug):
Because whitespace in the string will result in a checksum mismatch, the error
message was updated to make it a little clearer why the error was thrown.
For example: SRC_URI[md5sum] = " " or
SRC_URI[md5sum] = " 209f8326f5137d8817a6276d9577a2f1"
Now creates a message like this:
File: '/home/scott/yocto-cache/downloads/rsync-3.2.3.tar.gz' has md5
checksum '209f8326f5137d8817a6276d9577a2f1' when ' 209f8326f5137d8817a6276d9577a2f1' was expected
[YOCTO #14232]
Signed-off-by: Scott Weaver <weaverjs@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a13510d0028e234ea2f4744b0d0c38558395c70f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
svn was printing a message when encountering HTTP redirects.
This confused the revision parser.
Signed-off-by: Harald Brinkmann <Harald.Brinkmann@detectomat.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a944a335f8f4c4fe5df55f3d7d8e757bd2835146)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen cases where an OOM error causes bitbake server to hang:
9171 02:21:09.127810 Command Completed
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/bin/bitbake-server", line 51, in <module>
bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface)
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 550, in execServer
server.run()
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 108, in run
ret = self.main()
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 242, in main
ready = self.idle_commands(.1, fds)
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 370, in idle_commands
bb.event.fire(heartbeat, self.cooker.data)
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 216, in fire
fire_class_handlers(event, d)
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 123, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 93, in execute_handler
ret = handler(event)
File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 182, in defaultrun_buildstats
write_host_data(os.path.join(bsdir, "host_stats"), e, d, "interval")
File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 160, in write_host_data
output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8')
File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child
restore_signals, start_new_session, preexec_fn)
OSError: [Errno 12] Cannot allocate memory
We need to wrap the calls in the same high level wrapper as idle function calls
and trigger an exit upon an unhandled exception.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously the regex was maching x.y, but wasn't matching x, which is a problem
e.g. here:
https://download.gnome.org/sources/epiphany/
(the new gnome version scheme adds 40-series at the end).
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the hash of a task changes and that hash is a deferred task (e.g. a multiconfig
build), we need to ensure that the hash change propagates through to all the tasks
else the build will run multiple copies of the task, sometimes with oddly differing
results as the outhashes of native tasks built in differing locations can confuse
things.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2db571324f755edc4981deecbcfdf0aaa5a97627)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We were testing the validity of deferred tasks setscene status "up front" which
is very unlikely to succeed and leads to cache invalidation issues. With the
change to rebuild the deferred task list, this status becomes out of sync. The
result was tasks being executed when they should not have been leading to extra
work for the build unnecessarily.
Instead, don't process validity status for deferred tasks and assume their
data will become available. If it doesn't, this will now result in a build
error as the setscene task will fail and the main task will run instead.
In theory we could try and track the state changes in the deferred list and
re-test validity then but I'm not sure it is worth the effort when the other
code path and errors in setscene tasks will give a pretty good idea of what
is happening anyway.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit edcafac13b3b241b6687419e59018d21811507a1)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For some reason several git-annex files in Debian 10 buster
are read-only and removing them with "rm -rf" fails.
Fixes test failures like:
$ bitbake-selftest
...
rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied
rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied
EE..................................ssss.sssssssssssssss.sssss.......................................................................................................
======================================================================
ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex
fetcher, ud = self.fetch_shallow(uri)
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow
bb.utils.remove(ud.clonedir, recurse=True)
File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove
subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path))
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1.
Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod
the whole temporary directory which should cover any directory name differences between
different git-annex versions. Fixes tests failing after chmod call:
Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7729ef2983c72867e99fad82d671069ba5cb32b2)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Gitsm fetcher crashes when cloning a repository that contains LFS files.
This happens because the unpack method is called during download, but the
submodules have not been downloaded yet at this point.
This issue was introduced in this
commit: 977b7268bf4fd425cb86d4a57500350c9b829162
[YOCTO #14283]
Signed-off-by: Niels Avonds <niels@codebits.be>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 26caedc4d2e9b5a0f1d57f9291754a7f6c5e437e)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bitbake-server is spawned by process.py and passes the arguments it is
given to ProcessServer. There's some type confusion here:
bitbake-server is called with a string representation of the timeout,
which may be None. If the timeout is not set, pass 0 instead of None.
Inside bitbake-server a ProcessServer is created which expects the
timeout to be a float not a string, so always float() the value.
[ YOCTO #14350 ]
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c93ae1f861208f6d39fd15c84fbcd0e2b54331f5)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A CI user validating changes does not have any git push rights or
even a .gitconfig file so fix tests so that they run
by setting the user.name and user.email for the repo before
committing changes.
Fixes errors like:
ERROR: test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist (bb.tests.fetch.GitShallowTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 2055, in test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist
self.add_empty_file('a')
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1562, in add_empty_file
self.git(['commit', '-m', msg, path], cwd)
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1553, in git
return bb.process.run(cmd, cwd=cwd)[0]
File "/home/builder/src/base/poky/bitbake/lib/bb/process.py", line 184, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git commit -m a a' failed with exit code 128:
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 57c0811f1ee19b6619f4840a39e01e3cb98c34c4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In a multiconfig situation there are circumstances where firstly, tasks
are deferred when they shouldn't be, then later, tasks can end up as
both covered and not covered.
This patch fixes two related issues. Firstly, the stamp validity checking
is done up front in the build and not reevaulated. When rebuilding the
deferred task list after scenequeue hash change updates, we need therefore
need to check if a task was in notcovered *or* covered when deciding to
defer it. This avoids strange logs like:
NOTE: Running setscene task X of Y (mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch_setscene)
NOTE: Deferring mc:initrfs_guest:/A/alsa-state.bb:do_deploy_source_date_epoch after mc:host:/A/alsa-state.bb:do_deploy_source_date_epoch
where tasks have run but are then deferred.
Since we're recalculating the whole list, we also need to clear it before
iterating to rebuild it. By ensuring covered tasks aren't added to the
deferred queue, the covered + notcovered issue should also be avoided.
in the task deadlock forcing code.
[YOCTO #14342]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3c8717fb9ee1114dd80fc1ad22ee6c9e312bdac7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is further evidence of tasks ending up being "covered" and "notcovered"
which shouldn't happen and is bad. The code that caused this problem last
time appears to have issues where stamps for tasks already exist.
Split out the setscene stamp checking code to a separate function and
use this when checking "hard dependencies" (like pseudo-native) so
that if the stamps exist and it will be "covered", it is not put on
the notcovered list.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the new functionality in build.py to identify stale setscene tasks
and send an event to the metadata listing them. The metadata then
has the option of performing cleanup operations if it thinks that
appropriate.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a new function which compares the stamp filename we want (including
taskhash) with what is in the stamp directory (using the clean mask).
This tells us which stamp files are stale and are due to be rerun.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Commit 75f87db413 fixed the confusion between bitbake and python logger
but in codeparser still old method of setting debug level was used
causing TypeError, because debug level value was incorrectly returned
and assigned to event.msg.
| File "./bitbake/lib/bb/ui/knotty.py", line 660, in main
| event.msg = event.fn + ': ' + event.msg
| TypeError: can only concatenate str (not "int") to str
[YOCTO #14298]
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add logic to check that if REQUIRED_VERSION has been set that the cooker
class method findBestProviders properly handles the case where the
REQUIRED_VERSION has not been found.
Fixes [YOCTO #10096]
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This commit adds a new column to the Bitbake --show-versions command
called Required Version. This column will display any packages which
have a REQUIRED_VERSION successfully set.
Fixes [YOCTO #10096]
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before the REQUIRED_VERSION variable was introduced the
PREFERRED_VERSION variable allowed for a fallback to the next most
suitable version.
Since REQUIRED_VERSION does not allow a fallback to a different version
implement a check in the _filterProviders function to make sure that
if a requested REQUIRED_VERSION is not found then the function returns
no eligible providers have been found.
Fixes [YOCTO #10096]
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This commit adds checks during the findPreferredProvider function for a
new variable REQUIRED_VERSION. This can be set, in exactly the same
manner as PREFERRED_VERSION, on a per package basis to enforce the use
of a particular version of a package.
REQUIRED_VERSION is similar in behaviour to PREFERRED_VERSION except if
the version specified by REQUIRED_VERSION is not found an error occurs
and the execution of Bitbake stops.
Fixes [YOCTO #10096]
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The logging line for when no eligible providers are found only adds
unnecessary noise and is not required. As the stack unwinds better
logging occurs which satisfactorily describes the no eligible providers
error condition.
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adding the "if d is None" to the event handling code meant some of
these tests stopped working. The reason is that len(d) was zero but
not equal to None.
Passing the data object to the register() function in the test
correctly registers the event handler and avoids the problem, it
just happened to work previously, incorrectly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This should have been fixed in the previous patch, well spotted Chris!
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There were two issues in this code, firstly the code could stack duplicates
in the variable, secondly, calling "if data" caused the datastore to compute
len(data) which is comparitively expensive. Checking "if data is not None"
is much much faster/cheaper.
The issue was clear from "bitbake -p -P" output where the time in register()
showed large amounts of time in the __len__ function of the datastore.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using parse_recipes, honor quiet so that scripts and custom plugins
for recipetool are able to mute progress bars from bitbake that would
otherwise print to STDOUT.
Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if pseudo fails we can only see the path to pseudo.log. If we
have no access to server and can only rely on bitbake log then debugging
becomes impossible. This printing needs to be added in runqueue level,
not inside task execution, because in some cases task fails with pseudo
abort really early and we don't even see any log.
In this change I'm adding pseudo log printing in every fakeroot task
failure that logged `mismatch`, `error` or `fatal` to logfile, because
we have no other way to communicate with pseudo if it failed or not.
Only lines from last pseudo server execution will be printed.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
Signed-off-by: Adrian Walag
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Mikolaj Lasota <mikolaj.lasota@protonmail.com>
Signed-off-by: Wiktor Baura <wbaura@gmail.com>
Signed-off-by: Kamil Kwiek <kamil.kwiek@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A previous commit uses __BBHANDLERS_MC to record all handlers, but
it does not take into consideration of INVALIDCONF, thus causing regression.
We need to record the name before returned AlreadyRegistered, otherwise, when
reparsing due to INVALIDCONF, bascially all handlers are not called.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allows bitbake to fetch from an Azure Storage account.
The fetcher submodule is compatible with the az:// URI protocol, its
functionality is based on bitbakes wget fetcher, superior in performance
to using a propietary tool like azcopy which can handle cloud storage
account operations with more functionality (that we dont need in a fetcher)
but less compatibility.
A sample URI uses can be defined in the following way:
SRC_URI = "az://<azure-storage-account>.blob.core.windows.net/<container>/foo.tar.xz"
This fetcher can easily be used with PREMIRRORS and SSTATE_MIRRORS, e.g.:
SSTATE_MIRRORS = "file://.* az://<azure-storage-account>.blob.core.windows.net/sstate-cache/PATH;downloadfilename=PATH \n"
PREMIRRORS_prepend = "\
git://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \
ftp://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \
http://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \
https://.*/.* az://<azure-storage-account>.blob.core.windows.net/downloads/ \n \
"
Can also be used with non-public access Azure Storage accounts/containers via a
Shared Access Signature by declaring the AZ_SAS variable which will be
automatically used by the fetcher:
AZ_SAS="?sv=2000-01-01&ss=...&sig=somesignature"
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For a while I've been puzzled as to why debug logging from runqueue wouldn't
appear on the console with -DD. The logic in the bbdebug handling is inverted
so fix it and now we see the expected messages from runqueue with -D and -DD.
This should then let us debug other issues using those log messages.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen hard to debug issues where a task ends up in both the
covered and notcovered list. Add a sanity check to ensure if this
happens in future, we see it in the logs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen occasional issues where linux-yocto:do_compile_kernelmodules would
run without do_shared_workdir running before it. do_shared_workdir is an
setscene task but never has an sstate object generated so it will always
rerun. This should not happen since compile_kernemodules should only
execute if a setscene that depends on it didn't run and that should trigger
do_shared_workdir not to be marked as covered.
The issue is that build-appliance-image:do_package is one of the tasks which
covers linux-yocto:do_compile_kernelmodules but it is also a noexec task
and has a dependecy on pseudo-native:do_populate_sysroot.
In the problem case, pseudo-native:do_populate_sysroot is unavailable but
marked as covered since it is noexec. The "harddeps" code then also marks it
as notcovered. No task should ever be both covered and notcovered and this
is where the problems come from.
The solution is for the harddeps code only to to fail tasks if they've not
already been handled in some way. The code is assuming code couldn't have
handled revdeps at this point but we now have clear evidence they can.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If an exception is raised when running host python code, the fetcher
immediately crashes, this might be temporary depending on the servers
reliability.
Catch the exception when the connection was reset and try once again
to fetch the data.
File: '/usr/lib/python3.8/socket.py', lineno: 669, function: readinto
0665: if self._timeout_occurred:
0666: raise OSError("cannot read from timed out object")
0667: while True:
0668: try:
*** 0669: return self._sock.recv_into(b)
0670: except timeout:
0671: self._timeout_occurred = True
0672: raise
0673: except error as e:
Exception: ConnectionResetError: [Errno 104] Connection reset by peer
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
5f7fdf7b2d ("bitbake: event: Prevent bitbake from executing event
handler for wrong multiconfig target") broke multiconfig builds contain
a hyphen, since it's attempt to use the multiconfig as part of a
function name and python functions are not allowed to contain a hyphen.
Rework the bitbake multiconfig test to test a multiconfig with a hyphen
and one with an underscore to validate this doesn't break in the future.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Commit bebef58b21bdff7a3ee1fa2449b7df19144f26fd introduced forcing
parser shutdown as default in case of build abort.
In this case bitbake sometimes hangs after facing error during parsing,
waiting for child processes to finish. Killing it then will spawn zombie
processes.
Thus we force the shutdown after catching an exception.
Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If no eligible providers are found then an integer zero is returned.
This causes the following error, in two possible places in taskdata.py,
when the return value is used in a list comprehension:
[snip]
eligible = [p for p in eligible if not p in self.failed_fns]
TypeError: 'int' object is not iterable
[\snip]
Fix by returning the variable eligible itself, of type list.
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows metadata to rely upon BB_DEFAULT_UMASK.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently each task has to have a umask specified individually. This
is leading to determinism issues since it is easy to miss specifying
this for an extra task.
Add support for specifing the default task umask globally which
simplifies the problem.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When multiconfig is used bitbake might try to run events that don't
exist for specific mc target. In cooker.py we pass
`self.databuilder.mcdata[mc]` data that contains names of events'
handlers per mc target, but fire_class_handlers uses global _handlers
variable that is created during parsing of all the targets.
This leads to a problem where bitbake runs event handler that don't
exist for a target or even overrides them - if multiple targets use
event handler with the same name but different code then only one
version will be executed for all targets.
See [YOCTO #13071] for detailed bug information.
Add mc target name as a prefix to event handler name so there won't be
two different handlers with the same name. Add internal __BBHANDLERS_MC
variable to have the handlers lists per machine.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The bitbake logger overrode the definition of the debug() logging call
to include a debug level, but this causes problems with code that may
be using standard python logging, since the extra argument is
interpreted differently.
Instead, change the bitbake loggers debug() call to match the python
logger call and add a debug2() and debug3() API to replace calls that
were logging to a different debug level.
[RP: Small fix to ensure bb.debug calls bbdebug()]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
From tinfoil, if you edit bblayers.conf and break it, then call
parseConfiguration (e.g. by adding a bad layer with bitbake-layers),
the system doens't show any parse error yet it should.
Add in a call to the updateCache function so that things really
are reparsed when requested.
Partially fixes [YOCTO #14054]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
python function
If shell function name starts with 'python' or 'fakeroot' parser wrongly
assumes it's python/fakeroot function.
[YOCTO #14204]
Use regex lookahead assertions to check if 'python' expression is
followed by whitespace or '(' and if 'fakeroot' is followed by
whitespace.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When we run `devtool build mc` recipe's task dependencies are expanded
to "mc:do_populate_sysroot" where "mc" name is treated as multiconfig
and "do_package_sysroot" as multiconfigname.
| ERROR: Multiconfig dependency mc:do_populate_sysroot depends on
| nonexistent multiconfig configuration named do_populate_sysroot
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Insert an explicit pass to fetch all blobs needed by Git LFS, during the
fetch() function. This avoids the default behavior of Git LFS to wait
until 'git checkout' to begin downloading the blobs pointed to by LFS records.
Network access is not allowed at that point in the recipe's lifecycle.
[YOCTO #14191]
Signed-off-by: Matt Hoosier <matt.hoosier@garmin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Joshua Watt
Richard Purdie
Scott Weaver
Harald Brinkmann
Alexander Kanavin
Mikko Rapeli
Niels Avonds
Ross Burton
Michael Opdenacker
Tomasz Dziendzielski
Charlie Davies
Manuel Leonhardt
Chen Qi
Alejandro Hernandez Samaniego
Jan Brzezanski
Matt Hoosier