diff options
author | Richard Purdie <richard.purdie@linuxfoundation.org> | 2021-06-15 12:00:33 -1000 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2021-06-19 17:06:14 +0100 |
commit | 0e0af15b84e07e6763300dcd092b980086b9b9c4 (patch) | |
tree | cc8d397dc6168737068c10a89873977dad593f3e | |
parent | fe1b57f4d39c287526353387e164c863eb9df421 (diff) | |
download | bitbake-0e0af15b84e07e6763300dcd092b980086b9b9c4.tar.gz |
cooker: Avoid parser deadlocksyocto-3.1.9yocto-3.1.102020-04.9-dunfell2020-04.10-dunfell1.46.91.46.10
If you make parsing fail (e.g. add something like:
X := "${@d.getVar('MCMACHINES').split()[1]}"
to meson.bbclass, then run "while true; do bitbake -g bash; done"
it will eventually hang. It appears the cancel_join_thread() call the
parsing failure triggers, breaks the results_queue badly enough that it
sits in read() indefintely (called from self.result_queue.get(timeout=0.25)).
The timeout only applies to lock aquisition, not the read call.
I've tried various other approaches such as using cancel_join_thread()
in other places but the only way things don't lock up is to avoid
cancel_join_thread() entirely for results_queue.
I do have a concern that this may adversely affect Ctrl+C handling
but equally, its broken now already and this appears to improve
things.
[YOCTO #14034]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-rw-r--r-- | lib/bb/cooker.py | 10 |
1 files changed, 6 insertions, 4 deletions
diff --git a/lib/bb/cooker.py b/lib/bb/cooker.py index 4d2c62cd9..730cdc56f 100644 --- a/lib/bb/cooker.py +++ b/lib/bb/cooker.py @@ -1933,7 +1933,8 @@ class Parser(multiprocessing.Process): except queue.Empty: pass else: - self.results.cancel_join_thread() + self.results.close() + self.results.join_thread() break if pending: @@ -1942,6 +1943,8 @@ class Parser(multiprocessing.Process): try: job = self.jobs.pop() except IndexError: + self.results.close() + self.results.join_thread() break result = self.parse(*job) # Clear the siggen cache after parsing to control memory usage, its huge @@ -2060,8 +2063,6 @@ class CookerParser(object): bb.event.fire(event, self.cfgdata) - # Allow data left in the cancel queue to be discarded - self.parser_quit.cancel_join_thread() for process in self.processes: self.parser_quit.put(None) @@ -2081,7 +2082,8 @@ class CookerParser(object): process.join() self.parser_quit.close() - self.parser_quit.join_thread() + # Allow data left in the cancel queue to be discarded + self.parser_quit.cancel_join_thread() sync = threading.Thread(target=self.bb_cache.sync) self.syncthread = sync |