| Age | Commit message (Collapse) | Author |
|---|
|
In combination with the recent server reworking, this change actually
prevents messages sent from tasks from being logged properly. This will
of course give us the duplicated messages back, and I really hate to do
that effectively a second time, but that's better than seeing no error
at all in the case of a failure - we'll have to find the proper way of
avoiding the duplication that doesn't result in some messages going
missing.
This reverts commit 8a5bae76f91f2411187c638a42fa3c762052cf11.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If there are events in the UI queue we want to print them regardless of
whether we're handling an exception or something like SystemExit.
This improves error messages for some other failure cases where currently
no logging would get printed and leave the user confused as to what went
wrong.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"bitbake -g m4-native -c do_unpack" doesn't list any dependencies in
task-depends.dot. This is because no header describing the task was
added unless a task has dependencies. Tweak the code to fix this.
[YOCTO #10893]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This was originally based on some other code but its mostly misleading
comments now. Massively clean this up and accept its now a total fork.
There are no funciton changes here, just cleanup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When we create the child, if an exception occurred it was transfering
back into the parent context. We don't want to do that us use a try/finally
to ensure we exit.
We need to ensure a traceback is printed and any queued UI messages which
may not have made it to the client UI at this point.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We have cases where the server is being started but we're not seeing any messages
from it. Add some earlier logging so we can try and better understand where
issues may be occurring.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the connection control socket and the command channel close together,
we can race and hit EOFError exceptions before we close the channel.
Reorder the code to handle this in the correct order and ignore the
EOFError exceptions as they mean the client is disconnecting and shouldn't
terminate the server.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
My previous assertion about FusionForge appears to have been wrong, or
FusionForge has changed behaviour, or both.
FusionForge now mandates that downloads have the Accept header set, despite that
header being optional, and returns a 406 Not Acceptable error if it isn't set.
As we were pretending that 406 was actually 405 (Moved) and tried to handle it as a
redirect this results in an infinite loop until Python kills the recursion.
Delete the handling of 406 as 405, and pass Accept: */* in the headers.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The bitbake option "-t SERVERTYPE" was deprecated and can be
removed since the desired XMLRPC listener now the default.
The bitbake server port cannot be "-1" anymore and must be
explicitly passed.
There is a race condition for the bblock file to not only
be created but to actually be populated, so a delay test loop
is required (usually only one pass).
The 'xmlrpcclient' is now the class that allows Toaster to for
example kill builds.
The events for populating the recipe parsing now either show only
the final result or are skipped entiredly, so the progress
calculator needs to be changed to not block on the parsing phase.
[YOCTO #11851]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add support for Toaster aggregators with a set of api links that
return JSON data for (a) builds in progress, (b) builds completed,
(c) specific build data, and (d) an is-alive health ping link.
[YOCTO #11794]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Having this as one huge long line isn't easy to manipulate, split it into
multiple lines for ease of debugging issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The checkstatus() code was expecting checkstatus to throw exceptions if it
failed, but in general it should return False.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
`git fetch --tags` seems to interact badly with `mirror=fetch`, resulting in
the regular branches not being fetched, so drop the unnecessary `--tags`. This
fixes this unit test failure: `bb.fetch2.FetchError: Fetcher failure: Unable
to resolve 'master' in upstream git repository in git ls-remote output for
/tmp/tmp4ag_mgmn/gitsource`
[YOCTO #11698]
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the server isn't running return 0 as to do otherwise complicates
scripts which do cleanup of bitbake servers which would potentially
be memory resident.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This timeout path was commonly hit due to errors starting the server. Now we
have a better way to handle that, the retry logic can be improved and cleaned
up. This patch:
* Makes the timeout 5s rather than intervals of 1s with a message. Paul
noted some commands can take around 1s to run on a server which has just
been started on a loaded system.
* Allows a broke connection to exit immediately rather than retrying something
which will never work.
* Drops the Ctrl+C masking, we shouldn't need that anymore and any issues
would be better handled in other ways.
This should make things clearer and less confusing for users and is much cleaner
code too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we reconnect to the server, stop leaking pipes and clean up
after ourselves.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if the server fails to start, the user sees no error message and
the server will be repeatedly attempted to be started until some longer
timeouts expire. There are error messages in the cookerdeamon log but
nobody thinks to look there.
Add in a pipe which can be used to tell the starting process whether the cooker
did actually start or not. If it fails to start, no further attempts can be
made and if present, the log file can be shown to the user.
[YOCTO #11834]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This cleans up the socket keep alive into better class structured code
and adds cleanup of the open file descriptors upon shutdown.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the monitoring thread exits with some error code or runs indefinitely. Allow
closure of the pipe its monitoring to have the thread exit cleanly/silently.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Expose the underlying close() and fileno() methods which allow connection
monitoring and cleanup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code would duplicate messages to stdout and stderr when no UI connected
and there were error level messages.
Rework the code so it either uses stderr (for errors and above) or
stdout for warnings/debug but not both for the same messages.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We were bridging the gap between the server and UI here by calling a
bb.siggen.find_siginfo, a function defined and set on that module from
the metadata. This worked from the UI side before but since the recent
server changes is no longer accessible. Create a new command so this can
execute on the server side and return the result by way of a new event.
(We're still running compare_sigfiles() on the signature generator but
that isn't quite the same thing and does still work.)
Fixes [YOCTO #11844].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update stray usage of deprecated logger.warn(), which was introduced
to fetch2/__init__.py after all other instances had been replaced by
logger.warning():
http://git.openembedded.org/bitbake/commit/?id=5a53e7d7b017769a6eb0f0a6335735a1fe51a5ec
http://git.openembedded.org/bitbake/commit/?id=676a5f592e8507e81b8f748d58acfea7572f8796
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When wget is fetching a listing for a directory over FTP it writes to a
temporary file called .listing in the current directory. If there are many such
operations happening in parallel - for example during 'bitbake world -c
checkpkg' - then up to BB_NUMBER_THREADS instances of wget will be racing to
write to, read, and delete the same file.
This results in various failures such as the file disappearing before wget has
processed it or the file changing contents, which causes checkpkg to randomly
fail.
Mitigate the race condition by creating a temporary directory to run wget in
when doing directory listings.
[ YOCTO #11828 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The warning message currently shown can occur more frequently than previously
if a previous bitbake server is shutting down and we're reconnecting to a new
server. Change it to a note message to match the higher level connection
logging retry messages and so as not to interfer with selftests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that there is only one server, this abstraction is no longer needed
and causes indrection/confusion. The server shutdown is also broken with
the cooker post_server calls happening too late, leading to "lock held"
warnings in the logs if PRServ is enabled.
Remove the abstraction and put the shutdown calls in the right order
with respect to the locking.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The server shutdown is currenlty laggy and race prone. This patch:
* adds a waitpid so that no zombie server is left around if its not
running in daemon mode.
* adds a quit "sentinal" using a pipe so that we're not sitting in
a socket poll() until timeout in order just to quit.
* use a select() call to poll the socket and the pipe for a quit signal.
The net result of this change is that the prserv exits with the cooker server
and it does so immediately and doesn't wait for the select/poll calls to
timeout. This makes bitbake a lot more responsive for startup/shutdown and
doesn't cause UI timeout errors as often when prserv is used.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the sstate is accessed via HTTP, the existence check can fail due
to network issues, in which case bitbake silently continues without
sstate.
One such network issue is an HTTP server like Python's own SimpleHTTP
which closes the TCP connection despite an explicit "Keep-Alive" in
the HTTP request header. The server does that without a "close" in the
HTTP response header, so the socket remains in the connection cache,
leading to "urlopen failed: <urlopen error [Errno 9] Bad file
descriptor>" (only visible in "bitbake -D -D" output) when trying to
use the cached connection again.
The connection might also get closed for other reasons (proxy,
timeouts, etc.), so this is something that the client should be able
to handle.
This is achieved by checking for the error, removing the bad
connection, and letting the check_status() method try again with a new
connection. It is necessary to let the second attempt fail
permanently, because bad proxy setups have been observed to also lead
to such broken connections. In that case, we need to abort for real
after trying twice, otherwise a build would just hang forever.
[YOCTO #11782]
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Trim the entered built target value so that Toaster is not
confused with no real targets nor a ghost second target.
[YOCTO #11727]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Set the clone progress to be off by default for the benefit of
command line projects and 'Local Yocto' builds. For Toaster managed
projects that do use the clone feature the clone progress status
is already explicitly set by the existing code and thus displayed.
[YOCTO #11744]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Change the task progress counting from the task order field
which excludes setscene to the task completion field which
counts all completed tasks regardless of type.
[YOCTO #9971]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If BB_STRICT_CHECKSUMS is set to anything other than "1" i.e. we're not
going to raise an error, then fire an event so that scripts can listen
for it and get the checksums.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If BB_STRICT_CHECKSUMS is set to "ignore" then don't display a warning
if no checksums are specified in the recipe. This is not intended to be
used from recipes - it is needed when we move to using more standard
code paths to fetch new files from scripts i.e. where we don't know what
the checksums are in advance.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want this API to be easier to use, so add missing function
documentation to help with that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A common task for tinfoil-using scripts is to iterate over all recipes.
This isn't too difficult with the current API, but the pkg_* variables
are a little awkward and are really designed for bitbake's internal
usage - and it gets a bit more difficult when you want to access some of
the other information such as packages and rprovides. To resolve this,
create a new recipe info class and add an all_recipes() function to
generate this for all recipes. Also add a get_recipe_info() function to
get the information for a specific recipe (by PN).
(It might perhaps be suggested that we already have a structure similar
to this in the cache, however the one we add here is designed for
external use and allows the internal structures to change if needed
without affecting the API).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add access to fn_provides, packages, packages_dynamic and rproviders on
the recipecache object. This requires an additional corresponding
command plumbing to be added.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Up to this point, if you wanted to run build tasks in the normal way
they get run from a python script, there was no other way than to shell
out to bitbake. Worse than that, you couldn't have tinfoil active during
that because only one bitbake instance could be running at once. As long
as we're prepared to handle the events produced, we can create a wrapper
around calling the buildTargets command. Borrow code from knotty to do
this in such a way that we get the expected running task display
(courtesy of TermFilter) and Ctrl+C handling.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This isn't useful for knotty itself, but for use from tinfoil
in case we can't get access to either the console or errconsole, allow
either to be unspecified (None).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is useful for the caller to know whether the uihelper has handled the
event passed so that it can skip other event handling code if so.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We really ought to have just one place where the string representation
of these events is produced. This doesn't take any real control away
from the UI - if an alternative representation is desired, that can
still be made.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make it easy to determine if recipes are parsed (and thus information
about available recipes is in memory).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Python style recommends underscore based naming rather than camelCase,
and thus the former has been used for most of tinfoil's functions. Add
an underscored version of parseRecipes() for consistency and change the
one place we call it to use the new version.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In OE's devtool we want to repeatedly run build_file() without showing
unnecessary messages and triggering buildhistory for each call.
build_file() is just a wrapper around the buildFile command. Change
the final "hidewarning" parameter of the buildFile command to "internal"
and have this call a new buildFileInternal() function without triggering
any of the normal build events, silencing the normal info messages from
the runqueue ("Executing RunQueue Tasks", "Tasks Summary" etc.) and
avoiding calling parseConfiguration() which we've already done at this
point.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed:
$ bitbake --observe-only
FATAL: '--observe-only' can only be used by UI clients connecting to a server.
And even:
$bitbake --observe-only -B localhost:-1
FATAL: '--observe-only' can only be used by UI clients connecting to a server.
This was beucase the brackets were in a wrong position.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed 1:
$ . ../poky/oe-init-build-env-memres .
$ bitbake -m
$ bitbake -m # shutdown it again
$ bitbake -m
Starting bitbake server...
Terminated bitbake server.
It starts the server and kill it which is redundant.
$ bitbake -m
$ bitbake --status-only
Starting bitbake server...
It starts the server which is not what we need.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed:
$ bitbake --server-only -B localhost:-1
Infinite loop
Reconnecting to bitbake server...
[snip]
Break the loop when server-only can fix the problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is no global var "configuration", so the old code hang at
self.bitbake_lock.write(), and nothing wrote to bitbake.lock. I didn't
figure out why it hang (but not print errors).
Reproducer:
$ bitbake -B localhost:-1 world -k
Check bitbake.log, there was nothing, now fixed.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Paul Eggleton
Richard Purdie
Ross Burton
David Reyna
Christopher Larson
Mark Hatle
Andre McCurdy
Patrick Ohly
Robert Yang