| Age | Commit message (Collapse) | Author |
|---|
|
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The init function call was clobbering data hob relies on injecting, fix
this so the data injected by hob isn't overwritten.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Originally it seemed like a good idea to keep the parameters around. Having
seen this in real life use, its incorrect, we should pull all the data we need
into the cooker's configuguration and then use this to build the datastore.
Being able to just build the datastore from the parameters seemed like a good
idea but having a dummy cooker configuration object is now looking like
the better option.
This also fixes failures in hob since the parseFiles command can call
into cooker directly now and reset the configuration prefiles and postfiles
at will, rather than the indirect calls before which were breaking the datastore
(e.g. BBPATH wasn't set).
The cleanup this allows in tinfoil illustrates how this change makes more sense.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similarly to the execution context changes, establish better lifetime
management API of the class event handlers.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current execution context management for bitbake is ugly and the
use of a global variable is nasty. Fixing that is hard, however we
can improve things to start to establish an API for accessing
and changing that context.
This patch also adds in an explicit reset of the context when we reparse
the configuration data which starts to improve the lifecycle of the data
in setups like hob.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its confusing when an event handler in for example a bbclass receives
events for a recipe which doesn't use the class. This is due to the
class event handlers having confusing scope. Worse, the issue is not
deterministic and the events a handler will see depends on the parse
order.
To avoid these issues, remove the class event handler global scope
and only have class handlers recieve events for recipes using the
class.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Having a global method scope confuses users and with the introduction
of parallel parsing, its not even possible to correctly detect conflicting
functions. Rather than try and fix that, its simpler to retire the global
method scope and restrict functions to those locations they're defined
within. This is more what users actually expect too.
If we remove the global function scope, the need for methodpool is reduced
to the point we may as well retire it. There is some small loss of caching
of parsed functions but timing measurements so the impact to be neglibile
in the overall parsing time.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to have a memory resident bitbake and to allow task execution, we need
to be able to rebuild the base configuration without a cooker. This moves the
code into its own class so it can be built independently.
The interface is less than ideal here but I didn't want to add parsing methods
a subclassed DataSmart, at least until we've experimented further with this code
and are certain that makes sense. At the very least, the methods are ugly and need
cleaning up. Spliting the code out seems to be the right thing to do though and
should unblock various activities on BitBake so I believe this code is a step in
the right direction.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"status" is a relic of very early versions of bitbake, rename it to match what it now
means.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
updateCache() gets called by command.py when needed and needs to be iterated
over. The calls in cooker.py are therefore just plain wrong/confusing now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its clear looking at the code the path argument has never been used and doesn't
work. Tweak the function to be a little more readable.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The Cooker class is too large and needs to be split up into different
functional units. Splitting out the collections code into its own class
seems like a good place to start to try and disentangle things.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The <> operator is deprecated, replace with !=.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Current if the PR Service fails to start, bitbake carries on regardless or
hangs with no error message. This adds an exception and then handles it correctly
so the UIs correctly handle the error and exit cleanly.
[YOCTO #4010]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
So that the information added to cache_extra could
be accesed by hob, new variables were added in
the cooker.py.
Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
-extended cooker.py so that the extra information in cache_extra.py could
be used
Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reverting the pool changes, terminate does not work reliably on
bb.compat.Pool :(
[YOCTO #3978]
This reverts commit 8af519a49a3374bd9004864ef31ca8aa328e9f34.
|
|
Reverting the pool changes, terminate does not work reliably on
bb.compat.Pool :(
This reverts commit 9ca8e513dc142dd7aefadeb6db2ccdb00f38b39f.
|
|
Multiprocessing catches these and handles passing them between processes
itself, we don't need to do it ourselves anymore.
[YOCTO #3926]
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
|
|
I've sent a patch to poky, with some changes in sanity.bbclass. When bblayers.conf
is updated, it updates also a variable to know when to reparse configuration files.
This patch contains the changes to reparse conf files, after a sanity check which
changes bblayers.conf.
[YOCTO #3213]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
in BB_ORIGENV instead
Maintining hardcoded lists of user environmental variables is hacky,
replace these with the savedenv datastore. Allow access to that
through the BB_ORIGENV variable.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added a new command in bitbake to save a variable in a file; added a function
in cooker which is called by this command.
Added new command in bitbake to enable/disable data tracking.
The function saveConfigurationVar from cooker.py saves a variable in the file that
is received by argument. It checks all the operations made on that variable, using the history.
If it's the first time when it does some changes on a variable,it comments the lines where
an operation is made on it, and it sets it in a line to the end of file. If it's not
the first time(it has a comment before), it replaces the line.
Made some changes in hob to save the variables from bblayers.conf and local.conf
using the bitbake command.
[YOCTO #2934]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch adds tracking of the history of variable assignments.
The changes are predominantly localized to data_smart.py and
parse/ast.py. cooker.py and data.py are altered to display the
recorded data, and turn tracking on for the bitbake -e case.
The data.py update_data() function warns DataSmart.finalize()
to report the caller one further back up the tree.
In general, d.setVar() does what it used to do. Optionally,
arguments describing an operation may be appended; if none
are present, the operation is implicitly ignored. If it's
not ignored, it will attempt to infer missing information
(name of variable, value assigned, file and line) by examining
the traceback. This slightly elaborate process eliminates a
category of problems in which the 'var' member of the keyword
arguments dict is set, and a positional argument corresponding
to 'var' is also set. It also makes calling much simpler for
the common cases.
The resulting output gives you a pretty good picture of what
values got set, and how they got set.
RP Modifications:
a) Split from IncludeHistory to separate VariableHistory
b) Add dedicated copy function instead of deepcopy
c) Use COW for variables dict
d) Remove 'value' loginfo value and just use 'details'
e) Desensitise code for calling order (set 'op' before/after
infer_caller_details was error prone)
f) Fix bug where ?= "" wasn't shown correctly
g) Log more set operations as some variables mysteriously acquired
values previously
h) Standardise infer_caller_details to be triggered from .record()
where at all possible to reduce overhead in non-enabled cases
i) Rename variable parameter names to match inference code
j) Add VariableHistory emit() function to match IncludeHistory
k) Fix handling of appendVar, prependVar and matching flag ops
l) Use ignored=True to stop logging further events where appropriate
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code adds inclusion history to bitbake -e output, showing
which files were included, in what order. This doesn't completely
resolve timing questions, because it doesn't show you which lines
of a file were processed before or after a given include, but it
does let you figure out what the path was by which a particular
file ended up in your build at all.
How it works: data_smart acquires a .history member, which is an
IncludeHistory; this represents the inclusion of a file and all its
inclusions, recursively. It provides methods for including files,
for finishing inclusion (done as an __exit__), and for
dumping the whole tree.
The parser is modified to run includes inside a with() to push
and pop the include filename.
RP Modifications:
a) Split Include and Variable tracking
b) Replace deepcopy usage with dedicated copy function
c) Simplify some variable and usage
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Till now, a custom image made in Hob was using only the packages from
the base image. Now it is using everything declared in the base image.
Also next to hob-image.bb, it creates another .bb file which is used
in building process. Those images are ignored by git.
[YOCTO #2601]
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The data returned by get_file_depends() may me used in contexts like
checksums where order is important. The current usage of sets means
that some of the checksums can change in circumstances they should not.
This patch changes to use lists, thereby removing the problem.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"bitbake -e ASSUME_PROVIDED" should fail, but the error message wasn't
clear enough in the past:
$ bitbake -e bzip2-native
[snip]
ERROR: Command execution failed: Traceback (most recent call last):
File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 94, in
runAsyncCommand
commandmethod(self.cmds_async, self, options)
File "/buildarea/lyang1/poky/bitbake/lib/bb/command.py", line 323, in
showEnvironmentTarget
command.cooker.showEnvironment(None, pkg)
File "/buildarea/lyang1/poky/bitbake/lib/bb/cooker.py", line 325, in
showEnvironment
fnid = taskdata.build_targets[targetid][0]
KeyError: 0
[snip]
With this patch, the massage will be:
[snip]
ERROR: bzip2-native is in ASSUME_PROVIDED
ERROR: Command execution failed: Exited with 1
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
[YOCTO #3392]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Dumping the environment data into the console log files directory is
invariably not what the user wants or expects and leads to confusion
when looking at the log directory.
This change forces the logs to be disabled by default when using
the -e option.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make these a little easier to understand.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the '-s' option is run, change the heading above the list of recipes
to say "Recipe Name" instead of "Package Name".
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When making filename comparisons for recipes for priority calculations,
we need to split off any virtual prefix. Without this, BBCLASSEXTEND
version of recipes don't follow the priority settings they should.
[YOCTO #2933]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If an exception occurs during handling another exception we were
getting a useless traceback such as the following, after which
BitBake froze:
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/user/poky/poky/bitbake/lib/bb/command.py", line 84, in runAsyncCommand
self.cooker.updateCache()
File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1207, in updateCache
if not self.parser.parse_next():
File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1694, in parse_next
logger.error('Unable to parse %s', value.recipe,
AttributeError: 'exceptions.TypeError' object has no attribute 'recipe'
Fix this to print an actual traceback of the exception and exit
gracefully (well, as gracefully as possible under the circumstances).
The general fix for [YOCTO #2977].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This removes some unnecessary tracebacks
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently parsing failures still allow bitbake to continue on and try
and execute a build. This is clearly a bad idea and this patch adds in
more correct error handling and stops the build.
The use of sys.exit is nasty but this patches other usage in this function
so is at least consisent and its better than the current situation of
trying to execure a half parsed set of recipes. There are probably better
ways this could be improved to use to stop the build.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a recipe is skipped during parsing for whatever reason, check and
report this as an error rather than trying to use the data that is sent
back and failing.
Fixes [YOCTO #2976].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
All calls to parseConfiguration set self.status and call
self.handleCollections() afterwards, so just do this inside
parseConfiguration() itself.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
DISTRO.conf from local.conf)
We really need to pass the finalised data store into the parser init function
(and hence the siggen init function). This ensures any value changes get
passed into the correct code.
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
complete parse
Currently if any parsing failure occurs, there can be a long list of
bbappends that are "dangling" based on the fact that recipes were not
parsed. This change firstly ensures the error counter is incremented
and secondly that the bbappends list is only shown on a completed
parse list.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is for giving the user a clear list to show which pkg would be
built, we have the "bitbake -g", but it is not easy to read for people,
it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist:
$ bitbake -g core-image-sato
...
NOTE: PN build list saved to 'pn-buildlist'
[snip]
The contents of pn-buildlist:
busybox
shadow-native
pth
sysfsutils
qemu-helper-native
curl-native
ncurses-native
gdbm
xserver-xorg
linux-libc-headers
[snip]
[YOCTO #2404]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Alexandru DAMIAN
Andrei Dinu
Christopher Larson
Cristiana Voicu
Peter Seebach
Robert Yang
Paul Eggleton
Trevor Woerner
Martin Jansa