| Age | Commit message (Collapse) | Author |
|---|
|
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
|
|
Each time a runqemu() fails, the log handler would be left behind meaning
messages from any subsequent run would be duplicated (or worse/more).
This ensures we remove the handler regardless and means we no longer
have the duplication.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid warnings such as:
meta/lib/oeqa/utils/commands.py:213: ResourceWarning: unclosed file <_io.BufferedReader name=4>
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated
code from devtool/oeqa needs to set this to avoid warnings which break
various tests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
get_bb_vars was using variables.copy() to duplicate the list of variables passed
but this function only exists in lists [1,2] and not tuples (1,2).
Instead of throwing an exception if the variables are in a tuple, simply
construct a new list using the passed sequence-like object.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The logger handling in oeqa was confused at best. This patch:
a) Passes in a logger through various qemu runner pieces
b) Uses that logger consistently in the code
c) Creates a logger for QemuRunner outside the bitbake namespace
meaning we don't conflict with the tinfoil logging changes
The result of this is more consistency. For runtime tests in testimage,
the logs always contain the debug info, nothing is shwon on the console.
For the oe-selftests, logs are intercepted and only shown if the test
fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tests that use bitbake("my-test-image") can run for a long time
without any indication to the user of oe-selftest about what's going
on. The test author has to log the bitbake output explicitly,
otherwise it is lost in case of test failures.
Now it is possible to use bitbake("my-test-image",
output_log=self.logger) to get more output both on the console and in
the XML output (when xmlrunner is installed). Example output:
2017-06-23 12:23:14,144 - oe-selftest - INFO - Running tests...
2017-06-23 12:23:14,145 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-06-23 12:23:14,151 - oe-selftest - INFO - Running: bitbake my-test-image
2017-06-23 12:23:16,363 - oe-selftest - INFO - Loading cache...done.
2017-06-23 12:23:17,575 - oe-selftest - INFO - Loaded 3529 entries from dependency cache.
2017-06-23 12:23:18,811 - oe-selftest - INFO - Parsing recipes...done.
2017-06-23 12:23:19,659 - oe-selftest - INFO - Parsing of 2617 .bb files complete (2612 cached, 5 parsed). 3533 targets, 460 skipped, 0 masked, 0 errors.
2017-06-23 12:23:19,659 - oe-selftest - INFO - NOTE: Resolving any missing task queue dependencies
Because the implementation was already using threading, the same is
done to decouple reading and writing the different pipes instead of
trying to multiplex IO in a single thread. Previously the helper
thread waited for command completion, now that is done in the main
thread.
The most common case (no input data, joined stdout/stderr) still uses
one extra thread and a single read(), so performance should be roughly
the same as before.
Probably unintentionally, result.error was left as byte string when
migrating to Python3. OE-core doesn't seem to use runCmd() with split
output at the moment, so changing result.error to be treated the same
as result.output (i.e. decoded to a normal strings) seems like a
relatively safe API change (or rather, implementation fix).
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
merge: wait()
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This change allows quering for variables with the format A_B, i.e.
PREFERRED_PROVIDER_virtual/kernel instead of just A.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By default, QemuRunner avoids modifying the image files that it boots
into by enabling the qemu snapshot mode. However, some tests may want
to test changes that must persists across reboots, so this mode
should be optional.
This can be combined by copying the image file to a temporary location
first and then booting with that copy. It's also useful when testing
with additional drives attached to a virtual machine.
QemuTinyRunner doesn't use the snapshot parameter and therefore ignores
the new parameter.
Long term, a better way of passing these various configuration
parameters should be used, and perhaps QemuRunner and QemuTinyRunner
can be merged into one again to avoid code duplication. But for now
the patch follows the exiting style.
Also beware that QemuTarget.start() now acts in two different modes
(with or without explicit launch command), and depending on that mode
parameters like discard_writes must be ignored, i.e. not get passed to
launch().
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
runqemu() takes all parameters for the virtual machine from the
variables of the given recipe. By allowing the caller to provide a
hash with variables that get applied locally, the caller gets more
control.
Here's the intended usage:
<prepare internal-image in self.resultdir>
overrides = {
'DEPLOY_DIR_IMAGE': self.resultdir,
'IMAGE_LINK_NAME': 'internal-image-%s' % self.image_arch,
}
with runqemu('refkit-installer-image', ssh=False,
overrides=overrides) as qemu:
....
This can be used to replace the image completely with something else
or to copy it before allowing runqemu() to write into it.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sometimes it is useful to reconfigure the qemu virtual machine
directly. runqemu has the "qemuparams" parameter for that, and the
underlying start() methods also supported modifying that via their
"params" parameter. Only the runqemu() wrapper function lacked
a way to specify additional parameters.
One potential usage is to attach additional disks.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make it possible to limit the length of output lines shown in runCmd
exceptions. E.g when running bitbake we easily get thousands of lines of
log output, where only the last few (tens) are interesting or relevant
when an error occurs.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Modify the following files to test runqemu:
targetcontrol.py
utils/commands.py
utils/qemurunner.py
We need simulate how "runqemu" works in command line, so when test
"runqemu", the targetcontrol.py, utils/commands.py and
utils/qemurunner.py don't have to find the rootfs or set env vars.
[YOCTO #10249]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
qemu runner picks up first fsimage type from the hard-coded
list of supported types. This makes it impossible to test
particular image type unless it's not ext4(first type in
the hardcoded list of types).
Added image_fstypes argument to commands.runqemu and QemuTarget
__init__ to specify type of the image to run qemu with.
This will be used to pass wic image type to test efi wic images.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Added possibility to pass additional runqemu parameters
down the stack of APIs:
commands.runqemu -> QemuTarget.start -> QemuRunner.start
This will be used to pass ovmf parameter in testing of
efi wic images under qemu.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Tests may need to run a native tool that is not available on the host
filesystem, but can be built using one of the *-native recipes. In such case,
the tool will be available in native sysroot, and running in from that location
will require adjustments to PATH.
runCmd() can now take a path to native sysroot as one of its arguments and
setup PATH accordingly.
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Commit 9d55e9d489cd78be592fb9b4d6484f9060c62fdd broke calling get_bb_vars()
when called without arguments. This fix this issue.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use tinfoil.parse_recipe() in order to allow oe-selftest to be used in
memres mode.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
We don't need to see the parsing/cache loading message in the
oe-selftest output, so use the newly added quiet option to disable it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The function get_bb_vars will remove items for the list passed
as the function argument, this will leave the caller with an
empty list and the function never says it will consume the items.
This hasn't been found before because only get_bb_var uses this
function.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
multi-configuration builds
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When run a command sometimes the output isn't provided so validate
before trying to encode to utf-8, also some output like BIOS/EFI
contains characters that can't be codified into utf-8 for this reason
set errors='replace'.
[YOCTO #10019]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
updateEnv() can be used in other places so move the
function to utils/commands.py
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Useful if one wants to separate stdout and stderr.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Get rid of duplicate code.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
A new function for getting values of multiple bitbake variables at the
same time.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the changes introduced to test the eSDK
the runexported test failed during the execution.
This change fix runexported test in the least invasive
way, because of the release cycle.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added new parameter 'ssh' to targetcontrol 'start' method
to be able to test images without running ssh server.
[YOCTO #8498]
(From OE-Core rev: 1c3c66aadd43092bc19242b0651ee810cc31fe7c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Removed unused parameter 'test' from runqemu function.
(From OE-Core rev: c688b3bcbb57099fa72a9728bc708b109802f7fc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Bitbake logger stops working after tinfoil.shutdown removes console
handler from it. This makes bb.{error,warn,note,critical} messages
disappear from the console. Adding console handler to bitbake logger
again should fix this issue.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This matches the value in testimage.bbclass, which makes sense
since the autobuilders are usually contended.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a runqemu function which uses the QemuTarget() code from
oeqa.targetcontrol to setup the QEMU instance, with all of the added
robustness that that gives us. To do this, a datastore is needed for the
recipe in question (core-image-minimal) so we do the work needed to set
this up. We then use this runqemu function within the imagefeatures
tests instead of a hand-rolled implementation.
We can then use SSHControl to run the SSH tests rather than rolling our
own code to do that as an added bonus.
Fixed and extended by Paul Eggleton <paul.eggleton@linux.intel.com>.
Part of the fix for [YOCTO #7994].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Only split on the first equals character so that values that contain
equals characters (such as FAKEROOTENV) can be retrieved.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Locating which recipe provides a file in an image that you want to
modify and then figuring out how to bbappend the recipe in order to
replace it can be a tedious process. Thus, add a new appendfile
subcommand to recipetool, providing the ability to create a bbappend
file to add/replace any file in the target system. Without the -r
option, it will search for the recipe packaging the specified file
(using pkgdata from previously built recipes). The bbappend will be
created at the appropriate path within the specified layer directory
(which may or may not be in your bblayers.conf) or if one already exists
it will be updated appropriately.
Fairly extensive oe-selftest tests are also provided.
Implements [YOCTO #6447].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It searches using regex now and should be more accurate.
Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Bitbake can parse ~ in bblayer's paths.
Added this functionality to oeqa code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow get_bb_var() to work with unexported variable values such as
MACHINE - the workaround is a little crude but should suffice for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Python code should use spaces only for indentation.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
* Enable querying exported variables
* Use strip() to remove quotes so any internal quotes are not disturbed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adding support for postconfig option to the bitbake() and related methods.
This enables us to use 'bitbake -R postconfig_file <command>'.
Usage: bitbake(cmd, postconfig="some confguration")
'postconfig_file' would contain what we add in 'postconfig'
Other methods affected: get_bb_env(), get_bb_var()
Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
For use outside of tests themselves, we want a better error than
AssertionError, so create one and allow us to request it when calling
runCmd(). This enables us to avoid tracebacks during master image
operations if the power control command fails.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
various scripts
The purpose of oe-selftest is to run unittest modules added from meta/lib/oeqa/selftest,
which are tests against bitbake tools.
Right now the script it's useful for simple tests like:
- "bitbake --someoption, change some metadata, bitbake X, check something" type scenarios (PR service, error output, etc)
- or "bitbake-layers <...>" type scripts and yocto-bsp tools.
This commit also adds some helper modules that the tests will use and a base class.
Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the purpose of the tests.
The tests themselves will usually write to ".inc" files from the layer or in conf/selftest.inc
(which is added as an include in local.conf at the start and removed at the end)
It's a simple matter or sourcing the enviroment, adding the meta-selftest layer to bblayers.conf
and running: oe-selftest to get some results. It would finish faster if at least a core-image-minimal
was built before.
[ YOCTO #4740 ]
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Ross Burton
Patrick Ohly
Leonardo Sandoval
Markus Lehtonen
Robert Yang
Ed Bartosh
Maciej Borzecki
Joshua Lock
Mariano Lopez
Paul Eggleton
Aníbal Limón
Ed Bartosh
Lucian Musat
Corneliu Stoicescu
Stefan Stanacar