| Age | Commit message (Collapse) | Author |
|---|
|
Its possible some patterns may cause problems with the current path
manipulations, make a small tweak to try and avoid potential pathname
overlap issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If oe-selftest is run without -j, the separate build directory "build-st"
isn't cleaned up afterwards. Mirror the behaviour of the -j option to
handle this the same way, only preserve upon failure.
To do this, the remove function needs to be moved to the selftest
context module so that it can be accessed without requiring the
testtools and subunit modules the -j option requires.
A dummy wrapper class is used to wrap the tests and clean up afterwards.
[YOCTO #13953]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This ensures that builddir is updated correctly to point to the new
selftest build directory when we're given a list of test suites instead
of a list of test cases.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This may be useful if the parent directory of the original builddir is
not writable, on a lower performance drive, etc.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This avoids the seeing broken replacements like:
oe-selftest-centos/build/build-st-926tools/sysroots/x86_64-pokysdk-linux/etc/ssl/certs/ca-certificates.crt
which understandably break builds.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now selftest is using its own copied build directory, we can stop worrying about
copying files around as backup, and drop the SIGTERM handler to try and restore
them, simplifying the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
non-concurrent selftest
Currently oe-selftest reuses the current build directory and the concurrent
version run with -j does not.
Standardise and use a separate new build directory in both cases. This will lead
to simpler code and more reliable user run tests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We can see tracebacks where the SIGTERM handler catches things
it shouldn't. Avoid exit(1) unless we're the process that
it was intended for.
[YOCTO #13664]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the 'append' action of argparse instead of nargs. This changes the
behaviour of the option from "-t foo bar -r" to "-t foo -t bar -r".
Additionally rename the long form options to be consistent with
behaviour, such that they specifying a single tag at a time.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Change the -t/-T args to be optional so that they can be used together
with the existing -r/-a/... args to run a more flexible filtering of
test tags.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add '--run-only-tags' for running tests which match any of the provided
tags, and '--run-exclude-tags' for running all tests except those that
have any of the provided tags.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Ensure log directory exists to avoid the following error.
FileNotFoundError: [Errno 2] No such file or directory: '/.../build-selftest/tmp/log/oe-selftest-results-20181207043431.log'
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The existing logfile is simply placed in the current directory. Since the test
changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid
directory. We also see trackbacks if the symlink is invalid.
Improve things by:
* Placing logs in LOG_DIR (or BUILDDIR if unset).
* Using a full path to the log meaning the log and link are placed in the same directory.
* Using lexists instead of exists so invalid symlinks are handled correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
warnings
Use our own lsb function instead as used elsewhere by the codebase.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid the warning:
meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
In this case importlib is a direct replacement.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently sdk & sdkext will output json file to LOG_DIR, while
selftest will output json file to TOPDIR/log.
Standardize selftest json output file to LOG_DIR.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tweak the preceeding commit to:
* Add STARTTIME to the identifier to make it unique
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As part of the solution to replace Testopia to store testresult,
OEQA selftest need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.
By default, json testresult file will be written to "oeqa"
directory under the oe-selftest log directory.
To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Parsing all the recipes is annoying when trying to re-execute oe-selftest
and also unnecessary as its really just a sanity check. When the tests were
originally being developed the guard was useful but less so now.
Replace it with bitbake -e which is fast and checks the basic configuration
is valid.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
rm_work if enabled leads to some tests failing that rely on artifacts
being present. Check if rm_work.bbclass is included and show an error
and exit if it is.
Fixes [YOCTO #12694]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The warn method is deprecated. We should use the documented warning instead.
Quoting from the python's official doc:
"""
Note: There is an obsolete method warn which is functionally identical to warning.
As warn is deprecated, please do not use it - use warning instead.
"""
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This allows oe-selftest to take a -j option which specifies how much test
parallelisation to use. Currently this is "module" based with each module
being split and run in a separate build directory. Further splitting could
be done but this seems a good compromise between test setup and parallelism.
You need python-testtools and python-subunit installed to use this but only
when the -j option is specified.
See notes posted to the openedmbedded-architecture list for more details
about the design choices here.
Some of this functionality may make more sense in the oeqa core ultimately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A new command line argument (-R, which is the oposite of current -r) that allows
to run all test cases except the ones indicated through the command line.
Some command line examples:
* Run all except the distro test case:
$ oe-selftest -R distrodata
* Run all except the archiver test case and a single bblayers unit test
$ oe-selftest -R archiver bblayers.BitbakeLayers.test_bitbakelayers_add_remove
[YOCTO #11847]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
found
If some test module/case is specified to run and isn't found the OEQA
framework didn't notice it, so complete the implementation using
modules_required and validate for the test case prescense.
Raise an exception when the test module/case required isn't found.
[YOCTO #11645]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to avoid corrupt local.conf and bblayers.conf adds
signal handler for SIGTERM and use try/finally (KeyboardIntrrupt) block
to restore previously backuped configuration.
[YOCTO #11650]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
When add meta-selftest by the script the testlayer_path needs to be
reloaded to avoid None value.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Is better to log the summary at end to see in an easy way
the actual result of the test run.
[YOCTO #11622]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The new OEQA framework aims to re-use code into the different Test
components.
The previous oe-selftest implements it-self loading, run, and list test
cases in a non-standard way (unittest base) and other functionalities
like logging that is now on oeqa core. This ends on a compact oe-selftest
script.
All needed command line options was migrated but there are some of them
pending of implementation and others deprecated.
Deprecated options:
list-tags: The tag functionality into the old oeqa framework isn't
work, the selftest doesn't has tag decorators.
{run, list}-tests-by: Ambiguos options it accepts all the posibilites module,
class, name, id or tag.
Remaining to implement:
coverage: It enables covrage reports over a test run, currently isn't on
on use and some bugs [1], i filed a bug to add support to OEQA core module in
this way other Test components could enable it.
repository: It push XML results into a git repository and isn't in use,
i filed a bug to implement this into OEQA core module. [2]
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
Richard Purdie
Paul Barker
Nathan Rossi
Chen Qi
Yeoh Ee Peng
Anuj Mittal
Leonardo Sandoval
Aníbal Limón