aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2014-10-02oeqa/dmesg: Whitelist usbhid failuresRichard Purdie
We fixed these in parselog but not here. This test really can just be deleted now really. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02oeqa/parselogs: Ignore qemu usbhid errorsRichard Purdie
These are harmless from the USB pointer device we install, ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-29oeqa/utils: Added filter to LogResults decorator to enforce custom log level.Lucian Musat
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-22oeqa/selftest: Added decorators to buildoptions.pyLucian Musat
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
2014-09-10oeqa/runtime: Automatic test for ptestLucian Musat
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder. No cleanup is done for packages after the test is finished. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-10oeqa/utils/logparser.py: results based log parser utilityLucian Musat
A module for parsing results based logs like ptest, compliance and performance. Supports breaking the logs into multiple sections and also provides a result object to use the parser with. The parser is initialized with the regex required to identify results and section statements in the target log file. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-10oeqa: Added package installer to oetest to aid in future automatic install ↵Lucian Musat
of packages Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-09-01oeqa: sstatetests should now look for glibc-initialKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com>
2014-08-29oeqa: xorg log test is being replaced by parselogs so removeRichard Purdie
Now we have the parselogs test, this one can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/pasrselogs: Improve the machine/string whitelistRichard Purdie
Currently the whitelist is imcomplete, inaccurate and suffers duplication. These changes: * Add common groups of errors * Change to make the default whitelist clear * Correctly (or at least better) escape the regexp expressions * Add in missing machines to allow builds on the autobuilder to suceed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/runtime: Automatic test for parsing the logs on a machine and search ↵Lucian Musat
for certain error keywords. This adds a common new qa test for general processing of log files. One significant improvement is machine dependent ignore filters. This can be used to replace several weaker individual QA tests that are currently used. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-27oeqa/sdk/buildsudoku.py: add setUpModule method to run only when gtk+ in ↵Corneliu Stoicescu
installed. Adding setUpModule in order to skip the module when gtk+ is not installed in the toolchain. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-27oeqa/oetest.py: enable sdk tests to use hasFeature and hasPackage methods.Corneliu Stoicescu
In order to use the hasFeature and hasPackage methods, we need to make oeSDKTest extend oeTest and also set the test context (tc) attribute in the oeTest class when loading the tests. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23oeqa/utils/httpserver: Reset SIGTERM handlerRichard Purdie
With bitbake-worker installing a SIGTERM handler, we now need to reset the one here to ensure that when this process shuts down, it doesn't take the rest of the task with it. This does appear to be the only place in OE that we have this problem. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/sdk/: add sdk tests for sudoku, iptables and cvsCorneliu Stoicescu
Add test modules for sdk tests. NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/utils/targetbuild.py: add support for sdk testsCorneliu Stoicescu
- Create new abstract class BuildProject that provides basic functionality for a project/package building class * contains abstract method _run() that needs to be implemented by all extending classes. - The old TargetBuildProject class now extends the abstract BuildProjct class - Introducing new SDKBuildProject that extends the abstract BuildProjct class NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-18oeqa/oetest.py: enable sdk testingCorneliu Stoicescu
- add support for sdk tests in the loadTests and runTests methods - add new oeSDKTest test object NOTE: Original patch made by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11oeqa/runtime: add new cpp test and fileCorneliu Stoicescu
This tests out the limits header which we've noticed does have problems in some SDK builds. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11lib/oeqa/selftest: Don't match log level in outputTyler Hall
To facilitate changing the log level of the "Fetcher failure" message, search only for the message without the "Error:" prefix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-02oeqa/utils/decorators.py: add import osCorneliu Stoicescu
An 'import os' was omitted here while testing the previous decorators using runtime tests that import the os module before this one. Unfortunately oe-selftest fails because of this missing import. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-07-29selftest/buildoptions.py: fix QA_WARN test and add more output when failingCorneliu Stoicescu
The -ccleansstate should be done before building the package for the second time. Also printing the command output when failing. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25lib/oeqa: Fix accidental revert of codeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/rutime: Added testcase decorators for automated runtime tests. Also ↵Lucian Musat
added LogResults decorator for oeTest class in oetest.py Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/runtime: Added skipModule import for test modules that use it.Lucian Musat
The modules that use skipModule should import it themselves and not rely on somebody else to import it. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa: Refactor test skipping decorators to use the unittest result objectLucian Musat
In order to make the test skipping decorators independent of the oeTest object we rely on the unittest result object to construct skip, fail and error lists used by these decorators. Created a new object getResults that analyses upper frames and retrieves the unittest result object instance, then return a list of failed, skipped and error tests. Also removed the oetest import from decorators.py because it was no longer required. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqs/selftest: Added test case decorators for toasterLucian Musat
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqa: fix return status in pam.py to match shadow-4.2.1Chen Qi
The return statuses of commands like `su --help' or `passwd --help' in shadow 4.2.1 version are different from those in shadow 4.1.4.3 version. Now that we've upgraded shadow to 4.2.1, we need to fix these statuses in the pam.py to make things work as expected. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10oeqs/selftest: Added test case decorators for the rest of selftest testcases ↵Lucian Musat
except toaster Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-07-03Added testcase decorator to use in logging. Added class decorator LogResults ↵Lucian Musat
that outputs test results in separate log file. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-30oeqa/selftest/_toaster.py: fix 'IndentationError' issuesIonut Chisanovici
There were some indentation errors when the _toaster.py was merged into master. This patch fix those errors. Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-30oeqa/selftest/toaster.py: renamed to _toaster.pyIonut Chisanovici
In order not to run the toaster selftests by default, we renamed to _toaster.py Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-24oeqa/targetcontrol.py: Separate the matching of supported image fstypes from ↵Corneliu Stoicescu
the resulting value check. Because we used a bb.fatal call inside the get_image_fstype classmethod, this caused problems when accessed without instantiating the object with a valid bb environment. Separating the matching part of the classmethod(that is usable by outside scripts) from the check of the resulting value. The matching is done within a new classmethod and the latter keeps the old method name and internal functionality, this way we don't have to change any other target controllers code. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13controllers/masterimage.py: Make testimage kernel naming pattern universalCorneliu Stoicescu
YB: #6254 The QemuTargetControl class does not specify any kernel naming but the runqemu script it uses uses the " KERNEL_IMAGETYPE + MACHINE + '.bin' " naming scheme. Modifying the other major target controller class, MasterImageHardwareTarget, to use the same kernel naming scheme. This is usefull also to outside scripts that want to anticipate the kernel file name for all target controllers. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13oeqa/controllers/testtargetloader.py: add 'import os'Corneliu Stoicescu
YB: #6254 This module uses os but relies on other modules to import it. Adding 'import os' in order to be self-sustained. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13targetcontrol.py: Add a classmethod to get extra files needed by the target ↵Corneliu Stoicescu
controllers YB: #6254 Add a new classmethod that can be used by outside scripts to get the extra files needed by the target controllers. An outside script can predict rootfs, manifest and kernel files needed by a target controller, but sometimes there are other files needed. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13masterimage.py: enable dynamical determination of rootfs typeCorneliu Stoicescu
YB: #6375 Adding support for get_image_fstype() in the MasterImageHardwareTarget and GummibootTarget classes. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-13targetcontrol.py: make possible dynamical determination of rootfs typeCorneliu Stoicescu
YB: #6375 Added a new method get_image_fstype() that autodetermines what fstype to use for the rootfs file. This method uses a new list variable 'supported_image_fstypes' that contains image fstypes supported by the target controller. This method is also a classmethod which means outside scripts can get the image fstype. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-10oeqa/utils/commands.py: add support for postconfig optionCorneliu Stoicescu
Adding support for postconfig option to the bitbake() and related methods. This enables us to use 'bitbake -R postconfig_file <command>'. Usage: bitbake(cmd, postconfig="some confguration") 'postconfig_file' would contain what we add in 'postconfig' Other methods affected: get_bb_env(), get_bb_var() Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-06-06Added new test cases for bitbake modes (-e -n -p -r -R -c -k)Lucian Musat
Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-30toaster: toaster oe-selftest supportIonut Chisanovici
This patch adds toaster tests using the oe-selftest infrastructure. You need to have builds done - the tests will verify data integrity after the toaster collection phase. Once you have your toaster builds done, to run the automated backend tests via oe-selftest do the followings: 1. Update builddir/conf/bblayers.conf to contain the meta-selftest layer 2. From the builddir run: 'oe-selftest toaster' or if you just want to run a single test: 'oe-selftest toaster.Toaster_DB_Tests.testname' This first part adds the meta/lib/oeqa toaster file. Signed-off-by: Ionut Chisanovici <ionutx.chisanovici@intel.com> Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-04sstatetests: Use TUNE_ARCH instead of TARGET_ARCHRichard Purdie
The scripts extact variable values from bitbake -e. Unfortunately TARGET_ARCH is unset from that environment so we use TUNE_ARCH instead which will have the value we need. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-04sstatetests: Update after binutils-cross name change with PN additionRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30oeqa: add proper handling for command errors where neededPaul Eggleton
For use outside of tests themselves, we want a better error than AssertionError, so create one and allow us to request it when calling runCmd(). This enables us to avoid tracebacks during master image operations if the power control command fails. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30classes/testimage: if start fails, don't try to stopPaul Eggleton
If we couldn't start the target, it doesn't make sense to try and stop it here since logically it shouldn't now be in any kind of "started" state. (It's the start function's job to clean up after itself if it fails - to that end, fix up the QemuTarget class so that it does.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30oeqa/controllers/masterimage: more robust master image startupPaul Eggleton
Instead of powering up the target when the object is constructed, wait until deploy is called. Then there are basically two different scenarios: a) The device is booted into the master image already, in which case we can just use it b) The device is booted into another image or can't be contacted, in which case we need to power cycle it. Here we also now wait until it has booted up instead of trying to contact it immediately. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30oeqa/controllers/masterimage: add a serial control commandStefan Stanacar
Similar to power control command, this depends on a user's setup to get to the serial port of a board. For a local connected board this could just be: TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" and for a serial console server: "telnet 10.11.12.13 7003" or some conmux command. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30oeqa/targetcontrol: restart method shouldn't be abstractStefan Stanacar
And drop the un-needed and un-used restart methods. Only qemu ever used this and actually does it safely. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-30oeqa/controllers/masterimage: add a base class for hw targetsStefan Stanacar
Right now GummibootTarget is the only hardware TEST_TARGET with deployment, but we will add more, so let's make an abstract base class, that will do the common thing for all the hw targets. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29oeqa/utils: sshcontrol: realtime logging of outputStefan Stanacar
Log the output of the command as it runs not when it finished, else tail -f tmp/work/minnow-poky-linux/core-image-sato/1.0-r0/testimage/ssh_target_log isn't as useful as it could be. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-04-23selftest: DiskMonTest: use a high value for free spaceStefan Stanacar
Using the free space of the host works when oe-selftest is the only build running, but if something else on the host remove things this will fail (as seen on AB). Using an absurdly high value should fix this. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-12 07:51:06 +0000