aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
AgeCommit message (Collapse)Author
2015-07-22oeqa/runtime: Added decorators for the remaining auto tests.Lucian Musat
This helps for the automatic completion of the results in testopia. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added a new automated rpm test.Daniel Istrate
testcase 195: Check rpm install/removal log file size Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added one runtime testcase in connman.Daniel Istrate
(testcase 223) Check that only one connmand runs in background. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemumips64 runtime errorRichard Purdie
Similarly to qemumips, ignore these errors upon bootup so that we have a good QA baseline for new errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemuarm64 runtime errorsRichard Purdie
The qemuarm64 machine doesn't have graphics so whitelist the X server failures to start. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Fixed a problem in parselogsLucian Musat
When the test passed it returned an error because grep did not return any error and the variable that held the results was being referenced but not assigned any value. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23meta: Add explict getVar param for (non) expansionRichard Purdie
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11oeqa/parselogs: Added some more errors to the whitelist.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-11oeqa/parselogs: The logs are now copied and parsed locally.Lucian Musat
This is to fix a problem with reaching the limit of the whitelist size. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-04-09oeqa/runtime: Boot test for poky-tiny.Lucian Musat
Bug 6705. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/smart: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/rpm: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ping: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/parselogs: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ldd: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/systemd.py: skip instead of failing without avahiPatrick Ohly
The SystemdServiceTests assume that avahi-daemon is installed, which is not necessarily the case depending on the image being tested. Better check this dependency before starting the tests and skip them if the service is not installed. This has to be done for each test instead of for the entire module, because other tests in the module can run without avavi. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21oeqa/runtime: Added a new auto rpm testLucian Musat
The test tries to query the rpm list with a non-root user Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20oeqa/runtime: Fixed a problem in logparser where it failed to whitelist some ↵Lucian Musat
common errors. And some minor tweaks like moving some errors from qemu to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-16lib/oeqa/runtime/systemd: Add a basic runtime test for the journalRandy Witt
The test_systemd_journal() test will fail if journalctl exits with a non-zero exit status. The exit status is non-zero even if there are no journal files, which is what drove the creation of this test. [Yocto #7388] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-12oeqa/parselogs: Skip hda opcode errorsRichard Purdie
These occur when running images under virtualisation on machines with high load which sometimes trigger timeouts in the kernel DMA code. They're harmless to ignore in these cases. Adding this since "failing" the build due to this is more annoying thank useful. [YOCTO #7387] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-21oeqa/parselogs: Add exception for new mips error messageRichard Purdie
The 3.19 kernel introduces this error, ignore it for now. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-19oeqa/ptest: Fixed complementary package install detection and added ↵Lucian Musat
ptest-runner exit code check Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-15oeqa/ptest: Removed buildhistory as requirement for ptestLucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oeqa/smart: Added some new test casesLucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oe-pkgdata-util: improve command-line usagePaul Eggleton
* Use argparse instead of optparse for standardised help output, options and a much cleaner code structure * Look up pkgdata directory automatically so the user doesn't have to specify it * Use standard logging NOTE: this does mean a slight change in syntax - if you do want to specify the pkgdata directory (usually only necessary if you're calling it from within the build process) you need to use the parameter -p (or --pkgdata-dir) and specify this before the command, not after it. Examples: oe-pkgdata-util find-path /sbin/mke2fs oe-pkgdata-util lookup-recipe libelf1 oe-pkgdata-util read-value PKGSIZE libc6 oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-01-07oeqa/parselogs: Added a check in case the folder location does not contain ↵Lucian Musat
any log files Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31oeqa/parselogs: Add harmless failure warning introduced by new qemu version ↵Richard Purdie
on qemuppc Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25oeqa/runtime: Added some new filters for the log parser Including custom ↵Lucian Musat
filters for edgerouter, minnow and jasperforest. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-10-02oeqa/dmesg: Whitelist usbhid failuresRichard Purdie
We fixed these in parselog but not here. This test really can just be deleted now really. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02oeqa/parselogs: Ignore qemu usbhid errorsRichard Purdie
These are harmless from the USB pointer device we install, ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10oeqa/runtime: Automatic test for ptestLucian Musat
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder. No cleanup is done for packages after the test is finished. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-08-29oeqa: xorg log test is being replaced by parselogs so removeRichard Purdie
Now we have the parselogs test, this one can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/pasrselogs: Improve the machine/string whitelistRichard Purdie
Currently the whitelist is imcomplete, inaccurate and suffers duplication. These changes: * Add common groups of errors * Change to make the default whitelist clear * Correctly (or at least better) escape the regexp expressions * Add in missing machines to allow builds on the autobuilder to suceed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/runtime: Automatic test for parsing the logs on a machine and search ↵Lucian Musat
for certain error keywords. This adds a common new qa test for general processing of log files. One significant improvement is machine dependent ignore filters. This can be used to replace several weaker individual QA tests that are currently used. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11oeqa/runtime: add new cpp test and fileCorneliu Stoicescu
This tests out the limits header which we've noticed does have problems in some SDK builds. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25lib/oeqa: Fix accidental revert of codeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/rutime: Added testcase decorators for automated runtime tests. Also ↵Lucian Musat
added LogResults decorator for oeTest class in oetest.py Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/runtime: Added skipModule import for test modules that use it.Lucian Musat
The modules that use skipModule should import it themselves and not rely on somebody else to import it. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17oeqa: fix return status in pam.py to match shadow-4.2.1Chen Qi
The return statuses of commands like `su --help' or `passwd --help' in shadow 4.2.1 version are different from those in shadow 4.1.4.3 version. Now that we've upgraded shadow to 4.2.1, we need to fix these statuses in the pam.py to make things work as expected. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-31lib/oeqa: add a test target controller for EFI targetsStefan Stanacar
The purpose of this module is to deploy a test image on a EFI-enabled hardware and run our runtime tests. A bit of background: - testimage.bbclass uses the concept of TEST_TARGET which is a class name that is responsible for target deploying. A layer can provide it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget (ssh into an already up and running machine and run tests), the default one being qemu. - basically testimage does something like: target.deploy() try: target.start() runTests() finally: target.stop() This module assumes a running EFI machine with gummiboot as bootloader and core-image-testmaster installed (or similar). Also your hardware under test has to be in a DHCP-enabled network that gives it the same IP for each reboot. One time setup (master image): - build core-image-testmaster with EFI_PROVIDER = "gummiboot" - install the image on the target Test image setup: - build your test image, e.g core-image-sato as you usually do, but with these in local.conf: IMAGE_FSTYPES += "tar.gz" - Now run the tests: INHERIT += "testimage" TEST_TARGET = "GummibootTarget" TEST_TARGET_IP = "192.168.2.3" bitbake core-image-sato -c testimage Other notes: - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling. The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like TEST_POWERCONTROL_CMD="powercontrol.exp test 10.11.12.1 nuc1" in local.conf. Basically my expect script does: 'ssh test@10.11.12.1 "pyctl nuc1 <arg>" and runs a python script there that controls power for a label called nuc1'. The reason why my expect script has to ssh into another machine is because of network topology, and that machine is the one actually connected to the test rack and the power strip. That's why TEST_POWERCONTROL_CMD and _ARGS need to be customized for one's setup, the only requirement being that it accepts: on/off/cycle as the last argument. - if no command is defined it would use classic reboot. This is fine as long as the machine actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where some manual interaction is okay from time to time. [YOCTO #5614] Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02oeqa/runtime/dmesg: Ignore DMA timeout errorsRichard Purdie
We see these from qemu images on servers under load. They're not interesting and clutter the build output so lets ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-12oeqa/runtime: smart: don't run in auto mode when rpm is not the pmStefan Stanacar
TEST_SUITES = "auto" would run smart tests even for non-rpm images, which is pointless. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11oeqa/utils: targetbuild: take proxy into accountStefan Stanacar
A previous commit broke downloads when proxies are involved, let's fix it. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-04oeqa/runtime/systemd: remove race in settle()Ross Burton
The settle() function had a race where services could still be activating at two minutes but then when the final log is output, they've activated. Remove this race and generally clean up the code. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-02oeqa/runtime: syslog: update --help testStefan Stanacar
busybox 1.22 now returns exitcode 0 instead of 1 for --help options, so this test needs to be updated when busybox gets upgraded to 1.22. https://bugs.busybox.net/show_bug.cgi?id=5612 http://git.busybox.net/busybox/commit/?id=efd0698f74caab0a0c8a51228b923ee142e8e278 Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-01-29oeqa/runtime: smart: fix adding channelsStefan Stanacar
Without this it won't add core2-64 and similar channels, as the directory name in deploy_dir/rpm uses _ not - as the package arch. Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29oeqa/runtime: drop restartTarget methodStefan Stanacar
Restart is a property of a specific target, not of a test class, should a test really need to restart the target the direct method should be called. Also some tests used this to enforce more ram, which makes sense only for qemu targets only (and the inital reason this was needed isn't valid anymore, qemu machines had the default ram size bumped a while ago). Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28oeqa/runtime: rpm: fix finding the test rpm fileStefan Stanacar
Translate TUNE_PKGARCH to find the right file (this used to work because tune for qemux86-64 used to be x86_64 now it's core2-64) Also, while using packagedata was nice, it's harder to make the test exportable and runnable outside of the build system. (where oe.packagedata isn't available) Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-14oeqa/runtime/systemd: wait for services to start/failRoss Burton
When checking that no services have failed to start, actually wait for services to finish starting by waiting for there not be no units in the "activating" state. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-01-06lib/oeqa/runtime: add test for pythonTing Wang
Run a python script on the target 1)checks the output. 2)Call os.system method create a testfile Signed-off-by: Saul Wold <sgw@linux.intel.com>