aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
AgeCommit message (Collapse)Author
2015-10-19oeqa/runtime: Fix setUp and tearDown methodsMariano Lopez
Currently some of the runtime test overwrites the setUp and tearDown methods provided by oeRuntimeTest, this will avoid some checks required when running the test suit. This patch changes the setUp and tearDown methods for their local counterparts, so when these tests are called, it will run the parent setUp and tearDown and also the local ones. [YOCTO #8465] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-01Revert "oeqa/runtime: Added one runtime testcase in connman."Cristian Iorga
connman daemon forks during normal usage, so this test is no longer valid and generates sometimes false positives. This reverts commit 7d01c595c96eb3d67b90caf71050c4e9345257de. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28oeqa/sdk/gcc: Fix makefile testRichard Purdie
The makefile test was hardcoding a test for gcc. In the SDK context, we need to use $(CC) to use the cross compiler. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28oeqa/parselogs: Updated log parser whitelist.Lucian Musat
Added some fixes for meta-intel boards and moved some errors from specific images to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-23oeqa/runtime/parselogs.py: Fix dmesg log retrieve in satoAníbal Limón
Sato uses busybox that fails to write log using echo "" because dmesg output can contain special characters. [YOCTO #8377] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-11oeqa/runtime/date: stop systemd-timesyncd during testRoss Burton
There's a race between systemd-timesyncd manipulating the system time (with NTP lookups) and the test case's time manipulation. Prevent this by stopping systemd-timesyncd for the duration of the test case. Thanks to Khem Raj for root-causing this. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-09oeqa/parselogs: Updated log error whitelist.Cristina Agurida
[YOCTO #7898] [YOCTO #7761] [YOCTO #7804] Signed-off-by: Cristina Agurida <cristina-danielax.agurida@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30oeqa/runtime/multilib: add test for libcRoss Burton
Add a basic test to verify that /lib/libc.so.6 and /lib32/libc.so.6 have the right ELF class. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-29oeqa/runtime/multilib: fix and improve multilib testRoss Burton
Replace the previous shell magic with a Python function to parse the readelf output, and fix the package names to include the lib32- prefix. [ YOCTO #8219 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-29runtime/syslog.py: fix syslog test crashCostin Constantin
This patch fixes the ability to correctly identify syslog's package name for the built image. It is derived from modifying oeqa/oetest.py for [YOCTO #8170] Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/runtime/parselogs: don't use uname to get MACHINERoss Burton
Instead of running "uname -n" on the target (which actually returns the hostname) to get something approximating the hostname (or not, if that has been overriden), simply fetch MACHINE from the datastore. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/runtime/parselogs: add more x86 log exceptionsRoss Burton
Move "fail to add MMCONFIG..." to qemux86_common as we started to also see it on x86-64 with Linux 4.1. Add "can't claim BAR..." to qemux86_common. Generalise "Failed to access perfctr MSR" as the MSR number can change. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/syslog: Removed a pointless testcase and added skip for another.Lucian Musat
The testcase syslog --help was rather useless and also causing problems on images where syslog was non-busybox, like LSB images so I removed it. Added a skip condition for TC 202 for the same reason. If syslog is non-busybox then the test skips. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added decorators for the remaining auto tests.Lucian Musat
This helps for the automatic completion of the results in testopia. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added a new automated rpm test.Daniel Istrate
testcase 195: Check rpm install/removal log file size Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/runtime: Added one runtime testcase in connman.Daniel Istrate
(testcase 223) Check that only one connmand runs in background. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemumips64 runtime errorRichard Purdie
Similarly to qemumips, ignore these errors upon bootup so that we have a good QA baseline for new errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Whitelist qemuarm64 runtime errorsRichard Purdie
The qemuarm64 machine doesn't have graphics so whitelist the X server failures to start. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oeqa/parselogs: Fixed a problem in parselogsLucian Musat
When the test passed it returned an error because grep did not return any error and the variable that held the results was being referenced but not assigned any value. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23meta: Add explict getVar param for (non) expansionRichard Purdie
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11oeqa/parselogs: Added some more errors to the whitelist.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-06-11oeqa/parselogs: The logs are now copied and parsed locally.Lucian Musat
This is to fix a problem with reaching the limit of the whitelist size. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-04-09oeqa/runtime: Boot test for poky-tiny.Lucian Musat
Bug 6705. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/smart: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/rpm: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ping: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/parselogs: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/ldd: Added new decorators for existing automated tests.Lucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-09oeqa/runtime/systemd.py: skip instead of failing without avahiPatrick Ohly
The SystemdServiceTests assume that avahi-daemon is installed, which is not necessarily the case depending on the image being tested. Better check this dependency before starting the tests and skip them if the service is not installed. This has to be done for each test instead of for the entire module, because other tests in the module can run without avavi. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-21oeqa/runtime: Added a new auto rpm testLucian Musat
The test tries to query the rpm list with a non-root user Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20oeqa/runtime: Fixed a problem in logparser where it failed to whitelist some ↵Lucian Musat
common errors. And some minor tweaks like moving some errors from qemu to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-16lib/oeqa/runtime/systemd: Add a basic runtime test for the journalRandy Witt
The test_systemd_journal() test will fail if journalctl exits with a non-zero exit status. The exit status is non-zero even if there are no journal files, which is what drove the creation of this test. [Yocto #7388] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-03-12oeqa/parselogs: Skip hda opcode errorsRichard Purdie
These occur when running images under virtualisation on machines with high load which sometimes trigger timeouts in the kernel DMA code. They're harmless to ignore in these cases. Adding this since "failing" the build due to this is more annoying thank useful. [YOCTO #7387] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-21oeqa/parselogs: Add exception for new mips error messageRichard Purdie
The 3.19 kernel introduces this error, ignore it for now. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-19oeqa/ptest: Fixed complementary package install detection and added ↵Lucian Musat
ptest-runner exit code check Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-15oeqa/ptest: Removed buildhistory as requirement for ptestLucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oeqa/smart: Added some new test casesLucian Musat
Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-02-14oe-pkgdata-util: improve command-line usagePaul Eggleton
* Use argparse instead of optparse for standardised help output, options and a much cleaner code structure * Look up pkgdata directory automatically so the user doesn't have to specify it * Use standard logging NOTE: this does mean a slight change in syntax - if you do want to specify the pkgdata directory (usually only necessary if you're calling it from within the build process) you need to use the parameter -p (or --pkgdata-dir) and specify this before the command, not after it. Examples: oe-pkgdata-util find-path /sbin/mke2fs oe-pkgdata-util lookup-recipe libelf1 oe-pkgdata-util read-value PKGSIZE libc6 oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6 Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-01-07oeqa/parselogs: Added a check in case the folder location does not contain ↵Lucian Musat
any log files Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31oeqa/parselogs: Add harmless failure warning introduced by new qemu version ↵Richard Purdie
on qemuppc Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25oeqa/runtime: Added some new filters for the log parser Including custom ↵Lucian Musat
filters for edgerouter, minnow and jasperforest. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-10-02oeqa/dmesg: Whitelist usbhid failuresRichard Purdie
We fixed these in parselog but not here. This test really can just be deleted now really. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02oeqa/parselogs: Ignore qemu usbhid errorsRichard Purdie
These are harmless from the USB pointer device we install, ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10oeqa/runtime: Automatic test for ptestLucian Musat
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder. No cleanup is done for packages after the test is finished. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2014-08-29oeqa: xorg log test is being replaced by parselogs so removeRichard Purdie
Now we have the parselogs test, this one can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/pasrselogs: Improve the machine/string whitelistRichard Purdie
Currently the whitelist is imcomplete, inaccurate and suffers duplication. These changes: * Add common groups of errors * Change to make the default whitelist clear * Correctly (or at least better) escape the regexp expressions * Add in missing machines to allow builds on the autobuilder to suceed Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29oeqa/runtime: Automatic test for parsing the logs on a machine and search ↵Lucian Musat
for certain error keywords. This adds a common new qa test for general processing of log files. One significant improvement is machine dependent ignore filters. This can be used to replace several weaker individual QA tests that are currently used. Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11oeqa/runtime: add new cpp test and fileCorneliu Stoicescu
This tests out the limits header which we've noticed does have problems in some SDK builds. Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25lib/oeqa: Fix accidental revert of codeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25oeqa/rutime: Added testcase decorators for automated runtime tests. Also ↵Lucian Musat
added LogResults decorator for oeTest class in oetest.py Signed-off-by: Lucian Musat <georgex.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>