aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2015-08-26qemurunner: In the logging thread retry on EAGAIN, EWOULDBLOCKRandy Witt
On a nonblocking socket an exception can be generated for the EAGAIN and EWOULDBLOCK errnos. Since these aren't actually errors make sure to retry rather than bailing out. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-25qemurunner: Shut down logging thread successfully when test failsRandy Witt
Before this change on a test failure an exception would be generated due to runqemu being killed before the logging thread which was on the other end of the socket. The exception was actually correct saying there was no data on a socket marked readable, but this was because the qemu process was killed before the listener thread. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-24runqemu: Add a tcpserial optionRandy Witt
The option was added so that the qemurunner could start a second tcp serial port without adding machine conditional logic to qemurunner. The issue that made this necessary was that when "virt" is passed to qemu-system-aarch64, the normal mechanism for specifying a tcp serial port does not work. This is because the hardware for the "virt" machine is hardcoded in the device tree blob and the addition devices must be virtio devices. So runqemu can specify virtio for qemuarm64 whereas it seems all other qemu machines work with the "-serial tcp*" option. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-24qemurunner: Use two serial ports and log console with a threadRandy Witt
qemu can freeze and stop responding if the socket buffer connected to a tcp serial connection fills up. This happens of course when the reader of the serial data doesn't actually read it. This happened in the qemurunner code, because after checking for the "login:" sentinel, data was never again read from the serial connection. This patch solves the potential freeze by adding a thread to continuously read the data from the console and log it. So it also will give a full log of the console, rather than just up to the login prompt. To simplify this patch, another serial port was also added to use for the sole purpose of watching for the sentinel as well as being the interactive serial port. This will also prevent the possibility of lots of debug data on the console preventing the sentinel value from being seen due to interleaved text. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-24qemurunner: Make create_socket() return data and use exceptionsRandy Witt
So that create_socket() can be called more than once to create sockets, it now returns the socket and port rather than setting class variables directly. create_socket() now only uses exceptions for errors, not the return value from the function. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-24qemurunner.py: Move some class variables that should only be localRandy Witt
The bootlog and qemusock variables were set in the class as part of the create_socket() routine. However those variables are never used outside of the same function and thus serve no purpose as class variables. This initializes those variables near where they are used. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-24oe-selftest: replace assertTrue(False, ...) with fail(...)Paul Eggleton
I'd somehow missed the existence of fail() when I wrote these. It's preferable here so you don't get the somewhat useless "false is not true" message in the case of failure. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-24wic: Test creation of iso imageMihaly Varga
Added new wic testcase, for testing the creation of the hybrid iso image with isoimage-isohybrid plugin. Signed-off-by: Mihaly Varga <mihaly.varga@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-19oeqa/targetbuild: Remove bashismRichard Purdie
Use '.' instead of 'source' so this works with dash as /bin/sh. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-19devtool: fix handling of BBCLASSEXTENDed recipesPaul Eggleton
If a recipe is BBCLASSEXTENDed (e.g. to -native), its PN value and the name of the bbappend will be different; we were assuming them to be the same when reading in the workspace, leading to us seeing the base recipe name everywhere afterwards. Also add a test so we ensure this doesn't regress in future. Fixes [YOCTO #8157]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16oetest.py: Don't wait to write dump filesMariano Lopez
This allows to write the dump files immediately after get the data from the target. Before this, it would run all the commands and write the files. The old behavior could cause no log written at all if the serial console gets stuck. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16oetest.py: Added method tearDown for oeRuntimeTestMariano Lopez
The tearDown method is triggered when a tests ends it doesn't matter if fails or succeeds. Inside this method added an evalution to check if fails and then run some commands in the target to get the data for later debugging. [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16qemurunner.py: Performance improvements in run_serialMariano Lopez
This allow to search for the prompt after a command is run so it can avoid waiting for the timeout. Also corrected identation issues [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16qemurunner.py: Added raw mode in run_serialMariano Lopez
Raw mode allows to send the command without sending 'echo $?' for validation; Also this doesn't remove the command or the prompt from the output returned. In raw mode validation is done if there is output. This raw mode would be useful for validate the prompt when a user logs in. [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16qemurunner.py: Added login to start methodMariano Lopez
This adds the automatic login after the target finished booting. If the automatic login fails it won't stop the target or any test, it would only send a log to the file. [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-16qemurunner.py: Add method run_serialMariano Lopez
The only need for the console before this patch was to check if the target has booted. This allows to send commands to the terminal. This new method is based on the method with the same name of the QemuTinyRunner class. The difference here is it will remove the command and the prompt. The other diference is it will send an echo $? to check if the last command was successful. [YOCTO #8118] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-11oeqa/selftest: remove tests for TCs that are no longer validMihail Stanciu
Several toaster backend tests are no longer valid due to design changes. Signed-off-by: Mihail Stanciu <stanciux.mihail@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-09oeqa/qemurunner: Improve runqemu log output debugRichard Purdie
If runqemu fails, ensure the log output is shown as its invaluable to aid debugging. Its slightly convoluted since we need to ensure we don't block on reading the pipe which may still be executing hence the need for nonblocking IO. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09qemurunner: Improves checking for server and target IPs on qemus parametersAlejandro Hernandez
Fixes OS hanging infinitely waiting for qemus process to release bitbake.lock Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-31oeqa/commands: Increase qemu boot timeout to match testimage.bbclassRichard Purdie
This matches the value in testimage.bbclass, which makes sense since the autobuilders are usually contended. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-31oeqa/sstatetests.py: add annotator to test_sstate_32_64_same_hash [YOCTO #6006]Costin Constantin
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-31oeqa/sstatetests.py: add decorator to test_sstate_nativelsbstring_same_hashCostin Constantin
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-31oe-selftest: devtool: fix teardown warning in test_devtool_update_recipe_appendPaul Eggleton
We don't need to run bitbake -c cleansstate on this because it's being cleaned out as part of "devtool reset". Addresses [YOCTO #8031]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-31sshcontrol: Use os.environ.copy() instead of copy.copy()Richard Purdie
os.environ is special and copy.copy() doesn't do what we'd expect, changes in the child object change the parent. copy.deepcopy() is also known to have issues with it. Use the dedicated .copy() method which will not influence the parent. This fixes selftest failures where the DISPLAY variable disappears. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/qemurunner: Fix AttributeError: QemuRunner instance has no attribute ↵Richard Purdie
'server_socket' If start() returns False due to create_socker() failing, stop() may still get called and currently this gives a track back since server_socket doesn't exist. Avoid this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27qemurunner: Show last 25 line of log, not 5Richard Purdie
For debugging purposes, 5 lines often isn't useful as it doesn't even cover a full backtrace. Show 25 instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27targetcontrol: Improve logs by outputting qemu boot log in case of failureRichard Purdie
We've had a few cases on the autobuilder where we've lost logs of the boot and been unable to debug it further. Show this information onto the console to be more useful. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/devtool: use new runqemu functionPaul Eggleton
Use the common code we now have to run QEMU instead of running it ourselves, avoiding reliance on the machine showing up at 192.168.7.2. This also makes a copy of the image rather than using -snapshot so if we need to inspect the image after a failure, we can. Fixes [YOCTO #7928]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: remove gummiboot testsPaul Eggleton
These tests were _deleting_ meta-intel if it happened to appear under COREBASE, which could have been catastrophic if there was any work in progress in that directory. It turns out we don't even need meta-intel, but we do need a machine that's set up appropriately (e.g. genericx86-64). Tests that involve layers outside of OE-Core don't really belong in OE-Core, and genericx86-64 is in meta-yocto-bsp; however we will soon have the capability to have selftest tests in other layers, so remove this here so we can add a fixed version in meta-yocto-bsp after that happens. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: fix RPM4 testPaul Eggleton
* Use our new runqemu function * Don't hard-code the RPM4 version * Double-check the native version is RPM4 * Check that an rpm 4.x package is in the image manifest (this isn't strictly necessary, but "belt-and-braces" and it serves as an example of how to do that) * Check that the database is working on the target * Ensure the image actually has openssh in it so we can connect to it Initial runqemu adaptation by Richard Purdie <richard.purdie@linuxfoundation.org> Part of the fix for [YOCTO #7994]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: fix SSH without password testsRichard Purdie
* We need to set EXTRA_IMAGE_FEATURES outright or existing values will affect the test * For test case 1107 we need "empty-root-password" to match the behaviour described in the test case * For test case 1115 we shouldn't be able to connect as root with the features we are setting Test cases 1107 and 1115 have been updated in Testopia to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/selftest/imagefeatures: Use QemuTarget codeRichard Purdie
Create a runqemu function which uses the QemuTarget() code from oeqa.targetcontrol to setup the QEMU instance, with all of the added robustness that that gives us. To do this, a datastore is needed for the recipe in question (core-image-minimal) so we do the work needed to set this up. We then use this runqemu function within the imagefeatures tests instead of a hand-rolled implementation. We can then use SSHControl to run the SSH tests rather than rolling our own code to do that as an added bonus. Fixed and extended by Paul Eggleton <paul.eggleton@linux.intel.com>. Part of the fix for [YOCTO #7994]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/targetcontrol: write QemuRunner log output to a filePaul Eggleton
If we use this outside of testimage we don't have a task log; so let's just explicitly write the log output to a file all the time so it's always there to look at (particularly useful when runqemu exits immediately with an error.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/utils/qemurunner: fix error handling if runqemu exits with an errorPaul Eggleton
* Don't wait for QEMU to start if it's never going to (because runqemu exited with an error) * Don't error out if killing the process fails with "no such process" (we don't care if it's already dead) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/utils/qemurunner: avoid blocking on stty when running under oe-selftestPaul Eggleton
runqemu-internal runs stty to return the terminal to its previous state in case QEMU hasn't done that properly (which it at least used to do when it crashed). For some reason I have yet to determine, stty blocks (on tcsetattr() according to gdb) when run within QemuRunner() under oe-selftest, with the result that we always wait until the timeout and then we kill the script, which adds an extra delay after QEMU is stopped. Naturally you would assume that this is something to do with the nature of the terminal under which it is being run; however no amount of playing around with stdin/stdout/stderr seemed to fix the issue, apart from passing in subprocess.PIPE as stdin which makes stty error out with "stty: standard input: Inappropriate ioctl for device". I was also unable to come up with a reliable test for the terminal which we could use inside runqemu-internal to avoid calling stty. For now, go with the stdin=subprocess.PIPE workaround to at least avoid the delay with minimal ill effect. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/utils/qemurunner: fix loggingPaul Eggleton
OE-Core commit 519e381278d40bdac79add340e4c0460a9f97e17 unfortunately broke logging in two different ways: 1) it prevented logging to the task log from working within bitbake -c testimage. This is due to the logger object being set up too early which interferes with BitBake's own logging. If we prefix the name with "BitBake." everything works (and we don't need to set the logging level). 2) Additionally because it called the log functions on the logging module and not the logger object it set up, this caused the oe-selftest logging to start printing everything from that point forward. Fix these two issues and return us to the desired behaviour for do_testimage. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/targetcontrol: create test directory before copying rootfs imagePaul Eggleton
The test directory might not exist at this point so just go ahead and create it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27oeqa/recipetool: allow templayerdir overrideChristopher Larson
This is provided for use by subclasses. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-27oeqa/recipetool: add tests for appendsrcfile(s)Christopher Larson
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-27oeqa/recipetool: refactor / split out RecipetoolBaseChristopher Larson
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-25oeqa: Add exception handling around qemu process cleanupRichard Purdie
The qemu processes may no longer exist at the end of the test so don't error if that is the case by ignoring any exceptions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/qemurunner: don't assume hostname starts with qemuRoss Burton
It's possible to customise the hostname trivially so don't assume that the hostname hasn't been configured. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/runtime/parselogs: don't use uname to get MACHINERoss Burton
Instead of running "uname -n" on the target (which actually returns the hostname) to get something approximating the hostname (or not, if that has been overriden), simply fetch MACHINE from the datastore. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/selftest/imagefeatures: handle killpg victim not existingRoss Burton
If Pexpect throws an exception it may be because the child has already died, in which case killpg will fail. Catch the latter exception silently, and output the exception from Pexpect. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/runtime/parselogs: add more x86 log exceptionsRoss Burton
Move "fail to add MMCONFIG..." to qemux86_common as we started to also see it on x86-64 with Linux 4.1. Add "can't claim BAR..." to qemux86_common. Generalise "Failed to access perfctr MSR" as the MSR number can change. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24oeqa/sshcontrol: Ensure we don't trigger ssh-askpassRichard Purdie
If DISPLAY is set, ssh-askpass can be triggered which is not what we want in the middle of sanity tests. We can disable this by unsetting DISPLAY. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/buildoptions.py: automate test case 926, archive work dir and export sourceCostin Constantin
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/buildoptions.py: automate test case 563; build directfb imageCostin Constantin
Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/QemuRunner: don't use bb for loggingRoss Burton
Instead of using bb.note() etc for logging use logging.Logger directly, allowing the use of QemuRunner outside of bitbake. Also clean up the logging/errors by moving create_socket() out of __init__()/restart() and into start(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oeqa/syslog: Removed a pointless testcase and added skip for another.Lucian Musat
The testcase syslog --help was rather useless and also causing problems on images where syslog was non-busybox, like LSB images so I removed it. Added a skip condition for TC 202 for the same reason. If syslog is non-busybox then the test skips. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>