aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2016-11-01oeqa/selftest/kernel.py: Add new file destined for kernel related testsCostin Constantin
[YP#7202]: Test for linux-dummy The new kernel.py file is intended for kernel related test cases. The test for linux-dummy will ensure it is in good shape and can be used as a kernel replacement at build time. To do this, the test will first clean sstate for linux-dummy target, ensuring no file is present in the stamps directory. After, core-image-minimal is built, ensuring linux-dummy can be used as a kernel substitute. Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-15Remove RM_OLD_IMAGE, it's no longer usefulJoshua Lock
Since the move to put image deployment under sstate control in d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically removed before a new image is deployed (the default behaviour of the sstate logic). RM_OLD_IMAGE is therefore no longer required to provide this behaviour, remove the variable and its users. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-07oeqa/sshcontrol: Handle interrupted system call errorRichard Purdie
Deal with an interrupted system call gracefully: | File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/sshcontrol.py", line 55, in _run | if select.select([self.process.stdout], [], [], 5)[0] != []: | InterruptedError: [Errno 4] Interrupted system call Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07wic: selftest: add test for sdimage-bootpartEd Bartosh
Test creation of sdimage-bootpart image Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07wic: selftest: add test for systemd-bootdiskEd Bartosh
Test creation of systemd-bootdisk image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05oeqa.buildperf: measure apparent size instead of real disk usageMarkus Lehtonen
This change aligns disk usage measurements of the eSDK test with the old build-perf-test.sh script. And thus, also makes the results between the old and the new script comparable. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05parselogs.py: Add disabling eDP error to x86_common whitelistCalifornia Sullivan
The NUC6 firmware tells the kernel to try and initialize an embedded DisplayPort it does not have, causing this warning. Its harmless, so just whitelist it. Fixes [YOCTO #9434]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05oeqa/selftest: Update test after fetcher error changesBenjamin Esquivel
The following poky commit: 4359ef08 base.bbclass: Use bb.fatal() instead of raising FuncFailed changed the way the fetcher error is reported. Previous reporting: ...Function failed: Fetcher failure for URL:... New reporting: ...Fetcher failure for URL:... Updating how the check is done fixes the test error and accurately confirms the tested scenario for test_invalid_recipe_src_uri. [YOCTO #10370] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01oeqa/sstatetests: Add test for multilib allarch checksumsRichard Purdie
Switching between multilib configurations should not change allarch recipe or nativesdk checksums. Add a new sstate test for this based on the standard allarch test. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-30oeqa/utils: Add StreamHandler to loggerFrancisco Pedraza
StreamHandler was added due missing log information on the console in oe-selftest with Qemu Runner Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-29parselogs: Update uvesafb errors in qemu whitelistJianxun Zhang
This change only whitelists the timeout message in infinite wait case in uvesafb driver. With the latest timeout patch introducing infinite wait in uvesafb driver, we whitelist the timeout message since it works as a warning for issues related to timeout not to be fixed in build servers. We remove other errors for bug-discovering purposes in some cases where these lines are still worthy to be caught (not whitelisted): The removed errors show up again in the infinite wait case. It indicates a different root cause or the timeout patch doesn't work correctly. Timeout happens when developers explicitly set a non-negative timeout of a limited period to wait for task completion in uvesafb driver. Timeout or/and errors occur when kernel doesn't have the latest timeout patch in driver. Note: The latest timeout patch is tracked by: a2966330bcd29e99c0403235edb29433b0c53d56 [YOCTO #8245] Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28oeqa/sstatetests: Ensure we cover deb packaging backend for sstate testRichard Purdie
Currently we weren't testing the deb backaned for sstate correctness and there was a bug that had crept in. Ensure we cover all package backends with the test regardless of what the distro/conf sets. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28lib/oeqa: Regenerate galculator configureJussi Kukkonen
galculator configure seems to be so old it does not recognise --with-libtool-sysroot: regenerate configure. Fixes [YOCTO #10191]. Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28parselogs.py: Add amd_nb error to x86_common whitelistCalifornia Sullivan
This has always silently failed on hardware without AMD Northbridge, and a recent kernel patch made it not silent. It would be ideal to only whitelist the error for genericx86 MACHINEs and disable the CONFIG option that enables it in intel-* MACHINEs, but in order to disable this configuration option we would have to enable EXPERT and DEBUG_KERNEL, which we don't want. Instead just whitelist it on all x86 MACHINEs. Fixes [YOCTO #10261]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23oeqa/selftest/base: backup and restore local configuration filesJose Lamego
Selftests' cleanup method during test setup is not capable of restoring local configuration files that remain modified after aborting a test through a keyboard interruption. This change creates backups for local.conf and bblayers.conf at test setup, restore them when found, and deletes them at cleanup. [YOCTO #9390] Signed-off-by: Jose Lamego <jose.a.lamego@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-20oeqa/selftest/lic-checksum: don't report the expected failure to errors.ypRoss Burton
This test has a bitbake invocation that is expected to fail, so inhibit report-error running. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-20oeqa/runtime/parselogs.py: Add ignore of tsc calibration fail in x86Aníbal Limón
We are experimenting failures to calibrate CPU's using TSC in x86 VM's due to usage of nested KVM [1], this is a known issue [2][3] in virtualization environments, for detail explnation see [4]. Also we already have an ignore for 'TSC Fast calibration fail'. [1] http://errors.yoctoproject.org/Errors/Details/83684/ [2] https://bugzilla.redhat.com/show_bug.cgi?id=814231 [3] https://lists.nongnu.org/archive/html/qemu-devel/2010-09/msg01547.html [4] https://www.kernel.org/doc/Documentation/virtual/kvm/timekeeping.txt Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-16oeqa/sdkext/devtool: use a smaller module to test node.js functionalityPaul Eggleton
The "forever" package, despite its innocent description, actually drags in a surprising number of dependencies and as a result the nodejs test takes up to 10 minutes as a result. Pick a different example with a much more reasonable set of dependencies. Addresses part of [YOCTO #10254]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-16qemurunner: print out the runqemu commandbavery
This logs the launch command that was used for runqemu while running -c testimage. This way, if I'd like to easily launch qemu manually in order to debug a failed test, I know what commmand was run to create the qemu instance. Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-15oeqa/oetest: show stderr when running commandsRoss Burton
To help debug failures, redirect stderr to stdout in oeSDKTest.run() and oeSDKExtTest.run(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-15oeqa/utils/decorators: LogResults fix race condition in linkfileAníbal Limón
In order to avoid race condition when test if exists the linkfile use bb.utils.lock, the best solution is to create a unique name for the link file. There is no way to create a unique linkfile name at this decorator because is needed the machine and image variables, those variables can't be passed easily in this code. To avoid broke test export functionality use a try/except because bb isn't available when use test export [YOCTO #10225] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14Revert "oeqa.runtime.smart: work around smart race issues"Richard Purdie
We have a proper fix now so we can remove the workaround. This reverts commit 4d268abc2fc892c5d34449f78c8e9f2b1a9d6bac.
2016-09-13oeqa: Remove linux user utilized for rpm test.Edwin Plauchu
When trying to re-test smart rpm tests. A fail arises due to a linux user previously created upon the image. We've added a few lines to delete such user and his home dir when finishing test. [YOCTO #9204] Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-13oeqa.buildperf: another fix for splitting 'nevr' stringMarkus Lehtonen
When processing buildstats we determine recipe name, epoch, version and revision from the per-recipe buildstat directory name. One previous patch made an assumption that package version starts with a number. That might not be true because of a packaging mistake or whatever reason. Thus, if a version starting with a number is not found, fall back to the "old" method of just taking the second-last dash-delimited part (the one before revision). Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-09oeqa/parselogs: Don't use cwd for file transfersRichard Purdie
If you run: MACHINE=A bitbake <image> -c testimage MACHINE=B bitbake <image> -c testimage and A has errors in parselogs, machine B can pick these up and cause immense confusion. This is because the test transfers the log files to cwd which is usually TOPDIR. This is clearly bad and this patch uses a subdir of WORKDIR to ensure machines don't contaminate each other. Also ensure any previous logs are cleaned up from any existing transfer directory. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09oeqa.runtime.smart: work around smart race issuesMarkus Lehtonen
Yucku hack around test failures which ultimately are caused by a race in smartpm itself. Issuing smartpm commands in quick succession causes races in package cache of smartpm on some systems. This patch mitigates the problem by sleeping for 1 second after each smartpm command that modifies the system. [YOCTO #10244] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09oeqa/runtime/smart: Prune feeds to save memoryRichard Purdie
Full package feed indexes overload a 256MB image so reduce the number of rpms the feed. Filter to p* since we use the psplash packages and this leaves some allarch and machine arch packages too. [YOCTO #8771] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09oeqa: Use snapshot instead of copying the rootfs imageRichard Purdie
Rather than copying images, use the snapshot option to qemu. This fixes a regression caused by the recent runqemu changes where the wrong images were being testes since the image is copied without the qemuboot.conf file. This means the latest image is found by runqemu rather than the specified one, leading to various confused testing results. It could be fixed by copying more files but use snapshot mode instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09qemurunner.py/qemutinyrunner.py: remove runqemu-internalRobert Yang
There is no runqemu-internal any more. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09parselogs: Whitelist qemux86 error message with qemu 2.7.0Robert Yang
qemu 2.7.0 introduces kernel errors: [ 2.310768] pci 0000:00:00.0: [11ab:4620] type 00 class 0x060000 [ 2.311338] pci 0000:00:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size) [ 2.311604] pci 0000:00:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size) [ 2.311835] pci 0000:00:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size) [ 2.312063] pci 0000:00:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size) [ 2.312323] pci 0000:00:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size) [ 2.314320] pci 0000:00:0a.0: [8086:7110] type 00 class 0x060100 [ 2.315363] pci 0000:00:0a.1: [8086:7111] type 00 class 0x010180 Whitelist this for now since this is preferable to the random failures we're seeing from qemuppc with 2.6.0. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09parselogs: Whitelist qemuppc error message with qemu 2.7.0Richard Purdie
qemu 2.7.0 introduces kernel errors: [0.474981] pci 0000:00:0f.0: reg 0x10: [io 0x0400-0x041f] [0.483796] pci 0000:00:0f.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff82800000) [0.484204] pci 0000:00:0f.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.488077] pci 0000:00:0f.0: reg 0x30: [mem 0x83000000-0x8303ffff pref] [0.488903] pci 0000:00:10.0: [1af4:1005] type 00 class 0x00ff00 [0.490485] pci 0000:00:10.0: reg 0x10: [io 0x0480-0x049f] [0.496512] pci 0000:00:10.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff83800000) [0.496783] pci 0000:00:10.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.500345] pci 0000:00:11.0: [1af4:1001] type 00 class 0x010000 [0.501790] pci 0000:00:11.0: reg 0x10: [io 0x0500-0x053f] [0.507362] pci 0000:00:11.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff84000000) [0.507677] pci 0000:00:11.0: reg 0x20: [mem size 0x00800000 64bit pref] [0.513905] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00 [0.516493] PCI 0000:00 Cannot reserve Legacy IO [io 0x0000-0x0fff] [0.517512] pci 0000:00:0f.0: BAR 4: assigned [mem 0x80800000-0x80ffffff 64bit pref] [0.518877] pci 0000:00:10.0: BAR 4: assigned [mem 0x82800000-0x82ffffff 64bit pref] [0.519890] pci 0000:00:11.0: BAR 4: assigned [mem 0x83800000-0x83ffffff 64bit pref] Whitelist this for now since this is preferable to the random failures we're seeing from qemuppc with 2.6.0. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08oeqa.buildperf: be sure to use the latest buildstatsMarkus Lehtonen
Be sure to take the latest buildstats if multiple buildstats are found. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-08oeqa.buildperf: try harder when splitting 'nevr' stringMarkus Lehtonen
Try to be more intelligent when splitting out recipe name, epoch, version and revision from the buildstat directory name. Previous assumption was that package versions never contain a dash but obviously that is not necessarily true. The new assumption is that the package version starts with a number. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-06oeqa.buildperf: correct globalres time formatMarkus Lehtonen
Always use two digits for (integer part of) seconds, i.e. show '1:02.34' instead of '1:2.34'. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-06selftest/liboe: add a test for copyhardlinktree()Joshua Lock
Add a simple test to validate that the number of files in the destination matches the number of files in the source after the copyhardlinktree() has been performed. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05oeqa/parselogs: Add qemuarm64 warning from 4.8 kernel to whitelistRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05oeqa/parselogs/qemuarm: Whitelist amba and jitter for 4.8+ kernelsBruce Ashfield
With the update to the 4.8 kernel the versatile platform (and hence qemuarm) has switched to a device tree boot. We are using an ummodified mainline kernel versatilepb device tree, which includes definitions of multiple amba devices. These devices are not present in the qemu system emulation, hence throw warnings during boot. These warnings are not unique to oe-core, and rather than carry kernel patches to the device tree (for now), we whitelist the known warnings so qa testing will pass. We also can't turn amba off completely, since it is providing valid devices (like the serial port) and AMBA is force selected by other kconfig values. We also have a jitterentropy warning that shows up on some hosts. This warning is harmless, and like amba we can't turn it off in a fragment since it is force selected by crypto (and we'd rather not turn all crypto off). So we add it to the whitelist while investigations continue into what is needed in the host to support this fully. Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-04image: Deploy images to IMGDEPLOYDIREd Bartosh
Changed deployment directory from DEPLOY_DIR_IMAGE to IMGDEPLOYDIR to make sstate machinery to do final deployment and generate manifest. Renamed variable deploy_dir to deploy_dir_image in selftest code to avoid confusion with DEPLOYDIR variable. Updated the code of rootfs.py:Rootfs class to use IMGDEPLOYDIR variable as it's now used as a new deployment destination. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa/selftest Adds eSDK test cases to devtool verification.Francisco Pedraza
The covered functions are, install libraries headers and image generation binary feeds. Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03/oeqa/sdkext Adds verification for devtool on eSDK.Francisco Pedraza
The covered funcions are, build make, build esdk package, build cmake extend autotools recipe creation, kernel module, node.js installation and recipe creation. Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oe-selftest: devtool: fix test after recent changePaul Eggleton
OE-Core commit d3057cba0b01484712fcee3c52373c143608a436 fixed handling of wildcard bbappends, which means that this test's expectations about the bbappend file name are no longer met. devtool finish is meant to use wildcard bbappends so fix the test accordingly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.selftest.liboe: add test for xattr in copytreeJoshua Lock
Add a test to ensure that oe.path.copytree() preserves extended attributes on files. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.selftest: add a test for oe.path.copytree()Joshua Lock
One motivation for the use of cpio in oe.path.copytree() was to ensure that files with spaces in their names were copied. Add a new unittest module to test the OE module with a test case for copytree with a spaces in a filename. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: include commands log file name in results.jsonMarkus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: include buildstats file name in results.jsonMarkus Lehtonen
No need to do lsdir magic for finding buildstats when reading results. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: show skipped tests in results, tooMarkus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: convert buildstats into json formatMarkus Lehtonen
Instead of archiving buildstats in raw text file format convert all buildstats into one json-formatted file. Some redundant information, i.e. 'Event:', 'utime:', 'stime:', 'cutime:' and 'cstime:' fields, are dropped. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: measure io statMarkus Lehtonen
Add data from /proc/<pid>/io to system resource measurements. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: don't use Gnu timeMarkus Lehtonen
Use Python standard library functionality instead of the time utility for measuring elapsed (wall clock) time of commands. The time.* log files are also ditched. However, the same detailed resource usage data, previously found in time.* logs is now provided in results.json file. This data is collected through the resource module of Python. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03oeqa.buildperf: rename buildstats directoriesMarkus Lehtonen
Change directory name from 'buildstats-<test_name>' to just 'buildstats'. However, this patch adds the possibility to label buildstats directory name with a postfix which makes it possible to save multiple buildstats per test, for example. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>