summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2019-01-31oeqa/utils/logparser: Add in support for duration, exitcode and logs by sectionRichard Purdie
Allow parsing of the ptest duration, exit code and timeout keywords from the logs, returning data on each section. Also include the logs broken out per section. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31oeqa/logparser: Reform the ptest results parserRichard Purdie
Now we have a dedicated ptest parser, merge in the remaining ptest specific pieces to further clarify and simplify the code, moving to a point where we can consider extending/enhancing it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31oeqa/logparser: Further simplification/clarificationRichard Purdie
Rename the paster to be ptest specific and apply some further cleanups to the code to simplify and clarify what its doing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31oeqa/utils/logparser: Simplify ptest log parsing codeRichard Purdie
logparser is only used by ptest. Its slightly overcomplicated as it was intended to be reusable but wasn't. Simplify it as a dedicated parser is likely to me more readable and maintainable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31oeqa/runtime/ptest: Avoid traceback for tests with no sectionRichard Purdie
Some tests end up without a section, avoid tracebacks trying to use None as a string in that case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28oeqa/manual/sdk.json: Updated toolchain tarball to core-image-satosangeeta jain
Manual test step for SDK is updated. Previously toolchain was "poky-glibc-x86_64-core-image-sato-sdk<type-arch>-toolchain-<release-version>.sh" But toochain for core-image-sato-sdk is not available in releases after 2.1, hence changed it to "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh Other less significant update is to add exepcted result in intermediate test steps. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh" Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28oeqa/manual: Add manual runtime 'compliance' testcases to jsonSudhir Sharma
Added json file for compliance test cases to the manual runtime tests Signed-off-by: Sudhir Sharma <sudhirx.sharma@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26Change default debug split to make separate source packagesJoshua Watt
Changes the default PACKAGE_DEBUG_SPLIT_STYLE to generate separate source and debug packages. SDKIMAGE_FEATURES is updated to include the source packages so that there is not change for the SDK contents. [YOCTO #12931] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24testimage.bbclass: add support for passing runqemu paramsAlexander Kanavin
This is particularly useful when setting up GL tests. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24eSDK.py: unset BBPATH and BUILDDIR to avoid eSDK failureChen Qi
When executing eSDK test case, the following error appears. WARNING: attempting to use the extensible SDK in an environment set up to run bitbake - this may lead to unexpected results. Please source this script in a new shell session instead. FileExistsError: [Errno 17] File exists: '/.../tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-fcuyzsqu/tmp/sysroots/x86_64/bin/pigz' -> '/.../tmp/hosttools/pigz' So unset these two vars to avoid messing things up. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24runtime/ksample.py: correct a syntax and perfect item about architectureHongzhi.Song
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24eSDK.py: avoid error in tearDownClass due to race condistionChen Qi
When removing the temporary directory, it's possible that bitbake.lock file is removed by bitbake during the cleanup. And this leads to the following error. FileNotFoundError: [Errno 2] No such file or directory: 'bitbake.lock' So add a check to remove this file before cleaning up the temporary directory. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-21parselogs.py: ignore failure messages for beaglebone edgerouter and mpc8315e-rdbYi Zhao
Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-18oeqa/runtime/ksample.py: skip kprobe case except x86 and powerpcHongzhi.Song
A new commit was introduced to kernel, which caused kprobe sample case failed on arm. [kernel commit: e46daee53bb50b, ARM: 8806/1: kprobes: Fix false positive with FORTIFY_SOURCE] And according to author, the case just works on x86 and powerpc. So we just promise that the case can pass on x86 and powerpc. Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16selftest/wic: update test case test_qemuKai Kang
This test case boots the image in qemu and checks for mounted partitions. But the outputs of mount are different between sysvinit and systemd: sysvinit: /dev/root /\r\n/dev/sda1 /boot\r\n/dev/sda3 /media\r\n/dev/sda4 /mnt systemd: /dev/sda1 /boot\r\n/dev/sda2 /\r\n/dev/sda3 /media\r\n/dev/sda4 /mnt So check mounted partitions by egrep rather than check output of runqemu. Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16parselogs.py: whilelist one more amba error messageChen Qi
When using linux-yocto-dev, we will have one more amba error message for qemuarm, so ignore it too. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14oeqa/selftest/cases: Update test_ccache_toolRobert Yang
Now we can compile m4-native rather than target m4, this can save a lot of build time. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2019-01-11oeqa: make it work for multiple usersRobert Yang
There are failures when multiple users run oe-selftest on the same host: PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu' This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem. Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than /tmp/oe-saved-tests/ to fix the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-11selftest/distrodata: un-break the upstream version check testAlexander Kanavin
And fix the reported upstream check failures. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk: add test to exercise MesonRoss Burton
(From OE-Core rev: 29359493e391d68a5a6b4fa4d09ffdc1fe6db620) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk/assimp: cleanupRoss Burton
Unify style with the other tests. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-07oeqa/sdk/cases: clean up DL_DIR handlingRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-07buildoptions.py: use different STAMPS_DIR and SSTATE_DIRChen Qi
Use a different STAMPS_DIR and SSTATE_DIR in test_yocto_source_mirror. Otherwise, when executing `oe-selftest -a', we will get a lot of failures due to do_unpack failure. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/selftest/runqemu: Enable kvm when QEMU_USE_KVM is setRobert Yang
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/manual/bsp-qemu.json: Update for QEMU_USE_KVMRobert Yang
Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa: Fix for QEMU_USE_KVMRobert Yang
Fixed: MACHINE = "qemux86" QEMU_USE_KVM = "qemux86" IMAGE_CLASSES += "testimage" $ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs [snip] File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean raise ValueError("Invalid boolean value '%s'" % value) ValueError: Invalid boolean value 'qemux86' Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm will be enabled if target_arch == build_arch or both of them are x86 archs. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/utils/qemurunner: set timeout to 60s for run_serialRobert Yang
The 5s timeout for non-kvm is too short, especially when the load is high, which leads to unexpected errors, so set timeout to 60s by default. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-27oeqa/utils/qemurunner: Print output when failed to loginRobert Yang
This is useful for debugging. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26oeqa/concurrencytest: fix for locating meta-selftestRobert Yang
The previous code assumed builddir and meta-selftest are in the same dir, but this isn't always true, builddir can be anywhere, use get_test_layer() to locate meta-selftest can fix the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-18oeqa/selftest/distrodata: Drop now unneeded distrodata inheritRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15oeqa/selftest/distrodata: Port to use the new ↵Richard Purdie
recipeutils.get_recipe_upgrade_status() function Rather than use the obsolete do_checkpkg function, use the new recipeutils function which uses tinfoil to get the data rather than needing csv file manipulation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14oe-selftest: distrodata: change test_maintainers() to use tinfoilPaul Eggleton
Use tinfoil to enumerate recipes and get the value of RECIPE_MAINTAINER to make it a bit more reliable in the face of do_checkpkg issues we are currently seeing on the Yocto Project autobuilder. This also makes it a little less painful to re-execute test_maintainers() since you don't have to wait for bitbake -c checkpkg to complete every time. Note that the new test has been written in such a way that it will still function if RECIPE_MAINTAINER values are ever moved to the recipes. Also, the test still currently fails as there are recipes that don't have an assigned maintainer. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: rewrite cpio testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: rewrite lzip testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clean up galculator testRoss Burton
Drop redundant imports and variables, and use os.makedirs() instead of bb.utils.mkdirhier(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: show output if run() failsRoss Burton
Use oeqa.utils.subprocesstweak to monkey-patch the subprocess exception so that any output is shown, and remove any explicit try/catch handling that would have hidden this. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clarify ELF assertion messageRoss Burton
For example, instead of saying "3 != 62", say "Binary was x86-64 but expected i586". Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13selftest/package: Correct format arguments in test_gdb_hardlink_debugOla x Nilsson
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08oeqa/sdk/galculator: rewrite to use new helpersRoss Burton
2018-12-08oeqa/sdk/python: fix version typoRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-12-08oeqa/sdk/assimp: use helpersRoss Burton
2018-12-08oeqa/sdk/case: add fundamental helper methodsRoss Burton
2018-12-08oeqa/selftest/runcmd: Increase timeout deltaRichard Purdie
Expecting 1s accuracy on a 2s timeout on a heavily loaded system has proven to be unreliable. Update this to a 5s timeout with a 3s delta which should be achievable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08oeqa/selftest/context: ensure log directory existsChen Qi
Ensure log directory exists to avoid the following error. FileNotFoundError: [Errno 2] No such file or directory: '/.../build-selftest/tmp/log/oe-selftest-results-20181207043431.log' Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-07oeqa/selftest/esdk: Ensure parent directory existsRichard Purdie
INFO - ====================================================================== INFO - ERROR: setUpClass (eSDK.oeSDKExtSelfTest) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/eSDK.py", line 76, in setUpClass cls.tmpdirobj = tempfile.TemporaryDirectory(prefix="selftest-esdk-", dir=bb_vars["WORKDIR"]) File "/usr/lib/python3.5/tempfile.py", line 929, in __init__ self.name = mkdtemp(suffix, prefix, dir) File "/usr/lib/python3.5/tempfile.py", line 507, in mkdtemp _os.mkdir(file, 0o700) FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-debian/build/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-q7ln84gc' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/esdk: Fix typo causing test failureRichard Purdie
2018-12-06 23:19:24,564 - oe-selftest - INFO - Traceback (most recent call last): File "/media/build1/poky-sumo/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/media/build1/poky-sumo/meta/lib/oeqa/selftest/cases/eSDK.py", line 78, in setUpClass cls.tmpdir_eSDKQA = cls.tempdirobj.name AttributeError: type object 'oeSDKExtSelfTest' has no attribute 'tempdirobj' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05oeqa/selftest/esdk: run selftest inside workdir not /tmpRoss Burton
We've seen issues with rootfs size calculations and we've seen systems like opensuse which have btrfs mounted on /tmp causing selftest failures. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05oeqa: don't litter /tmp with temporary directoriesRoss Burton
If we need to create a temporary directory in targetbuild or buildproject use tempfile.TemporaryDirectory so that when the test case is finished, the directory is deleted. Also synchronise the logic and don't possibly store the temporary directory in self.tmpdir as nothing uses that. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05oeqa/utils/qemurunner: Avoid tracebacks on closed filesRichard Purdie
Reorder the shutdown/teardown to avoid: File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 224, in launch op = self.getOutput(output) File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 90, in getOutput fl = fcntl.fcntl(o, fcntl.F_GETFL) ValueError: I/O operation on closed file Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05oeqa/selftest/runqemu: Improve testcase failure handlingRichard Purdie
assertTrue doesn't give good debug information when things fail. Update several to use assertIn which gives information upon failure, for the others print the log information upon failure. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-25 16:34:18 +0000