summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/runtime
AgeCommit message (Collapse)Author
2018-06-04oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logicChen Qi
The test_file_connman should be executed only when 'lib32-connman' is installed and 'connman' is not installed. When lib32-connman and connman are both installed, the /usr/sbin/connmand could be from connman or lib32-connman, depending on the installation order. What we want to check is the connmand command from lib32-connman, so we need to make sure that connman is not there to cause chaos. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/runtime/cases/multilib.py: skip if needed packages are not availableChen Qi
1) The test cases use 'readelf' command to do the check. This command is from binutils. So skip the test if the needed binutils package is not installed. The related error message in log.do_testimage is like below. Output: sh: readelf: not found 2) The test case tests /lib/libc.so.6 from lib32-libc6. So skip the test if lib32-libc6 is not installed. The related error message in log.do_testimage is like below. Output: readelf: Error: 'lib/libc.so.6': No such file Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04oeqa/runtime/cases/rpm.py: skip if rpm not availableChen Qi
This test case should only run when rpm package is installed. So skip it if rpm package is not installed. This fixes: RESULTS - rpm.RpmBasicTest.test_rpm_help - Testcase 1059: FAILED Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-14runtime/dnf: Add new dnf test casesJose Perez Carranza
Add test cases to test “exclude” and “installroot“ options, also modify the logic of filtering packages on the feed to have all the packages needed by the tests. [YOCTO #10744] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04oeqa/runtime/cases/python: use python 3 rather than python 2Alexander Kanavin
For example, core-image-sato skipped the test alltogether, as it no longer pulls in Python 2.x at all. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04logrotate: update to 3.14.0Yi Zhao
Since the wtmp and btmp definitions had been moved from logrotate.conf to logrotate.d in this release, we also need to install them to /etc/logrotate.d/. Also update oeqa runtime logrotate test case. Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18oeqa/runtime/stap.py: add runtime test for systemtapVictor Kamensky
Add runtime test for stap to test basic SystemTap operations: can compile very basic module and run on target device. Note we disable (-DSTP_NO_VERREL_CHECK) SystemTap additional kernel release check since during OE testing mismatching kernel-devsrc and kernels are used. Signed-off-by: Victor Kamensky <kamensky@cisco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-03-30make-mod-scripts/kernel-devsrc: Fix objtool issuesRichard Purdie
Kernels which use tools/objtool can now fail when building external modules due to objtool being missing, the generated files can also cause problems for kernel-devsrc. Ensure objtool is generated in make-mod-scripts by also calling "make prepare". For devsrc, delete the generated binaries since they'd be native binaries and unsuitable for the target. The oeqa kernel module tests also need to have the additional "make prepare" step added. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04oeqa/runtime/opkg.py: add runtime test for opkgYeoh Ee Peng
Add runtime test for opkg to test that it can install ipk package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-03-04oeqa/runtime/apt.py: add runtime test for aptYeoh Ee Peng
Add runtime test for apt to test that it can install deb package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-29oeqa/runtime: add gobject-introspection testRoss Burton
(From OE-Core rev: 497a95b576e19e39e20ac280d0db24f51b7c9679) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06oeqa/runtime/buildcpio: Use our own mirror for sourceRichard Purdie
We see occasional network glitches which break this test. Use our own mirror (which has a .gz instead of .bz2) to avoid the errors, we're not trying to test network connectivity. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: fail when ptests fail on targetAlexander Kanavin
That's the whole point isn't it? Previously this testcase succeeded even if some of the underlying on-target tests failed; the only way to find out if anything was wrong was to manually inspect the logs. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run ↵Alexander Kanavin
only when ptest-runner is availalble; Previously the test would execute only when all available ptests for packages in the image were installed; some of those tests may be broken, never finish, take a very long time or simply irrelevant to the user who wants to check ptests of only a few specific packages, and does so by listing them explicitly via IMAGE_INSTALL_append or similar. Presence of ptest-runner means there is at least one ptest package installed as they pull it in via a class dependency; ptest-runner is not generally installed otherwise. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23runtime/cases/_ptest.py: rename it to ptest.pyRobert Yang
It works now. [YOCTO #11547] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23runtime/cases/_ptest.py: add skip statusRobert Yang
The packages' test cases maybe skipped, check and save them. [YOCTO #11547] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23runtime/cases/_ptest.py: revive itRobert Yang
* Make it work with current oeqa * Skip the test if ptest is not in DISTRO_FEATURES * Skip the test if ptest-pkgs is not in IMAGE_FEATURES * The logs are saved to: testimage/ptest_log -> testimage/ptest_log.<datetime> * This provides data that could be used to detect regressions in ptest results [YOCTO #11547] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23oeqa/parselogs: Ignore device node warnings on mipsRichard Purdie
THe newner kernels have ope firmware support enabled which leads to warnings during boot on mips. These aren't interesting and we should ignore them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-08oeqa/runtime: Replace buildiptables for buildlzip on runtime testsJose Perez Carranza
Buildiptables test cases are conflicting with images built with “musl” as standard C library, in order to avoid those issues lzip package was selected to be used on the tests as this does not have any "musl" dependency. This patch is applicable for testimage tests [YOCTO # 11713] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-05oeqa/runtime/context.py: Add support to specify port in target_ipAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/runtime/case: Don't use OEQA framework internal methodsAníbal Limón
The OEQA framework has internal methods for provide functionality in decorators so Test components aren't expected to override it. Use the base unittest methods for setUp and tearDown. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-10parselogs: whitelist failed to enable keyboard error on intel-corei7-64California Sullivan
This error is seen on the Braswell RVP platform we have been using for testing. The error is caused by atkbd.c keyboard driver, which is only for AT and PS/2 keyboards. I tested a PS/2 keyboard with the board, which worked fine, and the board does not have a separate AT connector, so this error won't cause any functional issues. [YOCTO #10110]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-10parselogs: whitelist bluetooth firmware load error for intel-corei7-64California Sullivan
The NUC6 has issues bringing up Bluetooth early in the boot sequence. We see: [ 4.091790] Bluetooth: hci0: Minimum firmware build 1 week 10 2014 [ 4.097326] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi [ 4.145317] Bluetooth: hci0: Failed to send firmware data (-38) Followed by this later on: [ 11.509870] Bluetooth: hci0: Minimum firmware build 1 week 10 2014 [ 11.509988] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi [ 13.090308] Bluetooth: hci0: Waiting for firmware download to complete [ 13.090829] Bluetooth: hci0: Firmware loaded in 1549114 usecs [ 13.090987] Bluetooth: hci0: Waiting for device to boot [ 13.101958] Bluetooth: hci0: Device booted in 10818 usecs Bluetooth does successfully come up and the firmware is loaded. This behavior is consistent across all kernels I've tested. [YOCTO #10628]. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-06oeqa/runtime/utils/targetbuildproject: use parent classes defaults tmpdirJoshua Lock
Rather than hard-coding the tmpdir for TargetBuildProject to /tmp allow the parent's default handling to define an appropriate tmpdir. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05runtime/dnf : Add ID to the dnf test casesJose Perez Carranza
Add the corresponding Testopia ID to the test cases for dnf Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31oeqa/runtime/cases: Migrate underscore casesMariano Lopez
There were two missing cases to be migrated to the new framework: _qemutiny and _ptest. qemutiny was straightforward. ptest on the other hand wasn't working even in previous releases; it has been migrated from smart to dnf, and how ptest packages are gathered to be installed, adapted to use unicode, and removed a lot of code that wasn't needed anymore. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22oeqa: use WORKDIR/oe-testimage-repo to look for RPM packagesAlexander Kanavin
Using RPM deploy dir was causing errors when pre-built images were used with these steps: https://wiki.yoctoproject.org/wiki/Quality_Assurance_yocto_project [YOCTO #11173] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21meta/lib/oeqa: change sdk test from cvs to cpiobrian avery
We currently fetch,configure,build, and install cvs as our test for the sdk. cvs unfortunately, requires a default editor in order to run. The change in 94790a8254d6 that checks to see if you have something like vi installed is fragile since you may have a different default editor. This patch switches from using cvs as a test to using cpio. cpio also uses autotools so the functionality tested is equivalent. Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-17dnf: add /usr/bin/dnf symlink that points to /usr/bin/dnf-2Alexander Kanavin
All documentation refers to dnf binary as 'dnf' yet make install does not create one - it's done by Fedora's spec file when building the rpm. Let's replicate this behavior. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13testimage.bbclass: fix runtime test for rpm, port smart tests to dnfAlexander Kanavin
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-02-23oeqa/runtime/context.py: Fix use of getTarget() with testexportMariano Lopez
The idea on getTarget is to use kwargs to send custom variables to different targets, instead of this, a new variable was added (just used for custom targets) and this broke testexport. So in order to fix it, just add the custom variable to kwargs. This fixes the use of getTarget() in testexport class that was introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15testimage: Implement custom target loadingJuro Bystricky
This patch implements custom target loading for testimage, currently missing due to major changes to the test framework. Custom targets can be defined in various meta-layers, so we need an extra path information in order to find them. Any other additional info is retrieved as usual via the variables TEST_TARGET and TEST_SUITES Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05oeqa/runtime: Improve failure log outputRichard Purdie
Printing a message which says "configure failed" without the log output is effectively useless. If a command fails, print the output by default and simplify the calling code which makes debugging any of these failures much easier. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31oeqa/runtime/cases: Rename syslog module to oe_syslogAníbal Limón
Debian based distros has a builtin syslog module so when try to load tests using unittest it references the builtin module instead of runtime/cases. [YOCTO #10964] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31runtime/cases: Fix case numbers, missing cases and unused classesMariano Lopez
gcc: Removed unneded lines. multilib: fixed case number. syslog: added a missing test. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSESMariano Lopez
smart test requires to build the image using rpm packages, this check was included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is not true when building packages for rpm and deb/ipk. So this would check IMAGE_PKGTYPE instead. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31runtime/cases/connman.py: Stop using oeRuntimeTestMariano Lopez
oeRuntimeTest class is not used anymore as part of runtime migration, this particular case was missed, so fix it. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23oeqa/runtime/context.py: Add defaults for runtime contextMariano Lopez
This adds default values to OERuntimeTestContextExecutor class in order to make easier the execution of exported test that were generated with testexport class. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23testimage.bbclass: Add package install featureMariano Lopez
This allows to use the package install feature with the new OEQA framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23runtime/cases/smart.py: Migrate smart testsMariano Lopez
This migrates the smart test from the old framework to the new one. This has its own commit because smart test was using bb and oe libraries that are available when exporting the test cases to run in a different host. Because of the removal of bb and oe libraries index and packages feeds creation will be managed in testimage bbclass. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/cases: Migrate runtime tests.Mariano Lopez
This migrates current runtime test suite to be used with the new framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/context.py: Add logger to getTargetMariano Lopez
Current targets (ssh and qemu) require a logger in their constructors, so in order to get a new target we need to provide the logger. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/context.py: Prepare for qemuAníbal Limón
This commit will prepare context to use qemu and not just simpleremote. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor classAníbal Limón
To avoid getVar calls inside a utils module, also moves get_host_dumper import inside testexport isn't needed. [YOCTO #10231] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/context: Move helper functions for process args to executorAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/context: Add runtime option group and options for target type ↵Aníbal Limón
and server ip. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters ↵Mariano Lopez
per line Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime/files: Move runtime files from old directoryMariano Lopez
As part of the refactor we require to move the files used in runtime testing to the new directory. This also adds the path to the runtime test context. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime: Add OEHasPackage decoratorMariano Lopez
This new decorator will be used to skip the test if the image under test doesn't have the required packages installed. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23oeqa/runtime: Add case, context and loader classes for runtime testingMariano Lopez
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader to be used for runtime testing. As expected there are some changes in runtime context: - Adds the target to be used for runtime testing, the default is a SSH connection to the device under test running a OE image. - Runtime context requires image manifest because several tests are skipped if a package is missing or installed. - Several tests require the output of the ps command and it changes its output and arguments if busybox o procps is installed, so the case must use the correct ps command. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-13 03:06:53 +0000