summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/manual
AgeCommit message (Collapse)Author
2020-12-10meta/lib/oeqa/manual/oe-core.json: Update test_bitbake_devshellsangeeta jain
Update command used for cross compilation to include ${CONFIGUREOPTS} to ensure right arguments are passed for cross compiling on any host. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e995e67bc7a3d30adf9f8d0d2f4df92a941baeae) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-08-29oeqa/manual/bsp-hw.json : remove X_server_can_start_up_with_runlevel_5_boot testTeohJayShen
remove the test as graphical mode is already being validated Signed-off-by: TeohJayShen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ad5b5ca9ec1d9106911a610bb36fb98e3f17748d) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-08-29oeqa/manual/bsp-hw.json : remove shutdown_system testTeohJayShen
remove the test as shutdown function is already being validated Signed-off-by: TeohJayShen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 58281082fbed57298247c019c3d8149e688de161) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-03-19oeqa/manual/bsp-hw.json: fix syntax errorYi Zhao
Remove the redundant comma to fix the json decode error: $ resulttool manualexecution ../meta/lib/oeqa/manual/bsp-hw.json Traceback (most recent call last): [snip] File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 948 column 1 (char 39810) Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18fix various, innocuous typosRobert P. J. Day
Corrections: - environment - accommodate - conversion - compatible Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02bsp-hw.json: remove mpc8315e-rdb manual testsArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21/oeqa/manual/compliance-test: remove obsolete testArmin Kuster
ltpstress was removed in Oct 2018 https://github.com/linux-test-project/ltp/commit/e752f7c19674d9b2f7d37aed123561a3f6410e97#diff-5231627fc8640e0adb955f9e69c3c08d Remove LTP stress tests ltpstress.sh runs stress.part[1-3]. But these runtest files just duplicate definitions: * stress.part1: fs, mm, nfs * stress.part2: ipc, math, nptl * stress.part3: net.multicast, pty, syscalls The definitions are outdated anyway. There is no point trying to keep them sync. Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21oeqa/manual/compliance-test: remove crashme testsArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21manual/bsd-hw: remove bash testsArmin Kuster
done as part of IncompatibleLicensePerImageTests Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21oeqa/manual/bsp-hw: remove usb and SDmicro testsArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21oeqa/manual/bsp-hw: remove reboot testArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21oeqa/manual/bsp-hw: remove rpm -ivh testArmin Kuster
done via runtime def test_rpm_install(self): Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/oe-core: remove crosstab test from manualArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manua/oe-core: remove manual PACKAGECONFIG_FLAGS testsArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove manual useradd testArmin Kuster
useradd tested is done via: test_non_root_user_can_connect_via_ssh_without_password and test_non_root_user_can_connect_via_ssh_without_password Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove manual bash testArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove postinit test done w/selftest runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove only_one_connmand_in_background test done at ↵Armin Kuster
runtime Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: drop xserver test done at runtimeArmin Kuster
xserver testing is done at runtime and selftest via sato image Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove KVM enabled which is already done in selftest ↵Armin Kuster
runqemu Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/manual/bsp-qemu: remove rpm tests already done in runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06sdkext: use simpler kernel module for devtool testMark Asselstine
The current devtool test for the building of an out-of-tree kernel module uses something which requires several "high order" kconfigs to be set. This results in the test failing, not for expected reasons, but rather because it depends on specific kernel configuration. You will get error messages such as ERROR: "video_ioctl2" [.../1.0-r5/testsdkext/workspace/sources/v4l2loopback-driver/v4l2loopback.ko] undefined! ERROR: "video_unregister_device" [.../1.0-r5/testsdkext/workspace/sources/v4l2loopback-driver/v4l2loopback.ko] undefined! Using a simpler hello-world kernel module example will only require that CONFIG_MODULE is enabled, thus avoiding a false positive. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-29Remove LSB supportAdrian Bunk
LSB as a standard isn't current and isn't well suited to embedded anyway. Its putting artifical constraints on the system and with modern layer technology, would now be better off as its own layer. As such its time to split it out. The only part with some (marginal) usage is lsb_release, which is split from the lsb package into an own lsb-release package. Signed-off-by: Adrian Bunk <bunk@stusta.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-12oeqa/selftest: Automate manual pybootchart testsRichard Purdie
Automate the current manual pybootchart tests. This includes a check for the cairo dependency, skipping the test if appropriate. Based on original patch from Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29manual compliance: remove bits done at runtimeArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-19oeqa/manual/toaster: updated test id namingYeoh Ee Peng
All test id (eg. @alias) inside manual testcase file shall follow the same test id naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore, there shall be only 1 unique test_module per each manual testcases file, where test_module match the file name itself. This file was using test_module name that does not match the file name itself. Fixed test_module name as well as the test_suite name. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18oeqa/manual/toaster: transfer manual toaster test cases to oeqaYeoh Ee Peng
As part of the solution to replace Testopia, manual test cases need to be migrated to OEQA. These manual test case json files will serve two use cases. Use case#1: as input to the future commandline-based manual test runner script, where this script will display actions and expected result information in user friendly text, then it will capture user input for test result and log, finally it will write test result and log into existing standardize test result json format from OEQA framework for automated tests. Use case#2: QA will open and read these json file manually for planning manual test execution. Any reader interested in understanding manual test cases will open and read these files. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18oeqa/manual/kernel-dev.json: test id updatedsangeeta jain
All test id (eg. @alias) inside manual testcase file shall follow the same test id naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore, there shall be only 1 unique test_module per each manual testcases file. This file was using more than 1 unique test_module for testcases. Furthermore, some of the testcases were defined using different test_suite where it was not needed. This patch fix the manual testcases file to have only 1 unique test_module as well as test_suite to simplify test id naming. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18oeqa/manual/compliance-test.json: test id updated and missing test cases addedsangeeta jain
Two changes made in oeqa/manual/compliance-test.json: 1. All test id (eg. @alias) inside manual testcase file shall follow the same test id naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore, there shall be only 1 unique test_module per each manual testcases file. This file was using more than 1 unique test_module for testcases. Furthermore, some of the testcases were defined using different test_suite where it was not needed. This patch fix the manual testcases file to have only 1 unique test_module as well as test_suite to simplify test id naming. 2. Added 2 missing test cases for Beaglebone Stress Test. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18oeqa/manual/bsp-hw.json: test id updated and obsolete test cases removedsangeeta jain
Two changes made in oeqa/manual/bsp-hw.json: 1. All test id (eg. @alias) inside manual testcase file shall follow the same test id naming convention from oeqa automated tests (eg. selftest, runtime, sdk, etc), where the test id consists of <test_module>.<test_suite>.<test_function>. Furthermore, there shall be only 1 unique test_module per each manual testcases file. This file was using more than 1 unique test_module for testcases. Furthermore, some of the testcases were defined using different test_suite where it was not needed. This patch fix the manual testcases file to have only 1 unique test_module as well as test_suite to simplify test id naming. 2. As per review by Intel and Windriver team, 7 testcases were found obsolete. Removed 7 testcases. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-16oeqa/manual/bsp-hw.json: add non-IA testsMazliana
QA team found that 10 manual bsp test cases from Testopia for Beaglebone, EdgeRouter, and MPC need to be up streamed [YOCTO #12650] Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28oeqa/manual/sdk.json: Updated toolchain tarball to core-image-satosangeeta jain
Manual test step for SDK is updated. Previously toolchain was "poky-glibc-x86_64-core-image-sato-sdk<type-arch>-toolchain-<release-version>.sh" But toochain for core-image-sato-sdk is not available in releases after 2.1, hence changed it to "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh Other less significant update is to add exepcted result in intermediate test steps. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh" Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28oeqa/manual: Add manual runtime 'compliance' testcases to jsonSudhir Sharma
Added json file for compliance test cases to the manual runtime tests Signed-off-by: Sudhir Sharma <sudhirx.sharma@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/manual/bsp-qemu.json: Update for QEMU_USE_KVMRobert Yang
Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-13oeqa/manual/build-appliance: Drop test which is already automatedRichard Purdie
The nightly-build-appliance target already tests this particular build. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11oeqa/manual/oe-core: Drop image build during taskexp testRichard Purdie
Building an image takes time and isn't necessary to test taskexp, we can drop this step. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11oeqa/manual: Remove bitbake-selftest executionRichard Purdie
The autobuilder runs bitbake-selftest already so we don't need to execute this manually. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11oeqa/manual: transfer manual test cases from testopia to oeqaYeoh Ee Peng
As part of the solution to replace Testopia, manual test cases need to be migrated to OEQA. These manual test case json files will serve two use cases. Use case#1: as input to the future commandline-based manual test runner script, where this script will display actions and expected result information in user friendly text, then it will capture user input for test result and log, finally it will write test result and log into existing standardize test result json format from OEQA framework for automated tests. Use case#2: QA will open and read these json file manually for planning manual test execution. Any reader interested in understanding manual test cases will open and read these files. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>