aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2019-04-09devtool: Split tests into multiple classesstable/sumo-nmutstable/sumo-nextRichard Purdie
This allows better parallelism between the different tests as currently this block takes the longest time to execute. devtool tests are still all grouped into the "devtool" module for ease of exection. This also makes it easier to execute some subset of devtool tests for testing devtool changes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-09oeqa: Default to buffer mode for testsRichard Purdie
Currently some tests run in buffer mode and some don't. Those that don't can corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent. If there is useful output on stdout/stderr, it will be displayed if the test fails. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-09oeqa/selftest/devtool: Ensure dbus is built befoe running testRichard Purdie
If dbus isn't build first the dbus dependency of dbus-wait can't be detected through pkgconfig and the test fails: AssertionError: {'DEPENDS': {'dbus'}} != {} - {'DEPENDS': {'dbus'}} + {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}} Ensure dbus is built and present in the sysroot. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-09logging: use warning instead warnChen Qi
The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-07pkgdata.py: avoid target-sdk-dummy-provides to mess things upChen Qi
Sometimes we meet the following failure for the test_lookup_recipe test case. AssertionError: 'zlib\nbusybox is in the RPROVIDES of target-sdk-provid[32 chars]ummy' != 'zlib\nbusybox' zlib + busybox- busybox is in the RPROVIDES of target-sdk-provides-dummy: - target-sdk-provides-dummy This is because target-sdk-provides-dummy rprovides busybox. So clean things up to avoid failure. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-03-25resulttool: Improvements to allow integration to the autobuilderRichard Purdie
This is a combined patch of the various tweaks and improvements I made to resulttool: * Avoid subprocess.run() as its a python 3.6 feature and we have autobuilder workers with 3.5. * Avoid python keywords as variable names * Simplify dict accesses using .get() * Rename resultsutils -> resultutils to match the resultstool -> resulttool rename * Formalised the handling of "file_name" to "TESTSERIES" which the code will now add into the json configuration data if its not present, based on the directory name. * When we don't have failed test cases, print something saying so instead of an empty table * Tweak the table headers in the report to be more readable (reference "Test Series" instead if file_id and ID instead of results_id) * Improve/simplify the max string length handling * Merge the counts and percentage data into one table in the report since printing two reports of the same data confuses the user * Removed the confusing header in the regression report * Show matches, then regressions, then unmatched runs in the regression report, also remove chatting unneeded output * Try harder to "pair" up matching configurations to reduce noise in the regressions report * Abstracted the "mapping" table concept used to pairing in the regression code to general code in resultutils * Created multiple mappings for results analysis, results storage and 'flattening' results data in a merge * Simplify the merge command to take a source and a destination, letting the destination be a directory or a file, removing the need for an output directory parameter * Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression mappings * Have the store command place the testresults files in a layout from the mapping, making commits into the git repo for results storage more useful for simple comparison purposes * Set the oe-git-archive tag format appropriately for oeqa results storage (and simplify the commit messages closer to their defaults) * Fix oe-git-archive to use the commit/branch data from the results file * Cleaned up the command option help to match other changes * Follow the model of git branch/tag processing used by oe-build-perf-report and use that to read the data using git show to avoid branch change * Add ptest summary to the report command * Update the tests to match the above changes Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-25resulttool: enable merge, store, report and regression analysisYeoh Ee Peng
OEQA outputs test results into json files and these files were archived by Autobuilder during QA releases. Example: each oe-selftest run by Autobuilder for different host distro generate a testresults.json file. These scripts were developed as a test result tools to manage these testresults.json file. Using the "store" operation, user can store multiple testresults.json files as well as the pre-configured directories used to hold those files. Using the "merge" operation, user can merge multiple testresults.json files to a target file. Using the "report" operation, user can view the test result summary for all available testresults.json files inside a ordinary directory or a git repository. Using the "regression-file" operation, user can perform regression analysis on testresults.json files specified. Using the "regression-dir" and "regression-git" operations, user can perform regression analysis on directory and git accordingly. These resulttool operations expect the testresults.json file to use the json format below. { "<testresult_1>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, ... "<testresult_n>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, } To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To store test result from oeqa automated tests, execute the below $ resulttool store <source_dir> <git_branch> To merge multiple testresults.json files, execute the below $ resulttool merge <base_result_file> <target_result_file> To report test report, execute the below $ resulttool report <source_dir> To perform regression file analysis, execute the below $ resulttool regression-file <base_result_file> <target_result_file> To perform regression dir analysis, execute the below $ resulttool regression-dir <base_result_dir> <target_result_dir> To perform regression git analysis, execute the below $ resulttool regression-git <source_dir> <base_branch> <target_branch> [YOCTO# 13012] [YOCTO# 12654] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-selftest: devtool: Support meta being a symbolic linkPeter Kjellerstedt
oe-selftest's devtool tests have been broken since commit 2457cd57 (oe-selftest: devtool: avoid parallel races by using temporary copy of core) if meta is a symbolic link. (From OE-Core rev: daba6c5a991b370709d17e51305334f55a3858ec) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-02-25oe-selftest: devtool: avoid parallel races by using temporary copy of corePaul Eggleton
Some of the devtool tests make changes to files under meta/ - legitimately since we want these tests to be working with real recipes and associated files. Unfortunately with the new oe-selftest parallelisation this can break other tests if files go missing at the wrong time (among other scenarios). To avoid this issue, simply take a copy of the core repository and use that for these tests. (We copy the entire repository since changing the path of meta/ influences COREBASE and thus we need to have things like scripts/ alongside as well). (From OE-Core rev: 2457cd57b4195924ef127f497efa2f34f411e660) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-02-25oeqa/selftest/runqemu: Enable kvm when QEMU_USE_KVM is setRobert Yang
(From OE-Core rev: 564de3681353fe8e203425388e8be9703a89d2da) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-02-25oeqa: replace Alex Kanavin's @intel email address with a personal oneAlexander Kanavin
As I will be leaving Intel, this address will no longer be valid, so swap it for my personal one for now. (From OE-Core rev: ea58ff101e28dfda3410de66d775df3d8a1e5a96) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-02-25testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation ↵Richard Purdie
warnings Use our own lsb function instead as used elsewhere by the codebase. (From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3) (From OE-Core rev: 570256a64af5a3fa994a20a5cc4c74d59ffc361f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/esdk: Fix typo causing test failureRichard Purdie
2018-12-06 23:19:24,564 - oe-selftest - INFO - Traceback (most recent call last): File "/media/build1/poky-sumo/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/media/build1/poky-sumo/meta/lib/oeqa/selftest/cases/eSDK.py", line 78, in setUpClass cls.tmpdir_eSDKQA = cls.tempdirobj.name AttributeError: type object 'oeSDKExtSelfTest' has no attribute 'tempdirobj' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/esdk: Ensure parent directory existsRichard Purdie
INFO - ====================================================================== INFO - ERROR: setUpClass (eSDK.oeSDKExtSelfTest) INFO - ---------------------------------------------------------------------- INFO - Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass clss.setUpClassMethod() File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/eSDK.py", line 76, in setUpClass cls.tmpdirobj = tempfile.TemporaryDirectory(prefix="selftest-esdk-", dir=bb_vars["WORKDIR"]) File "/usr/lib/python3.5/tempfile.py", line 929, in __init__ self.name = mkdtemp(suffix, prefix, dir) File "/usr/lib/python3.5/tempfile.py", line 507, in mkdtemp _os.mkdir(file, 0o700) FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-debian/build/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-q7ln84gc' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06image-buildinfo,oeqa/selftest/containerimage: Ensure image-buildinfo doesn't ↵Richard Purdie
break tests Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files to fail due to the presence of an unexpected file: ['./', './etc/', - './etc/build', './etc/default/', './etc/default/postinst', Tweak the class to allow it to be disabled and disable it from the test just in case it was enabled. (From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest: Standardize json logging output directoryYeoh Ee Peng
Currently sdk & sdkext will output json file to LOG_DIR, while selftest will output json file to TOPDIR/log. Standardize selftest json output file to LOG_DIR. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest: Improvements to the json loggingRichard Purdie
Tweak the preceeding commit to: * Add STARTTIME to the identifier to make it unique * Use LOG_DIR * Store the layer config in a more natural json format * Drop '_' function prefixes (From OE-Core rev: 173f59acf9722e2ef27fdd49c20f7d3d664917eb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/context: write testresult to json filesYeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. By default, json testresult file will be written to "oeqa" directory under the oe-selftest log directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/context: Improve log file handlingRichard Purdie
The existing logfile is simply placed in the current directory. Since the test changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid directory. We also see trackbacks if the symlink is invalid. Improve things by: * Placing logs in LOG_DIR (or BUILDDIR if unset). * Using a full path to the log meaning the log and link are placed in the same directory. * Using lexists instead of exists so invalid symlinks are handled correctly. (From OE-Core rev: 750ece11bed0e62a11e0003d1d16a81f7c219761) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/context: Replace deprecated imp module usageRichard Purdie
Avoid the warning: meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp In this case importlib is a direct replacement. (From OE-Core rev: db7a60c36a2d3eefc61ae6e1ede01680dc932035) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/esdk: run selftest inside workdir not /tmpRoss Burton
We've seen issues with rootfs size calculations and we've seen systems like opensuse which have btrfs mounted on /tmp causing selftest failures. (From OE-Core rev: 61be3cd748d1b7321a1fc4cfe84efa9b26a6aee0) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/runqemu: Improve testcase failure handlingRichard Purdie
assertTrue doesn't give good debug information when things fail. Update several to use assertIn which gives information upon failure, for the others print the log information upon failure. (From OE-Core rev: c29cb75d5ce6b0873a934f4709b0c8824f7164d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/oelib/path: don't leak temporary directoriesRoss Burton
setUp() is used to populate a directory of temporary files, and deleted in __del__. However setUp() is called once *per test* so __del__ would only be able to remove the last directory created. Fix the code by using the natural counterpart to setUp, tearDown(), to clean up. (From OE-Core rev: 68b4723e6fb11d171869185bccf28f32f6284c18) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/buildoptions: Ensure diskmon tests run consistentlyRichard Purdie
Heartbeat events default to once a second and we need to ensure we have enough time in the task to see them. Add a nostamp delay task 5s long so we can have a consistently timed task which doesn't need cleanup or have unneeded dependencies. This ensures we should deterministically see the disk moinitor events regardless of the state of the build. This is done in a way which doesn't corrupt build state or need cleanup and is efficient. (From OE-Core rev: ecc49ee8986929e2429d948000a0ca588fe63959) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/buildoptions: Improve ccache testRichard Purdie
This test occisionally fails as m4 doesn't recompile, meaning the logfile test then doesn't find mention of ccache. To ensure m4 does recompile, clean m4 before force compiling it. (Reading the test is confusing due to the test cleanup also involving a clean) (From OE-Core rev: 6e0b9214a0d57ed45a5df0ba5c9887a9045b89b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/buildoptions: Improve ccache test failure outputRichard Purdie
The current failure mode doesn't show us what the logs actually looked like and later cleans can lose them. Show the whole log in case of failure to aid debugging intermittent problems on the autobuilder. (From OE-Core rev: 7c3a0dc5978cea898b1ca51decf4d6e7cf9d519f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/case: Use bb.utils.remove() instead of shutil.remove()Richard Purdie
This avoids problems where shutil.remove will error with: File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra' when there are races over file deletion (gpg agent may be slow to exit). We already worked around speed and race issues in bb.utils. (From OE-Core rev: 00a8fd5b93a5c19ce0b7498e2bc653ce8ad58aaf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/signing: Use do_populate_lic target instead of do_packageRichard Purdie
This should speed the test up signficiantly without any loss of functionality for the purposes of the test. (From OE-Core rev: 3dde0b749643575878bfbca2f8d2d9ec30bad166) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/signing: Allow tests not to need gpg on the hostRichard Purdie
We ideally don't want to use gpg from the host. This is straightforward for package management but not for sstate. For sstate, create a second build directory to run the test in using gnupg-native from the original build directory. (From OE-Core rev: 10afa94c3f0d7eb7524a26deda86949073d55fde) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-06oeqa/selftest/signing: Skip tests if gpg isn't foundRichard Purdie
Raising an assertionError in the class setup isn't a particuarly good way to indicate gpg isn't installed. Instead skip the tests if the required binary isn't present. For the signing tests we do require it to be present and can't use a prebuilt one. (From OE-Core rev: 2d486af97e51b9daa9c40482c31d637c9ab4ae79) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-24oeqa/selftest/recipetool: Fix problems from changing upstream sourceMohamad Noor Alim Hussin
The upstream source tarball checksums changed. Use the copy from our source mirror to avoid failures. [YOCTO #12979] (From OE-Core rev: e97a31e6bbaec5cb56d4750bf5171dbba510ee33) Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-11-24selftest/wic: Improve error message for test_fixed_sizeRichard Purdie
Currently this can fail with a message like 127 != 0 which is unhelpful. If we remove the ignore_status=False, the debugging from runCmd is much more helpful printing status.output. Also remove the now unneeded exit code check. (From OE-Core rev: 1aa7471b11aedc68de5116c461fe73152e3985fd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-11-24oeqa/selftest/wic: Ensure initramfs exists for test_iso_imageRichard Purdie
AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal -o /var/tmp/wic.oe-selftest/' returned non-zero exit status 1: ERROR: _exec_cmd: gzip -f -9 -c /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio > /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio.gz returned '1' instead of 0 output: gzip: /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio: No such file or directory This is because in a clean build directory, the initramfs may not be rebuilt. Add a call to ensure it is built to avoid the error. (From OE-Core rev: 2a80fa234d31992691a157425e8990db30158fd1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-11-24oeqa/selftest/wic: Use a subdir of builddir, not /var/Richard Purdie
Using /var/ leave wic open to races with other processes on the system, use a subdir of builddir instead to avoid this. (From OE-Core rev: e07ec908ce7f26143a7bdf0a07a1230c0fd6ac87) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-11-16oeqa/selftest/runtime_test: Ensure we build/use gnupg-nativeRichard Purdie
Without this, we see errors if gpg is missing from the host system for "oe-selftest -r runtime_test.TestImage.test_testimage_dnf". (From OE-Core rev: e91838b63b506e2969582b2b8511fd3724d6aa3f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2018-04-20oeqa/selftest/runqemu: qemu nfs testing not ready for deploymentRichard Purdie
This test shouldn't have merged yet since we don't run portmap/rpcbind on the autobuilder infrastructure and the test therefore cannot succeed. We need to document this, set it up, then enable the test. The test itself is fine and good to have so its left in the code but disabled for now. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-19oe-selftest: runqemu: add tests for qemu boot and shutdownYeoh Ee Peng
QA team were testing qemu boot image and shutdown on each qemu architecture manually. Add automated test to test qemu boot on ext4 and nfs, finally check that it can shutdown properly. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-04-07devtool: Ensure added layer sets LAYERSERIES_COMPATRichard Purdie
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool needs to set this to avoid warnings which break various tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-12oe-selftest: add a test for failing package post-installation scriptletsAlexander Kanavin
The test runs a scriptlet that has an intentionally failing command in the middle and checks for two things: 1) that bitbake does warn the user about the failure 2) that scriptlet execution stops at that point. The test is run for all three package types: rpm, deb, ipk. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-06selftests: Add test case for booting a generic EFI boot partition imageCalifornia Sullivan
Simple test case that adds 'efi' to MACHINE_FEATURES, sets WKS_FILE to "efi-bootdisk.wks.in", installed required boot items, and attempts to boot the wic image. Quick check to make sure that the feature actually works. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-24sstatetests: pass the right files when comparing sigfilesRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: add a test for recipes without maintainersAlexander Kanavin
'bitbake -c checkpkg world' is moved to class initializer to avoid it being run twice in a row. The no-maintainers test checks only oe-core recipes, as other layers may be be configured, and assigning maintainership to specific people via maintainers.inc is known to be used only in oe-core. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: fix tests to reflect man->man-db transitionAlexander Kanavin
Rather than apply a patch twice, an incorrect patch is applied (as the new man-db recipe does not have any patches yet). Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-09oe-selftest: meta_ide: add tests for meta-ide-supportYeoh Ee Peng
QA team were testing meta-ide-support manually. Add automated tests to test that bibtake meta-ide-support will create the toolchain and environment setup script. Also test that after using environment setup script, one can compile c program and build cpio project. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-23imagefeatures: disable f2fs from test_image_features by defaultSaul Wold
Since the primary f2fs utilities are provided by the meta-openembedded meta-filesystems layer, we disable the testing of that functionality here. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-14selftest/cases/devtool.py: fix workspace layer checkingRobert Yang
Fixed: $ oe-selftest -r devtool.DevtoolTests.test_create_workspace [snip] 2017-12-20 18:28:59,404 - oe-selftest - INFO - Traceback (most recent call last): File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/workspace2/lyang1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 177, in test_create_workspace self.assertTrue('/workspace' not in result.output, 'This test cannot be run with a workspace layer in bblayers.conf') AssertionError: False is not true : This test cannot be run with a workspace layer in bblayers.conf [snip] $ bitbake-layers show-layers NOTE: Starting bitbake server... layer path priority ========================================================================== meta /workspace2/lyang1/poky/meta 5 meta-poky /workspace2/lyang1/poky/meta-poky 5 meta-yocto-bsp /workspace2/lyang1/poky/meta-yocto-bsp 5 meta-selftest /workspace2/lyang1/poky/meta-selftest 5 There is no workspace layer, but I'm in /workspace2, this patch can fix the problem. [YOCTO #12442] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-06bitbake.conf: Set AUTOREV to have a vardepvalueRichard Purdie
If you have a recipe which does not include SRCPV in PV but does set SRCREV = "${AUTOREV}" and you run do_fetch, then change the repo to a new commit then run do_unpack, do_unpack will fail since the new commit doesn't exist in the repo that was fetched. The problem is the revision chosen is not represented in the do_fetch task hash. It if were, the fetch would rerun first and the commit would be present. It works when PV includes SRCPV since that does contain the chosen commit from the AUTOREV. The solution is to include the SRCPV value into the representation of AUTOREV used for checksum calculation purposes. Add a selftest for this issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-02oe-selftest: devtool: switch away from mkelfimagePaul Eggleton
mkelfimage is about to be removed, so test_devtool_modify_git needs to use a different recipe. psplash is a reasonable choice given it uses a git repository (and probably will forever), and doesn't have too many dependencies, so change the test to use that recipe instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-21rpm: update to 4.14.0Alexander Kanavin
Previously oe-core had a development snapshot of rpm, it's better to update to something more stable. Removed patches: 0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch (upstream is using pkg-config) 0001-When-nice-value-cannot-be-reset-issue-a-notice-inste.patch (functionality has been moved to a plugin, we disable plugins for rpm-native) 0012-Use-conditional-to-access-_docdir-in-macros.in.patch (merged upstream) Changed patches: 0001-Fix-build-with-musl-C-library.patch (one previous musl issue has been resolved upstream; another has been added) Rest of the patches are trivial rebases. Update the signing oe-selftest so that the reference output matches the upstream changes. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-21oeqa/selftest/cases/wic: update OETestID decorator for test_qemuJoshua Lock
Test case ID 1422 is not correct for this issue, the correct test case ID for test_qemu is 1424. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>