aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2017-03-27oeqa/selftest: don't use cleanall in selftestsJoshua Lock
* devtool: oeqa.selftest.devtool.DevtoolTests is taking 4 or more hours on the autobuilder because the cleanall call deletes the tarball of the kernel source, forcing a large clone which is especially long on an NFS mount. The cleanall invocation isn't really required here and switching to clean instead should avoid hours of re-cloning the same repository each time we run oe-selftest on the autobuilder. * buildoptions: We only need to ensure the first image isn't an incremental image, clean will suffice here. * runtime-test: no need to clean up the artefacts generated during the test, let's leave that to a higher level process rather than using the cleanall hammer. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-26runtime-test.py: fix typoRobert Yang
roofs -> rootfs. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-26targetcontrol.py: use logger.info to replace of bb.noteRobert Yang
The bb.note prints multiple same lines when invoke this class again, but if we set mainlogger.propagate = False, nothing would be printed, according to logging's document: https://docs.python.org/3/library/logging.html Note If you attach a handler to a logger and one or more of its ancestors, it may emit the same record multiple times. In general, you should not need to attach a handler to more than one logger - if you just attach it to the appropriate logger which is highest in the logger hierarchy, then it will see all events logged by all descendant loggers, provided that their propagate setting is left set to True. A common scenario is to attach handlers only to the root logger, and to let propagation take care of the rest. We may need avoid using bb.note or bb.warn in oeqa since it attaches multiple log handlers which may cause confusions This patch only sets "mainlogger.propagate = False" in selftest/runqemu.py and use logger.info to replace bb.note in targetcontrol.py to minimize the impact. [YOCTO #10249] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-26selftest/runqemu.py: add it to test runqemuRobert Yang
Usage: $ oe-selftest -r runqemu Current test cases: $ runqemu nographic qemux86-64 $ runqemu nographic qemux86-64 ext4 $ runqemu nographic qemux86-64 iso $ runqemu nographic core-image-minimal $ runqemu nographic core-image-minimal vmdk $ runqemu nographic core-image-minimal vdi $ runqemu nographic tmp/deploy/images/qemux86-64 $ runqemu nographic tmp/deploy/images/qemux86-64 hddimg $ runqemu nographic qemux86-64 slirp $ runqemu nographic qemux86-64 slirp qcow2 $ runqemu nographic tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.qemuboot.conf $ runqemu nographic tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.ext4 Need more later: - Test initramfs - Test nfs - Test when set DEPLOY_DIR_IMAGE and OECORE_NATIVE_SYSROOT - And others which similate runqemu runs on SDK and eSDK. [YOCTO #10249] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22sign_rpm.bbclass: do not set/use RPM_GPG_PUBKEYAlexander Kanavin
This is entirely unnecessary (we can ask the signer backend to export the key to a file when needed), and was causing confusing selftest failures due to the variable being set from two different places. [YOCTO #11191] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22oe-selftest: add test_rawcopy_plugin test caseEd Bartosh
This test case builds wic image using rawcopy plugin running wic directly. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22selftest: pylinted wic test suite.Ed Bartosh
Fixed long lines, indentation and one unused variable. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22oe-selftest: test wic rawcopy pluginEd Bartosh
Added test_rawcopy_plugin_qemu test case and wks template. This test builds ext4 rootfs image, copies it to the root partition of the wic image using rawcopy plugin and boots wic image in qemu. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21oeqa/selftest/devtool: Enable missing cleanupRichard Purdie
Not sure why this cleanup is disabled but it clearly should be running so enable it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21wic: selftest: add tests for --fixed-size partition flagsMaciej Borzecki
wic has a new flag for setting a fixed parition size --fixed-size. Add tests that verify if partition is indeed sized properly and that errors are signaled when there is not enough space to fit partition data. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21wic: selftest: do not assume bzImage kernel imageMaciej Borzecki
Instead of assuming that bzImage is available, query bitbake enviroment for KERNEL_IMAGETYPE. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21wic: selftest: avoid COMPATIBLE_HOST issuesMaciej Borzecki
Wic tests will unconditionally attempt to build images that may require dependencies that are incompatible with current target. Resolve this by consulting HOST_ARCH first (which defaults to TARGET_ARCH) before proceeding to build images that may be incompatible. A convenience decorator only_for_arch() can be used to skip test cases for specific architectures. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21selftest: wic: replace directdisk with wictestdisk where possibleMaciej Borzecki
Use wictestdisk instead of directdisk thus allowing more tests to be run on non-x86 compatible machines. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21oeqa/selftest: remove test_sanity_unsafe_binary_referencesRoss Burton
This test was attempting to exercise a broken test, for some reason this broke with patches under review but investigation revealed that the test itself is broken. The test has been removed, so there's no need to test it. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21oe-selftest: tinfoil: add tests for recently enabled datastore operationsPaul Eggleton
A recent patch to bitbake fixes these datastore operations so that they actually affect the server end, so we should test that they work. (For full disclosure, some of these tests would probably pass without those fixes, since the operation would be done on the client side instead - but we are at least exercising the code paths.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-21selftest/devtool: Add test to verify "modify virtual/kernel"Jose Perez Carranza
The purpose of this test case is to verify that devtool modify works correctly when building the kernel. [YOCTO #10817] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-17selftest/signing.py: verify that images can be created from signed packagesAlexander Kanavin
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17rpm: add support for remote package feeds via PACKAGE_FEED_URIS variableAlexander Kanavin
I've used a previous patch (which was never merged) by Humberto Ibarra <humberto.ibarra.lopez@intel.com> as a model for how to do runtime testing of this feature (e.g. we need to boot an image, run dnf on it, and check that it is indeed able to access the remote repo over http). Here's his original commit message: ===== Testing that feeds specified with PACKAGE_FEED_URIS var are set correctly has two parts. First a build with this var set is required, and then smart update needs to be issued in the running taget. The previous is not a common selftest practice because this is a simple test, but requires building and running a specific image, which takes a lot of time. testimage is not a good fit either, since the images tested there do not have the PACKAGE_FEED_URIS var set. For this test, the runtime-test module is being used, which is a selftest module but runs a testimage command. The var and test environment were set in runtime-perf.py and the actual test is done in a new testcase added to meta-selftest layer. ===== [YOCTO #10872] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-16selftest: Disable SSTATE_MIRRORS for sstate signing testRichard Purdie
Building ed into an sstate mirror, then leaving it enabled for oe-selftest -r signing.Signing.test_signing_sstate_archive results in: NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Started WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Failed to fetch URL file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig, attempting MIRRORS if available ERROR: ed-1.14.1-r0 do_populate_lic_setscene: Fetcher failure: Unable to find file file://29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig;downloadfilename=29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz.sig anywhere. The paths that were searched were: /media/build1/poky/build/test-sstate /media/build1/poky/build/test-sstate WARNING: ed-1.14.1-r0 do_populate_lic_setscene: Cannot verify signature on sstate package /media/build1/poky/build/test-sstate/29/sstate:ed::1.14.1:r0::3:290bbc39f3b25cc27028dafce4b5aa48_populate_lic.tgz NOTE: recipe ed-1.14.1-r0: task do_populate_lic_setscene: Succeeded so we need to disable SSTATE_MIRRORS for this test. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14wic: selftest: account for occasional newline in debugfs file namesMaciej Borzecki
Debugfs output may contain a newline in file names in 'ls -p' output. Make sure that output is correctly split into lines by matching '/\n' and newlines are removed from file names. Fixes the following error appearing in AB tests: Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 388, in test_exclude_path files = [line.split('/')[5] for line in res.output.split('\n')] File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 388, in <listcomp> files = [line.split('/')[5] for line in res.output.split('\n')] IndexError: list index out of range Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13oe-selftest: no need to parse gpg output to determine the name of the ↵Alexander Kanavin
signing key The parsing fails on my machine, and we use a key with a hardcoded name, and so can provide it directly. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13gpg_sign.py: fix signing of rpm files using gpgAlexander Kanavin
This means a) calling rpmkeys and rpmsign instead of rpm b) instructing gpg to run non-interactively; otherwise on my machine it pops up windows requesting a key passphrase Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13selftest: update output checks in incremental image test to match what dnf ↵Alexander Kanavin
prints Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-13selftest: do not perform a full build in test_continueAlexander Kanavin
This was fetching and building the toolchain and everything else against empty download dir and sstate cache, and so was enormously slow. The test does not need that, it only checks that one fetch task fails and another succeeds when using bitbake's -k option. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-10lib/oeqa/selftest/bbtests: update test_non_gplv3 to use selftest-edJoshua Lock
GPLv2 recipes have been moved to a new layer (meta-gplv2), instead of readline perform this test on the selftest-ed recipe in meta-selftest which has gplv2 and gplv3 variants. Tested with oe-selftest -r bbtests.BitbakeTests.test_non_gplv3 Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-10lib/oeqa/selftest/oescripts: make test_cleanup_workdir use selftest-edJoshua Lock
Use a recipe which is bundled in the meta-selftest layer for this test, rather than relying on OE-Core remaining static (or updating the tests when OE-Core changes recipes). Tested with oe-selftest -r oescripts.TestScripts.test_cleanup_workdir Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08selftest: remove extra backslashes from debug outputEd Bartosh
Remove unneeded backslashes from the format strings that caused debug output to look confusing: 2017-03-06 16:52:42,428 - selftest.base - DEBUG - Removing from: ... \IMAGE_FSTYPES = "wic" WKS_FILE = "mkefidisk.wks" Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-08selftest: test wic efi image in qemuEd Bartosh
Added test_qemu_efi test case to wic test suite. It uses ovmf qemu extention to test mkefidisk image. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oeqa/selftest: Drop http sstate sharingRichard Purdie
Using httpServer from python for sharing sstate is known to be buggy, it can't cope with the number/type of requests coming from bitbake and quietly fails to share files. This causes intermittent build failures which are hard to debug. We can use a file:// url for the sstate mirror instead, removing the need for the http server. The sdk-update test is simply dropped since the SDK is never published to this location and hence it would never have any update. Its equiavalent to pointing at an empty web server. There is a better eSDK update test in testsdk so rather than improve this one, lets drop it and concentrate on the one there. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01oelib/buildhistory.py: Add unittest for buildhistory_analysisHumberto Ibarra
The buildhistory_analysis module (in which buildhistory-diff is based) was lacking unittest for its functions. Created selftest module for this and a few testcases to cover basic cases. [YOCTO #10727] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01selftest: Optimize get_bb_var useMariano Lopez
get_bb_var calls bitbake every time it is used and every call would take about 7 seconds. There are tests that calls get_bb_var several times when they can use get_bb_vars. Also there are tests that calls it to fetch the same variable over and over again. This will optimize the use of get_bb_var and get_bb_vars for a little speed up in the tests. [YOCTO #11037] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01selftest: Avoid sstate corruption by calling cleansstateMariano Lopez
Currently selftest doesn't use sstates because some tests clean sstate cache; using sstates would give a performance boost instead of building everything from scratch. With this sstates are not corrupted using different methods depending on tests: devtool: These tests needed to delete the cache so SSTATE_DIR as SSTATE_MIRRORS and set a temporal SSTATE_DIR. sstatetests: This module already used a temporal SSTATE_DIR, so just set up the SSTATE_MIRRORS. Rest: Removed cleansstate, some of them required to force a certain task, others were just removed or changed for another task. [YOCTO #10929] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23selftest/eSDK.py: Cleanup when there is an error in setUpClassMariano Lopez
Lately autobuilders are experiencing hangs with selftest, it seems it is cause if an error happens in setUpClass method of oeSDKExtSelfTest class because HTTP server keeps running in background. This patch will ensure tearDownClass will be run if there is an error in setUpClass. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-23oeqa: fix typoHumberto Ibarra
The word 'uninstall' was spelled incorrectly in various parts of oeqa files. (From OE-Core rev: af365e025030436f83b233fa51ecc9c58c58ce3c) Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-19selftest/containerimage.py: Add container IMAGE_FSTYPES testRandy Witt
This test checks to make sure only the files expected exist in a container image. Currently only ROOTFS_BOOTSTRAP_INSTALL, gets added to all images without the user specifying it. But this test should help if a developer in the future ever silently adds more than just ROOTFS_BOOTSTRAP_INSTALL, and that the developer can make sure it also gets removed from a container image. [YOCTO #9502] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-19oeqa/selftest/bbtests: Update after changes to dependency graph file generationRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15selftest/wic: Add tests for --exclude-dir option.Kristian Amlie
Based partially on an earlier patch by Maciej Borzecki. Note that because tools now reside in recipe specific sysroots, we need to import the path from bitbake and apply it during the test. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-07oeqa/selftest/pkgdata: use m4 instead of bashRoss Burton
The pkgdata test broke due to oe-core cf0fd8b which added alternatives support to bash, so the binary is now /bin/bash.bash. Instead of fixing the test case for this new name, use the m4 recipe instead as it has less build dependencies. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05sdkext/cases oeqa/selftest: Updates decoratorsFrancisco Pedraza
Updates decorators for eSDK in test cases at: sdkext/cases/devtool.py and oeqa/selftest/eSDK.py Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05selftest: wic: stop using iso image typeEd Bartosh
Removed 'IMAGE_FSTYPES = "iso" as this functionality depends on do_bootimg, which is going to be obsoleted soon. As wic doesn't depend on bootimg it's safe to remove this. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05selftest: wic: use wic-tools recipe to get STAGING_DIREd Bartosh
STAGING_DIR variable is used to get path to a boot dir. It's better to use wic-tools recipe to it as it contains all bootloader artifacts. Modified test_build_artifacts and test_rootfs_artifacts to use wic-tools target to get STAGING_DIR. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02selftest/runtime-tests: Modify positins test to reduce execution timeJose Perez Carranza
Currently test_postinst_roofs_and_boot is building a full-cmdline image this is taking a lot of time to execute the test, so a minimal image will be build instead. Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02selftest/bbtests: use write_config instead of local.conf fileLeonardo Sandoval
Extra configuration data should be write using the write_config method instead of manually appending to the local.conf file Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02selftest: wic: stop using hddimg in FSTYPESEd Bartosh
Removed hddimg from FSTYPES in wic test suite as wic doesn't depend on hddimg anymore. [YOCTO #10835] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31selftest: wic: fix test_iso_image test caseEd Bartosh
Added "iso" to IMAGE_FSTYPES to build iso artifacts required to fix test of isoimage-isohybrid wic plugin. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31selftest: stop using hddimg in the wic test suiteEd Bartosh
Removed hddimg from IMAGE_FEATURES as wic code doesn't use hddimg anymore. [YOCTO #10835] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: Look for image artifacts in a common locationTom Zanussi
Rather than have each image type look for artifacts in image-specific locations, move towards having them look for artifacts in a common location, in this case DEPLOY_DIR_IMAGE Use the existing deploy.bbclass to have the bootloaders put their binaries in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2017-01-31image-wic: remove HDDDIR from WICVARSEd Bartosh
Removed HDDDIR as it's not used by wic anymore. Stopped usage of HDDDIR in wic test suite. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: use INITRD_LIVE in isoimage-isohybridEd Bartosh
INITRD variable is not set if hddimg is disabled. isoimage-isohybrid can't get correct name for initrd if INITRD variable is not set. Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31selftest: check results in test_image_vars_dir*Ed Bartosh
Tested that result images are produced in test_image_vars_dir_long and test_image_vars_dir_short test cases. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-25 00:59:08 +0000