aboutsummaryrefslogtreecommitdiffstats
path: root/meta-selftest
AgeCommit message (Collapse)Author
2017-08-18oeqa: increase verbosity of dnf commands in dnf packagefeed testMarkus Lehtonen
Makes diagnosing failures easier. [YOCTO #11209] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16dnf: expand dnf selftest to test signed package feedsMarkus Lehtonen
[YOCTO #11209] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16oeqa: fix temp file handling in dnf package feed testMarkus Lehtonen
Prevent stale temp files and a possible (if unlikely) race in tempfile usage. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16oeqa: fix dnf testsMarkus Lehtonen
Rename one dnf runtime test that it will recognized as a python module and thus also found by the oe test loader. Also, fix value of TEST_SUITES in dnf selftest so that all test dependencies are satisfied and the runtime test may be successfully run from there. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16selftest: fix distrodata.py to use per-recipe UPSTREAM_VERSION_UNKNOWN settingAlexander Kanavin
... instead of a global exception list which was problematic. [YOCTO #11896] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09checkpkg_exceptions: add LLVMRoss Burton
The git mirror of LLVM doesn't have any tags we can track for release announcements, so exclude llvm from the checkpkg test. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-07-17selftest: Add vulkan-demos to checkpkg exceptionsJussi Kukkonen
vulkan-demos has not had any releases. Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-28selftest/seltest.py: Add test to check imports from other layersJose Perez Carranza
This tests adds a check of selftest itself to verify if can add test from other layers. [YOCTO #9770] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-28oe-selftest: add a test for upstream version check regressionsAlexander Kanavin
The test runs an upstream version check, and then compares the list of recipes that failed the check (i.e. those where latest upstream version could not be established) against the list of known-broken upstreams. Mismatches either way (upstream check failed, recipe not in the exception list, or upstream check worked, recipe in exception list) fail the test. [YOCTO #11031] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-06-09meta*: Add LAYERSERIES_COMPAT and LAYERSERIES_CORENAMES markup to layer.confRichard Purdie
This means mismatched layers are more clearly identified to the user in cases where compatibility has not been tested. This is perhaps not as needed for the core repository (other than CORENAMES) but lets lead by example. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-09runtime-test.py: Split different tests inside selftest.pyHumberto Ibarra
There were two completely different tests inside selftest.py, and the only reason for them to be together was that both needed the same mechanism to execute (run testimage from within selftest) This fixes the design issue and error-prone situation by separating these tests. In add a new module for dnf-runtime and also has an extra time added to running the tests, but it is minimal since the builds reuse data from previuous build. [YOCTO #11436] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-12wic-image-minimal: stop using core-image-minimalEd Bartosh
Removed dependency to core-image-minimal rootfs from the recipe and .wks file. This was a bad idea as recipes are not allowed to use other recipe's rootfs or anything else from workdir. This should also fix test_qemu test case when rm_work is enabled. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11wic-image-minimal.wks: vda -> sdaRobert Yang
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and would use vda if no grep result, now we have set QB_DRIVE_TYPE to "/dev/sd" by default, and the device will be /dev/sda, so use sda to replace vda in the test case. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22oe-selftest: test wic rawcopy pluginEd Bartosh
Added test_rawcopy_plugin_qemu test case and wks template. This test builds ext4 rootfs image, copies it to the root partition of the wic image using rawcopy plugin and boots wic image in qemu. [YOCTO #10618] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21selftest: wictestdisk: machine agnostic WKS for use with selftestMaciej Borzecki
Add a kickstart for an image that is not limited to use on x86 compatible machines. This allows us to run more wic tests on non-x86 architectures. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-17rpm: add support for remote package feeds via PACKAGE_FEED_URIS variableAlexander Kanavin
I've used a previous patch (which was never merged) by Humberto Ibarra <humberto.ibarra.lopez@intel.com> as a model for how to do runtime testing of this feature (e.g. we need to boot an image, run dnf on it, and check that it is indeed able to access the remote repo over http). Here's his original commit message: ===== Testing that feeds specified with PACKAGE_FEED_URIS var are set correctly has two parts. First a build with this var set is required, and then smart update needs to be issued in the running taget. The previous is not a common selftest practice because this is a simple test, but requires building and running a specific image, which takes a lot of time. testimage is not a good fit either, since the images tested there do not have the PACKAGE_FEED_URIS var set. For this test, the runtime-test module is being used, which is a selftest module but runs a testimage command. The var and test environment were set in runtime-perf.py and the actual test is done in a new testcase added to meta-selftest layer. ===== [YOCTO #10872] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13run-postinsts: simplify the logic of whether to install it to imagesAlexander Kanavin
The logic is scattered all over the place, but amounts to "install, unless the rootfs is read only". Let's express that directly. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
2017-03-10meta-selftest: add selftest-ed recipesJoshua Lock
The oe-selftest oescripts.TestScripts.test_cleanup_workdir was using gzip and the GPLv2 variant to test cleanup of the workdir. This broke with the removal of GPLv2 recipes from OE-Core. Instead of relying on recipes in OE-Core remaining static we should ensure that meta-selftest provides recipes required for the tests to pass. To that end we take a copy of the current GPLv2 and GPLv3 variants of ed and include them in meta-selftest as new recipes. We chose ed over gzip as gzip has dependencies which would require additional GPLv2 recipes to be included in meta-selftest. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-04wic: add WKS_FILE_DEPENDS variableEd Bartosh
This variable should be used to specify dependencies that will be added to DEPENDS only if wic image build is 'active' i.e. when 'wic' is in IMAGE_FSTYPES. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-23oeqa: fix typoHumberto Ibarra
The word 'uninstall' was spelled incorrectly in various parts of oeqa files. (From OE-Core rev: af365e025030436f83b233fa51ecc9c58c58ce3c) Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-19selftest/containerimage.py: Add container IMAGE_FSTYPES testRandy Witt
This test checks to make sure only the files expected exist in a container image. Currently only ROOTFS_BOOTSTRAP_INSTALL, gets added to all images without the user specifying it. But this test should help if a developer in the future ever silently adds more than just ROOTFS_BOOTSTRAP_INSTALL, and that the developer can make sure it also gets removed from a container image. [YOCTO #9502] Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-15m4: use wildcard for selftest bbappendRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23selftest/runtime-test.py: Adapt test to use new runtime frameworkMariano Lopez
This adapt the current runtime selftest to use the new runtime framework. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-16recipes-test: exclude recipes from world targetLeonardo Sandoval
These recipes should be excluded from target 'world' because these are just intended to be used internally by oe-selftest (devtool, recipetool, etc.) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16postinst: Add a test case to verify postinst scripts behaviorJose Perez Carranza
Add test case that verify behavior of postinst scripts at roofts time and when is delayed to the first boot directly on the target. Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16selftest: Test needed to verify postinst orderFrancisco Pedraza
It verifies the following: 1. Compile a minimal image. 2. The compiled image will add the layer with the recipe postinst, previously created at: "meta-selftest/recipes-test" 3. Run QEMU. 4. Validate the task execution order. [YOCTO #5319] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13recipetool: selftest: Add test for recipetool plugin loadingOla x Nilsson
Test that recipetool plugins are loaded in a well defined order. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13devtool: selftest: add test for devtool plugin loadingOla x Nilsson
Test that devtool plugins are loaded in a well defined order. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23oe-selftest: devtool: test that updating a file with subdir= worksPaul Eggleton
If you have a file:// entry in SRC_URI with a subdir= parameter that makes it extract into the source tree, then when you update that file in oe-local-files and run devtool update-recipe then you want the original file to be updated. This was made to work by OE-Core commit 9069fef5dad5a873c8a8f720f7bcbc7625556309 together with 31f1bbad248c36a8c86dde4ff57ce42efc664082, however until now there was no oe-selftest test to verify it. Note that in order to succeed this test also requires the fix "lib/oe/recipeutils: ignore archives by default in get_recipe_local_files()" since the test recipe uses a local tarball. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23devtool: update-recipe: fix handling of compressed local patchesPaul Eggleton
It is possible to use gzip or bzip2 to compress patches and still refer to them in compressed form in the SRC_URI value within a recipe. If you run "devtool modify" on such a recipe, make changes to the commit for the patch and then run devtool update-recipe, we need to correctly associate the commit back to the compressed patch file and re-compress the patch, neither of which we were doing previously. Additionally, add an oe-selftest test to ensure this doesn't regress in future. Fixes [YOCTO #8278]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-11-23oe-selftest: devtool: test update-recipe with only local filesPaul Eggleton
Add a test to ensure devtool update-recipe works properly on recipes that contain only local files (since the other tests we have didn't test that). Relates to [YOCTO #10563]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-28Remove LIC_FILES_CHKSUM from recipes without SRC_URIOlaf Mandel
LICENSE and LIC_FILES_CHKSUM apply to the sources specified by SRC_URI, not to the recipe itself. As such a license declaration for a source-less recipe makes little sense. The LICENSE declaration is mandatory, but LIC_FILES_CHKSUM can be removed in such cases. Remove the LIC_FILES_CHKSUM declarations from all recipes that do not need it. CC: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15Remove RM_OLD_IMAGE, it's no longer usefulJoshua Lock
Since the move to put image deployment under sstate control in d54339d4b1a7e884de636f6325ca60409ebd95ff old images are automatically removed before a new image is deployed (the default behaviour of the sstate logic). RM_OLD_IMAGE is therefore no longer required to provide this behaviour, remove the variable and its users. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-09-20image_types: add parted-native to do_image_wic dependsEd Bartosh
As parted is always used by wic it makes sense to make do_image_wic dependent on parted-native:do_populate_sysroot. This should help to avoid adding it to all wic image recipes. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-04selftest/runtime-test.py: Add test for testexport SDK featureMariano Lopez
This adds test_testexport_sdk() to test the SDK feature of testexport in the CI in order to avoid breaking it. [YOCTO #9765] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-08-01selftest/runtime-test.py: Add test for import test from other layersMariano Lopez
There are some features in testimage/testexport that are not tested; this might lead to break some of these features without notice. This adds a new test in order to test two features of testimage: - Import test from other layers. - Install/Unistall in the DUT without a package manager. [YOCTO #9764] [YOCTO #9766] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-05-21meta/selftest/scripts: Use print function for python3 compatibilityRichard Purdie
Used print function instead of print statement to make the code work in python 3. [Changes from both Ed and Richard] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11Drop unneeded LIC_FILES_CHKSUM valuesPaul Eggleton
With the changes to insane.bbclass we don't need LIC_FILES_CHKSUM to be set for recipes that don't actually pull in any source. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06test-empty-image: Fix LIC_FILES_CHKSUM typoRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06meta-selftest/images: Add LIC_FILES_CHKSUM to images using image.bbclassRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28image.bbclass: run wicenv task only for wic imagesEd Bartosh
Currently do_wicenv task is run for all images. However, its result is used only to produce wic image. It's better to run this task only for wic images. If another rootfs is required to produce wic image, dependency to its do_wicenv must be added to the wic image recipy. Stopped running do_wicenv for all images. Added explicit dependency to this task in wic-image-minimal recipe. [YOCTO #9095] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23selftest/signing.py: RPM_GPG_PASSPHRASE_FILE -> RPM_GPG_PASSPHRASERobert Yang
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09xcursor-transparent-theme: use a version glob in the selftest bbappendRoss Burton
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28oe-selftest: devtool: add an additional test for devtool upgradePaul Eggleton
Add a test for devtool upgrade with a recipe pointing to a git repository, since this uses several different code paths. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28oe-selftest: devtool: rework devtool upgrade testPaul Eggleton
* Use a more real-world test of a recipe pointing to a remote file * The cleanup tracking / teardown commands need to be added towards the top, or they won't have the desired effect of cleaning up if the test fails. * Check that a versioned subdirectory gets renamed to match the new version * Ensure the recipe contents gets changed as we expect it to * Check that the recipe directory is deleted by devtool reset at the end Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-15wic-image-minimal: use uuid for root partitionEd Bartosh
Used --uuid option to generate PARTUUID for root partion and gpt partition type. As this image is used in runtime testing under qemu it makes sense to ensure that this way of specifying root partition works. Added dependency to gptfdisk-native to wic-image-minimal recipe to be able to create gpt partition table. (From OE-Core rev: ba82867bfda54e86e1de5dc4a6ad7246254cf5c3) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15wic-image-minimal: update .wks to boot by qemuEd Bartosh
With below list of changes wic-image-minimal can be boot by qemu: - Changed device name sda -> vda. - Added root=/dev/vda2 to the kernel command line. - Changed mount point for core partition to /mnt to have it mounted on boot time. (From OE-Core rev: f085274711f561f97a2c89f40bb0e89094973054) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15wic-image-minimal: change IMAGE_FSTYPESEd Bartosh
Replaced wic.bz2 -> wic in the recipe. Replaced wic.bz2 -> wic in oe-selftest test case. wic-image-minimal is going to be booted in qemu and tested. As runqemu doesn't support compressed images this recipe needs to produce raw wic image. (From OE-Core rev: 3ce91db94f4b921566dce0dc6f91a422009be06b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22emptytest: exclude from world buildsRoss Burton
This recipe is for testing purposes and shouldn't be built in world builds, as it produces errors without being built in the selftest. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-19image: Fix wic environment issuesRichard Purdie
The wic environment function needs to run after the rootfs size is setup. We move this code to a specific task, and depend on that task from the wic images and other places its needed. This fixes: ====================================================================== FAIL: test_image_env (oeqa.selftest.wic.Wic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env self.assertTrue(var in content, "%s is not in .env file" % var) AssertionError: False is not true : ROOTFS_SIZE is not in .env file Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>