aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2016-01-07oeqa/selftest/recipetool: update for libjpeg-turbo migrationRoss Burton
The recipe creation test of libmatchbox now depends on libjpeg-turbo, not jpeg. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07scripts/oe-selftest: Allow to run tests on random/all MACHINEsDaniel Istrate
Add an option for random MACHINE into oe-selftest: --machine [random/all] 1. random: will set a random MACHINE for each test 2. all: will run tests for all machines Custom machine sets only weak default values (??=) for MACHINE in machine.inc. This let test cases that require a specific MACHINE to be able to override it, using (?= or =). e.g.: oe-selftest --run-tests signing --machine random --> will run all tests switching MACHINE randomly for each test oe-selftest --run-tests signing --machine all --> for each machine will run all tests oe-selftest --run-all-tests --machine random Also update oeqa/selftest/base.py to accomodate this feature. Fix for [YOCTO #5880]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: Added testcase decorators for 2 testsDaniel Istrate
1391 for test_devtool_add_fetch_simple from devtool 1392 for test_recipetool_create_simple from recipetool Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: moved tc test_buildhistory_does_not_change_signaturesDaniel Istrate
Moved test_buildhistory_does_not_change_signatures from buildhistory/BuildhistoryBase to buildoptions/BuildhistoryTests. The test being in the base class was causing it to run multiple times. Fix for [YOCTO #8867] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-28oeqa/selftest: support sets in devtool comparisonsRoss Burton
The devtool and recipetool tests do literal string comparisons, but for some fields the ordering could be irrelevant and potentially non-deterministic. For example, the recipetool_create_simple test started failing with: AssertionError: 'GPLv2 Unknown' != 'Unknown GPLv2' : values for LICENSE do not match The ordering of the LICENSE field isn't relevant. So, if the expected value is a set(), split the string into a set too and compare those. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-27selftest: Added MACHINE = "qemux86" to tests that use runqemuDaniel Istrate
It makes sense for tests that use runqemu to have MACHINE set as qemu. This also avoid issues when running oe-selftest with --arch random/all option. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22oe-selftest: devtool: add more explicit check for ls outputPaul Eggleton
test_devtool_deploy_target is failing on the Yocto Project autobuilder apparently when it attempts to cut out some fields from the list. It doesn't fail here and I can't see what the problem lines are, so add a check for lines with too few fields so we can get a look at them next time it fails. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22oe-selftest: add tests for simple devtool add / recipetool create URL casePaul Eggleton
Add an oe-selftest test case for the newly supported syntax with only the remote URL specified (auto-detecting name and version). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: basic extraction of name/version from filenamePaul Eggleton
Often the filename (e.g. source tarball) contains the name and version of the software it contains. (This isn't intended to be exhaustive, just to catch the common case.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: support extracting name and version from build scriptsPaul Eggleton
Some build systems (notably autotools) support declaring the name and version of the program being built; since we need those for the recipe we can attempt to extract them. It's a little fuzzy as they are often omitted or may not be appropriately formatted for our purposes, but it does work on a reasonable number of software packages to be useful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22selftest/buildhistory.py: Test buildhistory does not change sigsDaniel Istrate
[YOCTO #5953] Add a test to ensure buildhistory does not change signatures. Also removed unused imports. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-16oeqa/selftest: Added testcase decorators for 2 testcasesDaniel Istrate
bblayers: test_bitbakelayers_showrecipes 1384 wic: test_directdisk_bootloader_config 1385 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09selftest/wic.py: Add test for custom bootloader configMariano Lopez
This change just add anoher test to the wic module. It will try to create a image with a custom bootloader configuration. This test will use the example image directdisk-bootloader-config to test the configfile option for the bootloaders. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: disable creating workspace for extract and search subcommandsPaul Eggleton
For subcommands that don't actually involve the workspace, don't auto-create the workspace. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/sstatetests: prettier output for allarch testRoss Burton
Instead of creating two lists of full paths and comparing them which in failure produces a list of every stamp file (so all tasks, twice), reduce the filename down to a recipe/task->hash dictionary and compare those, meaning unittest shows the differences in the dictionaries. In the future get_files() should be generalised so all tests in this class can use it, and find a pair of hashes that don't match and run diffsigs on them. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/signing: Added new test for signing sstate.Daniel Istrate
[YOCTO #8182] Optional signing sstate archives and signature verification [YOCTO #8559] Signing sstate archives with custom dir for gpg keys Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-24oeqa/selftest/signing: New test for Signing packages in the package feeds.Daniel Istrate
[YOCTO # 8134] This test verifies features introduced in bug 8134. It requires as resources the files from meta-selftest/files/signing: For 'gpg --gen-key' the used input was: key: RSA key-size: 2048 key-valid: 0 realname: testuser email: testuser@email.com comment: nocomment passphrase: test123 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oe-selftest: add test for bitbake-layers show-recipesPaul Eggleton
Add a test for bitbake-layers show-recipes including the recently added -i option. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/layerappend: fix test if build directory is not inside COREBASEPaul Eggleton
Fix test_layer_appends to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/devtool: fix test if build directory is not inside COREBASEPaul Eggleton
Fix test_devtool_update_recipe_git to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24oeqa/selftest: Added testcase decorators.Daniel Istrate
Added testcase decorators for testopia integration. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: improve config writing and cleanupRoss Burton
The selftest.inc configuration file is deleted in both tearDown() and setUp() so there's no need to use addCleanup() to remove statements from it. Use write_config instead of append_config if the intention is to start from an empty config file, for clarity. Finally remove some misleading comments that claim that append_config() writes to local.conf when it doesn't. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: remove numbers from test namesRoss Burton
There isn't any required ordering of tests, and the numbers meant that the tests didn't appear in oe-selftest --list-classes. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: clean up selftest.inc in teardownRoss Burton
Test cases may want to do call bitbake in setUpClass() but at that point the previous selftest.inc is still present which could change the build configuration and result in any built artifacts being removed in the next bitbake invocation as part of the sysroot clean up. Resolve this by cleaning selftest.inc in the tearDown, the clean in setUp should be considered a safety net. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: fix cleaningRoss Burton
In some situations the native tools built in setUpClass() are wiped from the sysroot by the time the tests are executed, likely due to the cleanup performed in the base setUp() method. Avoid this by doing all of the preparatory building in setUpLocal. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: corrected testcase decorator for test18_iso_imageDaniel Istrate
Changed testcase decorator for TC test18_iso_image from 1264 to 1346. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: verify that devtool can use plugins in other layersRoss Burton
In selftest we want to exercise devtool's ability to use plugins from other layers, so add a basic command to reverse a string to meta-selftest and a test in oe-selftest to call it. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/buildoptions: Use the correct script for cleaning the workdirLeonardo Sandoval
The script 'cleanupworkdir' does not exit, so changing it to 'cleanup-workdir'. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: Updated bitbake TCsDaniel Istrate
- Added new TC test_force_task_1 (1354); check that do_package_write_rpm() re-executes upon changes in package image. - Updated TC test_force_task_2 (163); changed test recipe to zlib and added do_package() to the task execution list. - Removed unnecessary imports. Fix for bug [YOCTO #5875]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: clean up local DL_DIR/SSTATE_DIR safelyRoss Burton
Now that the selftests clean selftest.inc in the tearDown using bitbake in a function passed to addCleanup (which happens after teardown) will use the default configuration and not the customised one. As the intention was to clean away the temporary DL_DIR and SSTATE_DIR, simple use track_for_cleanup to delete the entire directory instead. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/manifest.py: Test support for manifestsBenjamin Esquivel
adding support for tests to verify that manifest contents contain entries that exist in an specified pkgdata dir tests added: - manifest.VerifyManifest.test_image_manifest_entries - manifest.VerifyManifest.test_SDK_manifest_entries test support written for future tests: -adding a setUpClass that supports other manifest tests -a get dir from bb var function that verifies if the dir exists -a ManifestEntry class defined with missing items list -check for the paths and fail gracefully if not there -debug prints for failure analysis [YOCTO#8028] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03oe-selftest: wic: fix LocalSetupEd Bartosh
Appended required features to build config every time LocalSetup is run to ensure that core-image-minimal is not rebuilt due to configuration changes. This should fix test22_mkgummidisk and test23_mkefidisk testcases as they depend on specific features set in LocalSetup. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/wic: Use SetupLocal instead of SetupRichard Purdie
If we replace the code Setup method, we don't get the cleanup it performs and this just resulted in failures on the autobuilder due to a stale config fragment. Setup will call SetupLocal so this should be a safe and easy fix to resolve the auobuilder failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add test that MACHINE doesn't change target sigsRichard Purdie
When we change between two machines with the same tune, we shouldn't see rebuilds of binaries. This adds a test for this using the qemux86copy machine. We also extend the test to cover multilibs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add check for same sigs for SDKMACHINERichard Purdie
Extend one of the sstate tests to also ensure that changing SDKMACHINE doesn't change the target task checksums. Also fix a typo and improve debugging by turning the diff filtering off in all cases (if the test fails, we want to full list). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/selftest/wic: Added testcase decorator to all testcases + fixed minor ↵Daniel Istrate
typos. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: better support for local source filesMarkus Lehtonen
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking repo statusMarkus Lehtonen
New method for checking the status of the working tree of a repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking srctree repoMarkus Lehtonen
Removes some code duplication. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add setup() methodMarkus Lehtonen
In order to remove some code duplication. The method currently only does checking of workspace dir. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-23oeqa/selftest/archiver: Test that archiver filters on recipe nameDaniel Istrate
[YOCTO #6929] this test validates the feature introduced in bug 6929 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22devtool: build-image: delete bbappend at end of buildPaul Eggleton
Upon further reflection, it seems to me that this bbappend ought to just be deleted at the end of the build. This keeps things simple; you never have to remember to delete any files to get back to where you were before with the image. This means we can also drop the slightly awkward message reminding the user how to do that. I've also updated the test to look at the image manifest to determine if the command has worked instead of looking for the (now deleted) bbappend. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22oeqa/selftest: buildoptions.py Removed unused importsDaniel Istrate
Removed unused imports: unittest, logging, pexpect Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-16oeqa/sstatetests: Add test for nativesdk stamp invariance with MACHINERichard Purdie
nativesdk-glbic should not rebuild when you change MACHINE but it was. We've fixed that, now add tests to ensure this doesn't happen again. Rather than add yet another stamps test, extend one of the existing ones to cover this instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09oeqa/sstatettests: Improve output of assertRichard Purdie
Currently if this fails you see: "AssertionError: 1 != 0" which is useless. Add the output of the failed command so we can stand some chance of debugging what went wrong. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: update-recipe: better 'auto' modeMarkus Lehtonen
Enhance the logic behind the 'auto' mode a bit by only updating the SRCREV if the changes are already found upstream. The logic is simple: update SRCREV only if the current local HEAD commit is found in the remote branch (i.e. 'origin/<branch_name>'). Otherwise resort to patching. This affects a couple of the oe-selftest tests so update those as well. [YOCTO #7907] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09devtool: add upgrade featureLeonardo Sandoval
Upgrades a recipe to a particular version and downloads the source code into a folder. User can avoid patching the source code. These are the general steps of the upgrade function: - Extract current recipe source code into srctree and create a branch - Extract upgrade recipe source code into srctree and rebase with previous branch. In case the rebase is not correctly applied, source code will not be deleted, so user correct the patches - Creates the new recipe under the workspace [YOCTO #7642] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02wic: add test cases for 3 imagesEd Bartosh
Added test cases to build qemux86_directdisk, mkgummidisk and mkefdisk images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-02oe-selftest: wic: configure a buildEd Bartosh
Added MACHINE_FEATURES and IMAGE_FSTYPES variables to the build config to ensure that bootimg and efi artifacts are built. This is needed to build canned wic images that are using those artifacts. Introduced class variable Wic.image_is_ready to avoid building image more than once. It would be better to build image in setUpClass, which is called only once, but add_config method can't be called in class method, so we have to use this trick with class variable in setUp method. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>