aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2015-11-16oe-selftest: add test for bitbake-layers show-recipesPaul Eggleton
Add a test for bitbake-layers show-recipes including the recently added -i option. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/layerappend: fix test if build directory is not inside COREBASEPaul Eggleton
Fix test_layer_appends to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/selftest/devtool: fix test if build directory is not inside COREBASEPaul Eggleton
Fix test_devtool_update_recipe_git to work when build directory is not inside COREBASE. Fixes [YOCTO #8639]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16qemurunner: Remove the timeout in run_serialMariano Lopez
Sometmes when there is high load in the server the commands executed in the target take a lot of time to complete and this lead to incorrect dump files or empty files. This is caused because run_serial has a timeout of five seconds when running the commands in the target. This change removes the timeout and just keep reading the socket until it finds the prompt from the target or when the socket is not ready to be read. [YOCTO #8510] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16qemurunner: Enable timestamps on kernel boot-upLeonardo Sandoval
Having the timestamps in the log is useful when debugging those cases when the booting time takes longer than expected. printk.times=1 enables stamping into every printk Kernel call. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/runtime/multilib: run the arch tests on connmand not connman-appletRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-16oeqa/runtime: remove dmesg testRoss Burton
This has been obsoleted by parselogs. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-27oeqa/utils/decorators: fix missing keyword arguments on decoratorsPaul Eggleton
We need to handle keyword arguments here or sending a keyword argument to a decorated function that accepts keyword arguments will trigger an error. (This showed up when testcase decorators were added to the recipetool.RecipetoolTests.test_recipetool_appendsrcfiles_basic_subdir test). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-24oeqa/selftest: Added testcase decorators.Daniel Istrate
Added testcase decorators for testopia integration. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24qemurunner: Show the output of runqemu scriptMariano Lopez
This change will show runqemu output in the logs when running the runtime tests. This is helpful for debugging testimage errors. [YOCTO #8550] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: improve config writing and cleanupRoss Burton
The selftest.inc configuration file is deleted in both tearDown() and setUp() so there's no need to use addCleanup() to remove statements from it. Use write_config instead of append_config if the intention is to start from an empty config file, for clarity. Finally remove some misleading comments that claim that append_config() writes to local.conf when it doesn't. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: remove numbers from test namesRoss Burton
There isn't any required ordering of tests, and the numbers meant that the tests didn't appear in oe-selftest --list-classes. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: clean up selftest.inc in teardownRoss Burton
Test cases may want to do call bitbake in setUpClass() but at that point the previous selftest.inc is still present which could change the build configuration and result in any built artifacts being removed in the next bitbake invocation as part of the sysroot clean up. Resolve this by cleaning selftest.inc in the tearDown, the clean in setUp should be considered a safety net. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: fix cleaningRoss Burton
In some situations the native tools built in setUpClass() are wiped from the sysroot by the time the tests are executed, likely due to the cleanup performed in the base setUp() method. Avoid this by doing all of the preparatory building in setUpLocal. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/wic: corrected testcase decorator for test18_iso_imageDaniel Istrate
Changed testcase decorator for TC test18_iso_image from 1264 to 1346. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest: verify that devtool can use plugins in other layersRoss Burton
In selftest we want to exercise devtool's ability to use plugins from other layers, so add a basic command to reverse a string to meta-selftest and a test in oe-selftest to call it. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/buildoptions: Use the correct script for cleaning the workdirLeonardo Sandoval
The script 'cleanupworkdir' does not exit, so changing it to 'cleanup-workdir'. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: Updated bitbake TCsDaniel Istrate
- Added new TC test_force_task_1 (1354); check that do_package_write_rpm() re-executes upon changes in package image. - Updated TC test_force_task_2 (163); changed test recipe to zlib and added do_package() to the task execution list. - Removed unnecessary imports. Fix for bug [YOCTO #5875]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/bbtests: clean up local DL_DIR/SSTATE_DIR safelyRoss Burton
Now that the selftests clean selftest.inc in the tearDown using bitbake in a function passed to addCleanup (which happens after teardown) will use the default configuration and not the customised one. As the intention was to clean away the temporary DL_DIR and SSTATE_DIR, simple use track_for_cleanup to delete the entire directory instead. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/utils/ftools: From functions that expect data, check if NoneLeonardo Sandoval
ftools functions that expect data may get 'None'; this patch does this check and return immediately if this is the case. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/utils/ftools: Ignore the exception if file does not existLeonardo Sandoval
There may be cases where the configuration file (path) does not exist, thus the remove_from_file should catch this exception. In case the exception is not the latter (errno.ENOENT), then re-raise it. [YOCTO #8540] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21oeqa/selftest/manifest.py: Test support for manifestsBenjamin Esquivel
adding support for tests to verify that manifest contents contain entries that exist in an specified pkgdata dir tests added: - manifest.VerifyManifest.test_image_manifest_entries - manifest.VerifyManifest.test_SDK_manifest_entries test support written for future tests: -adding a setUpClass that supports other manifest tests -a get dir from bb var function that verifies if the dir exists -a ManifestEntry class defined with missing items list -check for the paths and fail gracefully if not there -debug prints for failure analysis [YOCTO#8028] Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19oeqa/runtime: Fix setUp and tearDown methodsMariano Lopez
Currently some of the runtime test overwrites the setUp and tearDown methods provided by oeRuntimeTest, this will avoid some checks required when running the test suit. This patch changes the setUp and tearDown methods for their local counterparts, so when these tests are called, it will run the parent setUp and tearDown and also the local ones. [YOCTO #8465] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19oetest: Add tearDownLocal classMariano Lopez
In the current state there are some runtime test that don't run the tearDown method fromm oeRuntimeTest class because the tearDown class is overwritten in the child class. This change adds tearDownLocal method in oeRuntimeTest class that will run after tearDown. This method can be overwritten in the child classes to implement specific test functionality. [YOCTO #8465] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03oeqa/utils/qemurunner: Add support for Unicode from qemuMariano Lopez
The current state of qemurunner will drop the Unicode characters received from qemu, this is because error report web had problems with Unicode characters; now that the server support Unicode, it is possible to log all the output from qemu. So far the only Unicode character seen is the copyright symbol. This patch allows to get Unicode characters from the qemu target and save the log in an UTF-8 file for latter use. [YOCTO #8225] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-03oe-selftest: wic: fix LocalSetupEd Bartosh
Appended required features to build config every time LocalSetup is run to ensure that core-image-minimal is not rebuilt due to configuration changes. This should fix test22_mkgummidisk and test23_mkefidisk testcases as they depend on specific features set in LocalSetup. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/runexported: Replaced optionparser with argparse.Lucian Musat
Also added the default json file name as default for the first positional argument. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01testimage: Added IO commands to dumpsMariano Lopez
This change just add two more commands to the host dumps to get more information related to the IO. [YOCTO #8412] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01Revert "oeqa/runtime: Added one runtime testcase in connman."Cristian Iorga
connman daemon forks during normal usage, so this test is no longer valid and generates sometimes false positives. This reverts commit 7d01c595c96eb3d67b90caf71050c4e9345257de. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/wic: Use SetupLocal instead of SetupRichard Purdie
If we replace the code Setup method, we don't get the cleanup it performs and this just resulted in failures on the autobuilder due to a stale config fragment. Setup will call SetupLocal so this should be a safe and easy fix to resolve the auobuilder failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add test that MACHINE doesn't change target sigsRichard Purdie
When we change between two machines with the same tune, we shouldn't see rebuilds of binaries. This adds a test for this using the qemux86copy machine. We also extend the test to cover multilibs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/selftest/sstatetests: Add check for same sigs for SDKMACHINERichard Purdie
Extend one of the sstate tests to also ensure that changing SDKMACHINE doesn't change the target task checksums. Also fix a typo and improve debugging by turning the diff filtering off in all cases (if the test fails, we want to full list). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oeqa/utils/decorators: Append the testname without the full pathLeonardo Sandoval
When getting the failures/errors/skipped lists, include the unit test without the full path. This issue was found on this scenario | test_1_logrotate_setup (oeqa.runtime.logrotate.LogrotateTest) ... FAIL | test_2_logrotate (oeqa.runtime.logrotate.LogrotateTest) ... ok Where test_1_logrotate failed and test_2_logrotate should not have run because @skipUnlessPassed("test_1_logrotate_setup") def test_2_logrotate(self): Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01oetest: Change logic of a failed testMariano Lopez
Currently the logic to check if a test failed was to check for an exception in the thread, but some decorators used in the syslog runtime test would generate and handle exceptions; this will mess with the current check logic and will dump the host and the target as if the test failed. This patch changes the check logic to verify if the test that just happend is in the failure test list and dump the host and target accordingly. [YOCTO #8406] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/runexported: Fix a problem with ssh_target_log existing in folder.Lucian Musat
When copying the exported tests to a remote machine ssh_target_log can be transformed from softlink to file which will throw an error when trying to run again. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29qemurunner: Sanitize output from qemu and qemu pidMariano Lopez
Currently the output from qemu could contain control or Unicode characters; having such characters in the log will cause an internal server error when sending the report to error reporting web. Control characters can be found in the command line used to run quemu too. This change sanitize the output from qemu and the command line used to run qemu, this way the logs doesn't contain control or Unicode characters and this will solve the issue of error reporting web and generate better logs. The only Unicode character found in the qemu output is the copyright symbol, it really doesn't interfer removing Unicode characters with debugging. [YOCTO #8225] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/decorators: Fixed a problem with decorator logs link.Lucian Musat
When not doing actual tests with testimage, like for example exporting tests, the link to the log file was still created although the actual log file was not existent. Fixed it by moving the link creation part into the run() method. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29oeqa/selftest/wic: Added testcase decorator to all testcases + fixed minor ↵Daniel Istrate
typos. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: better support for local source filesMarkus Lehtonen
* extract: Copy all local source files (i.e. non-compressed/non-arcived SRC_URI files that have file:// URI prefix) - excluding patches - to the srctree repository. The files will be placed in a subdirectory called 'oe-local-files'. The oe-local-files directory is not committed to the Git repository, but, marked to be ignored by a .gitignore file. The developer can manually add and commit the files to Git if the changes to them need to be tracked. Before this patch, local source files (were copied (and committed) to the srctree repository only in some special cases (basically when S=WORKDIR) when doing devtool-extract. For most of the packages local files were not copied at all. * update-recipe: This patch causes the local files to be 'synced' from the srctree (i.e. from the 'oe-local-files' subdirectory) to the layer. Being 'synced' means that in addition to copying modified files over the original sources, devtool will also handle removing and adding local source files and updating the recipe accordingly. We don't want to create patches against the local source files but rather update them directly. Thus, 'oe-local-file' directory is ignored in patch generation when doing update-recipe, even if committed to Git. This functionality is only enabled if the 'oe-local-files' directory is present in srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking repo statusMarkus Lehtonen
New method for checking the status of the working tree of a repository. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add method for checking srctree repoMarkus Lehtonen
Removes some code duplication. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-29oe-selftest: devtool: add setup() methodMarkus Lehtonen
In order to remove some code duplication. The method currently only does checking of workspace dir. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-28oeqa/sdk/gcc: Fix makefile testRichard Purdie
The makefile test was hardcoding a test for gcc. In the SDK context, we need to use $(CC) to use the cross compiler. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28oeqa/parselogs: Updated log parser whitelist.Lucian Musat
Added some fixes for meta-intel boards and moved some errors from specific images to common. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28oeqa/decorators: Add timestamp to decorator logs.Lucian Musat
To avoid logs being overwriten when running the automated tests multiple times, log files include timestamps in their names and a link is created to point to the latest one. Signed-off-by: Lucian Musat <george.l.musat@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-28qemurunner: Handle qemu start failure correctlyRichard Purdie
If qemu didn't start correctly, we may not have registered the child signal. This results in a nasty traceback which confuses the underlying issue. Cleanup this code and make the handler cleanup conditional. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-26oeqa/oetest: Fix SDK command executionRichard Purdie
The SDK environment wasn't being sourced when running these test commands, meaning in some cases the cross compiler was being tested, not the SDK tools. This is clearly not the intent so fix this by ensuring the SDK environment is present. This fixes test failures in multilib SDKs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/selftest/archiver: Test that archiver filters on recipe nameDaniel Istrate
[YOCTO #6929] this test validates the feature introduced in bug 6929 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23oeqa/utils/dump: Add default commands and directoryMariano Lopez
Currently if qemu fails when running a selftest and tries to run some commands on the host it will fail because some variables required by the Dumper class do not exist because testimage was not included. This change adds a default parent directory to save the dumps for the host or target. Also adds default commands to run if no commands were provided to the class. With these changes the previous errors using selftest don't show anymore. [YOCTO #8306] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>