summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest/cases
AgeCommit message (Collapse)Author
2020-05-12selftest/imagefeatures: Enable sanity test for IMAGE_GEN_DEBUGFSYeoh Ee Peng
Add new testcase to check IMAGE_GEN_DEBUGFS. Test makes sure that debug filesystem is created accordingly. Test also check for debug symbols for some packages as suggested by Ross Burton. [YOCTO #10906] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-07virgl: skip test on Debian 9Alexander Kanavin
Debian 9 has an older version of mesa, where dri drivers do not link against glapi explicitly, which causes problems when they are loaded by newer mesa-native: pokybuild@debian9-ty-2:~$ ldd -d /usr/lib/x86_64-linux-gnu/dri/swrast_dri.so |grep undefined undefined symbol: _glapi_tls_Dispatch (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so) undefined symbol: _glapi_tls_Context (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so) compared to Debian 10: pokybuild@debian10-ty-2:~$ ldd -d /usr/lib/x86_64-linux-gnu/dri/swrast_dri.so|grep undefined pokybuild@debian10-ty-2:~$ Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26oeqa: wic: Add more tests for include_pathRicardo Ribalda Delgado
Make sure permissions are respected. Add new test for orig/destination option. Cc: Paul Barker <pbarker@konsulko.com> Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26oeqa: wic: Add tests for permissions and change-directoryRicardo Ribalda Delgado
Make sure that the permissions and username are respected when using all the rootfs modifiers. Add tests for change-directory command Cc: Paul Barker <pbarker@konsulko.com> Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26oeqa/selftest: Add test for conflicting sysroot providerRichard Purdie
sysroot-test depends on virtual/sysroot-test which we build for one machine, switch machine, switch provider of virtual/sysroot-test and check that the sysroot is correctly cleaned up. The files in the two providers overlap so can cause errors if the sysroot code doesn't function correctly. Yes, sysroot-test should be machine specific really to avoid this, however the sysroot cleanup should also work. This adds a test for bug: [YOCTO #13702] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26selftest/package: Add test to ensure ownership is preservedDaisuke Yamane
Yocto Bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13806 Add test_preserve_ownership to selftest/package. This test creates a file, a directory and a symbolic link and changes ownership, then compares with them installed in rootfs to ensure ownership is preserved. [Test without a commit 'bitbake: lib/bb/utils.py: Preserve ownership of symlink'] | 2020-03-14 10:01:14,519 - oe-selftest - INFO - test_preserve_ownership (package.PackageTests) | 2020-03-14 10:56:44,612 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/file | 2020-03-14 10:56:44,770 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/dir | 2020-03-14 10:56:44,822 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/symlink | 2020-03-14 10:56:44,879 - oe-selftest - ERROR - Incrrect ownership /etc/selftest-chown/symlink [root:root] | 2020-03-14 10:56:45,884 - oe-selftest - INFO - ... FAIL [Test with a commit 'bitbake: lib/bb/utils.py: Preserve ownership of symlink'] | 2020-03-14 10:58:49,599 - oe-selftest - INFO - test_preserve_ownership (package.PackageTests) | 2020-03-14 11:51:39,947 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/file | 2020-03-14 11:51:40,013 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/dir | 2020-03-14 11:51:40,063 - oe-selftest - INFO - Check ownership of /etc/selftest-chown/symlink | 2020-03-14 11:51:41,118 - oe-selftest - INFO - ... ok Signed-off-by: Daisuke Yamane <daisuke.yamane@cybertrust.co.jp> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24oeqa/selftest/tinfoil: Drop test for functionality removed upstreamRichard Purdie
bitbake dropped support for this piece of functionality (the test was the only user), drop the test too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24archiver.py: fix typo of "ARCHIVER_MIRROR_EXCLUDE" in commentrpjday@crashcourse.ca
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-19oescripts.py: use unittest.SkipUnless for class skipTrevor Gamblin
Yocto Bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13663 Prior to this fix, the try/except block in OEScriptTests' setUpClass did both the import and the raise unittest.SkipTest for the subsequent OEPybootchartguyTests. The consequence of the this meant that, when cairo was not available on the test host, the raise was counted only once as a class setup issue, instead of once for each of the tests to be run, resulting in incorrect skip counts in the test results. Instead of raising unittest.SkipTest in setUpClass, the more proper method is to use the unittest.skipUnless method along with importlib.util.find_spec to check for the presence of the python3-cairo module. This allows the entire class to be skipped if cairo is not present, while also correctly passing along the skip state for each of the class's tests, and avoiding the UNKNOWN results to appear in cases where the -j option was provided to oe-selftest, but the tests were still skipped. Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2020-02-19oeqa: reproducible: Include jquery in resultsJoshua Watt
Includes jquery in the output directory to make the diffoscope output easier to navigate Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18fix various, innocuous typosRobert P. J. Day
Corrections: - environment - accommodate - conversion - compatible Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15selftest: wic: Add APPEND to the optional variables listKevin Hao
The APPEND is an optional variable, so add it to the optional variables list to make the wic selftest happy. Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14oeqa: reproducible: Run diffoscope on saved outputJoshua Watt
If there are differing packages and they are being saved for review, automatically run diffoscope on them and include the output in the saved output. The output is currently done in HTML format since these are typically published on a webpage by the autobuilder. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08oeqa: Use cpio 2.13 as testcaseKhem Raj
cpio 2.12 was released in 2015 and might have used older autotools which could result in errors like https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779 Bumping to 2.13 will help in matching the tool versions A good change on top would be to run aclocal -I .; autoheader; autoconf; automake --add-missing -c before running configure step perhaps [YOCTO #13779] Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06oeqa/reproducible: Improve test output and ensure deb+ipk comparedRichard Purdie
Adding newline characters between the packages in the failure output massively improves readability. Also ensure to output ipk failures when there are deb failures by calling self.fail() at the end, else sometimes only partial differences are returned. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-03oeqa: reproducible: Do not strip packagesJoshua Watt
Do not strip packages when testing reproducible builds. In some cases, stripped data differs between builds, but then gets removed. However, the contents affect the generation of the GCC build-id, which then differs in the resulting ELF files, even though the data that caused this is no longer there. Inhibit stripping so that their causes can be evaluated. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02reproducible: Allow configuration of saved debug outputRichard Purdie
If OEQA_DEBUGGING_SAVED_OUTPUT is set in the environment, use this location to store reproducibile build failure output. This aids debugging on the YP autobuilder in particular. Use a date in the directory name to make it easier to find failure output. Also clean up empty directories as they're unnecessary distracting noise. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27distrodata/maintainers: Add PARSE_ALL_RECIPES flag to fix test issuesRichard Purdie
Currently the test depends on which DISTRO_FEATURES and targets are selected. Similar to SOURCE_MIRROR_FETCH, add PARSE_ALL_RECIPES to allow the test to see a much wider range of them. This avoids issues added by the new ucontext musl specific recipe and allows the current whitelist to be cleared out a bit (more cleanup can follow). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27oeqa/selftest/devtool: add npm recipe build testJean-Marie LEMETAYER
This commit adds a devtool build test for npm recipe: - devtool.DevtoolAddTests.test_devtool_add_npm Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27oeqa/selftest/recipetool: add npm recipe creation testJean-Marie LEMETAYER
This commit adds a recipetool creation test for npm recipe: - recipetool.RecipetoolTests.test_recipetool_create_npm Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22oeqa/reproducible: Fix unset save_dir variableRichard Purdie
Previous refactoring broke the case where save_dir was set. Fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22oeqa/reproducible: Fix extra data reportingJoshua Watt
A typo was preventing the extra data about the reproducible build from being reported in the test results Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22oeqa/reproducible: Add flag for building from sstateJoshua Watt
Adds a flag to control if the reproducible QA test should allow building from sstate or not. Building from sstate may not be reproducible depending on how the sstate is populated. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-21selftest: check maintainers.inc for entries without recipesAlexander Kanavin
Also remove a couple of entries found by the test :) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10oeqa: archiver: Tidy up test case descriptionsPaul Barker
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10archiver.bbclass: Add new mirror archiver modePaul Barker
We define a new method of populating a source mirror using the archiver bbclass instead of simply copying the contents of the downloads directory. This allows the archiver features such as copyleft license filtering and recipe type filtering to be used when preparing a source mirror. This new archiver mode is selected by setting `ARCHIVE_MODE[src]` to 'mirror'. The source mirror mode can either be 'split' (default) or 'combined', controlled by `ARCHIVER_MODE[mirror]`. Additionally, sources can be excluded as needed by setting `ARCHIVER_MIRROR_EXCLUDE` to a list of URI prefixes. These options are described in more detail in the new entries in the header of archiver.bbclass. New oeqa selftest cases are added to cover the mirror archiver mode. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10wic: Add --include-path argumentPaul Barker
This option adds the contents of the given path to a partition built with the rootfs source plugin. The path is relative to the directory in which wic is running not the rootfs itself so use of an absolute path is recommended. This option is most useful when multiple copies of the rootfs are added to an image and it is required to add extra content to only one of these copies. This option only has an effect with the rootfs source plugin. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-06sstate: Add extra directory levelRichard Purdie
We're having speed issues on the autobuilder due to the numbers of files in sstate directories. We previously split these by the first two characters of the hash. This change extends this to split by the next two characters as well, creating more layers of directories. This should signifiantly speed up eSDK builds on the autobuilder as the current sstate layout simply isn't scaling there but addresses a general complaint. gen-lockedsig-cache needed to be updated for the new split level sstate. Also update tests for new layout. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for nativeMike Crowe
Ensure that RECIPE_SYSROOT is the same for -native recipes whether multilib.conf is included or not. Without this change task signatures for -native recipes change when switching between MACHINEs that require multilib.conf and those that don't. This fix was one of the ones suggested by Khem Raj in http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html Add test_sstate_multilib_or_not_native_samesigs test case to sstatetests.py to ensure that this stays fixed. Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28oeqa/selftest/package: mark string as rawRoss Burton
This regex uses \ so is actually parsed incorrectly, mark it up as a raw string. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-12-18oeqa: reproducible: Test core-image-sato and core-image-full-cmdlineJoshua Watt
Adds core-image-sato and core-image-full-cmdline to the list of images that the reproducible build test builds. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16incompatible_lic.py: Add tests for incompatible licenses with wildcardsPeter Kjellerstedt
Suggested-by: Quentin Schulz <quentin.schulz@streamunlimited.com> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16license_image.bbclass: Report only the licenses that are incompatiblePeter Kjellerstedt
Instead of reporting ${LICENSE} when a package cannot be installed into an image because it is using an incompatible license, report the license(s) that are actually incompatible. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16base.bbclass: Report only the licenses that are incompatible for a packagePeter Kjellerstedt
Instead of reporting ${LICENSE} when a package is identified as using an incompatible license, report the license(s) that are actually incompatible. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16license.bbclass: Introduce AVAILABLE_LICENSES that lists all licensesPeter Kjellerstedt
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented variable that contained a static list of licenses. It was used by expand_wildcard_licenses() to expand any wildcards used in, e.g., INCOMPATIBLE_LICENSE. However, since this static list of licenses has not been kept up-to-date, many licenses were missing, with the result that if one tried to use any of those licenses with a wildcard, no licenses would be found, effectively ignoring that they should be marked as incompatible. This introduces a new (documented) variable, AVAILABLE_LICENSES, that is automatically updated to contain all licenses found in any directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding wildcards. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-09oeqa/selftest/runtime_test: split systemd and sysvinit tests outArmin Kuster
This should help debug which part of this selftest is failing. We can not tell which outloop is failing so split the tests into systemd and sysvinit. [YOCTO #13650] Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-09selftest: skip virgl gtk/sdl test on opensuse 15.0Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-04selftest/imagefeatures: blacklist zstRoss Burton
The dependency isn't in OE-Core so we need to disable this test. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-28oe-selftest: extend virgl gtk test to also check the SDL optionAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27oeqa: archiver: Add basic tests for all archiver modesPaul Barker
6 new test cases are added to cover the various archiver modes documented at the top of archiver.bbclass. Each test sets the appropriate configuration options, runs the `do_deploy_archives` task for the selftest-ed recipe and checks for the presence of the expected archive file. Signed-off-by: Paul Barker <paul@betafive.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27wic: 'wic cp' to copy from imageChee Yang Lee
currently 'wic cp' only works for copy file from local storage to wic image. enhance 'wic cp' to copy file/directory from wic image to local storage. include selftest and 'wic help' updates. [YOCTO#12169] Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27oeqa/selftest/cases/gcc.py: Split 'gcc' and 'g++' testsuitesNathan Rossi
Split the GccCrossSelfTest testcase into separate testcases for 'gcc' and 'g++' respectively. In order to split them use the "check-gcc-*" language make check targets. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-22oeqa/selftest/sstatetests: Ensure we don't use hashequiv for sstatesigs testsRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21selftest: check that 'devtool upgrade' correctly drops backported patchesAlexander Kanavin
There was a regression in this functionality that went unnoticed due to lack of tests. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21oeqa/devtool: Avoid unbound variable errorsRichard Purdie
inherits can be unset resulting in: UnboundLocalError: local variable 'inherits' referenced before assignment which can mask real errors. Avoid this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21selftest/wic: test wic rm with -r flagChee Yang Lee
selftest to make sure wic rm with -r flag for ext partition could remove directory and all its content. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/selftest/runtime_test: add crosstab selftestArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14oeqa/selftest/oescripts: move list-packageconfig-flags tests from manual to selfArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14buildhistory: fix "version went backwards" QA error messageDenys Dmytriyenko
Fix parentheses placement in the message from: Package version for package X went backwards which would break package feeds from (Y to Z) to this one: Package version for package X went backwards which would break package feeds (from Y to Z) Signed-off-by: Denys Dmytriyenko <denys@ti.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-07oeqa: reproducible: Add option to capture bad packagesJoshua Watt
Adds an option that can be used to copy the offending packages to a temp directory for later evaluation. This is useful on the Autobuilder to investigate failures. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-04-30 21:49:59 +0000