summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/sdk
AgeCommit message (Collapse)Author
2020-09-02oeqa: sdk: Capture stderr outputJoshua Watt
Redirect stderr to stdout when running subcommands while doing the SDK tests. The tests will show stdout when CalledProcessError is raised, but any output to stderr was lost. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 7cb4e9ab8c1596281060e94a216966060103956e) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-02-18fix various, innocuous typosRobert P. J. Day
Corrections: - environment - accommodate - conversion - compatible Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14oeqa: Use --disable-maintainer-mode configure optionKhem Raj
since the versions of autotools might differ on target and build host, plus difference in timestamps for configure and system can result in reconfigure lets avoid that by disabling maintainer mode Avoids error: newly created file is older than distributed files! [YOCTO #13779] Suggested-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08oeqa: Use cpio 2.13 as testcaseKhem Raj
cpio 2.12 was released in 2015 and might have used older autotools which could result in errors like https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779 Bumping to 2.13 will help in matching the tool versions A good change on top would be to run aclocal -I .; autoheader; autoconf; automake --add-missing -c before running configure step perhaps [YOCTO #13779] Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08oeqa/sdk: improve Meson testRoss Burton
Verify that the build inside the SDK is detected as a cross compilation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09oeqa/sdk: use bash to execute SDK test commandsMikko Rapeli
The commands only work with with bash. If /bin/sh is dash like in Debian, the command execution fails with errors like: Standard Output: /bin/sh: 5: export: --sysroot: bad variable name and all SDK tests fail. Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk: add test to exercise MesonRoss Burton
(From OE-Core rev: 29359493e391d68a5a6b4fa4d09ffdc1fe6db620) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk/assimp: cleanupRoss Burton
Unify style with the other tests. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-07oeqa/sdk/cases: clean up DL_DIR handlingRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-12-13oeqa/sdk: rewrite cpio testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: rewrite lzip testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clean up galculator testRoss Burton
Drop redundant imports and variables, and use os.makedirs() instead of bb.utils.mkdirhier(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: show output if run() failsRoss Burton
Use oeqa.utils.subprocesstweak to monkey-patch the subprocess exception so that any output is shown, and remove any explicit try/catch handling that would have hidden this. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clarify ELF assertion messageRoss Burton
For example, instead of saying "3 != 62", say "Binary was x86-64 but expected i586". Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08oeqa/sdk/galculator: rewrite to use new helpersRoss Burton
2018-12-08oeqa/sdk/python: fix version typoRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-12-08oeqa/sdk/assimp: use helpersRoss Burton
2018-12-08oeqa/sdk/case: add fundamental helper methodsRoss Burton
2018-12-03oeqa/sdk/python: add Python 2 and fix detectionRoss Burton
Add a Python 2 form to exercise that if present, and fix the setUp() so it actually looks for a package that exists (nativesdk-python3 is a virtual package, the interpretter is in nativesdk-python3-core). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20classes/testsdk: Split implementation into classesJoshua Watt
Splits the SDK test implementation into configurable Python classes. The classes used for the normal and extensible SDKs are ${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively. This allows SDK machines to override the classes used to implement the tests. For the traditional SDK, a common "run()" function is provided by the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions that can be overridden in child classes, making it easier to have consistent behavior. The extensible SDK class (oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but no hooks have yet been added as there is not currently a known use case for create derived classes. These changes should be purely organizational; no functional changes have been made to either the standard SDK or extensible SDK tests. [YOCTO #13020] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-13testsdk: fix skipped testcase output "UNKNOWN" status while multiprocess ↵Hongxu Jia
execution Usually skipped testcase output "SKIPPED" [snip serial execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: SKIPPED (0.01s) |RESULTS - python.PythonTest.test_python3 - Testcase -1: SKIPPED (0.01s) [snip serial execution] But if enable multiprocess execution, skipped testcase output "UNKNOWN" status [snip enable multiprocess execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: UNKNOWN |RESULTS - python.PythonTest.test_python3 - Testcase -1: UNKNOWN [snip enable multiprocess execution] Here is my investigation: There is a class pairs TestProtocolClient and TestProtocolServer provided by python3-subunit. The TestProtocolClient generates a subunit stream of TestResult from a test run, and TestProtocolServer parses the stream of subunit TestResult. The class ProtocolTestCase is a unittest.TestCase adapter and it uses TestProtocolServer to parse the stream of subunit TestResult. In Yocto testsdk, it forks multiple processes to execute testcases and use TestProtocolClient to generate TestResult stream; and then it creates multiple threads to use ProtocolTestCase to parse stream of subunit TestResult through pipe; finally it passes multiple ProtocolTestCase as TestCase instance to main process and output status result. The problem point is TestProtocolServer parses `skip:' directive after reading a `test:' directive. Without `test:' directive, `skip:' directive will be ignored. All above requires SkipTest should be raised inside a test method rather than setUpClass method. Throwing SkipTest inside setUp works correctly Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30oeqa/sdk: fixes related to hasPackage semanticsChen Qi
The current _hasPackage does a regex match when checking for the existence of packages. This will sometimes result in unexpected result. For example, the condition hasTargetPackage('gcc') is likely to be always true as it matches libgcc1. For most of the time, we should do exact match instead of regex match. So change _hasPackage function to do that. For the current sdk test cases, the only place that needs regex match is '^gcc-'. This is because there's no easy way to get multilib tune arch (e.g. i686) from testdata.json file. Besides, packagegroup-cross-canadian-xxx and gcc-xxx should be check in host manifest instead of the target one. So fix to use hasHostPackage. Also, as we are doing exact match, there's no need to use r'gtk\+3', just 'gtk+3' is enough. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30sdk/buldgalculator.py: check against multilib for gtk+3Chen Qi
When determining whether to skip the test case, the check should be done with consideration of multilib. Otherwise, we will meet the following error when testing against lib32 environment. No package 'gtk+-3.0' found Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30sdk/context.py: add ability to check for multilib version of target packageChen Qi
Add a named argument 'multilib' for the hasTargetPackage function. Its default value is False. When setting to True, it will try to get the correct multilib prefix from the sdk_env, the environment setup script. We need this because we don't want unexpected run of some sdk test cases. The following steps will generate error. 1. Enable multilib for qemux86-64 require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 ?= "core2-32" 2. bitbake core-image-sato -c populate_sdk 3. bitbake core-image-sato -c testsdk The error message is like below. No package 'gtk+-3.0' found RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: FAILED As we don't have lib32-gtk+3 installed, the test case should be skipped when testing against the lib32 environment setup script. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-21assimp.py: fix AttributeError in tearDownClassChen Qi
When running this test case, we will see the following error. AttributeError: type object 'BuildAssimp' has no attribute 'project' assimp.py test case does not make use of SDKBuildProject, so remove the import statement and the tearDownClass. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-10oeqa/sdk/buildgalculator: check for nativesdk-gettext-devRoss Burton
We don't need target gettext to build, but nativesdk-gettext-dev (for nls.m4). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07oeqa/sdk: add test that CMake worksRoss Burton
Add a new SDK testcase that builds assimp, a project that uses cmake. Using TARGET_ARCH and TARGET_OS which is now exported into the environment, check that the generated binaries match the ELF headers we expect. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26oeqa/sdk/sdkbuildproject: use os.path.joinRoss Burton
Instead of mushing two paths together and hoping that the slashes line up correctly, use os.path.join. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26oeqa: rationalise Perl testsRoss Burton
As with the Python test, this can be both better and faster. No need to copy a file, just run a one-liner. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18oeqa/sdk/python: clean up Python testRoss Burton
For the same reasons as the runtime Python test, clean up the SDK test. Also port from Python 2 to Python 3, as that's what is supported now. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-16testsdk: Enable multiprocess executionRichard Purdie
This uses the new concurrenttest code to enable parallel test execution if specified. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa/core/threaded: Remove in favour of using concurrenttestsRichard Purdie
We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/sdkbuildproject: Capture output to quieten selftest logsRichard Purdie
Only show the output in the failure case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-29oeqa/sdk/cases/buildgalculator.py: skip if gettext not availableChen Qi
We need to skip this testcase when gettext is not available. Otherwise, we will have the following error at configure. error: possibly undefined macro: AM_NLS Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-07-08oeqa/sdk: Replace buildiptables for buildlzip testsJose Perez Carranza
Buildiptables test cases are conflicting with images built with “musl” as standard C library, in order to avoid those issues lzip package was selected to be used on the tests as this does not have any "musl" dependency. [YOCTO #11713] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-12oeqa/sdk/cases: Fix skip of buildgcalculator testAníbal Limón
The tc.hasTargetPackage uses a re.search to see if gtk+3 is on the manifest but + in regex means 1 or more causing the test to be skipped. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/sdk/cases: Added validation for SDK compatibility tests with eSDKFrancisco Pedraza
The manifests for eSDK are generated using shared states so there is a need to validate to different "packages names" into the test cases. For example for perl: SDK provides nativesdk-perl eSDK provides perl-native [YOCTO #9090] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/sdk: Enable usage of OEQA thread modeAníbal Limón
oeqa/sdk/context.py - Use OETestContextThreaded. classes/testsdk.bbclass - Enable bb event thread mode to avoid corrupt the PIPE when multiple threads writes. [YOCTO #11450] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/sdk/context.py: Add return to OESDKTestContext.run() methodAníbal Limón
The run() methods of a OETestContext's are expected to return the results. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/sdk/context.py: Import argparse_oe at OESDKTestContext.run methodAníbal Limón
This import was at level of OESDKTestContext.register_commands but OESDKTestContext.run method need it to raise exceptions. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11oeqa/sdk/cases/buildcpio: enable use of cached cpio from DL_DIRJoshua Lock
All build project test cases will try to copy the requested source artefacts from DL_DIR before fetching. This testcase is referencing bzipped tarball whereas the recipe fetches a gzipped tarball. Switch to fetching the gzipped tarball in the test case so that we're able to use a cached tarball from DL_DIR Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21meta/lib/oeqa: change sdk test from cvs to cpiobrian avery
We currently fetch,configure,build, and install cvs as our test for the sdk. cvs unfortunately, requires a default editor in order to run. The change in 94790a8254d6 that checks to see if you have something like vi installed is fragile since you may have a different default editor. This patch switches from using cvs as a test to using cpio. cpio also uses autotools so the functionality tested is equivalent. Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-03-01oeqs/sdk*/case: Use universal_newlines for subprocess callsRichard Purdie
This removes the need for some of the ugly decode calls with hardcoded locales. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05oeqa/buildproject: Run gnu-configize by defaultRichard Purdie
The runtime oeqa tests were failing in the on-target case, only for aarch64. This was because an old config.sub was being used. Similar to the SDK testing, call gnu-configize in the on-target case too to resolve the failing tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31oeqa/sdk: Updates sanity tests for minimal eSDKFrancisco Pedraza
Updates sanity tests to cope with minimal eSDK installer 1. Skips the validation of sanity if packagegroup-cross-canadian is in host package. 2. Skips if SDK does not include toolchain at cases/devtool.py This should fix [YOCTO #10794] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23classes/oeqa: Replace subprocess.check_call() with check_output()Richard Purdie
If you use subprocess.check_output() the traceback will contain the output when the command fails which is very useful for debugging. There is no good reason not to use this everywhere. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23oeqa/sdkext/cases: Migrate test case to new OEQA frameworkAníbal Limón
Summary, - Changes base case class to OESDKExtTest. - Changes decorator classes to new ones. - Chnages variable names sdktestdir -> sdk_dir. - Added missing license to MIT. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23classes/testsdk: Remove the need of TEST_LOG_DIR variableAníbal Limón
The TEST_LOG_DIR was used for store sdk_target_log this log contains the output of the run of build commands now that information could be found also on log.do_testsdk under WORKDIR. The log will continue to store into SDK_DIR instead of TEST_LOG_DIR. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-01-23oeqa/sdk/cases: Migrate tests to the new OEQA frameworkAníbal Limón
Summary of the changes: - Remove auto extend_path using pkgutil at __init__, is not needed. - Change base class to OESDKTestCase. - Add td_vars attr to set dependencies of certain variables in test data. - Change skips from module level to class level because Test context (tc) now isn't at module level. - Variable names changes to be consistent (i.e. sdktestdir -> sdk_dir). [YOCTO #10599] - Don't use bb.utils functions use instead remove_safe and shutil for copy files. - SDKBuildProject pass test data variables instead of call getVar inside. [YOCTO #10231] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-06 14:02:25 +0000