summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/sdk
AgeCommit message (Collapse)Author
2022-04-26cases/buildepoxy.py: fix typoChen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-18buildtools-docs-tarball: Add test for building documentation using sphinxRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-12oeqa/buildtools: Switch to our webserver instead of example.comRichard Purdie
Similarly to the sanity connectivity check, switch away from example.com for the buildtools tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-16oeqa: Update cleanup code to wait for hashserv exitRichard Purdie
We sometimes see exceptions from code seeing the hashserv DB files being removed at directory cleanup time. Add a check to ensure the hashserv has written the data base journal (and hence likely exited) before cleaning up. This will hopefully avoid errors like: Traceback (most recent call last): File "[...]/meta/lib/oeqa/sdk/buildtools-cases/build.py", line 30, in test_libc delay = delay - 1 File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__ self.cleanup() File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup _rmtree(self.name) File "/usr/lib/python3.6/shutil.py", line 486, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/usr/lib/python3.6/shutil.py", line 424, in _rmtree_safe_fd _rmtree_safe_fd(dirfd, fullname, onerror) File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) FileNotFoundError: [Errno 2] No such file or directory: 'hashserv.db-wal' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-01oeqa/buildtools-cases: Allow bitbake time to shutdownRichard Purdie
bitbake may still be shutting down when the UI exits. Wait for the lock to disappear before trying to delete the directory to avoid errors. Traceback (most recent call last): File "/home/pokybuild/yocto-worker/buildtools/build/meta/lib/oeqa/sdk/buildtools-cases/build.py", line 23, in test_libc self._run('. %s/oe-init-build-env %s && bitbake virtual/libc' % (corebase, testdir)) File "/usr/lib64/python3.7/tempfile.py", line 807, in __exit__ self.cleanup() File "/usr/lib64/python3.7/tempfile.py", line 811, in cleanup _shutil.rmtree(self.name) File "/usr/lib64/python3.7/shutil.py", line 494, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/usr/lib64/python3.7/shutil.py", line 452, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/usr/lib64/python3.7/shutil.py", line 450, in _rmtree_safe_fd os.unlink(entry.name, dir_fd=topfd) FileNotFoundError: [Errno 2] No such file or directory: 'bitbake.sock' Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-18oeqa/sdk: add HTTPS test for buildtoolsRoss Burton
Verify that HTTPS connections work in both wget and Python, as this depends on variables correctly pointing to the certificate chain. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-18oeqa/sdk: add relocation test for buildtoolsRoss Burton
The buildtools-extended tarball includes GCC, which relies on being relocated correctly to work. Add a test case that verifies that the loader paths have all been relocated, as otherwise there are hard-to-debug errors at build time. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12oeqa/sdk: add some buildtools testsRoss Burton
These two tests are designed to exercise the buildtools-tarball. SanityTests simply verifies that inside the SDK, some commands are used from the SDK. BuildTests creates a new OE build directory and builds virtual/libc to verify that a basic build works correctly. DL_DIR is reused to avoid needless downloading, but sstate is not shared to ensure a build does happen. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-02oeqa: sdk: Capture stderr outputJoshua Watt
Redirect stderr to stdout when running subcommands while doing the SDK tests. The tests will show stdout when CalledProcessError is raised, but any output to stderr was lost. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-18buildgalculator: Patch to fix build with -fno-commonKhem Raj
We need to apply a patch to galculator for it to build with gcc-10+ Remove double definition of 'prefs' variable Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-18buildcpio.py: Apply patch to fix build with -fno-commonKhem Raj
This sed expression implements the needed patch to fix compilation with -fno-common, hopefully this patch will get included in 2.14 release and we can remove this operation Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18fix various, innocuous typosRobert P. J. Day
Corrections: - environment - accommodate - conversion - compatible Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14oeqa: Use --disable-maintainer-mode configure optionKhem Raj
since the versions of autotools might differ on target and build host, plus difference in timestamps for configure and system can result in reconfigure lets avoid that by disabling maintainer mode Avoids error: newly created file is older than distributed files! [YOCTO #13779] Suggested-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08oeqa: Use cpio 2.13 as testcaseKhem Raj
cpio 2.12 was released in 2015 and might have used older autotools which could result in errors like https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779 Bumping to 2.13 will help in matching the tool versions A good change on top would be to run aclocal -I .; autoheader; autoconf; automake --add-missing -c before running configure step perhaps [YOCTO #13779] Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08oeqa/sdk: improve Meson testRoss Burton
Verify that the build inside the SDK is detected as a cross compilation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09oeqa/sdk: use bash to execute SDK test commandsMikko Rapeli
The commands only work with with bash. If /bin/sh is dash like in Debian, the command execution fails with errors like: Standard Output: /bin/sh: 5: export: --sysroot: bad variable name and all SDK tests fail. Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk: add test to exercise MesonRoss Burton
(From OE-Core rev: 29359493e391d68a5a6b4fa4d09ffdc1fe6db620) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07oeqa/sdk/assimp: cleanupRoss Burton
Unify style with the other tests. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-07oeqa/sdk/cases: clean up DL_DIR handlingRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-12-13oeqa/sdk: rewrite cpio testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: rewrite lzip testRoss Burton
Don't use the helper class as it gets in the way more than it helps, exercise the out-of-tree paths, and verify the installed files match the expected architecture. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clean up galculator testRoss Burton
Drop redundant imports and variables, and use os.makedirs() instead of bb.utils.mkdirhier(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: show output if run() failsRoss Burton
Use oeqa.utils.subprocesstweak to monkey-patch the subprocess exception so that any output is shown, and remove any explicit try/catch handling that would have hidden this. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13oeqa/sdk: clarify ELF assertion messageRoss Burton
For example, instead of saying "3 != 62", say "Binary was x86-64 but expected i586". Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08oeqa/sdk/galculator: rewrite to use new helpersRoss Burton
2018-12-08oeqa/sdk/python: fix version typoRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-12-08oeqa/sdk/assimp: use helpersRoss Burton
2018-12-08oeqa/sdk/case: add fundamental helper methodsRoss Burton
2018-12-03oeqa/sdk/python: add Python 2 and fix detectionRoss Burton
Add a Python 2 form to exercise that if present, and fix the setUp() so it actually looks for a package that exists (nativesdk-python3 is a virtual package, the interpretter is in nativesdk-python3-core). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20classes/testsdk: Split implementation into classesJoshua Watt
Splits the SDK test implementation into configurable Python classes. The classes used for the normal and extensible SDKs are ${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively. This allows SDK machines to override the classes used to implement the tests. For the traditional SDK, a common "run()" function is provided by the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions that can be overridden in child classes, making it easier to have consistent behavior. The extensible SDK class (oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but no hooks have yet been added as there is not currently a known use case for create derived classes. These changes should be purely organizational; no functional changes have been made to either the standard SDK or extensible SDK tests. [YOCTO #13020] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-13testsdk: fix skipped testcase output "UNKNOWN" status while multiprocess ↵Hongxu Jia
execution Usually skipped testcase output "SKIPPED" [snip serial execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: SKIPPED (0.01s) |RESULTS - python.PythonTest.test_python3 - Testcase -1: SKIPPED (0.01s) [snip serial execution] But if enable multiprocess execution, skipped testcase output "UNKNOWN" status [snip enable multiprocess execution] |RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: UNKNOWN |RESULTS - python.PythonTest.test_python3 - Testcase -1: UNKNOWN [snip enable multiprocess execution] Here is my investigation: There is a class pairs TestProtocolClient and TestProtocolServer provided by python3-subunit. The TestProtocolClient generates a subunit stream of TestResult from a test run, and TestProtocolServer parses the stream of subunit TestResult. The class ProtocolTestCase is a unittest.TestCase adapter and it uses TestProtocolServer to parse the stream of subunit TestResult. In Yocto testsdk, it forks multiple processes to execute testcases and use TestProtocolClient to generate TestResult stream; and then it creates multiple threads to use ProtocolTestCase to parse stream of subunit TestResult through pipe; finally it passes multiple ProtocolTestCase as TestCase instance to main process and output status result. The problem point is TestProtocolServer parses `skip:' directive after reading a `test:' directive. Without `test:' directive, `skip:' directive will be ignored. All above requires SkipTest should be raised inside a test method rather than setUpClass method. Throwing SkipTest inside setUp works correctly Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30oeqa/sdk: fixes related to hasPackage semanticsChen Qi
The current _hasPackage does a regex match when checking for the existence of packages. This will sometimes result in unexpected result. For example, the condition hasTargetPackage('gcc') is likely to be always true as it matches libgcc1. For most of the time, we should do exact match instead of regex match. So change _hasPackage function to do that. For the current sdk test cases, the only place that needs regex match is '^gcc-'. This is because there's no easy way to get multilib tune arch (e.g. i686) from testdata.json file. Besides, packagegroup-cross-canadian-xxx and gcc-xxx should be check in host manifest instead of the target one. So fix to use hasHostPackage. Also, as we are doing exact match, there's no need to use r'gtk\+3', just 'gtk+3' is enough. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30sdk/buldgalculator.py: check against multilib for gtk+3Chen Qi
When determining whether to skip the test case, the check should be done with consideration of multilib. Otherwise, we will meet the following error when testing against lib32 environment. No package 'gtk+-3.0' found Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30sdk/context.py: add ability to check for multilib version of target packageChen Qi
Add a named argument 'multilib' for the hasTargetPackage function. Its default value is False. When setting to True, it will try to get the correct multilib prefix from the sdk_env, the environment setup script. We need this because we don't want unexpected run of some sdk test cases. The following steps will generate error. 1. Enable multilib for qemux86-64 require conf/multilib.conf MULTILIBS ?= "multilib:lib32" DEFAULTTUNE_virtclass-multilib-lib32 ?= "core2-32" 2. bitbake core-image-sato -c populate_sdk 3. bitbake core-image-sato -c testsdk The error message is like below. No package 'gtk+-3.0' found RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: FAILED As we don't have lib32-gtk+3 installed, the test case should be skipped when testing against the lib32 environment setup script. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-21assimp.py: fix AttributeError in tearDownClassChen Qi
When running this test case, we will see the following error. AttributeError: type object 'BuildAssimp' has no attribute 'project' assimp.py test case does not make use of SDKBuildProject, so remove the import statement and the tearDownClass. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-10oeqa/sdk/buildgalculator: check for nativesdk-gettext-devRoss Burton
We don't need target gettext to build, but nativesdk-gettext-dev (for nls.m4). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07oeqa/sdk: add test that CMake worksRoss Burton
Add a new SDK testcase that builds assimp, a project that uses cmake. Using TARGET_ARCH and TARGET_OS which is now exported into the environment, check that the generated binaries match the ELF headers we expect. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26oeqa/sdk/sdkbuildproject: use os.path.joinRoss Burton
Instead of mushing two paths together and hoping that the slashes line up correctly, use os.path.join. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26oeqa: rationalise Perl testsRoss Burton
As with the Python test, this can be both better and faster. No need to copy a file, just run a one-liner. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18oeqa/sdk/python: clean up Python testRoss Burton
For the same reasons as the runtime Python test, clean up the SDK test. Also port from Python 2 to Python 3, as that's what is supported now. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-16testsdk: Enable multiprocess executionRichard Purdie
This uses the new concurrenttest code to enable parallel test execution if specified. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa/core/threaded: Remove in favour of using concurrenttestsRichard Purdie
We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/sdkbuildproject: Capture output to quieten selftest logsRichard Purdie
Only show the output in the failure case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-29oeqa/sdk/cases/buildgalculator.py: skip if gettext not availableChen Qi
We need to skip this testcase when gettext is not available. Otherwise, we will have the following error at configure. error: possibly undefined macro: AM_NLS Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-07-08oeqa/sdk: Replace buildiptables for buildlzip testsJose Perez Carranza
Buildiptables test cases are conflicting with images built with “musl” as standard C library, in order to avoid those issues lzip package was selected to be used on the tests as this does not have any "musl" dependency. [YOCTO #11713] Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-12oeqa/sdk/cases: Fix skip of buildgcalculator testAníbal Limón
The tc.hasTargetPackage uses a re.search to see if gtk+3 is on the manifest but + in regex means 1 or more causing the test to be skipped. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/sdk/cases: Added validation for SDK compatibility tests with eSDKFrancisco Pedraza
The manifests for eSDK are generated using shared states so there is a need to validate to different "packages names" into the test cases. For example for perl: SDK provides nativesdk-perl eSDK provides perl-native [YOCTO #9090] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/sdk: Enable usage of OEQA thread modeAníbal Limón
oeqa/sdk/context.py - Use OETestContextThreaded. classes/testsdk.bbclass - Enable bb event thread mode to avoid corrupt the PIPE when multiple threads writes. [YOCTO #11450] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-30oeqa/sdk/context.py: Add return to OESDKTestContext.run() methodAníbal Limón
The run() methods of a OETestContext's are expected to return the results. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-13 09:51:06 +0000