aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib
AgeCommit message (Collapse)Author
2018-07-26oeqa/sdk/sdkbuildproject: use os.path.joinRoss Burton
Instead of mushing two paths together and hoping that the slashes line up correctly, use os.path.join. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26oeqa: rationalise Perl testsRoss Burton
As with the Python test, this can be both better and faster. No need to copy a file, just run a one-liner. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26logging: use warning instead warnChen Qi
The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-25selftest/bbtests: Add test for bitbake execution outside the build directoryRichard Purdie
Also fix a related test's error message to match what it does. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25selftest/bblayers: Place the test layer directory in builddirRichard Purdie
Placing the layer in meta means the directory is in an unclean state which may influence other tests. Use our build directory instead since we 'own' that. This helps keep oe-selftest parallelisation clean. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25selftest/wic: clean up runCmd error code handlingRichard Purdie
runCmd handles checking exit 0 internally unless specified otherwise and its error messages are much more useful including the failed command output. Clean up the unneeded 0 exit code checks in the wic tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25selftest/wic: Improve error message for test_fixed_sizeRichard Purdie
Currently this can fail with a message like 127 != 0 which is unhelpful. If we remove the ignore_status=False, the debugging from runCmd is much more helpful printing status.output. Also remove the now unneeded exit code check. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24oe-selftest: fix for changes to buildhistory-diffPaul Eggleton
Now we're not reporting the related fields (as of openembedded-core 8658b3677b9f7cb70806061c41570c709086ef05) we shouldn't expect to see PR reported here since it's not monitored by buildhistory-diff. However, with a bit of messing about we can check for the exact output that we should now see as a result of the test changing PR to go backwards. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24lib/oe/utils: Drop now unused multiprocess_execRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package/package_manager: multiprocess_exec -> multiprocess_launchRichard Purdie
After this replacement, the parent exception handling works so we don't need subprocess wrapping with bb.error in the underlying functions. The underlying contexts also have better module handling so the imports can be cleaned up. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Call file to determine elf status in parallelRichard Purdie
This allows the calls to is_elf (which calls file) to happen in parallel allowing a speedup of do_package and do_populate_sysroot for native recipes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Refactor to remove isElf/is_elf function duplicationRichard Purdie
There are probably further cleanups needed here but this at least removes the major code duplication between these two similar funcitons, keeping the kernel module ".ko" extension check for efficiency to avoid opening and reading file contents in the general case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24package: Don't use subshell to execute fileRichard Purdie
We don't need any functionality from the shell here, its just extra fork overhead. Therefore remove it and use subprocess directly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24utils: Add multiprocess_launch API and testcaseRichard Purdie
The current methods of spawning processes for parallel execution have issues around collection of results or exceptions. Take the code from package_ipk/deb, make it generic, add a results collection mechanism, fix the exception handling and for it into a standard library function. Also add a test case which tests both the success and failure modes of operation to stop this functionality regressiing again. In particular, compared to multiprocess_exec, this fork off the parent approach means we can pass in the datastore and functions work in the same scope as the parent. This removes some of the complexities found trying to scale multiprocess_exec to wider use. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18oeqa/sdk/python: clean up Python testRoss Burton
For the same reasons as the runtime Python test, clean up the SDK test. Also port from Python 2 to Python 3, as that's what is supported now. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18oeqa/runtime/python: clean up Python testRoss Burton
Currently this is three test cases: 1) test_python_exists. Fail if python3 isn't in PATH. 2) test_python_stdout. Run a Python script and check the output is as expected 3) test_python_testfile. Check that a file test_python_stdout wrote to exists. (1) should be a setup and skip the test module if it isn't present. (2) and (3) should be merged, there's no point copying over a two line Python file, and the test doesn't verify that the file doesn't exist in the first place. Rewrite the test to check that Python is present in a class setup so the entire test is skipped if it isn't and do some simple rot13 to verify that bytecode is being executed correctly. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18classes/buildhistory: handle packaged files with names containing spacesPaul Eggleton
The FILELIST field of the package info file in the buildhistory repository is a space-separated list of all of the files in the package. If a name of a file packaged by a recipe contains a space character then of course the result was that we didn't handle its name properly. To fix that, use quotes around any filename containing spaces and at the other end use these quotes to extract the proper entries. Fixes [YOCTO #12742]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18lib/oe/buildhistory_analysis: drop related field featurePaul Eggleton
The original idea here was that changes to certain fields might be able to be explained if there was a change to another field, for example if RDEPENDS changed it might be because DEPENDS changed. Thus we were printing this kind of thing out with each change. Unfortunately in practice this turned out to be noisy and not particularly useful, so we might as well remove it. Fixes [YOCTO #7336]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18oeqa/parselogs: remove references to BSPs that no longer existAnuj Mittal
These platform specifc BSPs were removed from meta-intel and superseded by intel-core* BSPs. Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-18oeqa/decorator: Improve reliabilityRichard Purdie
Checking if the dependency had any failure is unreliable, for example if the underlying data doesn't get transferred and the list is empty, success of the dependency is assumed. Since we now have success data available, change the code to use it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16testsdk: Enable multiprocess executionRichard Purdie
This uses the new concurrenttest code to enable parallel test execution if specified. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa: Remove xmlrunnerRichard Purdie
This isn't present on modern distros by default and doesn't work with testtools, needing multiple code paths in the code. Remove it in favour of finding a better replacement for results collection/analysis. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa/runner: Simplify codeRichard Purdie
There doesn't appear to be any reason we need this _results indirection any more so remove it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa/core/threaded: Remove in favour of using concurrenttestsRichard Purdie
We have several options for parallel processing in oeqa, parallel execution of modules, threading and mulitple processes for the runners. After much experimentation is appears the most scalable and least invasive approach is multiple processes using concurrenttestsuite from testtools. This means we can drop the current threading code which is only used by the sdk test execution. oeqa/decorator/depends: Remove threading code Revert "oeqa/sdk: Enable usage of OEQA thread mode" This reverts commit adc434c0636b7dea2ef70c8d2c8e61cdb5c703b1. Revert "oeqa/core/tests: Add tests of OEQA Threaded mode" This reverts commit a4eef558c9933eb32413b61ff80a11b999951b40. Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode" This reverts commit d3d4ba902dee8b19fa1054330cffdf73f9b81fe7. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-16oeqa: Add selftest parallelisation supportRichard Purdie
This allows oe-selftest to take a -j option which specifies how much test parallelisation to use. Currently this is "module" based with each module being split and run in a separate build directory. Further splitting could be done but this seems a good compromise between test setup and parallelism. You need python-testtools and python-subunit installed to use this but only when the -j option is specified. See notes posted to the openedmbedded-architecture list for more details about the design choices here. Some of this functionality may make more sense in the oeqa core ultimately. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/selftest/wic: Split tests into two class groupsRichard Purdie
This improves test parallelism. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/sdkbuildproject: Capture output to quieten selftest logsRichard Purdie
Only show the output in the failure case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15devtool: Split tests into multiple classesRichard Purdie
This allows better parallelism between the different tests as currently this block takes the longest time to execute. devtool tests are still all grouped into the "devtool" module for ease of exection. This also makes it easier to execute some subset of devtool tests for testing devtool changes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa: Default to buffer mode for testsRichard Purdie
Currently some tests run in buffer mode and some don't. Those that don't can corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent. If there is useful output on stdout/stderr, it will be displayed if the test fails. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/selftest/wic: Ensure initramfs exists for test_iso_imageRichard Purdie
AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal -o /var/tmp/wic.oe-selftest/' returned non-zero exit status 1: ERROR: _exec_cmd: gzip -f -9 -c /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio > /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio.gz returned '1' instead of 0 output: gzip: /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio: No such file or directory This is because in a clean build directory, the initramfs may not be rebuilt. Add a call to ensure it is built to avoid the error. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/selftest/wic: Use a subdir of builddir, not /var/Richard Purdie
Using /var/ leave wic open to races with other processes on the system, use a subdir of builddir instead to avoid this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/selftest/devtool: Ensure dbus is built befoe running testRichard Purdie
If dbus isn't build first the dbus dependency of dbus-wait can't be detected through pkgconfig and the test fails: AssertionError: {'DEPENDS': {'dbus'}} != {} - {'DEPENDS': {'dbus'}} + {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}} Ensure dbus is built and present in the sysroot. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-15oeqa/selftest/package: Use setUpLocal, not setUpRichard Purdie
The main setUp function needs to be called and tests are meant to define setUpLocal. For some reason this one didn't leading to errors with recent code changes. Fix it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-13oeqa/runner: Ensure we don't print misleading results outputRichard Purdie
The current code assumes if something isn't a failure of some kind, it was a pass. When test case IDs weren't matching, this lead to very confusing output where things would fail, then be listed as passing. This adds code to track successes, ensuring we don't end up in this position again with unmatched entries being listed as UNKNOWN. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-13oqqa/selftest/lic_checksum: Avoid cache reparseRichard Purdie
By having a consistent config for the test, a second reparse is avoided and the test runs slightly faster. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-13oeqa/selftest/image_tyypedep: Avoid undefined variable failureRichard Purdie
If something goes wrong with the test, dep can be undefined. Avoid a traceback for this and handle it more gracefully. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-13oeqa/core/runner: Improve test case comparisionRichard Purdie
We can directly compare the test case IDs rather than representations, then if we're using subunit to split the tests, the comparisions still work as intended. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-07rootfs-postcommands: split ssh_allow_empty_passwordJackie Huang
"allow root login" should not be bundled in ssh_allow_empty_password, because some distro may want only one of "allow root login" and "allow empty password", so split it out into ssh_allow_root_login and add new imagefeature allow-root-login so they can be controlled separately, debug-tweaks will still include both of them. Signed-off-by: Jackie Huang <jackie.huang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-07oe/copy_buildsystem.py: add SDK_LAYERS_EXCLUDE_PATTERNRobert Yang
It is helpful when exclude a lot of layers. It uses python re, and supports multiple patterns (separated by space). Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06sstatesig: Remove trailing whitespaceJoshua Watt
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05oeqa/runtime/cases/parselog.py: ignore a message from westonChen Qi
The following error message when starting core-image-weston is not critical as long as the image could start up correctly. So extend the common_errors list for parselog.py test case to ignore this message. logind: cannot setup systemd-logind helper (-61), using legacy fallback [YOCTO #12835] Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-05oe.types.boolean: treat None as FalseBinghua Guan
It is better to return False for None. E.g. checking an undefined variable returned d.getVar(). Signed-off-by: Binghua Guan <freebendy@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-02oe.path: add which_wild functionChristopher Larson
This is a function much like shutil.which or bb.utils.which, retaining shutil.which-like function semantics, bb.utils.which's support for returning available candidates for signatures, and most importantly, supports wildcards, returning only the first occurrance of each found pathname in the search path. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-02lib/oe/sstatesig: Fix task mappings from multilib<->non-multilib contextsRichard Purdie
If we're in a multilib context already and want a non-multilib context this function returned incorrect values. Try and retain optimisations for the common case not needing to request a datastore but allow the different multilib/non-multilib combinations to work too. This fixes bugs where rootfs generation of a multilib image would write into incorrect locations, or be unable to find sstate manifest files due to incorrect data stores being used to expand data. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02lib/oe/utils: Improve get_multilib_datastoreRichard Purdie
Currently this function assumes that no multilib is applied and that we're applying a multilib. This means if we're in multilib context and want the non-multilib context we can't obtain it (and no other function exists for this either). Improve the function to allow this to be requested. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02oe-selftest: Add bitbake-layer create-layer testJoshua Watt
Adds a test that creates a layer, adds it to bblayers.conf, then ensure that it appears properly in bitbake-layers show-layers. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02postinst-intercepts: do not execute any variant of delay_to_first_bootJoe Slater
As of commit 2c5c6e3ff we create multilib variants of intercept hooks but we did not account for delay_to_first_boot variants. This was covered up until commit a335e7867, but will now cause an error. Signed-off-by: Joe Slater <joe.slater@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02oeqa/runtime: Add testcases for kernel sampleHongzhi.Song
We are going to let runtime test support kernel tests. Now we just add kernel self-contained sample tests. And we plan to add overall kernel tests in the future. This patch is just add kernel samples test which contains about 13 tests enabled by kernel-sample.scc. So it needs statement, KERNEL_FEATURES_append += " features/kernel-sample/kernel-sample.scc" in local.conf. Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28bitbake-bblayers/create: Fix layer name generationJoshua Watt
The path to where the layer was being created was taken verbatim as the name of the layer when generating the layer.conf and README files from templates. This causes problems in the layer.conf file because it would result in strangely named variables like BBFILE_PATTERN_../my-layer = "..." Instead of blindly taking the path, use the name of the last component of the path as the layer name. Additionally, rework the template files to use python format strings with named parameters so that the same argument doesn't have to be repeated multiple times. [YOCTO #12808] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-27oeqa: add Git mirror testRoss Burton
Add an automated test to exercise that fetching a git: recipe... 1) works over git without mirrors 2) fails without git connectivity or mirrors 3) works without git connectivity but with a mirror (2) is done by setting GIT_PROXY_COMMAND to 'false' which should break any git network operations. [ YOCTO #12805 ] Signed-off-by: Ross Burton <ross.burton@intel.com>