| Age | Commit message (Collapse) | Author |
|---|
|
Add some tests for functions in meta/lib/oe/recipeutils.py, in
particular for a few issues I've just fixed. I haven't added tests for
all of the functions - some of them are already being tested via devtool
in any case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This avoids problems where shutil.remove will error with:
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
when there are races over file deletion (gpg agent may be slow to exit).
We already worked around speed and race issues in bb.utils.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This should speed the test up signficiantly without any loss of functionality
for the purposes of the test.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We ideally don't want to use gpg from the host. This is straightforward for package
management but not for sstate.
For sstate, create a second build directory to run the test in using gnupg-native
from the original build directory.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Raising an assertionError in the class setup isn't a particuarly good way to
indicate gpg isn't installed. Instead skip the tests if the required binary
isn't present. For the signing tests we do require it to be present and can't
use a prebuilt one.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
warnings
Use our own lsb function instead as used elsewhere by the codebase.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid the warning:
meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
In this case importlib is a direct replacement.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The upstream source tarball checksums changed. Use the copy from our source
mirror to avoid failures.
[YOCTO #12979]
Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
break tests
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently sdk & sdkext will output json file to LOG_DIR, while
selftest will output json file to TOPDIR/log.
Standardize selftest json output file to LOG_DIR.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've had a number of occasions where the Yocto Project source mirrors have not
been complete or functioning correctly. This adds a test so that if this happens
we find out out it sooner.
It also only works over http meaning we should be able to test that anyone behind
an http only proxy (no git protocol) also has functional fetches for OE-Core and
layers built by the core of the project.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tweak the preceeding commit to:
* Add STARTTIME to the identifier to make it unique
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As part of the solution to replace Testopia to store testresult,
OEQA selftest need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.
By default, json testresult file will be written to "oeqa"
directory under the oe-selftest log directory.
To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
oe-selftest's devtool tests have been broken since commit 2457cd57
(oe-selftest: devtool: avoid parallel races by using temporary copy of
core) if meta is a symbolic link.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It appends TCLIBCAPPEND to TMPDIR in meta/conf/distro/defaultsetup.conf:
TMPDIR .= "${TCLIBCAPPEND}"
It affects some oe selftest cases in sstatetests.py. Reset TCLIBCAPPEND
for these cases.
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The sigchld handler in runqemu can raise a SystemExit when qemu shuts down.
Rather than backtracing, accept this as a successful test result.
ERROR: runqemu.QemuTest.test_qemu_can_shutdown (subunit.RemotedTestCase)
testtools.testresult.real._StringException: Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 183, in test_qemu_can_shutdown
qemu_shutdown_succeeded = self._start_qemu_shutdown_check_if_shutdown_succeeded(qemu, shutdown_timeout)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/cases/runqemu.py", line 175, in _start_qemu_shutdown_check_if_shutdown_succeeded
time.sleep(1)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/qemurunner.py", line 100, in handleSIGCHLD
raise SystemExit
SystemExit
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The Navit recipe can depend on fribidi which is now part of oe-core. Since the
pkgconfig lookup is based on pkgdata the results can change depending on what
has been built, which leads to occasional failures.
Build gtk+ before the test to ensure that the pkgdata is populated with more of
the dependencies, and add fribidi to the generated DEPENDS checklist.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without this, we see errors if gpg is missing from the host system
for "oe-selftest -r runtime_test.TestImage.test_testimage_dnf".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BPF Linux ELF objects are generated with kernel-selftests with
>= 4.18 kernel and when clang is enabled which packages BPF objects
into packages, therefore recongnise this as a valid ELF target
Add a selftest for BPF
Do not flag BPF objects in target, since they pretty much will be ok for
most of kernels architectures we care do support BPF
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
allarch is disabled when multilib is used, so sstate oeqa case
test_sstate_allarch_samesigs_multilib is useless. Remove check for
allarch part and rename to test_sstate_nativesdk_samesigs_multilib.
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
runtime_test.TestImage.test_testimage_install after ssl changes
The test installs socat and expects it to run but assumes all its dependencies
are already installed (which includes libssl).
Recent changes mean this isn't the case so force libssl into the image in advance
to work around this issue for now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sumo release provides a transition period so that deferrals to first boot
via 'exit 1' can be converted to pkg_postinst_ontarget(). For the next release
however, postinst script failures should be treated as such.
[YOCTO #12607]
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tweak recipe selftest-hardlink
- addition of libexecdir to simulate multiple directories
- add gdb.sh to run gdb from script which is invoked at test time.
- rename `hello' -> `hello1' to workaround name confliction with the one in lmbench
Add test_gdb_hardlink_debug to selftest/package
- run a qemu and invoke gdb.sh to gdb binaries of selftest-hardlink
- check gdb to read symbols from separated debug hardlink file
- check debug symbols works correctly
[Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory']
2018-08-26 01:27:30,195 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests)
2018-08-26 01:30:29,005 - oe-selftest - INFO - gdbtest /usr/bin/hello1
2018-08-26 01:30:36,539 - oe-selftest - INFO - gdbtest /usr/bin/hello2
2018-08-26 01:30:43,568 - oe-selftest - INFO - gdbtest /usr/libexec/hello3
2018-08-26 01:30:50,157 - oe-selftest - ERROR - No debugging symbols found. GDB result:
Reading symbols from /usr/libexec/hello3...(no debugging symbols found)...done.^M
(gdb) Function "main" not defined.^M
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]^M
(gdb) Starting program: /usr/libexec/hello3 ^M
Hello World!^M
[Inferior 1 (process 320) exited normally]^M
(gdb) The program is not being run.^M
(gdb)
2018-08-26 01:30:51,180 - oe-selftest - INFO - ... FAIL
2018-08-26 01:30:51,181 - oe-selftest - INFO - Traceback (most recent call last):
File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 148, in test_gdb_hardlink_debug
self.fail('GDB %s failed' % binary)
AssertionError: GDB /usr/libexec/hello3 failed
[Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory']
[Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory']
2018-08-26 12:40:30,976 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests)
2018-08-26 12:42:15,149 - oe-selftest - INFO - gdbtest /usr/bin/hello1
2018-08-26 12:42:24,064 - oe-selftest - INFO - gdbtest /usr/bin/hello2
2018-08-26 12:42:31,078 - oe-selftest - INFO - gdbtest /usr/libexec/hello3
2018-08-26 12:42:38,646 - oe-selftest - INFO - gdbtest /usr/libexec/hello4
2018-08-26 12:42:46,824 - oe-selftest - INFO - ... ok
[Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory']
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The sparseness test was sometimes working and sometimes failing depending
on whether sstate was valid. This adds an explict test of sstate
to the test for both hardlinking and sparseness. Tweak the test name to
cover the fact its tests sparseness too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Parsing all the recipes is annoying when trying to re-execute oe-selftest
and also unnecessary as its really just a sanity check. When the tests were
originally being developed the guard was useful but less so now.
Replace it with bitbake -e which is fast and checks the basic configuration
is valid.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a new element to the hardlink test to check we also preseve file
sparseness during the packing process. This should ensure we don't regress this
issue again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The build target won't cause the package task to run if things were already
built and in sstate. Ensure we run the package task explicitly to ensure
the test works as intended.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some of the devtool tests make changes to files under meta/ -
legitimately since we want these tests to be working with real recipes
and associated files. Unfortunately with the new oe-selftest
parallelisation this can break other tests if files go missing at the
wrong time (among other scenarios). To avoid this issue, simply take a
copy of the core repository and use that for these tests. (We copy the
entire repository since changing the path of meta/ influences COREBASE
and thus we need to have things like scripts/ alongside as well).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
We keep breaking the preservation of hardlinks during the packaging process.
Add a selftest which tests this to try and prevent this breaking again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
rm_work if enabled leads to some tests failing that rely on artifacts
being present. Check if rm_work.bbclass is included and show an error
and exit if it is.
Fixes [YOCTO #12694]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The code was using Wic instead of WicTestCase for image_is_ready and
native_sysroot which was confusing some tests. Fix image_is_ready.
For native_sysroot, convert the only user to just run the call itself
as there is no point in executing this for all the other tests which
don't use the value.
Whilst here, move _get_image_env_path to the base class alongside
its parent variabe and fix it to use WicTestCase.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The warn method is deprecated. We should use the documented warning instead.
Quoting from the python's official doc:
"""
Note: There is an obsolete method warn which is functionally identical to warning.
As warn is deprecated, please do not use it - use warning instead.
"""
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Also fix a related test's error message to match what it does.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Placing the layer in meta means the directory is in an unclean state
which may influence other tests. Use our build directory instead
since we 'own' that. This helps keep oe-selftest parallelisation
clean.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
runCmd handles checking exit 0 internally unless specified otherwise and its
error messages are much more useful including the failed command output.
Clean up the unneeded 0 exit code checks in the wic tests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently this can fail with a message like 127 != 0 which is unhelpful.
If we remove the ignore_status=False, the debugging from runCmd is much
more helpful printing status.output.
Also remove the now unneeded exit code check.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now we're not reporting the related fields (as of openembedded-core
8658b3677b9f7cb70806061c41570c709086ef05) we shouldn't expect to see
PR reported here since it's not monitored by buildhistory-diff. However,
with a bit of messing about we can check for the exact output that we
should now see as a result of the test changing PR to go backwards.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current methods of spawning processes for parallel execution have
issues around collection of results or exceptions.
Take the code from package_ipk/deb, make it generic, add a results
collection mechanism, fix the exception handling and for it into a
standard library function.
Also add a test case which tests both the success and failure modes
of operation to stop this functionality regressiing again.
In particular, compared to multiprocess_exec, this fork off the parent
approach means we can pass in the datastore and functions work in the
same scope as the parent. This removes some of the complexities
found trying to scale multiprocess_exec to wider use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows oe-selftest to take a -j option which specifies how much test
parallelisation to use. Currently this is "module" based with each module
being split and run in a separate build directory. Further splitting could
be done but this seems a good compromise between test setup and parallelism.
You need python-testtools and python-subunit installed to use this but only
when the -j option is specified.
See notes posted to the openedmbedded-architecture list for more details
about the design choices here.
Some of this functionality may make more sense in the oeqa core ultimately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This improves test parallelism.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows better parallelism between the different tests as currently
this block takes the longest time to execute. devtool tests are still
all grouped into the "devtool" module for ease of exection.
This also makes it easier to execute some subset of devtool tests for
testing devtool changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently some tests run in buffer mode and some don't. Those that don't can
corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent.
If there is useful output on stdout/stderr, it will be displayed if the test
fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal -o /var/tmp/wic.oe-selftest/' returned non-zero exit status 1:
ERROR: _exec_cmd: gzip -f -9 -c /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio > /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio.gz returned '1' instead of 0
output: gzip: /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio: No such file or directory
This is because in a clean build directory, the initramfs may not be rebuilt.
Add a call to ensure it is built to avoid the error.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using /var/ leave wic open to races with other processes on the system, use
a subdir of builddir instead to avoid this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If dbus isn't build first the dbus dependency of dbus-wait can't be detected
through pkgconfig and the test fails:
AssertionError: {'DEPENDS': {'dbus'}} != {}
- {'DEPENDS': {'dbus'}}
+ {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}}
Ensure dbus is built and present in the sysroot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The main setUp function needs to be called and tests are meant to
define setUpLocal. For some reason this one didn't leading to errors
with recent code changes. Fix it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By having a consistent config for the test, a second reparse is avoided
and the test runs slightly faster.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If something goes wrong with the test, dep can be undefined. Avoid a
traceback for this and handle it more gracefully.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"allow root login" should not be bundled in ssh_allow_empty_password,
because some distro may want only one of "allow root login" and "allow
empty password", so split it out into ssh_allow_root_login and add new
imagefeature allow-root-login so they can be controlled separately,
debug-tweaks will still include both of them.
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Paul Eggleton
Richard Purdie
Mohamad Noor Alim Hussin
Yeoh Ee Peng
Peter Kjellerstedt
Kai Kang
Ross Burton
Khem Raj
Alexander Kanavin
Hongxu Jia
Anuj Mittal
Chen Qi
Jackie Huang