| Age | Commit message (Collapse) | Author |
|---|
|
We've seen issues with rootfs size calculations and we've seen systems
like opensuse which have btrfs mounted on /tmp causing selftest failures.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we need to create a temporary directory in targetbuild or buildproject use
tempfile.TemporaryDirectory so that when the test case is finished, the
directory is deleted.
Also synchronise the logic and don't possibly store the temporary directory in
self.tmpdir as nothing uses that.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reorder the shutdown/teardown to avoid:
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 224, in launch
op = self.getOutput(output)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/qemurunner.py", line 90, in getOutput
fl = fcntl.fcntl(o, fcntl.F_GETFL)
ValueError: I/O operation on closed file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
assertTrue doesn't give good debug information when things fail. Update
several to use assertIn which gives information upon failure, for the
others print the log information upon failure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen a cryptic:
"ERROR: Fatal errors occurred in subprocesses, tracebacks printed above"
message from oe-selftest with no other traceback information. Improve the
traceback logging to try and give a better indication of any errors that is
ocurring.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
setUp() is used to populate a directory of temporary files, and deleted in
__del__. However setUp() is called once *per test* so __del__ would only be
able to remove the last directory created.
Fix the code by using the natural counterpart to setUp, tearDown(), to clean up.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a Python 2 form to exercise that if present, and fix the setUp() so it
actually looks for a package that exists (nativesdk-python3 is a virtual
package, the interpretter is in nativesdk-python3-core).
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add some tests for functions in meta/lib/oe/recipeutils.py, in
particular for a few issues I've just fixed. I haven't added tests for
all of the functions - some of them are already being tested via devtool
in any case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These date from the time before Tinfoil's API covered this functionality
(back when you could actually access cooker from a tinfoil-based
script).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The code here wasn't correctly getting the variable history for
varflags, so for example if you did a devtool upgrade on a recipe where
the SRC_URI checksums were in the .inc file (typical for python recipes
in order to support both python 2 and 3) then after the upgrade the
new values would be set in the recipe and the old values were left in
the .inc, which is not right. Teach the code here how to get the history
for varflags so it works properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
includes/classes
If we were setting a variable and part of the variable's value was being
set in a class or a .inc file, we were still just setting the value
outright on the assumption that it was too hard to do otherwise. With
some careful use of the variable history we can do better for certain
situations i.e. when the recipe does not currently set the value
outright.
Additionally, correctly remove _appends for variables we are changing if
we're trying to remove the value added in the _append.
Fixes [YOCTO #12623] and partially fixes [YOCTO #9360].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The existing logfile is simply placed in the current directory. Since the test
changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid
directory. We also see trackbacks if the symlink is invalid.
Improve things by:
* Placing logs in LOG_DIR (or BUILDDIR if unset).
* Using a full path to the log meaning the log and link are placed in the same directory.
* Using lexists instead of exists so invalid symlinks are handled correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Heartbeat events default to once a second and we need to ensure we have
enough time in the task to see them.
Add a nostamp delay task 5s long so we can have a consistently timed
task which doesn't need cleanup or have unneeded dependencies. This
ensures we should deterministically see the disk moinitor events
regardless of the state of the build. This is done in a way which
doesn't corrupt build state or need cleanup and is efficient.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This test occisionally fails as m4 doesn't recompile, meaning the logfile test
then doesn't find mention of ccache.
To ensure m4 does recompile, clean m4 before force compiling it.
(Reading the test is confusing due to the test cleanup also involving a clean)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If runqemu fails it would leak an unclosed socket and file. Ensure we
close these in all cases to remove the resource warning.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Each time a runqemu() fails, the log handler would be left behind meaning
messages from any subsequent run would be duplicated (or worse/more).
This ensures we remove the handler regardless and means we no longer
have the duplication.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes:
Stderr:
/media/build1/poky/meta/lib/oeqa/utils/qemurunner.py:381: ResourceWarning: unclosed file <_io.BufferedWriter name=16>
self.runqemu = None
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This avoids problems where shutil.remove will error with:
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
when there are races over file deletion (gpg agent may be slow to exit).
We already worked around speed and race issues in bb.utils.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This should speed the test up signficiantly without any loss of functionality
for the purposes of the test.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We ideally don't want to use gpg from the host. This is straightforward for package
management but not for sstate.
For sstate, create a second build directory to run the test in using gnupg-native
from the original build directory.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Raising an assertionError in the class setup isn't a particuarly good way to
indicate gpg isn't installed. Instead skip the tests if the required binary
isn't present. For the signing tests we do require it to be present and can't
use a prebuilt one.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When tmux is built from a non-release Git version, its version number is
"next-X" where X appears to be the expected version number for the next
release. For example, when built from the current state of master, running
"tmux -V" yields:
tmux next-2.9
Currently check_tmux_pane_size only checks for the version being less than
1.9, so it seems unfair to fail with an obscure Python error in this case.
Let's just use the version number after the "next-" prefix if it is
present.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable qemu for slirp. Initialize Qemurunner with slirp. Setup ip
and port attribute to enable connection with qemu running with slirp.
[YOCTO#10713]
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable qemurunner for slirp. Retrieved the ip & port from host machine
to connect to qemu from host machine.
[YOCTO#10713]
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Qemu do not use the ip input from external. It will
retrieve ip from QemuRunner instance and assign
ip value.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Latest dnf versions have tweaked it.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Splits the SDK test implementation into configurable Python classes. The
classes used for the normal and extensible SDKs are
${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively.
This allows SDK machines to override the classes used to implement the
tests. For the traditional SDK, a common "run()" function is provided by
the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions
that can be overridden in child classes, making it easier to have
consistent behavior. The extensible SDK class
(oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but
no hooks have yet been added as there is not currently a known use case
for create derived classes.
These changes should be purely organizational; no functional changes
have been made to either the standard SDK or extensible SDK tests.
[YOCTO #13020]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
testimage.bbclass installs a SIGTERM handler which conflicts with the
use of multiprocessing here. This is paritcularly problematic if the http
service is terminated before its started and hence before its had a chance
to reset the default signal handler (as the code was written).
Instead, temporarily remove testimage's handler whilst forking the http process
which means the correct handler is installed and won't deadlock.
Also take the opportunity to add in some log messages about the server start
and shutdown so that future debugging is easier and its clearer what the code
is doing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
warnings
Use our own lsb function instead as used elsewhere by the codebase.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid an unclosed file per thread warning when running selftests concurrently
by closing the result stream.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
list_pkgs() for rpm calls RpmPM() which would try and create a copy of the
package feed. This can be called for example from buildhistory whilst some
other task may be working on an SDK or image construction, causing the package
feed to disappear part way through an operation.
Avoid the need to copy the package index just to list the installed
packages, avoiding the race.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix the warnings:
meta/lib/oeqa/utils/qemurunner.py:250: DeprecationWarning: invalid escape sequence \.
ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1])
meta/lib/oeqa/utils/qemurunner.py:343: DeprecationWarning: invalid escape sequence \-
if re.search("root@[a-zA-Z0-9\-]+:~#", output):
poky/meta/lib/oeqa/utils/qemurunner.py:350: DeprecationWarning: invalid escape sequence \-
if re.search("root@[a-zA-Z0-9\-]+:~#", output):
meta/lib/oeqa/utils/qemurunner.py:448: DeprecationWarning: invalid escape sequence \-
if re.search("[a-zA-Z0-9]+@[a-zA-Z0-9\-]+:~#", data):
by correctly marking the regexs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid the warning:
meta/lib/oeqa/selftest/context.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
In this case importlib is a direct replacement.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid warnings such as:
meta/lib/oeqa/utils/commands.py:213: ResourceWarning: unclosed file <_io.BufferedReader name=4>
return runCmd(cmd, ignore_status, timeout, output_log=output_log, **options)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid the warning "DeprecationWarning: invalid escape sequence \(" by marking
the regexs correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Clean up the warning:
meta/lib/oeqa/core/loader.py:27: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec()
_failed_test_args = inspect.getargspec(unittest.loader._make_failed_test).args
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want to see failures/errors listed last since this is the most easily
visible part of the log on consoles or autobuilder output and makes
human processing easier rather than having to scroll up and scan for
a single failure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its useful to have the counts of success/failure/error/skipped at the end of the
results to allow for easier human reading of what happened.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With this it's possible to build locale data for ARC
and not do it instead on the first boot.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Mark up these tests as needing a compiler, make and kernel source code
as appropriate, the image feature requirements can then be retired.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add the OEHasPackage decorator to a variety of tests so they determine
automatically if they should run against a given image.
To ensure tests can do this we need to move target operations such
as scp commands into the tests and out of the class startup/teardown.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
execution
Usually skipped testcase output "SKIPPED"
[snip serial execution]
|RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: SKIPPED (0.01s)
|RESULTS - python.PythonTest.test_python3 - Testcase -1: SKIPPED (0.01s)
[snip serial execution]
But if enable multiprocess execution, skipped testcase output "UNKNOWN" status
[snip enable multiprocess execution]
|RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: UNKNOWN
|RESULTS - python.PythonTest.test_python3 - Testcase -1: UNKNOWN
[snip enable multiprocess execution]
Here is my investigation:
There is a class pairs TestProtocolClient and TestProtocolServer
provided by python3-subunit. The TestProtocolClient generates a
subunit stream of TestResult from a test run, and TestProtocolServer
parses the stream of subunit TestResult.
The class ProtocolTestCase is a unittest.TestCase adapter and it
uses TestProtocolServer to parse the stream of subunit TestResult.
In Yocto testsdk, it forks multiple processes to execute testcases
and use TestProtocolClient to generate TestResult stream; and then
it creates multiple threads to use ProtocolTestCase to parse stream
of subunit TestResult through pipe; finally it passes multiple
ProtocolTestCase as TestCase instance to main process and output
status result.
The problem point is TestProtocolServer parses `skip:' directive
after reading a `test:' directive. Without `test:' directive,
`skip:' directive will be ignored. All above requires SkipTest should
be raised inside a test method rather than setUpClass method.
Throwing SkipTest inside setUp works correctly
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without doing this, the code can and sometimes does try and do "rm ~/"
which I think we'd all prefer it didn't.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should only be wiping out things on target if the tests have
actually run.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The nightly-build-appliance target already tests this particular build.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should not be finding GStreamer issues right around
release time but that's what we seem to do.
Port gstreamer tests from Intel Ref Kit.
https://github.com/intel/intel-iot-refkit/blob/master/meta-iotqa/
lib/oeqa/runtime/multimedia/gstreamer/gstreamer_cli.py
Add tests for gstreamer commandline tools (gst-inspect-1.0
& gst-launch-1.0).
[YOCTO #11418]
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The upstream source tarball checksums changed. Use the copy from our source
mirror to avoid failures.
[YOCTO #12979]
Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
break tests
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Ross Burton
Richard Purdie
Paul Eggleton
Mike Crowe
Yeoh Ee Peng
Alexander Kanavin
Joshua Watt
Alexey Brodkin
Hongxu Jia
Mohamad Noor Alim Hussin