| Age | Commit message (Collapse) | Author |
|---|
|
Don't ignore return values from the git command lines. If something goes
wrong, fail the test right away.
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A simple += operation is fine for working with TEST_RUNQEMUPARAMS so remove
the heavier append override usage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Split runqemu pre-requisites into a function which can be re-used by
other tests as well.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Builds core-image-minimal with testimage.bbclass and an ssh server
and then runs tests with slirp networking. Default test suite must
pass.
With full caches on x86_64 with x86_64 target the test passes in 416 seconds:
2023-08-21 09:01:12,295 - oe-selftest - INFO - Ran 1 test in 464.955s
2023-08-21 09:01:12,295 - oe-selftest - INFO - OK
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS:
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_slirp: PASSED (416.62s)
2023-08-21 09:01:19,711 - oe-selftest - INFO - SUMMARY:
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 464.956s
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
|
|
Don't overwrite TEST_RUNQEMUPARAMS since user may have added "slirp"
or similar things to it in site.conf, auto.conf or local.conf. Use
:append to add to it. Fixes failing tests when only "slirp" networking
works with qemu and tun based one is broken on the host machine.
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
|
|
explicitly
Running them in parallel is prone to races as postinsts from target sysroots
rely on executables from native sysroots which may or may not be fully prepared
yet. This was observed for example here:
https://autobuilder.yoctoproject.org/typhoon/#/builders/146/builds/468/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/147/builds/467/steps/12/logs/stdio
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
We have autobuilder workers which don't have the -b option unfortunately.
This reverts commit 5ec557467dda29309e25102b507bb919275bedbb.
|
|
Check return values of each git command and set initial branch name
to avoid this warning on every test run:
NOTE: Starting bitbake server...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
|
|
It's not a warning but a handled case and layer gets added
automatically. Very few build configs have this layer enabled
by default.
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
|
|
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
|
|
Adds support for creating FAT formatted file system images (useful for
boot partitions on some SoCs).
Note that FAT partitions are limited in what they can represent (no
symlinks or device files), so they can't really be used for general
purpose root file systems. As such, they are skipped when testing for
that purpose.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add basic test for the --no-table wic part option.
Signed-off-by: Markus Niebel <Markus.Niebel@ew.tq-group.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that SRCPV isn't needed we can simplify things in a few places...
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Source control information being present in PV used to be a hard requirement
for bitbake to operate correctly. Now that hashes are a required part of task
stamps, this requirement no longer exists.
This means we can defer the hash pieces to PKGV and simplify PV.
Use new bitbake fetcher API to inject the source revisions directly into the hash
allowing removal of some horrible code from base.bbclass and avoiding any hardcoding
about how SRCREV may or may not be used.
Use that API to object the string to append to PKGV and append that directly.
The user visible effect of this change is that PV will no longer have revision
information in it and this will now be appended to PV through PKGV when the
packages are written. Since PV is used in STAMP and WORKDIR, users will see
small directory naming and stamp naming changes.
This will mean that sstate reuse through hash equivalence where the source
revision changes but the output does not will become possible as the sstate
naming will become less specific and no longer contain the revision.
The SRCPV variable will no longer be needed in PV and is effectively now just
a null operation. Usage can be removed over time.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whenever a script needs to list tags, if it falls back to the third method
(reading local tags only), it emits a warning. While this warning is useful
for future diagnostic if some tagging issues re-appear, it makes buildperf
autobuilds status as "Passed with warnings", which is not desirable
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
e9cff55e73cc has switched tag listing from bare "git tag" to "git
ls-remote" to make sure not to miss remote tags which are not fetched
locally. This mechanism first checks for configured remote repository, next
for possibly passed url, and then fails if none worked.
However there are still cases where no remote repository is
configured and no url is provided (for instance: buildperf tests use an
empty git directory to store tests). Fix those cases by putting back the
old behavior (local tags check) as last resort, with at least a warning for
future diagnostics if we still encounter tagging issues
Fixes: e9cff55e73cc ("oeqa/utils/gitarchive: fix tag computation when creating archive")
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Propagate a "log" parameter to get_tags in order to know what method is
used to retrieve existing tags
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There's now a real fix in commits "linux-yocto/6.*: fix IRQ-80 warnings".
This reverts commit 1451df346a0e2433714774421ce8f339a37a844a.
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since setUp of RustCompileTest use cls.tc.copyTo, those tests needs the scp command:
NOTE: ======================================================================
NOTE: FAIL: test_cargo_compile (rust.RustCompileTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/case.py", line 53, in _oeSetUp
self.setUpMethod()
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/runtime/cases/rust.py", line 17, in setUp
cls.tc.target.copyTo(src, dst)
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 132, in copyTo
return self._run(scpCmd, ignore_status=False)
File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 81, in _run
raise AssertionError("Command '%s' returned non-zero exit "
AssertionError: Command '['scp', '-o', 'ServerAliveCountMax=2', '-o', 'ServerAliveInterval=30', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '-r', '-P', '2222', '/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/files/test.rs', 'root@127.0.0.1:/tmp/']' returned non-zero exit status 1:
sh: scp: not found
lost connection
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
rust-c-lib-example is a little rust code which provide
a single function to print a formatted date (via the chrono crate)
from an input timestamp in millisecond. It has the necessary FFI
annotation and inherit cargo_c class for the C ABI compatible
library generation.
rust-c-lib-example is meson project for the C code which
will call the print_date function from rust-c-lib-example
if no argument is provided, if any argument is provided
it will print "Hello world in rust from C!"
add a runtime test case to check if all went well.
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
configured
Add specific tests on gitarchive for when tag listing is required but no
remote is configured in target directory: it should either succeed if valid
url is provided, or fail is url is not provided or wrong
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sporadic errors have been observed in autobuilder when trying to store new
tests results:
error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.
The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push
Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. To do so, create a helper ("get_tags") which
manages both nominal case (target directory is a git repository with a
proper remote) and fallback case (target directory is not from a clone, no
remote has been configured)
Fixes [YOCTO #15140]
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a test suite for gitarchive.py. For now, only introduce tests on
methods which needs to read existing tags
The tests rely on tmpdirs to create local, "fake" results repository in
order to allow basic git commands
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 5a0a7da85a3acfd4a20a07478eabefdab60f313a.
This caused failres on the build performance tests on the autobuilder.
|
|
With the new 6.4 kernel we see this preempt-rt error. It is blocking changing
to the new kernel and has sat on mailing lists unresolved for a long time. Ignore
it in testing for now and allow upgrading until we can better understand the
issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sporadic errors have been observed in autobuilder when trying to store new
tests results:
error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.
The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push
Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. Two places which wrongly read only local tags has
been identified in gitarchive: expand_tag_strings and get_test_runs
Fixes [YOCTO #15140]
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
systemd now requires usrmerge, ensure this is always the case in
our test cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We can't have systemd here any longer without usrmerge. We don't really
want to enable the latter since having separate usr will likely result in
a class of reproducibility and host contamination issues that enabling it
might hide.
Also drop INHIBIT_PACKAGE_STRIP since we generally don't build with that
and the debug binaries should be generated regardless. I suspect this
is legacy from older issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously some dependencies couldn't be followed through their siginfo
files. This has been fixed, add a test to ensure this doesn't regress.
Signed-off-by: Yang Xu <yang.xu@mediatek.com>
|
|
we now have more rust/cargo recipes and tests that cover various use cases
so this is no longer needed.
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Capture the time taken for glibc test execution and
pass it for inclusion in the test report.
[YOCTO #15165]
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Capture the time taken for gcc test execution and
pass it for inclusion in the test report.
[YOCTO #15165]
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Capture the time taken for binutils test execution and
pass it for inclusion in the test report.
[YOCTO #15165]
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This was done in a selftest, but that is too late and creates
friction in integration as errors are not seen until autobuilder fails.
Bonus fix: SUMMARY check wasn't even working, as in the absence
of one set in the recipe there is a default value set from bitbake.conf.
I left DESCRIPTION check out for now, as many recipes don't actually
have it, and it's set from SUMMARY (plus a dot) if absent.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To match the other tests, round the test duration to an intger value
so the test reports are consistent.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Add the failing test cases in exclude_list to exclude them from testing during the Rust Oe-selftest.
* Drop meta/recipes-devtools/rust/files/rust-oe-selftest.patch file as the failing tests are
moved from being a patch to exclude_list in meta/lib/oeqa/selftest/cases/rust.py.
* When updating to a newer version of Rust, it is manually needed to update the exclude_list. The tests that fail are observed to
work on some versions and fail on others. These tests have been excluded in order to successfully complete testing of Rust Oe-selftest.
* The tests that are passed and skipped are as follows-
Target PASS SKIPPED
ARM 15507 428
ARM64 15535 400
MIPS64 15479 456
X86 15528 407
X86-64 15643 292
Also, time the test execution and pass the time taken for inclusion in the test report.
[RP: Note duration addiion in the commit log]
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It looks like there were further cases where orphaned processes may be left
behind since the .kill() calls may be unsuccessful if the process terminated
due to the terminate or through normal exit. In that situation .wait()
wouldn't have been called.
Further tweak the exit code paths to ensure .wait() is called to update the
returncode value before returning in all cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As spotted by Joshua Watt, the returncode isn't set until .poll() or .wait()
is called so we need to call this after the .kill() call.
This fixes return code reporting so that timeouts for example now return an
exit code when they didn't before.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When test command timeouts occur, there was no warning in the logs. Change
this to ignore "exit 1" but report all other exit codes so that timeouts
are clear.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On our slower arm server, the tests currently timeout leading to inconsistent test
results. Increase the timeout to avoid this and aim to make the test results
consistent.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We have a suspicion that the read() call may return EAGAIN on the non-blocking
fd and this may truncate test output leading to some of our intermittent failures.
Tweak the code to avoid this potential issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This provides a more reliable test execution when running tests that
write a large buffer/file and significantly reduces the localedata test
failures.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allows setting up NFS over TCP as well.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some of the tests trigger OOM and fail. Increase the amount of memory
available so we dont run into these issues.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The LTP test reporting appears to be a little fragile so I tried to make
it more reliable.
Primarily this is done by not passing -p to runltp, which results in
machine-readable logfiles instead of human-readable. These are easier
to parse and have more context in, so we can also report correctly
skipped tests.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Fix the override so we actually pass the correct value to glibc.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
- After introducing the CVE_STATUS and CVE_CHECK_STATUSMAP flag
variables, CVEs could contain a more information for assigned statuses.
- Add an example conversion in logrotate recipe.
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Ensure the test/class naming doesn't allow the to be triggered
without the toolchain decorator
* Add the toolchain-user decorator so it runs on non-IA targets
* Strip the leading "[XX] " prefix from the test names
* Ensure skipped test counts are passed through correctly (as SKIPPED, not SKIP)
* Avoid duplicate test results and show a warning if any are found (duplicates
were from other sources in the end but the code remains sensible to have)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The patch implements Rust testing framework similar to other selftest,
specifically the gcc selftest in OE. It uses the client and server
based method to test the binaries for cross-target on the image.
The test framework is a wrapper around the Rust build system as ./x.py
test. It tests many functionalities of Rust distribution like tools,
documentation, libraries, packages, tools, Cargo, Crater etc.
Please refer the following link for detailed description of Rust
testing:-
https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
To support the rust tests in oe-core, the following functions were
added:-
setup_cargo_environment(): Build bootstrap and some early stage tools.
do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemu target image.
Approximate Number of Tests Run in the Rust Testsuite :- 18000
Approximate Number of Tests that FAIL in bitbake environment :- 100-150
Normally majority of the testcases are present in major folder "test/"
It contributes to more than 80% of the testcases present in Rust test
framework. These tests pass as expected on any Rust versions without
much fuss. The tests that fail are of less important and contribute to
less than 2% of the total testcases. These minor tests are observed to
work on some versions and fail on others. They have to be added, ignored
or excluded for different versions as per the behavior.
These tests have been ignored or excluded in the Rust selftest
environment to generate success of completing the testsuite.
These tests work in parallel mode even in the skipped test mode as
expected. Although the patch to disable tests is large, it is very simple
in that it only disables tests. When updating to a newer version of Rust,
the patch can usually be ported in a day.
Tested for X86, X86-64, ARM, ARM64 and MIPS64 on Ubuntu 22.04.
Signed-off-by: pgowda <pgowda.cve@gmail.com>
Signed-off-by: Vinay Kumar <vinay.m.engg@gmail.com>
Signed-off-by: Yash Shinde <yashinde145@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix [Yocto #15085]
Co-authored-by: Fawzi KHABER <fawzi.khaber@smile.fr>
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Mikko Rapeli
Richard Purdie
Adrian Freihofer
Alexander Kanavin
Joshua Watt
Markus Niebel
Alexis Lothoré
Luca Ceresoli
Frederic Martinsons
Yang Xu
Yash Shinde
Anuj Mittal
Ross Burton
Andrej Valek
Yoann Congal