| Age | Commit message (Collapse) | Author |
|---|
|
removing this test since we move directfb out of oe-core
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This test refers to a function that no longer exists after the eSDK
install double execution of bitbake has been removed, and since
test_prepare_unexpected is the only test in this module, drop the
entire module. We can easily resurrect it if we have unit tests to add
in the future.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The progress patches change the output slightly, update the test to
deal with this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is a GTK+3 application, so we don't need to ship GTK+ 2 in Sato just for
the SDK test suite.
[ YOCTO #9780 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The test test_syslog_help doesn't exist, so skipping unless it passed just
produced a warning.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Using this option the script appends test results into a 'global results
file'. A CSV-formatted output of the results. This option is to provide
compatibility with the old build-perf-test.sh.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
BuildPerfTestRunner determines these from the Git repository under which
it is being run (i.e. where the build directory exists). The branch and
revision may be defined/overridden with OE_BUILDPERFTEST_GIT_BRANCH
and OE_BUILDPERFTEST_GIT_BRANCH environment variables, if needed. This
makes it possible to run the build performance test script even if the
top directory is not a git repository clone, for example.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
A new helper module for easier interaction with Git repositories.
Provides GitRepo class that represents one local Git repository clone.
The GitRepo class currently only has one method, run_cmd(), for running
arbitrary git commands in the repository. More specialized methods for
commonly used git operations can be added later.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Have the build/conf directory as part of test results.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test4" from build-perf-test.sh which measures eSDK metrics.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test3" from build-perf-test.sh which measures
bitbake parsing time.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test2" from build-perf-test.sh which measures
'bitbake core-image-sato -c rootfs'.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test1_p3" from build-perf-test.sh which measures
'bitbake core-image-sato' with rm_work enabled.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test1_p2" from build-perf-test.sh which measures
'bitbake virtual/kernel'.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Re-implement "test1_p1" from build-perf-test.sh which measures
'bitbake core-image-sato'.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The new class is responsible for actually running the tests and
processing their results. This commit also adds a decorator function for
adding new tests. No automatic test discovery, at least yet.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Add a new method to BuildPerfTest class for measuring the disk usage of
a file of directory.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Add new methods to BuildPerfTest class for running a shell
command and logging its output.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Extend BuildPerfTest class with a new method for measuring the system
resource usage of a shell command to BuildPerfTest class. For now,
easurement of the elapsed time is done with the Gnu time utility,
similarly to the build-perf-test.sh shell script. And, it currently only
records the elapsed (wall clock).
The measured values (currently, only the elapsed time) is actually a
dictionary, making it possible to extend it with additional resource
values, e.g. cpu time or i/o usage, in the future. In addition to the
actual values of the measurement each record contains a 'name' and
'legend' where name is supposed to function as a common key or id over
test runs, making comparison and trending easier, for example. Legend is
supposed to be a short human readable description.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The new class will be used as an abstract base class for build
performance tests. This implementation contains some common
functionality used in multiple tests, "copied" from the
build-perf-test.sh shell script.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Add a new utility class for dropping Linux kernel caches. It uses sudo
and tee to write to the drop_caches file. Checking if the user has the
permissions to drop caches (without a password) is done by trying to
writing an invalid value to the drop_caches file. This way, we will find
if writing (with tee) is possible but not really dropping caches, yet.
User can avoid giving the password by adding something like:
<user> ALL = NOPASSWD: /usr/bin/tee /proc/sys/vm/drop_caches
to the system sudoers file.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Wireframe of a new Python module for containing build performance tests
and utility functions.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Useful if one wants to separate stdout and stderr.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Get rid of duplicate code.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
A new function for getting values of multiple bitbake variables at the
same time.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Some upstreams need more than just gnu-configize ran before ./configure works,
such as ./autogen.sh or autoreconf. Add extra_args (defaulting to
gnu-configize) so that this can be done in test cases.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The re.sub() used to transform a tarball into a best guess folder name wasn't
right, as there isn't enough escaping and tar.xz was missing.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
hasPackage() was looking for the string provided as an RE substring in the
manifest, which resulted in a large number of false positives (i.e. libgtkfoo
would match "gtk+").
Rewrite the manifest loader to parse the files into a proper data structure,
change hasPackage to do full string matches, and add hasPackageMatch which does
RE substring matches.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Exported tests is not been tested right now and it get
broken very easily with import errors from bitbake.
This adds a new test in order to verify in the CI if
the testexport gets broken.
[YOCTO #8384]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add support to export the SDK tarball needed when a test
system doesn't have the required software to perform runtime
tests.
The support is when exporting the test and when running
the test on a remote system. The user of this feature just
need to set TEST_EXPORT_SDK_ENABLED to "1" and declare
the sdk packages in TEST_EXPORT_SDK_PACKAGES.
[YOCTO #7850]
(From OE-Core rev: a6041f81b81baa7564e4c712fc88de2b997e52e4)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add the functionality to install/unistall packages in the
DUTs without the use of the package manager. This is possible
with the extraction introduced in package manager class.
testimage and testexport bbclasses has been modified in order
to support this new feature.
[YOCTO #8694]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In python3 the functionality to import modules has been changed and
this broke the capability to add runtime tests from other layers.
This commit returns this capability to testimage and testexport.
[YOCTO #9705]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove deprecated attributes in order to use python3.
runexported was changed to use python3.
[YOCTO #9702]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We are observing cases (see below) where the 'WARNING:' prefix is not at the
beginning of a line, so instead of expecting it in the beginning, match it
within the string.
======================================================================
FAIL: test_recipetool_appendfile_patch (oeqa.selftest.recipetool.RecipetoolTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 285, in test_recipetool_appendfile_patch
self.fail('Patch warning not found in output:\n%s' % output)
AssertionError: Patch warning not found in output:
Parsing recipes..WARNING: File /etc/selftest-replaceme-patched is added by the patch /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta-selftest/recipes-test/recipetool/files/add-file.patch - you may need to remove or replace this patch in order to replace the file.
NOTE: Writing append file /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile.bbappend
NOTE: Copying /tmp/recipetoolqagci9tita/testfile to /tmp/recipetoolqaug0kdb4x/recipes-test/recipetool/selftest-recipetool-appendfile/testfile
done.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since the stream can contain invalid binary characters (e.g. from
ppc's bootloader) use surrogateescape decoding to ensure we do process
the character stream, else it can hang/timeout.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Current parsing was picking wrong targets, leading to the following problem:
AssertionError: Command 'bitbake Parsing recipes..done. -e' returned non-zero exit status 1:
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sets are safer when comparing internal elements and positions are not
important. This commit avoid errors observed on python3 builds as reported
on the below bugzilla entry.
[YOCTO #9661]
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Converted str to bytes before sending to socket.
Converted bytes to str after receiving from socket.
This should fix TypeError: 'str' does not support the buffer interface
for qemurunner.run_serial method.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sometimes is needed to have a package without extraction when
running a test. This patch adds the functionality.
[YOCTO #8536]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This new method extracts the content of package (RPM, DEB, or IPK)
to a directory inside of WORKDIR. The extraction is needed for later
install in the DUTs without using a package manager.
[YOCTO #8694]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This adds the functionality to use a json file to
specify the packages needed for a particular test.
The content of the json file is a dictionary with
dictionaries inside, using the test name as the hash.
The json file must have the same name as the class
module name and must be in the same path.
[YOCTO #7850]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
Maxin B. John
Paul Eggleton
Richard Purdie
Ross Burton
Markus Lehtonen
Mariano Lopez
Leonardo Sandoval
Ed Bartosh