aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2018-04-07devtool/oeqa: Ensure added layers set LAYERSERIES_COMPATRichard Purdie
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool/oeqa needs to set this to avoid warnings which break various tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30make-mod-scripts/kernel-devsrc: Fix objtool issuesRichard Purdie
Kernels which use tools/objtool can now fail when building external modules due to objtool being missing, the generated files can also cause problems for kernel-devsrc. Ensure objtool is generated in make-mod-scripts by also calling "make prepare". For devsrc, delete the generated binaries since they'd be native binaries and unsuitable for the target. The oeqa kernel module tests also need to have the additional "make prepare" step added. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-16buildperf: measure the size of core-image-sato rootfsRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-12oe-selftest: add a test for failing package post-installation scriptletsAlexander Kanavin
The test runs a scriptlet that has an intentionally failing command in the middle and checks for two things: 1) that bitbake does warn the user about the failure 2) that scriptlet execution stops at that point. The test is run for all three package types: rpm, deb, ipk. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-06selftests: Add test case for booting a generic EFI boot partition imageCalifornia Sullivan
Simple test case that adds 'efi' to MACHINE_FEATURES, sets WKS_FILE to "efi-bootdisk.wks.in", installed required boot items, and attempts to boot the wic image. Quick check to make sure that the feature actually works. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04oeqa/runtime/opkg.py: add runtime test for opkgYeoh Ee Peng
Add runtime test for opkg to test that it can install ipk package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-03-04oeqa/runtime/apt.py: add runtime test for aptYeoh Ee Peng
Add runtime test for apt to test that it can install deb package from remote source. [YOCTO# 11488] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-27package_manager: Filter to only rpms we depend uponRichard Purdie
Currently do_rootfs gets to see all rpms in the deploy directory. This filters that view to only rpms which the image recipe has actual depends upon which potentially removes some sources of confusion in the image construction. This makes builds more reproducibile and also fixes contamination issues where dnf picks up packages it shouldn't be able to 'see'. [YOCTO #12039] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-24sstatetests: pass the right files when comparing sigfilesRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: add a test for recipes without maintainersAlexander Kanavin
'bitbake -c checkpkg world' is moved to class initializer to avoid it being run twice in a row. The no-maintainers test checks only oe-core recipes, as other layers may be be configured, and assigning maintainership to specific people via maintainers.inc is known to be used only in oe-core. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-24oe-selftest: fix tests to reflect man->man-db transitionAlexander Kanavin
Rather than apply a patch twice, an incorrect patch is applied (as the new man-db recipe does not have any patches yet). Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-02-20oeqa/buildperf/base: Fix fetchall reference to use runall bitbake optionRichard Purdie
The fetchall task was removed, use its replacement bitbake option. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-09oe-selftest: meta_ide: add tests for meta-ide-supportYeoh Ee Peng
QA team were testing meta-ide-support manually. Add automated tests to test that bibtake meta-ide-support will create the toolchain and environment setup script. Also test that after using environment setup script, one can compile c program and build cpio project. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29oeqa/runtime: add gobject-introspection testRoss Burton
(From OE-Core rev: 497a95b576e19e39e20ac280d0db24f51b7c9679) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-23imagefeatures: disable f2fs from test_image_features by defaultSaul Wold
Since the primary f2fs utilities are provided by the meta-openembedded meta-filesystems layer, we disable the testing of that functionality here. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-14selftest/cases/devtool.py: fix workspace layer checkingRobert Yang
Fixed: $ oe-selftest -r devtool.DevtoolTests.test_create_workspace [snip] 2017-12-20 18:28:59,404 - oe-selftest - INFO - Traceback (most recent call last): File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f return func(*args, **kwargs) File "/workspace2/lyang1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 177, in test_create_workspace self.assertTrue('/workspace' not in result.output, 'This test cannot be run with a workspace layer in bblayers.conf') AssertionError: False is not true : This test cannot be run with a workspace layer in bblayers.conf [snip] $ bitbake-layers show-layers NOTE: Starting bitbake server... layer path priority ========================================================================== meta /workspace2/lyang1/poky/meta 5 meta-poky /workspace2/lyang1/poky/meta-poky 5 meta-yocto-bsp /workspace2/lyang1/poky/meta-yocto-bsp 5 meta-selftest /workspace2/lyang1/poky/meta-selftest 5 There is no workspace layer, but I'm in /workspace2, this patch can fix the problem. [YOCTO #12442] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-06bitbake.conf: Set AUTOREV to have a vardepvalueRichard Purdie
If you have a recipe which does not include SRCPV in PV but does set SRCREV = "${AUTOREV}" and you run do_fetch, then change the repo to a new commit then run do_unpack, do_unpack will fail since the new commit doesn't exist in the repo that was fetched. The problem is the revision chosen is not represented in the do_fetch task hash. It if were, the fetch would rerun first and the commit would be present. It works when PV includes SRCPV since that does contain the chosen commit from the AUTOREV. The solution is to include the SRCPV value into the representation of AUTOREV used for checksum calculation purposes. Add a selftest for this issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06oeqa/runtime/buildcpio: Use our own mirror for sourceRichard Purdie
We see occasional network glitches which break this test. Use our own mirror (which has a .gz instead of .bz2) to avoid the errors, we're not trying to test network connectivity. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: fail when ptests fail on targetAlexander Kanavin
That's the whole point isn't it? Previously this testcase succeeded even if some of the underlying on-target tests failed; the only way to find out if anything was wrong was to manually inspect the logs. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run ↵Alexander Kanavin
only when ptest-runner is availalble; Previously the test would execute only when all available ptests for packages in the image were installed; some of those tests may be broken, never finish, take a very long time or simply irrelevant to the user who wants to check ptests of only a few specific packages, and does so by listing them explicitly via IMAGE_INSTALL_append or similar. Presence of ptest-runner means there is at least one ptest package installed as they pull it in via a class dependency; ptest-runner is not generally installed otherwise. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-04core/loader.py: fix regex to include all available test casesLeonardo Sandoval
Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all. [YOCTO #12385] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-02oeqa/qemu: remove elf image typeSaul Wold
Since we are depercating the ELF image type, we should not test it here either. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-01-02oe-selftest: devtool: switch away from mkelfimagePaul Eggleton
mkelfimage is about to be removed, so test_devtool_modify_git needs to use a different recipe. psplash is a reasonable choice given it uses a git repository (and probably will forever), and doesn't have too many dependencies, so change the test to use that recipe instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-21rpm: update to 4.14.0Alexander Kanavin
Previously oe-core had a development snapshot of rpm, it's better to update to something more stable. Removed patches: 0001-Add-PYTHON_ABI-when-searching-for-python-libraries.patch (upstream is using pkg-config) 0001-When-nice-value-cannot-be-reset-issue-a-notice-inste.patch (functionality has been moved to a plugin, we disable plugins for rpm-native) 0012-Use-conditional-to-access-_docdir-in-macros.in.patch (merged upstream) Changed patches: 0001-Fix-build-with-musl-C-library.patch (one previous musl issue has been resolved upstream; another has been added) Rest of the patches are trivial rebases. Update the signing oe-selftest so that the reference output matches the upstream changes. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-21oeqa/selftest/cases/wic: update OETestID decorator for test_qemuJoshua Lock
Test case ID 1422 is not correct for this issue, the correct test case ID for test_qemu is 1424. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-20oeqa/selftest/buildoptions: Add test for fortran compiler buildingRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-20oeqa/selftest/archiver: Add test for srpm archiver mode dependenciesRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-18meta-selftest: wic: Add test for --use-uuid / --fsuuidTom Rini
- Mount a '/media' partition to wic-image-minimal.wks with a known UUID. - In test_qemu, sort our output from checking the output of 'mount' as it may not be stable. Also, do not check the exit code as passing any output to cut ensures a 0 exit code. - Check for a 'UUID=' line in /etc/fstab with out expected output. Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-18targetcontrol.py: use oe.types.boolean for QEMU_USE_KVMRobert Yang
So that both QEMU_USE_KVM = "True" and "1" will work. [YOCTO #12343] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10oe-selftest: devtool: fix test for changes in lzo recipePaul Eggleton
acinclude.m4 is about to be removed from the lzo recipe which breaks test_devtool_update_recipe_local_files_2. Create a synthetic recipe in meta-selftest with some local files and use that instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10utils: qemurunner.py: cleanup grammar and ensure consistencyMark Asselstine
Minor grammar correction along with making the term 'login banner' consistent throughout to make searching logs easier. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10utils: qemurunner.py: Log both 'failed to reach login banner" reasonsMark Asselstine
The current logging always assumes the boot timeout has expired yet there is a second reason we might have ended up in a position where no login banner was found, that being a socket disconnect. Add logging for the disconnect case and make the timeout expiration conditional on the timeout being exhausted. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-10utils: qemurunner.py: Add wall clock to 'timeout' loggingMark Asselstine
When debugging issues when timeouts are involved it is always best to have wall clock times included. This helps give confidence that the timeout is in fact run down at the right rate and that no unexpected events were the true cause of a premature running down of the timeout. Having these times in old logs also helps when debugging issues as we have a historic record as to what is a 'typical' time to complete an action. In addition to adding the wall clock times the time to 'login' is now printed making it consistent with the time to 'qemu pid'. Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-02oeqa/selftest/runtime_test: simplify postinst testingRoss Burton
Update the packages and file names to reflect the new postinst recipe. Fix a sh syntax error in the run_serial file exists test which was hidden by a logic problem in the status code. Remove the older test_verify_postinst as it's effectively a subset of test_postinst_rootfs_and_boot, and doesn't work: when booting under systemd the strings it searches for are not output to the console, but the test still passes. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-12-02oeqa/commands: don't break if get_bb_vars is passed a tupleRoss Burton
get_bb_vars was using variables.copy() to duplicate the list of variables passed but this function only exists in lists [1,2] and not tuples (1,2). Instead of throwing an exception if the variables are in a tuple, simply construct a new list using the passed sequence-like object. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-21qemurunner: Simplify binary data handlingRichard Purdie
I have concerns that bad timing of the flow of data from the logger might corrupt the output due to the way binary strings are handled in qemurunner. This simplifies the code to do the same thing it did before but much more safely. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-10oeqa/runner: Pass the value of buffer, don't force to TrueRichard Purdie
The value could be False in which case we should pass that through. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-10oeqa: Markup further tests for stdout/stderr bufferingRichard Purdie
This further cleans up the output of oe-selftest so that runqemu output is hidden unless tests fail. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09oeqa/target/ssh: Drop command/output logging to debug levelRichard Purdie
This ensures the console is kept clear of confusing output but that the main logs contain good debugging information. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09oeqa: Clean up logger handlingRichard Purdie
The logger handling in oeqa was confused at best. This patch: a) Passes in a logger through various qemu runner pieces b) Uses that logger consistently in the code c) Creates a logger for QemuRunner outside the bitbake namespace meaning we don't conflict with the tinfoil logging changes The result of this is more consistency. For runtime tests in testimage, the logs always contain the debug info, nothing is shwon on the console. For the oe-selftests, logs are intercepted and only shown if the test fails. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09oeqa/qemurunner: Use logger.debug, not logger.infoRichard Purdie
Bitbake logs info messages to the console. These messages are really there as debugging information. At the debug level, they will be shown in failure logs and in the task logs but not on the console which is what we want in this case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09oeqa/targetcontrol: Drop unused get_target_controller functionRichard Purdie
This funciton appears completely unused, drop it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09oeqa/runqemu: Only show stdout/stderr upon test failureRichard Purdie
In general we don't need to see the output of runqemu however if it fails we do. Use the buffer option that already exists in TestResult but allow us to trigger it on a per test basis. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-09qemurunner: Ensure logging handler is removedRichard Purdie
If we don't remove the handler we end up with duplicate log messages which is undesireable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-08sshcontrol.py: in copy_to() always use scpErik Botö
The current implementation is broken when the localpath is a link. Then only a symlink would be created on the target, instead of copying the actual file. [YOCTO #11524] Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-08masterimage.py: rename parameter "params" in start() to "extra_bootparams"Erik Botö
This matches how it is called, and how it is named in qmeu target. [YOCTO #11524] Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-08masterimage.py: fix stop()Erik Botö
The stop() function is called in the context of the masterimage, so self.master should be used instead of self.connection which is undefined at that time. [YOCTO #11524] Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-08masterimage.py: fix issue with calling reboot on masterimage/DUTErik Botö
On systemd systems calling reboot over an ssh connection doesn't return as expected causing an exception, therefore wrap the call to reboot in order to avoid this issue. Also sync the filesystems before rebooting cause otherwise, it will be done as part of the reboot and could take a very long time and testimage will fail to access the machine. This issue was observed consistently with one of our rootfs at Pelagicore. [YOCTO #11524] Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-08selftest/imagefeatures: add basic test for useradd-staticidsRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-08lib/oeqa/selftest/cases/gotoolchain: add selftest for the Go toolchainJoshua Lock
Add a simple test case to being testing of the Go toolchain: 1) build meta-go-toolchain 2) create a temp directory and install the generated Go toolchain within 3) fetch an archive of the Go Dep tool 4) create an appropriately laid out GOROOT and inflate the dep archive there 5) build the dep command with the SDK's Go toolchain and check it returned successfully. [YOCTO #12152] Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>