aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2016-03-29oe-selftest: generate .env only in test_image_envEd Bartosh
Most of wic functionality doesn't depend on .wicenv file, so it's better to generate it only in test_image_env test case where it's used. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28wic: Update after task ordering changesRichard Purdie
We need the wic env files to be available and this no longer happens automatically so ensure we have them by specifying a specific task dependency. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28selftest/devtool: Update after make PROVIDER changesRichard Purdie
Now that make/remake needs a PREFERRED_PROVIDER, we need to set this correctly during tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23selftest/signing.py: RPM_GPG_PASSPHRASE_FILE -> RPM_GPG_PASSPHRASERobert Yang
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23devtool: add: create git repository if URL specified as positional argumentPaul Eggleton
When we add from a fetched URL we are supposed to turn the resulting source tree into a git repository (if it isn't already one). However, we were using the older deprecated option name here instead of the positional argument, so "devtool add -f <url>" resulted in the repo being created but "devtool add <url>" didn't, which was wrong. Also update the oe-selftest tests to check that this worked. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20oeqa: remove RPM 4 self testJoshua Lock
We're dropping RPM 4 as it's not well maintained, therefore remove a selftest which tests RPM 4 availability. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20oeqa/selftest/buildoptions: test read-only-rootfsRoss Burton
Add a test to build core-image-sato with read-only-rootfs enabled. [ YOCTO #9214 ] Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20oeqa/selftest/sstatetests: verify more variables don't impact the hashRoss Burton
Add DL_DIR and TIME to the hash tests. We can't add DATE for some reason. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20selftest/buildoptions: Renamed one test caseDaniel Istrate
'test_layer_git_revisions_are_displayed_and_do_not_fail_without_git_repo' was renamed to 'test_layer_without_git_dir' which is shorter. fix for [YOCTO #9243] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20Revert "selftest: Added MACHINE = "qemux86" to tests that use runqemu"Paul Eggleton
This is not the correct way to handle this - it significantly increases the time taken to run oe-selftest anywhere MACHINE is set to some other value (for example "qemux86-64"), because all of the artifacts for qemux86 need to then be built as well when running the test. If we need to skip these tests on non-QEMU machines, the devtool test already demonstrates how to do that. This reverts commit 169e1eaa4fc5ed03e2307b68686a7f5b1db37a36. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20recipetool: create: don't create extra files directory unconditionallyPaul Eggleton
The extra directory next to the recipe should only be created if there are files to put into it; currently only the npm plugin does this. I didn't notice the issue earlier because the test was actually able to succeed under these circumstances if the recipe file came first in the directory listing, which was a fault in my original oe-selftest test; apparently on some YP autobuilder machines the order came out reversed. With this change we can put the oe-selftest test that highlighted the issue back to the way it was, with an extra check to reinforce that only a single file should be created. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-13oeqa/selftest/recipetool: Fix test_recipetool_create_simpleRichard Purdie
Correctly handle the any other files/directories that may exist during the test. ====================================================================== FAIL: test_recipetool_create_simple (oeqa.selftest.recipetool.RecipetoolTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/recipetool.py", line 414, in test_recipetool_create_simple self.fail('recipetool did not create recipe file; output:\n%s\ndirlist:\n%s' % (result.output, str(dirlist))) AssertionError: recipetool did not create recipe file; output: NOTE: Fetching http://www.dest-unreach.org/socat/download/socat-1.7.3.0.tar.bz2... NOTE: Unpacking /srv/www/vhosts/autobuilder.yoctoproject.org/current_sources/socat-1.7.3.0.tar.bz2 to /tmp/recipetool-Uj7MIh/ NOTE: Recipe /tmp/recipetoolqaebTo9s/recipe/socat_1.7.3.0.bb has been created; further editing may be required to make it fully functional dirlist: ['socat', 'socat_1.7.3.0.bb'] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-13oeqa/recipetool: Fix syntax errorRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12oeqa/recipetool: Improve debugging output by adding dirlistRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-11selftest/bblayers.py: Remove harcoded recipe filesMariano Lopez
Currently the recipe files are hardcoded and if the recipe change the version, the test will fail. This will change from using a harcoded file to look for the file using bitbake-layers. Now, just the recipe name must be specified. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-11selftest/prservice.py: Sanitize package version when looking for stampMariano Lopez
Currently when using a git version the check for the stamp, using regex, will fail because of plus sign in the version. With this change the version is escaped before adding it to the regex. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-09oeqa/selftest/buildoptions: Test build does not fail without git revDaniel Istrate
Test that layer git revisions are displayed and do not fail without git repository. fix for [YOCTO #8852] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-04oeqa/buildoptions: Improve unsafe references testsRichard Purdie
Fixing one of the recipes not to contain unsafe references to paths results in this QA test failing. Improve the test so that we test the recipe works, then if we intentionally break the recipe, the issue is detected. Also split out the binaries test from the scripts test. The binaries issue may also get 'fixed' in future and need the same fix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28oe-selftest: devtool: add an additional test for devtool upgradePaul Eggleton
Add a test for devtool upgrade with a recipe pointing to a git repository, since this uses several different code paths. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28oe-selftest: devtool: rework devtool upgrade testPaul Eggleton
* Use a more real-world test of a recipe pointing to a remote file * The cleanup tracking / teardown commands need to be added towards the top, or they won't have the desired effect of cleaning up if the test fails. * Check that a versioned subdirectory gets renamed to match the new version * Ensure the recipe contents gets changed as we expect it to * Check that the recipe directory is deleted by devtool reset at the end Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28oeqa/selftest/prservice: Added new TC: check pr-server starts and stop ↵Francisco Pedraza
correctly on localhost. This test case validates if prservice start and stop properly the server. Using bitbake commands also validates the exit status. [YOCTO #8258] Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28oeqa/selftest/signing: Added test for locked signaturesDaniel Istrate
fix for [YOCTO #8706] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-26oeqa/selftest/buildoptions: build -minimal instead of -sato imagesRoss Burton
When checking enabling buildhistory doesn't change anything but rootfs stamps, just build core-image-minimal instead of -sato to reduce the time this test takes. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18oeqa/selftest/bbtests: Test bitbake --setscene-only optionDaniel Istrate
Bitbake option to restore from sstate only within a build (i.e. execute no real tasks, only setscene) fix for [YOCTO #8876] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15oeqa: Update to handle domain specific references in build logsRichard Purdie
With the addition of the task name to recipe output, the sanity tests need updates where they are looking for specific messages. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15oe-selftest: don't use specific tasksEd Bartosh
Image should be build usual way, i.e. using bitbake <image>. Specifying do_mage_complete and do_rootfs_wicenv tasks should not be needed anymore as those tasks should be run by bitbake for every image. Removed specifying do_mage_complete and do_rootfs_wicenv tasks from bitbake calls. (From OE-Core rev: d8d7bd1b41eb846f18378a2581ff172cb2cc52b8) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15oe-selftest: pylinted wic testsEd Bartosh
Fixed the following pylint warnings: C: 45, 0: Line too long (91/80) (line-too-long) C:105, 0: Wrong continued indentation. ignore_status=True).status) ^ | (bad-continuation) W: 83, 8: Redefining built-in 'vars' (redefined-builtin) W:175, 8: Redefining built-in 'vars' (redefined-builtin) W: 27, 0: Unused import sys (unused-import) (From OE-Core rev: ecb5320e766d0fa031c269d6d9b2a24c81c7aa7e) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15oe-selftest: add new wic testcaseEd Bartosh
Added test_qemu testcase to boot wic-image-minimal and test that 2 partitions mentioned in .wks are mounted. [YOCTO #8499] (From OE-Core rev: 6fb015d0847fe7d259d654d4a99bf4c328f810ab) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15wic-image-minimal: change IMAGE_FSTYPESEd Bartosh
Replaced wic.bz2 -> wic in the recipe. Replaced wic.bz2 -> wic in oe-selftest test case. wic-image-minimal is going to be booted in qemu and tested. As runqemu doesn't support compressed images this recipe needs to produce raw wic image. (From OE-Core rev: 3ce91db94f4b921566dce0dc6f91a422009be06b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15oe-selftest: remove unused parameterEd Bartosh
Removed unused parameter 'test' from runqemu function. (From OE-Core rev: c688b3bcbb57099fa72a9728bc708b109802f7fc) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11oe-selftest: devtool: add another devtool add testPaul Eggleton
Add a test so we validate the following recently implemented/fixed aspects of "devtool add": * Adding from a local directory with no name specified * Auto-detecting the name and version from autoconf files * Setting SRC_URI and S from the local git repo * Showing the recipe file path in "devtool status" if it has been created within the workspace Incidentally also tests: * LICENSE and LIC_FILES_CHKSUM detection (though just for this piece of software) * Extracting a dependency from autoconf (though just for this narrow case since there's only one) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: modify: make -x the default behaviourPaul Eggleton
It's going to be more common for users not to have the prepared source tree for a recipe already, so the default behaviour ought to be to extract it for them from the recipe. Change the default to extract (effectively making the -x option a no-op) and add a --no-extract/-n option to disable it. Later we can look at trying to be smart and reusing an existing source tree instead of erroring out if it exists; for now this is just the default reversal. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11oeqa/selftest/sstatetests.py: check that PARALLEL_MAKE doesn't change signaturesRoss Burton
There are recipes that manage to have sstate signatures that depend on PARALLEL_MAKE, so verify that changing this variable doesn't change signatures. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-11oeqa/prservice: Fix whitespace problemRichard Purdie
Recent bitbake changes mean the whitespace formatting around the way functions are defined, changed which broke the rather ugly construct this test uses to append to a python function. This really needs to be rewritten and improved but fix the whitespace so at least the tests work again and other regressions don't creep in. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10oeqa: Improve test failure messagesRichard Purdie
When the test fails, print the incorrect values so aid debugging. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07sstatetests: Fix after change to sstate populate_lic SWSPECRichard Purdie
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9fd214d2e7a931cfb46f40cb76d49aeb07af612e changed SSTATE_SWSPEC, correctly however it did mean that some sstate files now appear where the tests don't expect them. Filter the test results to ignore the correct behaviour. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22devtool: build: ensure pkgdata is written outPaul Eggleton
When you run devtool build, you need to have the pkgdata written out at the end, so that if what you're adding is a library and the next thing you add is something that depends on that library, the necessary information to map the dependency back to the recipe is present. In practical terms all this means is we need do_packagedata to run in addition to do_populate_sysroot. This does mean that do_package needs to run which wasn't running before, and that means that the few package QA tests that run within do_package such as installed-vs-shipped will now be run. This may be a bit bothersome, and prompted a fix for one of our oe-selftest tests as a result, but I don't see an easy way around it. Ultimately if you care about using the recipe in an image you'll need to fix any such errors anyway. Fixes [YOCTO #8887]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22oeqa/selftest/signing: use temporary rpmdbMarkus Lehtonen
Use temporary rpmdb when importing gpg public key and checking rpm signature. This patch should fix a problem where test_signing_packages() sometimes fails with "pmdb: BDB1540 configured environment flags incompatible with existing environment" Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-19oe-selftest: devtool: fix test_devtool_add_library if python was built firstPaul Eggleton
If Python is in the sysroot then this test can fail due to some brokenness in libftdi's CMakeLists.txt file for its python bindings. Just disable it in order to have the test work more reliably. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19recipetool: create: add basic support for extracting dependencies from cmakePaul Eggleton
Add support for extracting dependencies from CMakeLists.txt. There's still a bunch of things missing that are outside the scope of OE-Core and we still lack a proper extension mechanism, but this is a good start. This also adds an oe-selftest test to exercise the new code a bit. Implements [YOCTO #7635]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19recipetool: create: improve extraction of pkg-config / lib depsPaul Eggleton
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict and thus we were skipping some macros. * Add support for PKG_CHECK_EXISTS * Avoid duplicates in warning on missing pkg-config dependencies * Ignore dependency on musl (since this may come up if it's the selected C library) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19selftest/buildhistory: Improve test to remove sources of errorRichard Purdie
When we add buildhistory, we expect do_rootfs to rerun, but depending on IMAGE_FSTYPES, the number of tasks which would execute after do_rootfs varies (e.g. live would add do_bootimg and we recently added do_image). Therefore limit the test to -c rootfs and then we're clear that only one task should re-run. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19image: Fix wic environment issuesRichard Purdie
The wic environment function needs to run after the rootfs size is setup. We move this code to a specific task, and depend on that task from the wic images and other places its needed. This fixes: ====================================================================== FAIL: test_image_env (oeqa.selftest.wic.Wic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env self.assertTrue(var in content, "%s is not in .env file" % var) AssertionError: False is not true : ROOTFS_SIZE is not in .env file Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19insane: Start to clean up do_configure_qa codeRichard Purdie
This cleans up the do_configure_qa code so that the exit status from package_qa_handle_error is handled correctly. It also converts package_qa_check_license to use our standard QA check configuration interface through package_qa_handle_error rather than a mix of return values, bb.fatal and bb.build.FuncFailed. Due to the merging of multiple lines into one message, we need to tweak a QA test to account for this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07oeqa/selftest/recipetool: update for libjpeg-turbo migrationRoss Burton
The recipe creation test of libmatchbox now depends on libjpeg-turbo, not jpeg. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07scripts/oe-selftest: Allow to run tests on random/all MACHINEsDaniel Istrate
Add an option for random MACHINE into oe-selftest: --machine [random/all] 1. random: will set a random MACHINE for each test 2. all: will run tests for all machines Custom machine sets only weak default values (??=) for MACHINE in machine.inc. This let test cases that require a specific MACHINE to be able to override it, using (?= or =). e.g.: oe-selftest --run-tests signing --machine random --> will run all tests switching MACHINE randomly for each test oe-selftest --run-tests signing --machine all --> for each machine will run all tests oe-selftest --run-all-tests --machine random Also update oeqa/selftest/base.py to accomodate this feature. Fix for [YOCTO #5880]. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: Added testcase decorators for 2 testsDaniel Istrate
1391 for test_devtool_add_fetch_simple from devtool 1392 for test_recipetool_create_simple from recipetool Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-07selftest: moved tc test_buildhistory_does_not_change_signaturesDaniel Istrate
Moved test_buildhistory_does_not_change_signatures from buildhistory/BuildhistoryBase to buildoptions/BuildhistoryTests. The test being in the base class was causing it to run multiple times. Fix for [YOCTO #8867] Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-28oeqa/selftest: support sets in devtool comparisonsRoss Burton
The devtool and recipetool tests do literal string comparisons, but for some fields the ordering could be irrelevant and potentially non-deterministic. For example, the recipetool_create_simple test started failing with: AssertionError: 'GPLv2 Unknown' != 'Unknown GPLv2' : values for LICENSE do not match The ordering of the LICENSE field isn't relevant. So, if the expected value is a set(), split the string into a set too and compare those. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-27selftest: Added MACHINE = "qemux86" to tests that use runqemuDaniel Istrate
It makes sense for tests that use runqemu to have MACHINE set as qemu. This also avoid issues when running oe-selftest with --arch random/all option. Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>