summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2020-12-14wic: Introduce empty plugin to create unformatted empty partitionsDiego Sueiro
The empty wic plugin is used to create unformatted empty partitions for wic images. To use it you must pass "empty" as argument for the "--source" parameter in the wks file. For example: part foo --source empty --ondisk sda --size="1024" --align 1024 Also adds a selftest for this plugin where the 'Fstype' column from 'wic ls' should be empty for the second partition as listed in test_empty_plugin.wks. Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-10selftest/reproducible: add an exclusion list for items that are not yet ↵Alexander Kanavin
reproducible Hopefully over time this list will be reduced to an empty one. Non-reproducible excluded packages are not given to diffoscope and do not cause a failure, but still saved side-by-side with non-reproducible failing ones to make investigation easier. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-10selftest/reproducible: enable world reproducibility testAlexander Kanavin
Add systemd and pam distro features, and commercial license flag to include more recipes into the world set. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-06oeqa/selftest/containerimage: update for improved cleanupRoss Burton
ldconfig/aux-cache isn't generated anymore, and the opkg directories are correctly removed now. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-03oeqa/devtool: use Yocto mirror for pv-1.5.3 tarballRoss Burton
Some of the selftests failed over the weekend with "access denied" errors fetching this tarball. Instead of relying on upstream when fetching the tarball, use the Yocto source mirrors instead. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24cases/bbtests.py: ensure PACKAGE_CLASSES is set to RPM for ↵Chris Laplante
bbtests.BitbakeTests.test_force_task_1 This is because the test expects to find "do_package_write_rpm" in the bitbake output. Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-13systemtap: split runtime material in its own packageYann Dirson
Note the _class-target qualifier, here to prevent a funky dependency of systemtap-native on systemtap-native-runtime-native. This possibly hints to something deeper ? Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-22imagefeatures: New test case, test_empty_image, addedKhairul Rohaizzat Jamaluddin
An empty image build file exists under the meta-selftest folder, test-empty-image.bb, which builds an image with no additional packages. However, there were no further selftest created to verify its emptiness This change consists of the selftest related to the said image to verify its emptiness and the 'import glob' moved as global import instead of local import. The expected outcome of the test should be TRUE or 1 if the .manifest file content is empty. [YOCTO #8455] Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20oeqa: Add sync call to command executionRichard Purdie
We previously put a sync call into devtool to try and combat the bitbake timeout issues on the autobuilder. It isn't enough as the timeouts occur mid test. They are also occurring on non-devtool tests. Add in sync calls around command execution instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-10oeqa/selftest/runtime_test: Exclude gpg directory from pseudo databaseRichard Purdie
Avoid pseudo abort()s like: path mismatch [1 link]: ino 6295376 db '/tmp/oeqa-feed-sign-2mw7z81v/S.gpg-agent.yocto-native' req '/tmp/jwkivmu6'. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-08selftest: add test for recipes with patches in overridesRoss Burton
devtool doesn't quite behave right when a recipe has patches applied in overrides, so add a test case to exercise that behaviour. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-08selftest: skip npm tests if nodejs-native isn't availableRoss Burton
The tests are actually skipped if meta-oe isn't present which isn't quite the same thing, but hopefully close enough. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30oeqa/selftest/devtool: Add sync call to test teardownRichard Purdie
Devtool tests are heavy on IO and if bitbake can't write out its caches, we see timeouts. Call "sync" around the tests to ensure the IO queue doesn't get too large, taking any IO hit here rather than in bitbake shutdown. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30oeqa/selftest/cases/devtool.py: avoid .pyc raceTim Orling
In certain conditions, most likely under heavy load on the AutoBuilder, the prebuilt .pyc files are attempting to be executed before they have been completely copied. Avoid this by not copying the .pyc files (nor the __pycache__ directory). The impact of python3-native recreating the .pyc files should hopefully be negligible. YOCTO#13421 YOCTO#13803 Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-23oeqa/selftest/containerimage: Update to match assumptions in configurationRichard Purdie
Assumptions in the test mean it doesn't work with ssh-pregen-hostkeys. It also doesn't work with systemd. Update the configuration to make sure neither of these effect the test. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-23oeqa/selftest/incompatible_lib: Fix append usageRichard Purdie
It's pure luck this has worked so far, add a missing space to the append. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-14wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi ↵Khairul Rohaizzat Jamaluddin
and bootimg-partition Due to recent changes in bootimg-efi to include IMAGE_BOOT_FILES, when both bootimg-partition and bootimg-efi occur in a single .wks and IMAGE_BOOT_FILES are defined, files listed in IMAGE_BOOT_FILES will be duplicated in both partition. Since IMAGE_BOOT_FILES are crucial for bootimg-partition, but optional for bootimg-efi, hence allowing bootimg-efi to have the option to ignore it. The new variable, IMAGE_EFI_BOOT_FILES, was added to help handle this issue. Its basic usage is the same as IMAGE_BOOT_FILES. Usage example: ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \ This commit is also squashed with the updated testcase to cover for this change. [YOCTO #14011] Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-08oeqa/concurrencytest: Improve builddir path manipulationsRichard Purdie
Its possible some patterns may cause problems with the current path manipulations, make a small tweak to try and avoid potential pathname overlap issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-08selftest/signing: Ensure build path relocation is safeRichard Purdie
Similarly to 04ee0e8b95cd8ed890374e0007f976684206b630, ensure only full build paths are replaced in the environment to avoid breaking buildtools. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-03package.bbclass: hash equivalency and pr serviceMark Hatle
When the PR service is enabled a number of small changes may happen to variables. In the do_package step a call to package_get_auto_pr will end up setting PRAUTO and modifying PKGV (if AUTOINC is there). PRAUTO is then used by EXTENDPRAUTO, which is then used to generate PKGR. Since this behavior typically happens BEFORE the BB_UNIHASH is calculated for do_package, we need a way to defer the expansion until after we have the unihash value. Writing out the pkgdata files w/o AUTOPR and PKGV (AUTOINC) expanded to placeholder values is the easiest way to deal with this. All other variables are expanded as expected. In the next task, typically do_packagedata, we will then use the UNIHASH from the do_package to get the PR (AUTOPR) as well as generate the AUTOINC replacement value (now PRSERV_PV_AUTOINC). The do_packagedata then translates the placeholders to the final values when copying the data from pkgdata to pkgdata-pdata-input. Also update the prservice test case. With unihash, just changing the do_package (via a _append) will not change the PR. So write the date to a specific file that is incorporated into the unihash to ensure it is always different for the test. Various assert messages were also updated to make it easier to figure out where/why a problem occured. Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-02selftest/prservice: Improve test failure messageRichard Purdie
When failing, give more information about why exactly a failure is happening such as the PR values in question. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-02oeqa: reproducible: Fix test not producing diffsJoshua Watt
Diffoscope changed the --exclude-directory-metadata option to require an argument. Add a test to validate that diffoscope is functioning as expected to ensure that future upgrades do not unintentionally break the reproducibility tests. [YOCTO #14025] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-26oeqa/selftest: Apply patch to fix cpio build with -fno-commonRichard Purdie
Patch a combined effort from Khem/RP, need to add$CONFIGURE_FLAGS to ensure tests still work. Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-26selftest/recipetool: Drop globbing SRC_URI test, no longer supportedRichard Purdie
Globbing is a bad idea in SRC_URI, it breaks the task checksums and can't really be fixed. Since we're removing it, drop the test for its interaction with recipetool. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-25oeqa: runtime_tests: Extra GPG debuggingJoshua Watt
This patch enables extra debugging for gpg to try and get more debugging information when [YOCTO #14003] occurs Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-22wic: Add 512 Byte alignment to --offsetJoshua Watt
Allows the --offset argument to use the "s" or "S" suffix to specify that it is reporting the number of 512 byte sectors. This is required for some SoCs where the mask ROM looks for an item at a sector that isn't aligned to a 1KB boundary. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-18selftest/tinfoil: Increase wait event timeoutRichard Purdie
We're seeing this on the autobuilder when IO load is high. Increase the timeout to give this a better chance of working out ok since there is no particular reason we only need to wait 5s and searching for files is IO sensitive. [YOCTO #14001] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-08oeqa/selftest/imagefeatures: Add testcase for fitImageUsama Arif
This testcase generates the Image Tree Source and the corresponding fitImage containing a kernel and a ramdisk. It then checks if the these files exist and if the right fields are present in the right order in the Image Tree Source. Tested with: oe-selftest -r imagefeatures.ImageFeatures.test_fit_image Signed-off-by: Usama Arif <usama.arif@arm.com> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-12oeqa/selftest/sstatetests: Avoid polluting DL_DIRRichard Purdie
Every selftest run is currently polluting the autobuilder DL_DIR. Avoid this by using a temporary directory for the test which is cleaned up. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-03oeqa/selftest: recipetool/devtool: Avoid load_plugin test raceRichard Purdie
This bug has plagued the autobuilder for a couple of years and we've struggled to reproduce/debug it. The problem is the "lib" directory in meta-poky used during the load_plugin tests for recipetool and devtool can race and one can delete the files from the other leading to test failures. Deleting the lib directory only if empty will avoid this. [YOCTO #13070] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02oescripts: ignore whitespaces when comparing linesChen Qi
list-packageconfig-flags.py determines the whitespace numbers according to the longest package name. This is reasonable for the tool as it's trying to generate pretty output. However, in this selftest case, the output is hardcoded. This results in "pinentry gtk2 libcap ncurses qt secret" not recognized as correct as the expected line is: "pinentry gtk2 libcap ncurses qt secret". The difference is only about whitespaces. So we should ignore the whitespaces when comparing lines. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02oescripts.py: fix typoChen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-28oeqa/selftest: Clean up separate builddir in success case when non-threadedRichard Purdie
If oe-selftest is run without -j, the separate build directory "build-st" isn't cleaned up afterwards. Mirror the behaviour of the -j option to handle this the same way, only preserve upon failure. To do this, the remove function needs to be moved to the selftest context module so that it can be accessed without requiring the testtools and subunit modules the -j option requires. A dummy wrapper class is used to wrap the tests and clean up afterwards. [YOCTO #13953] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-23oeqa/selftest/runcmd: Add better debug for thread count mismatch failuresRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-12buildhistory: Add simplistic file move detectionRichard Purdie
We'd like to use buildhistory more during patch review however its proving hard, particularly where whole subtrees of files move, such as a kernel version upgrade, or where a software module moves include directory. This adds file rename matching which covers our common case of library moves, kernel upgrades and more. A new test case is also added so that someone in the future can change the code and test the logic is still doing the expected things. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-10wic: Fix --extra-space argument handlingJoshua Watt
467f84e12b ("wic: Add --offset argument for partitions") broke the --extra-space argument handling in wic. Fix the option and add a unit test for the argument. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-06archiver: Speed up testsPaul Barker
When running tests locally without the advantage of a populated sstate cache, the archiver tests can take an unnecessarily long time. Using different target recipes can speed things up a lot by reducing the number of dependencies that are built. For the test case test_archiver_srpm_mode we can still test the conditions described in bug #11121 by using a target recipe that is known to inherit the nopackages class as well as a more normal recipe, instead of having to compute all the depepndencies of core-image-sato. Before: archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name: PASSED (20.50s) archiver.Archiver.test_archiver_filters_by_type: PASSED (14.16s) archiver.Archiver.test_archiver_filters_by_type_and_name: PASSED (79.08s) archiver.Archiver.test_archiver_mode_configured: PASSED (558.19s) archiver.Archiver.test_archiver_mode_diff: PASSED (14.59s) archiver.Archiver.test_archiver_mode_dumpdata: PASSED (14.41s) archiver.Archiver.test_archiver_mode_mirror: PASSED (14.02s) archiver.Archiver.test_archiver_mode_mirror_combined: PASSED (19.99s) archiver.Archiver.test_archiver_mode_mirror_excludes: PASSED (14.24s) archiver.Archiver.test_archiver_mode_mirror_gitsm: PASSED (13.47s) archiver.Archiver.test_archiver_mode_mirror_gitsm_shallow: PASSED (22.15s) archiver.Archiver.test_archiver_mode_original: PASSED (14.15s) archiver.Archiver.test_archiver_mode_patched: PASSED (14.57s) archiver.Archiver.test_archiver_mode_recipe: PASSED (14.56s) archiver.Archiver.test_archiver_srpm_mode: PASSED (83.03s) oe-selftest () - Ran 15 tests in 911.905s After: archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name: PASSED (23.66s) archiver.Archiver.test_archiver_filters_by_type: PASSED (15.16s) archiver.Archiver.test_archiver_filters_by_type_and_name: PASSED (15.33s) archiver.Archiver.test_archiver_mode_configured: PASSED (15.94s) archiver.Archiver.test_archiver_mode_diff: PASSED (15.79s) archiver.Archiver.test_archiver_mode_dumpdata: PASSED (15.39s) archiver.Archiver.test_archiver_mode_mirror: PASSED (15.28s) archiver.Archiver.test_archiver_mode_mirror_combined: PASSED (21.22s) archiver.Archiver.test_archiver_mode_mirror_excludes: PASSED (15.11s) archiver.Archiver.test_archiver_mode_mirror_gitsm: PASSED (13.61s) archiver.Archiver.test_archiver_mode_mirror_gitsm_shallow: PASSED (23.21s) archiver.Archiver.test_archiver_mode_original: PASSED (14.87s) archiver.Archiver.test_archiver_mode_patched: PASSED (15.50s) archiver.Archiver.test_archiver_mode_recipe: PASSED (15.62s) archiver.Archiver.test_archiver_srpm_mode: PASSED (20.66s) oe-selftest () - Ran 15 tests in 257.170s Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-05archiver: Capture git submodules in mirror archiverPaul Barker
Using the new Fetch.expanded_urldata() function we can get URL data for all git submodules. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04oe-selftest: Recursively patch test case pathsPaul Barker
This ensures that builddir is updated correctly to point to the new selftest build directory when we're given a list of test suites instead of a list of test cases. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04oe-selftest: Support verbose log outputPaul Barker
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04oe-selftest: Allow overriding the build directory used for testsPaul Barker
This may be useful if the parent directory of the original builddir is not writable, on a lower performance drive, etc. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04archiver: Fix test case for srpm archiver modePaul Barker
We can't assume that rpm packaging will always be enabled when running oe-selftest. In particular when using nodistro instead of poky this is not enabled by default. Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04wic: Add --offset argument for partitionsJoshua Watt
Add support for an --offset argument when defining a partition. Many SoCs require that boot partitions be located at specific offsets. Prior to this argument, most WKS files were using the --align attribute to specify the location of these fixed partitions but this is not ideal because in the event that the partition couldn't be placed in the specified location, wic would move it to the next sector with that alignment, often preventing the device from booting. Unlike the --align argument, wic will fail if a partition cannot be placed at the exact offset specified with --offset. Changes in V2: * Fixed a small typo that prevented test_fixed_size_error from passing Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-03apt: update to 1.8.2.1Alexander Kanavin
I took the opportunity to rewrite the recipe from scratch; there was just too much baggage in it. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-03meta-selftest: add test for .patch file with long filename and without subjectMartin Jansa
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-03meta-selftest: add test of .gitignore in tarballMartin Jansa
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-27qemu: enable virglrenderer and glx options subject to 'opengl' DISTRO_FEATUREAlexander Kanavin
Note that to actually use accelerated GL passthrough, there are two options 1) a suitable frontend need to be also enabled - gtk+ and SDL both seem to work well. Previously I struggled to make SDL work, but now it seems fine. 2) it is also possible to render off-screen with -display egl-headless option, and see the output with a VNC viewer (for which, qemu needs to be started with a VNC server): $ runqemu kvm egl-headless publicvnc Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-18image_types: declare support for wic.zstDiego
Declare images in wic.zst format as supported, as bmaptool now supports zstd: https://github.com/intel/bmap-tools/commit/1b8437d58447d3796dc11fd0f2c62bec5746e5d6 Given the very fast decompression speed of zstd over, for example, gzip, writing images with bmaptool can provide big write speed improvements when decompression speed is the limiting factor (especially with very sparse images). Signed-off-by: Diego Rondini <diego.rondini@kynetics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-12selftest/imagefeatures: Enable sanity test for IMAGE_GEN_DEBUGFSYeoh Ee Peng
Add new testcase to check IMAGE_GEN_DEBUGFS. Test makes sure that debug filesystem is created accordingly. Test also check for debug symbols for some packages as suggested by Ross Burton. [YOCTO #10906] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-07virgl: skip test on Debian 9Alexander Kanavin
Debian 9 has an older version of mesa, where dri drivers do not link against glapi explicitly, which causes problems when they are loaded by newer mesa-native: pokybuild@debian9-ty-2:~$ ldd -d /usr/lib/x86_64-linux-gnu/dri/swrast_dri.so |grep undefined undefined symbol: _glapi_tls_Dispatch (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so) undefined symbol: _glapi_tls_Context (/usr/lib/x86_64-linux-gnu/dri/swrast_dri.so) compared to Debian 10: pokybuild@debian10-ty-2:~$ ldd -d /usr/lib/x86_64-linux-gnu/dri/swrast_dri.so|grep undefined pokybuild@debian10-ty-2:~$ Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-07 05:49:19 +0000