summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/selftest
AgeCommit message (Collapse)Author
2021-07-30Convert to new override syntaxRichard Purdie
This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-08oeqa/selftest/recipetool: update socat version to fix failing downloadRoss Burton
If the recipetool tests are run with an empty DL_DIR the fetch of socat 1.7.3.0 fails as that URL doesn't exist anymore. Update the version to 1.7.4.1 to fix the test. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-07oeqa/selftest/multiprocesslauch: Fix test raceRichard Purdie
Having two possible failures in multiprocesslauch creates a race where one failure may occur and stop processes being lanuched meaning the second failure may not be seen. Rather than having periodic races appearing on the autobuilder, only have one failure, making the test much more deterministic. [YOCTO #13054] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/selftest/archiver: Allow tests to ignore empty directoriesRichard Purdie
If we tweak sstate to not remove empty directories under conditions where a race could occur, we see failures from: "oe-selftest -r archiver.Archiver.test_archiver_filters_by_type archiver.Archiver.test_archiver_filters_by_type_and_name" since an empty directory is left behind. Update the tests to ignore empty directories. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29oeqa/selftest/runcmd: Tweal test timeoutsRichard Purdie
Load on the autobuilder meant we see occasionaly timeout issues with these tests. Slightly increase the test timeouts to better reflect the real world timings we see. [YOCTO #14262] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27selftest/fetch: Avoid occasional selftest failure from poor temp file name ↵Richard Purdie
choice The temp file name may contain "_" characters. Switch to a temporary directory and a fixed filename to avoid this to avoid errors like: bb.data_smart.ExpansionError: Failure expanding variable PN, expression was ${@bb.parse.vars_from_file(d.getVar('FILE', False),d)[0] or 'defaultpkgname'} which triggered exception ParseError: ParseError in /tmp/tmpd_f2__to.bb: Unable to generate default variables from filename (too many underscores) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-21selftest: do not hardcode /tmp/sdkAlexander Kanavin
This races if there are several copies of the test running at the same time. [YOCTO #14438] (From OE-Core rev: deab11848036941771f2b3dc5cdaee83395280b5) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2021-06-03oeqa: remove Clutter usageRoss Burton
Remove the use of core-image-clutter in selftest and manual Toaster QA. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown testRichard Purdie
Rather than totally disabling the logging, inform it we're about to exit so we can log messages over the exit cleanly too. This aids debugging. It also avoids a race where the logging handler could still error whilst shutting down. Also remove a race window by notificing the handler of the shutdown first, before triggering it. This removes a race window I watched in local testing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06classes/lib/scripts: Use bb.utils.rename() instead of os.rename()Devendra Tewari
Incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than adding fallback code to every call site, use a new wrapper in bitbake which detects this case and falls back to shutil.move which is slower but will handtle the overlay docker filesystems correctly. [YOCTO #14301] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27oeqa/selftest: transition to weston imagesAlexander Kanavin
For readonly rootfs tests core-image-weston is appended; everywhere else it replaces core-image-sato. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23oeqa/selftest/bblayers: Add test case for bitbake-layers layerindex-show-dependsDiego Sueiro
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-18oeqa/selftest: Ensure packages classes are set correctly for maintainers testRichard Purdie
The dnf packages aren't parsed if rpm isn't in PACKAGE_CLASSES which means the aintainers test failes for OE-Core (where ipk is the default) but not for poky (where the default is rpm). Ensure PACKAGE_CLASSES is set so it works in all cases. [YOCTO #14277] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-16oeqa/selftest: Hardcode test assumptions about heartbeat event timingsRichard Purdie
Setting a value of 10 for heartbeat events causes the test to fail. Hardcode a value to ensure it works correctly even if the default is changed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06oeqa/selftest: add test case for SRC_URI dependency sniffingRoss Burton
Add tests to verify that SRC_URI dependency sniffing works correctly. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06uboot: Fixes SPL verified boot on corner casesKlaus Heinrich Kiwi
* The kernel-fitimage class adds a do_assemble_fitimage_initramfs task regardless of INITRAMFS_IMAGE_BUNDLE setting, which in some cases can result in that task running after do_uboot_assemble_fitimage and overwriting the u-boot-spl.dtb file with the pristine version (without public key). Fix this by making do_uboot_assemble_fitimage dependant on both do_assemble_fitimage_* tasks, regardless of the aforementioned setting. * Adjust 'type' and 'os' on the U-boot fitimage its script so that mkimage/dumpimage can recognize them. * Move the deployment of the u-boot-spl-nodtb files outside of concat_spl_dtb_helper(), so that we can better isolate the scenarios of creating an (unsigned) U-Boot fitimage versus also signing it. This prevents some stale files from being deployed in the images directory. * Remove any u-boot-fitImage and u-boot-its files from build tree, in case the build tree is being reused across bitbake calls. Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06oe-selftest: Add U-Boot fitImage signing testcasesKlaus Heinrich Kiwi
Derived from the similar kernel fitImage sign testcase, the U-Boot fitImage testcases exercises the following fitimage.FitImageTest scenarios: * test_uboot_fit_image - create unsigned U-Boot fitImage * test_uboot_sign_fit_image - create unsigned U-Boot fitImage in addition to signed Kernel fitImage * test_sign_standalone_uboot_fit_image - Create signed U-Boot fitImage without a Kernel fitImage * test_sign_cascaded_uboot_fit_image - Create and sign U-Boot and Kernel fitImages Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20selftest/wic: Fix dependency issue in rawcopy testRichard Purdie
The current test code contains a subtle race. It is building an ext4, then trying to reuse that image in a wic build for the same recipe. Whilst this happened to work, the code is within its rights to clean up the output before the wic code runs. Avoid this by using separate image targets instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20cve-check: CVE_VERSION_SUFFIX to work with patched releaseLee Chee Yang
CVE_VERSION_SUFFIX in "patch" to treat version string with suffix "pX" or "patchX" as patched release. also update testcases to cover this changes and set CVE_VERSION_SUFFIX for sudo. Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20selftest/reproducible: Drop ovmf exclusionRichard Purdie
The hardcoded path issue was fixed, drop the exclusion. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15meson: Work around determinism issueRichard Purdie
The meson determinism issue is the same frozenset issue we encountered with python itself. Remove the problematic pyc file until upstream work out the best way to address the issue (as with core python). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15distrodata.py: Test to track recipes without HOMEPAGE or DESCRIPTIONDorinda Bassey
This test is to keep track of recipes that do not have a HOMEPAGE or DESCRIPTION, with exceptions to recipes whose SRC_URI are not linked to external sources. Results of this test: it take about 2 to 3 minutes for this test to complete execution depending on the laptop used, Recipes that didn't have a homepage or description were found, and using list data structure instead of set() returned an ordered list of recipes that didn't have a homepage. Hence it's easier to view recipes with missing homepage and description. [YOCTO #13471] Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15go-dep: RemoveKhem Raj
go-dep was an effort for dependency management before go modules, which since 2020 has been deprecated in favor of go modules. Since its not developed any longer and go mdules is officially supported, this should be retired from OE-core as well. Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Otavio Salvador <otavio.salvador@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-12reproducibile: remove perf from exclusionsBruce Ashfield
We have fixes for perf reproducibility, so we can drop it from the exclusion list. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-10oeqa/selftest: add test to verify that poisoned sysroots are detectedRoss Burton
Add a recipe that explicitly searches /usr/include, and use that in oe-selftest to verify that host include paths are correctly causing build failures. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02python3-cython: Remove build paths from debug sourcesRichard Purdie
There are build paths which are in generated sources injected into ${PN}-src. These are in generated files and don't influence the binary output. As such we can correct them at do_package time by tweaking in an appropriately injected function. This fixes the reproducbility of the recipe. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02ltp: Fixing determinism issuesRichard Purdie
Add a patch adding sorting to a couple of points in the Makefiles which removes most of the determinism issues in ltp. Build swapon before the main build to ensure libswapon.o is built deterministically as it races with swapoff. All issues reported on the upstream mailing list. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02Add core-image-weston to reproducible build testsJan-Simon Moeller
This change adds core-image-weston to the reproducible build test. For this we also need to update the static-group and static-passwd. Besides the world package build which includes the wayland package, this adds a wayland/weston-based image to the reproducible build test. Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02lttng: Fix reproducibility issuesRichard Purdie
Add a hack to hardcode in specific rpaths which we then remove, allowing the build to be reproducible. Strip build patches out of one of the test scripts too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02qemu: Determinism fixesRichard Purdie
Add a patch to address two determinism issues and allow reproducible builds. Also strip full paths we don't need out of some ptest files, we can use the installed binaries. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02pinentry: update 1.1.0 -> 1.1.1Alexander Kanavin
Refresh xxx-config->pkg-config patches. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02selftest/reproducible: Add ability to pull some objects from sstateRichard Purdie
When debugging why a single recipe doesn't reproduce, its a pain to wait for the world to rebuild from scratch. Update the selftest to allow this to be configured, for example you could set targets as ['perf'] and sstate_targets as ['virtual/kernel'] and then it should only be rebuilding perf in the test rather than things like the toolchain (parts of the kernel may be unavoiable as they're not in sstate). Can be run as: OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds to save diffoscope output. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01systemd-bootchart: Disable LTO to fix reproducibilityRichard Purdie
LTO likely doesn't buy us much here, disable it to allow the binaries to be reproducible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01efivar: Fix reproducibility issueRichard Purdie
Add sorting to the globbing within the Makefile to make the output reproducible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01swig: Fix reproducibility issueRichard Purdie
Remove hardcoding the build configuration into the help/version output from swig to make the binaries reproducible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01syslinux: Fix reproducibility issuesRichard Purdie
Add sorting to wildcard expansion in the makefile to make builds reproducible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01gstreamer1.0-python: Set internal python library path correctyRichard Purdie
The library hardcodes paths to the python library internally and currently these are build paths. Fix this to use the correct target path and fix reproducibility in the process. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01apr-util: Fix CFLAGS used in buildRichard Purdie
We need to use CFLAGS with the correct WORKDIR in them, replace those in the sysroot file with the ones appropriate to the current recipe. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27libid3tag: Fix reproducibility issueRichard Purdie
Configure was swallowing our cflags meaning the resulting binaries were not reproducible. Tweak configure not to do that and fix reproducibility. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27igt-gpu-tools: Fix reproducibility issueRichard Purdie
Add a configuration option to pass in srcdir, removing hard coded build paths from the binaries. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27bootchart2: Fix manpage reproducibility issueRichard Purdie
The compressed docs contained a timestamp of the original file which meant the SDE clamping during package creation didn't work. The benefits of compression are minor, decompress the files to avoid the reproducibility issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26selftest/reproducible: Don't call sync between each file compareRichard Purdie
Calling sync between each file compare is horrible performance wise as we compare thousands of files. We don't care about IO latency here so disable. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26libhandy: Fix reproducibility issueRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26libsecret: Improve determimismRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26parted: Fix reproducibility issueRichard Purdie
We don't use tbe BUILDINFO line of host information in the Makefile so remove it for reproducibility. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gst-devtools: Fix reproducibility issueRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26epiphany: Fix reproducibility issueRichard Purdie
We don't want to encide BUILD_ROOT into target packages. This is used for build time tests but in our case those would be on target anyway do use the target paths. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gtk-doc: Fix reproducibility issueRichard Purdie
config_data was including a build system path to pkg-config, fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gcr: Fix reproducibility issueRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26cups: Fix reproducibility issuesRichard Purdie
configure inspects the host's /etc/group for these configuration options, fix this to the correct values by using configure options. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>