summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/sstatesig.py
AgeCommit message (Collapse)Author
2024-04-04sstatesig: fix netrc.NetrcParseError exceptionJose Quaresma
Looks like sometimes the e.filename and the e.lineno is not properly set by the netrc and this can cause TypeError. | File "/poky/meta/lib/oe/sstatesig.py", line 342, in init_rundepcheck | bb.warn("Error parsing %s:%d: %s" % (e.filename, e.lineno, e.msg)) | ~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | TypeError: %d format: a real number is required, not NoneType Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-26sstatesig: Warn on bad .netrcJoshua Watt
If there is an error parsing .netrc, warn the user Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-23sstatesig: Set hash server credentials from bitbake variablesJoshua Watt
Allows the hash server credentials to be specified in bitbake variables. If omitted, the users .netrc will be checked Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19sstatesig: Implement new siggen APIJoshua Watt
Implements the new API required for querying unihashes in parallel Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05lib/sstatesig/find_siginfo: raise an error instead of returning None when ↵Alexander Kanavin
obtaining mtime Suppressing the error and returning None can result in a delayed failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6254/steps/14/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6262/steps/14/logs/stdio It is not clear why the os.stat() error occurs to begin with (it shouldn't), so rather than adding further workarounds, let's get diagnostics at the source first, so we understand what is going on. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05sstatesig: Add version information for find_sigingfoRichard Purdie
Since we're changing the return values of the function, add a version so bitbake can ensure it is using a compatible function. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05sstatesig/find_siginfo: unify a disjointed APIAlexander Kanavin
find_siginfo() returns two different data structures depending on whether its third argument (list of hashes to find) is empty or not: - a dict of timestamps keyed by path - a dict of paths keyed by hash This is not a good API design; it's much better to return a dict of dicts that include both timestamp and path, keyed by hash. Then the API consumer can decide how they want to use these fields, particularly for additional diagnostics or informational output. I also took the opportunity to add a binary field that tells if the match came from sstate or local stamps dir, which will help prioritize local stamps when looking up most recent task signatures. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-09sstatesig/find_siginfo: special-case gcc-source when looking in sstate cachesAlexander Kanavin
This is already done for local stamps just above, and will allow enabling the full selftest that compares gcc-source signatures via printdiff (that is, both local stamp and sstate variants). Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S ↵Alexander Kanavin
lockedsigs This was writing out locked-sigs.inc into cwd with every 'bitbake -S' invocation. When the intent is only to to get task stamps (-S none), or print the difference between them (-S printdiff), the file is unnecessary clutter. A couple of selftests/scripts were however relying on this, so they're adjusted to explicitly request the file. eSDK code calls dump_lockedsigs() separately via oe.copy_buildsystem.generate_locked_sigs() and so isn't affected. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19sstatesig: be more precise and show the full path in exceptionsJose Quaresma
Also remove the warning than doesn't make sense as the code will generate an exception and bitbake will abort. Before: | WARNING: core-image-minimal-initramfs-1.0-r0 do_image_complete: KeyError in . | Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx' | Path . is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination. After: | Exception: Exception: KeyError: 'getpwuid(): uid not found: xxxxx' | Path /build/tmp/work/intel_corei7_64-lmp-linux/core-image-minimal-initramfs/1.0-r0/sstate-build-image_complete is owned by uid xxxxx, gid yyy, which doesn't match any user/group on target. This may be due to host contamination. Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
2023-10-09sstatesig: add a new info level for SIGGEN_LOCKEDSIGS_TASKSIG_CHECKJulien Stephan
as of now, SIGGEN_LOCKEDSIGS_TASKSIG_CHECK can take 2 values: "warn" and "error", displaying respectively a warning or a fatal error message only when a task is locked and the task signature is different from the locked one. The "info" level is introduced to add a "note" message to remind the user that a recipe is locked even if the signature is equivalent to the locked one. The "warn" and "error" level display the warn/error message for each task having a mismatch of the signature. Doing this with the "info" level would result in very verbose output if there are several tasks locked, so the info level will only print once the list of recipes that have locked signature. Signed-off-by: Julien Stephan <jstephan@baylibre.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2023-08-09sstatesig: Update to match bitbake changes to runtaskdepsRichard Purdie
Bitbake has changes to runtaskdeps in siginfo files to fix bugs in being able to locate them for sstate and hash debugging purposes. This patch updates to match the changes to the format. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-06sstatesig: Drop SPDX special casingRichard Purdie
Other changes in create-spdx code mean we shouldn't need to do this now. We need the various exclusions to allow the task hashes to behave correctly for the SPDX tasks too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15sstatesig: Improve output hash calculationMateusz Marciniec
Symbolic links to the files are included during the output hash calculation but symlinks to the directories are missed. So if the new symlink to a directory was the only change made, then the output hash won't change, and the Hash Equivalence server may change unihash. In the next run bitbake may use an older package from sstate-cache. To fix this followlinks=True flag could be set for os.walk but it can lead to infinite recursion if link points to a parent directory of itself. Also, all files from a directory to which symlink points would be included in depsig file. Therefore another solution was applied, I added code that will loop through directories and process those that are symlinks. Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-11lib/sstatesig: Drop OEBasic siggenRichard Purdie
We're now used to using hashes as part of the task hashes and the sstate code relies on this. The older OEBasic hash approach therefore wouldn't work and can be removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20sstatesig: emit more helpful error message when not finding sstate manifestEnrico Jörns
Since oe-core commit 64b89f3c8fc31842256c482a3039d90d3f12c1cc ("sstatesig.py: make it fatal error when sstate manifest isn't found") errors like: | Manifest [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot not found in imx8mm_dummy cortexa53-mx8mm cortexa53 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')? are fatal now and cannot be ignored but must be debugged. Unfortunately, the currently emitted error message is a bit imprecise with telling the reader what has actually gone wrong. This commit: * adds the word 'sstate' to the error message to clarify the scope we are dealing with ('sstate manifests', since there are other manifests, too) * does not randomly print the last manifest file searched for as THE manifest file that could not be found Instead, we print the name of the task the sstate was searched for * adds the word 'multilib' to variant to make clear which variant we are talking about * adds a separate line noting the searched pkgarchs and adds explicitly mentions this word ('pkgarchs') * prints a list of ALL manifest file locations attempted * removes the '?' at the end of the message since such errors indeed leave the question of what is the cause but the error message itself is more like a statement. The result for the exact same issue as noted above then looks as follows: | The sstate manifest for task 'dbus:populate_sysroot' (multilib variant '') could not be found. | The pkgarchs considered were: imx8mm_dummy, cortexa53-mx8mm, cortexa53, armv8a-crc, armv8a, aarch64, allarch, x86_64_x86_64-nativesdk. | But none of these manifests exists: | [..]/tmp/sstate-control/manifest-imx8mm_dummy-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-mx8mm-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-cortexa53-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-crc-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-armv8a-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-aarch64-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-allarch-dbus.populate_sysroot | [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot Signed-off-by: Enrico Jorns <ejo@pengutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2022-11-14sstatesig: skip the rm_work task signatureJose Quaresma
We can skip the rm_work task signature to avoid running the task when we remove some tasks from the dependencie chain. The inject_rm_work handler on the rm_work bbclass triggers the rm_work task running for any signature change in the dependencie chain of the task do_build of each recipe. i.e INHERIT:remove = "create-spdx" will trigger the do_rm_work when we collect the sstate cache with INHERIT = "create-spdx" Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2022-08-12lib: Add copyright statements to files without oneRichard Purdie
Where there isn't a copyright statement, add one to make it explicit. Also add license identifiers as MIT if there isn't one. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-19sstatesig: Include all dependencies in SPDX task signaturesJoshua Watt
SDPX generation involves looking through BB_TASKDEPDATA for dependencies, then linking to the generated documents for those dependencies. These document links use a checksum to validate the document, which means that if a upstream document changes, all downstream documents must be regenerated to get the new checksum, otherwise the compendium of documents produced by the build will have broken links; therefore all dependent task should be included in the signature (even from "ABI safe" recipes). Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2022-04-29lib/sstatesig: Fix find_siginfo to match sstate filename generationRichard Purdie
sstate filename generation was changed a while ago and taskname has to be passed into core functions for the correct filename to be generated. Update find_siginfo to match those changes and pass in taskname via SSTATE_CURRTASK. Thanks to Gregory Lumen <gregorylumen@microsoft.com> for spotting. [YOCTO #14774] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02sstate: do not add TARGET_ARCH to pkgarch for cross recipes.Alexander Kanavin
This is redundant (target arch is already in PN), and breaks compiling a cross-canadian toolchain, as that needs populating the sysroot with two different native-hosted toolchains built from cross recipes. Inserting TARGET_ARCH allows only one or the other. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-17classes/native: Propagate dependencies to outhashJoshua Watt
Native task outputs are directly run on the build system during the build after being built. Even if the output of a native recipe doesn't change, a change in one of its dependencies may cause a change in the output it generates (e.g. rpm output depends on the output of its dependent zstd library). This can cause poor interactions with hash equivalence, since this recipe's output-changing dependency is "hidden" and downstream tasks only see that this recipe has the same outhash and therefore is equivalent. This can result in different output in different cases and issues with reproducible builds in parcular (e.g. rpm compression changes for the same content). To resolve this, unhide the output-changing dependency by adding it's unihash to this tasks outhash calculation. Unfortunately, we don't know specifically know which dependencies are output-changing, so we have to add all of them. [YOCTO #14685] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-16reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variableRichard Purdie
We want things to be reproduicble and the variable doesn't really change much any more. Drop the remaining uses and make those code paths always active. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08sstatesig: Only apply group/other permissions to pseudo filesRichard Purdie
We hardlink some files into the build, such as licence files in do_populate_lic tasks. Depending on the umask that the source tree was checked out with, the group permissions would vary. This results in inconsistent task outhashes. Avoid this by ignoring the group/other bits unless we're under pseudo context. Bump the ABI numbers to ensure we don't see cache corruption from earlier builds. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-04sstatesig: Add processing for full build paths in sysroot filesRichard Purdie
Some files in the populate_sysroot tasks have hardcoded paths in them, particularly if they are postinst-useradd- files or crossscripts. Add some filtering logic to remove these paths. This means that the hashequiv "outhash" matches correcting in more cases allowing for better build artefact reuse. To make this work a new variable is added SSTATE_HASHEQUIV_FILEMAP which maps file globbing to replacement patterns (paths or regex) on a per sstate task basis. It is hoped this shouldn't be needed in many cases. We are in the process to developing QA tests which will better detect issues in this area to allow optimal sstate reuse. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-01sstatesig: Revert "Test cross/native hashserv method extension"Richard Purdie
This reverts commit 2a76082363d189880613765ad339718e3614049d. We have an issue where x86 host builds are not matching hashes with aarch64 host builds. We'd expect that for a given target, the target artefacts should work regardless of the host architecture, compiler version etc. but this isn't happening and the hashes are differing. This is due to issues from hash equivalence. I believe the commit being reverted was added as a test and there were other fixes at the time which resolved these issues. As illustration of that, different gcc versions are not cauing issues with hash equivalence. That should be similar to the aarch64 case vs. x86-64 and hence if we're not seeing gcc verison issues, we also don't need this special case. As such, revert it as we don't need it and it is in fact breaking sstate reuse cross platform. [YOCTO #14578] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-16sstatesig: Allow exclusion of the root directory for do_packageRichard Purdie
The package task references WORKDIR at it's top level and we can't easily make the timestamp for that determnistic due to writes to files there and in other subdirs. We could try and force it to a specific value but it is easier to just remove it from the package task, we don't need it there or care about it in this case. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-30meta: Manual override fixesRichard Purdie
The automated conversion of OE-Core to use the new override sytax isn't perfect. This patches some mis-converted lines and some lines which were missed by the automation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-30Convert to new override syntaxRichard Purdie
This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-03classes/buildhistory: Add option to strip path prefixJoshua Watt
Adds an option to strip a prefix from the paths reported in buildhistory. This makes it easier to compare task signatures in the build history when the builds were done from different directories. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-21sstatesig.py: make it fatal error when sstate manifest isn't foundMartin Jansa
* all known issues in this area were fixed, make it fatal that it cannot be overlooked if someone triggers this issue again Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26sstatesig.py: show an error instead of warning when sstate manifest isn't foundMartin Jansa
* not sure if there are some valid use-cases for missing manifest, but recently I'm seeing increasing number of build failures where something from native recipe is missing (seen it with pseudo, autoconf, nodejs recently) and the only indication that something is wrong (before showing sometimes misleading error like: recipe-sysroot-native/usr/bin/node: No such file or directory is this warning: NOTE: Running task 7844 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_prepare_recipe_sysroot) NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Started WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')? NOTE: Running task 7845 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_unpack) NOTE: recipe nodejs-12.20.2-r0: task do_unpack: Started WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')? NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Succeeded if I rebuild that native dependency, then it gets fixed and I don't see these failures in clean builds (as without sstate and with empty TMPDIR), only in incremental builds * but if there isn't valid reason for missing manifest file, then I think it would be better to error early (or even bb.fatal()) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03sstatesig: Add descriptive error message to getpwuid/getgrgid "uid/gid not ↵Tomasz Dziendzielski
found" KeyError If path is not owned by any user installed on target it gives insufficient error "getpwuid(): uid not found" which may be misleading. This exception occurs if uid/gid of path was not found in PSEUDO_PASSWD files, which simply means the path is owned by host user and there is host user contamination. Add more information to the exception message to make it easier for user to debug. [YOCTO #14031] Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24uninative: Don't use single sstate for pseudo-nativeRichard Purdie
pseudo-native is a bit special. It conditionally compiles in support for xattr, statx and statvfs amongst other options. If a pseudo-native binary is used on a system where these functions are present but it wasn't compiled in we see hard to debug permissions problems. An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target oe-selftest which shows a cryptic error: File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target self.assertEqual(filelist1, filelist2) File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual self.assertSequenceEqual(list1, list2, msg, seq_type=list) File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual self.fail(msg) File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail raise self.failureException(msg) AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] First differing element 0: '-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor' '-rwxr-xr-x root root /etc/init.d/mdmonitor' This is due to a version of pseudo without statx being used on a system where ls uses statx, hence the files are displayed as 6000.6000 instead of root.root. Avoid this by always building pseudo-native for the specific distro in question rather than using a universal sstate feed. This hopefully fixes one of the mysterious AB-INT issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-03sstatesig: Remove workaround for bitbake taskhash bugRichard Purdie
When trying to lock an individual signature, we see the checksum calculations of dependent tasks failing. The fix is to remove a bad optimisation within bitbake but with the removed, we need to remove some bogus code with OE-Core's sstatesig code too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30sstatesig: Log timestamps for hashequiv in reprodubile builds for do_packageRichard Purdie
Currently if a task generates the same output with different timestamps, hasequiv won't detect it but reproducibile builds will fail tests due to the different timestamps. Add do_package timestamps to the hash when reproducibile builds are enabled to avoid this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07base/bitbake.conf: Enable pseudo path filteringRichard Purdie
This is a pretty big change to the way pseudo operates when used in OpenEmbedded. Normally, pseudo monitors and logs (adds to its database) any file created or modified whilst in a fakeroot environment. There are large numbers of files we simply don't care about the permissions of whilst in that fakeroot context, for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control directories, This change uses new functionality in pseudo to ignore these directory trees, resulting in a cleaner database with less chance of "stray" mismatches if files are modified outside pseudo context. It also should reduce some overhead from pseudo as the interprocess round trip to the server is avoided. There is a possible complication where some existing recipe may break, for example, we found a recipe which was writing to "${B}/install" for "make install" in do_install and since we listed ${B} as not to be tracked, there were errors trying to chown root for files in this location. This patch fixes a few corner cases in OE-Core when used with this new ignore list: * The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed to something else since that directory does need its root permissions * The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR} * package_write_* task output (the debs/rpms/ipks) are now owned by the build user so we don't want the file ownership information in the hashequiv outhash calculation even if they are built under pseudo. * The fontcache postinstall intercept is run under qemu outside of pseudo context so delete files it may delete up front where pseudo can see this. * SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached by bitbake. We therefore need to trigger reparsing if this changes, which means SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST. Rework the variables to handle this. This otherwise breaks some of our sstate tests in oe-selftest. * Ignore the temp directory wic uses for rebuilding rootfs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-15sstatesig: Account for all dataCaches being passedJoshua Watt
Bitbake now passes all the dataCaches to the taskhash API, so use this to correctly filter mcdepends. [YOCTO #13724] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-25abi_version/sstatesig: Introduce HASHEQUIV_HASH_VERSIONRichard Purdie
We've found we need a way to cause a change in signatures and move to a new hash 'namespace' with hashequiv. This introduces a variable which allows us to do this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16sstatesig: Improve debug output if getpwuid() failsRichard Purdie
If getpwduid fails, we don't see which file it failed on which is key information to aid debugging. Print this information when exceptions are raised. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03sstatesig: Avoid resetting taskhash within siggen for locked sigsRichard Purdie
Since get_unihash uses taskhash as a key internally, changing it means different bebahour when locked sigs are active verses not active. Under corner cases this leads to a signature mismatch. Avoid this by by adding a wrapper for the place its externally exposed and then not changing the internals. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03sstatesig: Fix locked signature handling with unihashesRichard Purdie
get_taskhash will call get_unihash internally in the parent class. We need to disable our filter of it whilst this runs else incorrect hashes can be calculated. This is believed to be causing the locked signatures test to fail under some circumstances (depending on whether earlier hashes are being remapped). [YOCTO #13605] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02sstatesig: Test cross/native hashserv method extensionRichard Purdie
We can have one taskhash which represents multiple native/cross sstate objects since they're stored by BUILD_ARCH or possibly host distro (or host gcc version). We need to put these into separate namespaces on hashserv since their outhashes will never match and we need deterministic lookups for the different namespaces. Use this extramethod option to handle this. This fixes several problematic failures on the autobuilder. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16sstatesig: Further optimiseRichard Purdie
Optimise the call into the parent function to be only when needed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15sstatesig: Optimise get_taskhash for hashequivRichard Purdie
With hashequiv the get_taskhash function is called much more regularly and contains expensive operations. This these don't change based upon hash in a given build, improve the caching within the function to reduce overhead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27lib/sstatesig: Fix class inheritance problemsRichard Purdie
The locked sigs class needs to be inherited after the hashequiv mixin so that get_unihash can correctly wrap the underlying hashequiv function. To do this turn the locked sigs class into a second mixin, then the order can be correctly handled. Tweak the get/set_taskdata to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27sstatesig: Fix hash equivlanency locked signature issuesRichard Purdie
Using locked signatures with the hash equivalency server ran into problems. We need to: a) Ensure the lockedhashes data object is passed from the core to any individual tasks using the get/set_taskdata methods b) Return a locked singature instead of a unihash c) Write the unihash being used to locked signature lists rather than the calculated taskhash d) Skip warnings of hash mismatch if the hash is a unihash These changes fix esdk builds (which use locked sigs) when a hash equivalence server is in use. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18Remove SSTATE_HASHEQUIV_SERVERJoshua Watt
Removes all references to the SSTATE_HASHEQUIV_SERVER variable. This variable is redundant now that BB_HASHSERVE is present. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18sstatesig: Update server URIJoshua Watt
The server no longer uses a "http://" URI, since it has been updated to use a different protocol. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-08sstatesig: Fix leftover splitting issue from siggen changeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>