summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe
AgeCommit message (Collapse)Author
2020-02-04cmake: prefer CMAKE_BUILD_PARALLEL_LEVELDaniel McGregor
cmake 3.12 introduced this environment variable. Prefer it to passing PARALLEL_MAKE and PARALLEL_MAKEINST on the cmake command line, because it gets passed to second stage cmake invocations while command-line arguments do not (for example, multi-stage clang builds) Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27lib/oe/package: remove unneeded npm_split_package_dirs functionJean-Marie LEMETAYER
The npm_split_package_dirs function was used by the recipetool when creating npm recipes. This is not the case anymore. Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22lib/oe/package_manager: Improve locale-archive reproducibilityAlex Kiernan
The generation of locale-archive depends on the order of the input files. Fix the order by sorting the file list. Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19machine_dict: Add i686 to the ELF machine dictionaryAlejandro Enedino Hernandez Samaniego
An error like the following is thrown when building baremetal applications on some x86 architectures: (machine, osabi, abiversion, littleendian, bits) \ = oe.elf.machine_dict(d)[target_os][target_arch] Exception: KeyError: i686 Since the i686 (target_arch) key does not exist in the dictionary. Add the key to fix the error. Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19classes, conf, lib: Add support for powerpc64leKhem Raj
LE is default for modern powerpc64, power8+ PowerPC64 Little Endian Linux ABI specifies Power8 as the minimum ISA. The basic ABI can run on earlier versions of the 64 bit PowerPC ISA, but it was helpful to define a new, minimum instruction set for Linux distribution releases during the switch to Little Endian. Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16sstatesig: Improve debug output if getpwuid() failsRichard Purdie
If getpwduid fails, we don't see which file it failed on which is key information to aid debugging. Print this information when exceptions are raised. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10lib/oe/package_manager.py: put the sdk_provides_dummy_target upfrontAlexander Kanavin
Otherwise, the package utilizing this architecure does not get properly prioritized by rpm/dnf. This isn't seen single-lib builds (as the architecture ends up upfront anyway), but is triggered in multulib where the architecture may end up in the middle of the list due to multilib expansion/trimming duplicates. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03sstatesig: Avoid resetting taskhash within siggen for locked sigsRichard Purdie
Since get_unihash uses taskhash as a key internally, changing it means different bebahour when locked sigs are active verses not active. Under corner cases this leads to a signature mismatch. Avoid this by by adding a wrapper for the place its externally exposed and then not changing the internals. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03sstatesig: Fix locked signature handling with unihashesRichard Purdie
get_taskhash will call get_unihash internally in the parent class. We need to disable our filter of it whilst this runs else incorrect hashes can be calculated. This is believed to be causing the locked signatures test to fail under some circumstances (depending on whether earlier hashes are being remapped). [YOCTO #13605] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02sstatesig: Test cross/native hashserv method extensionRichard Purdie
We can have one taskhash which represents multiple native/cross sstate objects since they're stored by BUILD_ARCH or possibly host distro (or host gcc version). We need to put these into separate namespaces on hashserv since their outhashes will never match and we need deterministic lookups for the different namespaces. Use this extramethod option to handle this. This fixes several problematic failures on the autobuilder. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-29chrpath: Cleanup and fix previous patchRoss Burton
Ensure self.data isn't accessed without assignment. Also clean up old style popen use and replace with modern/simpler subprocess. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28recipeutils: set SRCPV for devupstream variantAlexander Kanavin
Otherwise bitbake is unable to expand it if the main variant is not using a SCM fetcher. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-12-28rootfs: don't use oe.cachedpathRoss Burton
Unless cachedpath is used correctly then it's just a glorified clone of os.walk, but without any of the recent optimisations in os.walk. In this codepath there is no point to using cachedpath. Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-12-16sstatesig: Further optimiseRichard Purdie
Optimise the call into the parent function to be only when needed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15sstatesig: Optimise get_taskhash for hashequivRichard Purdie
With hashequiv the get_taskhash function is called much more regularly and contains expensive operations. This these don't change based upon hash in a given build, improve the caching within the function to reduce overhead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10meta/lib/oe/package_manager.py: Enable sha256 checksums in opkg indexerHaris Okanovic
Pass `--checksum md5` and `--checksum sha256` to opkg-make-index. Sha256 checksum enables more reliable install-time validation of IPKs. This is particularly useful when installing from signed feeds -- I.e. feeds using signed Packages index files that deliver otherwise unsigned IPKs. Such feeds rely on hash validation of enclosed IPKs to thwart tampering. After download, opkg verifies IPK's checksum against the (signed) Packages index file. Weak hashes like md5 are prone to collision and therefore tampering. The md5 checksum is purely for backward compatibility. Sha256 validation was recently added to opkg. Newer builds of opkg will use it. Older builds still look for an md5 checksum. Md5 is deprecated and should be removed once old build are phased out. Testing: I ran `bitbake package-index` after building a few IPKs and verified MD5Sum and SHA256sum attributes are present in Packages. Using opkg-utils 0.4.0. Performance Impact: It takes about 40 seconds to cleanly re-index 8000 IPKs on an Intel Xeon E5-1620 machine. This was previously about 20 seconds. NOTE: It's recommended to delete all Packages* files after applying this patch. Otherwise, some IPKs won't have sha256. Signed-off-by: Haris Okanovic <haris.okanovic@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-31IMAGE_LINGUAS_COMPLEMENTARY: auto-add language packages other than localesAndreas Müller
This change allows layers adding packages other than locales automatically to images based on languages selected in IMAGE_LINGUAS. E.g if a layer has recipes creating packages as: | ${PN}-foo-en / ${PN}-foo-de / ${PN}-foo-it / .. it would set | IMAGE_LINGUAS_COMPLEMENTARY_append = " *-foo-%s" in its layer.conf to enable auto-adding. Signed-off-by: Andreas Müller <schnitzeltony@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-10-24buildhistory-analysis: filter out -src changes by defaultRoss Burton
Like the -dbg package, this package is automatically generated and contains source filenames. We expect this to change on every upgrade, so don't show the differences unless the user wants to see all changes. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23lib/oe/package_manager: Use with to control file handle lifetimeOla x Nilsson
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23lib/oe/packagedata: Use with to control file handle lifetimeOla x Nilsson
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23oe.types.path: Use with to control file handle lifetimeOla x Nilsson
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08lib/oe/terminal.py: fix gnome-terminal start behaviorTrevor Gamblin
[Bugzilla Bug 13201] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=13201 Newer versions of gnome-terminal (3.32.0 and up) are not starting as expected for commands e.g. "bitbake -c devshell zlib". This manifests as the instance appearing as a new tab rather than a new window. Fix this (and maintain new window preferred behavior) by changing the "-x" option to "--" as per the warning message, avoiding deprecated options: # Option “--command” is deprecated and might be removed in a later version of gnome-terminal. # Use “-- ” to terminate the options and put the command line to execute after it. Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27lib/oe/lsb: Make sure the distro ID is always lowercasedPeter Kjellerstedt
In commit 8689e561 (lib/oe/lsb: attempt to ensure consistent distro id regardless of source), the distro ID returned by oe.lsb.distro_identifier() was lowercased, but only if a release version is also present. This changes the code to always lowercase the distro ID, including the default distro ID "unknown", which is used if no other ID can be identified. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27lib/sstatesig: Fix class inheritance problemsRichard Purdie
The locked sigs class needs to be inherited after the hashequiv mixin so that get_unihash can correctly wrap the underlying hashequiv function. To do this turn the locked sigs class into a second mixin, then the order can be correctly handled. Tweak the get/set_taskdata to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27sstatesig: Fix hash equivlanency locked signature issuesRichard Purdie
Using locked signatures with the hash equivalency server ran into problems. We need to: a) Ensure the lockedhashes data object is passed from the core to any individual tasks using the get/set_taskdata methods b) Return a locked singature instead of a unihash c) Write the unihash being used to locked signature lists rather than the calculated taskhash d) Skip warnings of hash mismatch if the hash is a unihash These changes fix esdk builds (which use locked sigs) when a hash equivalence server is in use. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27sdk: Install nativesdk locales for all TCLIBC variantsKhem Raj
install_locales() here is actually operating on nativesdk and only glibc is the default library for nativesdk, since thats what most of desktop/server distros use, therefore bailing out based on TCLIBC is not needed here, since nativesdk-glibc would be required for all non-glibc targetting SDKs as well. Fixes SDK install time error ERROR: OE-core's config sanity checker detected a potential misconfiguration. Either fix the cause of this error or at your own risk disable the checker (see sanity.conf). Following is the list of potential problems / advisories: Your system needs to support the en_US.UTF-8 locale. ERROR: SDK preparation failed Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19populate_sdk_ext: Introduce mechanism to keep nativesdk* sstate in esdkJaewon Lee
When doing a devtool build-sdk from within an esdk all nativesdk components would be rebuilt. This patch introduces SDK_INCLUDE_NATIVESDK flag to toggle the inclusion of nativesdk packages when creating the esdk sstate Currently locked-sigs.inc is generated during do_sdk_depends which doesn't pull in nativesdk packages. Generating another locked-sigs.inc in do_populate_sdk_ext and pruning it to only nativesdk* packages by using a modified version of the already existing function prune_locked_sigs and merging it with the current locked-sigs.inc Also adding SDK_INCLUDE_NATIVESDK tasklistfn to the logic surrounding setting tasklist file to not prune esdk sstate during creation [YOCTO #13261] Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18Remove SSTATE_HASHEQUIV_SERVERJoshua Watt
Removes all references to the SSTATE_HASHEQUIV_SERVER variable. This variable is redundant now that BB_HASHSERVE is present. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18sstatesig: Update server URIJoshua Watt
The server no longer uses a "http://" URI, since it has been updated to use a different protocol. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16classextend.py: don't extend file for file dependencyChangqing Li
Fix error like: lib32-e2fsprogs-1.45.3-r0 do_package_qa: QA Issue: /usr/sbin/e2scrub_all contained in package lib32-e2fsprogs-e2scrub requires /bin/bash, but no providers found in RDEPENDS_lib32-e2fsprogs-e2scrub For some lib32 packages(eg: lib32-bash, lib32-sed) which probvides files, extend is not needed Eg: RPROVIDES of lib32-bash expects to have /bin/bash, with original extend, it will become lib32-/bin/bash, then will cause above error Fix by don't extend file dependency, and skip multilib check for file dependency in do_package_qa to avoid error like: WARNING: lib32-bash-5.0-r0 do_package: QA Issue: lib32-bash package lib32-bash - suspicious values '/bin/bash /bin/sh' in RPROVIDES [multilib] Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06lib/oe/gpg_sign.py: Remove unwanted importVijai Kumar K
Remove unwanted import of oe.utils. Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-08-08sstatesig: Fix leftover splitting issue from siggen changeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstatesig: Updates to match bitbake siggen changesRichard Purdie
Update the metadata to correspond to the bitbake siggen task specification format change. This standardises on "<fn>:<task>" everywhere rather than the "." delimiter that was being used in some places. This is an API breaking change but means we now have a consistent format being used throughout the codebase without compatibility APIs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstate/sstatesig: Update to new form of BB_HASHCHECK_FUNCTIONRichard Purdie
Bitbake has updated to a cleaned up form of BB_HASHCHECK_FUNCTION, adapt to this cleanup. This is an API breaking change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstatesig: Update to handle BB_HASHSERVERichard Purdie
Update the code to handle BB_HASHSERV, an autostarted bitbake internal hash equivalency server suited to local developer usage. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstatesig: Adpat to recent bitbake hash equiv runqueue changesRichard Purdie
Upstream bitbake now hanes preserving the unihash data itself, drop this usage of persist_data which was extremely problematic due to concurrent task access issues, particulary on loaded systems. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstatesig: Add debug for incorrect hash server settingsRichard Purdie
If the hash server settings are incorrect, show the user useful error messages instead of tracebacks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-06sstatesig: Move unihash siggen code to bitbakeRichard Purdie
This code is closely tied with the hash server in bitbake and also means we can't relibably test the hashserv runqueue functionality without OE metadata. Moving this to bitbake as a MixIn class makes most sense and encourages code collaboration and reuse as well as enabling easier and more accurate testing of the APIs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-03classes/lib: Remove bb.build.FuncFailedRichard Purdie
Whilst seemingly a good idea, this exception doesn't really serve any purpose that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't pythonic. Its not used in many places, lets clean up those and remove usage of it entirely. It may ultimately be dropped form bitbake entirely. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-26package: Improve determinismRichard Purdie
Its possible in cases with multiple shlib providers we were not being deterministic. Add in a couple of sorted() calls to fix the shlibs and pkgconfig cases with this potential issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18package_manager: Ensure the base-feed directory existsAlistair Francis
Ensure that the /etc/opkg directory exists before we try to create a file there. Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18oe/copy_buildsystem: move layer into layers directoryAndrej Valek
Layers could be located outside from poky but inside the build directory. This case should be covered in eSDK. meta-abc meta-def/meta-ghi meta-def/poky meta-def/meta-oe/meta-oe ... It should take all enabled layers and put them into 'layers' dir during build-time with respecting new relative path to poky. layers/meta-abc layers/meta-ghi layers/poky layers/meta-oe/meta-oe ... Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-05buildhistory_analysis: ignore ownership for sysroot diffsRoss Burton
The sysroot isn't populated under pseudo, so ownership differences should be ignored. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-28buildhistory: report sysroot changesRoss Burton
Now that the sysroot is written into the build history, write it out. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19rootfs: Fix dependency for every dpkg runRicardo Ribalda Delgado
Avoid getting a warning on do_rootfs. Fixes: NOTE: Installing the following packages: libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb WARNING: Unable to install packages. Command '/workdir/build/tmp/work/qt5122-poky-linux/bottlecam-image/1.0-r0/recipe-sysroot-native/usr/bin/apt-get install --force-yes --allow-unauthenticated libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb' returned 100: Reading package lists... Building dependency tree... Reading state information... Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10multiconfig: Adapt to bitbake switch 'multiconfig' -> 'mc'Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07gpg_sign/selftest: Fix secmem parameter handlingRichard Purdie
We keep seeing "cannot allocate memory" errors from rpm when signing packages on the autobuilder. The following were tried: * checking locked memory use (isn't hitting limits) * Restricting RPM_GPG_SIGN_CHUNK to 1 * Limiting to 10 parallel do_package_write_rpm tasks * Allowing unlimied memory overcommit * Disabling rpm parallel compression and the test still failed. Further invetigation showed that the --auto-expand-secmem wasn't being passed to gpg-agent which meant the secmem couldn't be expanded hence the errors when there was pressure on the agent. The reason this happens is that some of the early gpg commands can start the agent without the option and it sticks around in memory so a version with the correct option may or may not get started. We therefore add the option to all the key gpg calls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04lib/oe/reciputils.py: support character `+' in git pvHongxu Jia
While pv contains char `+' (such as ncurses 6.1+20181013), it was incorrectly converted to `6.1' In this commit: - Convert [^\+]* to .*? Support pv to contain char `+' and not greedy match (Such as ncurses 6.1+20181013) - Add [^\+] to sfx match Support sfx contains extra chars between `+' and `git' (such as asciidoc "8.6.9+py3-gitAUTOINC+618f6e6f6b") - Make sfx and rev greedy match Run `devtool check-upgrade-status --all' in poky and compare results, only one difference on ncurses version: Without the commit: INFO: ncurses 6.1 6.1+20181013 Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 With the commit: INFO: ncurses 6.1+20181013 MATCH Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09patch: add minver and maxver parametersRoss Burton
Add minver/maxver parameters to limit patch application by comparing PV. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-25 20:46:54 +0000