aboutsummaryrefslogtreecommitdiffstats
path: root/meta/recipes-core/meta
AgeCommit message (Collapse)Author
2019-01-14target-sdk-provides-dummy: add more perl modules to avoid populate_sdk failureChen Qi
When 'adduser' package, which is from meta-perl layer, is added to rootfs, we will get do_populate_sdk failure like below. Error: Problem: package perl-module-cwd-5.24.4-r0.core2_64 requires perl-module-dynaloader, but none of the providers can be installed - package perl-module-file-temp-5.24.4-r0.core2_64 requires perl-module-cwd, but none of the providers can be installed - package perl-module-dynaloader-5.24.4-r0.core2_64 requires perl-module-config, but none of the providers can be installed - package adduser-3.118-r0.core2_64 requires perl-module-file-temp, but none of the providers can be installed - package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with perl provided by perl-5.24.4-r0.core2_64 This is because adduser depends on some perl modules which are not listed in target-sdk-provides-dummy. So add these perl modules to avoid such failure. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03nativesdk-*-provides-dummy: Fixes to allow correct operation with opkgRichard Purdie
opkg needs the replaces/conflicts/provides in order for this package to function as intended. When enabled, this code caused failures for core-image-sato -c populate_sdk. The reason is that nativesdk-autoconf has several perl dependencies. We need to list more of the things which this package provides/conflicts with in order for it to function correctly. Therefore add the missing entries. (From OE-Core rev: 27412b377dda47daa87bfcb3a41babe53ff8b0a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-20buildtools-tarball: add nativesdk-rpcsvc-protoMingli Yu
Fedora28 repackages rpcgen program to rpcgen package and the program will no longer be part of the glibc-common package. fedora 28: $ rpm -qf /usr/bin/rpcgen rpcgen-1.3.1-4.fc28.x86_64 fedora 27: $ rpm -qf /usr/bin/rpcgen glibc-common-2.26-27.fc27.x86_64 Once build a project on fedora28 host without installing the extra rpcgen package, there comes below error: ERROR: Unable to start bitbake server ERROR: Last 10 lines of server log for this session (/yocto/builds/upgrade2/bitbake-cookerdaemon.log): self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset) File "/yocto/poky/bitbake/lib/bb/cooker.py", line 197, in __init__ self.initConfigurationData() File "/yocto/poky/bitbake/lib/bb/cooker.py", line 356, in initConfigurationData self.databuilder.parseBaseConfiguration() File "/yocto/poky/bitbake/lib/bb/cookerdata.py", line 317, in parseBaseConfiguration raise bb.BBHandledException bb.BBHandledException ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: rpcgen So add nativesdk-rpcsvc-proto to provide the program rpcgen to fix the gap. Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01meta-ide-support: add cmake-nativeChin Huat Ang
Include cmake as part of build tree toolchain so that it is populated into meta-ide-support's recipe-sysroot-native which is prepended to $PATH, this ensures cmake-native is picked up first before host cmake. Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-19dummy-sdk-package.inc: work around MACHINE_ARCH SSTATE_MANMACHMartin Jansa
* since following change: allarch: only enable allarch when multilib is not used the sstate-diff-machines.sh reports different signature for target-sdk-provides-dummy when multilib is enabled === Comparing signatures for task do_populate_sysroot.sigdata between qemux86 and qemux86copy === ERROR: lib32-target-sdk-provides-dummy different signature for task do_populate_sysroot.sigdata between qemux86 and qemux86copy basehash changed from b0a44b2c7003b6b4aa3a023d9cb9fe82 to 3a59fa25ddb6a95aff079d477ebf3457 Variable SSTATE_MANMACH value changed from 'qemux86' to 'qemux86copy' ERROR: target-sdk-provides-dummy different signature for task do_populate_sysroot.sigdata between qemux86 and qemux86copy basehash changed from 9e44f1deb3d15886ee96db1a3332764c to 6b417d08a5113c9b06d13b3681f5ab4f Variable SSTATE_MANMACH value changed from 'qemux86' to 'qemux86copy' It's using: inherit allarch python() { # Put the package somewhere separate to ensure it's never used except # when we want it # (note that we have to do this in anonymous python here to avoid # allarch.bbclass disabling itself) d.setVar('PACKAGE_ARCH', '${DUMMYARCH}') } and DUMMYARCH = "sdk-provides-dummy-target" The difference as shown with bitbake -e before and after reverting allarch.bbclass commit: before revert: $SSTATE_MANMACH [2 operations] set? oe-core/meta/classes/sstate.bbclass:61 "${SSTATE_PKGARCH}" set sstate.bbclass:100 [__anon_111_oe_core_meta_classes_sstate_bbclass] "machineName" pre-expansion value: "machineName" SSTATE_MANMACH="machineName" $SSTATE_PKGARCH set oe-core/meta/classes/sstate.bbclass:11 "${PACKAGE_ARCH}" SSTATE_PKGARCH="sdk-provides-dummy-target" $PACKAGE_ARCH [3 operations] set oe-core/meta/conf/bitbake.conf:150 [_defaultval] "${TUNE_PKGARCH}" set oe-core/meta/conf/documentation.conf:304 [doc] "The architecture of the resulting package or packages." set dummy-sdk-package.inc:12 [__anon_12_oe_core_meta_recipes_core_meta_dummy_sdk_package_inc] "${DUMMYARCH}" pre-expansion value: "${DUMMYARCH}" PACKAGE_ARCH="sdk-provides-dummy-target" after revert: $SSTATE_MANMACH set? oe-core/meta/classes/sstate.bbclass:61 "${SSTATE_PKGARCH}" SSTATE_MANMACH="allarch" $SSTATE_PKGARCH [2 operations] set oe-core/meta/classes/sstate.bbclass:11 "${PACKAGE_ARCH}" set sstate.bbclass:98 [__anon_111__oe_core_meta_classes_sstate_bbclass] "allarch" pre-expansion value: "allarch" SSTATE_PKGARCH="allarch" $PACKAGE_ARCH [4 operations] set oe-core/meta/conf/bitbake.conf:150 [_defaultval] "${TUNE_PKGARCH}" set oe-core/meta/conf/documentation.conf:304 [doc] "The architecture of the resulting package or packages." set oe-core/meta/classes/allarch.bbclass:5 "all" set dummy-sdk-package.inc:12 [__anon_12_oe_core_meta_recipes_core_meta_dummy_sdk_package_inc] "${DUMMYARCH}" pre-expansion value: "${DUMMYARCH}" PACKAGE_ARCH="sdk-provides-dummy-target" the relevant part of the anonymous python in sstate.bbclass: elif bb.data.inherits_class('allarch', d) and d.getVar("PACKAGE_ARCH") == "all": d.setVar('SSTATE_PKGARCH', "allarch") else: d.setVar('SSTATE_MANMACH', d.expand("${PACKAGE_ARCH}")) So with allarch.bbclass change, the PACKAGE_ARCH isn't set to "all" because multilib is enabled, but that causes sstate.bbclass to set SSTATE_MANMACH to MACHINE instead of SSTATE_PKGARCH allarch, where it got MACHINE is still a bit of mystery to me. Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30buildtools-tarball: add python3-testtools and python3-subunitRobert Yang
They are required by oeqa. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-24uninative-tarball: Add nativesdk-libnss-nis to resolve glibc symbol issuesRichard Purdie
We need this to avoid symbol mismatch issues for binaries that use this on newer systems which then won't run on older ones where it isn't present. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06buildtools-tarball: add nativesdk-libnss-nisChen Qi
Recent glibc change removed libnss-nis module from glibc and a new recipe libnss-nis.bb was added. After this change, we need to make sure nativesdk-libnss-nis is also included in buildtools-tarball, otherwise, we may encounter the following error when using 'tar' command from buildtools-tarball. tar: relocation error: /lib/x86_64-linux-gnu/libnss_nis.so.2: \ symbol _nsl_default_nss version GLIBC_PRIVATE not defined \ in file libnsl.so.1 with link time reference This error occured on my ubuntu16.04 host with 'nis' configured in /etc/nssswitch.conf. So add nativesdk-libnss-nis to buildtools-tarball to fix this problem. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-07toolchain-scripts: pass env to post-relocateMartin Kelly
It's useful for the post-relocate scripts to be able to see the SDK environment, for example to see the values of CC, CXX etc. in order to dynamically generate toolchain files. To enable this, source the SDK environment script prior to calling the relocate scripts. Signed-off-by: Martin Kelly <mkelly@xevo.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-15uninative-tarball: Add libjis and euc-jp gconv filesKhem Raj
packages like fontforge-native fail with mysterious errors like | ../../git/inc/gwwiconv.h:44:21: error: conflicting types for ‘gww_iconv_close’ | #define iconv_close gww_iconv_close | ^~~~~~~~~~~~~~~ | ../../git/inc/gwwiconv.h:37:13: note: previous declaration of ‘gww_iconv_close’ was here | extern void gww_iconv_close( gww_iconv_t cd); | ^~~~~~~~~~~~~~~ The reason behind this is that a check for iconv fails during native configure run, the check fails because the autoconf test to check for iconv pokes for these gconv's in test runs before declaring iconv support successful. Therefore when uninative is active the package fails to build but when uninative is inactive all works fine. this patch fixes that Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-04-07classes: Use ${MACHINE_ARCH} instead of ${MACHINE} for stamp-extra-info task ↵Peter Kjellerstedt
flag Without this change, there will be two sstate index files in tmp/sstate-control for any machine that contains a dash in the name. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-07nativesdk-glibc: Split glibc and libcrypt to use libxcrypt insteadRichard Purdie
Fedora28[1] has decided to go ahead and use libxcrypt to replace libcrypt from glibc despite the change not having merged into glibc upstream yet. This breaks the use of uninative in OE on fedora28 since binaries there are now using new symbols only found in libxcrypt. libxcrypt is meant to be backwards compatible with libcrypt but not the reverse. Since this will impact OE in the next release cycle, this changes nativesdk only to use this new model and adds libxcrypt to work in that case. This allows us to build a uninative which is compatible with fedora28 and previous other OSes. In order to work, recipes will now need to depend on virtual/crypt where they use libcrypt since its now a separate library and we can't depend on it from glibc to preseve backwards compatibility since glibc needs to build first. For now, only the problematic nativesdk recipes have been fixed up. For target use, the default provider remains glibc for now. Assuming this change is merged into upstream glibc, we will need to roll this change out for the target but we will do this in the next release cycle when we can better deal with the resulting bugs. [1] https://fedoraproject.org/wiki/Changes/Replace_glibc_libcrypt_with_libxcrypt Original patch from Charles-Antoine Couret <charles-antoine.couret@essensium.com>, tweaked by RP to add virtual provides, SkipRecipe for libxcrypt and other minor tweaks. Signed-off-by: Charles-Antoine Couret <charles-antoine.couret@essensium.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-06testexport-tarball: sync with buildtools-tarballRoss Burton
This is basically a copy/paste of buildtools-tarball so copy some of the recent changes to buildtools across. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-03-01populate_sdk: install UTF-8 locales in SDKsRoss Burton
As glibc 2.27 can't read older locale-archives, SDKs using glibc 2.27 on hosts using glibc earlier than 2.27 won't be able to find any locales, so bitbake won't start and Python can't use UTF-8. So by default install all locales into the SDK. Special-case Extensible SDKs by installing no locales as they ship glibc in a buildtools, and that will have the locales. Locale installation requires cross-localedef, so add that to DEPENDS. Also remove the explicit en_US addition in buildtools-tarball as it is now redundant. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01meta-world-pkgdata: This recipe is machine specific, mark as suchRichard Purdie
The recipe depends on many machine specific tasks and should be marked as machine specific itself. This fixes signature tests after some dependency issues were fixed at the bitbake level which exposed that issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-15toolchain-shar-extract: Add post-relocate scriptsJoshua Watt
Recipes can now install post-relocation scripts which will be run when the SDK is installed. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-10gnupg: use native version for signing, rather than one provided by hostAlexander Kanavin
Using host gpg has been problematic, and particularly this removes the need to serialize package creation, as long as --auto-expand-secmem is passed to gpg-agent, and gnupg >= 2.2.4 is in use (https://dev.gnupg.org/T3530). Sadly, gpg-agent itself is single-threaded, so in the longer run we might want to seek alternatives: https://lwn.net/Articles/742542/ (a smaller issue is that rpm itself runs the gpg fronted in a serial fashion, which slows down the build in cases of recipes with very large amount of packages, e.g. glibc-locale) Note that sstate signing and verification continues to use host gpg, as depending on native gpg would create circular dependencies. [YOCTO #12022] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-10nativesdk/sdk: Update sdk dummy providersRichard Purdie
When we migrated rpm v5 -> v4, we lost the ability to drop "per file" dependencies from the rpm backend for things like "/bin/bash" and "/usr/bin/env" which meant the sdks were becomming 'bloated'. This restores the functionality using a dummy package, similarly to the way the buildtools perl issue was addressed. It also removes the non-functional old code so as not to confuse people in future. I ran into this problem trying to filter dependencies to only rpms a build directly depends upon and it turns out we have some determinism issues in this area so this is something key to fix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-26wic: remove systemd-boot for x32Saul Wold
Currently systemd-boot actually incorporates libgcc, since the systemd-boot needs to be built with 64bit instructions it can not use the x32 based libgcc. Use the new override to ensure it gets overriden, linux-gnux32 could not be used because x86-64 has higher priority. Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21Add Go toolchain supportOtavio Salvador
This adds the meta-go-toolchain. It enables the generation of a Go toolchain allowing for cross compiling for target architecture. Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-08-30signing-keys: do not use DISTRO_VERSION in key namesAlexander Kanavin
DISTRO_VERSION may contain the current date, and so is prone to mismatches particularly when keys are created on one date, and dnf is configured to use the keys on another date. [YOCTO #11983] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: extend list of used toolsEd Bartosh
Added sfdisk, e2fsck, mkswap, resize2fs, mkdosfs to the list of used tools in Disk class. They're going to be used in 'wic write' implementation. Added dependency to util-linux to wic-tools to ensure that sfdisk and mkswap are available from wic-tools native sysroot. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16signing-keys: delete unused tasksMarkus Lehtonen
In particular, do_unpack needs to be deleted as it was causing missing subpackages. Do_unpack cleans the builddir and if it was run after do_get_public_keys it cleaned up all keys causing no packages to be built. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-27wic-tools: add dependency to e2fsprogs-nativeEd Bartosh
Added e2fsprogs-native to the list of dependencies for wic-tools as all fs-related utilities have to be in this list. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25buildtools-tarball: drop deltask package/packagedataMing Liu
They are redundant since nopackages are being inherited. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25uninative-tarball: drop deltask package/packagedataMing Liu
They are redundant since nopackages are being inherited. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25package-index: inherit nopackagesMing Liu
Drop deltask do_package* syntax, inheriting nopackages instead. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25wic-tools: ensure pseudo is availablePaul Eggleton
wic will attempt to use pseudo from the wic-tools sysroot to run, but it was only sure to be in there if do_install had executed - which is not the case if it had been restored from sstate, in which case it failed horribly as seen when running the wic.Wic.test_fs_types and test_mkfs_extraopts tests on the Yocto Project autobuilder recently. Add an explicit dependency on pseudo-native to ensure it's always there. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-17wic-tools: don't create wic-tools.envEd Bartosh
wic-tools.env was uses only when wic is run from bitbake. As wic doesn't use wic-tools anymore in this mode there is no need for this file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-04-05meta-ide-support: exclude this recipe from rm_workbrian avery
The results of meta-ide-support are used by utilities like runqemu and runqemu-extract-sdk. Since the usr/bin that meta-ide-support creates is subsequently removed bu rm_work, we exclude this recipe from the rm_work list. [YOCTO #11119] Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-31wic-tools: add btrfs-tools squashfs-tools to DEPENDSEd Bartosh
Added btrfs-tools-native and squashfs-tools-native to DEPENDS as wic uses these tools to support btrfs and squashfs filesystems. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-03-23uninative-tarball: glibc-gconv-{utf-16, cp1252} for binutils windresNathan Rossi
The windres binutils binary which is used for Windows resource files requires utf-16 and cp1252 encoding support in order to correctly generate resource files with strings. As such when using uninative to build mingw resources for a nativesdk target the windres binary is executed on the native host, thus using the uninative libc and gconv modules. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-02meta-estsdk-toolchain: use WORKDIR instead of D to hold locked-sigsRoss Burton
do_locked_sigs() uses ${D} to hold a file as input to the sstate staging code. Generally this isn't a problem as it happens after packaging, but sometimes (if the recipe is rebuild, perhaps) do_package() will see the file inside ${D} and fail because there is an uninstalled file. Solve this by using a directory under ${WORKDIR} so it can't be packaged. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15classes: Drop now unneeded update_data callsRichard Purdie
Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-02wic-tools: Do not remove sysroot with rm_work, because wic needs it.Kristian Amlie
The sysroot of wic-tools is needed for wic, but if rm_work is enabled, it will be removed before wic has a chance to use it, hence this fix. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-01meta-environment: Clean up the task structure to reduce manifest warningsRichard Purdie
This puts the dependencies on the correct task and removes pointless noexec tasks allowing for a slightly cleaner task structure. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31wic-tools: add dependency to systemd-bootEd Bartosh
Added systemd-boot to the list of dependencies of wic-tools as wic bootimg-efi plugin depends on it. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: change location of .env filesEd Bartosh
Current location of .env files $STAGING_DIR/imagedata. It doesn't depend on machine and be rewritten by the builds for different machines. Changed location to $STAGING_DIR/$MACHINE/imagedata to avoid .env files to be rewritten. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-28image/kernelsrc/packagegroups/recipes: Remove uneeded noexec tasksRichard Purdie
We used to have issues removing tasks like do_fetch due to implications for targets like world and universe. These have now been resolved. Removing uneeded tasks has advantages compared to noexec since it means that accidentally left in dependencies are no longer needed/processed (e.g. do_patch depends on quilt-native). This cleans up a number of cases which local analysis highlighted as being unneeded leading to slightly cleaner task graphs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23wic: add wic-tools recipeEd Bartosh
This meta recipe is for building tools used by wic. It allows wic to find tools in recipe specific sysroot as all tools will be present in wic-tools sysroot. NOTE: task do_build_sysroot is created to ensure that sysroot is re-populated when package is built. Otherwise it will be taken from sstate and sysroot will not be populated. Generated wic-tools.env file for wic to be able to get values of wic-tools variables when wic run from bitbake. Also add dependency to grub-efi Without grub-efi test_iso_image test case fails with this error: AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal' returned non-zero exit status 1: Error: Please build grub-efi first Fixed by adding dependency wic-tools -> grub-efi. [RP: Added syslinux exclusion for non-IA arches] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23Switch to Recipe Specific SysrootsRichard Purdie
This patch is comparatively large and invasive. It does only do one thing, switching the system to build using recipe specific sysroots and where changes could be isolated from it, that has been done. With the current single sysroot approach, its possible for software to find things which aren't in their dependencies. This leads to a determinism problem and is a growing issue in several of the market segments where OE makes sense. The way to solve this problem for OE is to have seperate sysroots for each recipe and these will only contain the dependencies for that recipe. Its worth noting that this is not task specific sysroots and that OE's dependencies do vary enormously by task. This did result in some implementation challenges. There is nothing stopping the implementation of task specific sysroots at some later point based on this work but that as deemed a bridge too far right now. Implementation details: * Rather than installing the sysroot artefacts into a combined sysroots, they are now placed in TMPDIR/sysroot-components/PACKAGE_ARCH/PN. * WORKDIR/recipe-sysroot and WORKDIR/recipe-sysroot-native are built by hardlinking in files from the sysroot-component trees. These new directories are known as RECIPE_SYSROOT and RECIPE_SYSROOT_NATIVE. * This construction is primarily done by a new do_prepare_recipe_sysroot task which runs before do_configure and consists of a call to the extend_recipe_sysroot function. * Other tasks need things in the sysroot before/after this, e.g. do_patch needs quilt-native and do_package_write_deb needs dpkg-native. The code therefore inspects the dependencies for each task and adds extend_recipe_sysroot as a prefunc if it has populate_sysroot dependencies. * We have to do a search/replace 'fixme' operation on the files installed into the sysroot to change hardcoded paths into the correct ones. We create a fixmepath file in the component directory which lists the files which need this operation. * Some files have "postinstall" commands which need to run against them, e.g. gdk-pixbuf each time a new loader is added. These are handled by adding files in bindir with the name prefixed by "postinst-" and are run in each sysroot as its created if they're present. This did mean most sstate postinstalls have to be rewritten but there shouldn't be many of them. * Since a recipe can have multiple tasks and these tasks can run against each other at the same time we have to have a lock when we perform write operations against the sysroot. We also have to maintain manifests of what we install against a task checksum of the dependency. If the checksum changes, we remove its files and then add the new ones. * The autotools logic for filtering the view of m4 files is no longer needed (and was the model for the way extend_recipe_sysroot works). * For autotools, we used to build a combined m4 macros directory which had both the native and target m4 files. We can no longer do this so we use the target sysroot as the default and add the native sysroot as an extra backup include path. If we don't do this, we'd have to build target pkg-config before we could built anything using pkg-config for example (ditto gettext). Such dependencies would be painful so we haven't required that. * PKDDATA_DIR was moved out the sysroot and works as before using sstate to build a hybrid copy for each machine. The paths therefore changed, the behaviour did not. * The ccache class had to be reworked to function with rss. * The TCBOOTSTRAP sysroot for compiler bootstrap is no longer needed but the -initial data does have to be filtered out from the main recipe sysroots. Putting "-initial" in a normal recipe name therefore remains a bad idea. * The logic in insane needed tweaks to deal with the new path layout, as did the debug source file extraction code in package.bbclass. * The logic in sstate.bbclass had to be rewritten since it previously only performed search and replace on extracted sstate and we now need this to happen even if the compiled path was "correct". This in theory could cause a mild performance issue but since the sysroot data was the main data that needed this and we'd have to do it there regardless with rss, I've opted just to change the way the class for everything. The built output used to build the sstate output is now retained and installed rather than deleted. * The search and replace logic used in sstate objects also seemed weak/incorrect and didn't hold up against testing. This has been rewritten too. There are some assumptions made about paths, we save the 'proper' search and replace operations to fixmepath.cmd but then ignore this. What is here works but is a little hardcoded and an area for future improvement. * In order to work with eSDK we need a way to build something that looks like the old style sysroot. "bitbake build-sysroots" will construct such a sysroot based on everything in the components directory that matches the current MACHINE. It will allow transition of external tools and can built target or native variants or both. It also supports a clean task. I'd suggest not relying on this for anything other than transitional purposes though. To see XXX in that sysroot, you'd have to have built that in a previous bitbake invocation. * pseudo is run out of its components directory. This is fine as its statically linked. * The hacks for wayland to see allarch dependencies in the multilib case are no longer needed and can be dropped. * wic needed more extensive changes to work with rss and the fixes are in a separate commit series * Various oe-selftest tweaks were needed since tests did assume the location to binaries and the combined sysroot in several cases. * Most missing dependencies this work found have been sent out as separate patches as they were found but a few tweaks are still included here. * A late addition is that extend_recipe_sysroot became multilib aware and able to populate multilib sysroots. I had hoped not to have to add that complexity but the meta-environment recipe forced my hand. That implementation can probably be neater but this is on the list of things to cleanup later at this point. In summary, the impact people will likely see after this change: * Recipes may fail with missing dependencies, particularly native tools like gettext-native, glib-2.0-native and libxml2.0-native. Some hosts have these installed and will mask these errors * Any recipe/class using SSTATEPOSTINSTFUNCS will need that code rewriting into a postinst * There was a separate patch series dealing with roots postinst native dependency issues. Any postinst which expects native tools at rootfs time will need to mark that dependency with PACKAGE_WRITE_DEPS. There could well be other issues. This has been tested repeatedly against our autobuilders and oe-selftest and issues found have been fixed. We believe at least OE-Core is in good shape but that doesn't mean we've found all the issues. Also, the logging is a bit chatty at the moment. It does help if something goes wrong and goes to the task logfiles, not the console so I've intentionally left this like that for now. We can turn it down easily enough in due course. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20meta-environment: Ensure all multilib dependencies are accounted forRichard Purdie
Currently the recipe depends on the mulitlib libcs all being built but the dependencies don't account for this. Fix the DEPENDS so that the requires pieces are all built first rather than relying on luck. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19classes/populate_sdk_ext: force a known value for TMPDIRPaul Eggleton
If TMPDIR is configured to be somewhere outside of TOPDIR (a not uncommon configuration where you have multiple disks and space on /home is at a premium) then our attempt to find out the location of paths under TMPDIR by using a relative path led to horribly broken paths ending up in the eSDK. To save pain, just force a known value for TMPDIR (i.e. ${TOPDIR}/tmp) and then we can assume that everywhere else. Fixes [YOCTO #10797]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16meta: remove True option to getVar callsJoshua Lock
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-10-28Remove LIC_FILES_CHKSUM from recipes without SRC_URIOlaf Mandel
LICENSE and LIC_FILES_CHKSUM apply to the sources specified by SRC_URI, not to the recipe itself. As such a license declaration for a source-less recipe makes little sense. The LICENSE declaration is mandatory, but LIC_FILES_CHKSUM can be removed in such cases. Remove the LIC_FILES_CHKSUM declarations from all recipes that do not need it. CC: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-07classes/populate_sdk_ext: add symlinks and unfsd to support Eclipse pluginPaul Eggleton
The Yocto Project Eclipse plugin requires that runqemu and unfsd are accessible within the SDK, and indeed the standard SDK has these. This turns out to be fairly easy to do - we just need to add unfsd and symlink it, runqemu and a few other scripts into the SDK's bin directory. Fixes [YOCTO #10214]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-26meta-environment: ensure corret TOOLCHAIN_CONFIGSITE_NOCACHE valueChen Qi
Expand TOOLCHAIN_CONFIGSITE_NOCACHE value immediately before inheriting cross-canadian to avoid HOST_ARCH being changed from TARGET_ARCH to SDK_ARCH, thus ensuring its correct value. [YOCTO #10255] Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23buildtools/uninative-tarball: Fix deployment overlap issuesRichard Purdie
We still have problems where deploying SDKMACHINE=i686 can cause removal of SDKMACHINE=x86_64 artefacts. The reason is that x86_64 is a BUILD_ARCH as well as an SDK_ARCH and the manifest namespaces overlap. To fix this, set PACKAGE_ARCH and the stamp-extra-into to include SDK_OS. SDK_OS may not be entirely correct but it is what sstate.bbclass uses for nativesdk and fixing that is a separate issue. This is confirmed to resolve artefact problems on the AB which have been delaying a new uninative release. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23buildtools-tarball: improve stamp independenceJoshua Lock
buildtools-tarball shouldn't be regenerated when MACHINE changes, nor should variants for other SDKMACHINE be removed from the deploy directory when SDKMACHINE changes. Remove target architecture dependencies so that deploy artefacts can overlap. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>