summaryrefslogtreecommitdiffstats
path: root/meta/classes
AgeCommit message (Collapse)Author
2020-11-24uninative: Don't use single sstate for pseudo-nativeRichard Purdie
pseudo-native is a bit special. It conditionally compiles in support for xattr, statx and statvfs amongst other options. If a pseudo-native binary is used on a system where these functions are present but it wasn't compiled in we see hard to debug permissions problems. An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target oe-selftest which shows a cryptic error: File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target self.assertEqual(filelist1, filelist2) File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual assertion_func(first, second, msg=msg) File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual self.assertSequenceEqual(list1, list2, msg, seq_type=list) File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual self.fail(msg) File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail raise self.failureException(msg) AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] First differing element 0: '-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor' '-rwxr-xr-x root root /etc/init.d/mdmonitor' This is due to a version of pseudo without statx being used on a system where ls uses statx, hence the files are displayed as 6000.6000 instead of root.root. Avoid this by always building pseudo-native for the specific distro in question rather than using a universal sstate feed. This hopefully fixes one of the mysterious AB-INT issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24fs-perms: Ensure /usr/src/debug/ file modes are correctRichard Purdie
If files are copied into /usr/src/debug directly from WORKDIR (e.g. makedevs) we'd get the permissions from the checkout which would depend on the host umask. Avoid this and be deterministic by setting the file modes consistently. Core code copies the files in so we're responsible for the permissions. Unfortunately to force this change to apply we need to invalidate both the package tasks and the hash equivalance mappings since file mode 'corruption' already made it into the output hashes (both input options were mapped to the output hashes). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24kernel-uboot: allow compression option to be configurableSinan Kaya
While some platforms might choose to compress the kernel by default especially when boot medium is slow, others want uncompressed kernel. The choice of decompression speed vs. load speed is very platform dependent. Allow platform to choose the option here. Signed-off-by: Sinan Kaya <okaya@kernel.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24distutils-common-base: fix LINKSHARED expansionAnuj Mittal
Add the missing $ so SECURITY_CFLAGS actually gets expanded. Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24cve-check: show real PN/PVRoss Burton
The output currently shows the remapped product and version fields, which may not be the actual recipe name/version. As this report is about recipes, use the real values. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24license_image.bbclass: use canonical name for license filesVyacheslav Yurkov
When copying license files to the image rootfs, i.e to /usr/share/common-licenses, a canonical name of a license should be used, otherwise duplicated files end up in common-licenses directory. For example, GPL-2.0 license according to conf/license.conf can be referenced in recipes as GPL-2, GPLv2, and GPLv2.0. If a license name is used directly, we end up with three files in the rootfs with the same content. If a canonical name used instead, then each license gets copied only once. Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24python3targetconfig.bbclass: Make py3 dep and tasks only for target recipesKhem Raj
python3targetconfig append target python3 to dependencies unconditionally, and here its inherited unconditionally too but distutils3-base is inherited in BBCLASSEXTEND'ed recipes and other not-target recipes as well. Hence the change added via 9c8f666097802cb594a759989edcf01603a22df3 is now bridging the native dependencies with target python3 and thats resulting all sorts of rebuilds for multimachine builds e.g. MACHINE=qemuarm bitbake python3-scons-native MACHINE=qemumips bitbake python3-scons-native results in rebuilds for python3-scons-native bitbake-diffsigs shows Hash for dependent task python/python3-scons-native_3.1.2.bb:do_populate_sysroot changed from 1cdb93193b416477df6faa137e83a967b433c7aa29033146b405153f73f36933 to 3cea1e7cbedd121ecb768fbc291cc4e4d7d3b5c0442897 0e3b97bd058d162065 Hash for dependent task python/python3-scons-native_3.1.2.bb:do_install changed from 8d6018fd03ffc6060a04532dc39a5b7ccca1be026a69d069cb4fb11aef86dd89 to c5f1d173596a8e910f45a2b6e0b4dab96cd0102be4d62bd3156 229cb0f5ebb11 Hash for dependent task python/python3-scons-native_3.1.2.bb:do_compile changed from e3ee4b52a15267e6ae7853ec19a666b2fb62608a597608793336382d1c45f8a0 to 1e582043dfe6b3e00aaa532f363ce6afb37652abe837dac 7cc9769194c43eae1 Hash for dependent task python/python3-scons-native_3.1.2.bb:do_configure changed from 770a4d5a77a96ebd9e1e7368f710bca3f88e3b1266dffa3b2d0360b1e3a81e27 to a366982778b03eee5165c3117ee778f848acdfaa2 b346650fbdf114ac70ab57b Hash for dependent task python/python3-scons-native_3.1.2.bb:do_prepare_recipe_sysroot changed from 958910037856ff5d5eb2b5162b3cdd02a3a710fc543b933cfeba771ee095cb72 to 474333fb565f908992fd3716 4935aaecf31a79e867826fe634cde4f44171d8e7 Hash for dependent task python/python3_3.9.0.bb:do_populate_sysroot changed from 7ac1c4fcbb2eacf98d2c32d991751bd2f3c7d55e2e32f2c9e485e7f5975fecf8 to 25dcfe74a95af19cce8df7c29311cc5edbbf6ad 08777e46a6fa6e417c0445018 ... Therefore limit effects of this class only for target recipes. Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Alexander Kanavin <alex.kanavin@gmail.com> Cc: Martin Jansa <Martin.Jansa@gmail.com> Cc: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-18kernel: relocate copy of module.lds to module compilation taskBruce Ashfield
There were two copies of this patch floating around, and the merged variant has the copy in the wrong place. module.lds is only created during modules_prepare, and that target is not invoked during our main build of the kernel. We aren't about to change the kernel build (there's no need), so we move the copy into the compile_kernelmodules task. After that runs, we have module.lds availble to copy. This has been tested against clean kernel + out of tree module builds, and the dependencies are correct that the file is copied before the out of tree module build starts. Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-17roofs_*.bbclass: fix missing vardeps for do_rootfsLoic Domaigne
As per lib/oe/rootfs.py and lib/oe/package_manager/???/__init__.py the PACKAGE_FEED baseurl is defined as the joined paths of: URIS/BASE_PATHS/ARCHS Therefore, the do_rootfs task should depend furthermore on PACKAGE_FEED_{BASE_PATHS,ARCHS} to properly retrigger a build if the value changes. Signed-off-by: Loic Domaigne (ljd) <tech@domaigne.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-17kernel: Set proper LD in KERNEL_KCONFIG_COMMANDWonmin Jung
With 'ld-is-gold' and linux kernel 5.4 or later, menuconfig task for kernel recipes will fail with: $ bitbake -c menuconfig virtual/kernel ... scripts/kconfig/mconf Kconfig scripts/Kconfig.include:43: gold linker 'x86_64-poky-linux-ld' not supported /OE/build/tmp/work-shared/qemux86-64/kernel-source/scripts/kconfig/Makefile:29: recipe for target 'menuconfig' failed make[2]: *** [menuconfig] Error 1 /OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:606: recipe for target 'menuconfig' failed make[1]: *** [menuconfig] Error 2 /OE/build/tmp/work-shared/qemux86-64/kernel-source/Makefile:185: recipe for target '__sub-make' failed make: *** [__sub-make] Error 2 Command failed. This is because that the KERNEL_LD variable already set in kernel-arch.bbclass isn't used by do_menuconfig function of cml1.bbclass. To fix this issue specify LD variable while calling the kernel menuconfig command through KERNEL_KCONFIG_COMMAND. Signed-off-by: Wonmin Jung <wonmin82@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16meta: drop _PYTHON_SYSCONFIGDATA_NAME hacksAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16distutils3-base.bbclass: use python3targetconfigAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16python3: split python target configuration into own classAlexander Kanavin
Setting _PYTHON_SYSCONFIGDATA_NAME in python3native class globally was problematic as it was leaking into host python environment, which was causing tracebacks depending on host distro and action (typically anything involving importing sysconfig module). The new class sets the variable only in specific tasks where it is needed, and should be inherited explicitly: - use python3native to run scripts with native python - use python3targetconfig to run scripts with native python if those scripts need to access target config data (such as correct installation directories). This also adds a dependency on target python, so should be used carefully to avoid lengthening builds. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-16kernel: provide module.lds for out of tree builds in v5.10+Bruce Ashfield
The upstream commit 596b0474d3d [kbuild: preprocess module linker script], adds a dependency on module.lds for external module building. Since module.lds is generated as part of 'modules_prepare', we must make it available with the other kernel artifacts in the kernel shared workdir, otherwise out of tree builds fail. This fixes errors like: | make[4]: *** No rule to make target 'scripts/module.lds', needed by 'build/tmp/work/qemuarm64-poky-linux/cryptodev-module/1.11-r0/git/cryptodev.ko'. Stop. | make[4]: *** Waiting for unfinished jobs.... We also ensure that kernel-devsrc has a copy to support on target module builds that are often prepared with 'make scripts prepare'. Those targets won't regenerate it, so the build fails. If 'make modules_prepare' is used, the file will be regenerated and overwrite our copy (as expected). Signed-off-by: Pan, Kris <kris.pan@intel.com> Signed-off-by: Lili Li <lili.li@intel.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-11testimage: print results for interrupted runsKonrad Weihmann
When a run is ended by overall timeout, print the already executed testcases, to provide some hints which testcase might made the test suite reach global timeout. Nonetheless make the testrun exit with an error Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-11add new extrausers command passwd-expireJoseph Reynolds
This enhances extrausers with a new passwd-expire command that causes a local user's password to be expired as if the `passwd --expire` command was run, so the password needs to be changed on initial login. Example: EXTRA_USERS_PARAMS += " useradd ... USER; passwd-expire USER;" Tested: on useradd accounts When configured with Linux-PAM, console login prompts for and can successfully change the password. OpenSSH server works. Dropbear SSH server notes the password must be changed but does not offer a password change dialog and rejects the login request. Signed-off-by: Joseph Reynolds <joseph-reynolds@charter.net> Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08cml1.bbclass: Handle ncurses-native being available via pkg-configNathan Rossi
The linux kernel will by default use pkg-config to get ncurses(w) paths, falling back to absolute path checks otherwise. If the build host does not have ncurses installed this will fail as pkg-config will not search the native sysroot for ncurses. To more all kernel/kconfig sources, inject the equivalent native pkg-config variables similar to what is done by the pkg-config-native script. This only affects the menuconfig python task itself and the oe_terminal call inside it. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08autotools: CONFIG_SHELL defaultsAndrej Valek
Do not hard-code default shell to /bin/bash even if CONFIG_SHELL is already set to other shell, but keep /bin/bash as a default. This will fix a shadow issue, where CONFIG_SHELL is exported to /bin/sh, but /bin/bash is used even if it's not installed. Signed-off-by: Andrej Kozemcak <andrej.kozemcak@siemens.com> Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08kernel-module-split.bbclass: identify kernel modconf files as configuration ↵Gratian Crisan
files Currently the modconf fragments representing the configuration for kernel modules are written out to appropriate .conf files and added to the FILES variable. However they are not identified as 'configuration files' and installing a new version of a kernel module results in a conflict and a failed installed because the respective .conf file is already in place from a previous install. Add the generated .conf files to the CONFFILES variable denoting their true nature. Signed-off-by: Gratian Crisan <gratian.crisan@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-02insane: add GitLab /archive/ testsAndrey Zhizhikin
Archives produced by GitLab should be avoided in the same way as those produced by GitHub. Extend SRC_URI check to include GitLab and inform user that recipe should be converted to use git protocol. Link: https://www.mail-archive.com/openembedded-devel@lists.openembedded.org/msg73109.html Link: https://lists.openembedded.org/g/openembedded-core/message/144035 Cc: Martin Jansa <martin.jansa@gmail.com> Signed-off-by: Andrey Zhizhikin <andrey.zhizhikin@leica-geosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30python: update 3.8.5 -> 3.9.0Alexander Kanavin
Drop 0001-Do-not-hardcode-lib-as-location-for-site-packages-an.patch, 0001-configure.ac-fix-LIBPL.patch and 0001-python3-Do-not-hardcode-lib-for-distutils.patch as they are all replaced by the new --platlibdir option to ./configure Rename 0001-Lib-sysconfig.py-fix-another-place-where-lib-is-hard.patch to 0001-Lib-sysconfig.py-use-libdir-values-from-configuratio.patch and describe the changes better. License-Update: documentation now dual license under PSF & BSD (not relevant for the recipe. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30qemuboot.bbclass: Fix a typoKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-23Revert "classes/buildhistory: also save recipe info for native recipes"Richard Purdie
This reverts commit d123606c4bef85c2436b40f51e47b602b7600c0b. This change contains races as it will start poking into do_package task directories from do_populate_sysroot. If we want to do this for native recipes, we need to add guards around the package code and only make this happen for native in populate_sysroot, not target in populate_sysroot too. Backtrace from an example problem below: ERROR: openssl-1.1.1g-r0 do_populate_sysroot: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:buildhistory_emit_pkghistory(d) 0003: File: '/home/ross/Yocto/poky/meta/classes/buildhistory.bbclass', lineno: 319, function: buildhistory_emit_pkghistory 0315: 0316: write_pkghistory(pkginfo, d) 0317: 0318: # Create files-in-<package-name>.txt files containing a list of files of each recipe's package *** 0319: bb.build.exec_func("buildhistory_list_pkg_files", d) 0320:} 0321: 0322:python buildhistory_emit_outputsigs() { 0323: if not "task" in (d.getVar('BUILDHISTORY_FEATURES') or "").split(): File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 256, function: exec_func 0252: with bb.utils.fileslocked(lockfiles): 0253: if ispython: 0254: exec_func_python(func, d, runfile, cwd=adir) 0255: else: *** 0256: exec_func_shell(func, d, runfile, cwd=adir) 0257: 0258: try: 0259: curcwd = os.getcwd() 0260: except: File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 503, function: exec_func_shell 0499: with open(fifopath, 'r+b', buffering=0) as fifo: 0500: try: 0501: bb.debug(2, "Executing shell function %s" % func) 0502: with open(os.devnull, 'r+') as stdin, logfile: *** 0503: bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)]) 0504: except bb.process.ExecutionError as exe: 0505: # Find the backtrace that the shell trap generated 0506: backtrace_marker_regex = re.compile(r"WARNING: Backtrace \(BB generated script\)") 0507: stdout_lines = (exe.stdout or "").split("\n") File: '/home/ross/Yocto/poky/bitbake/lib/bb/process.py', lineno: 184, function: run 0180: if not stderr is None: 0181: stderr = stderr.decode("utf-8") 0182: 0183: if pipe.returncode != 0: *** 0184: raise ExecutionError(cmd, pipe.returncode, stdout, stderr) 0185: return stdout, stderr Exception: bb.process.ExecutionError: Execution of '/yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804' failed with exit code 2: /yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804: 183: cd: can't cd to /yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/packages-split/openssl-engines Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20classes/buildhistory: also save recipe info for native recipesPaul Eggleton
If we want to also collect SRC_URI for native recipes we need to ensure that the code that writes out all of the recipe info is called - there isn't a do_packagedata for native recipes so we need to piggyback on do_populate_sysroot instead. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20classes/buildhistory: record SRC_URIPaul Eggleton
It can be useful to record SRC_URI into buildhistory for the purposes of tracking exactly which sources got built (we already have SRCREV) as well as getting an indication when changes to the SRC_URI relate to changes in the output. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17siteinfo: Recognize bigendian sh3be and sh4beMartin Jansa
* seems to be broken for many years, does someone still use sh3/sh4? scripts/tune/log.fake-sh3.sh3eb: Parsing recipes... ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/vala/vala_0.48.9.bb: Unable to determine endianness for architecture 'sh3eb' ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/vala/vala_0.48.9.bb: Please add your architecture to siteinfo.bbclass scripts/tune/log.fake-sh4.sh4aeb scripts/tune/log.fake-sh4.sh4eb: Parsing recipes... ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/vim/vim-tiny_8.2.bb: Unable to determine endianness for architecture 'sh4eb' ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/vim/vim-tiny_8.2.bb: Please add your architecture to siteinfo.bbclass Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17siteinfo: Recognize 32bit PPC LEMartin Jansa
* since this commit, all powerpc little-endians (both 32bit and 64bit) use "le" suffix: commit b6ac40f1cbabb20896bf113568f7735a462ed1a6 Author: Khem Raj <raj.khem@gmail.com> Date: Sun Dec 29 10:44:03 2019 -0800 powerpc, powerpc64: Append little-endianness to tune arch * 64bit variants were already fixed in: commit e62cdb9b88b575b5cfcdd65ca558edc237c43b2a Author: Khem Raj <raj.khem@gmail.com> Date: Sun Dec 29 10:44:02 2019 -0800 siteinfo: Recognize 64bit PPC LE * but 32bit are still failing: scripts/tune/log.fake-power5.powerpcle scripts/tune/log.fake-power5.powerpcle-nf scripts/tune/log.fake-power6.powerpcle scripts/tune/log.fake-power6.powerpcle-nf scripts/tune/log.fake-power7.powerpcle scripts/tune/log.fake-power7.powerpcle-nf scripts/tune/log.fake-power9.powerpcle scripts/tune/log.fake-power9.powerpcle-nf scripts/tune/log.fake-power9.ppcp9le scripts/tune/log.fake-ppc476.powerpcle scripts/tune/log.fake-ppc476.powerpcle-nf scripts/tune/log.fake-ppc603e.powerpcle scripts/tune/log.fake-ppc603e.powerpcle-nf scripts/tune/log.fake-ppc7400.powerpcle scripts/tune/log.fake-ppc7400.powerpcle-nf scripts/tune/log.fake-ppce300c2.powerpcle scripts/tune/log.fake-ppce300c2.powerpcle-nf scripts/tune/log.fake-ppce300c3.powerpcle scripts/tune/log.fake-ppce300c3.powerpcle-nf scripts/tune/log.fake-ppce500.powerpcle scripts/tune/log.fake-ppce500.powerpcle-nf scripts/tune/log.fake-ppce500mc.powerpcle scripts/tune/log.fake-ppce500mc.powerpcle-nf scripts/tune/log.fake-ppce500v2.powerpcle scripts/tune/log.fake-ppce500v2.powerpcle-nf scripts/tune/log.fake-ppce5500.powerpcle scripts/tune/log.fake-ppce5500.powerpcle-nf scripts/tune/log.fake-ppce6500.powerpcle scripts/tune/log.fake-ppce6500.powerpcle-nf Parsing recipes... ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/debianutils/debianutils_4.11.1.bb: Unable to determine endianness for architecture 'powerpcle' ERROR: /OE/build/oe-core/openembedded-core/meta/recipes-support/debianutils/debianutils_4.11.1.bb: Please add your architecture to siteinfo.bbclass ... Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17waf: add ${B} to do_configure[cleandirs]Ross Burton
As waf is always out-of-tree, we can delete ${B} before every build. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17waf: don't assume the waf intepretter is goodRoss Burton
Waf typically uses `python` as the intepretter but inside a task this does not exist. Typically this is solved by patching waf (see the glmark2 recipe) but not all versionf of Waf support Python 3 so we can't assume a specific interpretter. Instead, create a new variable WAF_PYTHON for the correct interpretter, and default this to `python3`. If the user has a recipe that needs Python 2 then this can be changed in the recipe. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17autotools.bbclass: Order CONFIG_SHELL before CACHED_CONFIGUREVARSKhem Raj
This helps in overriding CACHED_CONFIGUREVARS with wrappers to run configure under e.g. for static analysers like scan-build from clang, while it should not change the functionality in normal case. Since CONFIG_SHELL was introduced, it silently broke this use case and failed running static analyser on autotool based recipes Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17classes: Fixed the problem of undefined variables when compiling meta-toolchain.zangrc
The product name when compiling meta-toolchain is: poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.host.manifest poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.target.manifest poky-glibc-x86_64-${IMAGE_BASENAME}-cortexa57-qemuarm64-toolchain-3.1+snapshot.testdata.json poky-glibc-x86_64--cortexa57-qemuarm64-toolchain-3.1+snapshot.sh inherit image-artifact-names to solve this problem. Signed-off-by: Zang Ruochen <zangrc.fnst@cn.fujitsu.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-17externalsrc: No single-task lock if S != BDouglas Royds
Allow different recipes to build things from the one external source tree in parallel, but only if the build is happening outside the source tree. Signed-off-by: Douglas Royds <douglas.royds@taitradio.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13IMAGE_LOCALES_ARCHIVE: add option to prevent locale archive creationMichael Thalmeier
Under some circumstances it is not desirable to create a combined locale archive (/usr/lib/locale/locale-archive). The new variable IMAGE_LOCALES_ARCHIVE defaults to '1', so the default behaviour is not changed. Signed-off-by: Michael Thalmeier <michael.thalmeier@hale.at> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13uninative: Fix typo in error messageNaoki Hayama
Fix typo in an error message. s/verson/version/ Signed-off-by: Naoki Hayama <naoki.hayama@lineo.co.jp> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-13classes/waf: Add build and install argumentsJoshua Watt
Adds variables that can be used to allow a recipe to pass extra arguments to `waf build` and `waf install`. In most cases, you want to pass the same arguments to `build` and `install` (since install is a superset of `build`), so by default setting EXTRA_OEWAF_BUILD also affects `waf install`, but this can be overridded. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-10base.bbclass: warn when there is trailing slash in S or B variablesMartin Jansa
* to make sure we won't hit such corner cases in future add a warning to prevent trailing slashes and duplicated slashes even when they in most cases don't cause harm * only a few cases were found in layers included in my world builds: oe-core: 1 meta-oe: 7 meta-python2: 1 meta-qt5: 1 meta-aws: 1 will send patches for these once this warning is approved for oe-core Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-10base.bbclass: use os.path.normpath instead of just comparing WORKDIR and S ↵Martin Jansa
as strings * cannot use os.path.samefile, because S/B might not exist at this time yet * there is issue with PSEUDO_IGNORE_PATHS when some recipe sets e.g. S = "${WORKDIR}/" whole WORKDIR gets added to PSEUDO_IGNORE_PATHS and then the build can fail with various strange errors, in my case do_package was failing when do_package calls: fix_perms(.../1.0-r0/package/etc, 755, 0, 0, /etc) and fails with "[Errno 1] Operation not permitted:" Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-09abi_version,sanity: Tell users TMPDIR must be clean after pseudo changesRichard Purdie
After the recent path ignore changes, there are invalid entries in exiting pseudo databases which will trip up users with the new abort() (proving how common the pseudo potential corrpution is!). Inform them a clean TMPDIR is needed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-08devtool: fix modify with patches in override directoriesRoss Burton
If a recipe applies patches which are in machine-specific override directories, devtool will fail to fetch the patches that don't match the default configuration. For example where there are patches at qemux86/x86.patch and qemuarm/arm.patch: SRC_URI = "file://source" SRC_URI_append_qemuarm = " file://arm.patch" SRC_URI_append_qemux86 = " file://x86.patch" The patch apply phase sets OVERRIDES but does not set FILESOVERRIDES, so it cannot find the patch files as the search path isn't correct. Fix this by setting FILESOVERRIDES too. Also when iterating through the overrides we need to be sure that other overrides that are used are not enabled, so extend no_overrides instead of simply appending the current override. Fixes most but not all of [ YOCTO #14060 ]. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07wic: Handle new PSEUDO_IGNORE_PATHS variableRichard Purdie
Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid inode corruption issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07base/bitbake.conf: Enable pseudo path filteringRichard Purdie
This is a pretty big change to the way pseudo operates when used in OpenEmbedded. Normally, pseudo monitors and logs (adds to its database) any file created or modified whilst in a fakeroot environment. There are large numbers of files we simply don't care about the permissions of whilst in that fakeroot context, for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control directories, This change uses new functionality in pseudo to ignore these directory trees, resulting in a cleaner database with less chance of "stray" mismatches if files are modified outside pseudo context. It also should reduce some overhead from pseudo as the interprocess round trip to the server is avoided. There is a possible complication where some existing recipe may break, for example, we found a recipe which was writing to "${B}/install" for "make install" in do_install and since we listed ${B} as not to be tracked, there were errors trying to chown root for files in this location. This patch fixes a few corner cases in OE-Core when used with this new ignore list: * The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed to something else since that directory does need its root permissions * The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR} * package_write_* task output (the debs/rpms/ipks) are now owned by the build user so we don't want the file ownership information in the hashequiv outhash calculation even if they are built under pseudo. * The fontcache postinstall intercept is run under qemu outside of pseudo context so delete files it may delete up front where pseudo can see this. * SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached by bitbake. We therefore need to trigger reparsing if this changes, which means SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST. Rework the variables to handle this. This otherwise breaks some of our sstate tests in oe-selftest. * Ignore the temp directory wic uses for rebuilding rootfs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07classes/sanity: Bump minimum python version to 3.5Joshua Watt
Bumps the minimum python version to 3.5 to match bitbake and the test matrix Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-06kernel-fitimage: generate openssl RSA keys for signing fitimageUsama Arif
The keys are only generated if they dont exist. The key generation can be turned off by setting FIT_GENERATE_KEYS to "0". The default key length for private keys is 2048 and the default format for public key certificate is x.509. Signed-off-by: Usama Arif <usama.arif@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30populate_sdk_ext.bbclass: add ESDK_MANIFEST_EXCLUDESChen Qi
Add ESDK_MANIFEST_EXCLUDES to enable excluding items in sdk-conf-manifest. By default, files under conf/ are all added to sdk-conf-manifest, as the manifest file is set to 'conf/*'. However, there are situations where some configuration files under conf/ directory are not intended to be added to sdk-conf-manifest, thus adding ESDK_MANIFEST_EXCLUDES to enable users to do this. This variable takes the form of glob matching. e.g. ESDK_MANIFEST_EXCLUDES = "conf/autogen*" This would exclude all files under conf/ starting with 'autogen' from sdk-conf-manifest. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30cve-check: add CVE_CHECK_REPORT_PATCHED variable to suppress reporting of ↵Chris Laplante
patched CVEs Default behavior is not changed. To suppress patched CVEs, set: CVE_CHECK_REPORT_PATCHED = "" Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30cve-check: introduce CVE_CHECK_RECIPE_FILE variable to allow changing of ↵Chris Laplante
per-recipe check file The addition of this variable also makes it possible to change the output suffix of the check files, e.g. in local.conf: CVE_CHECK_MANIFEST_append = ".txt" CVE_CHECK_RECIPE_FILE_append = ".txt" Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30testimage: Add testimage_dump_target to kwargsSaul Wold
This passes the list of commands to run on the OEQemuTarget when the TargetDumper needs to run in a test context due to a failure on the target. This is added here as a kwargs because the 'd' dictionary is not available in the staticmethod getTarget in the OERuntimeTestContextExecutor class. The OEQemuTarget is different from the QemuTarget which already uses the list of commands from testimage_dump_target from 'd'. The create_dir() is needed to initialize the TargetDumper's dump_dir variable. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30Space-comma CleanupsJon Mason
Multiple files have " ," instead of ", " in expressions. This changes them to conform to the way the rest of them are done. Found and corrected via: git ls-files | xargs sed --follow-symlinks -i 's/ ,d/, d/g' Signed-off-by: Jon Mason <jon.mason@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30sstate: set mode explicitly when creating directories in sstate-cacheRoss Burton
When creating directories in the sstate-cache, explicitly set the mode passed to mkdir to 0775 so that the directories are group writable, as otherwise they cannot be shared with other users. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30staging: Ensure cleaned dependencies are addedRichard Purdie
Most recipe-sysroot dependencies are handled by these prefuncs. "configure" is special with a decidated task, prepare_recipe_sysroot which runs beforehand. do_prepare_recipe_sysroot does not have to be run before/after fetch/unpack/patch, they're independent tasks. If fetch/unpack/patch have sysroot dependencies and those tasks rerun, stale items from the sysroot could be uninstalled and since prepare_recipe_sysroot doesn't re-run, they could be missing by the time configure runs. Fix this by adding the prefunc for do_configure which would ensure the sysroot is in a good state. This issue can be reproduced by modifying do_unpack and watching binutils-native and vulkan-headers failing due to missing makeinfo or ninja. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>