| Age | Commit message (Collapse) | Author |
|---|
|
Move classes to classes-global or classes-recipe as appropriate to take
advantage of new bitbake functionality to check class scope/usage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As stated in our top level license files, the license is MIT unless
otherwise stated. Add SPDX identifers accordingly. Replace older
license statementa with the standardised syntax. Also drop "All
Rights Reserved" expression as it isn't used now, doesn't mean anything
and is confusing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* this doesn't fix any issue (at least AFAIK),
just keeps the log files more deterministic to avoid unnecessary churn like in:
--- /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/temp/log.do_patch.2213051 2022-05-26 11:35:44.110063749 +0200
+++ /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/temp/log.do_patch.2213372 2022-05-26 11:35:54.553062630 +0200
@@ -1,7 +1,7 @@
DEBUG: Executing python function extend_recipe_sysroot
-NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot']
+NOTE: Direct dependencies are ['/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot']
NOTE: Installed into sysroot: []
-NOTE: Skipping as already exists in sysroot: ['patch-native', 'quilt-native', 'attr-native', 'libtool-native', 'gettext-minimal-native', 'texinfo-dummy-native']
+NOTE: Skipping as already exists in sysroot: ['quilt-native', 'patch-native', 'attr-native', 'libtool-native', 'gettext-minimal-native', 'texinfo-dummy-native']
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_patch
DEBUG: Executing python function patch_do_patch
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We've seen failures on the autobuilder in oe-selftest where things which should
be in the sysroot aren't. The exact steps to reproduce the exact failure are
elusive and probably hash equivalance dependency but this set of steps does
reproduce corruption which is probably of the same origin:
Add DISTRO_FEATURES += "systemd"
bitbake dbus
Remove DISTRO_FEATURES += "systemd"
bitbake dbus
bitbake dbus -c clean
bitbake dbus -c configure
Add DISTRO_FEATURES += "systemd"
bitbake quilt-native
bitbake dbus -c populate_sysroot
Remove DISTRO_FEATURES += "systemd"
bitbake dbus -c compile
Where dbus will now fail as the compiler was no longer in the sysroot.
This works by clearing x11 and other values out of DISTRO_FEATURES so the x11
dependencies are removed from the sysroot. The configure stamp remains valid so
when the original configuration is restored, it becomes valid again but a load
of the sysroot disappeared and build failures result.
Fix this by removing stamps when we remove things from the sysroot.
Depends on a change to bitbake build.py to add the clean_stamp API.
[YOCTO #14790]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adding a dependency on ourselves in this function doesn't make sense, the hash
may change after hash equivalence is applied. Other code using BB_TASKDEPDATA does
handle the self reference correctly (which is there for a reason), update this
code to do likewise.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This matters when cross-compiling a cross-toolchain:
we need to specify the system where the toolchain
will run, not the system it will produce output for.
For everything else, HOST and TARGET are the same.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After other variables were renamed in bitbake, update OE-Core to match.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A regression form cpio CVE-2021-38185 caused the tool to hang for paths
greater than 128 character long. It was reported here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=992098
We were able to reliable reproduce this with dunfell, meta-freescale
recipe imx-boot
https://github.com/Freescale/meta-freescale/blob/dunfell/recipes-bsp/imx-mkimage/imx-boot_1.0.bb
Using relative path on the affected host fixes the issue as this is
always short, being in the same work dir. It would be harmless, and
useful to generally use the relative path for sysroot_stage_dir()
Signed-off-by: Ahsan Hussain <ahsan_hussain@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Target build output, stored in do_populate_sysroot or do_package can depend
not only upon direct dependencies but also indirect ones. A good example is
linux-libc-headers. The toolchain depends on this but most target recipes do
not. There are some headers which are not used by the toolchain build and do
not change the toolchain task output, hence the task hashes can change without
changing the sysroot output of that recipe yet they can influence others.
A specific example is rtc.h which can change rtcwake.c in util-linux but is not
used in the glibc or gcc build. To account for this, we need to account for the
populate_sysroot hashes in the task output hashes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When rebuilds are triggered, autoconf-native can fail with:
| DEBUG: Executing shell function update_gnu_config
| install: cannot stat '[BUILDPATH]tmp/work/x86_64-linux/autoconf-native/2.71-r0/recipe-sysroot-native/usr/share/gnu-config/config.guess': No such file or directory
which is due to update_gnu_config running before extend_recipe_sysroot.
This only happens rarely since usually the prepare_recipe_sysroot
function would already have set things up and only in the invalidated
task hash cases does this rebuild in this way from configure only.
Fix the code to prepend this function instead of appending which
resolves the ordering issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Deploy is a tricky task as it isn't in SSTATETASKS as it isn't always
present. We do need to ensure dependency chains around do_deploy tasks
are correctly handled as sstate tasks though. For now add to the list of
sstate tasks manually (like the other locale task reference).
Without this, missing manifest files could be reported now that do_deploy
tasks no longer have their dependencies added by the depvalid function.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The sstate code tries to be careful about racing around directory creation.
In particular, the copyhardlinktree code creates the directory tree first
allowing for "already exists" errors and ignoring them, then hardlinks the
files in.
Unfortunately the sstate removal code can race against this since it
will try and remove empty directories. If there is some bad timing,
a newly created directory can be removed before it was populated, leading
to build failures.
We could try and add locking but this would damage performance, we've been
there before. It is also unclear where to actually place locks just based on
the contents of a manifest file which may cover multiple sstate install
locations for a given task.
Instead, lets disable directory removal in the problematic "shared" core
path. This could result in a few more empty directories being left on disk
but those should be harmless and better than locking hurting performance
or rare build races.
[YOCTO #13999]
[YOCTO #14379]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that 022 is the default in BB_DEFAULT_UMASK in bitbake.conf, we
don't need any of these task flags, clean up.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are a variety of files being installed into $datadir which we
don't need. Pick the top "offenders" which amount of thousands of files
and simply don't install them. These include things like test data,
terminfo data, locale data for native tools and so on. This saves
copying these files into native and target sysroots and should improve
performance (smaller sstate, fewer files to copy around).
With this and the python recipe change, alsa-tools went from:
recipe-sysroot: 18357
recipe-sysroot-native: 14129
to
recipe-sysroot: 10809
recipe-sysroot-native: 8079
which is a decent improvement.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The '/sysroot-only' in SYSROOT_DIRS is to be used by recipes which
generate artifacts that are not included in the target filesystem.
Also, remove the ${D}/sysroot-only dir before copying D do PKGD to
generate the packages since it is not supposed to be included in
any package.
This will allow recipes to share non-target filesystem artifacts
without needing to use the DEPLOY_DIR and keep it tidy.
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Most recipe-sysroot dependencies are handled by these prefuncs. "configure"
is special with a decidated task, prepare_recipe_sysroot which runs
beforehand.
do_prepare_recipe_sysroot does not have to be run before/after
fetch/unpack/patch, they're independent tasks. If fetch/unpack/patch
have sysroot dependencies and those tasks rerun, stale items from the
sysroot could be uninstalled and since prepare_recipe_sysroot doesn't
re-run, they could be missing by the time configure runs.
Fix this by adding the prefunc for do_configure which would ensure
the sysroot is in a good state.
This issue can be reproduced by modifying do_unpack and watching
binutils-native and vulkan-headers failing due to missing makeinfo
or ninja.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A "broken" buildtools-extended-tarball has been released into the wild
where it is optimising binaries for the host processsor. This is fine in
local usage but in a non-homogeneous cluster like our autobuilder, this
results in SIGILL on other machines when the sstate is shared amongst them
and is painful to debug.
The buildtools tarball has been fixed but we need to invalidate the hash
equivalence and sstate data. This change does that. Adding to OE-Core
rather than autobuilder local changes as its good to illustrate how to
do this and the issue is potentially wider than just Yocto Project
infrastructure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there are different providers of a file and they are swiched when the
recipe isn't machine specific, we can get tracebacks due to the overlapping
files. The issue is that the previous provider isn't uninstalled since
the system can't tell whether some later task needs them.
By tracking which tasks we depend upon, the code can now choose to
uninstall more things since a later task can reinstall if/as needed.
The code here was to protect against code with two different tasks
running in parallel which is still protected agaisnt.
[YOCTO #13702]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until it has been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.
With the recent patch changes we've seen issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)
which is a symptom of this issue (libattr-native is a dependency of patch-native).
There are other ways to fix this such as disabling libattr in patch, installing
patch to a subdirectory and requiring PATH manipulation and so on.
We can simply fix the staging code to handle /bin/ after everything else so
do that and avoid all these other complications.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
They were not excluded when multilib variants were being built.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Do not reset libdir and base_libdir in sysroot_strip, and just pass crude
paths as they will be reset later in strip_execs.
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if files move between recipes, do_prepare_sysroot can fail with a message like
Exception: FileExistsError: [Errno 17] File exists:
'TMPDIR/sysroots-components/core2-64/libx11/usr/include/X11/extensions/XKBgeom.h' ->
'TMPDIR/work/core2-64-poky-linux/gtk+3/3.24.8-r0/recipe-sysroot/usr/include/X11/extensions/XKBgeom.h'
This is because files are removed and then added per package. What needs to
happen is all removes need to be processed, then all additions.
This patch changes the code to process in two phases, removals first, then additions,
which avoids the problem.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove remnants of the glibc-initial recipe.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With recent changes to runqueue, this fuction is unsafe as setscene tasks can run
at the same time as normal ones and doing things before do_fetch no longer
offers any guarantees.
There is other code which cleans out things from the sysroots as tasks rerun so
we should rely upon that instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
multiconfig dependencies no longer appear in BB_TASKDEPDATA so we can drop
this code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When mixing aarch64 and x86_64 host builds with common sstate, useradd failures
were found. This was due to references to:
/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/sysroots-components/aarch64/pseudo-native/usr/bin/pseudo
or
/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo
on the wrong architecture within the useradd postinst files.
Instead of using COMPONENTS_DIR as the FIXME, use PSEUDO_SYSROOT which contains
the arch component as well to avoid this.
Add extra comments to cause the relavent sstate to be rebuilt.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When api-documentation is enabled the GNOME stack builds API documentation. As
${datadir} is in SYSROOT_DIRS this documentation is in the sysroot but is never
used, wasting time and space.
Add ${datadir}/gtk-doc/html to the blacklist so that the generated documentation
isn't in the sysroot. Note that we don't blacklist all of ${datadir}/gtk-doc
because gtk-doc itself installs files there which are needed to use gtk-doc.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move prepare_recipe_sysroot's task dependency on populate_sysroot from
base.bbclass (where it was specified in the middle of do_configure's
definition) to staging.bbclass (where the rest of
do_prepare_recipe_sysroot is defined). This was a left-over from when
recipe specific sysroots were introduced in commit 809746f5 and the
task dependency on populate_sysroot was moved from do_configure to
do_prepare_recipe_sysroot.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bitbake gdk-pixbuf gdk-pixbuf-native
bitbake build-sysroots -c build_target_sysroot
can lead to tracebacks as gdk-pixbuf-native is being installed into the
target sysroot. The issue is that the x86_64 (common BUILD_ARCH) sysroot
components directory can contain a mix of native and target artefacts.
Differentiate by the "-native" in the recipe names. Should also trim
down the size of the sysroot used in eSDK.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The code currently only prints a single depchain due to overwriting data
instead of appending. Fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Occasionally we see warnings like:
WARNING: core-image-sato-1.0-r0 do_populate_sdk: Manifest /home/pokybuild/yocto-worker/nightly-qa-extras/build/build/tmp/sstate-control/manifest-allarch-nativesdk-linux-libc-headers.populate_sysroot not found in i686_linux allarch (variant '')?
which occur when do_populate_sdk is run in an otherwise empty TMPDIR.
It occurs because do_stash_locale is not recognised as a setscene task
and is removed from the taskgraph meaning the dependency chains fed
through setscene_depvalid don't match what was actually setscene'd.
That task is recipe specific and not in the global SSTATETASKS so we
hardcode the value for now to stop the build warnings. This is going to
need to be revisited for a more generic solution.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:
task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"
For the sake of simplicity consider the following example:
Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.
Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"
Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.
This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.
[YOCTO #10681]
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows the calls to is_elf (which calls file) to happen in parallel
allowing a speedup of do_package and do_populate_sysroot for native
recipes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without this, recipes can't find allarch data files like autoconf-archive.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the fixes to other parts of multilib, it was found that the fixup code's
assumptions about the recipe sysroot were incorrect. We need to use the value
calculated earlier in the function.
It turns out there is a rather neat way to do this which cleans up the code
as an added bonus.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if you enable multilib, then build an image, the multilib
recipe sysroot is build in the wrong WORKDIR. If you then clean and
rebuild the image you see "file exists" errors.
This patch ensures the real WORKDIR is used consistently and then
cleans/rebuilds also work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a common function for locating task manifest files rather than
several implementations with missing pieces.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is very useful for debugging. The similar to testsdk.bbclass.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After 02457ef7f600ce954874e2d11e74b1c6daaa3bfc, PSEUDO for
postinst-useradd-* scripts get to use only one PSEUDO_LOCALSTATEDIR
which is set under recipes ${WORKDIR}.
When the those scripts are run in a clean build environment that
is built from the sstate (populate_sysroot_setscene run for
postinst-useradd-* providers), pseudo fails to run because it cannot
access the PSEUDO_LOCALSTATEDIR (recipe ${WORKDIR}s do not exist).
This triggers a sysroot staging error.
Previously, the PSEUDO_LOCALSTATEDIR setting in useradd.bbclass
worked because the RSS sstate/staging logic automagically processed
${STAGING_DIR_TARGET} in postinst-useradd-* scripts to point under
the sysroot being built.
The fix uses the same fixme processing by adding PSEUDO_LOCALSTATEDIR
variable to it. Furthermore, LOGFIFO is added to be able to use
the logging fifo of the recipe that actually runs postinst-useradd-*.
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Search made with the following regex: getVar ?\((.*), True\).
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The staging changes were very verbose in their logging and whilst this is useful
when staging issues occur, those thankfully seem rare now and we can tune down
the logging to a sane level. This improves the readability of error messages from
functions that fail.
The code is still verbose when its replacing things in the sysroot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building the per-recipe sysroot keep track of what files we're installing
and where they came from, so we can detect when a file is installed by two
different recipes and tell the user what these recipes are (instead of just
showing a os.link() stack trace).
[ YOCTO #11631 ]
(From OE-Core rev: 606a8e0ca218f023e362c3678122d36d537f95de)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a logic error in the dependency cleanup code which meant
it would remove dependencies which other tasks still depended upon.
Fix the path names so the comparisions work as intended.
This fixes dependencies accidentally disappearing from sysroots
under certain reconfiguration situations.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently items are added to the sysroot, the obsolete items are removed. If
a change such as pkgconfig -> pkgconf is made, this leads to conflicts of
overlapping files in the sysroot.
In order to better support this, handle removing items before adding them.
This requires some minor refactoring to construct the installed list
before the main function loop, otherwise there are no changes in this
patch other than reordering the operations.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently a task could remove a dependency needed by another task leading
to build failures, often due to missing dependencies (e.g. dynamic libraries
not being found). This was often seen for all-arch recipes in package_write_rpm.
When removing a dependency, first check that no other task active for the
recipe has that same dependency.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes error "Exception: NameError: name 'errno' is not defined"
during build-sysroots.bb:do_build_target_sysroot
Signed-off-by: Marko, Peter <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These messages were added for debugging during the recipe specific
sysroot work. They may still be useful but they don't need to be notes -
if they are they show up in recipetool / devtool output when fetching
source.
(From OE-Core rev: a0e93d5c5dcf59d1898a3db727a5ab2d75e3d20e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Martin Jansa
Alexander Kanavin
Ahsan Hussain
Diego Sueiro
Junling Zheng
Nathan Rossi
Ross Burton
Peter Kjellerstedt
Alejandro Enedino Hernandez Samaniego
Robert Yang
Mikko Ylinen
Ming Liu
Marko, Peter
Paul Eggleton