| Age | Commit message (Collapse) | Author |
|---|
|
A "broken" buildtools-extended-tarball has been released into the wild
where it is optimising binaries for the host processsor. This is fine in
local usage but in a non-homogeneous cluster like our autobuilder, this
results in SIGILL on other machines when the sstate is shared amongst them
and is painful to debug.
The buildtools tarball has been fixed but we need to invalidate the hash
equivalence and sstate data. This change does that. Adding to OE-Core
rather than autobuilder local changes as its good to illustrate how to
do this and the issue is potentially wider than just Yocto Project
infrastructure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there are different providers of a file and they are swiched when the
recipe isn't machine specific, we can get tracebacks due to the overlapping
files. The issue is that the previous provider isn't uninstalled since
the system can't tell whether some later task needs them.
By tracking which tasks we depend upon, the code can now choose to
uninstall more things since a later task can reinstall if/as needed.
The code here was to protect against code with two different tasks
running in parallel which is still protected agaisnt.
[YOCTO #13702]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a long standing issue where a binary could be installed into the
sysroot before its library dependencies. We've always argued nothing should
use the binary until it has been installed by a dependency but there are issues
around binaries which conflict with the host system, for example patch,
python3, gzip and more.
With the recent patch changes we've seen issues like:
ERROR: gdb-cross-canadian-powerpc-8.3.1-r0 do_patch: Command Error: 'quilt --quiltrc /home/pokybuild/yocto-worker/qemuppc/build/build/tmp/work/x86_64-nativesdk-pokysdk-linux/gdb-cross-canadian-powerpc/8.3.1-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
Applying patch 0009-Change-order-of-CFLAGS.patch
patch: /lib64/libattr.so.1: version `ATTR_1.3' not found (required by patch)
Patch 0009-Change-order-of-CFLAGS.patch does not apply (enforce with -f)
which is a symptom of this issue (libattr-native is a dependency of patch-native).
There are other ways to fix this such as disabling libattr in patch, installing
patch to a subdirectory and requiring PATH manipulation and so on.
We can simply fix the staging code to handle /bin/ after everything else so
do that and avoid all these other complications.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
They were not excluded when multilib variants were being built.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Do not reset libdir and base_libdir in sysroot_strip, and just pass crude
paths as they will be reset later in strip_execs.
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if files move between recipes, do_prepare_sysroot can fail with a message like
Exception: FileExistsError: [Errno 17] File exists:
'TMPDIR/sysroots-components/core2-64/libx11/usr/include/X11/extensions/XKBgeom.h' ->
'TMPDIR/work/core2-64-poky-linux/gtk+3/3.24.8-r0/recipe-sysroot/usr/include/X11/extensions/XKBgeom.h'
This is because files are removed and then added per package. What needs to
happen is all removes need to be processed, then all additions.
This patch changes the code to process in two phases, removals first, then additions,
which avoids the problem.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove remnants of the glibc-initial recipe.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With recent changes to runqueue, this fuction is unsafe as setscene tasks can run
at the same time as normal ones and doing things before do_fetch no longer
offers any guarantees.
There is other code which cleans out things from the sysroots as tasks rerun so
we should rely upon that instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
multiconfig dependencies no longer appear in BB_TASKDEPDATA so we can drop
this code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When mixing aarch64 and x86_64 host builds with common sstate, useradd failures
were found. This was due to references to:
/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/sysroots-components/aarch64/pseudo-native/usr/bin/pseudo
or
/home/pokybuild/yocto-worker/qemuarm/build/build/tmp/sysroots-components/x86_64/pseudo-native/usr/bin/pseudo
on the wrong architecture within the useradd postinst files.
Instead of using COMPONENTS_DIR as the FIXME, use PSEUDO_SYSROOT which contains
the arch component as well to avoid this.
Add extra comments to cause the relavent sstate to be rebuilt.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When api-documentation is enabled the GNOME stack builds API documentation. As
${datadir} is in SYSROOT_DIRS this documentation is in the sysroot but is never
used, wasting time and space.
Add ${datadir}/gtk-doc/html to the blacklist so that the generated documentation
isn't in the sysroot. Note that we don't blacklist all of ${datadir}/gtk-doc
because gtk-doc itself installs files there which are needed to use gtk-doc.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move prepare_recipe_sysroot's task dependency on populate_sysroot from
base.bbclass (where it was specified in the middle of do_configure's
definition) to staging.bbclass (where the rest of
do_prepare_recipe_sysroot is defined). This was a left-over from when
recipe specific sysroots were introduced in commit 809746f5 and the
task dependency on populate_sysroot was moved from do_configure to
do_prepare_recipe_sysroot.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bitbake gdk-pixbuf gdk-pixbuf-native
bitbake build-sysroots -c build_target_sysroot
can lead to tracebacks as gdk-pixbuf-native is being installed into the
target sysroot. The issue is that the x86_64 (common BUILD_ARCH) sysroot
components directory can contain a mix of native and target artefacts.
Differentiate by the "-native" in the recipe names. Should also trim
down the size of the sysroot used in eSDK.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The code currently only prints a single depchain due to overwriting data
instead of appending. Fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Occasionally we see warnings like:
WARNING: core-image-sato-1.0-r0 do_populate_sdk: Manifest /home/pokybuild/yocto-worker/nightly-qa-extras/build/build/tmp/sstate-control/manifest-allarch-nativesdk-linux-libc-headers.populate_sysroot not found in i686_linux allarch (variant '')?
which occur when do_populate_sdk is run in an otherwise empty TMPDIR.
It occurs because do_stash_locale is not recognised as a setscene task
and is removed from the taskgraph meaning the dependency chains fed
through setscene_depvalid don't match what was actually setscene'd.
That task is recipe specific and not in the global SSTATETASKS so we
hardcode the value for now to stop the build warnings. This is going to
need to be revisited for a more generic solution.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:
task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"
For the sake of simplicity consider the following example:
Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.
Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"
Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.
This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.
[YOCTO #10681]
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows the calls to is_elf (which calls file) to happen in parallel
allowing a speedup of do_package and do_populate_sysroot for native
recipes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without this, recipes can't find allarch data files like autoconf-archive.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the fixes to other parts of multilib, it was found that the fixup code's
assumptions about the recipe sysroot were incorrect. We need to use the value
calculated earlier in the function.
It turns out there is a rather neat way to do this which cleans up the code
as an added bonus.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if you enable multilib, then build an image, the multilib
recipe sysroot is build in the wrong WORKDIR. If you then clean and
rebuild the image you see "file exists" errors.
This patch ensures the real WORKDIR is used consistently and then
cleans/rebuilds also work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a common function for locating task manifest files rather than
several implementations with missing pieces.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is very useful for debugging. The similar to testsdk.bbclass.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After 02457ef7f600ce954874e2d11e74b1c6daaa3bfc, PSEUDO for
postinst-useradd-* scripts get to use only one PSEUDO_LOCALSTATEDIR
which is set under recipes ${WORKDIR}.
When the those scripts are run in a clean build environment that
is built from the sstate (populate_sysroot_setscene run for
postinst-useradd-* providers), pseudo fails to run because it cannot
access the PSEUDO_LOCALSTATEDIR (recipe ${WORKDIR}s do not exist).
This triggers a sysroot staging error.
Previously, the PSEUDO_LOCALSTATEDIR setting in useradd.bbclass
worked because the RSS sstate/staging logic automagically processed
${STAGING_DIR_TARGET} in postinst-useradd-* scripts to point under
the sysroot being built.
The fix uses the same fixme processing by adding PSEUDO_LOCALSTATEDIR
variable to it. Furthermore, LOGFIFO is added to be able to use
the logging fifo of the recipe that actually runs postinst-useradd-*.
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Search made with the following regex: getVar ?\((.*), True\).
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The staging changes were very verbose in their logging and whilst this is useful
when staging issues occur, those thankfully seem rare now and we can tune down
the logging to a sane level. This improves the readability of error messages from
functions that fail.
The code is still verbose when its replacing things in the sysroot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building the per-recipe sysroot keep track of what files we're installing
and where they came from, so we can detect when a file is installed by two
different recipes and tell the user what these recipes are (instead of just
showing a os.link() stack trace).
[ YOCTO #11631 ]
(From OE-Core rev: 606a8e0ca218f023e362c3678122d36d537f95de)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a logic error in the dependency cleanup code which meant
it would remove dependencies which other tasks still depended upon.
Fix the path names so the comparisions work as intended.
This fixes dependencies accidentally disappearing from sysroots
under certain reconfiguration situations.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently items are added to the sysroot, the obsolete items are removed. If
a change such as pkgconfig -> pkgconf is made, this leads to conflicts of
overlapping files in the sysroot.
In order to better support this, handle removing items before adding them.
This requires some minor refactoring to construct the installed list
before the main function loop, otherwise there are no changes in this
patch other than reordering the operations.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently a task could remove a dependency needed by another task leading
to build failures, often due to missing dependencies (e.g. dynamic libraries
not being found). This was often seen for all-arch recipes in package_write_rpm.
When removing a dependency, first check that no other task active for the
recipe has that same dependency.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes error "Exception: NameError: name 'errno' is not defined"
during build-sysroots.bb:do_build_target_sysroot
Signed-off-by: Marko, Peter <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These messages were added for debugging during the recipe specific
sysroot work. They may still be useful but they don't need to be notes -
if they are they show up in recipetool / devtool output when fetching
source.
(From OE-Core rev: a0e93d5c5dcf59d1898a3db727a5ab2d75e3d20e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The commands:
bitbake nodejs-native;
bitbake nodejs-native -c clean;
bitbake nodejs-native;
bitbake -c addto_recipe_sysroot nodejs-native;
bitbake -c devshell nodejs-native;
bitbake -c addto_recipe_sysroot nodejs-native;
bitbake -c devshell nodejs-native;
never result in npm in the sysroot within devshell. The reason is the
addto_recipe_sysroot stamp isn't removed when do_fetch is run but the sysroot
is cleaned.
With this patch, the second devshell will contain npm, which I think is probably
the best outcome we can hope for here.
[YOCTO #11461]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make use of the library function oe.package.strip_execs for stripping
sysroot executables. oe.packge.strip_execs is based on code previously
residing in sysroot_strip.
Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The function is "sysroot_strip" instead of "split_and_strip_files".
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the limited dependency case we don't use any of the data from
BB_TASKDEPDATA. Restructure the code so this variable doesn't have
to be set. This allows the function to be called from other contexts
without creating artificial constructs. There should be no functional
change, behaviour remains unchanged.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The export of PSEUDO in useradd_sysroot() contains references to
${COMPONENTS_DIR}. These need to be handled when restoring
postinst-useradd-${PN} from the sstate cache.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The path to where to install and find the sysroot components is used
in many places. This warrants it to get its own variable.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Paths to host tools that have been copied to ${HOSTTOOLS_DIR} may end
up in the sstate cache. They thus need to be corrected when restoring
from the sstate cache.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Dependencies on svn:// urls were failing as the cleandirs on do_fetch was destroying
any sysroot setup by the extend_recipe_sysroot function.
Add code so that if the task do_fetch, we move the cleandirs to a separate function before
the extend_recipe_sysroot prefunc else we'd wipe out the sysroot we just created.
This allows fetcher do_populate_sysroot dependencies to work correctly again.
I did try various other approaches and a seperate function with cleandirs was the
cleanest way to add this without code duplication or too much complexity.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_fetch"
There seems to be an issue with the patch, revert for now.
This reverts commit cd5e0a32184d98beb8d81e6b5527166d3ca4fb3c.
|
|
Dependencies on svn:// urls were failing as the cleandirs on do_fetch was destroying
any sysroot setup by the extend_recipe_sysroot function.
Add code so that if the task do_fetch, we move the cleandirs to the extend_recipe_sysroot
task else we'd wipe out the sysroot we just created.
This allows fetcher do_populate_sysroot dependencies to work correctly again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if something is added to a sysroot, its hash remains unchanged,
and it continues to be buildable, it doesn't get removed from the sysroot.
This patch handles the case where something is removed from DEPENDS or
[depends].
It does introduce its own issue where something could get removed even
though some other task in parallel may have the same requirement. This
case should be extrememly rare and fixing the more common DEPENDS removal
is likely the bigger win though.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Changed were made to the code but not reflected in vardepsexclude, fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The original description for this was:
"""
Since we clean out do_populate_sysroot if do_configure runs, don't
allow do_populate_sysroot_setscene functions if we're going to
run do_configure.
"""
With RSS, we don't need to clean do_populate_sysroot any more. Since
we no longer do that, this function also has no purpose any longer
and can also be removed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With recipe specific sysroots its now pointless to do this, may as well
save the cpu cycles.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BB_SETSCENE_VERIFY_FUNCTION2 replaced BB_SETSCENE_VERIFY_FUNCTION
and due to our minimum bitbake requirements there is no point in
retaining the older version any more.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you abort a build mid execution of the staging extend_sysroot function
there are currently races and cleanup of that function may fail.
This change splits the code into separate phases so that the manifests are
manipulated before files are installed, meaning we should be able to reverse
actions if builds fail, crash or are interrupted.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This wastes space/time as the ptest files aren't going to be used in the
sysroot so add them to the blacklist.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Alexander Kanavin
Junling Zheng
Nathan Rossi
Ross Burton
Peter Kjellerstedt
Alejandro Enedino Hernandez Samaniego
Robert Yang
Mikko Ylinen
Ming Liu
Marko, Peter
Paul Eggleton
Tobias Hagelborn
Chen Qi