| Age | Commit message (Collapse) | Author |
|---|
|
Ensure that the plain variable X is also added to the dependencies
as well as X_<pkg>.
Allow the funciton to be called with a different variable list too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By default, we won't copy and strip static libraries. However, this
functionality can be useful in some cases where people are doing
development on the target, and don't generally want the larger debug
capable static libraries. To enable the new functionality set:
PACKAGE_DEBUG_STATIC_SPLIT = '1'
Add a new function splitstaticdebuginfo. Thus function will copy the
unmodified static library into the specific debug directory location.
By keeping an unmodified version, it is possible for a user trying
to debug something to use -L /usr/lib/.debug-static and their existing
build commands to switch from stripped to full debug versions.
The PACKAGE_DEBUG_SPLIT_STYLE will select between two different
approaches, /usr/lib/debug-static or <path>/.debug-static.
Additionally you can now choose to strip static libraries to conserve
space. If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC'
is set to 1, the static library will be stripped. (This is not on by
default, as it could make diagnosing static library usage difficult in
some cases.)
Add to insane.bbclass a skip to the staticdev warning for the specific
-dbg package versions.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Change the order of the skip processing to happen before any .ko and static
library processing. This will allow these types of files to be individually
skipped if necessary.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Format of the sources list is the [ (file, [source, ...]), ... ] before
this change, the static libraries were processed but the items were
included incorrectly causing no sources for static libraries to be
included.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When excluding a package from being packaged due to incompatible
licenses, report the license(s) that are actually incompatible.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that ldconfig may get installed from a feed, use it when it's
available on the target.
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the case where multiple packages provide the same file, we show an error.
Otherwise, python will generate a different build depending on which provider
appears first in the dictionary. On my system this order changes every time
I run bitbake causing intermittent build differences.
Add a sorted() to fix the determinism issue too.
Signed-off-by: Jonathan Marler <johnnymarler@hp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
PRIVATE_LIBS is used to exclude 'private' libraries from getting added to
automatic runtime dependency resolution. This variable currently has to list
all libraries by name, which becomes a maintenance issue if the list
of such libraries frequently changes, or is very large.
This change allows using shell-style wildcards in the variable, similar
to how FILES lists what gets packaged.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
do_package has PKGDESTWORK as a cleandir and do_packagedata has it as an sstate-input
dir. This means do_package wipes out the directory at both do_package and do_package_setscene.
do_package_setscene and do_packagedata_setscene can run in parallel when installing from
sstate which means they can wipe out parts of each other leading to interesting
build failures.
We therefore have to add in a hardlink copy so that the directories can work independently
of each other.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its possible in cases with multiple shlib providers we were not being
deterministic. Add in a couple of sorted() calls to fix the shlibs and
pkgconfig cases with this potential issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a package uses PKG_ variables to map package names to version specific
variants, on target postinstall functionality will be broken.
i.e. something like the following casuses rootfs assembly errors:
d.setVar('pkg_postinst_ontarget_linux-source', 'cd /usr/src/; ln -sf %s linux-source' % source_pkg)
This breakage is due to the fact that the original package name (as specified by
the PACKAGES variable) is logged by the intercept scripts, but the mapped /
specific version is actually installed to the rootfs (and hence logged by the
package manager).
When the runtime listing of on-target scripts is performed, we get a package
manager error due to a missing package, since it checks the generic version
logged by the intercept scripts.
We can fix this by ensuring that the PKG_ variable mapped package name
is logged by the intercept phase, and hence the package manager can locate
and execute the on target postinst script.
This variable check is consistent with other places in the code, and has
no impact if PKG_ variables are not used.
Signed-off-by: Bruce Ashfield <bruce.ashfield@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit b08976456c8ab7f29efd83644ce42746c0d6501b.
|
|
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
populate_packages relies on ``mkdir`` to both create a directory and set
its permissions. However, ``mkdir`` honors the ``umask`` value.
Therefore, some bits may be lost in the operation. In our case, the
setgid bit on the directories were lost.
This commit fixes this by having a distinct call to create the directory
and to set the permissions.
Signed-off-by: Jean-Tiare Le Bigot <jean-tiare.le-bigot@easymile.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This introduces a variable, PKGDATA_VARS, that contains the names of
the variables that are to be output in the runtime pkgdata files.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sorts the list of detected ELF files by path before processing. This
ensures that when multiple files are hardlinked together the first one
found is always the same. This is required to have reproducible builds.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Permanently adds the -src source package to ${PACKAGES} instead of
relying on creating it dynamically at packaging time. The source package
is now defined in bitbake.conf, just like -dev and -dbg packages.
For compatibility, the -src package is still added dynamically if it was
missing, since some recipes have a tendency to completely override
PACKAGES and do so without manually adding back the -src package.
This allows RDEPENDS on the -src packages, which wasn't previously
possible.
[YOCTO #13203]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently we parallel process the files we install running dwarfsrcfiles over
each one in parallel threads but requiring a lock to write the results to one
file. This is not ideal for performance and means we can't then use per file
data for other purposes such as source code license processing.
Rework the code so that the list of source files is generated per installed
file and is reusable.
The code still generates a null separated debugsources.list file since this
is used by a shell pipeline but it no longer needs locking.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Changes to pkg_postinst_ontarget were not triggering rebuilds, this fixes
that.
[YOCTO #13127]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.
Note that some show up as:
"""
meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.
"""
where the problem isn't on 1293 in package.bbclass but in some _prepend to a
package.bbclass function in a different file like mesa.inc, often from
do_package_split() calls.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Observing depsig.do_package for packages inbetween rebuilds indicated
that the following variables/files content was changing order randomly.
Make them deterministic by sorting the output:
RDEPENDS_<pkg>
RRECOMMENDS_<pkg>
FILERDEPENDSFLIST_<pkg>
packages-split/<pkg>.shlibdeps
The following variable was not observed to change, but it is
assumed that the same situation can occur, so do the same
sorting for consistency:
FILERPROVIDESFLIST_<pkg>
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We use subprocess.check_output() to run the command, which means that we need care
about the error, so the 2>/dev/null should not be used, otherwise it is hard to
debug when the error happens.
I guess it was copied from previous lines, but that command's error can be
ignored (excpet: pass):
try:
subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError:
# Can "fail" if internal headers/transient sources are attempted
pass
But we don't do this in the current location, so remove "2>/dev/null"
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
Observing depsig.do_package for a package inbetween rebuilds indicated
that FILES_INFO was changing content order randomly. Force it to be
deterministic by sorting with respect to the keys when serializing.
Suggested-by: Joshua Watt <jpewhacker@gmail.com>
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix the following warning.
ResourceWarning: unclosed file <_io.TextIOWrapper name='/.../systemd/1_239-r0/debugsources.list' mode='a' encoding='UTF-8'>
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some allarch packages rdepends non-allarch packages. when multilib is
used, it doesn't expand the dependency chain correctly, e.g.
core-image-sato -> ca-certificates(allarch) -> openssl
we expect dependency chain for lib32-core-image-sato:
lib32-core-image-sato -> ca-certificates(allarch) -> lib32-openssl
it should install lib32-openssl for ca-certificates but openssl is still
wrongly required.
Only enable allarch when multilib is not used to fix the issue.
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It's possible that the perms conf file specified by FILESYSTEM_PERMS_TABLES
does not exist. Currently, this situation is silently ignored, which
is likely to lead to further do_rootfs failures.
So fix to output a warning, telling user that the specified file
in FILESYSTEM_PERMS_TABLES cannot be found.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
directory
While multiple hardlinks of binary located in different dirs,
there are also multiple hardlinks of separated debug info file
with the same binary name in same debug dirs. But in each dir,
only one debug file with original name works. Because all of
binary hardlinks have one `.gnu_debuglink' which is added in
`splitdebuginfo'. It caused gdb could not find debugging
symbols.
[Before the patch]
$ find .
./usr/bin/foo
./usr/bin/foo-hd1
./usr/bin/.debug
./usr/bin/.debug/foo
./usr/bin/.debug/foo-hd1
./usr/libexec/foo-hd2
./usr/libexec/.debug
./usr/libexec/.debug/foo-hd2
$ readelf --debug-dump usr/libexec/foo-hd2
Contents of the .gnu_debuglink section:
Separate debug info file: foo
$ gdb usr/libexec/foo-hd2
Reading symbols from usr/libexec/foo-hd2...(no debugging symbols found)...done.
[Before the patch]
[Apply the patch]
$ find .
./usr/bin/foo
./usr/bin/foo-hd1
./usr/bin/.debug
./usr/bin/.debug/foo
./usr/libexec/foo-hd2
./usr/libexec/.debug
./usr/libexec/.debug/foo
$ gdb usr/libexec/foo-hd2
Reading symbols from usr/libexec/foo-hd2...Reading symbols from usr/libexec/.debug/foo...done.
[Apply the patch]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This does the same thing, but is more efficient in case st_nlinks
is (already) 1.
Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c .
Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Replaces usage of the deprecated oe.utils.getstatusoutput() with Python
subprocess calls.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
nativesdk-gpgme fails package_qa when setting PACKAGE_DEBUG_SPLIT_STYLE
= "debug-with-srcpkg".
ERROR: nativesdk-gpgme-1.10.0-r0 do_package_qa: QA Issue: non debug package contains .debug directory: nativesdk-python3-gpg path /work/x86_64-nativesdk-oesdk-linux/nativesdk-gpgme/1.10.0-r0/packages-split/nativesdk-python3-gpg/usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/usr/lib/python3.5/site-packages/gpg/.debug/_gpgme.cpython-35m-x86_64-linux-gnu.so [debug-files]
This turns out to be because the automatic moving of the -dbg package to
the beginning of the package list is disabled in that case, so the
python3-gpg packages that the recipe prepends to PACKAGES ends up before
the -dbg package.
It's not clear why the "and not split_source_package" was added when
debug-with-srcpkg was introduced. Presumably the intention was to
prevent the -dbg package to end up before the -src package, which we of
course need to. But at the same time, we still need -dbg packages to end
up before all other packages.
Using list.insert(0, ...) also means that if there happens to more than
one -dbg package, their relative ordering gets inverted in the new list.
This tries to fix these issues by sorting the packages by (priority,
original position), where priority is 10 for -src, 30 for -dbg and 50
for everything else. That guarantees that packages of the same "type"
preserve their relative ordering, while also ensuring that -dbg always
preceed other packages. This scheme is also quite extensible, and,
should the need arise, one could even expose the priorities as a knob
the recipe author could use to ensure specific orderings of packages
instead of the somewhat fragile and coarse-grained method of "prepend or
append, and ensure you do that in a proper order".
Probably the autodebug condition needs to stay, but I think the
split_source_package condition in the preceding elif should be removed,
so that that logic applies to all packages called -src, not just the one
we might have created a few lines above.
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want to preserve sparse files when building the system, add the option
to tar to ensure we do this when copying files.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Recent changes broke the preservation of hardlinks during processing due to a missing index.
Fix this, reducing the size of the git recipe packages in particular (it contains many hardlinks).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
of setting them
* this allows to work around some of the issues with file-rdeps
* e.g. in my case I have /usr/lib/libmali.so.0.1 which according to rpmdeps
provides only following libmali*
libmali.so(LIBMALI_1.0)
libmali.so.0.1
but many users of this link with libmali.so directly and according to
rpmdeps require libmali.so which causes file-rdeps for a lot of
recipes
* I was using simple work around to just set:
RPROVIDES_${PN} = "libmali.so libGLESv2.so libEGL.so"
but that doesn't work well with multilib, because the value gets
expanded with MLPREFIX and
RPROVIDES_${PN} = "lib32-libmali.so lib32-libGLESv2.so lib32-libEGL.so"
doesn't help at all.
[YOCTO #9217] Many unsolveable QA warnings from build-deps and file-rdeps
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The variable is PACKAGE_WRITE_DEPS as seen in the default below the
comment.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using 'sub' in one function just confuses things, standardise and fix
formatting of the parameters too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After this replacement, the parent exception handling works so we don't
need subprocess wrapping with bb.error in the underlying functions.
The underlying contexts also have better module handling so the imports
can be cleaned up.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Replace usage of oe.utils.getstatusoutput() with direct subprocess calls. The
call is just a wraper to the subprocess call of the same name and requires
the caller to handle exceptions themselves. We usually do this badly, failing
to show the output or the command or the return code.
Its much safer to rely on a call like subprocess.check_output() instead.
This also makes it easier to spot and remove cases where shell=True isn't
needed in a later cleanup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows the calls to is_elf (which calls file) to happen in parallel
allowing a speedup of do_package and do_populate_sysroot for native
recipes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are probably further cleanups needed here but this at least removes
the major code duplication between these two similar funcitons, keeping the
kernel module ".ko" extension check for efficiency to avoid opening and
reading file contents in the general case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This function is a bit more invasive to add parallelism to but
allows the shlibs analysis to happen in multiple threads. In order
to return values correctly/safely the data types needed tweaking
to avoid lists and use immutable objects.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adjust the code so that the splitting of debug symbols from files happens
in parallel. To to this we need to move some path handling code into the
main function and pass more parameters in.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't need any functionality from the shell here, its just extra fork
overhead. Therefore remove it and use subprocess directly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The command for running dwarfsrcfiles is simple and does not need a subshell
for each execution. By expanding out this function to use check_output()
from subprocess and a list of arguments, the shell overhead can be dropped.
For recipes with lots of files this gives a significant saving.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I think this lock dates from before we had sstate for do_packagedata.
Since WORKDIR is recipe specific and we write into WORKDIR, we no longer need
any write locks in the do_packagedata code itself, its handled by the sstate
task lock for the final copy in at the end. The final write lock can be simply
removed.
The only time we need read locking is when actually reading data from the
shared directory. We can therefore reduce the window the lock is held
significantly as well, hence improving the speed of packagedata tasks running
in parallel.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some packages containing shared libraries might be registered
as shlib providers when they shouldn't (for example, the lib is for
their private use and must not generate any dependency).
EXCLUDE_FROM_SHLIBS is targeted at that, but it could be set
for entire recipe only.
This patch allows to set list of packages that are not candidates
for shlib providers.
Variable EXCLUDE_PACKAGES_FROM_SHLIBS is used, e.g.:
EXCLUDE_PACKAGES_FROM_SHLIBS = "glibc-ptest glibc-mtest"
Cc: Andrii Bordunov <aborduno@cisco.com>
Signed-off-by: Oleksii Konoplitskyi <okonopli@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow debugsource listing using dwarfsourcefiles to fail for static
libraries when the archive content is not as expected.
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
PACKAGELOCK is there to protect readers of PKGDATA_DIR from writes and files
changing whilst they're being read. With various changes to the codebase,
the lock code has become confused as the files are now written by the sstate
code in do_packagedata, not in do_package directly any longer.
This change cleans up the code so read sites take the shared lock (anything in
do_package), write sites take the full lock (do_packagedata sstate).
The lock from do_package sstate is no longer needed since it doesn't write
outside WORKDIR.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
!= mingw*
"package.bbclass: Include dbgsrc for static libs" introduced a regression
on mingw, fix this by excluding on that TARGETOS.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Mark Hatle
Peter Kjellerstedt
Andreas Oberritter
Ola x Nilsson
Jonathan Marler
Alexander Kanavin
Bruce Ashfield
Joël Esponde
Joshua Watt
Jacob Kroon
Robert Yang
Chen Qi
Kai Kang
Hongxu Jia
Rasmus Villemoes
Rasmus Villemoes
Martin Jansa
Paul Eggleton
Oleksii Konoplitskyi