| Age | Commit message (Collapse) | Author |
|---|
|
Having these two separate functions handling PR values seems pointless,
and worse, there are impossible code branches mixed within them.
Merge them into one function and tweak comments so at least you
don't have to read both functions to figure out what is going on.
This does restructure the conditionals to try and aid readability.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
dictionary
shlib2 code puts the information about path where a provider
will be found. e.g.
{'/usr/lib/llvm3.3': ('libllvm3.3-llvm-3.3', '3.3')}
This is obtained from new shlib2 pkgdata from
llvm3.3/3.3-r0/pkgdata/shlibs2/libllvm3.3-llvm-3.3.list
However when we search for NEEDED libraries we ignore the
key above which is the path where the provider library is installed
and instead just seach in libdir and base_libdir and hence
libraries which are not in above standard search paths gets
ignored even if they appear in DT_NEEDED sections
and a note is emitted
NOTE: Couldn't find shared library provider for libLLVM-3.3.so, used by
files: ....
IMO this note should actually become an error since if we do
not have all DT_NEEDED libraries in image the system is dysfunctional.
This patch extracts this libpath from key and add it to seach paths
when looing for a provider of a shared library
[YOCTO #6798]
Change-Id: Ie5f08632e37ba8d3439c8aaae33bc68b8996792f
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In case a package is not generated (is empty and does not has allow
empty flag set) the package data regarding reverse runtime dependency
shouldn't be done.
This were causing a false-positive in the meta-fsl-arm layer, when
building mesa, as:
,----[ Error during build of MX53 in meta-fsl-arm ]
| ERROR: The recipe mesa is trying to install files into a shared area
| when those files already exist. Those files and their manifest
| location are:
| /.../build/build/tmp/sysroots/imx53qsb/pkgdata/runtime-reverse/libopenvg-dev
| Matched in manifest-imx53qsb-amd-gpu-x11-bin-mx51.packagedata
| Please verify which recipe should provide the above files.
`----
Fixes [YOCTO: #6795]
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This quite coincidentally invalidates the sstate for do_package which
is needed due to the correction of oe.utils.multiprocess_exec().
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It doesn't really make sense to run the remapping code on these
variables. If specific renaming is needed, it should be applied
manually. This means that the debian RPROVIDES of the original package
name can be preserved. There was also a bug report about this on the
OE-Core mailing list recently where someone else ran into this problem
too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For package groups, use the original package names and don't use the
remapped (e.g. debian) naming.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We may as well use the common function for this rather than
duplicating the code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We need to scan binaries as well as libraries for dependencies.
Also ensure if its not an object file (as found by otool), we handle
this case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, the darwin shlibs detection is done by parsing the .la file
dependency fields. This is very old code and is incomplete in some
cases so convert to using otool -l and otool -L to correctly load
the rpath and dependency information.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the code was rewritten we forgot to strip the pkg that is present
in the pkgdest path. This was fixed in the linux version of the code
but not the darwin one, this matches the fix.
Without this, the provider paths are broken.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_package() calls package_name_hook so that e.g. debian-style renaming through
debian.bbclass can happen. If there is no class providing a package_name_hook
then this causes "WARNING: Function package_name_hook doesn't exist" every time
do_package() is executed.
Silence this warning by providing an empty package_name_hook in package.bbclass.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
Currently, given a list of rdepends its near impossible to figure out
which ones are potentially invalid and which might be an RPROVIDES of a
package.
This problem is simple to solve, we can write lookup data into the
pkgdata. This patch does that, accounting for the fact that multiple
packages can RPROVIDE the same thing (particularly with locales). Its
done with symlinks so the performance overhead is minimal.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Change the do_package shlibs code to account for RPATHS. This means that
for library dependency purposes, only libraries in system paths or in
any declared RPATH will be seen. This is important to resolve problems
people have been having where similarly named libraries in "private"
paths were conflicting, e.g. with gstreamer.
For now this code assumes the default search path is libdir and
base_libdir and places ASSUME_SHLIBS in libdir so they are searched by
default.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of having a .list file and a .ver file, place the version
information into the .list file in a ":" delimited string.
Also place the path to the library here, this can then be used to
evaluate RPATHs in the shlib dependency code.
Since the disk format has changed, the easiest way to avoid build
failures in the same TMPDIR is to change the shlibs directory to
shlibs2. sstate dependency code with ensure everything rebuilds.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to do more advanced processing of the shared libraries, we need
to know where a given library is located on disk so we can know whether
its a system path or a private directory for example.
This patch adds this information into the 'sonames' data structure.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Improve the shlibs 'needed' data structure to include the file and
any rpath information. This allows various cleanups to the data structure
and moves us closer to being able to resolve shlibs providers issues
based on path in due course.
This commit doesn't change any stored data, just cleans up internal data
structures (for example dropping the needed_from dict).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Changes to CONFFILES should change the sstate checksum. To make that happen,
it needs to be listed in the list of package specific variables, therefore
add it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The reasons this task was introduced are lost in the mists of time. It
allowed for the a single "package_write" task instead of spelling out
the explicit package backends, however in all but one case we do that
anyway.
As such as might as well give in and delete the task, converting that
single reference into explicit dependencies.
This gives bitbake a bit less work to to when processing the runqueue
since there are less tasks (but more dependencies in some cases).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
So far, when do_split_packages is passed a non-existing folder, it will return nothing.
While building Mesa with PACKAGECONFIG="egl gles" for qemux86, do_package threw a nice exception on a line reading "pkgs += do_split_packages":
"Exception: TypeError: 'NoneType' object is not iterable"
I think a function should always return something, and in this specific case, returning an empty list should be right.
This patch simply fixes the return statement to do just that.
Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
Currently, if you bitbake xxxx, it will restore the package_write_,
packagedata, populate_sysroot *and* the package data from sstate.
We generally care about the packagedata and the actual packages, we
no longer need/care about the do_package data itself unless something
is going to repackage.
By relaxing this constraint (and relying on the packagedata target), we
can save things being installed from sstate and hence speedup build times.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Marks original commit message and variable documentation state that stripping and splitting are independent of eachother, but package.bbclass ANDs the two INHIBIT flags to see which files can be stripped and/or split.
Original behaviour:
INHIBIT_PACKAGE_STRIP: no strip, no debug split
INHIBIT_PACAKGE_DEBUG_SPLIT: no strip, no debug split
Behaviour after this patch:
INHIBIT_PACKAGE_STRIP: no strip, no debug split
INHIBIT_PACKAGE_DEBUG_SPLIT: strip, no split
Signed-off-by: Koen Kooi <koen.kooi@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* when log.do_package shows some unexpected dependency, people usually
need to grep package directory to find which binary was creating that
dependency, show it directly in the debug output
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* split PRIVATE_LIBS and don't use find(), so that libfoo cannot be
found in PRIVATE_LIBS = "libfoobar"
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Two variables are searched for value of additional package metadata:
* PACKAGE_ADD_METADATA_<PKGTYPE>
* PACKAGE_ADD_METADATA
First found variable with defined value wins.
<PKGTYPE> is a parameter of getter and expected to be a distinct name
of specific package type. For example: 'DEB' or 'RPM'.
Variable can contain multiple [one-line] metadata fields, separated by
literal sequence '\n'. Separator can be redefined through variable flag
'separator'. Getter returns found value with separator replaced with
newline character.
As side-effect, searched variables acquired flags 'type' (equals to
'list') and 'separator'.
Signed-off-by: Leonid Borisenko <ive.found@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
* move read_shlib_providers before registering package as provider
and show warning when different package tries to provide something
already provided.
[YOCTO #4628]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* prepare for reading shlibs providers only from dependency tree of
current recipe
[YOCTO #4628]
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since ca86603607a69a17cc5540d69de0e242b33382d3 we are now calling stat() on
all the pkgfiles[] during emit_pkgdata(). If symlink snapping has removed
some of the files then we will blow up trying to stat a path that no longer
exists. Fix that by ensuring that pkgfiles[] is updated when we process the
list of library renames.
Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_split_packages() is commonly used to split out plugin/module packages
dynamically within recipes. If it doesn't set SUMMARY for each of these
packages then they get a generic SUMMARY from the recipe, which isn't
particularly useful; so add a parameter to set this and default it from
the current description parameter (it ought to have been the other way
around, but the description parameter is what we currently have in use
by all recipes that use this function.)
Fixes [YOCTO #5406].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
When originally developed, it was thought a task may have more than one associated
sstate archive. The way the code has grown that idea is now not possible or needed.
We can therefore assume one sstate archive per task and drop the crazy name
mapping code. Simpler is better in this case.
The downside is that various sstate archives will change name so this forces a cache
rebuild. Given the other sstate changes going in at this time, this isn't really
a bad thing as things would rebuild anyway.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These are important parts of the version for every package, so we should
include them in PKGDATA just as we include PV/PR/PKGV/PKGR.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We were using "du -sk" to collect the total size of all files in each
package for writing out to PKGSIZE in each pkgdata file; however this
reports the total space used on disk not the total size of all files,
which means it is dependent on the block size and filesystem being used
for TMPDIR on the build host. Instead, take the total of the size
reported by lstat() for each packaged file, which we are already
collecting for FILES_INFO in any case.
Note: this changes PKGSIZE to be reported in bytes instead of kilobytes
since this is what lstat reports, but this is really what we should be
storing anyway so that we have the precision if we need it.
Fixes [YOCTO #5334]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The FILES_INFO entry in each pkgdata file stores the list of files for
each package. Make the following improvements to how this is stored:
* Store paths as they would be seen on the target rather than
erroneously including the full path to PKGDEST (which is specific to
the build host the package was built on)
* For simplicity when loading the data, store complete paths for each
entry instead of trying to break off the first part and use it as the
dict key
* Record sizes for each file (as needed by Toaster)
* Serialise the value explicitly using json rather than just passing it
through str().
Fixes [YOCTO #5443].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix false error report when a file that has a single quote by escaping
the single quote. Some packages might install files with quotes, such
as music files and other types, that will cause the problem.
Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
It is uncommon, but it is possible for upstream sources to contain files
that have wildcard characters in their names (Webmin is an example).
Because we were running glob.glob() on every entry in the list of
entries in FILES and then adding the result to the files list to be
processed, the process would loop infinitely if files whose names
contained wildcard characters were present. Fix this by avoiding
re-processing the output of glob.glob() with itself, and also "escape"
wildcard characters in FILES entries added automatically from
do_split_packages().
Fixes [YOCTO #1676].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
tar version 1.27 returns:
tar: --same-order option cannot be used with -c
with the commandlines we have been using. We can remove the -s option (which
is --same-order) to remove the error.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We switched to using os.mkdir with the file creation mode specified as the
second parameter. Python masks this with umask behind the scenes which isn't
what we want, we really want the permissions we specify.
To avoid this we zero the umask beforehand and restore afterwards. Other
solutions are possible but would not perform as well which is why
we're using os.mkdir in the first place.
Martin Jansa deserves the credit for debugging where the problem was.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
directory
Currently we have a hierarchy of pkgdata directories and the code has to put together
a search path and look through each in turn until it finds the data it needs.
This has lead to a number of hardcoded paths and file globing which
is unpredictable and undesirable. Worse, certain tricks that should be
easy like a GL specific package architecture become problematic with the
curretn search paths.
With the modern sstate code, we can do better and construct a single pkgdata
directory for each machine in just the same way as we do for the sysroot. This
is already tried and well tested. With such a single directory, all the code that
iterated through multiple pkgdata directories and simply be removed and give
a significant simplification of the code. Even existing build directories adapt
to the change well since the package contents doesn't change, just the location
they're installed to and the stamp for them.
The only complication is the we need a different shlibs directory for each
multilib. These are only used by package.bbclass and the simple fix is to
add MLPREFIX to the shlib directory name. This means the multilib packages will
repackage and the sstate checksum will change but an existing build directory
will adapt to the changes safely.
It is close to release however I believe the benefits this patch give us
are worth consideration for inclusion and give us more options for dealing
with problems like the GL one. It also sets the ground work well for
shlibs improvements in 1.6.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some packages like grub have already stripped binaries e.g.
ERROR: QA Issue: File '/boot/grub/kernel.img' from grub was already
stripped, this will prevent future debugging!
ERROR: QA run found fatal errors. Please consider fixing them.
We would like to have a possibility to skip it using something like
INSANE_SKIP_${PN} = "already-stripped"
This adds the logic to do so
it acts at PN level and not at package level. so something like
INSANE_SKIP_${PN}-misc = "already-stripped" wont work.
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
shlibs dependency calculations on darwin we not functioning correctly, we
need to process the filename without the complete path. If we don't,
"." characters in the path cause problems.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When copying the sources for the debug source package we use cpio -Ll
which means to copy files as hardlinks and to dereference symlinks.
It appears there is a bug in cpio since -Ll will copy symlinks and
not dereference them. We therefore do a second pass over copied symlinks
resolving them into files. Ideally we would copy these as hardlinks as well
however it doesn't seem worth the extra code and effort for what amounts
to a corner case for a minor space improvement.
This means that the -dbg packages no longer contain broken symlinks.
[YOCTO #5020]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The error case was gcc-dbg packing files from libgcc-dbg reported in [1]
[1] http://lists.openembedded.org/pipermail/openembedded-core/2013-July/080728.html
Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
When external commands such as debugedit fail, it can be useful to see
their output, so use oe.utils.getstatusoutput() instead of
subprocess.call() to capture this and print it on failure.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
There is the possibility that if we don't iterate through the multiprocessing
pool objects we might not catch return codes and this could lead to hung/zombie
processes either temproarily or on a wider scale. Adding this certainly doesn't
hurt anything and is better practise so we might as well do it.
Its not 100% clear if this fixes some issues or not.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Refactor packaging sanity tests from package.bbclass to insane.bbclass
so that the message can respect WARN_QA (print the warning message and
go on the task) and ERROR_QA (print the error message and fail the
task).
- For the bb.warn(), give it a message name and add it to WARN_QA, then
use package_qa_handle_error() to handle it.
- For the bb.error(), give it a message name and add it to ERROR_QA,
then use package_qa_handle_error() to handle it.
- All the bb.warn() and bb.error() have been replaced in
package.bbclass.
- A few bb.warn() and bb.error() in insane.bbclass have been kept since
they can not be replaced or doesn't have to, for example the
bb.error() in package_qa_check_license(), it will print the error
message and then invoke bb.fatal() to fail the task, I think that we
don't have to replace it with package_qa_handle_error().
- Put all the WARN_QA and ERROR_QA in one line, so that they can be
redefined by the user easily.
[YOCTO #3190]
[YOCTO #4396]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
RP's comment:
"What we're trying to do is move everything to use a standard mechanism
for reporting issues of this type (do_package). With insane.bbclass, you
can elect whether a given type of error is a warning or error and fails
the task."
* The package.bbclass had used package_qa_handle_error() which is from
insane.bbclass, and we will use it for handling other warnings and
errors, so let package.bbclass inherit insane.bbclass, this change will
make the insane as a requirement (always included).
* Change the "PACKAGEFUNCS ?=" to "+=", otherwise there would be an
error like:
Exception: variable SUMMARY references itself!
This is because we let package.bbclass inherit insane.bbclass, and
PACKAGEFUNCS has been set in insane.bbclass, so the "PACKAGEFUNCS ?="
will set nothing, then the "emit_pkgdata" doesn't run which will
cause this error.
* Add a QA_SANE variable in insane.bbclass, once the error type
is ERROR_QA, it will fail the task and stop the build.
[YOCTO #3190]
[YOCTO #4396]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
The bb.fatal() is defined as:
def fatal(*args):
logger.critical(''.join(args))
sys.exit(1)
So anything after bb.fatal() in the same code block doesn't have any
effect, e.g.:
bb.fatal("%s_%s: %s" % (var, pkg, e))
raise e
The "raise e" should be removed.
I searched all the files which use bb.fatal(), only the following 3
classes have this issues:
insane.bbclass
package.bbclass
package_rpm.bbclass
[YOCTO #4461]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c376f1f49cea182a2887945840ab97a20970a373
fixed a valid issue where the sources file was accumulating information
and subsequent task runs of do_packacge were not cleaning it. The
fix is wrong however since we're removing the file within a loop.
This fix removes the file outside the loop ensuring it is not truncated
and contains the correct information.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are various bits of cruft that have built up around our file accesses. This patch
cleans some of them up, specifically:
* Remove pointless "from __builtin__ import file"
* Use open(), not file()
* Wrap file usage in a with container to ensure files are closed
* Add missing .close() calls in some cases
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets
add a wrapper and have this work in both worlds.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The commands module is removed in python3. Use the subprocess module instead
and the pipes module to replace the mkargs usage.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Khem Raj
Otavio Salvador
Peter Kjellerstedt
Ross Burton
Josua Mayer
Koen Kooi
Martin Jansa
Leonid Borisenko
Phil Blundell
Paul Eggleton
Felipe F. Tonello
Andreas Müller
Robert Yang