| Age | Commit message (Collapse) | Author |
|---|
|
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:
d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')
which I've corrected (they happend to work by luck).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.
Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Previously do_populate_lic would bail writing the license files (such as the MIT
license text) to deploy/licenses/${PN}/ if there were no files listed in
LIC_FILES_CHKSUM. In general this isn't a problem since LIC_FILES_CHKSUM was
effectively a required field, and for packages which don't have traditional
content was generally set to a separate license such as
${COREBASE}/meta/COPYING.MIT.
However LIC_FILES_CHKSUM is now only required if there are sources in SRC_URI,
so oe-core b18fa5f removed these now redundant values. The unexpected side
effect of this is that that recipes that generate their content (such as
os-release) or are otherwise "interesting" (such as perf) don't have their
license files copied over, resulting in warnings from do_rootfs.
Resolve this by not returning early if LIC_FILES_CHKSUM is empty, and always
copy the references license files.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you set LIC_FILES_CHKSUM to point to a relative symlink then you'll
get "Could not copy license file" warnings in copy_license_files() since
the symlink won't be valid after it's copied. If the source is a symlink
then we need to dereference it first.
I encountered this when I used recipetool on the sources for capnproto,
where the c++ directory contains a LICENSE.txt symlink to the LICENSE
file in the parent directory, and this symlink ends up being pointed to
in LIC_FILES_CHKSUM.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This sets a good example and avoids unnecessarily contributing to
perceived complexity and cargo culting.
Motivating quote below:
< kergoth> the *original* intent was for the function/task to error via
whatever appropriate means, bb.fatal, whatever, and
funcfailed was what you'd catch if you were calling
exec_func/exec_task. that is, it's what those functions
raise, not what metadata functions should be raising
< kergoth> it didn't end up being used that way
< kergoth> but there's really never a reason to raise it yourself
FuncFailed.__init__ takes a 'name' argument rather than a 'msg'
argument, which also shows that the original purpose got lost.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The loop iterating over LICENSE_pn variables has never worked. In
addition, the LICENSE variable is supposed to contain all licenses
defined in LICENSE_pn variables. Thus, it is simpler just to use LICENSE
as the data we get is essentially the same.
[YOCTO #9499]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Previously, find_license_files() in license.bbclass just blindly assumed
that all different licenses specified in LIC_FILES_CHKSUM have unique
filenames. As a consequence, only the last one of these similarly named
license files was copied and the rest were "lost". This patch changes
the behavior so that all license files get copied. However, if multiple
identically named files are found, they are renamed to <file>.0,
<file>.1 etc.
The patch also changes the handling of NO_GENERIC_LICENSE slightly.
Previously, only basenames of NO_GENERIC_LICENSE and LIC_FILES_CHKSUM
were compared when searching for the correct license file. After this
patch NO_GENERIC_LICENSE must have the full path, matching what is
specified in LIC_FILES_CHKSUM. This is required in order to be able
to handle identical filenames (basenames) consistently. For example, if
you have:
LICENSE = "my-custom-license"
LIC_FILES_CHKSUM = "file://src/LICENCE;md5=d41d8cd98f00b204e9800998ecf8427e"
you must specify:
NO_GENERIC_LICENSE[my-custom-license] = "src/LICENCE"
[YOCTO #9663]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
|
|
There's no need to set these as the restore from sstate will create the
directories as required.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Hard links can still fail even if st_dev is the same for source
and destination. In case of EXDEV error, fall back to copying.
Signed-off-by: Manuel Huber <manuel.h87@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, write_deploy_manifest() was relying on
write_package_manifest() to create the subdirectory for the manifest
file. However, do_rootfs may be an empty function so that
write_package_manifest() will not be called and the manifest
subdirectory will not be created, causing a build failure. This patch
fixes that by creating the directory hierarchy inside
write_deploy_manifest().
[YOCTO #9446]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Call write_deploy_manifest() in image postprocessing phase, instead of
rootfs postprocessing. The reason being that not necessarily all
do_deploy tasks are dependencies of the do_rootfs and we might miss
something.
[YOCTO #9446]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
An unprivileged "container" environment like this[1] doesn't have root
account (uid 0) which causes tons of "Invalid argument" warnings:
$ bitbake ...
...
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
WARNING: Could not copy license file [src] to [dest]: [Errno 22] Invalid argument: '[src]'
...
Fix it by handling EINVAL similar to existing handling of EPERM (which
was added for when not running under pseudo).
[1]: The real environemnt is buildFHSUserEnv from NixOS/nixpkgs, but a
demonstration of the issue can be done like this:
$ touch f
$ unshare --user --mount chown 0:0 f
chown: changing ownership of ‘f’: Invalid argument
Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We get below host contamination warnings of license files for
each recipe, when we try to create a separate ${PN}-lic package (which
contains license files), by setting LICENSE_CREATE_PACKAGE equal to "1"
in local.conf.
-- snip --
WARNING: QA Issue: libcgroup: /libcgroup-lic/usr/share/licenses/libcgroup/generic_LGPLv2.1 is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
WARNING: QA Issue: attr: /attr-lic/usr/share/licenses/attr/libattr.c is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
WARNING: QA Issue: bash: /bash-lic/usr/share/licenses/bash/COPYING is owned by uid 5001, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]
-- CUT --
Since the license files from source and OE-core, are populated in a normal
shell environment rather in pseudo environment (fakeroot); the ownership of
these files will be same as host user running bitbake. During the do_package
task (which runs in pseudo environment (fakeroot)), os.link preserves the
ownership of these license files as host user instead of root user.
This causes license files to have UID same as host user id and resulting in
above warnings during do_package_qa task.
Changing ownership of license files to root user (which has UID and GID as 0)
under pseudo environment will solve above warnings, and on exiting pseudo
environment the license files will continue to be owned by host user. Perform
this manipulation within try/except statements, as tasks which are not exected
under pseudo (such as do_populate_lic) result in OSError when trying to
change ownership of license files.
Signed-off-by: Jagadeesh Krishnanjanappa <jkrishnanjanappa@mvista.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This class has a perform_packagecopy_prepend() that uses LICENSE_CREATE_PACKAGE,
but the BitBake variable dependency code doesn't know that oe.data.typed_value()
is a getter. Until BitBake knows this, add an explicit vardeps.
[ YOCTO #5635 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This sets the scene for removing the default False for expansion
from getVarFlag. This would later allow True to become the default.
On the most part this is an automatic translation with:
sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g' -i `grep -ril getVar *`
In this case, the default was False, but True was used since in most
cases here expansion would be expected.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We shouldn't try and expand what is a python dict object
and we don't want it to influence the task hashes, task
dependencies are already accounted for.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This change adds the license_deployed_manifest function
that will create the manifest for the packages deployed
next to the image but not installed in rootfs. Some
examples of these recipes would be the bootloaders, or
the kernel.
This new function was added to ROOTFS_POSTPROCESS_COMMAND
so it will run after every rootfs task.
This change also modify the write_license_files because
the image manifest is different from the root manifest.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This function will get the files that were deployed using
the sstate-control manifest file. This will give a better
view of what was deployed next to the image.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|
|
This change introduce a new function to get the dependencies
that were deployed. It uses BB_TASKDEPDATAto get all the
dependencies of the current task, so it is possible to get
different packages depending at what point this function is
called.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|
|
This function gets the dependencies from the classes that
create a boot image, this is required because sometimes
the bootloader dependecy is in these classes. The current
classes covered are bootimg and bootdirectdisk because
these are the only clases that add dependencies.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|
|
This changes moves the writing of the licenses to a
separated function that could be called for other packages.
With these change it will be easier to reuse the writing of
the license for the packages deployed but not installed in
the rootfs.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|
|
Currently there is no way to get the recipe version when
creating the rootfs. It is needed because the manifest
file for the image has to contain this important piece
of information.
This change writes a new file in the license folder for
every recipe. This file is called "recipeinfo" and have
the information used to write the manifest file for the
recipes deployed next to the image.
[YOCTO #6772]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
|
|
Fixes the following warning sometimes appearing during image builds:
WARNING: The license listed ABC was not in the licenses collected for recipe xyz
The files being looked for here, which runs during do_rootfs,
are written out by the do_populate_lic task for each recipe. However,
there was no explicit dependency between do_rootfs and all of the
do_populate_lic tasks to ensure they had run - only an implicit link via
do_build, so it is possible that sometimes they had not depending on how
the tasks were scheduled. Add an explicit set of dependencies to fix
this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.
[YOCTO #7664]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use canonical_license when doing evaluation of license expresion since
INCOMPATIBLE_LICENSE are already canonized.
[YOCTO #8080]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The CLOSED license isn't a generic license it is a set and can
be any closed source license.
[YOCTO #7752]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Use image_list_installed_packges instead of open manually the package
manifest and iterate over it.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When INCOMPATIBLE_LICENSE's is specified it need to be removed from
license.manifest and also avoid copy to target image.
Add ManifestVisitor that walk the license string searching for
INCOMPATIBLE_LICENSE's if found remove it.
[YOCTO #6765]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add dont_want_licenses as parameter to license_ok function and move it
to oe.license module in order to use in other modules.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reimplemented license_manifest_create from shell to python for
INCOMPATIBLE_LICENSE handle using oe.license module.
Optimizations are made to avoid license copy now uses a hardlink
and symbolic link this helps to save space during build.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some package like linux-firmware has many licenses that aren't in any
way common, and new ones will be added from time to time, in order to
avoid adding bunch of such common license files that are only applicable
to a specific package, NO_GENERIC_LICENSE is added to allow copying license
not in common licenses, it should be used in the recipe as:
NO_GENERIC_LICENSE[<license_name>] = "<license file in fetched source>"
e.g.
NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENCE.Abilis.txt"
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
|
|
If the package doesn't contain any file, then the license isn't relevant
as far as the final image is concerned. So we skip the license checking
in license_create_manifest if such case.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed:
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "bitbake/lib/bb/build.py", line 497, in exec_task
return _exec_task(fn, task, d, quieterr)
File "bitbake/lib/bb/build.py", line 437, in _exec_task
exec_func(func, localdata)
File "bitbake/lib/bb/build.py", line 212, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/home/nxadm/nx/ala-blade44.1/builds-2015-03-09-163005/qemuppc_world_oe_bp/bitbake/lib/bb/build.py", line 237, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: 'bitbake_build/tmp/work/ppc7400-wrs-linux/taglib/1.9.1-r0/build'
When running setscene, the cwd is $B which maybe removed by
autotools.bbclass or cmake.bbclass when rebuild.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
* add quotes around pkged_lic so that it works correctly with spaces
* fixes following error:
run.license_create_manifest.50601: 193: [: GPLv2: unexpected operator
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Use argparse instead of optparse for standardised help output, options
and a much cleaner code structure
* Look up pkgdata directory automatically so the user doesn't have to
specify it
* Use standard logging
NOTE: this does mean a slight change in syntax - if you do want to
specify the pkgdata directory (usually only necessary if you're calling
it from within the build process) you need to use the parameter -p (or
--pkgdata-dir) and specify this before the command, not after it.
Examples:
oe-pkgdata-util find-path /sbin/mke2fs
oe-pkgdata-util lookup-recipe libelf1
oe-pkgdata-util read-value PKGSIZE libc6
oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Strictly speaking not all of these characters are operators, so reword
the message to describe them as separators. Also use the standard
"recipename: message" format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed:
* The os.link() reqiures write permission on the src file (suppose the
src file belongs to another user, then you need write permission to harlink to
it since the link count would change)
* Print more info when failed to copy
The warning was like:
WARNING: Could not copy license file COPYING: [Errno 1] Operation not permitted
We couldn't know which recipe print the warning from this message.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add check_license_format function that shows warning if LICENSE don't have
valid operators and also if have space separated entries without operator,
add check_license_format validation into base class.
[YOCTO #6758]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Drop removal of [|&()*] operators in pkged_lic because this removal is only
needed to validate if license is collected.
[YOCTO #6757]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The whitelist processing in code in base.bbclass does not play well with
wildcards in INCOMPATIBLE_LICENSES. The code expects bad_licenses to
contain actual license names, not wildcards.
Add incompatible_license_contains to replace bb.utils.contains(
"INCOMPATIBLE_LICENSE", **, **, **, d)
[YOCTO #5592]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
|
|
If INCOMPATIBLE_LICENSE=GPLv3, GPLv3+ should be excluded
as well but not now since there is no SPDXLICENSEMAP for
licenses named with 'X+', we can add all the SPDXLICENSEMAP
settings for licenses named with 'X+' in licenses.conf,
but it's more like a duplication, so improve the canonical_license
function to auto map for 'X+' if SPDXLICENSEMAP for 'X' is
available, so GPLv3+ becomes GPL-3.0+.
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Currently copies of the license files are made which wastes disk space
and adversely affects performance. We can link these instead in most
cases for small performance gains.
(From OE-Core rev: 0b0f3631fd22f731b6aeedb73965e367b695028b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If INCOMPATIBLE_LICENSE=GPL-3.0 but the recipe sets LICENSE=GPLv3, the current
code won't trigger because they're different strings.
Fix this by attempting to canonicalise every license name to a SPDX name, so
both names in this example become GPL-3.0.
[ YOCTO #5622 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we don't fix this, the fix for bug 6502 will trigger warnings
that the write_package_manifest function contains tabs.
Related to fix for [YOCTO #6502].
Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Joshua Lock
Ross Burton
Paul Eggleton
Ulf Magnusson
Markus Lehtonen
Manuel Huber
Bjørn Forsman
Jagadeesh Krishnanjanappa
Mariano Lopez
Jackie Huang
Alex Franco
Aníbal Limón
Chen Qi
Robert Yang
Martin Jansa
Hongxu Jia
Roxana Ciobanu