| Age | Commit message (Collapse) | Author |
|---|
|
In expectation of SPDX 3.0 support, move the create-spdx.bbclass ->
create-spdx-2.2.bbclass. The create-spdx.bbclass class still exists and
can be used if a user doesn't care about which specific version of SPDX
they get.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
[YOCTO #14948]
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Signed-off-by: Keiya Nobuta <nobuta.keiya@fujitsu.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
When a link is pointing to location inaccessible to build user (e.g. "/root/something"),
filepath.is_file() throws "PermissionError: [Errno 13] Permission denied".
Fix this by first checking if it is a link.
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Otherwise spdx can have references for data that is not packed
in the package delivered because this contol data is temporarly
and only exist while the package is been write.
During do_package_write_ipk task in do_package_ipk the control
files is cleaned up at the end. This can create a race condiction
when the do_create_spdx task runs the add_package_files function
and these files is been deleted at same time in the
task do_package_write_ipk.
ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_create_spdx(d)
0003:
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 567, function: do_create_spdx
0563: package_doc.add_relationship(package_doc, "DESCRIBES", spdx_package)
0564:
0565: package_archive = deploy_dir_spdx / "packages" / (package_doc.name + ".tar.zst")
0566: with optional_tarfile(package_archive, archive_packaged) as archive:
*** 0567: package_files = add_package_files(
0568: d,
0569: package_doc,
0570: spdx_package,
0571: pkgdest / package,
File: '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', lineno: 234, function: add_package_files
0230: info.mtime = source_date_epoch
0231:
0232: archive.addfile(info, f)
0233:
*** 0234: sha1 = bb.utils.sha1_file(filepath)
0235: sha1s.append(sha1)
0236: spdx_file.checksums.append(oe.spdx.SPDXChecksum(
0237: algorithm="SHA1",
0238: checksumValue=sha1,
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: sha1_file
0555: """
0556: Return the hex string representation of the SHA1 checksum of the filename
0557: """
0558: import hashlib
*** 0559: return _hasher(hashlib.sha1(), filename)
0560:
0561:def sha384_file(filename):
0562: """
0563: Return the hex string representation of the SHA384 checksum of the filename
File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher
0524:
0525:def _hasher(method, filename):
0526: import mmap
0527:
*** 0528: with open(filename, "rb") as f:
0529: try:
0530: with mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) as mm:
0531: for chunk in iter(lambda: mm.read(8192), b''):
0532: method.update(chunk)
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control'
ERROR: Logfile of failure stored in: /srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/temp/log.do_create_spdx.998864
INFO: recipe alsa-topology-conf-1.2.5.1-r0: task do_create_spdx: Failed
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
|
|
The correct field name is "supplier" according to SPDX schema.
The "supplier" field translates to "PackageSupplier", but that's for
tag-value format.
Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds an option to make the SPDX more human-readable (at the expense of a
larger files)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building FreeRTOS this dependency generates an error because
bitbake cannot find any provider for "virtual/kernel".
>From a dependency analysis the task is executed independently from
this so it can be safely removed.
This patch has been discussed in this ML thread:
https://lists.openembedded.org/g/openembedded-core/topic/90602531
Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If IMAGE_NAME and IMAGE_LINK_NAME are the same an
exception is raised in image_combine_spdx because
it cannot create a symlink with the same exact
path of the original file. In that case there is
no point in creating a link in the first place.
Refactor image_combine_spdx to avoid code duplication
Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
create-spdx.bbclass:43: DeprecationWarning: invalid escape sequence \W
lic_regex = re.compile(b'^\W*SPDX-License-Identifier:\s*([ \w\d.()+-]+?)(?:\s+\W*)?$', re.MULTILINE)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is no reason to first search for all available licenses using
avail_licenses() and then search through the same paths looking for
the actual license texts. Just look for the license texts directly
instead.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
We can directly call the function rather than using the variable indirection.
As this is the last user of the variable, it then allows removal of it in a
followup patch.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is no reason to include a path in foo[dirs] if it is also in
foo[cleandirs] (except if it is the last path in foo[dirs]).
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, SPDX SBOMs are only created for images. Add support for
SDKs.
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch will read the begining of source files and try to find
the SPDX-License-Identifier to populate the licenseInfoInFiles
field for each source file. This does not populate licenseConcluded
at this time, nor rolls it up to package level.
We read as binary file since some source code seem to have some
binary characters, the license is then converted to ascii strings.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A superfluous package was being created, but it was never used.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Per NIST SBoM recommendations, include the Package Supplier field for
all SPDX packages that are created. This field should generally be set
to the person or organization that is performing the build, since they
would be considered the "supplier" of the SPDX packages.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since the kernel source is stored in work-shared, we need to add it
to the search path so the kernel and kernel-modules source code can
be found correctly.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If IMAGE_LINK_NAME is set to "" (supported in other classes) then skip
creating symlinks for the spdx manifest files.
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, the do_create_runtime_spdx task fails with a Key Error if a
dependency is not contained in the package providers dictionary. Add a
check before using "dep" as a key in "providers".
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If LICENSE_PATH is not set, then the split() will fail on a NoneType.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a function with the same name is_work_shared() in the archiver class
this causes a conflict when both classes are included. Use work-shared as the
check in WORKDIR to allow for other packages beyond the kernel and gcc that
use a common shared-work source directory.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, the "Organization" field for SBOMs is hard-coded in
create-spdx. Create a new variable SPDX_ORG to make this field more
generic.
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Recipes that inherit cross should also be categorized as isNative
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
This allows code reuse and future usage with relationship annotations
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Create a small function that checks for 'isNative' as part of an Annotation
When the collect_dep_sources() runs, it collects sources from both native
and non-native recipes. Later when the GENERATED_FROM matching occurs it
may find the file (via checksum) from the native recipe since it's the
same checksum as the target file. The that are generated DocumentRefs
point to the native recipe rather than the target recipe DocumentRef.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before, even if the code was seemingly written to search through all
licenses in ${COMMON_LICENSE_DIR} and ${LICENSE_PATH}, it would
actually bail out after only searching ${COMMON_LICENSE_DIR} due to
the exception handling.
Also refrain from using f-strings.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The special CLOSED license means that this is closed source code with
no other licensing assertions. It's not a generic license in oe-core,
and not a SPDX license, so transform it to NONE explicitly.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The trailing comma turns ref_id into an object instead of a string
when added to the relationship table.
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Instead of putting the full license text for non-SPDX licenses into the
recipe and every package, use links to the recipe from the packages if
possible.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For licenses which are not known to SPDX, find and embed the actual
license text in an ExtractedLicesingInfo block.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
add_extracted_document() is only called if the license isn't known to
SPDX, so there's no need to check again.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
spdx-licenses.json contains an array of licenses objects. As we'll be
searching it often, convert that to a dictionary when we parse it.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't want this warning causing problems on the AB, so leave it
comment out for now
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Native recipes do not produce packages and should not process them,
otherwise it can trigger an error in read_subpackage_metadata
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Stop filtering the runtime dependencies based on do_create_sdpx (makes
it only pick up things in DEPENDS) and instead include all task
dependencies that are not the current PN. This allows other dependency
methods to be picked up correctly, for example the dependency on the
kernel used by kernel modules.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The Public Domain license (PD) needs a special exception in the license
processing since there is no common license text to be extracted for
these licenses.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
file:// URIs should not be included as the downloadLocation. Instead,
loop until a non-file:// URI is found, or set the location to
NOASSERTION if none is found
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the bb.utils.sha* utilities to hash files since they are much faster
than the loops we were rolling ourselves
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Licenses reported in the SPDX documents should be either:
A) A valid SPDX identifier cross referenced from the SPDX license
database
B) A "LicenseRef" to a license described in the SPDX document
The licensing code will now add a placeholder extracted license with
corresponding "LicenseRef" for any licenses that are not matched to the
SPDX database
Parenthesis in the license expression are now handled correctly
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This will create a more uniq DocumentRef, which will allow
the individual spdx files to be merged into a single SBOM
file reflecting the image. Do the same with the runtime dependencies
also
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes another creator that was missed earlier
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a debug source cannot be found, mark it as NOASSERTION so that other
tools at least know we were unable to locate it.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add the index to DEPLOYDIR in addition to adding it to the SPDX archive
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update the creator name since this no longer lives in meta-doubleopen
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds a class as a first attempt to create SPDX SBoM documents during the
build. This initial work was influenced by [meta-doubleopen][1],
although almost completely rewritten.
[1]: https://github.com/doubleopen-project/meta-doubleopen
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Joshua Watt
Michael Opdenacker
Keiya Nobuta
Peter Marko
Richard Purdie
Jose Quaresma
Mihai Lindner
Davide Gardenal
Peter Kjellerstedt
Andres Beltran
Saul Wold
Paul Eggleton
Ross Burton