add upstr. source tracing functionality in unpack

do_unpack currently unpacks all SRC_URI entries into WORKDIR, and can
even mix files coming from multiple SRC_URI entries into the same
subdir, making it hard to trace each source file found in WORKDIR back
to its corresponding upstream source.

Being able to trace source files to their corresponding upstream source
is fundamental for Software Composition Analysis (SCA), Software Bill
of Materials (SBoM) generation (create-spdx.bbclass), license compliance
checking and CVE checking.

To solve this issue, this patch implements a process that consists of:

1) unpacking each SRC_URI element into a temporary directory
2) collecting relevant metadata for Software Composition Analysis (file
   sha1, upstream download location (in SPDX-compliant format), relative
   path in the upstream repo/package, etc.);
3) moving everything to WORKDIR, and iterate with the next SRC_URI
   element;
4) saving metadata in a json file after all SRC_URI elements have been
   processed

By patching the relevant fetcher modules and adding a bb.trace module,
this patch implements steps 1,3,4 , while it provides only a bare-bone
implementation of step 2, in which all relevant raw metadata (file
paths, url, urldata, real destination dir, npmsw dependency tree, git
submodule revisions) are collected, but not processed nor saved.

This should allow to develop a full implementation of step 2 (data
collection) in a separate module independently from the development
of the rest of bb code, i.e. without the need of further patching bb
fetchers.

Signed-off-by: Alberto Pianon <alberto@pianon.eu>

1 files changed, 1 insertions, 0 deletions

diff --git a/bin/bitbake-selftest b/bin/bitbake-selftest
index f25f23b1a..7be354f9e 100755
--- a/bin/bitbake-selftest
+++ b/bin/bitbake-selftest
@@ -26,6 +26,7 @@ tests = ["bb.tests.codeparser",
          "bb.tests.data",
          "bb.tests.event",
          "bb.tests.fetch",
+         "bb.tests.trace_base",
          "bb.tests.parse",
          "bb.tests.persist_data",
          "bb.tests.runqueue",