| Age | Commit message (Collapse) | Author |
|---|
|
There are times when exluding or including a layer
may be desired. This provide the framwork for that via
two variables. The default is all layers in bblayers.
CVE_CHECK_LAYER_INCLUDELIST
CVE_CHECK_LAYER_EXCLUDELIST
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5fdde65ef58b4c1048839e4f9462b34bab36fc22)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
|
|
Lets include whcih layer a package belongs to and
add it to the cve logs
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 00d965bb42dc427749a4c3985af56ceffff80457)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
|
|
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.
refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.
Also update testcases to cover new changes.
(From OE-Core rev: 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 86b42289bda5bc2a4eff221ab476f170dd3d3794)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
|
|
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.
Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.
included selftest for the version class.
[YOCTO#14127]
(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3807c6d9a78ac8ade24c9c69cfe2b9624c49a20d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
|
|
The output currently shows the remapped product and version fields,
which may not be the actual recipe name/version. As this report is about
recipes, use the real values.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 18827d7f40db4a4f92680bd59ca655cca373ad65)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
|
|
patched CVEs
Default behavior is not changed. To suppress patched CVEs, set:
CVE_CHECK_REPORT_PATCHED = ""
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
per-recipe check file
The addition of this variable also makes it possible to change the
output suffix of the check files, e.g. in local.conf:
CVE_CHECK_MANIFEST_append = ".txt"
CVE_CHECK_RECIPE_FILE_append = ".txt"
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of inventing a new task to fetch the CVE data, use the existing
fetch task.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For example, if you just run 'bitbake cve-update-db-native' in a clean
build system, |cve_tmp_file| won't exist yet.
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously CVE_CHECK_DB_FILE / CVE_CHECK_DB_DIR was the same across
multiconfigs which led to a race condition wherein multiple
cve-update-db-native:do_populate_cve_db tasks could attempt to write to
the same sqlite database. This led to the following task failure:
Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_populate_cve_db(d)
0003:
File: '/mnt/data/agent/work/74f119cccb44f133/yocto/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 103, function: do_populate_cve_db
0099: if year == date.today().year:
0100: cve_f.write('CVE database update : %s\n\n' % date.today())
0101:
0102: cve_f.close()
*** 0103: conn.commit()
0104: conn.close()
0105:}
0106:
0107:def initialize_db(c):
Exception: sqlite3.OperationalError: disk I/O error
Use a lockfile to ensure multiple tasks don't step over each other.
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The cve-check file should be saved always, it has good info.
Put a copy in the log dir as cve-summary with symlinks to latest run.
[Yocto #13974]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the generated cve.log files, include the epoch in the product
version. This better matches how versions are displayed elsewhere,
in particular the bb.warn("Found unpatched CVE...") that appears
on the terminal when CVEs are found.
Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Certain recipes e.g. bash readline ( from meta-gplv2 ) download patches instead of having them in
metadata, this could fail cve_check
ERROR: readline-5.2-r9 do_cve_check: File Not found: qemuarm/build/../downloads/readline52-001
This patch ensures that download is done before running CVE scan, even
though these will be external patches and may not contain CVE tags as it
expects, but it will fix the run failures as seen above
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.
Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.
Non-valid CVE to be whitelisted separately.
[YOCTO #13617]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fix below error for whitelisted recipe and recipe skip cve check.
Error:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_cve_check(d)
0003:
File: '/poky-master/meta/classes/cve-check.bbclass', lineno: 59, function: do_cve_check
0055: try:
0056: patched_cves = get_patches_cves(d)
0057: except FileNotFoundError:
0058: bb.fatal("Failure in searching patches")
*** 0059: whitelisted, patched, unpatched = check_cves(d, patched_cves)
0060: if patched or unpatched:
0061: cve_data = get_cve_info(d, patched + unpatched)
0062: cve_write_data(d, patched, unpatched, whitelisted, cve_data)
0063: else:
Exception: ValueError: not enough values to unpack (expected 3, got 2)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
change whitelisted CVE status from "Patched" to "Whitelisted".
[Yocto #13687]
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With out these changes, a traceback displayed when a file
is listed in the SRC_URI but the file does not exist.
raise FileNotFoundError and print the patch then mark the task as failed.
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that
Due to changes required to support CVSS v3.1 scoring, the JSON
vulnerability feeds must be modified. This will require the consumers
of this data to update their internal processes. We will be providing
the JSON 1.1 schema on the data feeds page and the information below
to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At
that time the current JSON 1.0 data feeds will no longer available.
This change was tested briefly by issuing 'bitbake core-image-minimal'
with 'cve-check.bbclass' inherited via local.conf, and then comparing
the content between the resulting two
'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not
seem to contain any other change, except total of 167 entries like
CVSS v3 BASE SCORE: 0.0
were replaced with similar 'CVSS v3 BASE SCORE:' entries which had
scores that were greater than '0.0' (up to '9.8').
Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code used to construct a single SQL statement that fetched the NVD data for
every CVE requested. For recipes such as the kernel where there are over 2000
CVEs to report this can hit the variable count limit and the query fails with
"sqlite3.OperationalError: too many SQL variables". The default limit is 999
variables, but some distributions such as Debian set the default to 250000.
As the NVD table has an index on the ID column, whilst requesting the data
CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time
different is insignificant: 0.05s verses 0.01s on my machine.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Remove obsolete Python 2 code, and use convenience methods for neatness.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A previous optimisation was premature and resulted in false-negatives in the report.
Rewrite the checking algorithm to first get the list of potential CVEs by
vendor:product, then iterate through every matching CPE for that CVE to
determine if the bounds match or not. By doing this in two stages we can know
if we've checked every CPE, instead of accidentally breaking out of the scan too
early.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The patch scanner works with patch files in the layer, not in the workdir, so it
doesn't need to unpack.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
|
|
CVEs that are whitelisted or were not vulnerable when there are version
comparisons were not included in the report, so alter the logic to ensure that
all relevant CVEs are in the report for completeness.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
djb2 hash algorithm was found to do collisions, so the database was
sometime missing data. Remove this hash mechanism, clear and populate
elements from scratch in PRODUCTS table if the current year needs an
update.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CVE_CHECK_WHITELIST does not contain version anymore, as it was not
used. This variable should be set per recipe.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some product names are too vague to be searched without also matching the
vendor, for example Flex could be the parser compiler we ship, or Adobe Flex, or
Apache Flex, or IBM Flex.
If entries in CVE_PRODUCT contain a colon then split it as vendor:product to improve the search.
Also don't use .format() to construct SQL as that can lead to security
issues. Instead, use ? placeholders and lets sqlite3 handle the escaping.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes build failure with core-image-minimal:
Exception: UnboundLocalError: local variable 'to_append' referenced before assignment
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As glibc will be scanned for CVEs, we don't need to scan glibc-locale,
glibc-mtrace, and glibc-scripts which are all separate recipes for technical
reasons.
Exclude the recipes by setting CVE_PRODUCT in the recipe, instead of using the
global whitelist.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CVE-2014-2524 is a readline CVE that was fixed in 6.3patch3 onwards, but the
tooling wasn't able to detect this version. As we now ship readline 8 we don't
need to manually whitelist it, and if we did then the whitelisting should be in
the readline recipe.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that cve-update-db added CPE information to NVD database. We can
check for unpatched versions with operators '<', '<=', '>', and '>='.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_populate_cve_db is a native task.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the NVD url is not accessible, print a warning on top of the CVE
report, and continue. The database will not be fully updated, but
cve_check can still run on the previous database.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of generating a series of indexes via range(len(list)), just iterate the
list.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the NVD json CVE feed, affected versions can be strictly matched to a
version, but they can also be matched with the operator '<='.
Add a new condition in the sqlite query to match affected versions that
are defined with the operator '<='. Then use LooseVersion to discard all
versions that are not relevant.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In some rare cases (eg. curl recipe) the CVE_PRODUCT contains more than
one name.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use the new update-cve-db recipe to update database.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are both "curl" and "libcurl" CPEs in NVD.
All "curl" CVEs are currently missing in the reports.
Hence, switch "CVE_PRODUCT" to a space separated list.
It is useful for recipes generating several packages,
that have different product names in NVD.
Signed-off-by: Grygorii Tertychnyi <gtertych@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Disable downloading of the vulnerability DB in do_check_cves() task.
When invoked in this task, cve-check-tool attempts re-download of the CVE DB
if the latter is older than certain threshold. While reasonable for a
stand-alone CVE checker, this behavior can cause errors in parallel builds
if the build time is longer than this threshold:
* Other tasks might be using the DB.
* Several packages can start the download of the same file at the same time.
This check is not really needed, as the DB has been downloaded by
cve_check_tool:do_populate_cve_db() which is a prerequisite of any do_build().
The DB will be at most (threshold + build_time) old.
Signed-off-by: Konstantin Shemyak <konstantin.shemyak@ge.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some backported patches fix multiple CVEs and list the corresponding
identifiers on multiple lines, rather than on a single line.
cve-check.bbclass yields false positive warnings when CVE IDs are
presented on multiple lines because re.search() returns only
the first match.
An example of this behavior may be found when running do_cve_check() on
the wpa-supplicant recipe while in the rocko branch. Only CVE-2017-13077
is reported to be patched by commit de57fd8, despite the patch including
fixes for a total of 9 CVEs.
This is resolved by iterating over all regular expression matches,
rather than just the first.
Signed-off-by: Jon Szymaniak <jon.szymaniak.foss@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The binary 'cve-check-update' downloads the CVE database from the Internet.
If the system is behind a web proxy, the download fails, as proxy-related
variables are not exported.
In turn, 'cve-check-tool' does not connect to the network and correspondingly
does not need exported proxies.
Exported all proxy-related environment variables to 'cve-check-update' and
removed the unneeded export from 'cve-check-tool'.
Signed-off-by: Konstantin Shemyak <konstantin.shemyak@ge.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
For some recipes is is meaningless to do a CVE check, for example packagegroups
or images. Check that CVE_PRODUCT is set and short-circuit the scan if it
isn't.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
For reasons which I don't understand, the Berkeley DB tarball is version 5.3.28
but in CVE reports the version is 11.2.5.3.28.
To handle this allow recipes to override their version as well as their name.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This is where the other task logs go, so it's a sensible place to put it.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
While poky master branch has been fixed so that all CVE patch files have
the:
CVE: CVE-2017-1234556
strings in the patch comments, many older versions of poky and other meta
layers are not, but the CVE patches quite often have the CVE id in the
patch file name.
If the CVE: string also found, there are no duplicates in the report.
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
PV is the recipe major version number. cve-check tries to map that to
NVD database release versions of the component. If the recipe sources
are taken from git, the PV can be automatically modified to include
git details, but the syntax is like 233+gitAUTOINC+a1e2ef7ec9.
In CVE checks we want to remove the git details and just use the major
version information, in this example 233.
Thus use "+git" as the separator and use the first part before the separator
as SW product version number in CVE check.
Fixes version number for e.g. systemd recipe. If systemd PV is
233+gitAUTOINC+a1e2ef7ec9 there will be no matches from CVE database where
latest release mentioned is plain 233. If the filter is set to +git, then
CVE PV is 233 and issues like this are detected by do_cve_check:
https://nvd.nist.gov/vuln/detail/CVE-2017-1000082
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This way also bbclasses can override it. For example kernel.bbclass
could set CVE_PRODUCT to linux_kernel for all users of the class
which compile Linux kernels.
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since do_rootfs depends on cve-check results of all recipes,
we need to recursively depend on recipe do_cve_check.
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Sometimes there are control messages in cve-check-tool printed to
stderr. These lead to parsing error and thus failed build.
This can happen for instance when cve database needs to be
refreshed during build.
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If there is cve report for a recipe in previous build and there
is no result for current one, old cves are kept in CVE_CHECK_DIR.
This happens on version upgrade or when cve/recipe is whitelisted.
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
akuster
Lee Chee Yang
Ross Burton
Chris Laplante
Ralph Siemsen
Khem Raj
Niko Mauno
Ross Burton
Pierre Le Magourou
Mikko Rapeli
Grygorii Tertychnyi
Konstantin Shemyak
Jon Szymaniak
Peter Marko