summaryrefslogtreecommitdiffstats
path: root/meta/recipes-core/meta/cve-update-db-native.bb
AgeCommit message (Collapse)Author
2020-11-29cve-update-db-native: handle all-wildcard versionsRoss Burton
If a CPE version field is just *:*:*:* it should be handled the same as -:*:*:*, that is 'all versions'. To ease handling, transform this case to use -. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-14cve-update-db-native: remove unused variableChris Laplante
Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-14cve-update-db-native: move -journal checking into do_fetchChris Laplante
It was always questionable to do this in an anonymous function, but now with multiconfig it is a critical mistake and leads to more strange "Exception: sqlite3.OperationalError: disk I/O error" errors. Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-14cve-update-db-native: be less magical about checking whether the cve-check ↵Chris Laplante
class is enabled Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-11cve-update-db-native: use fetch taskRoss Burton
Instead of inventing a new task to fetch the CVE data, use the existing fetch task. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-11cve-update-db-native: add more logging when fetchingRoss Burton
Add some debug logging when fetching the CVE data. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-11cve-update-db-native: remove unused importRoss Burton
This import isn't used anymore, so remove it. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-09cve-update-db-native: use context manager for cve_fChris Laplante
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-09cve-check/cve-update-db-native: use lockfile to fix usage under multiconfigChris Laplante
Previously CVE_CHECK_DB_FILE / CVE_CHECK_DB_DIR was the same across multiconfigs which led to a race condition wherein multiple cve-update-db-native:do_populate_cve_db tasks could attempt to write to the same sqlite database. This led to the following task failure: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_populate_cve_db(d) 0003: File: '/mnt/data/agent/work/74f119cccb44f133/yocto/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 103, function: do_populate_cve_db 0099: if year == date.today().year: 0100: cve_f.write('CVE database update : %s\n\n' % date.today()) 0101: 0102: cve_f.close() *** 0103: conn.commit() 0104: conn.close() 0105:} 0106: 0107:def initialize_db(c): Exception: sqlite3.OperationalError: disk I/O error Use a lockfile to ensure multiple tasks don't step over each other. Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-09cve-update-db-native: add progress handlerChris Laplante
Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27cve-update: handle baseMetricV2 as optionalKonrad Weihmann
Currently in NVD DB an item popped up, which hasn't set baseMetricV2. Let the parser handle it as an optional item. In case use baseMetricV2 before baseMetricV3 Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-02cve-update-db-native: fix DB file version for '-'Lee Chee Yang
fix logic for CVE DB update so that when the CPE version is '-', it keeps the version as '-' in the DB file too and leave other operation as blank. Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-31cve-check: CPE version '-' as all versionLee Chee Yang
CPE version could be '-' to mean no version info. Current cve_check treat it as not valid and does not report these CVE but some of these could be a valid vulnerabilities. Since non-valid CVE can be whitelisted, so treat '-' as all version and report all these CVE to capture possible vulnerabilities. Non-valid CVE to be whitelisted separately. [YOCTO #13617] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29cve-update-db-native: clean DB if temporary file existLee Chee Yang
when do_populate_cve_db forced stop at certain point, the DB execution are stoped however the temporary database file (DB-JOURNAL) are not removed. This db-journal file indicates that DB is incomplete and set DB in readonly mode. So when db-journal exist, remove both DB and the db-journal and build the DB again from scratch. [YOCTO #13682] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-25cve-update-db-native: prevent fatal error cause by networkLee Chee Yang
prevent cve-check from fatal error cause by network issue. [YOCTO #13680] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-09cve-check: Switch to NVD CVE JSON feed version 1.1Niko Mauno
Switch to recently released version 1.1 of NVD CVE JSON feed, as in https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release it is mentioned that Due to changes required to support CVSS v3.1 scoring, the JSON vulnerability feeds must be modified. This will require the consumers of this data to update their internal processes. We will be providing the JSON 1.1 schema on the data feeds page and the information below to prepare for this transition. ... The JSON 1.1 data feeds will be available on September 9th, 2019. At that time the current JSON 1.0 data feeds will no longer available. This change was tested briefly by issuing 'bitbake core-image-minimal' with 'cve-check.bbclass' inherited via local.conf, and then comparing the content between the resulting two 'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not seem to contain any other change, except total of 167 entries like CVSS v3 BASE SCORE: 0.0 were replaced with similar 'CVSS v3 BASE SCORE:' entries which had scores that were greater than '0.0' (up to '9.8'). Signed-off-by: Niko Mauno <niko.mauno@iki.fi> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: clean up proxy handlingRoss Burton
urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: add an index on the CVE ID columnRoss Burton
Create an index on the PRODUCTS table which contains a row for each CPE, drastically increasing the performance of lookups for a specific CVE. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: don't hardcode the database nameRoss Burton
Don't hardcode the database filename, there's a variable for this in cve-check.bbclass. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14cve-update-db-native: don't refresh more than once an hourRoss Burton
We already fetch the yearly CVE metadata and check that for updates before downloading the full data, but we can speed up CVE checking further by only checking the CVE metadata once an hour. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-26cve-update-db-native: fix https proxy issuesChin Huat Ang
When https_proxy is set, use proxy opener to open CVE metadata and database URLs, otherwise fallback to the urllib.request.urlopen. Also fix a minor issue where the json database which has been gzip decompressed as byte object should be decoded as utf-8 string as expected by update_db. Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: clean up JSON fetchingRoss Burton
Currently the code fetches the compressed JSON, writes it to a temporary file, uncompresses that with gzip and passes the fake file object to update_db(). Instead, uncompress the gzip'd data in memory and pass the JSON directly to update_db(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: improve metadata parsingRoss Burton
The metadata parser is fragile: first it coerces a bytes() to a str() (so the string is b'LastModifiedDate:2019...'), assumes the first line is the date, and then uses a regex to parse (which then includes the trailing quote as part of the date). Clean this up by parsing the bytes as UTF-8 (ASCII is probably fine, but this is safer), iterate through the lines and split on colons to find the right key/value pair. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: use executemany() to optimise CPE insertionRoss Burton
Instead of calling execute() repeatedly, rewrite the function to be a generator and use executemany() for performance. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-19cve-update-db: actually inherit nativeRoss Burton
The recipe was called -native but didn't inherit native. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-19cve-update-db-native: use os.path.join instead of +Ross Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18cve-update-db-native: Remove hash column from database.Pierre Le Magourou
djb2 hash algorithm was found to do collisions, so the database was sometime missing data. Remove this hash mechanism, clear and populate elements from scratch in PRODUCTS table if the current year needs an update. Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-18cve-update-db-native: use SQL placeholders instead of format stringsRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-10cve-update-db-native: Skip recipe when cve-check class is not loaded.Pierre Le Magourou
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-09cve-update-db: Use NVD CPE data to populate PRODUCTS tablePierre Le Magourou
Instead of using expanded list of affected versions that is not reliable, use the 'cpe_match' node in the 'configurations' json node. For cve-check to correctly match affected CVE, the sqlite database need to contain operator_start, operator_end and the corresponding versions fields. Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-09cve-check: Depends on cve-update-db-nativePierre Le Magourou
do_populate_cve_db is a native task. Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-21 05:57:45 +0000