summaryrefslogtreecommitdiffstats
path: root/meta/recipes-core/meta
AgeCommit message (Collapse)Author
2021-04-27cve-update-db-native: skip on empty cpe23UriKonrad Weihmann
Recently an entry in the NVD DB appeared that looks like that {'vulnerable': True, 'cpe_name': []}. As besides all the vulnerable flag no data is present we would get a KeyError exception on acccess. Use get method on dictionary and return if no meta data is present Also quit if the length of the array after splitting is less than 6 Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 00ce2796d97de2bc376b038d0ea7969088791d34) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2021-03-24cve-update-db-native: consider version suffix when update CVE dbLee Chee Yang
some record from NVD can merge or split suffix from version, for example: CVE-2017-15906 "cpe23Uri" : "cpe:2.3:a:openbsd:openssh:5.0:p1:*:*:*:*:*:*" "cpe23Uri" : "cpe:2.3:a:openbsd:openssh:4.7p1:*:*:*:*:*:*:*" in such case include the suffix into version when update local CVE db. Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 13cc68197f81bb7c76fa1abecc5dd720b8bdb8d5) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2021-03-04buildtools-extended-tarball: Add glibc-gconvs needed for buildRichard Purdie
When building vim it tries to rebuild files using iconv. If this fails the build continues anyway but the output is not determnistic as builds using a hosttools tarball are different from builds where there isn't a hosttools tarball. Add the needed gconvs to the tarball when iconv is present to become determistic and generate vim locales consistently. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b945652a088f430a2adec6b968cd00c5928d4272) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2021-01-04buildtools-tarball.bb: Fix PATH for environment setup scriptRobert Yang
It only added ${SDKPATHNATIVE}/usr/bin to PATH which didn't work when files were installed to other bin dirs such as /bin or /sbin, for example, nativesdk-pigz installs the files to /bin, now fix it to keep align with sdk's PATH. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 67cac575a5696af5bad1aab888b65ea2686adff7) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-12-11buildtools-tarball: add wic dependency into extended buildtoolsChangqing Li
fix below error: wic ls ./core-image-minimal-intel-x86-64.wic ERROR: Can't find executable parted wic depend on some tools like parted/mtools/..., and we have those tools in native_sysroot. so above problem can be avoided by run command like wic ls imagename.wic --native-sysroot <path> but this cannot cover condition that usr don't have build the image, the image just copy from somewhere. Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c0eda6ec09395f3a04cb80107a3ca33a063b21f8) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-12-10cve-update-db-native: handle all-wildcard versionsRoss Burton
If a CPE version field is just *:*:*:* it should be handled the same as -:*:*:*, that is 'all versions'. To ease handling, transform this case to use -. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 04a9bc4ca5294fe6834513669c7746a824d12b04) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-30buildtools-tarball: Fix conflicts with oe-selftest and other toolingRichard Purdie
OECORE_NATIVE_SYSROOT is used by tools like oe-run-native and hence we were seeing selftest failures when newer buildtools-tarballs that use this were run on the autobuilder. Unset the variable after use to avoid these issues. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ea5128fde10eb9be7fefbbb87ae6181edd2bcdff) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-30buildtools: Handle generic environment setup injectionRichard Purdie
Currently buildtools has its own hacked environment setup. We added capability for generic script fragements but this code was never updated to use it even if several components it contains needs it. Remove a hardcoded variable in favour of the generic scripts to avoid bugs in this area in future. The others should probably be migrated to environment files in future too but there isn't such a file for those at present. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit eb210cee4bd71614dd108027e6d5aa8f8d451b27) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-28cve-update-db-native: remove unused variableChris Laplante
Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 790f8a5bfbd1851226fa5fc5b09e79f6396e747d) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-28cve-update-db-native: move -journal checking into do_fetchChris Laplante
It was always questionable to do this in an anonymous function, but now with multiconfig it is a critical mistake and leads to more strange "Exception: sqlite3.OperationalError: disk I/O error" errors. Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fcbc9d0e8421a3a559db05005ba042337cd5d864) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-28cve-update-db-native: be less magical about checking whether the cve-check ↵Chris Laplante
class is enabled Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit cbc717350b21fb417fa6f83621022ac633a26124) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-28buildtools-tarball: Add python3-jinja2Richard Purdie
This is used by various pieces of the result handling code, particularly the performance testing and we're seeing autobuilder failures that are easiest resolved using this. Acked-by: Tim Orling <ticotimo@gmail.com> Signed-off-by: Tim Orling <ticotimo@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit dc6d4d438b71365f5c6f54b8482d4f01f1d0dc8e) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-14cve-update-db-native: use context manager for cve_fChris Laplante
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ff422652e1b5db62205fafc75ce56bb5951d478d) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-14cve-check/cve-update-db-native: use lockfile to fix usage under multiconfigChris Laplante
Previously CVE_CHECK_DB_FILE / CVE_CHECK_DB_DIR was the same across multiconfigs which led to a race condition wherein multiple cve-update-db-native:do_populate_cve_db tasks could attempt to write to the same sqlite database. This led to the following task failure: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_populate_cve_db(d) 0003: File: '/mnt/data/agent/work/74f119cccb44f133/yocto/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 103, function: do_populate_cve_db 0099: if year == date.today().year: 0100: cve_f.write('CVE database update : %s\n\n' % date.today()) 0101: 0102: cve_f.close() *** 0103: conn.commit() 0104: conn.close() 0105:} 0106: 0107:def initialize_db(c): Exception: sqlite3.OperationalError: disk I/O error Use a lockfile to ensure multiple tasks don't step over each other. Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 24e9380643a2ae3fcae193519cb64aedaf682153) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-09-14cve-update-db-native: add progress handlerChris Laplante
Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 79ae2e82b8ec11578177f428060b568d6c7d44ca) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-07-27cve-update: handle baseMetricV2 as optionalKonrad Weihmann
Currently in NVD DB an item popped up, which hasn't set baseMetricV2. Let the parser handle it as an optional item. In case use baseMetricV2 before baseMetricV3 Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit fdcbf3f28289188c5a97664d1421d4a5c4991eda) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-06-18buildtools-tarball: export OPENSSL_CONF in environment setupSteve Sakoman
The autobuilder has been experiencing SSL: CERTIFICATE_VERIFY_FAILED errors during error report uploads when using buildtools due to looking for certs in /opt/poky Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 197f1d5d14b8e57295f5a81c03c86abba5328614) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-06-16build-sysroots: add sysroot paths with native binaries to PATHAlexander Kanavin
staging_populate_sysroot_dir() collects postinsts from the sysroot and executes them. These postinsts, in turn, may call binaries that are only available from the sysroot. This works fine with recipe-specific sysroots, as all necessary paths are already in PATH, but breaks down in this recipe which imitates the old global sysroot way but doesn't adjust the PATH to include the binary paths from global sysroot. To reproduce the failure: $ bitbake docbook-xml-dtd4-native $ bitbake -c build_native_sysroot build-sysroots ... Exception: subprocess.CalledProcessError: Command '/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127. Subprocess output: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 5: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 8: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 6b5f7bda4204d45cd29670cefcd53dc5da031095) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-05-01buildtools-extended-tarball: add libgomp-devAlexander Kanavin
This is needed in particular for newer versions of rpm which would otherwise fail to build due to absence of omp.h header. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-04-30buildtools-tarball: add nativesdk-mtools for `wic ls'hongxu
On ubuntu 18.04.1, it does not provides `mdir' by default which caused `wic ls **.wic' failed on fat partition ... $ wic ls build/tmp-glibc/deploy/images/xilinx-zynqmp/wrlinux-image-std-xilinx-zynqmp.wic ERROR: Can't find executable 'mdir' ... Add nativesdk-mtools to buildtools-tarball and use buildtools to provide mdir Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-04-30buildtools-extended-tarball: Add libstc++.aJeremy Puhlman
Builds like native-openjdk, really wants a to link some tools against the static version. Since when using the extended tarball, its the only place to get it, add the library. Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2020-04-02cve-update-db-native: fix DB file version for '-'Lee Chee Yang
fix logic for CVE DB update so that when the CPE version is '-', it keeps the version as '-' in the DB file too and leave other operation as blank. Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-31cve-check: CPE version '-' as all versionLee Chee Yang
CPE version could be '-' to mean no version info. Current cve_check treat it as not valid and does not report these CVE but some of these could be a valid vulnerabilities. Since non-valid CVE can be whitelisted, so treat '-' as all version and report all these CVE to capture possible vulnerabilities. Non-valid CVE to be whitelisted separately. [YOCTO #13617] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29cve-update-db-native: clean DB if temporary file existLee Chee Yang
when do_populate_cve_db forced stop at certain point, the DB execution are stoped however the temporary database file (DB-JOURNAL) are not removed. This db-journal file indicates that DB is incomplete and set DB in readonly mode. So when db-journal exist, remove both DB and the db-journal and build the DB again from scratch. [YOCTO #13682] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-25cve-update-db-native: prevent fatal error cause by networkLee Chee Yang
prevent cve-check from fatal error cause by network issue. [YOCTO #13680] Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-11buildtools-extended-tarball: Add locale commandRichard Purdie
The eSDK installation code checks installed locales with the locale command which is from glibc-utils. Add this so that we find the correct locales from the buildtools. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06buildtools-extended-tarball: add nativesdk-libxcrypt-devJeremy Puhlman
virtual/crypt-native is assume provided in bitbake.conf, so buildtools-extended-tarball shoud provide crypt since it doesn't use the host's headers/libraries. [YOCTO #13714] Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15image.bbclass, package-index.bb: Delete do_populate_lic taskKhem Raj
These are meta packages Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06patch: Extend to native/nativesdk and depend uponRichard Purdie
There is a bug in patch 2.7.3 and earlier where index lines in patches can change file modes when they shouldn't: http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4 This leaks into debug sources in particular (e.g. tcp-wrappers where source files are read-only). Add the dependency to target recipes to avoid this problem until we can rely on 2.7.4 or later. We could try and remove all index lines from patch files but it will be a losing battle. We could try and identify all the recipes which change modes on files in patches but again, its a losing battle. Instead, compromise and have patch-native as a dependency for target recipes. We use patch-replacement-native since patch-native is in ASSUME_PROVIDED. Also add nativesdk-patch to buildtools-tarball. [YOCTO #13777] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22buildtools-tarball: Add an ld.so.conf for nativesdk-binutilsRichard Purdie
We need to search our own libdirs, then fall back to the system ones as our customised dynamic loader will. Have ld.so.conf reflect that. This ensures that binutils finds libraries here when linking too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11meta-extsdk-toolchain: Mark as nostampRichard Purdie
This recipe needs to generate the latest hashes. With unihash we may not rebuild as previous dependencies may be marked as equivalent. eSDK which uses this is nostamp already, mark this task as nostamp also for the same reason. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10dummy-sdk-package.inc: do multilib expanesion for RREPLACES as wellAlexander Kanavin
Otherwise, multilib packages such as lib64-perl aren't properly replaced: Error: Problem: package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl provided by lib64-perl-5.30.1-r0.x86_64 - package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-strict provided by lib64-perl-5.30.1-r0.x86_64 - package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-warnings provided by lib64-perl-5.30.1-r0.x86_64 - package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-vars provided by lib64-perl-5.30.1-r0.x86_64 Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-01buildtools-extended-tarball: add recipe with build-essentialsTim Orling
* For some aging distros, such as CentOS 7, the native version of gcc is simply too ancient and is a constant source of headaches for moving forward. * Add an extended version of buildtools-tarball which adds all of build-essential, so that the host is now modernized and capable of compiling the latest versions of components. Fixes [YOCTO #13714] Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30nativesdk-buildtools-perl-dummy: add dependencies for autoconf and automakeTim Orling
* For buildtools-extended-tarball, where we are adding all of build-essentials to the nativesdk, we need additional perl modules for autoconf and automake. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-09cve-check: Switch to NVD CVE JSON feed version 1.1Niko Mauno
Switch to recently released version 1.1 of NVD CVE JSON feed, as in https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release it is mentioned that Due to changes required to support CVSS v3.1 scoring, the JSON vulnerability feeds must be modified. This will require the consumers of this data to update their internal processes. We will be providing the JSON 1.1 schema on the data feeds page and the information below to prepare for this transition. ... The JSON 1.1 data feeds will be available on September 9th, 2019. At that time the current JSON 1.0 data feeds will no longer available. This change was tested briefly by issuing 'bitbake core-image-minimal' with 'cve-check.bbclass' inherited via local.conf, and then comparing the content between the resulting two 'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not seem to contain any other change, except total of 167 entries like CVSS v3 BASE SCORE: 0.0 were replaced with similar 'CVSS v3 BASE SCORE:' entries which had scores that were greater than '0.0' (up to '9.8'). Signed-off-by: Niko Mauno <niko.mauno@iki.fi> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-28dummy-sdk-package.inc: do not include files into RREPLACESAlexander Kanavin
rpm 4.15 no longer allows it, which makes sense. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-28dummy-sdk-package: Add DUMMYPROVIDES_PACKAGESRichard Purdie
We're about to need to use this variable in the main include file so restructure the users of it to all set it appropriately. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: clean up proxy handlingRoss Burton
urllib handles adding proxy handlers if the proxies are set in the environment, so call bb.utils.export_proxies() to do that and remove the manual setup. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: add an index on the CVE ID columnRoss Burton
Create an index on the PRODUCTS table which contains a row for each CPE, drastically increasing the performance of lookups for a specific CVE. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21cve-update-db-native: don't hardcode the database nameRoss Burton
Don't hardcode the database filename, there's a variable for this in cve-check.bbclass. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14cve-update-db-native: don't refresh more than once an hourRoss Burton
We already fetch the yearly CVE metadata and check that for updates before downloading the full data, but we can speed up CVE checking further by only checking the CVE metadata once an hour. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14buildtools-tarball: export OPENSSL_CONF for opensslLiwei Song
export OPENSSL_CONF to aviod SDK openssl can not find openssl.cnf. Signed-off-by: Liwei Song <liwei.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27Revert "meta-extsdk: Either an sstate task is a proper task or it isn't"Richard Purdie
This reverts commit db5f08c16eff01f3af0b99119c09f68628e2331d. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19meta-extsdk: Either an sstate task is a proper task or it isn'tRichard Purdie
Ensure the task is properly regsistered as an sstate task as this "half way" state confuses new code in bitbake and it isn't supported. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-16target-sdk-provides-dummy: extend packages for multilib caseChen Qi
If we have installed some lib32 package which depends on perl/bash, then populating sdk for that image would fail with the following error. Error: Problem: package lib32-libxml-namespacesupport-perl-1.12-r0.corei7_32 requires lib32-perl, but none of the providers can be installed package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with /usr/bin/perl provided by lib32-perl-5.30.0-r0.corei7_32 package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with libperl.so.5 provided by lib32-perl-5.30.0-r0.corei7_32 This could be produced by the following steps: 1. IMAGE_INSTALL_append = " lib32-valgrind" 2. bitbake core-image-minimal -c populate_sdk We need to extend all packages in DUMMYPROVIDES to avoid such problems. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-26cve-update-db-native: fix https proxy issuesChin Huat Ang
When https_proxy is set, use proxy opener to open CVE metadata and database URLs, otherwise fallback to the urllib.request.urlopen. Also fix a minor issue where the json database which has been gzip decompressed as byte object should be decoded as utf-8 string as expected by update_db. Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: clean up JSON fetchingRoss Burton
Currently the code fetches the compressed JSON, writes it to a temporary file, uncompresses that with gzip and passes the fake file object to update_db(). Instead, uncompress the gzip'd data in memory and pass the JSON directly to update_db(). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: improve metadata parsingRoss Burton
The metadata parser is fragile: first it coerces a bytes() to a str() (so the string is b'LastModifiedDate:2019...'), assumes the first line is the date, and then uses a regex to parse (which then includes the trailing quote as part of the date). Clean this up by parsing the bytes as UTF-8 (ASCII is probably fine, but this is safer), iterate through the lines and split on colons to find the right key/value pair. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-20cve-update-db-native: use executemany() to optimise CPE insertionRoss Burton
Instead of calling execute() repeatedly, rewrite the function to be a generator and use executemany() for performance. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-19cve-update-db: actually inherit nativeRoss Burton
The recipe was called -native but didn't inherit native. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-25 05:13:20 +0000