Age | Commit message (Collapse) | Author |
|
do_deploy should clean up ${DEPLOYDIR} before running, just like do_install
cleans up ${D} before running. This reduces the risk of DEPLOYDIR being
accidentally contaminated by files from previous runs, possibly even with
different config, in case of incremental builds.
It is convenient to have this in deploy.bbclass, so it doesn't have to be
duplicated in every recipe, considering for example meta-freescale, which
has 23 affected recipes.
All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky,
meta-openembedded and meta-freescale look like they either benefit from
this or are at least not affected negatively by it. The only exception
I've noticed was uboot-sign.bbclass, which was however fixed by the
previous patch.
Signed-off-by: Daniel Klauer <daniel.klauer@gin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The native version of grub-efi only installs the tools
/usr/bin/grub-editenv
/usr/bin/grub-mkimage
to sysroots-components/, but equivalent tools are already provided by
grub-native, the difference on x86_64 being 4 hardwired paths in grub-mkimage
(values taken from grub-native):
LOCALEDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/share/locale
GRUB_DATADIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/share
GRUB_LIBDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/lib
GRUB_SYSCONFDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/etc
If grub-native and grub-efi-native are built with the following patch
--- grub-2.04.orig/configure.ac
+++ grub-2.04/configure.ac
@@ -1980,10 +1980,10 @@ grub_libdir="$(eval echo "$libdir")"
grub_localedir="$(eval echo "$localedir")"
grub_datadir="$(eval echo "$datadir")"
grub_sysconfdir="$(eval echo "$sysconfdir")"
-AC_DEFINE_UNQUOTED(LOCALEDIR, "$grub_localedir", [Locale dir])
-AC_DEFINE_UNQUOTED(GRUB_LIBDIR, "$grub_libdir", [Library dir])
-AC_DEFINE_UNQUOTED(GRUB_DATADIR, "$grub_datadir", [Data dir])
-AC_DEFINE_UNQUOTED(GRUB_SYSCONFDIR, "$grub_sysconfdir", [Configuration dir])
+AC_DEFINE_UNQUOTED(LOCALEDIR, "/non-existent", [Locale dir])
+AC_DEFINE_UNQUOTED(GRUB_LIBDIR, "/non-existent", [Library dir])
+AC_DEFINE_UNQUOTED(GRUB_DATADIR, "/non-existent", [Data dir])
+AC_DEFINE_UNQUOTED(GRUB_SYSCONFDIR, "/non-existent", [Configuration dir])
the produced grub-editenv/grub-mkimage binaries become binary equivalent,
assuming reproducible builds is active. Since the unpatched values of
LOCALEDIR/GRUB_DATADIR/GRUB_LIBDIR/GRUB_SYSCONFDIR point to directories that
are not expected to exist at runtime, they can be ignored.
Therefore:
* remove grub-efi-native and instead rely on the same tools from
grub-native
* replace references to grub-efi-native with grub-native
* remove unused grub-efi-native security flags overrides
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The autobuilder has been experiencing SSL: CERTIFICATE_VERIFY_FAILED
errors during error report uploads when using buildtools due to looking
for certs in /opt/poky
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
staging_populate_sysroot_dir() collects postinsts from the sysroot
and executes them. These postinsts, in turn, may call binaries that
are only available from the sysroot. This works fine with recipe-specific
sysroots, as all necessary paths are already in PATH, but breaks down
in this recipe which imitates the old global sysroot way but doesn't adjust
the PATH to include the binary paths from global sysroot.
To reproduce the failure:
$ bitbake docbook-xml-dtd4-native
$ bitbake -c build_native_sysroot build-sysroots
...
Exception: subprocess.CalledProcessError: Command '/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
Subprocess output:
/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 5: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found
/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 8: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Extend the functionality provided by commit [1] to the SDK as well. This way we
can make sure that nativesdk-binutils finds SDK libraries first rather than
host ones.
This is useful for example when trying to build the linux kernel using
nativesdk-gcc. This scenario currently fails because it tries to link to host
libraries rather than SDK host ones:
make x86_64_defconfig
make bzImage
...
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile:1101: recipe for target 'prepare-objtool' failed
make: *** [prepare-objtool] Error 1
....
/../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
...
[1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is needed in particular for newer versions of rpm
which would otherwise fail to build due to absence of omp.h header.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow specifying an optional destination to include-path and make the
option aware of permissions and owners.
It is very useful for making a partition that contains the rootfs for a
host and a target Eg:
/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)
Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.
With this patch we can use something like:
part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello
on the .wks file.
Cc: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On ubuntu 18.04.1, it does not provides `mdir' by default
which caused `wic ls **.wic' failed on fat partition
...
$ wic ls build/tmp-glibc/deploy/images/xilinx-zynqmp/wrlinux-image-std-xilinx-zynqmp.wic
ERROR: Can't find executable 'mdir'
...
Add nativesdk-mtools to buildtools-tarball and use buildtools
to provide mdir
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Builds like native-openjdk, really wants a to link
some tools against the static version. Since when
using the extended tarball, its the only place to
get it, add the library.
Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fix logic for CVE DB update so that when the CPE version is '-',
it keeps the version as '-' in the DB file too and leave other
operation as blank.
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.
Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.
Non-valid CVE to be whitelisted separately.
[YOCTO #13617]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
when do_populate_cve_db forced stop at certain point, the
DB execution are stoped however the temporary database
file (DB-JOURNAL) are not removed. This db-journal file
indicates that DB is incomplete and set DB in readonly
mode. So when db-journal exist, remove both DB and the
db-journal and build the DB again from scratch.
[YOCTO #13682]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
prevent cve-check from fatal error cause by network issue.
[YOCTO #13680]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The eSDK installation code checks installed locales with the locale command which is
from glibc-utils. Add this so that we find the correct locales from the buildtools.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
virtual/crypt-native is assume provided in bitbake.conf, so
buildtools-extended-tarball shoud provide crypt since it doesn't
use the host's headers/libraries.
[YOCTO #13714]
Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These are meta packages
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a bug in patch 2.7.3 and earlier where index lines
in patches can change file modes when they shouldn't:
http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4
This leaks into debug sources in particular (e.g. tcp-wrappers where
source files are read-only). Add the dependency to target recipes
to avoid this problem until we can rely on 2.7.4 or later.
We could try and remove all index lines from patch files but it will be a
losing battle. We could try and identify all the recipes which change
modes on files in patches but again, its a losing battle.
Instead, compromise and have patch-native as a dependency
for target recipes. We use patch-replacement-native since patch-native
is in ASSUME_PROVIDED.
Also add nativesdk-patch to buildtools-tarball.
[YOCTO #13777]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We need to search our own libdirs, then fall back to the system ones as our
customised dynamic loader will. Have ld.so.conf reflect that.
This ensures that binutils finds libraries here when linking too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This recipe needs to generate the latest hashes. With unihash we may not
rebuild as previous dependencies may be marked as equivalent.
eSDK which uses this is nostamp already, mark this task as nostamp also
for the same reason.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Otherwise, multilib packages such as lib64-perl aren't properly replaced:
Error:
Problem: package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl provided by lib64-perl-5.30.1-r0.x86_64
- package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-strict provided by lib64-perl-5.30.1-r0.x86_64
- package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-warnings provided by lib64-perl-5.30.1-r0.x86_64
- package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with lib64-perl-module-vars provided by lib64-perl-5.30.1-r0.x86_64
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* For some aging distros, such as CentOS 7, the native version
of gcc is simply too ancient and is a constant source of
headaches for moving forward.
* Add an extended version of buildtools-tarball which adds all
of build-essential, so that the host is now modernized and
capable of compiling the latest versions of components.
Fixes [YOCTO #13714]
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* For buildtools-extended-tarball, where we are adding all of build-essentials
to the nativesdk, we need additional perl modules for autoconf and automake.
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that
Due to changes required to support CVSS v3.1 scoring, the JSON
vulnerability feeds must be modified. This will require the consumers
of this data to update their internal processes. We will be providing
the JSON 1.1 schema on the data feeds page and the information below
to prepare for this transition.
...
The JSON 1.1 data feeds will be available on September 9th, 2019. At
that time the current JSON 1.0 data feeds will no longer available.
This change was tested briefly by issuing 'bitbake core-image-minimal'
with 'cve-check.bbclass' inherited via local.conf, and then comparing
the content between the resulting two
'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not
seem to contain any other change, except total of 167 entries like
CVSS v3 BASE SCORE: 0.0
were replaced with similar 'CVSS v3 BASE SCORE:' entries which had
scores that were greater than '0.0' (up to '9.8').
Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
rpm 4.15 no longer allows it, which makes sense.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're about to need to use this variable in the main include file so
restructure the users of it to all set it appropriately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
urllib handles adding proxy handlers if the proxies are set in the environment,
so call bb.utils.export_proxies() to do that and remove the manual setup.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create an index on the PRODUCTS table which contains a row for each CPE,
drastically increasing the performance of lookups for a specific CVE.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Don't hardcode the database filename, there's a variable for this in
cve-check.bbclass.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We already fetch the yearly CVE metadata and check that for updates before
downloading the full data, but we can speed up CVE checking further by only
checking the CVE metadata once an hour.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
export OPENSSL_CONF to aviod SDK openssl can not find openssl.cnf.
Signed-off-by: Liwei Song <liwei.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit db5f08c16eff01f3af0b99119c09f68628e2331d.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Ensure the task is properly regsistered as an sstate task as this
"half way" state confuses new code in bitbake and it isn't supported.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we have installed some lib32 package which depends on perl/bash,
then populating sdk for that image would fail with the following error.
Error:
Problem: package lib32-libxml-namespacesupport-perl-1.12-r0.corei7_32 requires lib32-perl, but none of the providers can be installed
package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with /usr/bin/perl provided by lib32-perl-5.30.0-r0.corei7_32
package target-sdk-provides-dummy-1.0-r0.sdk_provides_dummy_target conflicts with libperl.so.5 provided by lib32-perl-5.30.0-r0.corei7_32
This could be produced by the following steps:
1. IMAGE_INSTALL_append = " lib32-valgrind"
2. bitbake core-image-minimal -c populate_sdk
We need to extend all packages in DUMMYPROVIDES to avoid such problems.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When https_proxy is set, use proxy opener to open CVE metadata and
database URLs, otherwise fallback to the urllib.request.urlopen.
Also fix a minor issue where the json database which has been gzip
decompressed as byte object should be decoded as utf-8 string as
expected by update_db.
Signed-off-by: Chin Huat Ang <chin.huat.ang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the code fetches the compressed JSON, writes it to a temporary file,
uncompresses that with gzip and passes the fake file object to update_db().
Instead, uncompress the gzip'd data in memory and pass the JSON directly to
update_db().
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The metadata parser is fragile: first it coerces a bytes() to a str() (so the
string is b'LastModifiedDate:2019...'), assumes the first line is the date, and
then uses a regex to parse (which then includes the trailing quote as part of
the date).
Clean this up by parsing the bytes as UTF-8 (ASCII is probably fine, but this is
safer), iterate through the lines and split on colons to find the right
key/value pair.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of calling execute() repeatedly, rewrite the function to be a generator
and use executemany() for performance.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The recipe was called -native but didn't inherit native.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
djb2 hash algorithm was found to do collisions, so the database was
sometime missing data. Remove this hash mechanism, clear and populate
elements from scratch in PRODUCTS table if the current year needs an
update.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of using expanded list of affected versions that is not
reliable, use the 'cpe_match' node in the 'configurations' json node.
For cve-check to correctly match affected CVE, the sqlite database need to
contain operator_start, operator_end and the corresponding versions fields.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_populate_cve_db is a native task.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the NVD url is not accessible, print a warning on top of the CVE
report, and continue. The database will not be fully updated, but
cve_check can still run on the previous database.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To be able to populate NVD database on a fetchall
(bitbake <image> --run-all=fetch), set the do_populate_cve_db task to be
executed before do_fetch.
Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE
variable because do_populate_cve_db can be called in a context where
cve-check class is not loaded.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With postgresql added to IMAGE_INSTALL, we will get the following
error when building for 64bit BSPs.
Problem: package postgresql-11.3-r0.corei7_64 requires libperl.so.5()(64bit), but none of the providers can be installed
A previous patch has added libperl.so.5 to DUMMY_PROVIDES, but this
is not enough. Because for 64bit BSP, it should also provide libperl.so.5()(64bit).
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If https_proxy environment variable is defined, manage proxy to be able
to download meta and json data feeds from https://nvd.nist.gov
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
urllib3 was used in this recipe but it was not set as a
dependency. As it is not specifically needed, rewrite the recipe with
urllib from the standard library.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
cve-check-tool-native do_populate_cve_db task was using deprecated NVD
xml data feeds, cve-update-db uses NVD json data feeds.
Sqlite database schema was updated to take into account CVSSv3 CVE
scores and operator in affected product versions.
A new META table was added to store the last modification date of the
NVD json data feeds.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|