| Age | Commit message (Collapse) | Author |
|---|
|
BitBake can optionally 'import yaml' if BB_LOGCONFIG specifies a yaml
file. This is a 3rd party module, so that this works out of the box
when buildtools is used -- either explicitly via buildtools-tarball or
implicitly via eSDK -- we can add pyyaml to the buildtools.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the change to bitbake, update the references in OE-Core to match the updates.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some of the buildtools tests test network access so allow this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The class adds an emtpy PACKAGES setting but most code now uses the
nopackages class which is much clearer. It also adds recursive do_build
dependencies which don't really serve any useful purpose any more.
Simplify the code and drop the class use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if you change the site files, nothing rebuilds since they are
not accounted for in task checksums. They could/should be through the
file-checksums task flag. We need to cache all the files looked for,
whether the exist or not so that if they do exist and didn't,
the checksum also changes.
This gets complicated by the need to clean out hardcoded build
paths from the variable and that other layers can have site files.
This patch adds this functionality. A new variable, SITEINFO_PATHVARS
is added which controls which substitutions to make on the file-checksum
values to remove the hardcoded paths. Layers adding site files will need
to set this to a variable that has the layer path in it and is excluded
from task hashes (COREBASE is the one the core layer uses).
This patch will cause yocto-check-layer to fail for some layers
where site files are added yet the layer isn't a machine specific layer.
This is arguable correct since these additional site files apply to
all recipes and things from a layer like core could be changed by such
changes so it is right they should rebuild. There is a determinism issue
potentially there if not. meta-openembedded does have some such references
but looking at them they should move to core or likely just be removed as
most look obsolete anyway.
[YOCTO #13729]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a testsdk task, which is essentially the same as testsdk.bbclass but
the test case directory is changed. This lets us exercise the
buildtools tarballs at build time.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since we start to require these for builds, we need to include them in buildtools-tarball
so that older systems can access them easily.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Recently an entry in the NVD DB appeared that looks like that
{'vulnerable': True, 'cpe_name': []}.
As besides all the vulnerable flag no data is present we would get
a KeyError exception on acccess.
Use get method on dictionary and return if no meta data is present
Also quit if the length of the array after splitting is less than 6
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With this small patch, it's possible to overrule the public
URL with a local mirror for those without Internet access.
Signed-off-by: Jan Vermaete <jan.vermaete@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
some record from NVD can merge or split suffix from version, for
example:
CVE-2017-15906
"cpe23Uri" : "cpe:2.3:a:openbsd:openssh:5.0:p1:*:*:*:*:*:*"
"cpe23Uri" : "cpe:2.3:a:openbsd:openssh:4.7p1:*:*:*:*:*:*:*"
in such case include the suffix into version when update local CVE db.
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds the debug symbols for the binaries included in the uninative
tar ball. These are needed if one wants to run valgrind on a native
binary when uninative is used. Or get complete backtraces using gdb.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building vim it tries to rebuild files using iconv. If this fails
the build continues anyway but the output is not determnistic as builds
using a hosttools tarball are different from builds where there isn't a
hosttools tarball. Add the needed gconvs to the tarball when iconv is
present to become determistic and generate vim locales consistently.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When used in SDKs we need to provide the perl modules used by autoconf.
Add new ones needed by recent changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows the hardcoded nativesdk dependency hacking in the
automake recipe to be dropped and matches what autoconf is doing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These both now work on arm64, so add them to the dependencies.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we're not building syslinux, then there's no point building
syslinux-native.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It only added ${SDKPATHNATIVE}/usr/bin to PATH which didn't work when files
were installed to other bin dirs such as /bin or /sbin, for example,
nativesdk-pigz installs the files to /bin, now fix it to keep align with sdk's
PATH.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fix below error:
wic ls ./core-image-minimal-intel-x86-64.wic
ERROR: Can't find executable parted
wic depend on some tools like parted/mtools/..., and we have those
tools in native_sysroot. so above problem can be avoided by
run command like wic ls imagename.wic --native-sysroot <path>
but this cannot cover condition that usr don't have build the
image, the image just copy from somewhere.
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a CPE version field is just *:*:*:* it should be handled the same as
-:*:*:*, that is 'all versions'. To ease handling, transform this case
to use -.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
OECORE_NATIVE_SYSROOT is used by tools like oe-run-native and hence
we were seeing selftest failures when newer buildtools-tarballs that
use this were run on the autobuilder.
Unset the variable after use to avoid these issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is used by various pieces of the result handling code, particularly the
performance testing and we're seeing autobuilder failures that are
easiest resolved using this.
Acked-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It was always questionable to do this in an anonymous function, but now
with multiconfig it is a critical mistake and leads to more strange
"Exception: sqlite3.OperationalError: disk I/O error" errors.
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
class is enabled
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of inventing a new task to fetch the CVE data, use the existing
fetch task.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add some debug logging when fetching the CVE data.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This import isn't used anymore, so remove it.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously CVE_CHECK_DB_FILE / CVE_CHECK_DB_DIR was the same across
multiconfigs which led to a race condition wherein multiple
cve-update-db-native:do_populate_cve_db tasks could attempt to write to
the same sqlite database. This led to the following task failure:
Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_populate_cve_db(d)
0003:
File: '/mnt/data/agent/work/74f119cccb44f133/yocto/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 103, function: do_populate_cve_db
0099: if year == date.today().year:
0100: cve_f.write('CVE database update : %s\n\n' % date.today())
0101:
0102: cve_f.close()
*** 0103: conn.commit()
0104: conn.close()
0105:}
0106:
0107:def initialize_db(c):
Exception: sqlite3.OperationalError: disk I/O error
Use a lockfile to ensure multiple tasks don't step over each other.
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By doing this we can revert b18c32ab6bc9c4f1953e9f79aa39bc92d1c4e30d which
was a pretty ugly hack anyway and now means the different providers are all
being handled consistently.
Anyone with SDK recipes will need to ensure nativesdk-sdk-provides-dummy
is included in those builds (or an equivalent). This is a good thing to
do anyway.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently buildtools has its own hacked environment setup. We added
capability for generic script fragements but this code was never updated
to use it even if several components it contains needs it.
Remove a hardcoded variable in favour of the generic scripts to
avoid bugs in this area in future. The others should probably be
migrated to environment files in future too but there isn't such
a file for those at present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently in NVD DB an item popped up, which hasn't set baseMetricV2.
Let the parser handle it as an optional item.
In case use baseMetricV2 before baseMetricV3
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_deploy should clean up ${DEPLOYDIR} before running, just like do_install
cleans up ${D} before running. This reduces the risk of DEPLOYDIR being
accidentally contaminated by files from previous runs, possibly even with
different config, in case of incremental builds.
It is convenient to have this in deploy.bbclass, so it doesn't have to be
duplicated in every recipe, considering for example meta-freescale, which
has 23 affected recipes.
All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky,
meta-openembedded and meta-freescale look like they either benefit from
this or are at least not affected negatively by it. The only exception
I've noticed was uboot-sign.bbclass, which was however fixed by the
previous patch.
Signed-off-by: Daniel Klauer <daniel.klauer@gin.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The native version of grub-efi only installs the tools
/usr/bin/grub-editenv
/usr/bin/grub-mkimage
to sysroots-components/, but equivalent tools are already provided by
grub-native, the difference on x86_64 being 4 hardwired paths in grub-mkimage
(values taken from grub-native):
LOCALEDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/share/locale
GRUB_DATADIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/share
GRUB_LIBDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/usr/lib
GRUB_SYSCONFDIR = $TMPDIR/work/x86_64-linux/grub-native/2.04-r0/recipe-sysroot-native/etc
If grub-native and grub-efi-native are built with the following patch
--- grub-2.04.orig/configure.ac
+++ grub-2.04/configure.ac
@@ -1980,10 +1980,10 @@ grub_libdir="$(eval echo "$libdir")"
grub_localedir="$(eval echo "$localedir")"
grub_datadir="$(eval echo "$datadir")"
grub_sysconfdir="$(eval echo "$sysconfdir")"
-AC_DEFINE_UNQUOTED(LOCALEDIR, "$grub_localedir", [Locale dir])
-AC_DEFINE_UNQUOTED(GRUB_LIBDIR, "$grub_libdir", [Library dir])
-AC_DEFINE_UNQUOTED(GRUB_DATADIR, "$grub_datadir", [Data dir])
-AC_DEFINE_UNQUOTED(GRUB_SYSCONFDIR, "$grub_sysconfdir", [Configuration dir])
+AC_DEFINE_UNQUOTED(LOCALEDIR, "/non-existent", [Locale dir])
+AC_DEFINE_UNQUOTED(GRUB_LIBDIR, "/non-existent", [Library dir])
+AC_DEFINE_UNQUOTED(GRUB_DATADIR, "/non-existent", [Data dir])
+AC_DEFINE_UNQUOTED(GRUB_SYSCONFDIR, "/non-existent", [Configuration dir])
the produced grub-editenv/grub-mkimage binaries become binary equivalent,
assuming reproducible builds is active. Since the unpatched values of
LOCALEDIR/GRUB_DATADIR/GRUB_LIBDIR/GRUB_SYSCONFDIR point to directories that
are not expected to exist at runtime, they can be ignored.
Therefore:
* remove grub-efi-native and instead rely on the same tools from
grub-native
* replace references to grub-efi-native with grub-native
* remove unused grub-efi-native security flags overrides
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The autobuilder has been experiencing SSL: CERTIFICATE_VERIFY_FAILED
errors during error report uploads when using buildtools due to looking
for certs in /opt/poky
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
staging_populate_sysroot_dir() collects postinsts from the sysroot
and executes them. These postinsts, in turn, may call binaries that
are only available from the sysroot. This works fine with recipe-specific
sysroots, as all necessary paths are already in PATH, but breaks down
in this recipe which imitates the old global sysroot way but doesn't adjust
the PATH to include the binary paths from global sysroot.
To reproduce the failure:
$ bitbake docbook-xml-dtd4-native
$ bitbake -c build_native_sysroot build-sysroots
...
Exception: subprocess.CalledProcessError: Command '/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog' returned non-zero exit status 127.
Subprocess output:
/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 5: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found
/home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: 8: /home/akanavin/build/tmp/sysroots/x86_64/usr/bin/postinst-docbook-xml-dtd4-native-xmlcatalog: xmlcatalog: not found
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Extend the functionality provided by commit [1] to the SDK as well. This way we
can make sure that nativesdk-binutils finds SDK libraries first rather than
host ones.
This is useful for example when trying to build the linux kernel using
nativesdk-gcc. This scenario currently fails because it tries to link to host
libraries rather than SDK host ones:
make x86_64_defconfig
make bzImage
...
error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
Makefile:1101: recipe for target 'prepare-objtool' failed
make: *** [prepare-objtool] Error 1
....
/../../../../x86_64-wrlinuxsdk-linux/bin/ld: /lib/x86_64-linux-gnu/libpthread.so.0: undefined reference to `__libc_vfork@GLIBC_PRIVATE'
...
[1] 15049c610b [buildtools-tarball: Add an ld.so.conf for nativesdk-binutils]
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is needed in particular for newer versions of rpm
which would otherwise fail to build due to absence of omp.h header.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow specifying an optional destination to include-path and make the
option aware of permissions and owners.
It is very useful for making a partition that contains the rootfs for a
host and a target Eg:
/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)
Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.
With this patch we can use something like:
part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello
on the .wks file.
Cc: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On ubuntu 18.04.1, it does not provides `mdir' by default
which caused `wic ls **.wic' failed on fat partition
...
$ wic ls build/tmp-glibc/deploy/images/xilinx-zynqmp/wrlinux-image-std-xilinx-zynqmp.wic
ERROR: Can't find executable 'mdir'
...
Add nativesdk-mtools to buildtools-tarball and use buildtools
to provide mdir
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Builds like native-openjdk, really wants a to link
some tools against the static version. Since when
using the extended tarball, its the only place to
get it, add the library.
Signed-off-by: Jeremy Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fix logic for CVE DB update so that when the CPE version is '-',
it keeps the version as '-' in the DB file too and leave other
operation as blank.
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.
Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.
Non-valid CVE to be whitelisted separately.
[YOCTO #13617]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
when do_populate_cve_db forced stop at certain point, the
DB execution are stoped however the temporary database
file (DB-JOURNAL) are not removed. This db-journal file
indicates that DB is incomplete and set DB in readonly
mode. So when db-journal exist, remove both DB and the
db-journal and build the DB again from scratch.
[YOCTO #13682]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
prevent cve-check from fatal error cause by network issue.
[YOCTO #13680]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Ross Burton
Richard Purdie
Konrad Weihmann
jan
Lee Chee Yang
Peter Kjellerstedt
Robert Yang
Changqing Li
Chris Laplante
Daniel Klauer
Jacob Kroon
Steve Sakoman
Alexander Kanavin
Ovidiu Panait
Ricardo Ribalda Delgado
hongxu
Jeremy Puhlman