| Age | Commit message (Collapse) | Author |
|---|
|
Adds extended package data which is encoded as JSON which allows it to
encode more structure than the "flat" package data files. The extended
data might be much larger than the standard package data, so it is not
read by default and instead requires
oe.packagedata.read_subpkgdata_extended() to be called
Currently, the file sizes and ELF debug sources are saved off into the
extended package data
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Reviewed-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, depends.dot includes per-file dependencies but not the packages
providing those files. This makes it hard to obtain all package
dependencies by just looking at depends.dot.
Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of
their values to the package providing the component. Include runtime
packages as dependencies in depends.dot, together with the component
provided by the package as a label.
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch is a follow on of the the PR server rework in bitbake to add
read-only support. The shift to using the bb.asyncrpc code in the PR
server and client brings issues with respect to reuse of the same
asyncio loop in different processes. This patch removes the PR service
connection caching to avoid one source of this problem. It is believed
that in practice this should have little impact on overall performance.
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add entries for Android to the mappings.
Signed-off-by: Hsia-Jun Li <randy.li@synaptics.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix read_subpkgdata_dict to account for the new override operator being
":" instead of "_"
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Moving the function will allow other classes to capture which CVEs have
been patched, in particular SBoM generation.
Also add a function to capture the CPE ID from the CVE Product and
Version
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Removes the /var/cache/ldconfig auxiliary cache directory from
the rootfs when:
- read-only-rootfs is in IMAGE_FEATURES,
- ldconfig is not in DISTRO_FEATURES.
In both cases the /var/cache/ldconfig/aux-cache is useless.
Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
install_complementary
When attempt_only flag is set to True all installation failures are
ignored. Disable attempt_only, that is the default in install function,
to raise error in install_complementary and to prevent do_populate_sdk
task to treat errors as warnings.
[RP: Historically some of our addtional packages were in bad shape and
wouldn't all install. This is no longer the case so we shouldn't need
a hack like this any longer.]
Signed-off-by: Fabio Berton <fabio.berton@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds an option to strip a prefix from the paths reported in
buildhistory. This makes it easier to compare task signatures in the
build history when the builds were done from different directories.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Parse systemctl error to provide more info
[Yocto #14395]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* all known issues in this area were fixed, make it fatal that it
cannot be overlooked if someone triggers this issue again
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A stray space made it into the command for verifying gpg signatures.
This caused verification to fail, at least on my host. Removing the
space makes it work as expected.
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the scripts/postinst-intercepts is owned by root/root then the copyfile() calls
will fail due to chown issues. We don't care about ownership of these files so
use shutil.copy() instead which won't perform any chown.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* with xz PACKAGECONFIG enabled in kmod and xz module compression enabled in kernel
the do_rootfs task doesn't run depmod in the image, because it thinks there are no modules:
NOTE: No Kernel Modules found, not running depmod
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Christophe Chapuis <chris.chapuis@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The new prserv connect() function decouples the code in oe-core from the
exact classes and implementation details used within bitbake. This
allows us to more easily switch over to a new asyncrpc based prservice.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If not postinstall applied, some nativesdk command could not be found
in sdk due to update-alternatives in postinst not be executed, such as chroot:
$ which chroot
/sbin/chroot
$ which chroot.coreutils
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils
After applying the fix
$ which chroot
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot
$ which chroot.coreutils
path-to-sdk/sysroots/x86_64-wrlinuxsdk-linux/usr/bin/chroot.coreutils
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
`tmux new -c` fails on tmux older than 1.9, when that flag was added.
We can omit the flag for older versions of tmux, and the working
directory gets set even without it.
Signed-off-by: Peter Budny <pbbudny@amazon.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since pyc cache files are really system specific, no real reason to copy or
generate them during the eSDK build process. Also generating them has the
possibility of re-using inodes that pseudo may have been tracking, leading
a build failure.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The run-postinsts package runs post installation scripts
on target if packages request delayed post installations. When
no delayed post installations are found the sysV style scripts
are disabled for the package and hence it did not run on sysV
based systems. However, the package provides systemd service
as well which still ran on systems based on systemd even when
no post installations were found.
Rather than disabling/masking scripts for different initialization
managers we now simply remove/uninstall the run-postinsts package
when no post installations are found to be delayed till runtime.
This is also more aligned with the function (_uninstall_unneeded)
this functionality is triggered through.
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For the purpose of image construction using du on a rootfs directory isn't
entirely satisfactory. Bare "du" will report the actual disk usage so file
systems which can compress the data will report less than the actual space
required. Using "du --apparent-size" will report the actual space used, but as
this simply sums the bytes used for content across an entire file system can
result in significant under-reporting due to block size overhead.
Attempt to solve these problems by implementing our own function to calculate
how large a rootfs will be. This function handles hardlinks correctly but
rounds up all sizes to multiples of the block size (currently, 4KB is the
hard-coded block size).
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
CVE_VERSION_SUFFIX in "patch" to treat version string with suffix "pX"
or "patchX" as patched release.
also update testcases to cover this changes and set CVE_VERSION_SUFFIX
for sudo.
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* not sure if there are some valid use-cases for missing manifest, but
recently I'm seeing increasing number of build failures where something
from native recipe is missing (seen it with pseudo, autoconf, nodejs
recently) and the only indication that something is wrong (before showing
sometimes misleading error like:
recipe-sysroot-native/usr/bin/node: No such file or directory
is this warning:
NOTE: Running task 7844 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_prepare_recipe_sysroot)
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: Running task 7845 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_unpack)
NOTE: recipe nodejs-12.20.2-r0: task do_unpack: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Succeeded
if I rebuild that native dependency, then it gets fixed and I don't
see these failures in clean builds (as without sstate and with empty
TMPDIR), only in incremental builds
* but if there isn't valid reason for missing manifest file, then I think
it would be better to error early (or even bb.fatal())
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Ensure the fallback value if used is written to the SDE file
and hence stored in sstate, reducing any confusion within the
code over '0' values.
Bump the HASHEQUIV_VERSION since we've had a ton of trouble
with ensuring this rolls out correctly on the autobuilder so
others may too, take a clean slate for it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
GPLv2 and GPLv2+ are two difference licenses with different meanings
and we can't just pretend they're the same thing. Change the code
to treat them separately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
`devtool` uses `copy_recipe_files` for the upgrade operation when
creating the new, workspace recipe. Before handling the copy operations,
the function checks the entry in `SRC_URI` against `FILE` while in turn
uses absolute paths. When BBLAYERS contains entries that are not
normalised, this check will fail resulting in having the recipe in the
workspace without the initial patches.
Signef-off-by: Robert Drab <robert.drab@huawei.com>
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building SDKs with the deb backend you could see errors like:
Setting up nativesdk-python3-ndg-httpsclient (0.5.1-r0) ...
mkdir: cannot create directory ‘/usr/lib/opkg’: Permission denied
dpkg: error processing package nativesdk-python3-ndg-httpsclient (--configure):
which is due to environment misconfiguration when removing packages.
Fix this by setting the same environment variables as used for installation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Bitbake changed the debug() logging call to make it compatible with
standard python logging by no longer including a debug level as the
first argument. Fix up the few places this was being used.
Tweaked version of a patch from Joshua Watt.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
found" KeyError
If path is not owned by any user installed on target it gives
insufficient error "getpwuid(): uid not found" which may be misleading.
This exception occurs if uid/gid of path was not found in PSEUDO_PASSWD
files, which simply means the path is owned by host user and there is
host user contamination.
Add more information to the exception message to make it easier for user
to debug.
[YOCTO #14031]
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The "devtool modify" could remove message body before scissors line, so
patches re-generated from git tree were incorrectly modified.
Adding --no-scissors to "git am" invocation to prevent this behaviour.
[YOCTO #12674]
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If a function returns any stderr it will be passed to extractPatches and
used as path to patch.
For example subprocess command output can be:
| sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
| /tmp/oepatchhuqle8fj/0001-foo.patch
| /tmp/oepatchhuqle8fj/0002-bar.patch
that will result in:
| FileNotFoundError: [Errno 2] No such file or directory: 'sh:'
To fix this I separated output, made the function return stdout and
print stderr only in case of command error.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.
refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.
Also update testcases to cover new changes.
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The cmd redirected stderr to stdout that was then assigned to variable
with pkgs to install. Then this variable was passed to package manager
that then tried to install it and generated confusing warnings.
For example this variable could contain:
| ['(en_US.UTF-8)', 'LC_ALL:', 'bash:', 'cannot', 'change', 'locale', 'setlocale:', 'warning:']
and the warning was:
| WARNING: addon-bci-1.0-r0 do_populate_sdk: Unable to install packages.
| Command 'PATH/usr/bin/opkg ... install (en_US.UTF-8) LC_ALL: bash:
| cannot change locale setlocale: warning:' returned 255:
| Collected errors:
| * opkg_prepare_url_for_install: Couldn't find anything to satisfy '(en_US.UTF-8)'.
In this change I remove stderr redirection to stdout and pass it to
bb.note instead.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When building package indexes for RPM feeds, delete repodata first to
be absolutely sure that the indexes are up to date.
We've seen some mysterious failures where the repodata doesn't match
the repository, so hopefully this will stop that happening.
[ YOCTO #14190 ]
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.
Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.
included selftest for the version class.
[YOCTO#14127]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
- The dir /etc/apt was created in package apt, if package apt was not
installed, there is no need to insert package feed. Otherwise, it
will fail with no such dir
- Output the result of apt install
- Explicitly trust the deb package repository from build
This could avoid apt install warning:
...
WARNING: The following packages cannot be authenticated!
...
- Also trust the inserted deb package repository from PACKAGE_FEED_URIS
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Example use case in OpenBMC: rootfs is squashfs and the system has either
overlayfs for whole rootfs or for some parts (e.g. /etc).
This option will allow to create migration one-shot postinsts using
"pkg_postinst_ontarget_${PN}" routines defined in recipes to fix
files under upper workdir in overlayfs.
Signed-off-by: Anton D. Kachalov <rnouse@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make
function and if PARALLEL_MAKE is set to empty string then this variable
is exported as "None" causing cmake to fail with:
"'CMAKE_BUILD_PARALLEL_LEVEL' environment variable
invalid number 'None' given."
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[RP: Update OEQA selftest to match change]
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
oe.path.canonicalize() is used to canonicalize paths (i.e., remove
symbolic links and "..", and make them absolute). It takes a string
with paths separated by commas, and returns the canonicalized path in
the same format.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
timestamps
This in particular addresses vulkan-samples reproducibility which made me scratch my
head for a while.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
/var/cache/opkg wasn't being deleted, and /var/lib/opkg doesn't need
to exist as there are no lockfiles that write into it after this step.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
oe.path.join handles path components starting with / for us.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pseudo-native is a bit special. It conditionally compiles in support for
xattr, statx and statvfs amongst other options. If a pseudo-native binary is
used on a system where these functions are present but it wasn't compiled in
we see hard to debug permissions problems.
An example is the devtool.DevtoolExtractTests.test_devtool_deploy_target
oe-selftest which shows a cryptic error:
File "/home/pokybuild/yocto-worker/oe-selftest-fedora/build/meta/lib/oeqa/selftest/cases/devtool.py", line 1388, in test_devtool_deploy_target
self.assertEqual(filelist1, filelist2)
File "/usr/lib64/python3.9/unittest/case.py", line 831, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib64/python3.9/unittest/case.py", line 1037, in assertListEqual
self.assertSequenceEqual(list1, list2, msg, seq_type=list)
File "/usr/lib64/python3.9/unittest/case.py", line 1019, in assertSequenceEqual
self.fail(msg)
File "/usr/lib64/python3.9/unittest/case.py", line 670, in fail
raise self.failureException(msg)
AssertionError: Lists differ: ['-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8'] != ['-rwxr-xr-x root root /etc/init.d/mdmonitor', '-rw-r-[10124 chars]n.8']
First differing element 0:
'-rwxr-xr-x 6000 6000 /etc/init.d/mdmonitor'
'-rwxr-xr-x root root /etc/init.d/mdmonitor'
This is due to a version of pseudo without statx being used on a system where
ls uses statx, hence the files are displayed as 6000.6000 instead of root.root.
Avoid this by always building pseudo-native for the specific distro in question rather
than using a universal sstate feed.
This hopefully fixes one of the mysterious AB-INT issues.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Dynamic loading of package managers will allow other layers to simply
add their package manager code in package_manager/ and have bitbake find
it according to the package manager configuration. This is useful for
adding new (faster) package managers to Open Embedded while not increasing the
test scope or require Open Embedded to support more package managers.
How this is tested:
* Build core-image-minimal with all three package managers
* Build the sdk with all three package managers. dpkg fails, but
it fails on master as well.
* Run the complete test suite, all tests passed except 16
* Run those 16 tests on master and verify that they fail there as well
* Fix errors making tests works on master but not with this patch.
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Joshua Watt
Andres Beltran
Scott Murray
Hsia-Jun Li
Vyacheslav Yurkov
Richard Purdie
Damian Wrobel
Fabio Berton
Armin Kuster
Martin Jansa
Daniel McGregor
Robert P. J. Day
Devendra Tewari
Christophe Chapuis
Paul Barker
hongxu
Peter Budny
Mark Hatle
Awais Belal
Ross Burton
Lee Chee Yang
Andrei Gherzan
Tomasz Dziendzielski
Ross Burton
Anton Kachalov
Michael Ho
Peter Kjellerstedt
Alexander Kanavin
Fredrik Gustafsson