| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
|
|
Implement support for remote signing using obs-signd. It is now possible
to sign both RPM packages and package feeds with this method. The user
just needs to set RPM_GPG_BACKEND and/or PACKAGE_FEED_GPG_BACKEND
variables to 'obssign' in the bitbake config. Of course, in addition,
one needs to setup the signing server and the configure the 'sign'
client command on the build host. The *_PASSPHRASE_FILE settings are not
used when the obssign backend is enabled.
[YOCTO #8755]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
|
|
Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g. package
signing and sstate signing). Currently, only local signing with gpg is
implemented.
[YOCTO #8755]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
|
|
Currently this code installs into the standard sysroot, however this causes
some conflicts when linking since the linker can look specifically for
versioned .so files (e.g. like libpthreads.so.0). This breaks builds
of util-linux-native for example.
The easiest solution is to install uninative into its own separate sysroot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_stage was obsoleted years ago, drop the test now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
xz is slow at compressing the SDKs, we can speed it up by using the
parallel compressor, pixz.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Originally, the idea was that the init environment would handle
fetching or providing the binary shim that uninative needs.
This turns out to be ugly, especially when you consider proxy
environments and so on getting involved. Instead, lets therefore
support our fetcher which already handles all this.
The distro is expected to setup configuration like:
UNINATIVE_URL ?= "http://mydomain/mypath/"
UNINATIVE_CHECKSUM[i586] =
"md5sum1"
UNINATIVE_CHECKSUM[x86_64] = "md5sum2"
and then it should all work if the user inherits the uninative class.
This patch also improves the error handling in the class to give more
user readable error messages.
If the shim binary is already provided, the system will just use that
and ignore the url information.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the script files we use to construct the SDK installer change then
that really ought to trigger re-execution of the do_populate_sdk(_ext)
task, so add file-checksums varflags to ensure that happens.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
At the end of the extensible SDK installation, if we've successfully
prepared the build system then we don't need ext-sdk-prepare.py. I had
thought earlier that this would be used when updating, but a different
mechanism was needed there so this script isn't used for that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make the following improvements to the SDK update process:
* Use a manifest file with sha256sums to track files other than sstate
and metadata that we need to update - e.g. conf files. This allows us
to handle where files such as auto.conf may or may not be present,
as well as the configuration changing without affecting task signatures
- we still want the config files copied in that case rather than it
saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
so that local.conf remains static (since we don't want to trigger an
update every time). Also, If there is an SSTATE_MIRRORS value already
set in the configuration we can skip this and assume it contains the
needed packages.
* Allow the update process to be run in any directory, don't assume
we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
then move them to the desired location at the end, to avoid a
failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
instead of using the SDK targets. This ensures any item installed
through dependencies after installation (e.g. when running
"devtool build") won't go stale.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A warning that doesn't say what the PN is doesn't really help.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The parameter "path" would be redefined inside package_qa_walk() which
is useless, so remove it.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed:
If more than one files have the same QA issue, it only prints the last
one, others are overrided, for example:
messages["host-user-contaminated"] = "foo1"
messages["host-user-contaminated"] = "foo2"
Only foo2 will be printed.
Add package_qa_add_message() to hanlde messages, so that all of them
will be printed. The package_qa_add_message() is from RP.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed race issue when:
WARN_QA_append = " buildpaths"
PACKAGE_CLASSES = "package_ipk"
$ bitbake e2fsprogs
[snip]
*** 0595: with open(path) as f:
0596: file_content = f.read()
0597: if tmpdir in file_content:
0598: messages["buildpaths"] = "File %s in package contained reference to tmpdir" % package_qa_clean_path(path,d)
0599:
Exception: IOError: [Errno 2] No such file or directory: '/path/to/tmp/work/core2-64-poky-linux/e2fsprogs/1.42.9-r0/packages-split/e2fsprogs-ptest/CONTROL/control'
The similar to package_deb.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When cleaning old builds from the sysroots, also print the sysroot architecture.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The config.log written by autoconf includes many pages of useless output,
followed by an obfuscated error message, and then more pages of every variable
that's been set. It's only understandable if you're well versed in how autoconf
behaves, and often in simple failure modes doesn't actually make it clear what
the problem was.
Instead of outputting the whole config.log to the console when do_configure()
fails, use bbfatal_log so the human-readable configure output (not the
config.log) is shown to the user, and tell the user where config.log can be
found if they need it.
[ YOCTO #8856 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.
With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.
The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.
[YOCTO #7427]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If there are lots of changes between the previous build and the build about to
start bitbake will potentially print pages of:
DEBUG: Stamp $BUILD/stamps/corei7-64-poky-linux/libdrm/2.4.66-r0 is not reachable, removing related manifests
Instead of spamming the console with this list, write the list of manifests only
to the debug log and simply write a count to the console. This way the user
doesn't get spammed but still knows what is happening if bitbake appears to
stall with heavy I/O.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
do_rootfs task needs to depend on signing-keys:do_export_public_keys.
The rpm signing public key needs to be present in order to prevent a
crash because it is imported into the rootfs rpmdb before rootfs
creation starts.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If we "bitbake X -c packagedata" and the packagedata comes from sstate, we
don't need any of the tasks dependencies. This is similar to the
populate_lic case, we only care about the end result.
Therefore short circuit the dependencies so packagedata doesn't pull in
any other dependencies.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix a regression caused by OE-Core revision
eabeb26335b1a4eb1e68218160dbdbe8fdf36272 where we lost the task
dependency on packages in TOOLCHAIN_HOST_TASK (such as
meta-environment-extsdk-${MACHINE}) which led to errors about missing
packages when building the extensible SDK. This happened because I only
moved half of the dependencies coming from the standard SDK task to
do_sdk_depends.
While I'm at it, tidy up the do_populate_sdk_ext[depends] line.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Shell function names can't contain '-' characters, which means our image
task names also can't. Add some mapping to use '_' instead of the '-' so
images like "rpi-sdimg" work again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that image generation consists of separate tasks, we can improve
the dependencies in these classes. We can also incorporate the tasks
within do_image_complete instead of do_build so we have a specific task
for when the deploy directory is populated with images.
This fixes various expectations about do_image_completed and the way
I'd tried to use it to fix some sanity test problems.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic command can be used externally but for this to work, the wic
environment file needs to be present. Therefore write this out
universally, it runs in parallel with other image construction so
any performance implications are negligible.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, iso and hddimg links don't respect RM_OLD_IMAGE. This
updates them to use the common symlinks code so that they behave
like the rest of the system.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When RM_OLD_IMAGE is set, old manifest files should be removed
along with old image files and our QA tests expect this.
This patch ensures this happens.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similarly to DATETIME, don't expand TMPDIR in image commands.
This ensures some of the stamp comparisons we make in the
QA tests work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_image_* tasks contained the expanded version of DATETIME. Due
to the expansion, we couldn't exclude the value from the task checksum
which meant the task would rerun.
We fix this by deleting the DATETIME value during expansion so we don't
expand any references to at that time. This means the task's hash can be
stable rather than having hardcoded date/time values. It will get expanded
at execution time.
This also fixes errors shown by -S:
NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/locked-sigs.inc
ERROR: Bitbake's cached basehash does not match the one we just generated (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-sato/images/core-image-sato.bb.do_image_tar)!
ERROR: The mismatched hashes were 77872792556367f1dde49a1425caf1a0 and 9bb0aca6286ab7dd22d3c69964beb665
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:
======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.
It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.
Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst in theory return values are nice, handling the complex
aggregation of the return values is ugly. The code already uses QA_SANE
as a marker to signal problems. Whilst that isn't as idealistic, it
makes the code massively cleaner, so rely on this instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move do_kernel_link_vmlinux() from kernel-yocto.bbclass into
kernel.bbclass so that it's available to any kernel recipe.
Note that the task is not enabled by default in kernel-yocto.bbclass,
so don't enable by default in kernel.bbclass either. To enable, see
the example in linux-yocto.inc, ie:
addtask kernel_link_vmlinux after do_compile before do_install
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_kernel_link_vmlinux() task modifies the build directory (not
the source tree) and should not be skipped when externalsrc is being
used.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
SDK artifacts were being dumped alongside the image file dump.
However, the dump was not being triggered correctly, as it
should be attached as a postfunc to do_populate_sdk, rather than
do_rootfs, as do_rootfs doesn't always fire when populating
the SDK directory.
By adding a postfunc to do_populate_sdk, the postfunc is fired
in the correct situations.
Also fixes the directory which is scanned for SDK artifacts, as
this is different from the image file directory.
[YOCTO #7603]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This removes unnecessary spaces inserted before semicolons in the
modified USERADD_PARAM_${PN} and GROUPADD_PARAM_${PN} variables. This
should not affect the handling of the variables as the only one that
actually sees the semicolons is the code in useradd.bbclass that uses
cut to split the variables at them, and any whitespace preceeding or
following the semicolons will be properly ignored.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Read and merge the passwd/group files before parsing the user and
group definitions. This means they will only be read once per
recipe. This solves a problem where if a user was definied in multiple
files, it could generate group definitions for groups that should not
be created. E.g., if the first passwd file read defines a user as:
foobar::1234::::
and the second passwd file defines it as:
foobar:::nogroup:The foobar user:/:/bin/sh
then a foobar group would be created even if the user will use the
nogroup as its primary group.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The original code was near impossible to follow, and missed a couple
of cases. For example, if one added the following line to the passwd
file specified in USERADD_UID_TABLES:
foobar:x:12345:nogroup::/:/bin/sh
and then specified the user as:
USERADD_PARAM_${PN} = "--system foobar"
one would then assume that the foobar user would be created with the
primary group set to nogroup. However, it was not (the primary group
would be foobar), and the only way to get it correct was to explicitly
add --gid nogroup to the USERADD_PARAM_${PN}.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The [<on_true>, <on_false>][not <condition>] construct may solve the
problem of implementing a conditional operator, but it is not very
readable. At least I find this:
uaargs.groupid = field[3] or uaargs.gid or uaargs.groupname
a lot more readable than this:
uaargs.groupid = [uaargs.gid, uaargs.groupname][not uaargs.gid]
uaargs.groupid = [field[3], uaargs.groupid][not field[3]]
Also, the official conditional operator since Python 2.5 (<on_true> if
<condition> else <on_false>) does not evaluate both <on_false> and
<on_true> as [<on_true>, <on_false>][not <condition>] does.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If --no-user-group is specified in USERADD_PARAM_${PN} for a user and
no --gid is specified, then we should not assume that the group name
for the user is the user name.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The useradd options --create-home/--no-create-home and
--user-group/--no-user-group are mutually exclusive and should be
treated as such.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The absolute path (/path/to/configure) caused VPATH in Makefile to be an
absolute path, and then it will be in elf files, use relative path to
run configure can fix the problem.
This will reduce a lot of buildpaths QA issues in a world buld.
[YOCTO #8894]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
kernel ARCH
For a bare-bone kernel recipe which specifies 32 bit x86 target,
a 64 bit .config will be generated from do_configure task when
building 32-bit qemux86, once all of these conditions are true:
* arch of host is x86_64
* kernel source tree used in build has commit ffee0de41 which
actually chooses i386 or x86_64 defconfig by asking host when
ARCH is "x86" (arch/x86/Makefile)
* bare-bone kernel recipe inherits directly from kernel without
other special treatments.
Build will fail because of the mismatched kernel architecture.
The patch sets ARCH i386 or x86_64 explicitly to configure
task to avoid this host contamination. Kernel artifact is also
changed so that it can map i386 and x64 back to arch/x86 when
needed.
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).
The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.
Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.
(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.
The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.
The solution is to check QA_SANE in places we use the handle_error
function.
[YOCTO #8907]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Markus Lehtonen
Richard Purdie
Paul Eggleton
Ross Burton
Robert Yang
Mariano Lopez
Jackie Huang
Andre McCurdy
Elliot Smith
Peter Kjellerstedt
Jianxun Zhang
Otavio Salvador