| Age | Commit message (Collapse) | Author |
|---|
|
At the end of the extensible SDK installation, if we've successfully
prepared the build system then we don't need ext-sdk-prepare.py. I had
thought earlier that this would be used when updating, but a different
mechanism was needed there so this script isn't used for that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make the following improvements to the SDK update process:
* Use a manifest file with sha256sums to track files other than sstate
and metadata that we need to update - e.g. conf files. This allows us
to handle where files such as auto.conf may or may not be present,
as well as the configuration changing without affecting task signatures
- we still want the config files copied in that case rather than it
saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
so that local.conf remains static (since we don't want to trigger an
update every time). Also, If there is an SSTATE_MIRRORS value already
set in the configuration we can skip this and assume it contains the
needed packages.
* Allow the update process to be run in any directory, don't assume
we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
then move them to the desired location at the end, to avoid a
failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
instead of using the SDK targets. This ensures any item installed
through dependencies after installation (e.g. when running
"devtool build") won't go stale.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix a regression caused by OE-Core revision
eabeb26335b1a4eb1e68218160dbdbe8fdf36272 where we lost the task
dependency on packages in TOOLCHAIN_HOST_TASK (such as
meta-environment-extsdk-${MACHINE}) which led to errors about missing
packages when building the extensible SDK. This happened because I only
moved half of the dependencies coming from the standard SDK task to
do_sdk_depends.
While I'm at it, tidy up the do_populate_sdk_ext[depends] line.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.
(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
newline
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The user of the extensible SDK doesn't need to see these.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
During extensible SDK installtion, if the build system preparation step
fails we try to put something at the end of the environment setup script
to show an error when it is sourced, in case the user doesn't realise
that the partially-installed SDK is broken. However, an apostrophe in
the message (actually a single quote) appears to terminate the string
and therefore breaks the command. Drop it to avoid that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Ensure extensible SDK and standard SDK go into their own directories
* Record extra extensible SDK variables
* Write sstate-package-sizes.txt and sstate-task-sizes.txt files so you
can analyse the size of the contents
* Add BUILDHISTORY_SDK_FILES (similar to BUILDHISTORY_IMAGE_FILES) and
default it to pick up config files installed in the extensible SDK
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If the installation of buildtools fails then we should fail the entire
installation instead of blindly continuing on.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The configuration of the build system within the extensible SDK is
fixed, so there's not a lot of point in showing it; plus it just gets in
the way of the output that's interesting to the user in this context. So
let's hide it within the extensible SDK.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We inherit uninative in the extensible SDK configuration, and uninative
sets NATIVELSBSTRING to a fixed value, so we don't need to force the
value ourselves.
Fixes [YOCTO #8662].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should always pass a parameter to getVar, add missing default value.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We want to support different names for the buildtools tarball. The
name may not always be of the default oe-core format.
For instance, at Wind River we define the built-tools name to be:
${SDK_ARCH}-buildtools-nativesdk-standalone-${DISTRO_VERSION}
because thes standard SDK_NAME has additional information that is not
relevant to the builtools tarball.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The extensible SDK relies upon uninative, and with the way that
uninative works, the build system architecture must be the same as the
SDK architecture or the extensible SDK won't be usable. At some point in
future hopefully we can remove this limitation, but until then it's
disingenuous to allow this to build, so add a check to ensure
SDK_ARCH == BUILD_ARCH and fail if it isn't.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you have a local workspace layer enabled when building the
extensible SDK, we explicitly exclude that from the SDK (mostly because
the SDK has its own for the user to use). Adjust the message we print
notifying the user of this so it's clear that we're excluding it from
the SDK, and scale it back from a warning to a note printed with
bb.plain().
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When SDK preparation fails:
* Insert an ERROR: in front of the error message
* Add an error message to the environment setup script
Hopefully this should make it more obvious when this happens.
Fixes [YOCTO #8658].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Use a variable for the log file which includes the full path; this is
not only neater but avoids us writing the first part (the output of
oe-init-build-env) to a file in another directory since we are
changing directory as part of this subshell.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If you run the extensible SDK environment setup script in a shell
session where oe-init-build-env has been run already, and attempt to use
the two together, strange things happen - you may not even be running
devtool from the extensible SDK, but the OE tree. This isn't a supported
use case anyway, so show a warning recommending starting a new shell
session.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Print a note at the end of the environment setup script pointing to
devtool.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to prepare the build system within the extensible SDK, we
actually go ahead and build the targets specified by SDK_TARGETS (by
default the image the SDK was built for). Assuming that's an image, we
don't actually need to build the image itself - we just need to have
everything done up to the point before building the image, so that we
have everything needed in the sysroot.
In order to do this, create temporary bbappends for each of the targets
in the workspace layer that stub out do_rootfs and related tasks if they
exist. This is a little bit of a hack but is the least intrusive fix at
this point. To make things a bit tidier, I have split out the
preparation commands into a separate script so we can run that in the
appropriate environment rather than all the commands separately.
Fixes [YOCTO #7590].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make indentation consistent here in preparation for the changes that
follow.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is not actually used for anything - I thought that we would need to
use it within devtool to set global configuration, but we're able to do
everything we need within the bbappends it creates, which also saves on
parse time. If we're not going to use work-config.inc let's just drop it
completely.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Provide the ability to define a function containing extra configuration
values to be added to the local.conf file that goes into the SDK. For
example, this could be used to set up SSTATE_MIRRORS within the SDK.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
installer with sh
If you ran the extensible SDK installer file with sh (instead of bash),
then the additional call to buildtools environment setup, extension of
PATH to support running devtool, and setting of OE_SKIP_SDK_CHECK
weren't being added to the end of the script. This is because apparently
bash is happy to expand wildcards in the target of a redirection, but
bash running in POSIX sh mode won't (although it apparently does work on
the sh command line rather than within a script run as an argument to
sh). In any case using a wildcard here is a bit of a crutch which we
don't need, so replace it with the proper path to the environment setup
script.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable updating the installed extensible SDK from a local or remote
server, avoiding the need to install it again from scratch when
updating. (This assumes that the updated SDK has been built and then
published somewhere using the oe-publish-sdk script beforehand.)
This plugin is only enabled when devtool is used within the extensible
SDK since it doesn't make sense to use it next to a normal install of
the build system.
E.g.
devtool sdk-update /mnt/sdk-repo/
devtool sdk-update http://mysdkhost/sdk
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When publishing SDK, what we want is basically its metadata and sstate
cache objects. We don't want the SDK to be prepared with running bitbake
as it takes time which reproduces meaningless output for the published SDK.
So this patch adds an option to allow for SDK to be extracted without
preparing the build system.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These dependencies were deliberately removed because it was assumed that
they were provided by nativesdk packages. On the one hand, nativesdk packages
in extensible SDK don't have these packages; on the other hand, even if we
add these nativesdk packages, they are still not useful because we we need
runqemu to run correctly.
So we don't remove these native qemu dependencies.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Copy the contents of local.conf under TOPDIR into the final generated
local.conf. In this way, custom settings are also made into the final
local.conf like IMAGE_INSTALL, DISTRO_FEATURES, VIRTUAL-RUNTIME_xxx,
etc. Comments and blank lines are filtered out.
Before this change, installing extensible SDK would usually report failure
when preparing the build system if the user has custom configuration for
DISTRO_FEATURES in local.conf. Also, items in IMAGE_INSTALL_append in
local.conf also don't get built correctly.
This patch solves the above problem by making use of bb.utils.edit_metadata.
In addition, we check to avoid any setting that might lead to host paths
bleeding into the SDK's configuration. Basically, variables with values
starting with '/' are removed. A whitelist mechanism is introduced so that
users could specify variables that should not be ignored. The name of the
whitelist is SDK_LOCAL_CONF_WHITELIST.
The SDK_META_CONF_WHITELIST is removed as it's of no use after this
change.
SDK_LOCAL_CONF_BLACKLIST can be used to prevent copying specific
variable settings to the extensible SDK's local.conf; the default is to
exclude PRSERV_HOST (since this is likely to be internal). Similarly,
SDK_INHERIT_BLACKLIST to forbit local.conf in SDK to inherit certain
classes such as 'buildhistory' or 'icecc' that would not normally make
sense in an SDK environment.
[YOCTO #7616]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we do `bitbake buildtools-tarball' and then after one day do `bitbake
core-image-minimal -c populate_sdk_ext', we would meet errors like below.
| install: cannot stat '/buildarea2/chenqi/poky/build-systemd/tmp/deploy/sdk/
poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone
-1.8+snapshot-20150429.sh': No such file or directory
The problem is that the output name for buildtools-tarball has ${DATE} in it.
So if populate_sdk_ext task is executed but buildtools-tarball is not rebuilt,
the above error appears.
Instead of hardcoding ${DISTRO_VERSION} which consists of ${DATE} in the
install_tools() function, we should find the latest buildtools-tarball based
on the modification time and install it.
[YOCTO #7674]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Only poky sets SDK_NAME to include ${IMAGE_BASENAME} (i.e. ${PN}), so we
can't assume the buildtools filename will include it here. Change it to
look for a file with "buildtools-nativesdk-standalone" in the name
(the buildtools-tarball recipe itself sets TOOLCHAIN_OUTPUTNAME to
include this.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Not all hosts are running sufficiently new coreutils.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Since the extensible sdk uses bitbake, which can't run as root, the sdk
shouldn't be installed as root.
Previously it would error out late into setup when bitbake errored
saying not to run bitbake as root.
Now the script errors with a message saying the extensible sdk can't be
installed as root.
[Yocto #7545]
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Show a friendly title when running the SDK installer, so the user knows
what SDK they are installing. The title is controlled by the
SDK_INSTALLER_TITLE variable and includes the distro name and SDK
version by default.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It may be tempting to edit the configuration of the encapsulated version
of the build system, however that is not the way it is intended to be
used, so add a warning against doing this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Most of the time we shouldn't be downloading anything within the
extensible SDK (since it's all pre-built and we have the sstate
artifacts) therefore there's really no need for a connectivity
check, in fact it may just get in the way.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using bitbake to do the setscene as part of sdk setup, it would be
useful to have a log in the case where it fails.
The log is called preparing_build_system.log and is in the top level
directory of the extracted sdk.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This bbclass will create an SDK with a copy of bitbake and the metadata
and sstate for the target specified for the task. The idea is to let
"system" developers both work on applications and then test adding them
to an image without having to switch between workspaces or having to
download separate items.
Rather than running bitbake directly however, the primary way of running
builds within the extensible SDK is to use the "devtool" command. The
rest of the build system is fixed via locked shared state signatures,
and thus only the recipes you have added get built.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Paul Eggleton
Randy Witt
Richard Purdie
Mark Hatle
Qi.Chen@windriver.com
Christopher Larson