| Age | Commit message (Collapse) | Author |
|---|
|
Now that image generation consists of separate tasks, we can improve
the dependencies in these classes. We can also incorporate the tasks
within do_image_complete instead of do_build so we have a specific task
for when the deploy directory is populated with images.
This fixes various expectations about do_image_completed and the way
I'd tried to use it to fix some sanity test problems.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic command can be used externally but for this to work, the wic
environment file needs to be present. Therefore write this out
universally, it runs in parallel with other image construction so
any performance implications are negligible.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, iso and hddimg links don't respect RM_OLD_IMAGE. This
updates them to use the common symlinks code so that they behave
like the rest of the system.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When RM_OLD_IMAGE is set, old manifest files should be removed
along with old image files and our QA tests expect this.
This patch ensures this happens.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similarly to DATETIME, don't expand TMPDIR in image commands.
This ensures some of the stamp comparisons we make in the
QA tests work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_image_* tasks contained the expanded version of DATETIME. Due
to the expansion, we couldn't exclude the value from the task checksum
which meant the task would rerun.
We fix this by deleting the DATETIME value during expansion so we don't
expand any references to at that time. This means the task's hash can be
stable rather than having hardcoded date/time values. It will get expanded
at execution time.
This also fixes errors shown by -S:
NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/locked-sigs.inc
ERROR: Bitbake's cached basehash does not match the one we just generated (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-sato/images/core-image-sato.bb.do_image_tar)!
ERROR: The mismatched hashes were 77872792556367f1dde49a1425caf1a0 and 9bb0aca6286ab7dd22d3c69964beb665
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:
======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This cleans up the do_configure_qa code so that the exit status from
package_qa_handle_error is handled correctly.
It also converts package_qa_check_license to use our standard QA check
configuration interface through package_qa_handle_error rather than
a mix of return values, bb.fatal and bb.build.FuncFailed.
Due to the merging of multiple lines into one message, we need to
tweak a QA test to account for this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst in theory return values are nice, handling the complex
aggregation of the return values is ugly. The code already uses QA_SANE
as a marker to signal problems. Whilst that isn't as idealistic, it
makes the code massively cleaner, so rely on this instead.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The license manifest is wrong when there is no spaces before and after separates |&()
and we can get warning like:
WARNING: The license listed Artistic-1.0GPLv1+ was not in the licenses collected for recipe
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move do_kernel_link_vmlinux() from kernel-yocto.bbclass into
kernel.bbclass so that it's available to any kernel recipe.
Note that the task is not enabled by default in kernel-yocto.bbclass,
so don't enable by default in kernel.bbclass either. To enable, see
the example in linux-yocto.inc, ie:
addtask kernel_link_vmlinux after do_compile before do_install
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_kernel_link_vmlinux() task modifies the build directory (not
the source tree) and should not be skipped when externalsrc is being
used.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
SDK artifacts were being dumped alongside the image file dump.
However, the dump was not being triggered correctly, as it
should be attached as a postfunc to do_populate_sdk, rather than
do_rootfs, as do_rootfs doesn't always fire when populating
the SDK directory.
By adding a postfunc to do_populate_sdk, the postfunc is fired
in the correct situations.
Also fixes the directory which is scanned for SDK artifacts, as
this is different from the image file directory.
[YOCTO #7603]
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This removes unnecessary spaces inserted before semicolons in the
modified USERADD_PARAM_${PN} and GROUPADD_PARAM_${PN} variables. This
should not affect the handling of the variables as the only one that
actually sees the semicolons is the code in useradd.bbclass that uses
cut to split the variables at them, and any whitespace preceeding or
following the semicolons will be properly ignored.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Read and merge the passwd/group files before parsing the user and
group definitions. This means they will only be read once per
recipe. This solves a problem where if a user was definied in multiple
files, it could generate group definitions for groups that should not
be created. E.g., if the first passwd file read defines a user as:
foobar::1234::::
and the second passwd file defines it as:
foobar:::nogroup:The foobar user:/:/bin/sh
then a foobar group would be created even if the user will use the
nogroup as its primary group.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The original code was near impossible to follow, and missed a couple
of cases. For example, if one added the following line to the passwd
file specified in USERADD_UID_TABLES:
foobar:x:12345:nogroup::/:/bin/sh
and then specified the user as:
USERADD_PARAM_${PN} = "--system foobar"
one would then assume that the foobar user would be created with the
primary group set to nogroup. However, it was not (the primary group
would be foobar), and the only way to get it correct was to explicitly
add --gid nogroup to the USERADD_PARAM_${PN}.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The [<on_true>, <on_false>][not <condition>] construct may solve the
problem of implementing a conditional operator, but it is not very
readable. At least I find this:
uaargs.groupid = field[3] or uaargs.gid or uaargs.groupname
a lot more readable than this:
uaargs.groupid = [uaargs.gid, uaargs.groupname][not uaargs.gid]
uaargs.groupid = [field[3], uaargs.groupid][not field[3]]
Also, the official conditional operator since Python 2.5 (<on_true> if
<condition> else <on_false>) does not evaluate both <on_false> and
<on_true> as [<on_true>, <on_false>][not <condition>] does.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If --no-user-group is specified in USERADD_PARAM_${PN} for a user and
no --gid is specified, then we should not assume that the group name
for the user is the user name.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The useradd options --create-home/--no-create-home and
--user-group/--no-user-group are mutually exclusive and should be
treated as such.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The absolute path (/path/to/configure) caused VPATH in Makefile to be an
absolute path, and then it will be in elf files, use relative path to
run configure can fix the problem.
This will reduce a lot of buildpaths QA issues in a world buld.
[YOCTO #8894]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
kernel ARCH
For a bare-bone kernel recipe which specifies 32 bit x86 target,
a 64 bit .config will be generated from do_configure task when
building 32-bit qemux86, once all of these conditions are true:
* arch of host is x86_64
* kernel source tree used in build has commit ffee0de41 which
actually chooses i386 or x86_64 defconfig by asking host when
ARCH is "x86" (arch/x86/Makefile)
* bare-bone kernel recipe inherits directly from kernel without
other special treatments.
Build will fail because of the mismatched kernel architecture.
The patch sets ARCH i386 or x86_64 explicitly to configure
task to avoid this host contamination. Kernel artifact is also
changed so that it can map i386 and x64 back to arch/x86 when
needed.
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).
The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.
Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Create a separate task where we can just have the recursive dependencies
for the tasks corresponding to constructing the SDK content (i.e. from
the image contents). This avoids us recursing into dependencies from
buildtools and getting a bunch of nativesdk stuff, for example.
(This isn't an ideal way to have to implement it, but without
overcomplicating things on the BitBake side just for this use-case I
can't see a better way.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When using package_qa_handle_error(), we need to check QA_SANE and
give a fatal error if issues were raised. The main insane checks do
this, the anonymous python in insane does not, nor does the code in
package.bbclass. This was likely missed when the function was
introduced to package.bbclass.
The impact is that errors can be shown but don't stop the build
(do_package succeeds), the initial build fails due to the errors
shown but subsequent builds succeed. This is clearly broken.
The solution is to check QA_SANE in places we use the handle_error
function.
[YOCTO #8907]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
newline
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES. For instance, when using mingw32 a dependency
error will occur:
NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']
This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext
Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.
Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.
Also bring in some additional mappings to Debian architecture names
that weren't being handled.
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
prune_lockedsigs expects excluded_targets to be a list, whereas
previously it was passed in as a string.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Right now, the locked task hashes list for the extensible SDK locks
down only the sstate tasks.
Whilst asthetically pleasing, this gives two problems:
* Half the task are left floating meaning checksum mismatches
are a pain to debug
* The later code which copies relavent data files out the sstate
cache can't use any of this data.
This patch modifies things so all the checksums are listed in the locked
file. An exclusion of tasks probably makes more sense for the library
function rather than an allowed list.
The only sstate task being deliberaly excluded here was do_package
so add in a function to explictly exclude those sstate object files.
The net result of this that siginfo files for all tasks are included in
the SDK, which means commands like "bitbake -S printdiff" now function.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
xz gives *much* faster decompression times for the SDK which in itself
is a good reason to use it. It also gives better compression.
One downside is its slower but we care about the end user case
first, build performance secondary.
It also assumes the SDK user has a tar capable of understanding a xz
compressed file but that should be common enough now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the separation of do_rootfs, some rootfs references need changing
to image_complete.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I've heard complaints from people trying to create more interesting image
types about how hard it is to understand the rootfs/image generation code
and that its a pain to develop/test/debug.
Having looked at it myself, the internal construction of shell functions which
then gets passed into a multiprocessing pool is rather convoluted and it places
rather odd constraints on when variables are expanded. Its therefore no wonder
people find it confusing/complex.
This patch starts the process of splitting this up by separating out image
generation from the do_rootfs task into a new do_image task.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The user of the extensible SDK doesn't need to see these.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
During extensible SDK installtion, if the build system preparation step
fails we try to put something at the end of the environment setup script
to show an error when it is sourced, in case the user doesn't realise
that the partially-installed SDK is broken. However, an apostrophe in
the message (actually a single quote) appears to terminate the string
and therefore breaks the command. Drop it to avoid that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This was supposed to be removed by a previous patch but was readded.
Really remove it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The procces to do a wic image is to save a file with
variables required by wic and then call wic using this
file. Because this is external to bitbake if the vars
change, the image won't be rebuild; an example of such
is IMAGE_BOOT_FILES.
This patch adds these variables to vardeps of do_rootfs
when a wic image is build. This will rebuild the image
if a variable needed by wic changes.
[YOCTO #8693]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Now that x11vnc won't be in Sato images we don't want to test it.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
I was using bytes here for the sizes and writing out KiB as the suffix.
Change it to actually write out size values in KiB.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
missing in PACKAGE_ARCHS
* sometimes it's hard to see what's wrong here
* error message:
Error, the PACKAGE_ARCHS variable does not contain TUNE_PKGARCH (cortexa7t2hfhf-vfp-vfpv4-neon).
doesn't help much to understand what exactly went wrong and where
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Qt4 is no longer supportd upstream, but it is still needed for LSB compliancy.
Qt4 recipes have been moved to a separate meta-qt4 layer to be consistent with
meta-qt3 and meta-qt5.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Various prefuncs and flags and addtask statements make sense to belong together
in one clearer function now, this patch cleans things up a bit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reading image.bbclass is a little difficult as it has many post rootfs
helper functions and its hard to separate those from the core contents
of the rootfs/image code.
Moving it to a separate class would be one way of making it clearer
what these functions are. There are some comment layout improvements
but no code changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It was added in aa3141e979326a9d931ff03bad75923faa1d9dc6 back in 2008
but I don't see why multiple images would need this now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Jackie Huang
Andre McCurdy
Elliot Smith
Peter Kjellerstedt
Robert Yang
Jianxun Zhang
Otavio Salvador
Paul Eggleton
Mark Hatle
Matt Madison
Randy Witt
Mariano Lopez
Ross Burton
Martin Jansa
Alexander Kanavin