| Age | Commit message (Collapse) | Author |
|---|
|
With the enhanced functionality, the term "compression" is no longer
accurate, because the mechanism also gets used for conversion
operations that do not actually compress data.
It is possible to remove this naming problem in a backward-compatible
manner by including COMPRESSIONTYPES in CONVERSIONTYPES and checking for
the old COMPRESS_CMD/DEPENDS as fallbacks.
[YOCTO #9346]
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This task runs all functions in IMAGE_QA_COMMANDS after the image
construction has completed in order to validate the resulting image.
Image sanity checks should either be Python functions which raise
bb.build.FuncFailed on failure or shell functions with return a
non-zero exit code.
Python functions may instead raise an oe.utils.ImageQAFailed
Exception which takes an extra argument, a description of the
failure.
python image_check_python_ok () {
if True:
raise bb.build.FuncFailed('This check always fails')
else:
bb.note("Nothing to see here")
}
image_check_shell_ok () {
if true
exit 1
else
exit 0
fi
}
[YOCTO #9448]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Use the new task progress functionality to report progress during
do_rootfs. This is a little coarse and ideally we would have some
progress within the installation section, but it's better than
nothing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There's no need to chdir() when creating image symlinks, and using chdir()
changes the state for future tasks.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Do exact match for rootfs type, instead of pattern match, to avoid
unexpected build error due to redundant rootfs type build.
E.g. when building ext2.gz.u-boot, both .gz.u-boot and .u-boot are matched,
the following build error will appear, actually .u-boot is not needed.
| mkimage: Can't open .../core-image-minimal-<machine>-<yyyymmddhhmmss>.rootfs.ext2.gz: No such file or directory
Signed-off-by: Zhenhua Luo <zhenhua.luo@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If /init is just a symlink to /sbin/init, debugfs creation
fails with the following error:
ERROR: Error: The image creation script '<...>/debugfs.create_image.cpio' returned 1:
touch: cannot touch '<...>/cpio_append/init': Permission denied
WARNING: exit code 1 from a shell command.
ERROR: Function failed: do_rootfs
The reason is that IMAGE_CMD_cpio() is run twice on the same
WORKDIR. The first run creates a symlink in WORKDIR/cpio_append/init
to point to /sbin/init, while the 2nd run then tries to 'touch'
that link, which will fail, of course since /sbin/init is not
usually writable by non-root users.
Fix this by providing knowledge to the IMAGE_CMD_xxx() scripts
with regards to the fact that they are being executed in the
context of debugfs creation. The IMAGE_CMD_cpio() can now be
intelligent in the sense that it can avoid all additional symlink
handling during the debugfs run. The symlinks do not need to
be part of the debugfs, so we can skip that part altogether
in that case.
Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When a symlink does not get created, it is useful for debugging to log
what would have been created and why it was skipped.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
When a derived distro adds a certain type, say zip, to
COMPRESSIONTYPES and later OE-core does the same, we end up with the
type being listed twice, and that would have undesired effects
(commands generated twice).
So to support such loosely coupled extension, we de-duplicated the
list of types first.
Alternatively, such a situation could also be treated as error. But that
seems unnecessary because typically commands for the same type will also
do the same thing.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ed Bartosh <eduard.bartosh@intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
remain_features uses a dict which means the order is not deterministic. This
can lead to the task hash changing depending on the state of the memory at
parse time. This is particularly noticeable under python v3.
Since the dict is helpful in constructing the data, pass the data through
sort() so the order is always deterministic.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In case of chained conversion methods are used via COMPRESS_CMD_*
there is chance that some of steps would be executed multiple times.
[YOCTO #9482]
Signed-off-by: Alexander D. Kanevskiy <kad@kad.name>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
IMAGE_CMD_xxx commands are always inlined within do_image_xxx.
When IMAGE_CMD_xxx is defined as a function (e.g. IMAGE_CMD_btrfs,
IMAGE_CMD_cpio, etc), a redundant copy of the function will be emitted
by default. Remove IMAGE_CMD_xxx 'func' flags to prevent that.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
[YOCTO #9487]
The debug filesystem file name is ending in "debug_tar", it should be simply
"tar". Strip the "debug_" piece as necessary.
To avoid deleting the tar ball, when we've asked for just the tarball we need
to check 't' and not 'realt'.
The two hunks were suggested by RP. I've implemented and verify they work
with the settings:
PACKAGE_CLASSES = "package_rpm"
IMAGE_GEN_DEBUGFS = '1'
IMAGE_FSTYPES_DEBUGFS = "tar.bz2"
IMAGE_FSTYPES_DEBUGFS = "tar.gz"
and
IMAGE_FSTYPES_DEBUGFS = "tar"
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
rootfs_command_variables list
Remove duplicate ROOTFS_POSTPROCESS_COMMAND in the rootfs_command_variables list.
Add DEB_PREPROCESS_COMMANDS and DEB_POSTPROCESS_COMMANDS to rootfs_command_variables
list for consistency with the RPM_ and OPKG_ versions of those variables.
Note: the package manager specific pre and post process commands
may removed entirely in Yocto 2.2 or later.
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
There's some code dotted around OE that uses (a, b)[foo < bar] instead of the
more idiomatic "test and a or b". Or in this case, just max().
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The debugfs prefix is striped from t, but not from baset.
Therefore baset never matches t.
Signed-off-by: Freudiger Raphael <raphael.freudiger@siemens.com>
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move the common code to live_vm_common.bbclass and remove duplicated ones.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.
This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently do_wicenv task is run for all images. However, its
result is used only to produce wic image. It's better to
run this task only for wic images. If another rootfs is
required to produce wic image, dependency to its do_wicenv
must be added to the wic image recipy.
Stopped running do_wicenv for all images. Added explicit
dependency to this task in wic-image-minimal recipe.
[YOCTO #9095]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Make it can build vm and live (e.g., iso + vmdk) together as we did
for syslinux.
* GRUBCFG -> GRUB_CFG as other GRUB_FOO vars
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Fixed:
- Found potential conflicted var LABELS ...
Set LABELS to "boot install" would build out broken images when build
vm + live together, use set_live_vm_vars() to fix the problem.
- Use ROOT and LABEL in boot-directdisk.bbclass and image-foo.bbclass,
they are not only used by syslinux.bbclass, but also grub-efi.bbclass,
add "SYSLINUX_" prefix would mislead users.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The list of variables influencing do_rootfs was not updated when
introducing ROOTFS_POSTUNINSTALL_COMMAND. As a result, making changes
in commands listed there or the variables they depend on did not trigger
a re-run of do_rootfs.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
It makes sense to use the compression mechanism also for conversion,
for example of a whole-disk image into .vdi (VirtualBox). That part
already works, like this:
COMPRESSIONTYPES_append = " vdi"
COMPRESS_CMD_vdi = "qemu-img convert -O vdi ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.vdi"
IMAGE_DEPENDS_vdi = "qemu-native"
But then it also makes sense to allow compressing the resulting image,
which only works after enhancing the image.bbclass.
For example, suppose a custom image command produces "dsk" images. Then
it becomes possible to set
IMAGE_FSTYPES = " dsk.xz dsk.vdi.xz"
and do_image_dsk will automatically produce the intermediate images,
convert to dsk.xz resp. dsk.vdi -> dsk.vdi.xz and delete all
intermediate images. Symlinks are also set correctly.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The patch for making the .rootfs configurable was incomplete: in the
python create_symlinks() method the new variable must be expanded
explicitly.
Not doing so broke the symlink creation and that led to hard build
failures in image types depending on the boot-directdisk.bbclass (like
qcow2) because the build_boot_dd() method relied on the symlink.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By default, the image file name contains ".rootfs" to distinguish the
image file from other files created during image building. However,
for certain image types (for example, .hddimg) the ".rootfs" suffix is
redundant because the type suffix alone already uniquely identifies
the main image file (core-image-minimal-intel-corei7-64.hddimg instead
of core-image-minimal-intel-corei7-64.rootfs.hddimg).
With this change, distros that prefer the shorter image name can
override the .rootfs suffix unconditionally with
IMAGE_NAME_SUFFIX ?= '' in their distro configuration
or with some condition check like this:
python () {
if <whole-disk image format active>:
d.setVar('IMAGE_NAME_SUFFIX', '')
}
The exact logic when to remove the extra suffix depends on the distro
and how it enables its own image type.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
do_image can modify the content of the rootfs directory so we need to run
do_rootfs_wicenv after do_image compeltes or the command can fail.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, do_rootfs depends on variables like SDK_OS, SDK_OUTPUT, etc.
And changing variables like POPULATE_SDK_POST_HOST_COMMAND doesn't cause
do_populate_sdk to rerun.
This patch separates variables so that do_rootfs and do_populate_sdk could
correctly depend on their related variables.
[YOCTO #8670]
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Usually, the initramfs' maxsize can be 1/2 of ram size since modern
kernel uses tmpfs as initramfs by dafault, and tmpfs allocates 1/2 of
ram by default at boot time, which will be used to locate the initramfs.
Set INITRAMFS_MAXSIZE to 131072K (128M) by default (ram 256M), the
initramfs is small usually, for example, core-image-minimal-initramfs is
about 21M (uncompressed, 17M * 1.3) by default, but if the user add a
lot pkgs to initramfs, we can error and stop to let the user know ealier
rather than fail to boot (e.g., OOM-killer) at boot time.
Please see the bug for more info:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=5963
[YOCTO #5963]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Similarly to "-", "." doesn't work well in task names but is used in
some real world image classes. Work around this with some replacements
for now to unbreak layers.
(Issues don't show themselves until runtime, e.g. with --dry-run)
Tested-By: Otavio Salvador <otavio.salvador@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Shell function names can't contain '-' characters, which means our image
task names also can't. Add some mapping to use '_' instead of the '-' so
images like "rpi-sdimg" work again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic command can be used externally but for this to work, the wic
environment file needs to be present. Therefore write this out
universally, it runs in parallel with other image construction so
any performance implications are negligible.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, iso and hddimg links don't respect RM_OLD_IMAGE. This
updates them to use the common symlinks code so that they behave
like the rest of the system.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similarly to DATETIME, don't expand TMPDIR in image commands.
This ensures some of the stamp comparisons we make in the
QA tests work correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The do_image_* tasks contained the expanded version of DATETIME. Due
to the expansion, we couldn't exclude the value from the task checksum
which meant the task would rerun.
We fix this by deleting the DATETIME value during expansion so we don't
expand any references to at that time. This means the task's hash can be
stable rather than having hardcoded date/time values. It will get expanded
at execution time.
This also fixes errors shown by -S:
NOTE: Reparsing files to collect dependency data
Writing locked sigs to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/locked-sigs.inc
ERROR: Bitbake's cached basehash does not match the one we just generated (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-sato/images/core-image-sato.bb.do_image_tar)!
ERROR: The mismatched hashes were 77872792556367f1dde49a1425caf1a0 and 9bb0aca6286ab7dd22d3c69964beb665
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The wic environment function needs to run after the rootfs size is
setup. We move this code to a specific task, and depend on that task
from the wic images and other places its needed.
This fixes:
======================================================================
FAIL: test_image_env (oeqa.selftest.wic.Wic)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f
return func(*args, **kwargs)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env
self.assertTrue(var in content, "%s is not in .env file" % var)
AssertionError: False is not true : ROOTFS_SIZE is not in .env file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
IMAGE_TYPES_MASKED support was accidentally removed. The original
idea behind it was to remove some of the hardcoding in the core
image code, so do that for image-live and ensure the dependency
and masked variables correctly reflect the needs of the class.
This means we can remove all the hardcoded special cases since
image-vm already has the needed markup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a base image type has an implicit dependency (from IMAGE_TYPEDEP)
this has to be taken into account. This is a regression introduced by
OE-Core:c2dab18 (image: Create separate tasks for rootfs construction).
The issue has been found when building meta-fsl-arm based images which
does not include the rootfs image type explicitly in IMAGE_FSTYPES but
instead is added, using IMAGE_TYPEDEP, for the 'sdcard.gz' image.
Reported-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
populate_sdk_ext requires uninative support, which is only available on
glibc based SDKMACHINES. For instance, when using mingw32 a dependency
error will occur:
NOTE: Runtime target 'nativesdk-glibc' is unbuildable, removing...
ERROR: Required build target 'core-image-minimal' has no buildable providers.
Missing or unbuildable dependency chain was: ['core-image-minimal', 'uninative-tarball', 'nativesdk-glibc']
This is dues to populate_sdk_ext.bbclass having:
do_populate_sdk_ext[depends] += "buildtools-tarball:do_populate_sdk uninative-tarball:do_populate_sdk"
addtask populate_sdk_ext
Since bitbake can't determine for dependency resolution if the task is going
to be run yet, it blows up and says it simply can't be resolved.
Workaround this problem by making the inherit conditional on the SDK_OS
containing 'linux'.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After the separation of do_rootfs, some rootfs references need changing
to image_complete.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I've heard complaints from people trying to create more interesting image
types about how hard it is to understand the rootfs/image generation code
and that its a pain to develop/test/debug.
Having looked at it myself, the internal construction of shell functions which
then gets passed into a multiprocessing pool is rather convoluted and it places
rather odd constraints on when variables are expanded. Its therefore no wonder
people find it confusing/complex.
This patch starts the process of splitting this up by separating out image
generation from the do_rootfs task into a new do_image task.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This was supposed to be removed by a previous patch but was readded.
Really remove it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Various prefuncs and flags and addtask statements make sense to belong together
in one clearer function now, this patch cleans things up a bit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Reading image.bbclass is a little difficult as it has many post rootfs
helper functions and its hard to separate those from the core contents
of the rootfs/image code.
Moving it to a separate class would be one way of making it clearer
what these functions are. There are some comment layout improvements
but no code changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It was added in aa3141e979326a9d931ff03bad75923faa1d9dc6 back in 2008
but I don't see why multiple images would need this now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the same way it's done for openssh.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
regardless of init system in use
Previously it was done only if sysvinit was in DISTRO_FEATURES.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When image-vm.bbclass was introduced, it indirectly also introduced a
".hdddirect" image type based on boot-directdisk.bbclass. However, one
could only get that image when also enabling at least one of the
virtual machine images.
The .hdddirect images are useful by themselves. By registering
image-vm.bbclass as implementation of it, it becomes possible to
select them with:
IMAGE_FSTYPES = "hdddirect"
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
We want do_rootfs to rerun if the fstype or compression commands or
dependencies change for any of our configured fstypes (IMAGE_FSTYPES).
IMAGE_TYPEDEP isn't explicitly handled, as it's traversed already, so the end
result will change if it does, and we only really care about the results, not
how we got there. This uses oe.image.Image()._get_image_types() to get the
info about the image and compression types in use.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This function is intended to be used in ROOTFS_POSTPROCESS_COMMAND, and checks
for any paths outside of /home which are owned by the user running bitbake.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Patrick Ohly
Joshua Lock
Paul Eggleton
Ross Burton
Zhenhua Luo
André Draszik
Richard Purdie
Alexander D. Kanevskiy
Andre McCurdy
Mark Hatle
Bill Randle
Freudiger Raphael
Robert Yang
Peter Kjellerstedt
Ed Bartosh
Chen Qi
Otavio Salvador
Alexander Kanavin
Christopher Larson