| Age | Commit message (Collapse) | Author |
|---|
|
This patch splits the code in lib/oe/image into separate tasks, one per
image type. This removes the need for the simple task graph code and defers
to the bitbake task management code to handle this instead.
This is a good step forward in splitting up the monolithic code and starting
to make it more accessible to people.
It should also make it easier for people to hook in other tasks and processes
into the rootfs code.
Incidentally, the reason this code was all combined originally was due to
limitations of fakeroot where if you exited the session, you lost permissions
data. With pseudo this constraint was removed.
We did start to rework the rootfs/image code previously and got so far with
untangling it however we did prioritise some performance tweaks over splitting
into separate tasks and in hindsight, this was a mistake and should have been done
the other way around. That work was suspended due to changes in the people working
on the project but this split has always been intended, now is the time to finish
it IMO.
There were some side effects of doing this:
* The symlink for the manifest moves to the rootfs-postcommands class and into
the manifest function.
* There is no seperate "symlink removal" and "symlink creation", they are merged
* The date/time stamps of the manifest and the built images can now be different since
the tasks can be run separately and the datetime stamp will then be different
between do_rootfs and the do_image_* tasks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
As the next step in splitting up do_image, move the pre and post processing
commands to separate tasks. This also creates the do_image_complete task
which acts as the end marker task for image generation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If multiple images are being built simultaneously against the same
sysroot then the call to os.makedirs in Image._write_wic_env can fail
with:
File: '.../meta/lib/oe/image.py', lineno: 341, function: _write_wic_env
0337: """
0338: stdir = self.d.getVar('STAGING_DIR_TARGET', True)
0339: outdir = os.path.join(stdir, 'imgdata')
0340: if not os.path.exists(outdir):
*** 0341: os.makedirs(outdir)
0342: basename = self.d.getVar('IMAGE_BASENAME', True)
0343: with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
0344: for var in self.d.getVar('WICVARS', True).split():
0345: value = self.d.getVar(var, True)
File: '/usr/lib/python2.7/os.py', lineno: 157, function: makedirs
0153: if e.errno != errno.EEXIST:
0154: raise
0155: if tail == curdir: # xxx/newdir/. exists if xxx/newdir exists
0156: return
*** 0157: mkdir(name, mode)
0158:
0159:def removedirs(name):
0160: """removedirs(path)
0161:
Exception: OSError: [Errno 17] File exists: '.../tmp-glibc/sysroots/cheetah/imgdata'
Using bb.utils.mkdirhier() protects against this.
There's also little point in checking to see if the directory already
exists - we might as well just try and create it regardless.
Once the directory has been created, there's no race on the actual file
since the filename contains IMAGE_BASENAME.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Creating a file for every image containing a few variables isn't
necessary if wic is not being used, so don't write the file if WICVARS
is empty.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This doesn't cause any issues right now but it make sense to standardise
on consistently using strings in the data store.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The dependency code needs to also include the dependency of base
types. For example:
- sdcard.gz image with ext4
The dependency chain needs to include:
- sdcard
- ext4
- gz
Until this change, the ext4 dependency were not being taken into
account when using the compressed one.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The companion debug filesystem, enabled with IMAGE_GEN_DEBUGFS, was
creating the companion filesystem but was missing the code to actually
package it into a usable filesystem.
The code (and associated documentation) will allow the debugfs to generate a
companion tarball or other image.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a floating point multiplication done of a base image size
and an "overhead factor", which is currently rounded up to the next
integer. If the multiplication results in a whole number, the value
will still be a float. When this float is used to generate a shell
script, a buggy script is generated.
Fix this by always forcing to an integer.
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When IMAGE_LINK_NAME is empty, OE will try to create a "blank" link instead
of just skipping it. The code checks for "link_name is not None" which will
never evaluate to true. Change the test to a simple "if link_name:" so it
no longer attempt to create links when the variable is an empty string.
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Renamed this function as it's too generic name for it.
It writes variables, which are used by wic to .env file,
so _write_wic_env is better name for it.
Thanks Christopher Larson for poining out to this.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Write set of bitbake variables used by wic into
build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in
meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without
running 'bitbake -e'.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Let's add output of image creation script to the bitbake log
as it can contain useful information.
One good example of such an information is wic report about
artifacts and .wks file used for image creation.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This variable is going to be used by wic to set partition
size. Setting it in image.py makes it possible for wic to
use it without calculating it again.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
The following IMAGES_FSTYPES defaulted to ext3:
"vmdk", "vdi", "qcow2", "live", "iso", "hddimg"
This patch changes the default for those IMAGES_FSTYPES to
ext4 in order to bring the images more in line with other BSPs.
Besides improvements in performance and reliability ext4 provides
additional functionality as well (option to turn off the journaling,
dynamic resizing of VDI volumes etc.).
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added support for VirtualBox VDI format. The support was
implemented by merging with the already existing VMDK support
for VM player by creating a new class image-vm.bbclass.
This class replaces the previous VMDK only image-vmdk.class.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
An invalid value in IMAGE_FSTYPES was triggering a traceback. Add a
check and a reasonable error message instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
When computing the dependency graph for the image generation, we need
to take into account the compression type and identify the base type
it relates to. This allow for a more robust graph generation even when
using composed image types.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The graph sorting algorithm for image dependencies does a look for an
occurrence of a searched string instead of comparing the chunk to the
searched string. This leads to the problem that ubifs is recognized as ubi aswell.
This fixes this by splitting up the string into chunks.
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Check the rootfs size against IMAGE_ROOTFS_MAXSIZE (if set)
* Add comments for IMAGE_ROOTFS_SIZE to not confuse with IMAGE_ROOTFS_MAXSIZE
[YOCTO #2610]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
When RM_OLD_IMAGE = "1", we delete old images but we didn't check they
actually exist...
[YOCTO #6029]
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
calling check_output()
If a command executed with subprocess.check_output() fails, the
subprocess.CalledProcessError.output contains only STDOUT and the user
needs to check the log.do_rootfs to see any other details.
This commit forwards stderr to stdout so that, in case of failure, the
entire error output will be displayed in terminal.
[YOCTO #5902]
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
This commit adds a dependency mechanism to image creation, so that we can
split the images creation execution in groups, that can be executed in
parallel, having the dependencies satisfied in the same time. The old
code didn't need this since everything was serialized.
Technically, it adds a dependency graph topological sort class that the
main Image class can use to sort out the dependencies.
Images that have dependencies have to declare them using the NEW
IMAGE_TYPEDEP variable, like in the example below:
For:
IMAGE_FSTYPES = "i1 i2 i3 i4 i5"
IMAGE_TYPEDEP_i4 = "i2"
IMAGE_TYPEDEP_i5 = "i6 i4"
IMAGE_TYPEDEP_i6 = "i7"
IMAGE_TYPEDEP_i7 = "i2"
We'll get the following image groups, sorted out by their dependencies:
[['i1', 'i3', 'i2'], ['i4', 'i7'], ['i6'], ['i5']]
The algorithm can probably be optimized but, given the small size of the
graphs, it'll do.
[YOCTO #5830]
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
|
|
This reverts commit 3f49597225a58965124503ca5f3cc4011b04b3c0.
This change appears to cause more problems than it fixes since the
compression commands usually work in the deploy dir but the archive ones
have always worked in the rootfs dir (which is clear from the tar command
we use).
|
|
The working directory needs to be changed before the image creation
commands instead of afterwards.
Signed-off-by: Jonathan Liu <net147@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previous commit added support for sum 'compression' (rather
postprocessing) of jffs2 images. Drop support for sum.jffs2 image type.
Signed-off-by: Dmitry Eremin-Solenikov <dmitry_eremin@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Print the entire command output in case of errors.
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
|
|
The IMAGE_ROOTFS_EXTRA_SPACE value was not correctly added to the base
size.
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
|
|
The value of IMAGE_ROOTFS_EXTRA_SPACE is "0 + 51200",
we should use eval rather than int in python.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
|
|
This will replace the old bash image creation code. This needs the
rootfs to be already generated in order to work.
Usage:
Image(d).create()
or using the provided wrapper function:
create_image(d)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
|
Richard Purdie
Mike Crowe
Otavio Salvador
Mark Hatle
Patrick Williams
Mike Looijmans
Ed Bartosh
Juro Bystricky
Paul Eggleton
Pascal Bach
Robert Yang
Laurentiu Palcu
Jonathan Liu
Dmitry Eremin-Solenikov
Hongxu Jia