| Age | Commit message (Collapse) | Author |
|---|
|
python3-compression
* the /usr/lib/python3.5/_compression.py file is possibly incorrectly included
in python3-misc. This runtime dependency is needed in order to use e.g. gzip.py in runtime:
>>> import tarfile, zlib, gzip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/gzip.py", line 12, in <module>
import _compression
ImportError: No module named '_compression'
* at least python3-tests and lzma and bz2 still in python3-misc are using this as well:
$ grep -R import.*_compression tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-compression/usr/lib/python3.5/gzip.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/lzma.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_lzma.py:import _compression
and python3-tests are using it as well, so add new runtime dependency
on python3-compression
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
runqemu uses stty to change terminal settings to give users
better control to qemu. However, stty does not work when
runqemu is run directly or indirectly via oe-selftest in
a Docker container (presumably some problems with Docker's
pseudo-tty implementation).
The error reported is:
stty: 'standard input': Inappropriate ioctl for device
As runqemu recently moved to subprocess.check_call() for
stty calls we now get thrown an error and all runqemu
runs fail.
sys.stdin.isatty() does proper job in detecting if the stty
calls can work so we use that check before running the stty
subprocess operations.
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added support for removing directories using mdeltree
utility to Disk.del method
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added implementation of Disk.del method and wic_r
function that removes files from the vfat partition
using mdel utility.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added empty 'wic rm' command that does nothing.
The functionality will be added by the next commits.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added wic_rm_help and wic_rm_usage variables to help.py.
These variables contain help content that will be used in
'wic rm help' and 'wic rm --help' output.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add parser for 'wic rm' subcommand.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added implementation of Disk.copy method and wic_cp
function that copies files/directories to the vfat partition
of the partitioned image.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This method copies partition image into the wic image.
It will be used in 'wic cp' and 'wic rm' subcommands
to copy changed partition back into wic image.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed calculation of the dst file size using skip, seek and
length parameters. Current code does it incorrectly which
causes sparse_copy API to create unnecessary big output files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If lenght parameter is provided to sparse_copy call
it's mandatory to check if the output file is fully
written after reading unmapped block from input file.
If it's not done then sparse_copy can write more data
than specified length.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Renamed parameter offset->skip to match names of dd
parameters.
Changed affected sparse_copy calls.
Added explanation of the parameters to docstring.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added property that points to the mcopy executable.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added generic helper to use in property methods to
access commands in a lazy manner.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added empty 'wic cp' command that does nothing.
The functionality will be added by the next commits.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added wic_cp_help and wic_cp_usage variables to
help.py. These variables contain help content that
will be used in 'wic cp help' and 'wic cp --help'
output.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add parser for 'wic cp' subcommand and a custom argument type.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Implemented 'wic ls' functionality:
- list image partitions
- list directory content of vfat partitions
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added empty 'wic ls' command that does nothing.
The functionality will be added by the next commits.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added wic_ls_help and wic_ls_usage variables to
help.py. These variables contain help content that
will be used in 'wic ls help' and 'wic ls --help'
output.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added parser for 'wic ls' command.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Plugin code uses boot.img file name for an image file. If there are
two partitions that use bootimg-pcbios wic breaks with an error
"file already exists: boot.img"
Made image file name unique by adding wks like number to it to fix
the issue.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added parameter 'length' to specify amount of data
to write into destination file. This is useful when only
part of source file should be written into destination file.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixed bug in processing 'skip' parameter:
don't read input file if end of bmap block is less than skip
Simplified logic of positioning to the start of data inside a
partially skipped bmap block.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
RPC ports (also known as rpc program numbers) are values:
- given to rpcbind (aka portmapper) to allow nfsv3 clients that don't
know the tcp/udp port number of nfsd and mountd to look it up the
tcp/udp port number, and to
- allow a single transport (ie: tcp/udp port) to provide multiple
sunrpc services.
OE has carried patches to nfsutils & linux for some time to support the
mountprog & nfsprog options.
In the case of runqemu-export-rootfs, we don't need to use custom rpc
program numbers because runqemu-export-rootfs tells unfsd not to
register with the portmapper, and unfsd runs the nfs and mount rpc
services on tcp/udp ports unfsd binds itself (iow: the tcp/udp ports are
not shared in the sunrpc sense).
Linux's nfs client does not query rpcbind when tcp/udp port numbers are
specified (in net/sunrpc/clnt.c, call_bind checks for the tcp/udp port
with xprt_bound() and skips the call to rpcbind if xprtsock.c's
xs_setup_udp() or xs_setup_tcp() has found a non-zero tcp/udp port).
The program numbers _are_ sent over the mount & nfs protocol (really,
over sunrpc), and checked to match at both ends. As a result, even when
rpcbind is unused, using different program numbers in unfsd vs linux
nfs client causes mounts to fail (and nfsroot mounts to timeout).
The result is that specifying custom program numbers in
runqemu-export-rootfs doesn't solve any conflicts, it simply requires
that users of runqemu-export-rootfs carry a kernel patch & adds 2 extra
parameters to the kernel command line unnecessarily.
Change runqemu-export-rootfs to use the default program numbers.
For now, I have not dropped the custom program number patches to linux,
nfs-utils, and unfsd just in case someone is using them in a
non-runqemu-export-rootfs context.
CC: Bruce Ashfield <bruce.ashfield@gmail.com>
CC: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Cody P Schafer <dev@codyps.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If for any reason the parse_recipe fail in extract command
the process gets locked because Cooker is expecting the
finish event by tinfoil.
For example:
$ devtool extract remake /tmp/remake
ERROR: remake is unavailable:
remake was skipped: PREFERRED_PROVIDER_virtual/make set to make, not remake
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
I got a following error when I run buildstats-diff against a invalid
buildstats file:
| Traceback (most recent call last):
| File "/poky/scripts/buildstats-diff", line 548, in <module>
| sys.exit(main())
| File "/poky/scripts/buildstats-diff", line 534, in main
| bs1 = read_buildstats(args.buildstats1, args.multi)
| File "/poky/scripts/buildstats-diff", line 222, in read_buildstats
| return read_buildstats_dir(path)
| File "/poky/scripts/buildstats-diff", line 165, in read_buildstats_dir
| os.path.join(recipe_dir, task))]
| File "/poky/scripts/buildstats-diff", line 124, in read_buildstats_file
| bs_task['elapsed_time'] = end_time - start_time
| UnboundLocalError: local variable 'end_time' referenced before assignment
the root cause is that a task was terminated by me on the terminal,
so the generated buildstats file was invalid, supposing that it would
make the buildstats.sh fail, but the script should give more graceful
error messages.
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
We are using a temp directory, use the realpath for output log
to store the results in the original BUILDDIR.
[YOCTO #11571]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
If whatever layer tested fails returns 2 to indicate the
failure.
[YOCTO #11482]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Also clean up the logic in the script to be more Pythonic.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Also clean up the logic in the script to be more Pythonic.
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This was overlooked when msger was removed in 28014087b8091.
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
By default python logging module uses stderr as default stream
for output but is unix-like to use stdout instead, so change it.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
separated with dash
* I had some components where the MACHINE name was also included in PV of allarch recipe
which was tripping the script into saying that they have different signatures (because
for one MACHINE was the string in PV replaced with 'MACHINE' and not for other MACHINEs
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The new OEQA framework aims to re-use code into the different Test
components.
The previous oe-selftest implements it-self loading, run, and list test
cases in a non-standard way (unittest base) and other functionalities
like logging that is now on oeqa core. This ends on a compact oe-selftest
script.
All needed command line options was migrated but there are some of them
pending of implementation and others deprecated.
Deprecated options:
list-tags: The tag functionality into the old oeqa framework isn't
work, the selftest doesn't has tag decorators.
{run, list}-tests-by: Ambiguos options it accepts all the posibilites module,
class, name, id or tag.
Remaining to implement:
coverage: It enables covrage reports over a test run, currently isn't on
on use and some bugs [1], i filed a bug to add support to OEQA core module in
this way other Test components could enable it.
repository: It push XML results into a git repository and isn't in use,
i filed a bug to implement this into OEQA core module. [2]
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
The oe-selftest creates include files to store custom configuration to
make specific tests, every class executes a different test and may be
uses custom configuration.
So move to case class in order to simplify oe-selftest script and later
implement later a build folder per class.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
In order to maintain compatibility with oe-selftest, the
load_test_components needs to be re-used, so the script
executor needs to pass to only load components supported
by certain script (oe-test, oe-selftest).
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
Python function subprocess.call() returns the return value of the
executed process. If return values are not checked, errors may
go unnoticed and bad things can happen.
Change all callers of subprocess.call() which do not check for
the return value to use subprocess.check_call() which raises
CalledProcessError if the subprocess returns with non-zero value.
https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module
All users of the function were found with:
$ git grep "subprocess\.call" | \
egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call'
Tested similar patch on top of yocto jethro. Only compile tested
core-image-minimal on poky master branch.
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
isoimage-isohybrid plugin fails with this error when
trying to find initrd image:
initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, target_arch, image_type))[0]
IndexError: list index out of range
as it uses TRANSLATED_TARGET_ARCH value as part of the image.
This approach stopped to work due to changes in oe core code.
initrd file name is made using MACHINE variable. wic can't get value
of this variable as it's not included into bitbake -e output.
Used basename of deploy dir as MACHINE value to fix the breakage.
(From OE-Core rev: cfbb3cc1279ea88ca3e2867f8a409c5120aa1f05)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The remote script is run with `set -e`, so doing rm without -f on a
$file that is already gone will exit the whole script, failing any
redeployment. Assume a use case where packages sometimes produces
certain test binaries stored on volatile media (tmpfs), and where the
system is occasionally rebooted.
Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Either both or none of the paths must be passed through
os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will
cause unintentional mismatches even when B = "${S}".
Using os.path.abspath for both seems more likely to be correct as that
will also handle the case where ${B} != ${S} but the abspaths are
equal.
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
find may be provided by busybox, which might be compiled without support
for -exec.
Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
This pre runner will serve to allow Test components executes code
previously of the run a suite.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Included error output from qemu-system into the runqemu error message.
Made error output more visible by printing new line before it.
[YOCTO #11542]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move config-*/Makefile in libdir from misc package to dev package for
python3, because it is only needed in development process.
Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Perf is a tool build from the kernel source, which is normally available
in /work-shared/..., but when devtool is used to modify the kernel
source code, perf is not buildable since it gets an error about being unable
to add a depends to a non-exisit task do_patch.
This patch removes do_patch from the SRCTREECOVEREDTASKS and creates an empty
do_patch task to enable the VarFlags code to have someplace to attach depends
information to.
[YOCT #11120]
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Since recipe-specific sysroots were implemented, devtool add and devtool
upgrade operations that fetch from a URL that requires native sysroot
dependencies will fail to work as there is no recipe-specific sysroot
set up for them during fetching. An example was any URL pointing to a
tarball compressed with xz, e.g. devtool upgrade on gnutls.
The most expedient way to fix this is to set up a dummy recipe-specific
sysroot to use for the fetch/unpack operations. We do this in the same
manner as bitbake -b does, so we're just taking all of the sysroot
components available and creating a sysroot from those rather than
ensuring the correct dependencies are there - this means that we're
still going to have problems if e.g. xz-native hasn't been built yet,
but that issue will be trickier to solve and is tracked separately.
Fixes [YOCTO #11474].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Buildstats from oe-build-perf-test results have been optimized to not
have child rusage values at all. There, rusage is the sum of parent and
child rusage values. This patch makes buildstats-diff compatible with
this format.
[YOCTO #11355]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
For dumping buildstats from the test runs being reported. The output
directory where buildstats are copied is 'oe-build-perf-buildstats/'.
Buildstats can be then further analyzed with buildstats-diff script, for
example.
[YOCTO #11355]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
|
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
|
Martin Jansa
Mikko Ylinen
Ed Bartosh
Cody P Schafer
Aníbal Limón
Ming Liu
Ross Burton
Kristian Amlie
Leonardo Sandoval
Mikko Rapeli
Daniel Lublin
Ola x Nilsson
Li Zhou
Saul Wold
Paul Eggleton
Markus Lehtonen