aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2017-06-14wic: add mcopy propertyEd Bartosh
Added property that points to the mcopy executable. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add Disk._prop helperEd Bartosh
Added generic helper to use in property methods to access commands in a lazy manner. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add 'wic cp' commandEd Bartosh
Added empty 'wic cp' command that does nothing. The functionality will be added by the next commits. [YOCTO #11283] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add help and usage content for 'wic cp'Ed Bartosh
Added wic_cp_help and wic_cp_usage variables to help.py. These variables contain help content that will be used in 'wic cp help' and 'wic cp --help' output. [YOCTO #11283] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add wic_init_parser_cpEd Bartosh
Add parser for 'wic cp' subcommand and a custom argument type. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14engine: implement listing wic imagesEd Bartosh
Implemented 'wic ls' functionality: - list image partitions - list directory content of vfat partitions [YOCTO #11283] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add 'wic ls' commandEd Bartosh
Added empty 'wic ls' command that does nothing. The functionality will be added by the next commits. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add help and usage content for 'wic ls'Ed Bartosh
Added wic_ls_help and wic_ls_usage variables to help.py. These variables contain help content that will be used in 'wic ls help' and 'wic ls --help' output. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14wic: add wic_init_parser_lsEd Bartosh
Added parser for 'wic ls' command. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14bootimg-pcbios: make boot image file uniqueEd Bartosh
Plugin code uses boot.img file name for an image file. If there are two partitions that use bootimg-pcbios wic breaks with an error "file already exists: boot.img" Made image file name unique by adding wks like number to it to fix the issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14filemap: add parameter 'length' to sparse_copyEd Bartosh
Added parameter 'length' to specify amount of data to write into destination file. This is useful when only part of source file should be written into destination file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14filemap: fix skip logicEd Bartosh
Fixed bug in processing 'skip' parameter: don't read input file if end of bmap block is less than skip Simplified logic of positioning to the start of data inside a partially skipped bmap block. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12runqemu-export-rootfs: don't change RPC portsCody P Schafer
RPC ports (also known as rpc program numbers) are values: - given to rpcbind (aka portmapper) to allow nfsv3 clients that don't know the tcp/udp port number of nfsd and mountd to look it up the tcp/udp port number, and to - allow a single transport (ie: tcp/udp port) to provide multiple sunrpc services. OE has carried patches to nfsutils & linux for some time to support the mountprog & nfsprog options. In the case of runqemu-export-rootfs, we don't need to use custom rpc program numbers because runqemu-export-rootfs tells unfsd not to register with the portmapper, and unfsd runs the nfs and mount rpc services on tcp/udp ports unfsd binds itself (iow: the tcp/udp ports are not shared in the sunrpc sense). Linux's nfs client does not query rpcbind when tcp/udp port numbers are specified (in net/sunrpc/clnt.c, call_bind checks for the tcp/udp port with xprt_bound() and skips the call to rpcbind if xprtsock.c's xs_setup_udp() or xs_setup_tcp() has found a non-zero tcp/udp port). The program numbers _are_ sent over the mount & nfs protocol (really, over sunrpc), and checked to match at both ends. As a result, even when rpcbind is unused, using different program numbers in unfsd vs linux nfs client causes mounts to fail (and nfsroot mounts to timeout). The result is that specifying custom program numbers in runqemu-export-rootfs doesn't solve any conflicts, it simply requires that users of runqemu-export-rootfs carry a kernel patch & adds 2 extra parameters to the kernel command line unnecessarily. Change runqemu-export-rootfs to use the default program numbers. For now, I have not dropped the custom program number patches to linux, nfs-utils, and unfsd just in case someone is using them in a non-runqemu-export-rootfs context. CC: Bruce Ashfield <bruce.ashfield@gmail.com> CC: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Cody P Schafer <dev@codyps.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-12devtool/standard: Fix lock in _prep_extract_operationAníbal Limón
If for any reason the parse_recipe fail in extract command the process gets locked because Cooker is expecting the finish event by tinfoil. For example: $ devtool extract remake /tmp/remake ERROR: remake is unavailable: remake was skipped: PREFERRED_PROVIDER_virtual/make set to make, not remake Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-09buildstats-diff: show more graceful error messagesMing Liu
I got a following error when I run buildstats-diff against a invalid buildstats file: | Traceback (most recent call last): | File "/poky/scripts/buildstats-diff", line 548, in <module> | sys.exit(main()) | File "/poky/scripts/buildstats-diff", line 534, in main | bs1 = read_buildstats(args.buildstats1, args.multi) | File "/poky/scripts/buildstats-diff", line 222, in read_buildstats | return read_buildstats_dir(path) | File "/poky/scripts/buildstats-diff", line 165, in read_buildstats_dir | os.path.join(recipe_dir, task))] | File "/poky/scripts/buildstats-diff", line 124, in read_buildstats_file | bs_task['elapsed_time'] = end_time - start_time | UnboundLocalError: local variable 'end_time' referenced before assignment the root cause is that a task was terminated by me on the terminal, so the generated buildstats file was invalid, supposing that it would make the buildstats.sh fail, but the script should give more graceful error messages. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-09scripts/yocto-compat-layer-wrapper: Use realpath of output_logAníbal Limón
We are using a temp directory, use the realpath for output log to store the results in the original BUILDDIR. [YOCTO #11571] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-09scripts/yocto-compat-layer.py: Return non-zero when layer test failAníbal Limón
If whatever layer tested fails returns 2 to indicate the failure. [YOCTO #11482] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-06python3: add python3-modules-native to RPROVIDES for python3-nativeRoss Burton
Also clean up the logic in the script to be more Pythonic. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-06python: add python-modules-native to RPROVIDES for python-nativeRoss Burton
Also clean up the logic in the script to be more Pythonic. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-06wic: Remove obsolete reference to msger logger.Kristian Amlie
This was overlooked when msger was removed in 28014087b8091. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-05scripts/oe-{self,}test: Logger change default stream to stdoutAníbal Limón
By default python logging module uses stderr as default stream for output but is unix-like to use stdout instead, so change it. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-06-05sstate-diff-machines.sh: Replace MACHINE_ARCH only at the beginning and ↵Martin Jansa
separated with dash * I had some components where the MACHINE name was also included in PV of allarch recipe which was tripping the script into saying that they have different signatures (because for one MACHINE was the string in PV replaced with 'MACHINE' and not for other MACHINEs Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05scripts/oe-selftest: Migrate to new framework into oeqa.selftest.contextLeonardo Sandoval
The new OEQA framework aims to re-use code into the different Test components. The previous oe-selftest implements it-self loading, run, and list test cases in a non-standard way (unittest base) and other functionalities like logging that is now on oeqa core. This ends on a compact oe-selftest script. All needed command line options was migrated but there are some of them pending of implementation and others deprecated. Deprecated options: list-tags: The tag functionality into the old oeqa framework isn't work, the selftest doesn't has tag decorators. {run, list}-tests-by: Ambiguos options it accepts all the posibilites module, class, name, id or tag. Remaining to implement: coverage: It enables covrage reports over a test run, currently isn't on on use and some bugs [1], i filed a bug to add support to OEQA core module in this way other Test components could enable it. repository: It push XML results into a git repository and isn't in use, i filed a bug to implement this into OEQA core module. [2] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0 [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0 Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-06-05scripts/oe-selftest: Move {add,remove}_include files to caseLeonardo Sandoval
The oe-selftest creates include files to store custom configuration to make specific tests, every class executes a different test and may be uses custom configuration. So move to case class in order to simplify oe-selftest script and later implement later a build folder per class. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-06-05scripts/oe-test: Move load_test_components to oeqa.utilsAníbal Limón
In order to maintain compatibility with oe-selftest, the load_test_components needs to be re-used, so the script executor needs to pass to only load components supported by certain script (oe-test, oe-selftest). Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-06-03scripts: Fix return value checks from subprocess.call()'sMikko Rapeli
Python function subprocess.call() returns the return value of the executed process. If return values are not checked, errors may go unnoticed and bad things can happen. Change all callers of subprocess.call() which do not check for the return value to use subprocess.check_call() which raises CalledProcessError if the subprocess returns with non-zero value. https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module All users of the function were found with: $ git grep "subprocess\.call" | \ egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call' Tested similar patch on top of yocto jethro. Only compile tested core-image-minimal on poky master branch. Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-03isoimage-isohybrid: don't use TRANSLATED_TARGET_ARCHEd Bartosh
isoimage-isohybrid plugin fails with this error when trying to find initrd image: initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, target_arch, image_type))[0] IndexError: list index out of range as it uses TRANSLATED_TARGET_ARCH value as part of the image. This approach stopped to work due to changes in oe core code. initrd file name is made using MACHINE variable. wic can't get value of this variable as it's not included into bitbake -e output. Used basename of deploy dir as MACHINE value to fix the breakage. (From OE-Core rev: cfbb3cc1279ea88ca3e2867f8a409c5120aa1f05) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03devtool: deploy-target: Let script continue even if rm failsDaniel Lublin
The remote script is run with `set -e`, so doing rm without -f on a $file that is already gone will exit the whole script, failing any redeployment. Assume a use case where packages sometimes produces certain test binaries stored on volatile media (tmpfs), and where the system is occasionally rebooted. Signed-off-by: Daniel Lublin <daniel@lublin.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-03devtool: Compare abspath of both B and SOla x Nilsson
Either both or none of the paths must be passed through os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will cause unintentional mismatches even when B = "${S}". Using os.path.abspath for both seems more likely to be correct as that will also handle the case where ${B} != ${S} but the abspaths are equal. Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-06-03devtool: deploy-target: Don't use find -execDaniel Lublin
find may be provided by busybox, which might be compiled without support for -exec. Signed-off-by: Daniel Lublin <daniel@lublin.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-30oeqa/core/context: Include a _pre_run methodLeonardo Sandoval
This pre runner will serve to allow Test components executes code previously of the run a suite. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-29runqemu: output qemu-system errorsEd Bartosh
Included error output from qemu-system into the runqemu error message. Made error output more visible by printing new line before it. [YOCTO #11542] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-29python3: Move config-*/Makefile from misc package to dev packageLi Zhou
Move config-*/Makefile in libdir from misc package to dev package for python3, because it is only needed in development process. Signed-off-by: Li Zhou <li.zhou@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25devtool/standard.py: Enable perf to be buildableSaul Wold
Perf is a tool build from the kernel source, which is normally available in /work-shared/..., but when devtool is used to modify the kernel source code, perf is not buildable since it gets an error about being unable to add a depends to a non-exisit task do_patch. This patch removes do_patch from the SRCTREECOVEREDTASKS and creates an empty do_patch task to enable the VarFlags code to have someplace to attach depends information to. [YOCT #11120] Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-25scriptutils: fix fetch_uri() to work with RSSPaul Eggleton
Since recipe-specific sysroots were implemented, devtool add and devtool upgrade operations that fetch from a URL that requires native sysroot dependencies will fail to work as there is no recipe-specific sysroot set up for them during fetching. An example was any URL pointing to a tarball compressed with xz, e.g. devtool upgrade on gnutls. The most expedient way to fix this is to set up a dummy recipe-specific sysroot to use for the fetch/unpack operations. We do this in the same manner as bitbake -b does, so we're just taking all of the sysroot components available and creating a sysroot from those rather than ensuring the correct dependencies are there - this means that we're still going to have problems if e.g. xz-native hasn't been built yet, but that issue will be trickier to solve and is tracked separately. Fixes [YOCTO #11474]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23scripts/buildstats-diff: support optimized rusage valuesMarkus Lehtonen
Buildstats from oe-build-perf-test results have been optimized to not have child rusage values at all. There, rusage is the sum of parent and child rusage values. This patch makes buildstats-diff compatible with this format. [YOCTO #11355] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23oe-build-perf-report: implement --dump-buildstatsMarkus Lehtonen
For dumping buildstats from the test runs being reported. The output directory where buildstats are copied is 'oe-build-perf-buildstats/'. Buildstats can be then further analyzed with buildstats-diff script, for example. [YOCTO #11355] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23oe-build-perf-report: two verbosity levels for --listMarkus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23oe-build-perf-report: use correct x-axis max value in html chartsMarkus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23recipetool: create: extract name of package from a repositoryPaul Eggleton
For git repositories in the absence of any other indicator, it's not an unreasonable assumption that the name of the repository is the name of the software package it contains, so use that as PN if we don't have anything else. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23recipetool: create: skip values extracted from spec files containing macrosPaul Eggleton
If a value we extract from a spec file contains an unexpanded macro (e.g. %{macroname}) then we should discard it since we're not seeing the actual value and we don't have an easy way of expanding it at the moment. This fixes for example getting %{name} as the recipe name when running the following: recipetool create https://github.com/gavincarr/mod_auth_tkt.git Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23create-pull-request: support format-patch optionsEd Bartosh
Added possibility to specify extra format-patch options in the create-pull-request command line: create-pull-request -u contrib -r master -- -v3 Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-23Revert "create-pull-request: add "-t in-reply-to" option"Ed Bartosh
Rasons: - It breaks the script if script is used without -t - Its functionality covered by the next patch This reverts commit 3ad3fda6c5f084f4fa1485b30aa333287989bee7. Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-21wic: Add missing text to usage and help stringsAndreas J. Reichel
Add missing parameters -c to and remove non-existent parameter -i from usage string for 'wic create'. Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-21wic: Use argparse instead of optparseAndreas J. Reichel
* optparse is deprecated and will not be developed further (see: https://docs.python.org/2/library/optparse.html) * argparse supports subcommands, which simplifies definition of arguments and options * reimplement help mechanism through sub-subcommands [YOCTO #9636] Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-18build-perf-test-wrapper.sh: support uploading test reportsMarkus Lehtonen
Implement new '-R' command line option for specifying an rsync destination where to upload test reports. [YOCTO #5049] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-18oe-build-perf-report: allow slashes in {branch} field in tag namesMarkus Lehtonen
The target branch name might contain slashes. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-18oe-build-perf-report: accept parenthesis in tag namesMarkus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-18scripts/runqemu.README: fix typoMartin Kelly
Signed-off-by: Martin Kelly <mkelly@xevo.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-05-16runqemu: Automatically add a TFTP directory for slirp bootAlistair Francis
When booting QEMU with slirp networking we want to use QEMUs TFTP server to make the images in deploy accessible to the guest. Signed-off-by: Alistair Francis <alistair.francis@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>