summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2015-12-27scripts/oe-selftest: Add support for selftest log with timestampCostin Constantin
Each time oe-selftest runs, the oe-selftest.log file is overwritten. This patch solves it by adding time stamp to each selftest log file and doing a symlink named as oe-selftest.log to the last one created. Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22recipetool: create: fix error when extracting source to a specified directoryPaul Eggleton
Having fetched the source and unpacked it to a temporary directory, we then move part of it to the destination directory, or if the source is at the top level we move the whole temporary directory, but in the latter case we were later attempting to delete the temporary directory which no longer existed. Clear out the variable so that doesn't happen. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: improve autotools supportPaul Eggleton
* tar and binutils we can assume are there * libsocket is only relevant on BSD systems, so we can ignore it. * Detect more things implying gettext/intltool is needed * Detect glib-2.0 requirement. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: sync: tweak help / messagesPaul Eggleton
Much of this was copy/pasted from the extract subcommand code; make it specific to sync. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: reset: print message about leaving source tree behindPaul Eggleton
We deliberately leave the source tree alone when resetting in case it contains any work in progress belonging to the user; tell them that we're doing this so they aren't surprised about it still existing later on. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: status: list recipe file within workspace if one existsPaul Eggleton
If a recipe in the workspace actually exists as a file within the workspace (e.g. after doing "devtool add" or "devtool upgrade") then show the path to the recipe file on the status line for the recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: modify: default source tree pathPaul Eggleton
As per the changes to "devtool add", make the source tree path optional and use the default path if none is specified. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: allow specifying URL as positional argumentPaul Eggleton
Having to specify -f is a little bit ugly when a URI is distinctive enough to recognise amongst the other positional parameters, so take it as an optional positional parameter. -f/--fetch is still supported, but deprecated. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: figure out recipe name from recipetoolPaul Eggleton
recipetool create now has all the logic in it for auto-detecting the name and version, and using those in the file name - so we can make the name an optional parameter for devtool add and we pick up the file name that recipetool has used after the fact. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: allow source tree to be omittedPaul Eggleton
Assuming we're fetching source remotely (from a URI) we can default the source tree that will be extracted from it to a "sources" directory under the workspace in order to save the user specifying it if they don't have a preferred location. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22scripts/lib/argparse_oe: handle intermixing of optional positional argumentsPaul Eggleton
Python's argparse module can't handle when several optional positional arguments (set with nargs='?') are intermixed with other options. If the positional arguments aren't optional then this isn't an issue; thus when changing positional arguments to optional (as we are doing with devtool) we need this workaround. This is a pretty horrible hack, but we don't want this flexibility of ordering to disappear simply because we made some arguments optional. Unfortunately the corresponding bug remains unresolved upstream even in Python 3, and argparse is not really designed to be subclassed so it doesn't make things like this easy. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: update-recipe: use correct method to get bbappend filenamePaul Eggleton
The bbappend already exists at this point, so we know what its path is - there's no need to figure it out from scratch here. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: split out function for naming bbappendPaul Eggleton
We're repeating this in a couple of places, so we might as well have a function to do it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: tweak help textPaul Eggleton
Add a few clarifying words. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: edit-recipe: add new subcommandPaul Eggleton
Add an "edit-recipe" subcommand that runs your default editor (as specified by the EDITOR environment variable) on the specified recipe in the workspace. Note that by default the recipe file itself must be in the workspace - i.e. as a result of "devtool add" or "devtool upgrade"; however there is a -a/--any-recipe option to override this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: basic extraction of name/version from filenamePaul Eggleton
Often the filename (e.g. source tarball) contains the name and version of the software it contains. (This isn't intended to be exhaustive, just to catch the common case.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: support extracting name and version from build scriptsPaul Eggleton
Some build systems (notably autotools) support declaring the name and version of the program being built; since we need those for the recipe we can attempt to extract them. It's a little fuzzy as they are often omitted or may not be appropriately formatted for our purposes, but it does work on a reasonable number of software packages to be useful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: set up priority system for recipe handlersPaul Eggleton
Sometimes we want to force one handler to run before another; if the two handlers are in different plugins that's difficult without some kind of priority number, so add one and sort by it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: detect when specified URL returns a web pagePaul Eggleton
If the user specifies a URL that just returns a web page, then it's probably incorrect (or broken); attempt to detect this and show an error if it's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: prevent attempting to unpack entire DL_DIRPaul Eggleton
If you specify a URL ending in /, BitBake's fetcher returns a localpath of ${DL_DIR}, and if you then try to unpack that it will attempt to copy the entire DL_DIR contents to the destination - which at least on my system filled my entire /tmp. Obviously we should fix the fetcher, but at least detect and stop that from happening here for now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: minor fix for potential issue in python handlingPaul Eggleton
If SRC_URI happened not to be in the pre-generated lines then this code would error out. This is unlikely to happen with the way the create code is structured at the moment, but handle it just in case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: fix do_install handling for makefile-only softwarePaul Eggleton
In my testing here it appears make -qn returns an error (exit code 2) whereas make -n doesn't; I can't immediately tell why based on the documentation. We don't actually care for it to be quiet since we're capturing the output, so let's just leave -q off and have this work properly as a result. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: avoid traceback on fetch errorPaul Eggleton
If a fetch error occurs, the fetcher already prints a reasonable error - we don't need the traceback as well, so catch that and exit if it occurs. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: handle https://....git URLsPaul Eggleton
When you grab a URL for a github repository you'll almost certainly find it in https://github.com/path/to/repository.git format; but bitbake's fetcher can't handle that because it'll see https:// at the start and assume it should use wget to fetch it. If the URL starts with http:// or https:// and the path part ends with .git then assume it's a git repository and adjust it accordingly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22scripts: print usage in argparse-using scripts when a command-line error occursPaul Eggleton
For scripts that use Python's standard argparse module to parse command-line arguments, create a subclass which will show the usage the usage information when a command-line parsing error occurs. The most common case would be when the script is run with no arguments; at least then the user immediately gets to see what arguments they might need to pass instead of just an error message. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: sdk-update: fix traceback without update server setPaul Eggleton
If the SDK update server hasn't been set in the config (when building the extensible SDK this would be set via SDK_UPDATE_URL) and it wasn't specified on the command line then we were failing with a traceback because we didn't pass the default value properly - None is interpreted as no default, meaning raise an exception if no such option exists. Additionally we don't need the try...except anymore either because with a proper default value, NoSectionError is caught as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22wic/help.py: document that mountpoint is optional for part commandJoshua Lock
If no mountpoint is specified for a partition command the partition will be created but not mounted — mention this in the kickstart help text. [YOCTO #8820] Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22oe-buildenv-internal: fix return codeJuro Bystricky
The script oe-buildenv-internal is called from oe-init-build-env. Make sure oe-init-buildenv does not return an error if BB_ENV_EXTRAWHITE is already set, otherwise this will cause oe-init-build-env to fail. Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-22buildhistory-collect-srcrevs: hide empty sectionsChristopher Larson
Cc: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-18combo-layer: Stop using filterdiffRichard Purdie
I ran into an issue where a patch just deleting a single file within the repository (meta/recipes-devtools/m4/m4/make.patch) would get skipped by combo-layer. It turns out this has the patch header (commented to avoid breaking scripts): : diff --git a/meta/recipes-devtools/m4/m4/make.patch b/meta/recipes-devtools/m4/m4/make.patch : deleted file mode 100644 : index 79fb415..0000000 : --- a/meta/recipes-devtools/m4/m4/make.patch : +++ /dev/null : @@ -1,42 +0,0 @@ and this is classed as > 5 headers in filterdiff. When we piped the path through filterdiff, the --- line disappears, then the second time we pass through filterdiff, it shows no lines changed and the patch is assumed to be empty and skipped. Changing MAX_HEADERS in filterdiff is one way to fix this, another would be to grep out "deleted file mode" lines. Instead, we can use new git syntax to exclude files from the git format-patch instead and avoid filterdiff entirely. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-16sstate-sysroot-cruft: Add php, python, lua, fontcache generated files to ↵Martin Jansa
whitelist Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-12scripts: oe-selftest Added new features.Daniel Istrate
[YOCTO #8750] Allow oe-selftest to run custom test suites based on different criteria 1. Can run custom lists of tests based on different criteria: --run-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags> eg: --run-tests-by module imagefeatures signing recipetool --run-tests-by id 1377 1273 935 --run-tests-by tag wic sstate bitbake 2. Can list tests based on different criteria: --list-tests-by <name|class|module|id|tag> <list of tests|classes|modules|ids|tags> eg: --list-tests-by module imagefeatures signing recipetool --list-tests-by id 1377 1273 935 --list-tests-by tag wic sstate bitbake 3. Can list all tags that have been set to test cases: --list-tags The list of tags should be kept as minimal as possible. This helps preview the tags used so far. To take advantage of the 'tag' feature: - add @tag(feature=<>) to testcases eg: @tag(feature='signing') for a single tag @tag(feature=(('signing', 'sstate')) or @tag(feature=['signing', 'sstate']) for multiple tags Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-12oe-buildenv-internal: preserve existing BB_ENV_EXTRAWHITEJuro Bystricky
According to the BitBake User Manual the environment variable BB_ENV_EXTRAWHITE specifies an additional set of variables to allow through (whitelist) from the external environment into BitBake's datastore. However, running: $ source oe-init-build-env build-name will overwrite any pre-existing BB_ENV_EXTRAWHITE variables. This patch modifies this behaviour: all oe-build-internal BB_ENV_EXTRAWHITE variables are appended to any potentially already existing BB_ENV_EXTRAWHITE variables. (The variables are only appended if not already in BB_ENV_EXTRAWHITE) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09devtool: extract: update SRCTREECOVEREDTASKS for kernelMarkus Lehtonen
Add 'do_kernel_configme' and 'do_kernel_configcheck' to SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run because kernel meta in the srctree is not necessarily up-to-date or even present which causes build failures and/or invalid kernel config. Especially so because 'do_patch' which is a dependency of 'do_kernel_configme' is not being run. We now store .config in the srctree and 'do_configure' task is able to run successfully. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09devtool: extract: copy kernel config to srctreeMarkus Lehtonen
This makes the correct kernel config to be used when building kernel from srctree (extrernalsrc). If no kernel config is present in the builddir 'do_configure' task copies .config from the srctree. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09directdisk-bootloader-config.wks: Add example for custom bootloader configMariano Lopez
Add new wks file as a example for a custom bootloader configuration. This change also includes the configuration that file that will be used. This example is using syslinux with MBR, the configuration file is almost the same as the one generated by wic. As stated before this is just an example. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic/help.py: Document the new option "configfile"Mariano Lopez
This just adds the "configfile" option for the bootloader to wic help. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic: Allow to use a custom config for bootloadersMariano Lopez
This change will allow to use a user defined file as the configuration for the bootloaders (grub, gummiboot, syslinux). The config file is defined in the wks file with the "configfile" option in the bootloader line. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic/utils/misc.py: Added function to search for files in canned-wksMariano Lopez
This change add two new function to search for files in the canned-wks folder for all the layers included in bblayers.conf. This will be used to search for custom configuration files for the bootloaders. There are similar functions in the wic engine, but these are focused in wks files only, so it was needed to create new ones. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic: Prepare wicboot to allow custom bootloader configMariano Lopez
Currently wic does the bootloader configuration file on the fly. This change introduce a configfile variable for the bootloader; this is to have a user defined configuration file for the bootloaders (grub, syslinux, and gummiboot). This is particular useful when having a multiboot system or scripts embedded in the configuration file. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-08scripts/oe-pkgdata-util: sort the packages in list-pkg-filesRoss Burton
Sort the list of packages in list-pkg-files to make the output easier to read. Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-08wic: insert local Python paths at frontMatt Madison
This follows how bitbake performs path insertion, and fixes a failure to start wic on Ubuntu 15.10 with the distribution's version of python-ply installed. Signed-off-by: Matt Madison <matt@madison.systems> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01python3: Upgrade from 3.4.3 to 3.5Alejandro Hernandez
python3-native_3.4.3.bb -> python3-native_3.5.0.bb: - changed version - changed cheksum - no license change, just dates python3_3.4.3.bb -> python3_3.5.0.bb: - changed version - changed cheksum - no license change, just dates New: - use_packed_importlib.patch: Fixes importlib on cross-compile environments Rebased: - Manifest - 000-cross-compile.patch - 020-dont-compile-python-files.patch - 04-default-is-optimized.patch - python-3.3-multilib.patch - distutils3-base.bbclass - distutils3-native-base.bbclass - python3native.bbclass Upstream: - makerace.patch Misc: - pip2 is handled as default on major distros, modified python3-pip to leave /usr/bin/pip available for pip2 - Fixed importing pip3 from python3 interpreter Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01oe-selftest: Enable code coverage on unit testsLeonardo Sandoval
Enable code coverage through the library 'python coverage'. In case the environment variable COVERAGE_PROCESS_START is present (one of the requisites for measuring sub-processes; the second one is including some coverage statements into the python sitecustomize.py file) it will be taken into account, otherwise it is exported with value '.coveragerc'. The latter value is a configuration file (also automatically created) with some default settings. Once tests are executed, a coverage report is shown on the log and the coverage output data is stored with name '.coverage.<args>' where '<args>' is the name of the unit tests executed or 'all_tests' when running with --run-all-tests. This output data can be latter used for better reporting using the same tool (coverage). As briefly indicate before, measuring sub-process implies setting the env variable COVERAGE_PROCESS_START (done automatically by the oe-selftest code with this patch if not already set) and creating a sitecustomize.py as explained on [1]. If either one of these is missing, complete coverage will be incomplete. Current measurements for 'oe-selftest --run-all-tests' indicate that current coverage is around 42 % taking into account BBLAYERS, bitbake and scripts folders. More details on [2], indicating the coverage per file/module. This tasks has been done together with Humberto Ibarra <humberto.ibarra.lopez@linux.intel.com> [YOCTO #8679] [1] http://coverage.readthedocs.org/en/latest/subprocess.html [2] https://bugzilla.yoctoproject.org/attachment.cgi?id=2854 Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01runqemu-internal: Replace wacom-tablet with tablet for usbdeviceKhem Raj
When booting weston-core-image with latest wayland/weston/libinput mouse/touchpad would not work on qemux86, this fixes the issue Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01recipetool: make plugin registration function name consistent with devtoolPaul Eggleton
This should have been register_commands rather than register_command; I used register_commands in devtool so lets change this here to be consistent with that. (Since this is extensible through layers though we need to remain compatible with the old name, so fall back to that if the new function name isn't there.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01recipetool: add setvar subcommandPaul Eggleton
Add a recipetool subcommand "setvar" to set a variable in a file. This uses our existing logic such that it doesn't matter if the variable is already set in the recipe, if it's set in the recipe or some inc file, and if the variable is not currently set that the line setting the variable gets inserted in the right place in the file. Implements [YOCTO #7676]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()Paul Eggleton
Use bb.utils.edit_metadata() to replace some of the logic in this function; this avoids us effectively having two implementations of the same thing. In the process fix the following issues: * Insert values before any leading comments for the next variable instead of after them * Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place * Properly handle replacing varflag settings (e.g. SRC_URI[md5sum]) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: clarify help textPaul Eggleton
* Make some minor clarifications to help text * Drop ArgumentDefaultsHelpFormatter and just put the defaults in the text itself where needed (because otherwise you get defaults shown for store_true options which is somewhat confusing). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: build: enable showing default task in helpPaul Eggleton
Enable access to the configuration object in register_commands() so that we can read configuration values there; this allows us to show the task that will be run in the command line help for the build subcommand. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>