aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2016-02-28devtool: upgrade: print new recipe namePaul Eggleton
Make this consistent with "devtool add" so that the user knows where to find the new recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: drop PR on upgradePaul Eggleton
The PR value should be reset to the default when upgrading, so we need to drop it from the newly created file. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: eliminate unnecessary datastore copyPaul Eggleton
We aren't modifying the datastore copy here, so we don't need a copy at all. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: fix several issues with extraction of new sourcePaul Eggleton
Fix several issues when extracting the new version source over the top of the old one (when the recipe is not fetching from a git repo): * Delete the old source first so we ensure files deleted in the new version are deleted. This also has the side-effect of fixing any issues where files aren't marked writeable in the old source and thus overwriting them failed (harfbuzz 1.1.3 contains such files). * Fix incorrect variable name in abspath statement that made it a no-op Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: fix constructing new branch from tarball releasesPaul Eggleton
When we do an upgrade from one tarball version to another we want to: 1) Check out the old version as a new branch 2) Record the changes between the old and new versions as a commit 3) Check out the old version with patches applied 4) Rebase that onto the new branch Where we went wrong was step #1 where instead we checked out the old version with patches applied as the new branch, which meant the rebase didn't do anything and any changes made by the patches to files still in the new version were wiped out. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: fix renaming of recipe if PV is not in namePaul Eggleton
If the actual value of PV isn't in the name of the recipe (for example, a git or svn recipe) there's no point trying to rename it. Additionally, we already have the original filename, there's no need to guess it - just pass it in. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: fix moving version-specific files directoryPaul Eggleton
We were trying to move this from the current directory instead of the path. Let's just use shutil.move() instead of shelling out to mv. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: fix version argument checkingPaul Eggleton
For recipes that specify SRCREV, the code here wasn't quite doing the right thing. If the recipe has a SRCREV then that needs changing on upgrade, so ensure that the user specifies it. If it doesn't, then it'll be "INVALID" not None since the former is the actual default, so handle that properly as well. Additionally an unset variable was being erroneously passed when raising the error about the version being the same leading to a traceback, so fix that as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: drop superfluous call to validate_pnPaul Eggleton
The recipename argument to devtool upgrade specifies an existing recipe, so by definition the name will be valid (or it won't exist) - we don't need to validate it ourselves, that's only needed for situations like in devtool add where we're creating a new recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: upgrade: make source tree path optionalPaul Eggleton
Make devtool upgrade consistent with devtool add/modify in defaulting to sources/<recipename> under the workspace if no source tree path is specified. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: modify: fix source tree default name when mapping virtualsPaul Eggleton
If you for example ran devtool modify virtual/libusb0 without specifying a source tree path, the default was <workspace>/sources/virtual/libusb0 which isn't correct - it should be using the mapped name i.e. libusb-compat (in the default OE-Core configuration). Reorder some of the code to ensure that the mapped name is used. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: add: tweak auto-determining name failure messagePaul Eggleton
As suggested by Khem Raj. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28oe-selftest: Add support for lib/oeqa/selftest subdirectoriesCostin Constantin
This patch adds functionality to allow creating subdirectories inside lib/oeqa/selftest for all layers present in BBLAYERS. Like this, test cases can be grouped into organized directories. Addresses [YOCTO #7865] Signed-off-by: Costin Constantin <costin.c.constantin@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: update-recipe: create config fragmentMarkus Lehtonen
Create config fragment if the user makes modifications to kernel config. User may change .config e.g. by directly editing it or by running the 'do_menuconfig' bitbake task. Devtool generates one monolithic fragment by simply doing a diff between .config and .config.baseline files in the source directory. If either of these files is missing, the config fragment is not gerenrated or updated. The output is a file, 'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well as copied into the 'oe-local-files' directory if that is present in the source tree). ${S}/.config will be a symlink to ${B}/.config. We need to do this as devtool is not able to access ${B} because ${B} is set in a .bbappend in the workspace layer which is not parsed by devtool itself. [YOCTO #8999] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28devtool: sync: update kernel configMarkus Lehtonen
Copy kernel config is copied to the source directory at a later phase in _extract_source() so that it gets copied when devtool sync is done, too. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28scripts/oe-selftest: Use site.USER_SITE to run coverage configuration code ↵Humberto Ibarra
for sub-process Coverage in oe-selftest currently requires to create or modify a sitecustomize.py file according the coverage tool setup instructions (http://coverage.readthedocs.org/). This file has to be located in the system's python folder, which is not a good solution since this folder is not accesible to non-privileged users. The best solution so far is to create this file in the home directory. This is implemented by creating the temporal file in the user site default folder. [Yocto #8930] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-28scripts/oe-selftest: Add filtering to the coverage data gathered by oe-selftestHumberto Ibarra
When --coverage is used, oe-selftest gathers coverage data from the testcases executed. The command lacks a way of filtering which files to gather coverage data from. This patch adds three options to specify which files should be considered. The --coverage-source option specifies folders, while --coverage-include and --coverage-omit specify patterns to have an extra level of filtering. Some examples: 1. oe-selftest --run-all-tests --coverage Gathers coverage data from the default poky folders 2. oe-selftest --run-all-tests --coverage --coverage-include /home/me/poky/scripts/* Gathers coverage data only for the files located under '/home/me/poky/scripts' 3. oe-selftest --run-all-tests -coverage --coverage-omit /home/me/poky/meta* Gathers coverage data. Files inside all the folders starting with 'meta' under '/home/me/poky' are omited 4. oe-selftest --run-all-tests --coverage --coverage-source /home/me/poky/bitbake Gathers coverage data only from files inside the folder: '/home/me/poky/bitbake' [Yocto #8920] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-21recipetool: create: improve CMake package mappingPaul Eggleton
* Package names are actually case sensitive near as I can tell, so we shouldn't be lowercasing them everywhere. * Look for CMake packages in pkgdata and map those back to recipes, so we aren't dependent on the hardcoded mappings (though those are still preserved). * Avoid duplicates in the unmapped package list Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21recipetool: create: add additional extension mechanismsPaul Eggleton
Add a means of extending the dependency extraction for autotools and cmake. Note: in order to have this work, you need to have an __init__.py in the lib/recipetool directory within your layer along with the module implementing the handlers, and the __init__.py needs to contain: # Enable other layers to have modules in the same named directory from pkgutil import extend_path __path__ = extend_path(__path__, __name__) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: modify: tweak help description for behaviour changePaul Eggleton
I should have adjusted this in OE-Core commit 80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: deploy-target: preserve existing filesPaul Eggleton
If files would be overwritten by the deployment, preserve them in a separate location on the target so that they can be restored if you later run devtool undeploy-target. At the same time, also check for sufficient space before starting the operation so that we avoid potentially failing part way through. Fixes [YOCTO #8978]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: undeploy-target: support undeploying all recipesPaul Eggleton
If you want to put the target device back to exactly how it was before devtool deploy-target started poking things into it, then it would make things easier if you didn't have to figure out which recipes were deployed. Now that we have the list stored on the target we can determine this reliably, so add a -a/--all option to undeploy-target to undeploy everything that has been deployed. One of the side-effects of this is that the dry-run functionality for undeploy-target had to be reimplemented to actually run the script on the target, since we have no way of knowing what's been deployed from the host side. We don't need to do the same for deploy-target though since we know exactly which files will be deployed without referring to the target. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: deploy-target: write deployed files list to targetPaul Eggleton
When running devtool deploy-target, we save a list of deployed files, and this list is used by devtool undeploy-target (or the next time deploy-target is run if the list is present, in case any files have been renamed or deleted since the first time). We were writing this file to the host, but it makes more sense to write the list to the target instead, so that if we for example swap in a different board, or switch hosts, things will work as expected. In order to do this properly we have to construct a shell script and ship it over to the target so we can run it. The manifest is written out to a hidden directory in the root (/.devtool). Fixes [YOCTO #7908]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: sdk-update: tweak command-line handling of updateserverPaul Eggleton
Get the default value for updateserver from the configuration file and show it in the help; also only make the parameter optional if it's specified. This means we can also drop the check in the function as argparse will then ensure it's specified if there's no config setting. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: (un)deploy-target: add help descriptionsPaul Eggleton
Add a long description used when running --help on the specific command. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21scripts/lib/argparse_oe: tweak title above optionsPaul Eggleton
Naming these as "optional arguments" is perhaps slightly confusing since some of the positional arguments might also be optional; in addition it's rare (though possible) for options to be mandatory - up until recently we had a recipetool option (-o) that was mandatory. It's not perfect, but change it to "options" so it's at least a bit more appropriate. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: categorise and order subcommands in help outputPaul Eggleton
The listing of subcommands in the --help output for devtool was starting to get difficult to follow, with commands appearing in no particular order (due to some being in separate modules and the order of those modules being parsed). Logically grouping the subcommands as well as being able to exercise some control over the order of the subcommands and groups would help, if we do so without losing the dynamic nature of the list (i.e. that it comes from the plugins). Argparse provides no built-in way to handle this and really, really makes it a pain to add, but with some subclassing and hacking it's now possible, and can be extended by any plugin as desired. To put a subcommand into a group, all you need to do is specify a group= parameter in the call to subparsers.add_parser(). you can also specify an order= parameter to make the subcommand sort higher or lower in the list (higher order numbers appear first, so use negative numbers to force items to the end if that's what you want). To add a new group, use subparsers.add_subparser_group(), supplying the name, description and optionally an order number for the group itself (again, higher numbers appear first). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: update-recipe: don't show workspace recipe warning if no updatePaul Eggleton
If we didn't make any changes to the file then there's no point warning the user that we have done. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: reset: fix preserving patches/other files next to recipesPaul Eggleton
If files had been created next to the recipe (for example devtool add, edit the source and commit and then devtool update-recipe), running devtool reset failed to preserve those files and gave an error due to trying to rmdir the directory containing them which wasn't empty. Fix the preservation of files in the "attic" directory properly so we catch anything under the directory for the recipe, and replicate the same structure in the attic directory rather than slightly flattening it as we were before. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool / recipetool: use common code for launching editorPaul Eggleton
Looking at Chris Larson's code for starting the user's editor for "recipetool newappend" it was slightly better than what I wrote for "devtool edit-recipe" in that it checks VISUAL as well as EDITOR and defaults to vi if neither are set, so break this out to its own function and call it from both places. The broken out version passes shell=True however in case it's a more complicated command rather than just a name of an executable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: minor fix for error messagePaul Eggleton
There is no -N/--name option for devtool, that's a recipetool option - with devtool you just specify the name as a positional argument. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21devtool: Don't recursively look for .devtoolbase in --basepathRandy Witt
If the user specifies --basepath on the commandline, only the directory specified should be searched for .devtoolbase. Otherwise when --basepath is a child of the sdk directory, .devtoolbase will always be found and devtool will only show options meant to be used within an sdk. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15linux-yocto: Work around PAT issue on qemux86Richard Purdie
We have an issue with PAT handling on older processes with limited PAT bits, see the patch description for the full problem. This replaces the runqemu workaround with a kernel patch until we can get the kernel trees sorted out and discuss a proper fix with upstream. It should be safe everywhere so is applied unconditionally. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15wic: fix processing of --use-uuidEd Bartosh
Processing of this option was lost during recent change of wks parsting. It was discovered during the work on booting wic images under qemu. Now, when -use-uuid is fixed it's possible to specify root partition by partition uuid. This will be done in the following commit. (From OE-Core rev: b4882e0b84d7fd4c85ee95386e94722485eafc2b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: support path/to/<image>-<machine>.wicEd Bartosh
Supported providing wic image path to runqemu: runquemu path/to/<image>-<machine>.wic [YOCTO #8691] (From OE-Core rev: 58a3bfb1e4b493200820cdf0bf3fc79e31e792de) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: don't set KERNEL for wic imagesEd Bartosh
Wic images should be boot as is, without pointing qemu to the kernel binary. Current code doesn't use kernel, but sets KERNEL variable and shows kernel path in the console output. This can confuse users. Changed runqemu and runqemu-internal code to avoid setting KERNEL variable and show kernel path. (From OE-Core rev: 474caa7ed5ff05caa5d49d270b283882fa616ed1) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15runqemu: add support for wic imagesEd Bartosh
Quemu should be able to run wic images this way: runqemu <machine> <image recipe> wic Tested with 'runqemu qemux86-64 wic-image-minimal wic' (From OE-Core rev: 8716be799949cb8bde7fa49cbea61312a3a93bb7) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15scripts/sstate-cache-management.sh: Change wordingJan Sarenik
Use "out of" rather than "from" in the output. Signed-off-by: Jan Sarenik <jajomojo@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-12scripts/runqemu-internal: Work around qemux86 PAT bugs in linux 4.4.1Richard Purdie
Since the upgrade of qemux86 to 4.4.1 we're seeing PAT issues when starting the X server. We need to fix the problem but the failing sanity tests mask out other issues and we need a workaround. Merge this for now until we can figure out the full issue. This is better than changing the kernel defconfig or reverting to old kernel versions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: set S when we set SRC_URI from local git repoPaul Eggleton
If you specify a local directory which happens to be a git repository with an origin remote (and it is in fact remote), we can use that for SRC_URI as implemented by OE-Core revision b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S if the recipe is going to be of any use fetching from that SRC_URI later. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: convert http git URLs that don't end in .git but contain ↵Paul Eggleton
/git/ When recipetool create is given a URL that starts with http(s):// and contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe to assume it's a git repository and not something that should be fetched with wget, so rewrite the URL. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: ensure URL parameters don't make it into the namePaul Eggleton
When auto-detecting the name for a recipe from the URL, strip off any parameters (";name=value...") before parsing the URL, otherwise this just ends up in the recipe name which is undesirable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: add: fix adding from a local source directoryPaul Eggleton
Fix a regression introduced in in OE-Core revision aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source tree without specifying a name resulted in a traceback. Fixes [YOCTO #9086]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: modify: make -x the default behaviourPaul Eggleton
It's going to be more common for users not to have the prepared source tree for a recipe already, so the default behaviour ought to be to extract it for them from the recipe. Change the default to extract (effectively making the -x option a no-op) and add a --no-extract/-n option to disable it. Later we can look at trying to be smart and reusing an existing source tree instead of erroring out if it exists; for now this is just the default reversal. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: determine name/version from github/bitbucket URLsPaul Eggleton
Both BitBucket and GitHub provide "release" downloads that unfortunately don't include the actual project name. For recipetool this means we have to have special handling for these URLs or else the current code assumes the name of the project is something like "v0.9.1". Borrow and adapt some code from autospec to do this, and put it in its own function for clarity. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool: create: support cmake find_library directivePaul Eggleton
CMake supports a find_library() directive to find named libraries, so detect dependencies from this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11devtool: commit for extra tasks that modify source when extractingPaul Eggleton
When extracting source for a recipe, if there are additional custom tasks run that make changes to the source, create a commit in the generated git branch so they are contained. This is particularly useful for tasks that come before do_patch since otherwise the changes might get incorporated in the first applied patch, but otherwise it helps avoid the tree being dirty at any point. Fixes [YOCTO #7626]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11recipetool.newappend: add -e/--edit argumentChristopher Larson
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-11python: merge python-elementtree into python-xmlRoss Burton
python-xml depends on python-elementtree as the latter just contains a C library used by the former. However there's no point to this split apart from increasing the number of packages, so merge -elementtree into python-xml. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07isoimage-isohybrid.py: fix cpio working directoryIoan-Adrian Ratiu
Take `pwd` to be <initial-dir>. The %s path is relative to it. The value of %s is "output_folder/build". The current code works as follows: Changing directory to %s and finding the sources (after cd'ing) to cpio with output redirection to %s/initrd.cpio triggers the following error "Error: exec_cmd: cd output_folder/build/INITRD && find . | cpio -o -H newc >output_folder/build/initrd.cpio returned '1' instead of 0" This happens because after the cd, `pwd` is <initial-dir>/%s and by the redirect we write the result to to <initial-dir>/%s/%s/initrd.cpio which obviously does not exist. Fix this by getting the sources with "find %s" instead of "cd && find ." Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>