aboutsummaryrefslogtreecommitdiffstats
path: root/scripts/lib
AgeCommit message (Collapse)Author
2015-12-22devtool: split out function for naming bbappendPaul Eggleton
We're repeating this in a couple of places, so we might as well have a function to do it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: add: tweak help textPaul Eggleton
Add a few clarifying words. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: edit-recipe: add new subcommandPaul Eggleton
Add an "edit-recipe" subcommand that runs your default editor (as specified by the EDITOR environment variable) on the specified recipe in the workspace. Note that by default the recipe file itself must be in the workspace - i.e. as a result of "devtool add" or "devtool upgrade"; however there is a -a/--any-recipe option to override this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: basic extraction of name/version from filenamePaul Eggleton
Often the filename (e.g. source tarball) contains the name and version of the software it contains. (This isn't intended to be exhaustive, just to catch the common case.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: support extracting name and version from build scriptsPaul Eggleton
Some build systems (notably autotools) support declaring the name and version of the program being built; since we need those for the recipe we can attempt to extract them. It's a little fuzzy as they are often omitted or may not be appropriately formatted for our purposes, but it does work on a reasonable number of software packages to be useful. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: set up priority system for recipe handlersPaul Eggleton
Sometimes we want to force one handler to run before another; if the two handlers are in different plugins that's difficult without some kind of priority number, so add one and sort by it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: detect when specified URL returns a web pagePaul Eggleton
If the user specifies a URL that just returns a web page, then it's probably incorrect (or broken); attempt to detect this and show an error if it's the case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: prevent attempting to unpack entire DL_DIRPaul Eggleton
If you specify a URL ending in /, BitBake's fetcher returns a localpath of ${DL_DIR}, and if you then try to unpack that it will attempt to copy the entire DL_DIR contents to the destination - which at least on my system filled my entire /tmp. Obviously we should fix the fetcher, but at least detect and stop that from happening here for now. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: minor fix for potential issue in python handlingPaul Eggleton
If SRC_URI happened not to be in the pre-generated lines then this code would error out. This is unlikely to happen with the way the create code is structured at the moment, but handle it just in case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: fix do_install handling for makefile-only softwarePaul Eggleton
In my testing here it appears make -qn returns an error (exit code 2) whereas make -n doesn't; I can't immediately tell why based on the documentation. We don't actually care for it to be quiet since we're capturing the output, so let's just leave -q off and have this work properly as a result. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: avoid traceback on fetch errorPaul Eggleton
If a fetch error occurs, the fetcher already prints a reasonable error - we don't need the traceback as well, so catch that and exit if it occurs. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22recipetool: create: handle https://....git URLsPaul Eggleton
When you grab a URL for a github repository you'll almost certainly find it in https://github.com/path/to/repository.git format; but bitbake's fetcher can't handle that because it'll see https:// at the start and assume it should use wget to fetch it. If the URL starts with http:// or https:// and the path part ends with .git then assume it's a git repository and adjust it accordingly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22scripts: print usage in argparse-using scripts when a command-line error occursPaul Eggleton
For scripts that use Python's standard argparse module to parse command-line arguments, create a subclass which will show the usage the usage information when a command-line parsing error occurs. The most common case would be when the script is run with no arguments; at least then the user immediately gets to see what arguments they might need to pass instead of just an error message. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22devtool: sdk-update: fix traceback without update server setPaul Eggleton
If the SDK update server hasn't been set in the config (when building the extensible SDK this would be set via SDK_UPDATE_URL) and it wasn't specified on the command line then we were failing with a traceback because we didn't pass the default value properly - None is interpreted as no default, meaning raise an exception if no such option exists. Additionally we don't need the try...except anymore either because with a proper default value, NoSectionError is caught as well. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-22wic/help.py: document that mountpoint is optional for part commandJoshua Lock
If no mountpoint is specified for a partition command the partition will be created but not mounted — mention this in the kickstart help text. [YOCTO #8820] Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09devtool: extract: update SRCTREECOVEREDTASKS for kernelMarkus Lehtonen
Add 'do_kernel_configme' and 'do_kernel_configcheck' to SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run because kernel meta in the srctree is not necessarily up-to-date or even present which causes build failures and/or invalid kernel config. Especially so because 'do_patch' which is a dependency of 'do_kernel_configme' is not being run. We now store .config in the srctree and 'do_configure' task is able to run successfully. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09devtool: extract: copy kernel config to srctreeMarkus Lehtonen
This makes the correct kernel config to be used when building kernel from srctree (extrernalsrc). If no kernel config is present in the builddir 'do_configure' task copies .config from the srctree. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09directdisk-bootloader-config.wks: Add example for custom bootloader configMariano Lopez
Add new wks file as a example for a custom bootloader configuration. This change also includes the configuration that file that will be used. This example is using syslinux with MBR, the configuration file is almost the same as the one generated by wic. As stated before this is just an example. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic/help.py: Document the new option "configfile"Mariano Lopez
This just adds the "configfile" option for the bootloader to wic help. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic: Allow to use a custom config for bootloadersMariano Lopez
This change will allow to use a user defined file as the configuration for the bootloaders (grub, gummiboot, syslinux). The config file is defined in the wks file with the "configfile" option in the bootloader line. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic/utils/misc.py: Added function to search for files in canned-wksMariano Lopez
This change add two new function to search for files in the canned-wks folder for all the layers included in bblayers.conf. This will be used to search for custom configuration files for the bootloaders. There are similar functions in the wic engine, but these are focused in wks files only, so it was needed to create new ones. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-09wic: Prepare wicboot to allow custom bootloader configMariano Lopez
Currently wic does the bootloader configuration file on the fly. This change introduce a configfile variable for the bootloader; this is to have a user defined configuration file for the bootloaders (grub, syslinux, and gummiboot). This is particular useful when having a multiboot system or scripts embedded in the configuration file. [YOCTO #8728] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01recipetool: make plugin registration function name consistent with devtoolPaul Eggleton
This should have been register_commands rather than register_command; I used register_commands in devtool so lets change this here to be consistent with that. (Since this is extensible through layers though we need to remain compatible with the old name, so fall back to that if the new function name isn't there.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01recipetool: add setvar subcommandPaul Eggleton
Add a recipetool subcommand "setvar" to set a variable in a file. This uses our existing logic such that it doesn't matter if the variable is already set in the recipe, if it's set in the recipe or some inc file, and if the variable is not currently set that the line setting the variable gets inserted in the right place in the file. Implements [YOCTO #7676]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()Paul Eggleton
Use bb.utils.edit_metadata() to replace some of the logic in this function; this avoids us effectively having two implementations of the same thing. In the process fix the following issues: * Insert values before any leading comments for the next variable instead of after them * Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place * Properly handle replacing varflag settings (e.g. SRC_URI[md5sum]) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: clarify help textPaul Eggleton
* Make some minor clarifications to help text * Drop ArgumentDefaultsHelpFormatter and just put the defaults in the text itself where needed (because otherwise you get defaults shown for store_true options which is somewhat confusing). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: build: enable showing default task in helpPaul Eggleton
Enable access to the configuration object in register_commands() so that we can read configuration values there; this allows us to show the task that will be run in the command line help for the build subcommand. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: build: use bbappend to set PARALLEL_MAKEPaul Eggleton
Use a bbappend file to set PARALLEL_MAKE instead of a postfile; this is a bit neater and only affects the specified recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: ensure we change back to the original dir on errorPaul Eggleton
This is just belt-and-braces but we ought to use try..finally in this kind of situation, so just do it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: search: print SUMMARY valuePaul Eggleton
Print the SUMMARY value for each matched item assuming it's not the default. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: drop unused plugin_init() functionsPaul Eggleton
This function is no longer required to be defined for a plugin, so drop it where it's a no-op. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: package: use DEPLOY_DIR_<pkgtype> to get deploy directoryPaul Eggleton
Rather than reconstructing the output path for packages, use the proper variable. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: disable creating workspace for extract and search subcommandsPaul Eggleton
For subcommands that don't actually involve the workspace, don't auto-create the workspace. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: provide a means to update the source branchPaul Eggleton
If you're upgrading a git recipe to a revision on a release branch that's different to the branch for the current revision, then you'll need to update the branch parameter in SRC_URI, so add a --srcbranch/-B command-line parameter to let you do that easily. It handles both when the branch is stated verbatim in the recipe, and when a reference to another variable is used (a common convention is to use a SRCBRANCH variable for this, though the code doesn't care what variable is used if any). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fetch remote repository before checking out new revisionPaul Eggleton
If we're upgrading a recipe that fetches from git, and we've simply fetched a tarball of the repo instead of directly from the upstream repo (this can happen if you have PREMIRRORS set up as in poky with a core recipe, e.g. kernelshark) then we won't have any new revisions, and the checkout will fail with "fatal: reference is not a tree: <hash>". To avoid this, do a "git fetch" before checking out the new revision. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: remove erroneous error when not renaming recipePaul Eggleton
If we're upgrading a git recipe the recipe file usually won't need renaming; for some unknown reason we were throwing an error here which isn't correct. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fix updating PV and SRCREVPaul Eggleton
This code was clearly never tested. Fix the following issues: * Actually set SRCREV if it's been specified * Enable history tracking and reparse so that we handle if variables are set in an inc file next to the recipe * Use a more accurate check for PV being in the recipe which will work if it's in an inc file next to the recipe Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: upgrade: fix removing other recipes from workspace on resetPaul Eggleton
If you did a "devtool add" followed by "devtool upgrade" and then did a "devtool reset" on the recipe you upgraded, the first recipe would also be deleted from the workspace - this was because we were erroneously adding the entire "recipes" subdirectory and its contents to be tracked for removal on reset. Remove the unnecessary call to os.path.dirname() that caused this. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01devtool: include do_patch in SRCTREECOVEREDTASKSTzu-Jung Lee
The external source of kernel has been patched during the construction of git repository. Include the do_patch task in the SRCTREECOVEREDTASKS. Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01wic: exec_native_cmd: implement support for pseudoEd Bartosh
Wic runs some tools through pseudo, which makes exec_native_cmd to fail and throw cryptic error message when tool is not baked: For example: Error: exec_cmd: 'export PSEUDO_PREFIX=/media/ssd/poky-build/tmp/sysroots/x86_64-linux/usr;export PSEUDO_LOCALSTATEDIR=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/ ... PSEUDO_PASSWD=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/ ... PSEUDO_NOSYMLINKEXP=1;/media/ssd/poky-build/tmp/sysroots/ ... mkfs.ext4 -F -i 8192 /var/tmp/wic/build/rootfs_platform.7.ext4 -L platform -d /media/ssd/poky-build/tmp/work/qemux86-poky-linux/core-image-minimal/... returned '1' instead of 0 Made exec_native_cmd aware of pseudo and properly report errors when command is not found. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-25devtool: add sync commandTzu-Jung Lee
The sync command is similar to the extract command, except it fetches the sync'ed and patched branch to an existing git repository. This enables users to keep track the upstream development while maintaining their own local git repository at the same time. Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-11-25recipetool.append: don't choke on a trailing ; in a urlChristopher Larson
Apparently bb.fetch.URI can't handle this at the moment. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-24wic/utils/oe/misc.py: Preserve PATH when running native toolsRandy Witt
Previously exec_native_cmd() would remove all items from PATH except for the native sysroot. This can cause issues for the tools that are created using create_wrapper(). Now instead of wiping out the PATH, run a sanity check to check if the command is in the native sysroot. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-19devtool: extract: fix error handlingPaul Eggleton
If recipe parsing fails we need to exit out instead of attempting to use the data object that's set to None, which resulted in a traceback. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-03recipetool: create: fix change in path structure if --extract-to path existsPaul Eggleton
If the directory specified by --extract-to exists, because we were using shutil.move() to move the temporary extracted directory to the specified path, a subdirectory was being created under that directory instead of moving the contents, which was a different result than if the directory didn't previously exist. We could try to always move the contents but that's complicated when any symlinks are involved; the simplest thing is just to remove the directory (which should be empty anyway) before moving the temporary directory across in its place. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-03devtool: update-recipe: avoid updating patches that have not changedPaul Eggleton
Use "git cherry" against the original tag that we made when we extracted the source in order to find the revisions that are definitely new. This allows you to modify a commit in the middle of the series and then run devtool update-recipe and not have the subsequent patches unnecessarily modified. Fixes [YOCTO #8388]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: modify: use correct local files directory nameMarkus Lehtonen
The name of the directory for local source files under srctree is 'oe-local-files', not 'local-files'. Fixes a bug that slipped through in b7ab82485e4514e07ab8a76e554da27ddc92e6c0. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01devtool: update-recipe: enable var history trackingMarkus Lehtonen
Enable variable history tracking so that the variables are updated in the correct file - i.e. in the file they are already defined. [YOCTO #7715] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-29devtool: modify: make bitbake use local files from srctreeMarkus Lehtonen
This change makes it possible to have local files (non-remote SRC_URI files, i.e. files that are located in the "recipe space") under the srctree even if S!=WORKDIR. The files must be placed under the 'local-files' subdirectory. Complements the previous patch that imports local files into srctree. [YOCTO #7602] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>