aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/recipeutils.py
AgeCommit message (Collapse)Author
2016-07-14devtool: add finish subcommandpaule/devtool19-oePaul Eggleton
Add a subcommand which will "finish" the work on a recipe. This is effectively the same as update-recipe followed by reset, except that the destination layer is required and it will do the right thing depending on the situation - if the recipe file itself is in the workspace (e.g. as a result of devtool add), the recipe file and any associated files will be moved to the destination layer; or if the destination layer is the one containing the original recipe, the recipe will be overwritten; otherwise a bbappend will be created to apply the changes. In all cases the layer path can be loosely specified - it could be a layer name, or a partial path into a recipe. In the case of upgrades, devtool finish will also take care of deleting the old recipe. This avoids the user having to figure out the correct actions when they're done - they just do "devtool finish recipename layername" and it saves their work and then removes the recipe from the workspace. Addresses [YOCTO #8594]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2016-07-14devtool: upgrade: record original recipe filesPaul Eggleton
This provides us with the information we need to remove the original version recipe and associated files when running "devtool finish" after "devtool upgrade". Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2016-07-14lib/oe/recipeutils: fix a few issues in find_layerdir()Paul Eggleton
* Allow the function to be called with the base layer path (in which case it will just return the same path) * Ensure that the function doesn't recurse indefinitely if it's called on a file that's not inside a layer * Correct the doc comment for accuracy Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2016-06-02classes/lib: Complete transition to python3Richard Purdie
This patch contains all the other misc pieces of the transition to python3 which didn't make sense to be broken into individual patches. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02scripts: Fix deprecated dict methods for python3Ed Bartosh
Replaced iteritems -> items, itervalues -> values, iterkeys -> keys or 'in' Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2016-06-02classes/lib: Update xrange -> range for python3Richard Purdie
xrange() no longer exists in python 3, use range() Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02classes/lib: Update to explictly create lists where neededRichard Purdie
Iterators now return views, not lists in python3. Where we need lists, handle this explicitly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02classes/lib: Update to match python3 iter requirementsRichard Purdie
python3 standardises its use of iteration operations. Update the code to match the for python3 requires. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30lib/oe/recipeutils: fix insertion of variable valuesPaul Eggleton
Add some more variables in appropriate places in recipe_progression such that the patch_recipe_* functions are able to insert variables in the right place within a recipe. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30lib/oe/recipeutils: patch_recipe_lines: allow omitting trailing newlinesPaul Eggleton
This function was assuming that what you wanted was that output lines had trailing newline characters. If you're just outputting each line verbatim to a text file then that's fine, but sometimes you start with the assumption that the lines don't have trailing newlines; thus we shouldn't allow for the possibility that the caller doesn't want them and add a parameter to control it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-30lib/oe/recipeutils: split out patch_recipe_lines()Paul Eggleton
Split out a function from patch_recipe_file() that takes just the lines as input so we can edit recipe lines in memory. This will be used within recipetool to ensure we insert new values in the right place. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15devtool: update-recipe: handle where SRC_URI is appended to with +=Paul Eggleton
If a recipe sets SRC_URI and then appends more items to it with += (such as the current rpm recipe in OE-Core), the code in patch_recipe_file() was failing with a traceback. Work around the problem for now by dropping the existing lines if we understand the operation, else just set the value outright at the end. This leaves something to be desired as it either doesn't respect the existing structure or leaves a mess but it's better than the current breakage. We'll need to come up with a better solution later. Part of the problem is the existing code structure doesn't allow for patch_recipe_file() to know what's being added or removed - it only knows the final value that the caller wants set. Fixes [YOCTO #9458]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01lib/oe/recipeutils: refactor patch_recipe_file() to use edit_metadata()Paul Eggleton
Use bb.utils.edit_metadata() to replace some of the logic in this function; this avoids us effectively having two implementations of the same thing. In the process fix the following issues: * Insert values before any leading comments for the next variable instead of after them * Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place * Properly handle replacing varflag settings (e.g. SRC_URI[md5sum]) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-12-01lib/oe/recipeutils: check in validate_pn() for names instead of filenamesPaul Eggleton
Ensure that the user specifies just the name portion instead of a file name with extension. (We can't just look for . since there are recipe names such as "glib-2.0" that legitimately contain .). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-10-21devtool: handle virtual providersPaul Eggleton
For modify / extract / upgrade, if the specified "recipe" is not actually a recipe but a virtual target such as virtual/kernel, map it correctly to the actual recipe and make sure we use that name within the workspace. Thanks to Chris Larson for reminding me this was still broken and for a hint on how to fix it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-29recipeutils: implement get_recipe_local_files()Markus Lehtonen
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
2015-09-28recipeutils.py: don't hardcode the upstream version as 1.0 when SRC_URI is ↵Alexander Kanavin
empty or absent Instead, simply return the version of the recipe, so that the upstream version checker will declare that they match. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-09-22devtool / lib/oe/recipeutils: ensure we can parse without bbappendsPaul Eggleton
These functions ostensibly allowed parsing a recipe without bbappends but this clearly hadn't been tested because a variable was unassigned in both of them in that case. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-22lib/oe/recipeutils: properly split unexpanded variable valuesPaul Eggleton
Variables such as SRC_URI which are space-separated may also contain Python expressions (${@...}) which themselves contain spaces that shouldn't be split when splitting the value into items. In order to ensure this we need to use a custom splitting function instead of just string.split(). This issue could be seen when doing "devtool modify sudo", adding a commit to the resulting source repository then "devtool update-recipe" - the Python expression in SRC_URI was being unnecessarily broken onto multiple lines. Fixes [YOCTO #8046]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19lib/oe/recipeutils: avoid parsing in get_var_files()Paul Eggleton
Let's have the caller do this and then the function is a bit more flexible (e.g. we can choose to parse with bbappends or not); fix up calls to this function appropriately (of which there are only two, both within devtool). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-08-09recipeutils.py: allow all characters in regexes used to parse version stringsAlexander Kanavin
Previously only numeric characters were matches and anything else was discarded, so 4.0-rc3, 2005e, 1.0.2a and similar versions got truncated. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22recipeutils.py: get_recipe_pv_without_srcpv remove prefixes from pvAníbal Limón
Some recipes uses v or r prefixes in versions that makes wrong comparisions over recipes like lz4 r123 > 128. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oe/recipeutils.py: get_recipe_upstream_info update to get revisionAníbal Limón
Bitbake fetcher latest_versionstring now returns a tuple with (version, revision) that helps SCM's like git to build current upstream version. [YOCTO #7605] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22oe/recipeutils.py: get_recipe_upstream_info only use sfx and pfx when exitsAníbal Limón
Don't use pfx and sfx when not exist because cause formatting errors like 2.9HASH instead of 2.9+gitAUTOINC+HASH. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16lib/oe/recipeutils: fix replace_dir_vars to return the correct variablesPaul Eggleton
If we sort by length of name here we get the variables we are interested in first. I've tested this with all of the variables we care about (the ones at the top of bitbake.conf) and it returns the right results. This fixes the failures we've been seeing in the oe-selftest test_recipetool_appendfile_* tests. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-07-12oe/recipeutils.py: get_recipe_upstream_version returns 1.0 when not SRC_URIAníbal Limón
Recipes that don't have SRC_URI means that don't use upstream sources so returns 1.0. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12lib/oe/recipeutils: Ensure order of data store doesn't influence the resultRichard Purdie
The order of the keys from the data store is not prescribed. If target_datadir comes before datadir the selftests fail since the 'wrong' variable is used for substitutions. This highlights an issue with the replace_dir_vars() function. Fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27oe.recipeutils: fix line.split error in bbappend_recipeChristopher Larson
Cc: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Christopher Larson <kergoth@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-08recipeutils: Improve get_recipe_pv_without_srcpv functionAníbal Limón
Use pv instead of rd this make the function more generic and avoid copy recipe data. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2015-06-03recipeutils: Add get_recipe_upstream_version and get_recipe_pv_without_srcpv ↵Aníbal Limón
functions The get_recipe_upstream_version functions tries to get the current version of recipe in upstream it uses bb.fetch2 latest_versionstring method also latest_revision when is SCM. The get_recipe_pv_without_srcpv discards the SRCPV in SCM's recipe like git it returns a tuple with the version, prefix and suffix of a PV. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2015-05-19recipetool: add appendfile subcommandPaul Eggleton
Locating which recipe provides a file in an image that you want to modify and then figuring out how to bbappend the recipe in order to replace it can be a tedious process. Thus, add a new appendfile subcommand to recipetool, providing the ability to create a bbappend file to add/replace any file in the target system. Without the -r option, it will search for the recipe packaging the specified file (using pkgdata from previously built recipes). The bbappend will be created at the appropriate path within the specified layer directory (which may or may not be in your bblayers.conf) or if one already exists it will be updated appropriately. Fairly extensive oe-selftest tests are also provided. Implements [YOCTO #6447]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-15lib/oe/recipeutils: add a parse_recipe_simple() functionPaul Eggleton
Add a function that simply parses a recipe by name and optionally the bbappends that apply to it. (Note that if you're using tinfoil you need to have initialised it with config_only=False so that it can map the recipe name to a recipe file.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2015-04-27devtool: handle . in recipe namePaul Eggleton
Names such as glib-2.0 are valid (and used) recipe names, so we need to support them. Fixes [YOCTO #7643]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-27devtool: include bbappends in recipe parsingMarkus Lehtonen
In order to get correct metadata, SRCREV for example. Fixes [YOCTO #7648]. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23lib/oe/recipeutils: minor improvements to patch_recipe()Paul Eggleton
* Ensure it knows where to put SRCREV and S * Handle prepend/append and functions in general Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21lib/oe: add recipeutils modulePaul Eggleton
Add a module to help provide utility functions for dealing with recipes. This would typically be used by external tools. Substantial portions of this module were borrowed from the OE Layer index code; other functions originally contributed by Markus Lehtonen <markus.lehtonen@intel.com>. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>