aboutsummaryrefslogtreecommitdiffstats
path: root/lib/bb/data_smart.py
AgeCommit message (Collapse)Author
2021-11-24bitbake: adjust parser error check for python 3.10 compatibilityAlexander Kanavin
The change was introduced in https://github.com/python/cpython/commit/a698d52c3975c80b45b139b2f08402ec514dce75 Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8d3c6cbbe6ee734495713ae3b99c609527842506) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-24bitbake: correct the collections vs collections.abc deprecationAlexander Kanavin
This becomes a hard error in python 3.10. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit ae219e1f7460077f4492b31ac91cef4cf9b17277) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-24data_smart: Improve error display for handled exceptionsRichard Purdie
We don't need tracebacks for BBHandledException. Reduces confusing output like: ERROR: /meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio ERROR: ExpansionError during parsing /meta/recipes-core/images/core-image-tiny-initramfs.bb Traceback (most recent call last): File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_qa', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_cpio', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1038, in follow_chain(task='do_image_complete', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in chain: > bb.fatal("Circular task dependencies as %s depends itself via the chain %s?!" % (task, " -> ".join(chain))) chain.append(task) File "/bitbake/lib/bb/__init__.py", line 165, in fatal: mainlogger.critical(''.join(args), extra=kwargs) > raise BBHandledException() to the real error: ERROR: /media/build1/poky/meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 551d4c0576a0a0c3406000029df9238b312f2263) Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
2021-07-30data_smart: Allow colon in variable expansion regexRichard Purdie
Now that ":" is a valid character in variable key names, it needs to be allowed by the variable expansion code too, to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-30data_smart/parse: Allow ':' characters in variable/function namesRichard Purdie
It is becomming increasingly clear we need to find a way to show what is/is not an override in our syntax. We need to do this in a way which is clear to users, readable and in a way we can transition to. The most effective way I've found to this is to use the ":" charater to directly replace "_" where an override is being specified. This includes "append", "prepend" and "remove" which are effectively special override directives. This patch simply adds the character to the parser so bitbake accepts the value but maps it back to "_" internally so there is no behaviour change. This change is simple enough it could potentially be backported to older version of bitbake meaning layers using the new syntax/markup could work with older releases. Even if other no other changes are accepted at this time and we don't backport, it does set us on a path where at some point in future we could require a more explict syntax. I've tested this patch by converting oe-core/meta-yocto to the new syntax for overrides (9000+ changes) and then seeing that builds continue to work with this patch. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-19data_smart: Ensure hash reflects vardepvalue flags correctlyRichard Purdie
The get_hash() function is used to decide if the base configuration has changed and hence whether a reparse is required. The vardepvalue flag's value was not expanded but it is often used in contexts like: METADATA_REVISION = "${@base_detect_revision(d)}" METADATA_REVISION[vardepvalue] = "${METADATA_REVISION}" which in it's unexpanded form means reparsing doesn't happen when it should as the data appears unchanged. Update get_hash to expand the values of vardepvalue so reparsing works as expected. This avoids basehash mismatch errors such as the one recently caused by using METADATA_REVISION in poky.conf's DISTRO_VERSION variable. The issue there could be exposed by a recipe using DISTRO_VERSION with the sequence: bitbake os-release <change the revision of the metadata with a dummy commit> bitbake os-release -C install which was caused because METADATA_REVISION changed but the metadata didn't reparse. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-25lib: fix most undefined code picked up by pylintFrazer Clews
Correctly import, and inherit functions, and variables. Also fix some typos and remove some Python 2 code that isn't recognised. Signed-off-by: Frazer Clews <frazerleslieclews@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24data_smart: Handle hashing of datastores within datastores correctlyRichard Purdie
If there is a datastore within a datastore (e.g. BB_ORIGENV) then get-hash() doesn;t correclty handle the contents using the memory address instead of the contents. This is a patch from dominik.jaeger@nokia.com which addresses this problem. Its been low priority since we don't include BB_ORIGENV anywhere this would cause an issue as standard. [YOCTO #12473] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24tinfoil: Simplify remote datastore connectionsRichard Purdie
The current approach to remote datastores used in tinfoil is breaking. For example, adding a devupstream extension to a recipe with a git upstream, making it the preferred version and then running "devtool modify" on it causes get_srcrev() circular dependency issues. The problem is the override handling in the datastore is broken. This gets broken since remotedata:recieve_datastore() sets d.dict but doesn't update d.overridedata (or d.inchistory or d.varhistory). We could play whack-a-mole but the current implementation seems to be flawed to me. It also doesn't cover, or only partially covers some datastore operations and each needs new dedicated command API. Instead, step back and reimplement the way the datastore connector works. With this change, the datastore is either remote or local but the data is not spread on two sides of the connection. All the API is proxied over the connection by a single function for the datastore (and two to support variable history and include history). This code does not support using the datastore as a parameter to any data store functions. We did have one case of that but its just bad code and can be replaced. The result is something which is much simpler and less invasive to the datastore code itself, meaning its behaviour should be much more consistent. The existing tests for the remote data no longer make any sense and are removed. The one bug this code would have is if key/value pairs are returned over the IPC and those values contained a DataSmart object since we don't recurse into return values to find such things. Nothing appears to do that currently so lets worry about it if its ever an issue. This change should simplfy a ton of other issues and avoid a ton of other bugs so is a huge net gain. Tested with bitbake's and OE's selftests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-23data_smart: Don't pass unneeded datastoreRichard Purdie
The datastore is already available to this function internally so don't also try and pass the datastore as a parameter. This is clearly broken API when you look at the existing calls to it. This then doesn't break the planned tinfoil data connector changes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-19data: Don't allow renameVar calls with equivalent keysKyle Russell
While usually a programming error, the behavior can cause a Parser instance to eventually gobble up a significant amount of memory, greatly affecting system performance. Try to avoid getting into that situation and alert the user about what they attempted to do. Signed-off-by: Kyle Russell <bkylerussell@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04bitbake: Strip old editor directives from file headersRichard Purdie
There are much better ways to handle this and most editors shouldn't need this in modern times, drop the noise from the files. Its not consitently applied anyway. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04bitbake: Drop duplicate license boilerplace textRichard Purdie
With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie
This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-18bb.data_smart: only try to expand refs to valid variable namesChristopher Larson
This aligns the behavior of expansion with the recipe parser, only attempting to expand references to valid variable names. This avoids adding references for things like `${foo#${TOPDIR}}` to our vardeps without imposing much additional processing overhead beyond the change to the expansion regexp. YOCTO #12987 Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08data/siggen: Switch md5 -> sha256Richard Purdie
Similarly to the codeparser change, change to sha256 hashes due to worries over collisions. The main impact of this change is slightly slower parsing time as well as longer sstate file names. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14data_smart: Allow numeric characters in overridesRichard Purdie
We're seeing problems due to the way x86-64 is handled (or not handled) as an override. Relax the containts on overrides from being lowercase to being lowercase or numeric. This fixes problem where MACHINE=qemux86 would work but MACHINE=qemux86-64 would fail the same tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14data_smart: Add missing regexp markupRichard Purdie
Fix some further python3 warnings about unescaped regexs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16data_smart: fix filename for compile()Robert Yang
Fixed: Add the following two lines to conf/local.conf: FOO = "${@foo = 5}" HOSTTOOLS += "${FOO}" * Before the patch $ bitbake -p Check the first lines of bitbake bitbake-cookerdaemon.log [snip] File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 125, in python_sub codeobj = compile(code.strip(), self.varname or "<expansion>", "eval") File "FOO", line 1 [snip] There isn't a file named 'FOO', but a variable name. * After the patch $ bitbake -p [snip] File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 129, in python_sub codeobj = compile(code.strip(), varname, "eval") File "Var <FOO>", line 1 foo = 5 Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16data_smart: Add original traceback to ExpansionErrorRobert Yang
This can make it print clearer errors, for exmaple: Add Runtime_error to 'def oe_import(d)" 16 def oe_import(d): 17 import sys 18 Runtime_error [snip] * Before the patch: $ bitbake -p ERROR: Unable to parse /buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py Traceback (most recent call last): File "/buildarea1/lyang1/poky/bitbake/lib/bb/data_smart.py", line 430, in DataSmart.expandWithRefs(s='${@oe_import(d)}', varname='OE_IMPORTED[:=]'): except Exception as exc: > raise ExpansionError(varname, s, exc) from exc bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception NameError: name 'Runtime_error' is not defined This error message has two problems: - "Unable to parse data_smart.py": This isn't the real cause. - It pionts to "raise ExpansionError(varname, s, exc) from exc" which isn't clear enough. * After the patch: $ bitbake -p ERROR: Unable to parse OE_IMPORTED[:=] Traceback (most recent call last): File "OE_IMPORTED[:=]", line 1, in <module> File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 18, in oe_import(d=<bb.data_smart.DataSmart object at 0x7f9257e7a0b8>): import sys > Runtime_error bb.data_smart.ExpansionError: Failure expanding variable OE_IMPORTED[:=], expression was ${@oe_import(d)} which triggered exception NameError: name 'Runtime_error' is not defined This one is more clearer than before. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-19data_smart: Fix removal handling interaction issue with overridesRichard Purdie
If a variable has a _remove applied to it but that variable is in turn 'renamed' through OVERRIDES, the removal gets lost with the current code. TEST = "foo" TEST_someval = "bar" TEST_someval_remove = "bar" OVERRIDES = "someval" currently gives "bar" for TEST but should give "". This fixes the code to track the removal and adds a test case to ensure this doesn't regress again. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-17bitbake: data: Ensure task checksums account for remove dataRichard Purdie
Currently remove operations are not being accounted for in the task checksums. This is a fairly serious oversight and needs to be fixed. To do so, we need internal data from getVarFlag combined with the expanded variable data so that only "active" remove operators are accounted for in the task checksum. We can get this from the new optional removes attribute in the returned parser object. The code can then use the data on active remove operators to account for the removals in task checksum but only when the removal is active. We have to be careful here not to reference any expanded data since this may for example contain build paths. This means we can only map back and reference the unsplit (and hence unexpanded) remove string which may expand to multiple removal values. [YOCTO #12913] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-17data/data_smart: Allow getVarFlag to return the variable parser objectRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-17data_smart: Fix expand_cache and _remove operator interaction issuesRichard Purdie
The contents of the expand_cache is meant to match the return value of getVarFlag() but the implementation was mostly in expandWithRefs(). If an incorrect key was passed to expandWithRefs(), or a variable was only partially expanded with no remove processing, the cache could become corrupted. Move the code to getVarFlag making the data lifecycle very clear, meaning other calls to expandWithRefs() cannot corrupt the cache. The expand_cache reset code needs to be moved ahead of any remote data connectors too, since the expand_cache is now on the local side of the connection. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-17data_smart: Micro optimise _remove handlingRichard Purdie
This brings _remove handling into line with _append/_prepend with regard to the parsing flag to getVarFlag. This is an internal flag and the only times this is used is through getVar during renameVar operations and when processing ?= operations to see if a variable is set. In either case we don't need to process remove operations. Therefore take the minor speedup and skip processing for parsing=True. [YOCTO #10945] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-16data: Fix whitespace on _remove operationsRichard Purdie
We have some slightly odd behaviours with the current implementation of _remove operations. For example: TEST = " A B" TEST_remove = "C" would trigger TEST to become "A B" even thought it doesn't contain "C". In particular, this means that an inactive remove operator added in a bbappend could change the task checksum which is not desireable. Fix the operation to preserve whitespace, adding new tests to make this explict and test further corner cases. Also update the manual to match. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-16data_smart: Preserve history when renaming variablesRichard Purdie
Currently when a variable is renamed, all the variable history is lost. This makes it hard for users to understand/debug where values came from. Improve the code so that history is preserved across renamed variables. (Expanded variable key names are a special case of variable renaming) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-16data_smart: Don't cache/process capitalised overridesRichard Purdie
Bitbake now only processes overrides which are lowercase since this allows variables like SRC_URI not to pollute the cache. There was a corner case where XXX_append_SomeThing was still being processed (yet XXX_append_SomeThing_SomeOtherThing would not be). This patch ensures we're consistent and only process lowercase _append/_prepend and _remove operators too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11data_smart: Ensure _remove operations on newly set variables are clearedRichard Purdie
We clear append/prepend on newly set variables, we should also clear remove operations. If we don't do this, there is no way we can actually delete a remove operation. Bitbake internally uses parsing=True to avoid these side effects when making its own internal calls. Also add a testcase to bitbake-selftest to ensure we remain consistent going forward from here. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21data_smart: implement missing remote datastore operationsPaul Eggleton
Enable the following operations from a remote datastore to affect the other end: * setVarFlag() * delVar() * delVarFlag() * renameVar() In practice I don't expect these to be used much, but they should be present so that the implementation is at least reasonably filled out and that the tests pass. Also add tests for the interface, mostly by subclassing the existing local test classes so that they are using a remote datastore. (These don't actually test remote usage via tinfoil, just that the datastore's interface can be used.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-21tinfoil: fix override handling in remote datastoresPaul Eggleton
There was a huge gap in the remote datastore code introduced in the tinfoil2 rework - we weren't handling overrides at all, since these are stored separately from the actual data in the DataSmart object. Thus, when a datastore actually represents a remote datastore we need to go back to that remote datastore to get the override data as well, so introduce code to do that. To avoid a second round-trip I had to modify the _findVar() function to return the override data as well. This will increase the overhead a little when that data is superfluous, but without making the function even uglier I don't think there's a way to avoid that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-13lib: Drop now unneeded update_data callsRichard Purdie
Now that the datastore works dynamically we don't need the update_data calls so we can just remove them. They're not actually done anything at all for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20data_smart: Fix unneeded variable manipulationRichard Purdie
If was pointed out that if we have: XXX = " A" XXX_remove_inactive-override = "YY" then XXX can become "A" and the leading space can be removed. This is because the remove override code changes the variable value even when there is no removals active. In the process it dirties the cache. We don't really need to do this so tweak the code accordingly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14data_smart: support serialisationPaul Eggleton
The COW object used within VariableHistory can't be serialised itself, but we can convert it to a dict when serializing and then back when deserialising. This finally allows DataSmart objects to be serialized. NOTE: "serialisation" here means pickling, not over XMLRPC or any other transport. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14tinfoil: pass datastore to server when expanding python referencesPaul Eggleton
If you're expanding a value that refers to the value of a variable in python code, we need to ensure that the datastore that gets used to get the value of that variable is the client-side datastore and not just the part of it that's on the server side. For example, suppose you are in client code doing the following: d.setVar('HELLO', 'there') result = d.expand('${@d.getVar("HELLO", True)}') result should be "there" but if the client part wasn't taken into account, it would be whatever value HELLO had in the server portion of the datastore (if any). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14data_smart: implement remote datastore functionalityPaul Eggleton
This allows you to maintain a local reference to a remote datastore. The actual implementation of the remote connection is delegated to a connector object that the caller must define and supply. There is support for getting variable values and expanding python references (i.e. ${@...} remotely, however setting variables remotely is not supported - any variable setting is done locally as if the datastore were a copy (which it kind of is). Loosely based on an earlier prototype implementation by Qing He. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14data_smart: fix resetting of reference on variablehistoryPaul Eggleton
There is no "datasmart" member, only dataroot. This dates back to the original implementation of variable history support - it's surprising we haven't noticed the issue until now, but I guess it's rare to change a copy of a datastore in a manner which using the old reference would cause an issue. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: remove True option to getVarFlag callsJoshua Lock
getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: remove True option to getVar callsJoshua Lock
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23data_smart.py: don't reorder internal bitbake variables when calculating hashPatrick Ohly
Commit 260ced745 added __BBTASKS, __BBANONFUNCS, __BBHANDLERS to the data that gets hashed, but only after reordering these lists. The intention probably was to make the hash deterministic, but that's unnecessary (the content of the variables should already be deterministic) and hides potential reasons that might require re-parsing. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23data_smart: Default to expansion for getVar/getVarFlagsRichard Purdie
We've been building to this for a while, default to return expanded values for getVar/getVarFlags. We can then go through and remove the "True" option to many of the calls to this function, all function calls should have a default by now though since the parameter has been required for a while. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-13data_smart/utils: Add 'd' to the context used for better_eval in python ↵Richard Purdie
expansion If a line like: foo=${@' '.join([d.getVar('D', True) + x for x in (' '.join([d.getVar('FILES_bash-' + p, True) or '' for p in ['lib', 'dev', 'staticdev', 'doc', 'locale', 'ptest']])).split()])} is added to a function like do_install, it fails with Exception name 'd' is not defined. This is due to a change of behaviour in python 3 compared to python 2. Generator expressions, dict comprehensions and set comprehensions are executed in a new scope but list comprehensions in python 2.x are not. In python 3 they all use a new scope. To allow these kinds of expressions to work, the easiest approach is to add 'd' to the global context. To do this, an extra optional parameter is added to better_eval and we use that to add 'd'. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01data_smart: Simplify ExpansionError exceptionRichard Purdie
This seemingly convoluted syntax doesn't work in python3. Instead use the chained exception handling syntax which appears to make more sense here. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01bitbake: Convert to python 3Richard Purdie
Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06bb.data_smart: use iter() for __len__Christopher Larson
It seems the frozenset constructor in pypy runs len(), so we can't pass the DataSmart instance directly to it, instead pass the iterator. Fixes pypy support. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-09data_smart: Restrict expansion regexp to not include : charactersRichard Purdie
Bitbake variables don't include ":" characters so exclude these from the variable expansion regexp. This assists when parsing shell code which does A=${B:-C} as we don't want a dependency on a variable called "B:-C". Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18data_smart: Drop default expand=False to getVarFlag [API change]Richard Purdie
At some point in the future, getVarFlag should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18data_smart: Drop default expand=False to getVar [API change]Richard Purdie
At some point in the future, getVar should expand by default. To get there from the current position, we need a period of time where the expand parameter is mandatory. This patch starts that process. Clear errors will result from any code which doesn't provide this. Layers can be fixed with an expression like: sed -e 's:\(\.getVar([^,()]*\)):\1, False):g' -i `grep -ril getVar *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10data_smart: Avoid expanding anonymous python functionsRichard Purdie
We don't expand anonymous python before execution, so nor should we do this when calculating checksums for them. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04lib/bb: Add expansion parameter to getVarFlagRichard Purdie
This sets the scene for removing the default False for expansion from getVarFlag. This would later allow True to become the expand default. On the most part this is an automatic translation with: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, False):g' -i `grep -ril getVar *` There should be no functional change from this patch. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>