summaryrefslogtreecommitdiffstats
path: root/scripts/lib/recipetool
AgeCommit message (Collapse)Author
2022-02-05recipetool: Fix circular reference in SRC_URISaul Wold
When creating a new recipe.bb file for a binary, don't use BP which includes the version information, instead use BPN which is just the name base Package Name. Since PB is not specified, it takes the default: PV = "1.0+git${SRCPV}" But SRCPV is defined in terms of the SRC_URI, which leads to infinite recursion (traceback below). Here are the pertinent variables which cause the recursion: SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}" BP = "${BPN}-${PV}" PV = "1.0+git${SRCPV}" SRCPV = "${@bb.fetch2.get_srcrev(d)}" def get_srcrev(d, method_name='sortable_revision'): # ... trimmed scms = [] fetcher = Fetch(d.getVar('SRC_URI').split(), d) # ... trimmed [YOCTO #14040] Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12recipetool/create_buildsys_python.py: less distutilsTim Orling
'distutils' is deprecated in Python 3.10 with removal in Python 3.12 (~October 2023). Replace 'distutils.command.build_py' with 'setuptools.command.build_py'. To avoid an AttributeError, we call super().__init__() which provides the missing 'distribution' attribute. However, for some reason, __init__() in 'setuptools.command.build_py.build_py' class requires a 'dist' positional argument which must be a 'Distribution' instance. It is not clear why 'distutils.command.build_py.build_py' class does not require this. There is still a check which decides to inherit setuptools3 vs distutils3 that will need to be refactored when we add pyproject.toml and setup.cfg support for more modern PEP 517 packaging. Once distutils3.bbclass is dropped, any recipe inheriting distutils3 will throw a parsing error. The plan is to move distutils*.bbclasses to meta-python. However if meta-python is not in bblayers, the parsing error would still occur. [YOCTO #14610] Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20recipetool: Separate licenses with & operatorStefan Herbrechtsmeier
Separate licenses with & operator since it should be satisfied most use cases and it is a reasonable assumption that all the licenses apply. Furthermore flat, split and sort the licenses to minimize license string changes. Separate package licenses with & operator: -LICENSE:${PN} = "MIT ISC" +LICENSE:${PN} = "ISC & MIT" Respect | and brackets in LICENSE: -LICENSE = "BSD-3-Clause & (ISC & | & MIT)" +LICENSE = "BSD-3-Clause & (ISC | MIT)" Sort licenses: -LICENSE = "MIT & BSD-3-Clause & ISC" +LICENSE = "BSD-3-Clause & ISC & MIT" Remove duplicates: -LICENSE = "MIT & ISC & MIT" +LICENSE = "ISC & MIT" Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20recipetool: Sort output of guess_license function to be deterministicStefan Herbrechtsmeier
Sort the output of guess_license function by license file to be deterministic and to support self-testing. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09recipetool: Set master branch only as fallbackStefan Herbrechtsmeier
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)' sets the branch= parameter too early to master and thereby breaks the -B/--srcbranch option. ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03recipetool: extend curl detection when creating recipesRoss Burton
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03recipetool: handle GitLab URLs like we do GitHubRoss Burton
GitHub URLs are automatically transformed to git: fetches, so handle GitLab URLs too. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30meta/scripts: Manual git url branch additionsRichard Purdie
Following the scripted conversion adding branches to git:// SRC_URI entries, add the remaining references, mainly in the selftests and recipetool. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-23recipetool: Simplify common source files skip in guess_licenseStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Rework crunch_license to recognize more variantsStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: npm: Use README as license fallbackStefan Herbrechtsmeier
Use the README as license fallback if a license file is missing. Use the linenumbers parameter of get_license_md5sums function to determine the license text inside the README. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUMStefan Herbrechtsmeier
The package.json files doesn't contain any licenses. The name of the license doesn't comply the license requirements of most liceneses. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Add support for linenumbers to licenses.csvStefan Herbrechtsmeier
Add support for linenumbers (begin and end lines) to licenses.csv. Add an optional linenumbers parameter to get_license_md5sums to support different use cases. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Add logger info for missing license entriesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: ignore empty license filesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Change default paramter fallback_licenses of function ↵Stefan Herbrechtsmeier
split_pkg_licenses from None to [] Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Skip common source files in guess_licenseStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Move license md5sums into CSV filesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03recipetool/create_buildsys_python: treat BSD as BSD-3-ClauseRoss Burton
The PyPI license classifiers include "OSI Approved: BSD", which does not specify which of the many variations of BSD license it actually means. The generic "BSD" license in the oe-core set is actually BSD-3-Clause. >From a random sample of ten PyPI modules that use "BSD", they are all BSD-3-Clause. As we expect the recipe maintainer to verify the license anyway, and this matches the previous license text, explicitly set the license to BSD-3-Clause. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-17package/scripts: Fix FILES_INFO handlingRichard Purdie
There is a long standing bug where FILES_INFO isn't written into pkgdata with a package suffix. This means if the files are read into the datastore as intended, the last one "wins". Fix this to work as intended. Most of the call sites using the data need to be updated to handle this and the overrides change correctly. Also fix some other problematic references noticed along the way. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-30Convert to new override syntaxRichard Purdie
This is the result of automated script conversion: scripts/contrib/convert-overrides.py <oe-core directory> converting the metadata to use ":" as the override character instead of "_". Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-16recipetool: create: only add npmsw url if requiredKamel Bouhara
Before adding a npmsw fetcher to a recipe we should first check if the generated shrinkwrap file contains dependencies. Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-12scripts/lib/recipetool/create.py: fix regex stringsTim Orling
Python now expects regex strings to be prepended with r. Silence pylint/autopep8 and similar warnings by identifying these regex patterns as... regex patterns. Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-17recipetool: create: fix SRCBRANCH not being passed to paramsTuomas Salokanto
When explicitly passing a branch using --srcbranch in 'devtool add' or 'recipetool create', the branch name is not included in the params of bb.fetch2.encodeurl and default 'master' branch is used instead. Signed-off-by: Tuomas Salokanto <tuomas.salokanto@gmail.com> Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-12recipetool: Fix list concatenation when using editAlex Kiernan
If there are multiple appends, ensure we concatenate compatible things: Traceback (most recent call last): File "/home/akiernan/poky/scripts/recipetool", line 111, in <module> ret = main() File "/home/akiernan/poky/scripts/recipetool", line 100, in main ret = args.func(args) File "/home/akiernan/poky/scripts/lib/recipetool/edit.py", line 38, in edit return scriptutils.run_editor([recipe_path] + appends, logger) TypeError: can only concatenate list (not "tuple") to list Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27recipetool/create: npm: remove the 'noverify' url parameterJean-Marie LEMETAYER
This commit removes the 'noverify' parameter which was added to the url to fix warnings with the shrinkwrap / lockdown file generation. This is not needed anymore with the new npm fetcher. Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27recipetool/create_npm: handle the licenses of the dependenciesJean-Marie LEMETAYER
As usual the 'LICENSE' and the 'LIC_FILES_CHKSUM' values reflects all the license files discovered in the source tree (including the dependencies). For npm recipes the 'LIC_FILES_CHKSUM' value contains also the status of the 'package.json' file of every packages as it contains license informations. Finally each package has a separate 'LICENSE_${PN}-package-name' value which describes its license. Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27recipetool/create_npm: refactor the npm recipe creation handlerJean-Marie LEMETAYER
This commit refactors the npm recipe creation handler to use the new npm behavior. The process is kept as simple as possible and only generates the shrinkwrap file. To avoid naming issues the recipe name is now extracted from the npm package name and not directly mapped. Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27devtool: npm: rename npm command line optionsJean-Marie LEMETAYER
This commit renames the '--fetch-dev' option into '--npm-dev' as it is a npm only option. Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30recipetool: always use python 3 in recipe creationAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21devtool: fix devtool upgrade with reproducible_builds classPaul Eggleton
If the reproducible_build class is inherited then there may be a "source-date-epoch" subdirectory in a fetched source tree; devtool upgrade was not expecting that in the upgraded source. Take a small snippet of code from recipetool create which already handles this, and make it a shared function that can be used in both places. Additionally, fix an assumption that the source is always in a subdirectory in the cleanup code that blocked debugging this. [YOCTO #13635] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21recipetool/create: Fix to work with reproducible_buildsRichard Purdie
The extra diretory created by reproducibile_builds was confusing recipetool and hence devtool. Ignore this directory to restore the correct behaviour and allow it to work with the class. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-11recipetool: add MD5 hash for the line-wrapped MPL-1.1 licenseRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03recipetool: ignore zero-length setup.py filesPaul Eggleton
If a setup.py file exists it ought to have something in it before we consider the source tree to be a Python module and treating it as such. (A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz - it's not clear why this has a zero-length setup.py in it but we should pay no attention to it.) Fixes [YOCTO #12923]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-30package: Build pkgdata specific to the current recipeRichard Purdie
This switches the code to build pkgdata specific to the current recipe which means that its filtered to the recipes dependencies and can perform better as we can drop the lockfile. It uses a similar method to the staging code to do this, using BB_TASKDEPDATA to construct a list of packagedata task output which this recipe should "see". The original pkgdata store is left unaltered so existing code works. The lock file was there to prevent files disappearing as they were read or as directories were listed. Since we have a copy of the data and only access output from completed tasks (as per their manifests), we can remove the lock. The lock was causing starvation issues on systems with parallelism. There was also a potential determinism problem as the current code could "see" data from recipes which it doesn't depend upon. [YOCTO #13412] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10recipetool: add python3 supportMaciej Pijanowski
Add support for generating python3 recipes using the recipetool / devtool. Drop python2 support at the same time. Tested with: oe-selftest -r recipetool.RecipetoolTest [YOCTO #13264] Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan
If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11recipetool: add missed moduleLei Yang
When I use recipetool to add a new recipe, it says: [snip] sys.exit(14) NameError: name 'sys' is not defined [snip] Signed-off-by: Lei Yang <Lei.Yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26logging: use warning instead warnChen Qi
The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-03recipetool: add 'edit' subcommandChristopher Larson
This edits the recipe and any bbappends for the specified target. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-05-04recipetool: fixed uncaught NameError exceptionSarah Marsh
When packaging a node application, a `NameError` can be thrown in create_npm.py if an optional npm dependency does not support Linux. Signed-off-by: Sarah Marsh <sarah.marsh@arm.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-04-03recipetool: create: fix port number parsing issueMing Liu
A flaw was found when I run: $ recipetool create "ssh://git@xxx.xxx:7999/xxx.git" the url turned out to be: "git://git@xxx.xxx/7999/xxx.git;protocol=ssh" after parsing, the port number was parsed as part of the path, this is definitely wrong and lead to fetching failures. This issue could be fixed in reformat_git_uri, by filtering out port numbers when formatting ":". Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-18recipetool: create: fix conflict between SRCREV and tagChang Rebecca Swee Fun
If you specify 'tag=' for a git URL and passed to recipetool create, you will get into Bitbake expansion error shown below: ----- snip ----- $ devtool add --version 2.4.2 mbedtls "git://github.com/ARMmbed/mbedtls;tag=mbedtls-2.4.2" ... bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Conflicting revisions (abeccb9dbd7e19ae91ac50e1edd3803111c5f9b6 from SRCREV and mbedtls-2.4.2 from the url) found, please specify one valid value ----- snip ----- Assuming the tag is valid, we should get the tag commit hash and drop the usage of 'tag=' from SRC_URI. By using a commit hash corresponding to the tag will prevent bitbake from accessing remote repository in order to expand SRCPV. Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-21recipetool: create: fix failure handling included dictsPaul Eggleton
If a setup dict in a python setup.py file pulled in the contents of another dict (e.g. **otherdict), then we got an error when mapping the keys because the key is None in that case. Skip those keys to avoid the error (we pick up the values directly in any case). A quick reproducer for this issue: recipetool create https://files.pythonhosted.org/packages/source/p/pyqtgraph/pyqtgraph-0.10.0.tar.gz Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-10recipetool: create: show a warning for github archive URLsPaul Eggleton
github archive URLs are not guaranteed to be stable [1] and thus we should show a warning if a user specifies one to recipetool create (or devtool add). [1] http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142519.html Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-10recipetool: create: drop debug printPaul Eggleton
This looks like some debug printing that was left in by accident. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-10recipetool: ignore incidental kernel module sourcePaul Eggleton
If the source tree happens to contain a kernel module as an example, a test or under a "contrib" directory then we shouldn't be picking it up and making the determination that the entire thing is a kernel module. An example that triggered this is zstd, which ships a kernel module under contrib/linux-kernel: https://github.com/facebook/zstd Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-11-10recipetool: pass absolute source tree path to pluginsPaul Eggleton
We shouldn't be passing a relative path to the plugins if that's what's been specified on the recipetool command line. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-09-13devtool/standard: set a preferred provider when adding a new recipe with devtoolJuan M Cruz Alcaraz
A recipe added with "devtool add" requires to be able to take precedence on recipes previously defined with PREFERRED_PROVIDER. By adding the parameter "--provides" to "devtool add" it is possible to specify an element to be provided by the recipe. A devtool recipe can override a previous PREFERRED_PROVIDER using the layer configuration file in the workspace. E.g. devtool add my-libgl git@git://my-libgl-repository --provides virtual/libgl [YOCTO #10415] Signed-off-by: Juan M Cruz Alcaraz <juan.m.cruz.alcaraz@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-06 21:31:28 +0000