summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2022-03-16perf-build-test/report: Drop phantomjs and html email reports supportRichard Purdie
phantomjs isn't reliable and we've moved to sharing the reports via a webserver. Update the scripts to more match those being used in the autobuilder helper where the html email support was removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15scripts/runqemu: Fix memory limits for qemux86-64Richard Purdie
When setting memory to 4GB, qemu is only running with 2GB for x86_64. Avoid this by removing the mem= option to the kernel and letting the qemu configuration handle it for x86 in a similar way to mips. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-11oe-pkgdata-util: Adapt to the new variable override syntaxPeter Kjellerstedt
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-09meta/scripts: Improve internal variable namingSaul Wold
Update internal variable names to improve the terms used. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-08wic: Use custom kernel path if providedBill Pittman
If the custom kernel path is provided in options, then use that path instead of the default path. Signed-off-by: Bill Pittman <bill.pittman@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04convert-variable-renames: Fix output stringSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02recipetool/devtool: Further SPDX identifier cleanupsRichard Purdie
Some of these are hard to know what to do with since the original source files for the checksum aren't present. The safe option is to use "-only" as often the main license is ambiguous and the source files themselves determine the "or-later" possibility. The "-only" option therefore is realistically what we need to use in this code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02scripts: Various typo/grammar/punctuation fixesRobert P. J. Day
Among other things, fix misspellings of: - absolute - deprecated - suitable - handle and a bunch of other things. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool: Use SPDX license identifiersPeter Kjellerstedt
There are still a couple of cases where the license may be set as, e.g., "GPL" or "GPL-2.0" since there is not enough information to decide the actual SPDX license. It is then assumed that the developer will have to correct the information. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool/create_buildsys_python: Add support for more known licensesPeter Kjellerstedt
Add all OSI approved licenses from https://pypi.org/classifiers/. Also add support for Other/Proprietary (Proprietary) and Public Domain (PD). Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25scripts/documentation-audit: Use renamed LICENSE_FLAGS_ACCEPTED variableKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-spdx-licensesScott Murray
The convert-spdx-licenses.py script needs the same file closing fix as was made to convert-variable-renames to ensure modified file contents get flushed out. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-variable-renamesScott Murray
In my test environments (Fedora 35 and Debian 10.10 on AMD 2970WX), running the convert-variable-renames.py rename script was resulting in empty files instead of updated ones. From inspection, the new files are never flushed/closed before moving them into place, which seems inherently racy. Adding an explicit close to flush the modified contents out before moving into place fixes the issue for me. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21meta: Rename LICENSE_FLAGS variableSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21icecc: Improve variables/terminologyRichard Purdie
The SYSTEM and USER seperation between variables seems odd and not necessary, drop it. Avoid the use of whitelist/blacklist and also change "packages" to "recipes" since that misuse causes confusion. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21scripts: Add convert-variable-renames script for inclusive language variable ↵Saul Wold
renaming This script searches for a list of variable that have been renamed and converts them to their more descriptive names. It also searches for a list of variables that have been removed or deprecated and prints a message. It will print a message to inform the user that there are terms that need to be updated in their files. Many of these changes are context sensitive and may not be modified as they might be existing calls to other libraries. This message is informational only. I have tested this on poky and meta-openembedded so far. (From OE-Core rev: 50fe7ba8dba05a9681c9095506f798796cfc2750) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21meta/scripts: Change BB_ENV_EXTRA_WHITE -> BB_ENV_PASSTHROUGH_ADDITIONSRichard Purdie
After the change to bitbake, update the references in OE-Core to match the updates. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21yocto-check-layer: add ability to perform tests from a global bbclassDenys Dmytriyenko
This is useful when needing to test layer's recipes, where this special bbclass can define a global python function that gets called on each recipe parsing during "bitbake -S none world" signature dump and be able to fail layer's check accordingly. First test being added is to detect recipes skipping "installed-vs-shipped" QA check. As "installed-vs-shipped" is a packaging QA check, it happens very late in the build process and failing it could mean some potential issues with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and resulting package is used in an SDK. In OE-Core failing this QA check leads to an error, but other layers can suppress it or change it to a warning. Detecting weird packaging problems with SDKs is quite difficult and time consuming. Also, waiting for the actual "installed-vs-shipped" packaging QA check to fail means that all recipes in the layer under test have to run through all standard tasks in the build chain, equivalent to a multi-hour world-build. Hence yocto-check-layer takes a shortcut and only detects a mere attempt at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during initial parsing when dumping the signature information for the layer. Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21devtool: explicitly set main or master branches in upgrades when availableAlexander Kanavin
In particular this resolves devtool's inability to pick a branch when the same tagged commit is avaiable in main and in a release maintenance branch. Thanks to Peter Kjellerstedt for the suggestion. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-20scripts: Add a conversion script to use SPDX license namesRichard Purdie
We're standardising on SPDX license names so it is overdue to change the ones in the LICENSE fileds to the SPDX values. Add a conversion script which makes this straightforward on the most part. Ultimately this allows the core code to be more efficient and not need to consult the mappings. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16wic: rawcopy: Add support for packed imagesStefan Herbrechtsmeier
Add support for packed images to wic rawcopy handler do minimize disk usage in deploy directory and reuse of packed images between wic and swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz archives. Example: part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack" Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16wic: partition: Support valueless keys in sourceparamsStefan Herbrechtsmeier
Accept valueless keys in sourceparams without equals sign (=) to match the comment and support Boolean entries. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-12scriptutils: Fix handling of srcuri urlsRichard Purdie
Bitbake is now requiring there be a call to get_srcrev() when urls contain floating revisions. This breaks some recipetool workflows since it is running these queries to determine those revisions. Add a PV entry which triggers that call to avoid the errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07runqemu: preload uninative libraries when host gl drivers are in useAlexander Kanavin
Some of the host distributions build the drivers in a way (RPATH/RUNPATH) that tricks uninative loader into loading pieces of the host libc, if the same pieces haven't been previously loaded by native binaries. Mixing the two libc versions leads to failures. This change ensures that the correct (uninative) versions are always in use. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07qemu: replace a gtk wrapper with directly setting environment from runqemuAlexander Kanavin
The wrapper is executed by host bash, and host bash refuses to run when pseudo libc is preloaded via LD_PRELOAD (which is the case when gl is enabled). Only the fontconfig setting is carried over as local testing showed that only that is necessary for the gui to look ok nowadays; adjust further if necessary. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07devtool: deploy-target: Remove stripped binaries in pseudo contextFlorian Amstutz
deploy-target may fail the second time with "pseudo abort" because devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot context. Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05yocto-check-layer: check for duplicate layers when finding layersRoss Burton
detect_layers() is very greedy and if it recurses into poky or bitbake it will find the test suite layers, such as bitbake/lib/layerindexlib/tests/testdata/layer4. This is a dummy layer which claims to be openembedded-layer, so if the real openembedded-layer is a dependency then layer4 may be used instead, which will cause errors: initially because it's only compatible with Sumo, but later because it doesn't contain any recipes. Add a check that the set of layers we've found doesn't contain any duplicate collection names with different patterns, and abort if that is the case as the test will be non-deterministic. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05scripts/runqemu-ifdown: Don't treat the last iptables command as specialDaniel Müller
The runqemu-ifup script performs a bunch of setup steps that runqemu-ifdown attempts to undo later on. While a bunch of said setup operations are considered fatal should they fail, the iptables based NAT setup notably is not. The tear down procedure in runqemu-ifdown, however, has the iptables based tear down as the last operation, with the status of it determining the overall status of the script. Hence, if this step fails, the script is considered a failure overall. That is arguably inconsistent: If the NAT setup did not succeed, the tear down cannot succeed either. To ensure similarity of the two paths, let's not treat the last iptables tear down operation any special and allow it to fail the runqemu-ifdown script, but just ignore failures. Background: we have seen a NAT related setup problem on the ifup path (which didn't cause script failure), but then saw an issue bubbled up when this operation was meant to be undone on the ifdown path. Signed-off-by: Daniel Müller <muellerd@fb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05recipetool: Fix circular reference in SRC_URISaul Wold
When creating a new recipe.bb file for a binary, don't use BP which includes the version information, instead use BPN which is just the name base Package Name. Since PB is not specified, it takes the default: PV = "1.0+git${SRCPV}" But SRCPV is defined in terms of the SRC_URI, which leads to infinite recursion (traceback below). Here are the pertinent variables which cause the recursion: SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}" BP = "${BPN}-${PV}" PV = "1.0+git${SRCPV}" SRCPV = "${@bb.fetch2.get_srcrev(d)}" def get_srcrev(d, method_name='sortable_revision'): # ... trimmed scms = [] fetcher = Fetch(d.getVar('SRC_URI').split(), d) # ... trimmed [YOCTO #14040] Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05prelink: Drop support for itRichard Purdie
Prelink is being dropped by glibc in 2.36. It already causes issues with binary corruption, has a number of open bugs and is of questionable benefit without disabling load address randomization and PIE executables. We disabled it by default a while back but left people able to use it. We would be unable to maintain it alone without glibc support so remove the remaining pieces. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-03devtool: sdk-update: Remove an unnecessary \n from SSTATE_MIRRORSPeter Kjellerstedt
Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror entries) there is no need to separate the entries in SSTATE_MIRRORS with "\n". Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-01yocto-check-layer: add debug output for the layers that were foundRoss Burton
When debugging weird yocto-check-layer output it is useful to know what the tool found when looking for layers. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12recipetool/create_buildsys_python.py: less distutilsTim Orling
'distutils' is deprecated in Python 3.10 with removal in Python 3.12 (~October 2023). Replace 'distutils.command.build_py' with 'setuptools.command.build_py'. To avoid an AttributeError, we call super().__init__() which provides the missing 'distribution' attribute. However, for some reason, __init__() in 'setuptools.command.build_py.build_py' class requires a 'dist' positional argument which must be a 'Distribution' instance. It is not clear why 'distutils.command.build_py.build_py' class does not require this. There is still a check which decides to inherit setuptools3 vs distutils3 that will need to be refactored when we add pyproject.toml and setup.cfg support for more modern PEP 517 packaging. Once distutils3.bbclass is dropped, any recipe inheriting distutils3 will throw a parsing error. The plan is to move distutils*.bbclasses to meta-python. However if meta-python is not in bblayers, the parsing error would still occur. [YOCTO #14610] Signed-off-by: Tim Orling <tim.orling@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12buildtools-installer: Update to use 3.4Richard Purdie
This updates buildtools to use the tarball from the 3.4 release which contains some bug fixes and is what the autobuilder currently uses for testing on older distros. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-11qemuboot/runqemu: fully form the ip= kernel parameterAlexander Kanavin
New systemd is actually parsing this in systemd-network-generator and fails if it is not fully formed. 'off' means 'static ip, do nothing': https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-05scripts: Update to use exec_module() instead of load_module()Richard Purdie
This is deprecated in python 3.12 and Fedora 35 is throwing warnings so move to the new functions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-04scripts/buildhistory-diff: drop use of distutilsTim Orling
The use of distutils.version.LooseVersion to check for GitPython > 0.3.1 is not really needed anymore since any supported distribution has at least 1.0.0 (centos-7 via epel7, debian-9, ubuntu-16.04) If we want to reinstate this check, alternatives would be to require python3-packaging on all hosts and use packaging.version.Version or use an imported LooseVersion in bb.version. [YOCTO #14610] Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20recipetool: Separate licenses with & operatorStefan Herbrechtsmeier
Separate licenses with & operator since it should be satisfied most use cases and it is a reasonable assumption that all the licenses apply. Furthermore flat, split and sort the licenses to minimize license string changes. Separate package licenses with & operator: -LICENSE:${PN} = "MIT ISC" +LICENSE:${PN} = "ISC & MIT" Respect | and brackets in LICENSE: -LICENSE = "BSD-3-Clause & (ISC & | & MIT)" +LICENSE = "BSD-3-Clause & (ISC | MIT)" Sort licenses: -LICENSE = "MIT & BSD-3-Clause & ISC" +LICENSE = "BSD-3-Clause & ISC & MIT" Remove duplicates: -LICENSE = "MIT & ISC & MIT" +LICENSE = "ISC & MIT" Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20recipetool: Sort output of guess_license function to be deterministicStefan Herbrechtsmeier
Sort the output of guess_license function by license file to be deterministic and to support self-testing. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-17runqemu: additional setting to force software rendering with sdl 2.0.18Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09recipetool: Set master branch only as fallbackStefan Herbrechtsmeier
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)' sets the branch= parameter too early to master and thereby breaks the -B/--srcbranch option. ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09runqemu: check the qemu PID has been set before kill()ing itRoss Burton
If runqemu is killed, check that we have a valid PID for the qemu before sending a kill() to it. [ YOCTO #14651 ] Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-04yocto-check-layer: Relax README case checksJoshua Watt
Relaxes the case requirements for checks in the README file so that word like "Maintainer" and "Patch" are allowed Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-04yocto-check-layer: disregard checks if referencing another README fileJon Mason
It is permissible to have a layer's README reference another README. If this is the case, avoid other checks. Do this by checking for the word README (case insensitive) in the README file. This might be too permissive, but follows the pattern of looking for exact words (like "patch" or "maintainer") in READMEs. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03yocto-check-layer: Add additional README checksDhruva Gole
This adds: - Test case to review if the README file specifies the maintainer. - Test case to review if the README file specifies the word patch. - Test case to review if the README file contains an email address. [YOCTO #11131] Signed-off-by: Dhruva Gole <goledhruva@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03recipetool: extend curl detection when creating recipesRoss Burton
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03recipetool: handle GitLab URLs like we do GitHubRoss Burton
GitHub URLs are automatically transformed to git: fetches, so handle GitLab URLs too. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-01scripts/checklayer/common.py: Fixed a minor grammatical errorDhruva Gole
Signed-off-by: Dhruva Gole <goledhruva@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-26native/cross: Add ar wrapper for determinismJacob Kroon
Add a wrapper around ar calls for native/cross recipes. This wrapper adds the -D option so that deterministic archives are built for native/cross output. This improves the changes of hash equivalence matches and hence build artefact reuse. We don't need this in the target case since we compile binutils-cross with an option making this the default. We need a wrapper since we need to remove the "u" option and replace it with "D" but also allow things like "--version" to continue to work too. Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-26nativesdk: Handle chown/chgrp calls in nativesdk do_install tasksRichard Purdie
We disable the useradd code for nativesdk targets since we don't support postinstalls or multiple users in those cases. This means any usage of chown/chgrp inside do_install tasks won't work and would have to be conditional. Rather than require all recipes to do that, add intercepts of the calls and map those to root/root user/groups. We can't just ignore them as some calls are used to remove host contamination from the host user ID so they need to be made, just as root. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-04-18 10:31:14 +0000