summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2022-08-02wic: add target tools to PATH when executing native commandsRoss Burton
We might want to run a cross tool, such as objcopy, in wic. These are in a TARGET_SYS/ subdirectory under /usr/bin, so add that directory to the search path too. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
2022-07-28runqemu: Add missing space on default display optionMark Hatle
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org> Signed-off-by: Mark Hatle <mark.hatle@amd.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-28devtool: error out when workspace is using old override syntaxRoland Hieber
When the workspace bbappends are still using the old override syntax with EXTERNALSRC_pn-*, externalsrc_re will not match, and pn will never be assigned, leading to a nondescript UnboundLocalError being raised on the user's terminal. Try to detect that situation and give the user a hint how to solve it. Signed-off-by: Roland Hieber <rhi@pengutronix.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-23wic/plugins/rootfs: Fix NameError for 'orig_path'Mihai Lindner
Fix "NameError: name 'orig_path' is not defined". It's a typo from when this error was handled outside this function. Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-14devtool: finish: handle patching when S points to subdir of a git repoPaul Eggleton
If devtool finish needs to create a patch and have it applied to the sources for a recipe where S points to a subdirectory of the sources, then the patch needs to be applied at the root of the repo i.e. we need to add a patchdir= parameter to the SRC_URI entry. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-12devtool: ignore pn- overrides when determining SRC_URI overridesPaul Eggleton
If (perhaps foolishly) at your configuration level you have e.g. SRC_URI_append_pn-recipename = " file://patchname.patch" and then run devtool modify on a different recipe, an error occurs: INFO: SRC_URI contains some conditional appends/prepends - will create branches to represent these ... ERROR: [Errno 2] No such file or directory: '/path/to/downloads/patchname.patch' pn- overrides would not constitute an alternative configuration that we should handle in this context, so just ignore them to avoid the issue. Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-01scriptutils: fix style to be more PEP8 compliantMarius Kriegerowski
Signed-off-by: Marius Kriegerowski <marius.kriegerowski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-01recipetool/devtool: Fix python egg whitespace issues in PACKAGECONFIGThomas Roos
Substitute expressions or whitespace from python egg requires.txt when generating PACKAGECONFIG Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements. Recipetool parses this information to generate the PACKAGECONFIG. These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG. This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable. Also adding an oe-selftest for this. [YOCTO #14446] Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29pybootchartgui: render cpu and io pressureAryaman Gupta
Add two new, separate charts showing the avg10 and delta total pressure over time for the CPU and IO resources. The height of the avg10 data in each chart represents the percentage of time "some" task was delayed over the specific resource during the last 10 seconds of the build. The height of the delta total data in each chart represents the total time "some" task was delayed since the last sample was collected. If the reduced_proc_pressure data is not present in the buildstats log, then the new charts are not shown at all rather than being present but unpopulated. Note that the delta total graphs may appear "spikey", oscillating from high values to low. This behaviour is fixed in a subsequent commit. Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com> Signed-off-by: Randy MacLeod <randy.macleod@windriver.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-22wic: fix WicError messageMartin Jansa
* add missing % to print the values instead of: | INFO: Build artifacts not found, exiting. | INFO: (Please check that the build artifacts for the machine | INFO: selected in local.conf actually exist and that they | INFO: are the correct artifacts for the image (.wks file)). | | ERROR: ("The artifact that couldn't be found was %s:\n %s", 'kernel-dir', '/OE/build/deploy/images/qemux86-64') Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-17runqemu: add QB_KERNEL_CMDLINERaju Kumar Pothuraju
runqemu auto generating the KERNEL_CMDLINE values and specifying using -append option to qemu boot command which will lead to override the kernel_cmdline/bootargs which are specified in DTB when using -dtb option. Add new macro QB_KERNEL_CMDLINE to specify not to add the runqemu generated KERNEL_CMDLINE values instead use which are in the DTB if value defined as 'none'. Add provision to override bootargs using # runqemu bootparams="root=/dev/ram0" Signed-off-by: Raju Kumar Pothuraju <raju.kumar-pothuraju@xilinx.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11devtool: Fix _copy_file() TypeErrorXiaobing Luo
when devtool finish, the _copy_file() failed. -------------------------------------------- TypeError: _copy_file() got an unexpected keyword argument 'base_outdir' -------------------------------------------- Fixes: 05f2d5d2ce00 ("devtool: finish: add dry-run option") Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27wic/plugins/images/direct: Allow changes in fstab on rootfsTobias Schmidl
Allow wic to also manipulate the rootfs entry in fstab, which it currently refuses to write. Reasons one might want to do that include using systemd-growfs via --fsoptions on / With this change / is now handled exactly the same as other mountpoints, the former exception seemingly was not even documented. Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27oe-depends-dot: Handle new format for task-depends.dotRusty Howell
The .dot file created by `bitbake -g` changed formats a while ago, which broke oe-depends-dot. Also add some useful examples to the --help output. Signed-off-by: Rusty Howell <rustyhowell@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23scripts/patchreview: Make json output human parsableRichard Purdie
Sort dict keys in the json output and use tab spacing. This means when commited into git, the diffs are human readable but it is more compact filesize than space indentation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23scripts/patchreview: Add commit to stored json dataRichard Purdie
Save commit data when writing to the json file so the results can be copared/extended later. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-16wic: added fspassno parameter to partitionClaudius Heine
The `fspassno` parameter allows to overwrite the value of the last column (`fs_passno`) in the /etc/fstab of the target root file system. This allows to have periodic file system checks. Signed-off-by: Claudius Heine <ch@denx.de> Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-15image_types/runqemu-addptable2image: Fix a minor typoLeon Anavi
Fix a minor typo and replace "partion" with "partition". Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-11scripts/autobuilder-worker-prereq-tests: add additional limit testingMichael Halstead
Check that open file and user process limits are greater than or equal to what the autobuilder uses. Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-11scripts/autobuilder-worker-prereq-tests: update to use yocto 4.0Michael Halstead
Use files from the current release to avoid the depreciated -show-cursor option. Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-08wic/plugins/rootfs: Fix permissions when splitting rootfs folders across ↵Felix Moessbauer
partitions This patches makes locating the file database containing the file and folder usernames and permissions more reliable. In addition to locating it relative to the partition directory, we also try to locate it relative to the IMAGE_ROOTFS. Prior to this patch, the database was not found when using --rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file permissions and ownership. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-07scripts/git: Ensure we don't have circular referencesRichard Purdie
This is horrible but I'm running out of better ideas. We hit circular reference issues which we were trying to avoid in the core HOSTTOOLS code. When building the eSDK, there can be two copies of the script. Therefore assume git will never be in a directory called scripts. This fixes eSDK build failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-07scripts: Make git intercept globalRichard Purdie
The previous minimially invasive git intercept simply isn't enough. For example, meson used in the igt-gpu-tools recipe hardcodes the path to git in the configure step so at install time, changing PATH has no effect. There are lots of interesting things we could do to try and avoid problems but making the git intercept and dropping fakeroot privs for git global is probably the least worst solution at this point. It will add slight overhead to git calls but we don't make many so the overall impact is likely minimal. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-30scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for ↵Steve Sakoman
phantomjs and optipng Use of those tools was removed in b5c131006e3fad0a15e6cdf81f71dc1e96647028 perf-build-test/report: Drop phantomjs and html email reports support Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-27wic: Add dependencies for erofs-utilsSean Anderson
In order to build erofs filesystems, wic must have the erofs-utils package installed into its sysroot. Fixes: 30375ce97 ("Add support for erofs filesystems") Signed-off-by: Sean Anderson <sean.anderson@seco.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-22wic: do not use PARTLABEL for msdos partition tablesHenning Schild
When using "msdos" partition tables and "--label" but not "--use-uuid" one can generate images which will not find their root, because PARTLABEL does not work for "msdos". Fix that by simply not going the PARTLABEL path in case of "msdos". Fixes: 2fb247c5ecf0 ("wic: support rootdev identified by partition label") Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-22install/devshell: Introduce git intercept script due to fakeroot issuesPaul Gortmaker
In a devshell, recent versions of git will complain if the repo is owned by someone other than the current UID - consider this example: ------ bitbake -c devshell linux-yocto [...] kernel-source#git branch fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else) To add an exception for this directory, call: git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source kernel-source# ------ Of course the devshell has UID zero and the "real" UID is for "paul" in this case. And so recent git versions complain. As the whole purpose of the devshell is to invoke a shell where development can take place, having a non-functional git is clearly unacceptable. Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I suggested we probably will see other similar instances like this and should make use of PATH to intercept via devshell wrappers - conveniently we already have examples of this. Here, we copy the existing "ar" example and tune it to the needs of git to combine Richard's suggestion and mine. As such we now also can store commit logs and use send-email with our user specific settings, instead of "root", so in additon to fixing basic commands like "git branch" it should also increase general usefulness. RP: Tweaked the patch so the PATH change only applies to the devshell task and is a generic git intercept rather than devshell specific. RP: Also apply the PATH change to do_install tasks since that also runs under fakeroot and several software projects inject "git describe" output into their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing reproducibility issues from systems with different git versions. Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19update_udev_hwdb: fix multilib issue with systemdKai Kang
It duplicates udevadm in systemd recipe to make it could run update_udev_hwdb with multilib enabled. Since systemd last update, it deploys a shared library libsystemd-shared-250.so in /lib/systemd/. The library will be overwritten when multilib enabled. Then if both udev-hwdb and lib32-udev-hwdb are installed, it fails to run the multilib version postinstall intercept update_udev_hwdb: | /path/to/build/tmp-glibc/work/intel_x86_64-wrs-linux/wrlinux-image-small/1.0-r1/rootfs/usr/libexec/lib32-udevadm: error while loading shared libraries: libsystemd-shared-250.so: wrong ELF class: ELFCLASS64 `udevadm hwdb --update` just concatenates .hwdb files in dirs /etc/udev/hwdb.d and /lib/udev/hwdb.d. The output file hwdb.bin is identical with the one created by lib32-udevadm. So do NOT duplicate lib32-udevadm in systemd and eudev. And update intercept script update_udev_hwdb that re-run udevadm with same arch qemuwrapper if run ${binprefix}qemuwrapper failed. Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19recipetool: Do not use mutable default arguments in PythonStefan Herbrechtsmeier
Remove mutable default arguments in Python because they can lead to all sorts of nasty and horrible bugs. https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/ Revert `recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []` and instead check fallback_licenses before use. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-15runqemu: Do not auto detect graphics if publicvnc is specifiedScott Murray
The graphics option auto detection logic added in 7f78bb7a was not checking if the publicvnc option had been specified, meaning that it would be ignored and the auto detection result used instead. Add setting a flag variable in the argument parsing and check it along with the ones for the other graphics backend options. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-14runqemu: Allow auto detection of the correct graphics optionsRichard Purdie
Running "runqemu qemux86 kvm" when qemu is configured for sdl and/or gtk display output currently leads to a poor user experience with no cursor and corrupted fonts in the gtk case. This is due to no options being passed to qemu which leads to the loss of the font envirornment variable and the show-cursor option. If the user hasn't specified a display type, grep the output of "qemu-system-xxx --help" for the display types and pick the "best" which ensures our config is passed in. That resolves the gtk font issue and the cursor issue with both sdl and gtk. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05oe-init-build-env: add quotes around variables to prevent word splittingAbongwa Amahnui Bonalais
Used shellcheck to add quotes to the variables. This is to make sure that directories with names that have space between, such as "Desktop/projects/test repo/poky" will not be considered as 2 separate words. With this modification, running the command "source oe-init-build-env" will not give the error "bash: oe-init-build-env: No such file or directory" Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05meta: scripts - relocation script adapted to support big-endian machinesSundeep KOKKONDA
relocate_sdk.py was developed for little-endian architures and when tries to install SDK for big-endian machines errors like below will be shown. Error: struct.error: unpack requires a string argument of length 32. SDK could not be set up. Relocate script failed. Abort! Error: IOError: [Errno 22] Invalid argument. SDK could not be set up. Relocate script failed. Abort! To fix this, script is modified to support big-endian architecture. Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-31convert-variable-renames: Fix typo in descriptionSimon Kuhnle
Signed-off-by: Simon Kuhnle <simon.kuhnle@methodpark.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2022-03-18scripts/patchreview: handle Inactive-Upstream statusAlexandre Belloni
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-16perf-build-test/report: Drop phantomjs and html email reports supportRichard Purdie
phantomjs isn't reliable and we've moved to sharing the reports via a webserver. Update the scripts to more match those being used in the autobuilder helper where the html email support was removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15scripts/runqemu: Fix memory limits for qemux86-64Richard Purdie
When setting memory to 4GB, qemu is only running with 2GB for x86_64. Avoid this by removing the mem= option to the kernel and letting the qemu configuration handle it for x86 in a similar way to mips. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-11oe-pkgdata-util: Adapt to the new variable override syntaxPeter Kjellerstedt
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-09meta/scripts: Improve internal variable namingSaul Wold
Update internal variable names to improve the terms used. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-08wic: Use custom kernel path if providedBill Pittman
If the custom kernel path is provided in options, then use that path instead of the default path. Signed-off-by: Bill Pittman <bill.pittman@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04convert-variable-renames: Fix output stringSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02recipetool/devtool: Further SPDX identifier cleanupsRichard Purdie
Some of these are hard to know what to do with since the original source files for the checksum aren't present. The safe option is to use "-only" as often the main license is ambiguous and the source files themselves determine the "or-later" possibility. The "-only" option therefore is realistically what we need to use in this code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02scripts: Various typo/grammar/punctuation fixesRobert P. J. Day
Among other things, fix misspellings of: - absolute - deprecated - suitable - handle and a bunch of other things. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool: Use SPDX license identifiersPeter Kjellerstedt
There are still a couple of cases where the license may be set as, e.g., "GPL" or "GPL-2.0" since there is not enough information to decide the actual SPDX license. It is then assumed that the developer will have to correct the information. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool/create_buildsys_python: Add support for more known licensesPeter Kjellerstedt
Add all OSI approved licenses from https://pypi.org/classifiers/. Also add support for Other/Proprietary (Proprietary) and Public Domain (PD). Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25scripts/documentation-audit: Use renamed LICENSE_FLAGS_ACCEPTED variableKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-spdx-licensesScott Murray
The convert-spdx-licenses.py script needs the same file closing fix as was made to convert-variable-renames to ensure modified file contents get flushed out. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-variable-renamesScott Murray
In my test environments (Fedora 35 and Debian 10.10 on AMD 2970WX), running the convert-variable-renames.py rename script was resulting in empty files instead of updated ones. From inspection, the new files are never flushed/closed before moving them into place, which seems inherently racy. Adding an explicit close to flush the modified contents out before moving into place fixes the issue for me. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21meta: Rename LICENSE_FLAGS variableSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21icecc: Improve variables/terminologyRichard Purdie
The SYSTEM and USER seperation between variables seems odd and not necessary, drop it. Avoid the use of whitelist/blacklist and also change "packages" to "recipes" since that misuse causes confusion. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>