summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2022-06-15wic: Use ROOTFS_DIR to locate pseudo_dirrbt/wicRobert Yang
Fixed when wks is: part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4096 --fixed-size 256M part / --source rootfs --exclude-path=var/ --ondisk mmcblk0 --fstype=ext4 --label root --align 4096 part /var --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/var --ondrive mmcblk0 --fstype=ext4 --label var --align 4096 --fixed-size 1024M --fsoptions=ro,relatime,sync $ bitbake <image> $ wic create <wks> -e <image> WARNING: /path/to/rootfs/var/../pseudo folder does not exist. Usernames and permissions will be invalid The --rootfs-dir can be anywhere, so the ../pseudo may not exist, use ROOTFS_DIR to fix the problem. From: Rath Anil Kumar <AnilKumar.Rath@windriver.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2022-06-11devtool: Fix _copy_file() TypeErrorXiaobing Luo
when devtool finish, the _copy_file() failed. -------------------------------------------- TypeError: _copy_file() got an unexpected keyword argument 'base_outdir' -------------------------------------------- Fixes: 05f2d5d2ce00 ("devtool: finish: add dry-run option") Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27wic/plugins/images/direct: Allow changes in fstab on rootfsTobias Schmidl
Allow wic to also manipulate the rootfs entry in fstab, which it currently refuses to write. Reasons one might want to do that include using systemd-growfs via --fsoptions on / With this change / is now handled exactly the same as other mountpoints, the former exception seemingly was not even documented. Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27oe-depends-dot: Handle new format for task-depends.dotRusty Howell
The .dot file created by `bitbake -g` changed formats a while ago, which broke oe-depends-dot. Also add some useful examples to the --help output. Signed-off-by: Rusty Howell <rustyhowell@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23scripts/patchreview: Make json output human parsableRichard Purdie
Sort dict keys in the json output and use tab spacing. This means when commited into git, the diffs are human readable but it is more compact filesize than space indentation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23scripts/patchreview: Add commit to stored json dataRichard Purdie
Save commit data when writing to the json file so the results can be copared/extended later. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-16wic: added fspassno parameter to partitionClaudius Heine
The `fspassno` parameter allows to overwrite the value of the last column (`fs_passno`) in the /etc/fstab of the target root file system. This allows to have periodic file system checks. Signed-off-by: Claudius Heine <ch@denx.de> Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-15image_types/runqemu-addptable2image: Fix a minor typoLeon Anavi
Fix a minor typo and replace "partion" with "partition". Signed-off-by: Leon Anavi <leon.anavi@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-11scripts/autobuilder-worker-prereq-tests: add additional limit testingMichael Halstead
Check that open file and user process limits are greater than or equal to what the autobuilder uses. Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-11scripts/autobuilder-worker-prereq-tests: update to use yocto 4.0Michael Halstead
Use files from the current release to avoid the depreciated -show-cursor option. Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-08wic/plugins/rootfs: Fix permissions when splitting rootfs folders across ↵Felix Moessbauer
partitions This patches makes locating the file database containing the file and folder usernames and permissions more reliable. In addition to locating it relative to the partition directory, we also try to locate it relative to the IMAGE_ROOTFS. Prior to this patch, the database was not found when using --rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file permissions and ownership. Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-07scripts/git: Ensure we don't have circular referencesRichard Purdie
This is horrible but I'm running out of better ideas. We hit circular reference issues which we were trying to avoid in the core HOSTTOOLS code. When building the eSDK, there can be two copies of the script. Therefore assume git will never be in a directory called scripts. This fixes eSDK build failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-07scripts: Make git intercept globalRichard Purdie
The previous minimially invasive git intercept simply isn't enough. For example, meson used in the igt-gpu-tools recipe hardcodes the path to git in the configure step so at install time, changing PATH has no effect. There are lots of interesting things we could do to try and avoid problems but making the git intercept and dropping fakeroot privs for git global is probably the least worst solution at this point. It will add slight overhead to git calls but we don't make many so the overall impact is likely minimal. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-30scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for ↵Steve Sakoman
phantomjs and optipng Use of those tools was removed in b5c131006e3fad0a15e6cdf81f71dc1e96647028 perf-build-test/report: Drop phantomjs and html email reports support Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-27wic: Add dependencies for erofs-utilsSean Anderson
In order to build erofs filesystems, wic must have the erofs-utils package installed into its sysroot. Fixes: 30375ce97 ("Add support for erofs filesystems") Signed-off-by: Sean Anderson <sean.anderson@seco.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-22wic: do not use PARTLABEL for msdos partition tablesHenning Schild
When using "msdos" partition tables and "--label" but not "--use-uuid" one can generate images which will not find their root, because PARTLABEL does not work for "msdos". Fix that by simply not going the PARTLABEL path in case of "msdos". Fixes: 2fb247c5ecf0 ("wic: support rootdev identified by partition label") Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-22install/devshell: Introduce git intercept script due to fakeroot issuesPaul Gortmaker
In a devshell, recent versions of git will complain if the repo is owned by someone other than the current UID - consider this example: ------ bitbake -c devshell linux-yocto [...] kernel-source#git branch fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else) To add an exception for this directory, call: git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source kernel-source# ------ Of course the devshell has UID zero and the "real" UID is for "paul" in this case. And so recent git versions complain. As the whole purpose of the devshell is to invoke a shell where development can take place, having a non-functional git is clearly unacceptable. Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I suggested we probably will see other similar instances like this and should make use of PATH to intercept via devshell wrappers - conveniently we already have examples of this. Here, we copy the existing "ar" example and tune it to the needs of git to combine Richard's suggestion and mine. As such we now also can store commit logs and use send-email with our user specific settings, instead of "root", so in additon to fixing basic commands like "git branch" it should also increase general usefulness. RP: Tweaked the patch so the PATH change only applies to the devshell task and is a generic git intercept rather than devshell specific. RP: Also apply the PATH change to do_install tasks since that also runs under fakeroot and several software projects inject "git describe" output into their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing reproducibility issues from systems with different git versions. Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19update_udev_hwdb: fix multilib issue with systemdKai Kang
It duplicates udevadm in systemd recipe to make it could run update_udev_hwdb with multilib enabled. Since systemd last update, it deploys a shared library libsystemd-shared-250.so in /lib/systemd/. The library will be overwritten when multilib enabled. Then if both udev-hwdb and lib32-udev-hwdb are installed, it fails to run the multilib version postinstall intercept update_udev_hwdb: | /path/to/build/tmp-glibc/work/intel_x86_64-wrs-linux/wrlinux-image-small/1.0-r1/rootfs/usr/libexec/lib32-udevadm: error while loading shared libraries: libsystemd-shared-250.so: wrong ELF class: ELFCLASS64 `udevadm hwdb --update` just concatenates .hwdb files in dirs /etc/udev/hwdb.d and /lib/udev/hwdb.d. The output file hwdb.bin is identical with the one created by lib32-udevadm. So do NOT duplicate lib32-udevadm in systemd and eudev. And update intercept script update_udev_hwdb that re-run udevadm with same arch qemuwrapper if run ${binprefix}qemuwrapper failed. Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19recipetool: Do not use mutable default arguments in PythonStefan Herbrechtsmeier
Remove mutable default arguments in Python because they can lead to all sorts of nasty and horrible bugs. https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/ Revert `recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []` and instead check fallback_licenses before use. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-15runqemu: Do not auto detect graphics if publicvnc is specifiedScott Murray
The graphics option auto detection logic added in 7f78bb7a was not checking if the publicvnc option had been specified, meaning that it would be ignored and the auto detection result used instead. Add setting a flag variable in the argument parsing and check it along with the ones for the other graphics backend options. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-14runqemu: Allow auto detection of the correct graphics optionsRichard Purdie
Running "runqemu qemux86 kvm" when qemu is configured for sdl and/or gtk display output currently leads to a poor user experience with no cursor and corrupted fonts in the gtk case. This is due to no options being passed to qemu which leads to the loss of the font envirornment variable and the show-cursor option. If the user hasn't specified a display type, grep the output of "qemu-system-xxx --help" for the display types and pick the "best" which ensures our config is passed in. That resolves the gtk font issue and the cursor issue with both sdl and gtk. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05oe-init-build-env: add quotes around variables to prevent word splittingAbongwa Amahnui Bonalais
Used shellcheck to add quotes to the variables. This is to make sure that directories with names that have space between, such as "Desktop/projects/test repo/poky" will not be considered as 2 separate words. With this modification, running the command "source oe-init-build-env" will not give the error "bash: oe-init-build-env: No such file or directory" Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05meta: scripts - relocation script adapted to support big-endian machinesSundeep KOKKONDA
relocate_sdk.py was developed for little-endian architures and when tries to install SDK for big-endian machines errors like below will be shown. Error: struct.error: unpack requires a string argument of length 32. SDK could not be set up. Relocate script failed. Abort! Error: IOError: [Errno 22] Invalid argument. SDK could not be set up. Relocate script failed. Abort! To fix this, script is modified to support big-endian architecture. Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-31convert-variable-renames: Fix typo in descriptionSimon Kuhnle
Signed-off-by: Simon Kuhnle <simon.kuhnle@methodpark.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2022-03-18scripts/patchreview: handle Inactive-Upstream statusAlexandre Belloni
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-16perf-build-test/report: Drop phantomjs and html email reports supportRichard Purdie
phantomjs isn't reliable and we've moved to sharing the reports via a webserver. Update the scripts to more match those being used in the autobuilder helper where the html email support was removed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15scripts/runqemu: Fix memory limits for qemux86-64Richard Purdie
When setting memory to 4GB, qemu is only running with 2GB for x86_64. Avoid this by removing the mem= option to the kernel and letting the qemu configuration handle it for x86 in a similar way to mips. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-11oe-pkgdata-util: Adapt to the new variable override syntaxPeter Kjellerstedt
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-09meta/scripts: Improve internal variable namingSaul Wold
Update internal variable names to improve the terms used. Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-08wic: Use custom kernel path if providedBill Pittman
If the custom kernel path is provided in options, then use that path instead of the default path. Signed-off-by: Bill Pittman <bill.pittman@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04convert-variable-renames: Fix output stringSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02recipetool/devtool: Further SPDX identifier cleanupsRichard Purdie
Some of these are hard to know what to do with since the original source files for the checksum aren't present. The safe option is to use "-only" as often the main license is ambiguous and the source files themselves determine the "or-later" possibility. The "-only" option therefore is realistically what we need to use in this code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02scripts: Various typo/grammar/punctuation fixesRobert P. J. Day
Among other things, fix misspellings of: - absolute - deprecated - suitable - handle and a bunch of other things. Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool: Use SPDX license identifiersPeter Kjellerstedt
There are still a couple of cases where the license may be set as, e.g., "GPL" or "GPL-2.0" since there is not enough information to decide the actual SPDX license. It is then assumed that the developer will have to correct the information. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01recipetool/create_buildsys_python: Add support for more known licensesPeter Kjellerstedt
Add all OSI approved licenses from https://pypi.org/classifiers/. Also add support for Other/Proprietary (Proprietary) and Public Domain (PD). Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25scripts/documentation-audit: Use renamed LICENSE_FLAGS_ACCEPTED variableKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-spdx-licensesScott Murray
The convert-spdx-licenses.py script needs the same file closing fix as was made to convert-variable-renames to ensure modified file contents get flushed out. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22scripts: fix file writing in convert-variable-renamesScott Murray
In my test environments (Fedora 35 and Debian 10.10 on AMD 2970WX), running the convert-variable-renames.py rename script was resulting in empty files instead of updated ones. From inspection, the new files are never flushed/closed before moving them into place, which seems inherently racy. Adding an explicit close to flush the modified contents out before moving into place fixes the issue for me. Signed-off-by: Scott Murray <scott.murray@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21meta: Rename LICENSE_FLAGS variableSaul Wold
Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21icecc: Improve variables/terminologyRichard Purdie
The SYSTEM and USER seperation between variables seems odd and not necessary, drop it. Avoid the use of whitelist/blacklist and also change "packages" to "recipes" since that misuse causes confusion. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21scripts: Add convert-variable-renames script for inclusive language variable ↵Saul Wold
renaming This script searches for a list of variable that have been renamed and converts them to their more descriptive names. It also searches for a list of variables that have been removed or deprecated and prints a message. It will print a message to inform the user that there are terms that need to be updated in their files. Many of these changes are context sensitive and may not be modified as they might be existing calls to other libraries. This message is informational only. I have tested this on poky and meta-openembedded so far. (From OE-Core rev: 50fe7ba8dba05a9681c9095506f798796cfc2750) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21meta/scripts: Change BB_ENV_EXTRA_WHITE -> BB_ENV_PASSTHROUGH_ADDITIONSRichard Purdie
After the change to bitbake, update the references in OE-Core to match the updates. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21yocto-check-layer: add ability to perform tests from a global bbclassDenys Dmytriyenko
This is useful when needing to test layer's recipes, where this special bbclass can define a global python function that gets called on each recipe parsing during "bitbake -S none world" signature dump and be able to fail layer's check accordingly. First test being added is to detect recipes skipping "installed-vs-shipped" QA check. As "installed-vs-shipped" is a packaging QA check, it happens very late in the build process and failing it could mean some potential issues with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and resulting package is used in an SDK. In OE-Core failing this QA check leads to an error, but other layers can suppress it or change it to a warning. Detecting weird packaging problems with SDKs is quite difficult and time consuming. Also, waiting for the actual "installed-vs-shipped" packaging QA check to fail means that all recipes in the layer under test have to run through all standard tasks in the build chain, equivalent to a multi-hour world-build. Hence yocto-check-layer takes a shortcut and only detects a mere attempt at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during initial parsing when dumping the signature information for the layer. Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21devtool: explicitly set main or master branches in upgrades when availableAlexander Kanavin
In particular this resolves devtool's inability to pick a branch when the same tagged commit is avaiable in main and in a release maintenance branch. Thanks to Peter Kjellerstedt for the suggestion. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-20scripts: Add a conversion script to use SPDX license namesRichard Purdie
We're standardising on SPDX license names so it is overdue to change the ones in the LICENSE fileds to the SPDX values. Add a conversion script which makes this straightforward on the most part. Ultimately this allows the core code to be more efficient and not need to consult the mappings. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16wic: rawcopy: Add support for packed imagesStefan Herbrechtsmeier
Add support for packed images to wic rawcopy handler do minimize disk usage in deploy directory and reuse of packed images between wic and swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz archives. Example: part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack" Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16wic: partition: Support valueless keys in sourceparamsStefan Herbrechtsmeier
Accept valueless keys in sourceparams without equals sign (=) to match the comment and support Boolean entries. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-12scriptutils: Fix handling of srcuri urlsRichard Purdie
Bitbake is now requiring there be a call to get_srcrev() when urls contain floating revisions. This breaks some recipetool workflows since it is running these queries to determine those revisions. Add a PV entry which triggers that call to avoid the errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07runqemu: preload uninative libraries when host gl drivers are in useAlexander Kanavin
Some of the host distributions build the drivers in a way (RPATH/RUNPATH) that tricks uninative loader into loading pieces of the host libc, if the same pieces haven't been previously loaded by native binaries. Mixing the two libc versions leads to failures. This change ensures that the correct (uninative) versions are always in use. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07qemu: replace a gtk wrapper with directly setting environment from runqemuAlexander Kanavin
The wrapper is executed by host bash, and host bash refuses to run when pseudo libc is preloaded via LD_PRELOAD (which is the case when gl is enabled). Only the fontconfig setting is carried over as local testing showed that only that is necessary for the gui to look ok nowadays; adjust further if necessary. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>