summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2021-11-24wic: support rootdev identified by partition labelSchmidt, Adriaan
We already support specifying the rootfs by PARTUUID. This adds general support for letting the kernel find the rootfs by PARTLABEL. Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21wic: use shutil.whichMingli Yu
Use shutil.which to find the executable instead to silence the below warning: $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic [snip] DEBUG: Executing shell function do_image_wic /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import spawn INFO: Creating image(s)... [snip] [RP: Added conversion for missed function reference] Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21scripts/lnr: removeRoss Burton
lnr is a script that implements the same behaviour as 'ln --relative --symlink', as at the time of creation --relative was only available in coreutils 8.16 onwards which was too new for the older supported distros. Now, however, everyone has a new enough coreutils, so we can remove this script. All users of lnr should be replaced with ln --relative --symbolic. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-13runqemu: support rootfs mounted roAdrian Freihofer
Optionally allow to set QB_KERNEL_ROOT to e.g. "/dev/vda ro" to mount the rootfs reay-only in Qemu. Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11scripts/lib/wic/help.py: Update Fedora Kickstart URLsJon Mason
The URLs describing Kickstart are no longer valid and do not redirect to the correct location. Update them with the correct location. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11scripts/oe-package-browser: Handle no packages being builtRichard Purdie
Give the user a proper error message if there aren't packages built, rather than a less friendly traceback. [YOCTO #14619] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11scripts/oe-package-browser: Fix after overrides changeRichard Purdie
After the overrides change, the format of pkgdata changed and this usage of configparser no longer works. This change is a bandaid to make things work but the pkgdata format isn't very similar to ini files so this may need to be reimplmented in a better way in the long run. [YOCTO #14619] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11convert-srcuri.py: use regex to check space in SRC_URIKai Kang
There may be none, one or more spaces including tab before backslash in SRC_URI. Use regex to check and update. It helps to avoid malformed uri such as recipe open-iscsi-user in meta-openstack: SRC_URI = "git://github.com/open-iscsi/open-iscsi.git;protocol=https ;branch=master \ And help to check more recipes such as concurrent-ruby in the same layer: SRC_URI = "git://github.com/ruby-concurrency/concurrent-ruby.git;protocol=https;tag=v1.1.6\ Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-10wic: Use os.rename instead of bb.utils.renameKhem Raj
This is not invoked with bitbake context as a result bb.utils is not visible when this function is called during image creation and builds fail e.g. NameError: name 'bb' is not defined Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Devendra Tewari <devendra.tewari@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-03runqemu: work without SERIAL_CONSOLES being definedJon Mason
Not all machine definitions need to have SERIAL_CONSOLES defined, but runqemu currently will fail with the following script error if not present: Traceback (most recent call last): File "/builder/meta-arm/poky/scripts/runqemu", line 1604, in main config.setup_final() File "/builder/meta-arm/poky/scripts/runqemu", line 1446, in setup_final self.setup_serial() File "/builder/meta-arm/poky/scripts/runqemu", line 1381, in setup_serial self.kernel_cmdline_script += ' console=%s' %entry.split(';')[1] IndexError: list index out of range To get around this issue, add a sanity check to runqemu to avoid the parsing of SERIAL_CONSOLES if empty. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-03scripts/convert-srcuri: Update SRC_URI conversion script to handle github ↵Richard Purdie
url changes Github are dropping support for git:// protocol fetching. Update the script to learn about corner cases found in the previous conversion and support remapping the github urls as needed too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30meta/scripts: Manual git url branch additionsRichard Purdie
Following the scripted conversion adding branches to git:// SRC_URI entries, add the remaining references, mainly in the selftests and recipetool. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30meta: Add explict branch to git SRC_URIsRichard Purdie
There is uncertainty about the default branch name in git going forward. To try and cover the different possible outcomes, add branch names to all git:// and gitsm:// SRC_URI entries. This update was made with the script added to contrib in this patch which aims to help others convert other layers. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30runqemu: unbreak non-gl displaysAlexander Kanavin
Correct two issues: 1. Looking for dri.pc is only needed when gl is enabled. 2. virtio-vga-gl works only when gl is enabled via -display, otherwise virtio-vga needs to be selected. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2021-10-23recipetool: Simplify common source files skip in guess_licenseStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-15sstate: Fixes for eSDK generation after zstd switchRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14scripts/autobuilder-worker-prereq-tests: jinja2 checkMichael Halstead
Ensure the jinja2 module is available during bringup. Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14sstate: Switch to ZStandard compressor supportHenry Kleynhans
This patch switches the compressor from Gzip to ZStandard for ssate cache files. Zstandard compression provides a significant improvement in decompression speed as well as improvement in compression speed and disk usage over the 'tgz' format in use. Furthermore, its configurable compression level offers a trade-off between time spent compressing sstate cache files and disk space used by those files. The reduced disk usage also contributes to saving network traffic for those sharing their sstate cache with others. Zstandard should therefore be a good choice when: * disk space is at a premium * network speed / resources are limited * the CI server can sstate packages can be created at high compression * less CPU on the build server should be used for sstate decompression Signed-off-by: Henry Kleynhans <hkleynhans@fb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14runqemu: add DEPLOY_DIR_IMAGE replacement in QB_OPT_APPENDJon Mason
Add the ability to replace DEPLOY_DIR_IMAGE with that path in QB_OPT_APPEND. This allows for anything present in DEPLOY_DIR_IMAGE to be passed into the qemu parameters. This is especially useful if you want to run multiple flash images (as -bios only allows for one). Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Rework crunch_license to recognize more variantsStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: npm: Use README as license fallbackStefan Herbrechtsmeier
Use the README as license fallback if a license file is missing. Use the linenumbers parameter of get_license_md5sums function to determine the license text inside the README. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUMStefan Herbrechtsmeier
The package.json files doesn't contain any licenses. The name of the license doesn't comply the license requirements of most liceneses. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Add support for linenumbers to licenses.csvStefan Herbrechtsmeier
Add support for linenumbers (begin and end lines) to licenses.csv. Add an optional linenumbers parameter to get_license_md5sums to support different use cases. Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Add logger info for missing license entriesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: ignore empty license filesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Change default paramter fallback_licenses of function ↵Stefan Herbrechtsmeier
split_pkg_licenses from None to [] Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Skip common source files in guess_licenseStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14recipetool: Move license md5sums into CSV filesStefan Herbrechtsmeier
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11devpyshell: rename to pydevshellMichael Opdenacker
For consistency with "pydevshell" which is also used. This addresses [YOCTO #14531] Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
2021-10-11runqemu: correct vga-virtio option to keep virgl enabledAlexander Kanavin
qemu 6.1 has (perhaps unintentionally) disabled virgl in -vga virtio: https://gitlab.com/qemu-project/qemu/-/issues/586 Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-01wic/bootimg-efi: Add Unified Kernel Image optionKristian Klausen
"A unified kernel image is a single EFI PE executable combining an EFI stub loader, a kernel image, an initramfs image, and the kernel command line. [...] Images of this type have the advantage that all metadata and payload that makes up the boot entry is monopolized in a single PE file that can be signed cryptographically as one for the purpose of EFI SecureBoot."[1] This commit adds a create-unified-kernel-image=true option to the bootimg-efi plugin for creating a Unified Kernel Image[1] and installing it into $BOOT/EFI/Linux/ with a .efi extension per the the Boot Loader Specification[1][2]. This is useful for implementing Secure Boot. systemd-boot is the only mainstream bootloader implementing the specification, but GRUB should be able to boot the EFI binary, this commit however doesn't implement the necessary changes to the GRUB config generation logic to boot the Unified Kernel Image. [1] https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images [2] https://systemd.io/BOOT_LOADER_SPECIFICATION/ Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-01scriptutils.py: Add check before deleting pathChandana kalluri
Add a check before deleting path when using recipetool commands to avoid the following type of errors: Traceback (most recent call last): File "<workdir>/sources/core/scripts/lib/scriptutils.py", line 218, in fetch_url shutil.rmtree(path) File "/usr/local/lib/python3.7/shutil.py", line 476, in rmtree onerror(os.lstat, path, sys.exc_info()) File "/usr/local/lib/python3.7/shutil.py", line 474, in rmtree orig_st = os.lstat(path) FileNotFoundError: [Errno 2] No such file or directory: '<workdir>/build/tmp/work/recipetool-usg7o81n/work/recipe-sysroot' ERROR: Command 'script -e -q -c "recipetool --color=always create --devtool -o /tmp/devtool5sq_op37 'file:///<SRCTREE>' -x <workdir>/build/workspace/sources/devtoolsrcxc1b9zjq -N test" /dev/null' failed Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-26wic:direct.py: ignore invalid mountpoints during fstab updateMarkus Volk
wic fstab-update creates invalid entries for partitons that are not supposed to be mounted from userspace eg u-boot partitions. The following lines were added to fstab on a rock-pi-4: /dev/mmcblk1p1 loader1 vfat defaults 0 0 /dev/mmcblk1p2 reserved1 vfat defaults 0 0 /dev/mmcblk1p3 reserved2 vfat defaults 0 0 /dev/mmcblk1p4 loader2 vfat defaults 0 0 /dev/mmcblk1p5 atf vfat defaults 0 0 /dev/mmcblk1p6 /boot vfat defaults 0 0 With this patch only valid entries should be added /dev/mmcblk1p6 /boot vfat defaults 0 0 Signed-off-by: MarkusVolk <f_l_k@t-online.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23meta: Drop useless classRichard Purdie
This class was added by me back in 2007 and has changed one for a whitespace issue since. It only has two lines and neither are particularly useful, one was replaced by the nopackages class, the other adding recursive dependencies also is now mainly problematic adding tons of unneeded dependencies. The name is hard to understand and the class doesn't have a clear purpose. Drop it. Remove the references in devtool (which may be the one reason to keep it around but in my view still not worth it). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23scripts/oe-publish-sdk: Disable git gc to avoid build errorsRichard Purdie
I've puzzled over why I see local eSDK test failures. It is due to having a messy git tree locally, git then runs git gc in the background which races against the deletion code for the publish test. Disable git gc to avoid this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-21pybootchart: Avoid divide by zeroRichard Purdie
Avoid a rare divide by zero error if there isn't data point spread. [YOCTO #14547] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17wic: keep rootfs_size as integerAlexander Kanavin
The corrected line accidentally converted it to float, which causes problems later on with python 3.10: | File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext | os.ftruncate(sparse.fileno(), rootfs_size * 1024) | TypeError: 'float' object cannot be interpreted as an integer Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17meta: correct collections vs collections.abc deprecationAlexander Kanavin
collections.abc is available since 3.3 and doesn't need special handling. Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-04wic: Add extra-space argumentKristian Klausen
This allows extra space to be added after the last partition and is especially useful when free space is needed for ex: adding partitions on first boot with ex: systemd-repart[1] and the image is tested in QEMU. [1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-03recipetool/create_buildsys_python: treat BSD as BSD-3-ClauseRoss Burton
The PyPI license classifiers include "OSI Approved: BSD", which does not specify which of the many variations of BSD license it actually means. The generic "BSD" license in the oe-core set is actually BSD-3-Clause. >From a random sample of ten PyPI modules that use "BSD", they are all BSD-3-Clause. As we expect the recipe maintainer to verify the license anyway, and this matches the previous license text, explicitly set the license to BSD-3-Clause. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-27buildhistory: Add output file listing package informationAndres Beltran
Currently, buildhistory does not produce a single file combining relevant information of installed packages. Produce an output file "installed-package-info.txt" listing a package's runtime name, buildtime name, its recipe, version, and size to avoid having to look up each package externally. Leave the existing package list files as-is for backwards compatibility. In order to support this efficiently, extend oe-pkgdata-util to accept multiple keys for its read-value argument. Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-26buildhistory-collect-srcrevs: Adapt to the new variable override syntaxPeter Kjellerstedt
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-23qemu: add a hint on how to enable CPU render nodes when a suitable GPU is absentAlexander Kanavin
This is particularly useful for llvm-accelerated GL rendering from qemu guest to an offscreen buffer (accessible over vnc or spice) using llvmpipe on the host, rather than using unaccelerated swrast renderer in the guest. This is the best that can be done in the absence of a host GPU with render node support (such as old Matrox cards common in servers, or fully virtualized cloud environments with no GPU at all). Note: even though NVidia blob drivers do support render nodes, they do not support gbm (yet?), and so rendering will fall back to llvmpipe as well even when the system has a 3000 euro NVidia GPU. Cue Linus picture. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-20wic: Add --no-fstab-update part optionDaniel Gomez
When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we might want to keep the stock fstab for that image. In such a case, use this option to not update the fstab and use the stock one instead. This option allows you to specify which partitions get the fstab updated and which get the stock fstab. The option matches the argument you can pass to wic itself where the same action is performed but for all the partitions. Example: part /export --source rootfs --rootfs-dir=hockeycam-image --fstype=ext4 --label export --align 1024 --no-fstab-update part / --source rootfs --fstype=ext4 --label rootfs --align 1024 Signed-off-by: Daniel Gomez <daniel@qtec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-17package/scripts: Fix FILES_INFO handlingRichard Purdie
There is a long standing bug where FILES_INFO isn't written into pkgdata with a package suffix. This means if the files are read into the datastore as intended, the last one "wins". Fix this to work as intended. Most of the call sites using the data need to be updated to handle this and the overrides change correctly. Also fix some other problematic references noticed along the way. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-17scripts/lib/wic/help/py: "Redhat" -> "Red Hat"Robert P. J. Day
Company name is properly spelled as "Red Hat". Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12wic: don't forcibly pass -T defaultRoss Burton
This reverts part of oe-core eecbe62555, which was a previous attempt to solve the Y2038 problem. This is now solved centrally in e2fsprogs, so doesn't need to be dealt with in wic. We don't revert the commit entirely, to retain the warning if a filesystem has small inodes. Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12convert-overrides.py: also convert comments without a leading whitespaceChen Qi
Currently lines like below are converted. e.g. # IMAGE_INSTALL_append = " A" But lines without a leading whitespace are not converted. e.g. #IMAGE_INSTALL_append = " A" We should be converting both. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-05convert-overrides.py: allow dots before override in vars_re and shortvars_reMartin Jansa
e.g. VIRTUAL-RUNTIME_com.webos.service.flowmanager_armv4 weren't replaced with VIRTUAL-RUNTIME_com.webos.service.flowmanager:armv4 or when package name contains a dot like in: RDEPENDS:gstreamer1.0-meta-base:remove Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04image/image_types: Convert CONVERSION_CMD/COMPRESS_CMD to new override syntaxRichard Purdie
For consistency, use override syntax for these variables as well since it is more consistent with the rest of the image code. We may be able to use these as proper overrides in due course. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-01 12:01:10 +0000