aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2017-01-19oeqa.utils.metadata: allow storing any bitbake config variablesMarkus Lehtonen
Make it possible to store any bitbake config variables in the metadata. Config values will be stored under a new config element in the xml report: <config> <variable name="MACHINE">qemux86</variable> </config> The value of MACHINE is moved there instead of having a dedicated <machine> element. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: rename 'revision' to 'commit'Markus Lehtonen
Revision is a bit vague and could point to a tag, for example. Git commit objects are unambiguous and persistent so be explicit that the element should contain git commit hash. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: drop 'unknown' git data elementsMarkus Lehtonen
It's better just to not have the xml elements than to have elements with faux data. One could have git branch named 'unknown', for example. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: re-organise distro informationMarkus Lehtonen
Use the same format, based on /etc/os-release, as for host distro information. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19runqemu-export-rootfs: fix inconsistent var namesRobert Yang
Fixed: $ runqemu nfs qemux86-64 [snip] On your target please remember to add the following options for NFS nfsroot=IP_ADDRESS:/path/to/nfsroot,nfsvers=3,port=,mountprog=,nfsprog=,udp,mountport= [snip] Note that the values are null, this is because their var names are inconsistent. [YOCTO #10519] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19scripts/runqemu: fix checking for <file>.cpio.gzRobert Yang
When "runqemu /path/to/<file>.cpio.gz", it used the last suffix "gz" as the fstype which was wrong. Check filename against self.fstypes firstly can fix the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19wic: partitionedfs: account for non-table partitions when checking if ↵Maciej Borzecki
logical parititon is needed Commit 8c1c43b7901a9fcd8b279eb4250b08157ad345b7 `wic: Create a logical partition only when it is really mandatory` did not account for partitions that are not present in partition table. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16scripts: python3 fixes and new tool ksumTom Zanussi
'ksum.py' generates a combined summary of vmlinux and module sizes for a built kernel, as a quick tool for comparing the overall effects of systemic tinification changes. Execute from the base directory of the kernel build you want to summarize. Setting the 'verbose' flag will display the sizes for each file included in the summary. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16scripts/oe-selftest: fix typoChen Qi
Change 'agains' to 'against'. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16runqemu: Allow the user to specity no kernel or rootFSAlistair Francis
In some cirsumstances the user doesn't want to supply a kernel, rootFS or DTB to QEMU. This will occur more now that QEMU supports loading images using a '-device loader' method. Allow users to specify 'none' for QB_DEFAULT_FSTYPE or QB_DEFAULT_KERNEL to avoid supplying these options to QEMU. Signed-off-by: Alistair Francis <alistair.francis@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic: _exec_cmd: produce error if exit code is not 0Ed Bartosh
Current code doesn't always show error output of the external command and even ignores non-zero exit code. Moved checking of exit code value to the lowest level possible: to _exec_cmd. This should make wic to always check exit code of the external command and issue an error if it's not 0. [YOCTO #10816] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16direct.py: fix getting image nameEd Bartosh
part.rootfs_dir was used as an image name in the code. However, when multi-rootfs feature is used this attribute points to the name of the rootfs, e.g. if --rootfs command line is rootfs1=core-image-minimal partf.rootfs_dir is 'rootfs1'. The code also fails when image name is not provided in wic commandline. For example, when wic is called with --rootfs-dir=<path> part.rootfs_dir will contain path and wic will crash trying to call bitbake -e <path> to get value of ROOTFS_SIZE variable. Fixed the code by getting image name properly and checking if it's not a path. [YOCTO #10815] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16canned-wks: remove mpc8315e-rdb.wksEd Bartosh
This file has been moved to meta-yocto-bsp/wic/ Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic/isoimage-isohybrid: remove do_stage_partition()Ioan-Adrian Ratiu
The purpouse of this function was to check dependencies for building a hybrid iso and build them using bitbake if not found. Calling bitbake in this context means this wic plugin itself cannot be instrumented inside bitbake recipes which is undesirable, the benefits of this are clear: there is no need to maintain outside scripts to generate an iso using wic and the isohybrid building logic can be further abstracted away into an isohybrid.bbclass in the future which can be easily inherited or something similar. So remove the function and add all dependencies to NATIVE_RECIPES so that wic can print useful errors when they're not built. To automate building the isohybrid image dependencies, add the following somewhere in your image build inheritence hierarcy (or maybe create a bbclass in the future to do these sort of things automatically): DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \ parted-native dosfstools-native mtools-native grub-efi-native" Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16python-3.5-manifest: Add http module to the netclient packageDerek Straka
Adding http module from Python's standard library. This allow use of the http module without installing all python-misc modules. Signed-off-by: Derek Straka <derek@asterius.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic: fix parsing of 'bitbake -e' outputEd Bartosh
Current parsing code can wrongly interpret arbitrary lines that are of 'key=value' format as legitimate bitbake variables. Implemented more strict parsing of key=value pairs using regular expressions. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16gummiboot: Remove old gummiboot recipe, related class and wks fileAlejandro Hernandez
Since the gummiboot project is no longer being maintained and we are using systemd-boot as a replacement instead, we can now clean up all remaining gummiboot files. [YOCTO #10332] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09oe-selftest: fix behaviour if oe-selftest.log is a dangling symlinkPaul Eggleton
If you delete the log file that the oe-selftest.log symlink points to but not the symlink itself, because we were using os.path.exists() here the code assumed that the symlink didn't exist when in fact it still did. Use os.path.lexists() instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09gummiboot: Remove/change gummiboot references with systemd-bootAlejandro Hernandez
After systemd-boot was introduced, its been tested for a while with no major issues being found until now, this patch completely replaces all gummiboot instances with systemd-boot ones, taking the next step into cleaning up systemd-boot/gummiboot. [YOCTO #10332] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05oe-selftest: Improve BUILDDIR environment handlingRichard Purdie
Its possible something (like bitbake/tinfoil2) may mess around with the environment and using the enviroment as a global variable store isn't particularly nice anyway. This patch changes the BUILDDIR usages so that the environment isn't used as a global store and a global variable is used instead. Whilst that is still not perfect, it does avoid the current double and triple backtraces we're seeing where tinfoil2/bitbake has trampled the enviroment leading to failures of failures making debugging even harder. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05runqemu: let command line parameters override defaultsPatrick Ohly
It may be necessary to override the parameters gathered for the qemu invocation. For example, the qemux86 machine configuration sets "-vga vmware", but when using OVMF as BIOS, only "-vga std" is supported. By putting the parameters derived from custom runqemu parameters like "qemuparams" after the parameters derived from the machine configuration the user gets the possibility to override those. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: add kickstart file for MPC8315Ed Bartosh
Added kickstart file to produce partitioned image for MPC8315 reference hardware. [YOCTO #8719] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: add --fixed-size wks optionMaciej Borzecki
Added new option --fixed-size to wks. The option can be used to indicate the exact size of a partition. The option cannot be added together with --size, in which case an error will be raised. Other options that influence automatic partition size (--extra-space, --overhead-factor), if specifiec along with --fixed-size, will raise an error. If it partition data is larger than the amount of space specified with --fixed-size option wic will raise an error. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22devtool: modify: fix usage on the kernelPaul Eggleton
When using devtool modify on the kernel, we have to do a bit of a dance with tinfoil instances because we only find out that we're working on a kernel recipe after tinfoil is initialised, but then we need to build kern-tools-native which we're doing just by running bitbake directly. With the tinfoil2 changes, a datastore for the recipe that we were keeping around across the opening and closing of tinfoil is no longer able to be used. Re-parse the recipe to avoid this problem. (In future this whole thing will be able to be done in the same tinfoil instance thanks to tinfoil2, but that refactoring is yet to be done.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: look for wks files in <layer>/wicEd Bartosh
Currently wic looks for wks files in <layer dir>/scripts/lib/wic/canned-wks/ directories. This path is too nested and doesn't look consistent with the naming scheme of layer directories. Added <layer>/wic directory to the list of paths to look for wks files. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-19oe-selftest: import git module only when neededEd Bartosh
git module is not included into standard Python library and therefore causes import errors on the systems where PythonGit is not installed. As git module only used in the code implementing --repository functionality it's better to import git only in the scope that requires it. [YOCTO #10821] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17wic: obey the rootfs size from the metadataChristopher Larson
When no --size is specified for the rootfs in the .wks, we want to obey the rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result in image construction failure, if the size determined by du was insufficient to hold the files without the aforementioned extra space. This fallback from --size to ROOTFS_SIZE was already implemented when --rootfs-dir is specified in the .wks, but it did not occur otherwise, neither when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used. This made a certain amount of sense, as this fallback logic happened at such a level that it wasn't able to identify which partitions were rootfs partitions otherwise. Rather than doing it at that level, we can do it in prepare_rootfs(), which is run by the rootfs source plugins. Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now both be applied when --size isn't specified in the .wks. A warning is added about this, though a user won't see it unless wic fails or they examine the do_image_wic log. Fixes [YOCTO #10815] Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16scripts: remove True option to getVarFlag callsJoshua Lock
getVarFlag() now defaults to expanding by default, thus remove the True option from getVarFlag() calls with a regex search and replace. Search made with the following regex: getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16scripts: remove True option to getVar callsJoshua Lock
getVar() now defaults to expanding by default, thus remove the True option from getVar() calls with a regex search and replace. Search made with the following regex: getVar ?\(( ?[^,()]*), True\) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-16scripts/runqemu: Allow to use qemu from host.Mariano Lopez
This will add support to use qemu from the running host, with this is possible to put qemu-native in ASSUME_PROVIDED variable. By default it will try to get qemu from the build sysroot, and only if it fails will try to use the host's qemu. Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14devtool: prevent BBHandledException from showing tracebackPaul Eggleton
If we don't catch this then attempting to run devtool in non-memres mode when bitbake is already running will produce a traceback instead of just an error message. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14devtool: extract: disable basehash mismatch errorsPaul Eggleton
Using the setVariable commands here followed by buildFile will result in "basehash mismatch" errors, and that's expected since we are deviating *at runtime* from what was previously seen by changing these variable values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14devtool: fix extraction of source to work in memres modePaul Eggleton
Extracting the source for a recipe (as used by devtool's extract, modify and upgrade subcommands) requires us to run do_fetch, do_unpack, do_patch and any tasks that the recipe has inserted inbetween, and do so with a modified datastore primarily so that we can redirect WORKDIR and STAMPS_DIR in order to have the files written out to a place of our choosing and avoid stamping the tasks as having executed in a real build context respectively. However, this all gets much more difficult when in memres mode since we can't call internal functions such as bb.build.exec_func() directly - instead we need to execute the tasks on the server. To do this we use the buildFile command which already exists for the purpose of supporting bitbake -b, and setVariable commands to set up the appropriate datastore. (I did look at passing the modified datastore to the buildFile command instead of using setVar() on the main datastore, however its use of databuilder makes that very difficult, and we'd also need a different method of getting the changes in the datastore over to the worker as well.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14recipetool: add OE lib pathPaul Eggleton
The autotools code imports oe.package; we weren't experiencing a problem with this probably due to OE itself adding that path previously. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14classes/patch: move in logic to commit for additional tasksPaul Eggleton
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for additional tasks between do_unpack and do_patch, make a git commit. This logic was previously implemented in devtool itself, but it makes more sense for it to be implemented in the patch class since that's where the rest of the logic is for this (or in lib/oe/patch.py). It also makes it possible for this to work with tinfoil2. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14devtool / recipetool: use tinfoil parsing APIPaul Eggleton
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of the recipeutils equivalents, and replace any local duplicate implementations. This not only tidies up the code but also allows these calls to work in memres mode. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14devtool: package: don't try to initialise tinfoil twicePaul Eggleton
setup_tinfoil() already calls prepare(), we don't need to call it again ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of bitbake should be run against a build directory". Calling prepare() twice should probably still be allowed, so that ought to be fixed separately, but in the mean time this code is still wrong so fix it here. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-14oe-pkgdata-util: Make read-value handle override variablesOla x Nilsson
Some variables in pkgdata files have a package-name override. When the bare variable can not be found, try with the override-variant. PKGSIZE is one such variable, and already had special code to handle this. Test included. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-buildenv-internal: show usage outputEd Bartosh
Show usage text if script is not sourced. Tested in bash, zsh and dash. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-find-native-sysroot: create usage outputEd Bartosh
Created usage output for oe-find-native-sysroot script. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-git-proxy: create usage outputEd Bartosh
Created usage output for oe-git-proxy script. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oepydevshell-internal.py: standardize usage outputEd Bartosh
Made usage output of oepydevshell-internal.py to look similar to the output of other oe scripts. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-setup-builddir: create usage outputEd Bartosh
Created usage output for oe-setup-builddir script. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-setup-rpmrepo: standardize usage outputEd Bartosh
Made usage output of oe-setup-rpmrepo to look similar to the output of other oe scripts. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-trim-schemas: create usage outputEd Bartosh
Created usage output for oe-trim-schemas script. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13oe-run-native: standardize usage outputEd Bartosh
Made usage output of oe-run-native to look similar to the output of other oe scripts. [YOCTO #10751] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13recipetool: Load plugins in a well defined orderOla x Nilsson
To allow recipetool plugins in one layer to shadow another in a well defined way, first search BBPATH/lib/recipetool directories and then scripts/lib/recipetool and load only the first found. The previous search and load loop would load all found plugins with the ones found later replacing any found before. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13devtool: Load plugins in a well defined orderOla x Nilsson
To allow devtool plugins in one layer to shadow another in a well defined way, first search BBPATH/lib/devtool directories and then scripts/lib/devool and load only the first found. The previous search and load loop would load all found plugins with the ones found later replacing any found before. Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-13wic: rename command line option -p -> -sEd Bartosh
Short variant of wic command line option --skip-build-check is incorretly named -p. It's named -s in wic help and Yocto documentation. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>