aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2017-01-28scripts/buildstats-diff: simplify timestamp handlingMarkus Lehtonen
Simply use floats instead of datetime and timedelta objects for handling timestamps. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26runqemu: more verbose error message about missing qemuboot.confPatrick Ohly
When invoking "runqemu" with a mistyped image or architecture name, the resulting error message is about the missing qemuboot.conf, without any indication about the root cause: $ runqemu core-image-mimimal ext4 intel-corei7-64 runqemu - INFO - Assuming MACHINE = intel-corei7-64 runqemu - INFO - Running MACHINE=intel-corei7-64 bitbake -e... runqemu - INFO - MACHINE: intel-corei7-64 runqemu - INFO - DEPLOY_DIR_IMAGE: /fast/build/refkit/intel-corei7-64/tmp-glibc/deploy/images/intel-corei7-64 Traceback (most recent call last): File "/work/openembedded-core/scripts/runqemu", line 1095, in <module> ret = main() File "/work/openembedded-core/scripts/runqemu", line 1082, in main config.read_qemuboot() File "/work/openembedded-core/scripts/runqemu", line 643, in read_qemuboot raise Exception("Failed to find <image>.qemuboot.conf!") Exception: Failed to find <image>.qemuboot.conf! Including the name of the actual file the scripts expects to find plus adding some hints what to check for might help. The error now is: $ runqemu core-image-mimimal ext4 intel-corei7-64 ... Exception: Failed to find <image>.qemuboot.conf = .../tmp-glibc/deploy/images/intel-corei7-64/core-image-mimimal-intel-corei7-64.qemuboot.conf (wrong image name or BSP does not support running under qemu?). The comment about the BSP is included because that would be the real reason why the file might be missing. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26wipe-sysroot: removeRoss Burton
This script was useful to delete the sysroot so that future builds happen from a clean sysroot. Now that recipes have their own sysroot this script doesn't need to exist anymore, and if in fact if used will break your TMPDIR. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23selftest: automatically add the meta-selftest layerBenjamin Esquivel
adding the meta-selftest layer is a small bump you always encounter when you want to run selftest. Adding an automatic procedure to include the layer if it is not yet present. Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23create-pull-request: fix for OE cgit URL changePaul Eggleton
The /cgit.cgi/ part is no longer valid in cgit URLs on git.openembedded.org as of recent infrastructure changes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23devtool: check locale and refuse to start if it isn't UTF-8Paul Eggleton
We need to ensure the locale is UTF-8 or otherwise strange errors will occur later on during execution - the same reason we check this in BitBake itself. Unfortunately this check has to be before command line parsing and therefore showing the help text in response to --help, since that relies upon parsing bitbake's configuration (as we need to load plugins in other layers). Fixes [YOCTO #10908]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23runqemu: fixes for slirp, network device and hostfwdRobert Yang
Fixed: - Add QB_NETWORK_DEVICE to set network device, it will be used by both slirp and tap. - Set QB_NETWORK_DEVICE to "-device virtio-net-pci" in qemuboot.bbclass but runqemu will default to "-device e1000" when QB_NETWORK_DEVICE is not set, this is because oe-core's qemu targets support virtio-net-pci, but the one outside of oe-core may not, "-device e1000" is more common. - Set hostfwd by default: 2222 -> 22, 2323 -> 23, and it will choose a usable port when the one like 222 is being used. This can avoid conflicts when multilib slirp qemus are running. We can forward more ports by default if needed, and bsp.conf can custom it. - Use different mac sections for slirp and tap to fix conflicts when running both of them on the same host. [YOCTO #7887] CC: Nathan Rossi <nathan@nathanrossi.com> CC: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2017-01-23runqemu: support multiple qemus running when nfsRobert Yang
Fixed: * In build1: $ runqemu nfs qemux86-64 In build2: $ runqemu nfs qemux86-64 It would fail before since the port numerbs and conf files are conflicted, now make runqemu-export-rootfs work together with runqemu to fix the problem. * And we don't need export PSEUDO_LOCALSTATEDIR in runqemu, the runqemu-export-rootfs can handle it well based on NFS_EXPORT_DIR. * Remove "async" option from unfsd to fix warning in syslog: Warning: unknown exports option `async' ignored * Fixed typos Both slirp and tap can work. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2017-01-23oe-build-perf-test: remove unused imports and fix indentMarkus Lehtonen
[YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23oe-build-perf-test: save test metadata in a separate fileMarkus Lehtonen
The patch introduces a new metadata (.json or .xml) file in the output directory. All test meta data, e.g. git revision information and tester host information is now stored there. The JSON report format is slightly changed as the metadata is not present in results.json anymore. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23oe-build-perf-test: enable xml reportingMarkus Lehtonen
Add --xml command line option to oe-build-perf-test script for producing a test report in JUnit XML format instead of JSON. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23testexport.bbclass: Migrate testexport to use new frameworkMariano Lopez
This migrates current testexport implmentation to use the new OEQA framework. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23scripts/oe-test: Add new oe-test scriptAníbal Limón
The new oe-test script will be use to run test components with one single script. The oe-test script search for test components inside meta/lib/oeqa, the test components needs to implement OETestContextExecutor inside context module in order to be supported by oe-test. [YOCTO #10230] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23wic: bootimg-pcbios: use wic-tools targetEd Bartosh
Used wic-tools target to get location of syslinux. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23wic: rebuild wic-tools if its sysroot doesn't existEd Bartosh
Rebuild wic-tools if its native sysroot doesn't exist to ensure that all required tools are available. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23wic: misc.py: add parameter 'cache' to get_bitbake_varsEd Bartosh
This parameter will be used to allow or prevent variable caching. For example, we don't want to cache value of RECIPE_SYSROOT_NATIVE wic-tools variable as we may decide to rebuild wic-tools. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23wic: fixed test_iso_image test caseEd Bartosh
Fixed isoimage-isohybrid plulgin and correspondent wic tet case: - used wic-tools target when getting varlue of STAGING_LIBDIR variable - ensured that image is built with efi and hddimg enabled Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23wic: fixed recipe nameEd Bartosh
Renamed grub-efi-native -> grub-efi in NATIVE_RECIPES dictionary as '-native' suffix is added to the recipe name in the code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23partition.py: use FAKEROOTCMD variable to get path to pseudoEd Bartosh
wic used native sysroot to get path to pseudo utility. This approach doesn't work with recipe specific sysroots. Using FAKEROOTCMD should fix the issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-23wic: fix getting path of native sysrootEd Bartosh
wic used STAGING_DIR_NATIVE variable as a path to native sysroot. This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE points to the native sysroot of the current recipe. Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe to fix the issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-19oe-selftest: Error if the user has buildhistory enabledRichard Purdie
buildhistory interfers with various tests so error if the user has it enabled, hence avoiding time spent running the tests only for them to fail. For example, if there is an sstate cache and something comes from that cache, you can get a "version went backwards" warning from buildhistory which would then change the outcome of a test. Its safer/easier to disable it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19devtool: fix source extraction to work with recipe specific sysrootsPaul Eggleton
When extracting source for a recipe within devtool (for extract, modify or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a temporary directory so that: (a) we pick up all files that get unpacked to the WORKDIR, and (b) we don't disturb the existing build However, with recipe-specific sysroots the sysroots for the recipe will be prepared under WORKDIR, and if we used the system temporary directory i.e. usually /tmp) as used by mkdtemp by default, then our attempts to hardlink files into the recipe-specific sysroots will fail on systems where /tmp is a different filesystem, and we'd have to fall back to copying the files which is a waste of time. Put the temp directory under the WORKDIR to prevent that from being a problem. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> [RP: Add needed mkdirhier call] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19oeqa.utils.metadata: allow storing any bitbake config variablesMarkus Lehtonen
Make it possible to store any bitbake config variables in the metadata. Config values will be stored under a new config element in the xml report: <config> <variable name="MACHINE">qemux86</variable> </config> The value of MACHINE is moved there instead of having a dedicated <machine> element. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: rename 'revision' to 'commit'Markus Lehtonen
Revision is a bit vague and could point to a tag, for example. Git commit objects are unambiguous and persistent so be explicit that the element should contain git commit hash. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: drop 'unknown' git data elementsMarkus Lehtonen
It's better just to not have the xml elements than to have elements with faux data. One could have git branch named 'unknown', for example. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19oeqa.utils.metadata: re-organise distro informationMarkus Lehtonen
Use the same format, based on /etc/os-release, as for host distro information. [YOCTO #10590] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19runqemu-export-rootfs: fix inconsistent var namesRobert Yang
Fixed: $ runqemu nfs qemux86-64 [snip] On your target please remember to add the following options for NFS nfsroot=IP_ADDRESS:/path/to/nfsroot,nfsvers=3,port=,mountprog=,nfsprog=,udp,mountport= [snip] Note that the values are null, this is because their var names are inconsistent. [YOCTO #10519] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19scripts/runqemu: fix checking for <file>.cpio.gzRobert Yang
When "runqemu /path/to/<file>.cpio.gz", it used the last suffix "gz" as the fstype which was wrong. Check filename against self.fstypes firstly can fix the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-19wic: partitionedfs: account for non-table partitions when checking if ↵Maciej Borzecki
logical parititon is needed Commit 8c1c43b7901a9fcd8b279eb4250b08157ad345b7 `wic: Create a logical partition only when it is really mandatory` did not account for partitions that are not present in partition table. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16scripts: python3 fixes and new tool ksumTom Zanussi
'ksum.py' generates a combined summary of vmlinux and module sizes for a built kernel, as a quick tool for comparing the overall effects of systemic tinification changes. Execute from the base directory of the kernel build you want to summarize. Setting the 'verbose' flag will display the sizes for each file included in the summary. Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16scripts/oe-selftest: fix typoChen Qi
Change 'agains' to 'against'. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16runqemu: Allow the user to specity no kernel or rootFSAlistair Francis
In some cirsumstances the user doesn't want to supply a kernel, rootFS or DTB to QEMU. This will occur more now that QEMU supports loading images using a '-device loader' method. Allow users to specify 'none' for QB_DEFAULT_FSTYPE or QB_DEFAULT_KERNEL to avoid supplying these options to QEMU. Signed-off-by: Alistair Francis <alistair.francis@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic: _exec_cmd: produce error if exit code is not 0Ed Bartosh
Current code doesn't always show error output of the external command and even ignores non-zero exit code. Moved checking of exit code value to the lowest level possible: to _exec_cmd. This should make wic to always check exit code of the external command and issue an error if it's not 0. [YOCTO #10816] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16direct.py: fix getting image nameEd Bartosh
part.rootfs_dir was used as an image name in the code. However, when multi-rootfs feature is used this attribute points to the name of the rootfs, e.g. if --rootfs command line is rootfs1=core-image-minimal partf.rootfs_dir is 'rootfs1'. The code also fails when image name is not provided in wic commandline. For example, when wic is called with --rootfs-dir=<path> part.rootfs_dir will contain path and wic will crash trying to call bitbake -e <path> to get value of ROOTFS_SIZE variable. Fixed the code by getting image name properly and checking if it's not a path. [YOCTO #10815] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16canned-wks: remove mpc8315e-rdb.wksEd Bartosh
This file has been moved to meta-yocto-bsp/wic/ Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic/isoimage-isohybrid: remove do_stage_partition()Ioan-Adrian Ratiu
The purpouse of this function was to check dependencies for building a hybrid iso and build them using bitbake if not found. Calling bitbake in this context means this wic plugin itself cannot be instrumented inside bitbake recipes which is undesirable, the benefits of this are clear: there is no need to maintain outside scripts to generate an iso using wic and the isohybrid building logic can be further abstracted away into an isohybrid.bbclass in the future which can be easily inherited or something similar. So remove the function and add all dependencies to NATIVE_RECIPES so that wic can print useful errors when they're not built. To automate building the isohybrid image dependencies, add the following somewhere in your image build inheritence hierarcy (or maybe create a bbclass in the future to do these sort of things automatically): DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \ parted-native dosfstools-native mtools-native grub-efi-native" Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16python-3.5-manifest: Add http module to the netclient packageDerek Straka
Adding http module from Python's standard library. This allow use of the http module without installing all python-misc modules. Signed-off-by: Derek Straka <derek@asterius.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16wic: fix parsing of 'bitbake -e' outputEd Bartosh
Current parsing code can wrongly interpret arbitrary lines that are of 'key=value' format as legitimate bitbake variables. Implemented more strict parsing of key=value pairs using regular expressions. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-16gummiboot: Remove old gummiboot recipe, related class and wks fileAlejandro Hernandez
Since the gummiboot project is no longer being maintained and we are using systemd-boot as a replacement instead, we can now clean up all remaining gummiboot files. [YOCTO #10332] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09oe-selftest: fix behaviour if oe-selftest.log is a dangling symlinkPaul Eggleton
If you delete the log file that the oe-selftest.log symlink points to but not the symlink itself, because we were using os.path.exists() here the code assumed that the symlink didn't exist when in fact it still did. Use os.path.lexists() instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-09gummiboot: Remove/change gummiboot references with systemd-bootAlejandro Hernandez
After systemd-boot was introduced, its been tested for a while with no major issues being found until now, this patch completely replaces all gummiboot instances with systemd-boot ones, taking the next step into cleaning up systemd-boot/gummiboot. [YOCTO #10332] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-06meta/scripts: Various getVar/getVarFlag expansion parameter fixesRichard Purdie
There were a few straggling expansion parameter removals left for getVar/getVarFlag where the odd whitespace meant they were missed on previous passes. There were also some plain broken ussages such as: d.getVar('ALTERNATIVE_TARGET', old_name, True) path = d.getVar('PATH', d, True) d.getVar('IMAGE_ROOTFS', 'True') which I've corrected (they happend to work by luck). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05oe-selftest: Improve BUILDDIR environment handlingRichard Purdie
Its possible something (like bitbake/tinfoil2) may mess around with the environment and using the enviroment as a global variable store isn't particularly nice anyway. This patch changes the BUILDDIR usages so that the environment isn't used as a global store and a global variable is used instead. Whilst that is still not perfect, it does avoid the current double and triple backtraces we're seeing where tinfoil2/bitbake has trampled the enviroment leading to failures of failures making debugging even harder. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-05runqemu: let command line parameters override defaultsPatrick Ohly
It may be necessary to override the parameters gathered for the qemu invocation. For example, the qemux86 machine configuration sets "-vga vmware", but when using OVMF as BIOS, only "-vga std" is supported. By putting the parameters derived from custom runqemu parameters like "qemuparams" after the parameters derived from the machine configuration the user gets the possibility to override those. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: add kickstart file for MPC8315Ed Bartosh
Added kickstart file to produce partitioned image for MPC8315 reference hardware. [YOCTO #8719] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: add --fixed-size wks optionMaciej Borzecki
Added new option --fixed-size to wks. The option can be used to indicate the exact size of a partition. The option cannot be added together with --size, in which case an error will be raised. Other options that influence automatic partition size (--extra-space, --overhead-factor), if specifiec along with --fixed-size, will raise an error. If it partition data is larger than the amount of space specified with --fixed-size option wic will raise an error. Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22devtool: modify: fix usage on the kernelPaul Eggleton
When using devtool modify on the kernel, we have to do a bit of a dance with tinfoil instances because we only find out that we're working on a kernel recipe after tinfoil is initialised, but then we need to build kern-tools-native which we're doing just by running bitbake directly. With the tinfoil2 changes, a datastore for the recipe that we were keeping around across the opening and closing of tinfoil is no longer able to be used. Re-parse the recipe to avoid this problem. (In future this whole thing will be able to be done in the same tinfoil instance thanks to tinfoil2, but that refactoring is yet to be done.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-22wic: look for wks files in <layer>/wicEd Bartosh
Currently wic looks for wks files in <layer dir>/scripts/lib/wic/canned-wks/ directories. This path is too nested and doesn't look consistent with the naming scheme of layer directories. Added <layer>/wic directory to the list of paths to look for wks files. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-12-19oe-selftest: import git module only when neededEd Bartosh
git module is not included into standard Python library and therefore causes import errors on the systems where PythonGit is not installed. As git module only used in the code implementing --repository functionality it's better to import git only in the scope that requires it. [YOCTO #10821] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17wic: obey the rootfs size from the metadataChristopher Larson
When no --size is specified for the rootfs in the .wks, we want to obey the rootfs size from the metadata, otherwise the defined IMAGE_ROOTFS_EXTRA_SPACE and IMAGE_OVERHEAD_FACTOR will not be obeyed. In some cases, this can result in image construction failure, if the size determined by du was insufficient to hold the files without the aforementioned extra space. This fallback from --size to ROOTFS_SIZE was already implemented when --rootfs-dir is specified in the .wks, but it did not occur otherwise, neither when --rootfs-dir= was passed to `wic create` nor when IMAGE_ROOTFS was used. This made a certain amount of sense, as this fallback logic happened at such a level that it wasn't able to identify which partitions were rootfs partitions otherwise. Rather than doing it at that level, we can do it in prepare_rootfs(), which is run by the rootfs source plugins. Note that IMAGE_OVERHEAD_FACTOR and a --overhead-factor in the .wks will now both be applied when --size isn't specified in the .wks. A warning is added about this, though a user won't see it unless wic fails or they examine the do_image_wic log. Fixes [YOCTO #10815] Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>