summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2019-12-06hosttools: no longer check for or provide host python 2 to buildsAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-05scriptutils: remove the useless import lineChen Qi
imp was replaced by importlib as it's deprecated, but the import line was left, so remove this useless line. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27wic: 'wic cp' to copy from imageChee Yang Lee
currently 'wic cp' only works for copy file from local storage to wic image. enhance 'wic cp' to copy file/directory from wic image to local storage. include selftest and 'wic help' updates. [YOCTO#12169] Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21runqemu: add options that enable virgl with the SDL frontendAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21devtool: fix devtool upgrade with reproducible_builds classPaul Eggleton
If the reproducible_build class is inherited then there may be a "source-date-epoch" subdirectory in a fetched source tree; devtool upgrade was not expecting that in the upgraded source. Take a small snippet of code from recipetool create which already handles this, and make it a shared function that can be used in both places. Additionally, fix an assumption that the source is always in a subdirectory in the cleanup code that blocked debugging this. [YOCTO #13635] Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21Revert "devtool/standard.py: Not filtering devtool workspace for devtool finish"Alexander Kanavin
This reverts commit 41d225f4a37d02e9f79bdbfb79caac8cd3d291ce. Unfortunately this change broke 'devtool upgrade' functionality, causing 'devtool finish' to write out an upgraded recipe that no longer includes the original upstream source in SRC_URI. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21recipetool/create: Fix to work with reproducible_buildsRichard Purdie
The extra diretory created by reproducibile_builds was confusing recipetool and hence devtool. Ignore this directory to restore the correct behaviour and allow it to work with the class. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21wic: rm with -r flag supportChee Yang Lee
wic currently unable to remove non-empty directory in ext* partition. enable wic rm to remove non-empty directory and all the sub-content with -r flag. update help documents for 'wic rm'. [YOCTO #12404] Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14scripts/native-intercept: Add chgrp interceptPaul Barker
Some installation rules have been seen to run chgrp for native recipes which leads to 'Operation not permitted' errors. To prevent this we need a chgrp intercept script to go with the existing chown intercept. Signed-off-by: Paul Barker <paul@betafive.co.uk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14resulttool/store.py: Enable add extra test environment dataYeoh Ee Peng
Enable the option to add extra test environment data to the configuration of each test result (as optional). Example of optional test environment data include: - custom packages included for runtime test - detail machine specification used as target - detail host environment used for bitbake Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14scripts/resulttool/report: Add total statistic to test result.Yeoh Ee Peng
Add total passed, failed, and skipped statistic to test result. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14scripts/resulttool/report: Enable output raw test resultsYeoh Ee Peng
In case of debugging, report user need to acccess the raw test result. Instead of going back to source file/directory/URL to manually pull out the raw result, provide alternative way to let report showing raw test results by providing the result id (optional). Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14scripts/resulttool/report: Enable report to use regression_mapYeoh Ee Peng
By default, report will use the store_map to generate the key to reference each result set. In some situation when using store_map with multiple set of tests sharing similar test configurations, the report will only showing partial result set for results that having identical result_id (use of multiconfig to run tests where it generate identical result_id). Enable report to have the option to use the regression_map (optional) instead of the default store_map, where it will take larger set of configurations to generate the key to reference each result set, this will prevent the report from only showing partial result set. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14wic: beautify 'wic help'Chee Yang Lee
The Wic help returned to the user is unreadable. Use a custom ArgumentParser to override argparse help message. change help message as suggest in https://bugzilla.yoctoproject.org/show_bug.cgi?id=12205 [YOCTO #12205] changes applies to 'wic help', 'wic -h', 'wic --h' and 'wic --help' Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10isoimage-isohybrid.py: Parameterize ESP partition sizeHaris Okanovic
Add "esp_extra_blocks" plugin parameter so that caller may change ESP's free space from the default 100 blocks. Signed-off-by: Haris Okanovic <haris.okanovic@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10isoimage-isohybrid.py: Parameterize ESP labelHaris Okanovic
Add "esp_label" plugin parameter so that caller may override default ESP partition label 'EFIimg'. Signed-off-by: Haris Okanovic <haris.okanovic@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-04scripts/oe-pkgdata-util: Enable list-pkgs to print ordered packagesYeoh Ee Peng
The list-pkgs currently print packages in unordered format. Enable list-pkgs to print ordered packages that will ease viewing. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-10-23wic/engine: use 'linux-swap' for swap file systemChee Yang Lee
[YOCTO #13312] see https://bugzilla.yoctoproject.org/show_bug.cgi?id=13312 wic/engine.Disk._get_part_image was looking at variable fstypes for supported fstype which is 'swap' but image build with 'linux-swap'. supported fstype should be 'linux-swap'. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23wic/help: change 'wic write' help descriptionChee Yang Lee
update the help desciption for 'wic write' command to match behaviour. see commit: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/scripts/wic?id=74b44e59be89c738f702948f9e16e367c0ba43ba Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23devtool: add ssh key option to deploy-target paramEduardo Abinader
Signed-off-by: Eduardo Abinader <eduardoabinader@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-10-22devtool: Add --remove-work option for devtool reset commandSai Hari Chandana Kalluri
Enable --remove-work option for devtool reset command that allows user to clean up source directory within workspace. Currently devtool reset command only removes recipes and user is forced to manually remove the sources directory within the workspace before running devtool modify again. Using devtool reset -r or devtool reset --remove-work option, user can cleanup the sources directory along with the recipe instead of manually cleaning it. syntax: devtool reset -r <recipename> Ex: devtool reset -r zip devtool finish -r <recipename> <layer-name> Ex: devtool finish -r zip meta-yocto-bsp Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-10-19wic/rawcopy: Support files in sub-directoriesEugene Smirnov
If the source file is located in a subdirectory of DEPLOY_DIR rawcopy will currently fail in sparse_copy function on open(dst_fname, 'wb'), as the parent directory for destination file does not exist. This patch helps to avoid that by recursively creating parent directories. Signed-off-by: Eugene Smirnov <evgenii.smirnov@here.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-15runqemu: Remove disabling of high resolution timerJoerg Vehlow
The option 'highres=off' sneaked itself into the runqemu script for all configurations, where the root filesystem type is not 'cpio' or 'cpio.gz'. See: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13590 Signed-off-by: Joerg Vehlow <joerg.vehlow@aox-tech.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-11runqemu: unset another environment variable for 'egl-headless'Alexander Kanavin
Some host distributions (opensuse for example) are using 'pkgconf', which, unlike the original pkg-config, appends PKG_CONFIG_SYSROOT_DIR to every directory from the .pc file. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09scripts/gen-lockedsig-cache: Don't list paths which don't existRichard Purdie
This avoids failures seen on the autobuilder when generating eSDKs and release sstate copies. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-07wic/direct: Partition numbering is broken for MBR primary partition #4Michael Cooper
When wks describes extra partitions that aren't in the partition table (e.g. boot loader) and exactly four primary MBR partitions, the last partition gets added to fstab as partition #5 instead of #4. [YOCTO #13560] Signed-off-by: Michael Cooper <michaelcooper81@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-07devtool/standard.py: Not filtering devtool workspace for devtool finishJaewon Lee
All devtool commands right now are filtering out the devtool workspace bbappends in build/workspace/appends when calling parse_recipe. While this may make sense for devtool add and modify, we need devtool finish to include those appends. A specific breakage that is caused because devtool finish filters devtool appends is the cmake/cml1 flow where a file is created in the WORKDIR that finish needs access to, to commit those files. Particularly for git packages with SRCPV in PV, SRCPV is only changed to 999 when using external source, hence when creating the cfg or cmake config files using for instance bitbake -c diffconfig, these files are created in the git999 workdir correctly (as in the devtool bbapends, we are inheriting externalsrc class). But when devtool finish is run, the devtool appends are not parsed, hence SRCPV is not changed to 999 and devtool is looking for the fragment files in the wrong WORKDIR. Changing the parse_recipe call just in devtool finish to not filter out the devtool workspace. Fixes [YOCTO #13533] Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02gen-lockedsig-cache: Replace glob lookup with hash to filename lookupKonrad Scherer
Using the glob function to map signatures to sstate files is very slow when the sstate is large and accessed over nfs. The lookup now only loads the necessary prefixes and doesn't use glob as all. Unfortunately I don't have access to the systems where the performance isse was noticed and on my test system the glob is fast enough that the performance numbers aren't useful. I could verify that file list returned by the new code is the same. [YOCTO #13539] Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27oeqa/core/case.py: Encode binary data of logNathan Rossi
Do not decode the log content into a string only to re-encode it as binary data again. Some logs might un-intentionally contain bytes that do not decode as utf-8, as such preserve the log file content as it was on disk. Handle the decoding on the resulttool side, but also handle the failure to decode the data. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27devtool: finish: Add suppport for the --no-clean optionPeter Kjellerstedt
This works just like the already existing --no-clean option to the `devtool reset` command. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27devtool: finish: Keep patches ordered when updating bbappendNiclas Svensson
The _get_patchset_revs() function returns the patches in an OrderedDict to keep them ordered. However, this information was lost when the patches were added to the bbappend file. Signed-off-by: Niclas Svensson <niclas.svensson@axis.com> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27wic: Using the right rootfs size during prepare_rootfsAlessio Igor Bogani
The commit 8e48b4d6c4 makes wic ignores IMAGE_ROOTFS_SIZE for rootfs size and makes it uses the computed one only. Re-add support for IMAGE_ROOTFS_SIZE variable and compute roots size only if the former is not defined. Moreover the size of a provided directory with --rootfs-dir="" in the .wks file should always be computed on the fly, else every partition will be constrained to be the same size as what ever value was in ROOTFS_SIZE. Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu> Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18runqemu: Mention snapshot in the help outputRichard Purdie
This is a useful option but not documented in the help text. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16runqemu: decouple gtk and gl optionsAlexander Kanavin
This will allow not having to multiply these options for the sdl frontend, instead combining them as needed. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15oe-git-proxy: fix dash "Bad substitution"Henning Schild
The script claims it works with dash, make sure that is actually the case. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15oe-git-proxy: NO_PROXY suffix matching without wildcard for match_hostHenning Schild
NO_PROXY can also contain just suffixes that do not start with a "*". We failed to match those so far. Just add an extra "*" to also match those suffixes. If one was there we get "**" which does not hurt. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15oe-git-proxy: disable shell pathname expansion for the whole scriptHenning Schild
This truly fixes the issue that cbc148d5d93d5f3531434fee7b234a16196b3088 wanted to solve, without breaking the iteration over multiple entries. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15Revert "oe-git-proxy: Avoid resolving NO_PROXY against local files"Henning Schild
This reverts commit cbc148d5d93d5f3531434fee7b234a16196b3088. The quoting causes H to be one string with spaces, so looping over multiple entries does not work anymore. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-15oe-git-proxy: allow setting SOCAT from outsideHenning Schild
This allows to write selftests where we can mock the real socat. Signed-off-by: Henning Schild <henning.schild@siemens.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-11oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpersNathan Rossi
Add the OEPTestResultTestCase class as a mix-in class to provide helper functions for interacting with ptestresults within the extraresults object generated by the test case. This class also provides default compression of log text and log files. Also add support to resulttool for decoding/decompressing log files embedded in the test results. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-11resulttool: Handle multiple series containing ptestresultsNathan Rossi
Handle multiple results series having ptestresults content. The contents are merged on a per-result basis where duplicates are ignored (with a warning message printed). The 'ptestresults.sections' collection is also merged on a per-suite basis. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06resulttool: Add reproducible log extractionJoshua Watt
Adds an argument to the log subcommand to extract the raw logs from the reproducible selftest. To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility. [YOCTO #13324] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-06runqemu: Fix typo in help text.Drew Moseley
Signed-off-by: Drew Moseley <drew.moseley@northern.tech> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-09-06oe-selftest: Implement console 'keepalive' outputNathan Rossi
Similar to bitbake, implement a 'keepalive' output to the console to ensure CI systems do not kill the process. The default timeout for bitbake is 5000s. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-01qemuriscv: Generate a wic rootFS with a larger filesystemAlistair Francis
This allows us to generate a rootFS with a large filesystem for use with QEMU. Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-31devtool: Avoid failure for recipes with S == WORKDIR and no local filesPeter Kjellerstedt
When extracting the sources for a recipe that has S == WORKDIR and no local files in the SRC_URI (which, e.g., can happen for a recipe with a URI that has the unpack=false attribute), the extraction fails with the following backtrace: Traceback (most recent call last): File ".../scripts/devtool", line 344, in <module> ret = main() File ".../scripts/devtool", line 331, in main ret = args.func(args, config, basepath, workspace) File ".../poky/scripts/lib/devtool/standard.py", line 762, in modify initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) File ".../poky/scripts/lib/devtool/standard.py", line 647, in _extract_source bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree) File ".../poky/bitbake/lib/bb/process.py", line 178, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git commit -a -m "Committing local file symlinks %% ignore"' failed with exit code 1: On branch devtool nothing to commit, working tree clean This is because no files were found in the oe-local-files directory and consequently no symbolic links were added using `git add`, but the `git commit` command was still executed. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-30qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mipsAlexander Kanavin
This is the qemu default since qemu 2.2, is generally supported better, and is recommended by upstream. It also has already been in use for arm/risc and ovmf. Additional information: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13466 https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/ '-vga virtio' emulated hardware remains in use when virgl is enabled via a runqemu override. Also, adjust the error whitelist, as there is a number of new messages coming from the drivers that are not actual errors. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-30yocto-check-layer: Allow any case for README file detectionMark Hatle
It's become more commone for files to be named "readme" or "Readme" on github servers in recent time. So adjust the scanning to allow any mix of case. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-29qemu: set default RAM to 256M for all machinesAlexander Kanavin
There was a discussion about what amount of RAM is appropriate for a default; the outcome was that for now it is still 256M. Some qemu machine definitions have however set this to 512M so for the sake of treating all architectures fairly, they are reset back to 256M. Also runqemu is adjusted to use 256M if QB_MEM is not set at all. http://lists.openembedded.org/pipermail/openembedded-core/2019-August/285900.html Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28scripts/lib/resulttool/report.py: Add more result typesNathan Rossi
Add additional result types into the dictionary to handle dejagnu style test results. These include PASS, FAIL, XPASS, XFAIL, UNSUPPORTED, UNTESTED, UNRESOLVED and ERROR. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>