summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2019-06-19wic/bootimg-efi: allow multiple initrdChee Yang Lee
Allow plugin bootimg-efi to configure with multiple initrd through source parameter. Uses ; to separate each initrd. e.g: --sourceparams="loader=${EFI_PROVIDER},initrd=initrd1;initrd2" Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/merge: Enable control TESTSERIES and extra configurationsYeoh Ee Peng
Current QA team need to merge test result files from multiple sources. Adding TESTSERIES configuration too early will have negative implication to report and regression. Enable control to add TESTSERIES when needed. Also enable adding EXECUTED_BY configuration when needed. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/store: Enable add EXECUTED_BY config to resultsYeoh Ee Peng
Current results stored does not have information needed to trace who executed the tests. Enable store to add EXECUTED_BY configuration to results file in order to track who executed the tests. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/resultutils: Enable add extra configurations to resultsYeoh Ee Peng
Current resultutils library always add "TESTSERIES" configuration to results. Enhance this to allow control of adding "TESTSERIES" configuration as well as allow adding extra configurations when needed. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-11wic/plugins: kernel image refer to KERNEL_IMAGETYPEChee Yang Lee
replaced hardcoded kernel image with KERNEL_IMAGETYPE. set kernel image to "bzImage" incase KERNEL_IMAGETYPE not set. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10recipetool: add python3 supportMaciej Pijanowski
Add support for generating python3 recipes using the recipetool / devtool. Drop python2 support at the same time. Tested with: oe-selftest -r recipetool.RecipetoolTest [YOCTO #13264] Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10runqemu: QB_FSINFO to support fstype wic imagesAdrian Freihofer
wic images are handled as vmtype images. Starting qemu with "-kernel" parameter and an image of type wic is not supported. Especially for "-machine virt" the combination of wic with -kernel parameter would be beneficial. The new parameter QB_FSINFO allows to pass image type specific flags to runqemu. QB_FSINFO is a space separated list of parameters. Parameters are structured according to the following pattern: image-type:flag. For now two parameters are supported: - wic:no-kernel-in-fs The wic image is treated as rootfs only image. A -kernel option is passed to qemu. - wic:kernel-in-fs The wic image is treated as VM image including a bootloader and a kernel. This is still the default behavior. Example: QB_DEFAULT_FSTYPE = "wic" QB_FSINFO = "wic:no-kernel-in-fs" QB_KERNEL_ROOT = "/dev/vda1" QB_SYSTEM_NAME = "qemu-system-aarch64" QB_MACHINE = "-machine virt" ... [YOCTO #13336] Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-08wic/filemap: handle FIGETBSZ failingRoss Burton
Some file systems don't support fetching the block size (notably the file system Docker uses for containers), so handle the iotctl() failing and raise the expected error. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07wic/engine: include .wks.in in wic search and listChee Yang Lee
allow wic to list and search for kickstart file in .wks.in extension. basename show by wic list images to fully exclude extension. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07runqemu: Add the support to pass multi ports to tcpserial parameterKevin Hao
In some cases(such as the oeqa's qemurunner), we need to setup multi serial devices via the '-serial 127.0.0.1:xx" and the order of them is significant. The mixing use of "tcpserial" and "-serial 127.0.0.1:xx" cause ambiguous issues and we can't fix it by only adjusting the order of them. So add the support to pass multi ports to the tcpserial parameter, this will make sure that the order of setting up the serial is really what we want. [YOCTO Bug 13309] Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool: Remove prints if no tests occurJon Mason
Printing the lack of a test is not necessary (per feedback). Remove this from the template to quieten it. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool/manualexecution: Enable creation of test case configurationsangeeta jain
Allow the creation of test case configuration file based on user inputs. Where this testcase configuration file will be used by the the manual execution to run selected test cases for a module rather than compulsory run all test cases in manual json file. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool: modify to be multi-machineJon Mason
Currently, the code will sum all of the different machine results into a single report of the tests results. This can lead to confusion as to which machine may be experiencing issues. Modify the code to store the results in a per machine basis and report them accordingly. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-31runqemu: Add support for kvm on aarch64Richard Purdie
The main issue is to make the x86 checks apply to x86 targets only. We may end up with better checks on other architectures but this adapts the code to allow for that and its still controlled by whether QB_CPU_KVM is set. The code needed minor refactoring so the qemu-system-XXX name is set earlier so the kvm code can use it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-30wic: bootimg-efi: add label source parameterChee Yang Lee
Add new source parameter label to allow custom boot.conf/grub.cfg label, so far it's hardcoded to "Boot". Default label to "Boot" for systemd-boot and blank for grub-efi when source parameter label are not set. Signed-off-by: Chee Yang Lee <chee.yang.lee at intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-15wic/bootimg-efi: replace hardcoded volume name with labelChee Yang Lee
volume name should refer to --label in .wks. Replace the hardcoded volume name with label. set "ESP" as default name when no lable specified. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08resulttool: Add option to dump all ptest logsJoshua Watt
Adds an option to dump all the ptest logs to individual files in a specified directory. If multiple test runs are present, the '--prepend-run' argument will create separate directories for each test run under the target directory and put the logs there to prevent each test run from clobbering the others. [YOCTO #13331] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08scripts/pybootchart/draw: Fix some bounding problemsRichard Purdie
The chart size extents were being incorrectly reported, not accounting for the width of the legend. Set a minimum width to account for that (its fixed size). Also stop printing the chart background off the bottom of the chart extents. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08scripts/pybootchart/draw: Clarify some variable namesRichard Purdie
This tweaks some intermediate variable names to make it clearer what is being done. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08scripts/pybootchart: Port to python3Richard Purdie
This updates the pybootchart code (used for viewing build timing profiles) to use python3. The bulk of the changes are to use gi instead of pygtk, i.e. port from gtk+2 to gtk+3. The main change is to make the bootchart widget inherit gtk.Scrollable and change the way the scrollbars are implemented to match the new method upstream. The drawing code used cairo already so can remain unchanged, Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08scripts/pybootchart: Fix mixed indentationRichard Purdie
The script had a toxic mix of tabs and spaces, fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan
If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04wic: add global debug optionRoss Burton
Add a global --debug option to assist debugging. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04wic: change expand behaviour to match docsRoss Burton
The documentation says that --expand takes a comma-separated list of partition:size pairs, but the code was splitting on hyphens. Hyphens are not a transitional separator for a list of items, so change the code to reflect the documentation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29resulttool: add LTP compliance sectionArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29resulttool: add ltp test supportArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26resulttool/manualexecution: Enable test case configuration optionsangeeta jain
Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual There are cases when all test cases all not required to run for a module on specific DUT. Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help in reducing testing time and reporting unneccesary skip or failures. Example pre-defined json format test case configuration file (for build-applince): { "testcases" : [ "build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance", "build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)" ] } Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26Revert "wic: Set a miniumum FAT16 volume size."Angus Lees
This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83. FAT16 volumes do not have a minimum size of 8250 blocks (== slightly over 4MB). Exhibit A: floppy disks. The original commit message suggests this was a workaround for a bug in parted - in which case we should fix it there, or use the wic `--fixed-size` option to pass down an enforced minimum from a more context-aware point in the callstack. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26resultool/resultutils: Fix module import errorRichard Purdie
Fix AttributeError: module 'urllib' has no attribute 'request' when using remote http urls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-25resulttool: Add log subcommandJoshua Watt
Adds a subcommand for dumping various logs from test results Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-25resulttool: Load results from URLJoshua Watt
Adds support for resulttool to load JSON files directly from a http:// or https:// URL Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-12runqemu: do not check return code of tputChen Qi
The subprocess.run was replaced by subprocess.check_call because of compatibility support down to python 3.4. But we really don't care about whether that command succeeds. Some user reports that in some tmux environment, this command fails and gives some unpleasant traceback output. So we use 'call' instead of 'check_call' to avoid such problem. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11resulttool/manualexecution: Refactor and remove duplicate codeYeoh Ee Peng
Remove duplicate codes. Replace unnecessary class variables with local variables. Rename variables and arguments with simple and standard name. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11recipetool: add missed moduleLei Yang
When I use recipetool to add a new recipe, it says: [snip] sys.exit(14) NameError: name 'sys' is not defined [snip] Signed-off-by: Lei Yang <Lei.Yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-10resulttool/manualexecution: Enable creation of configuration option fileYeoh Ee Peng
Allow the creation of configuration option file based on user inputs. Where this configuration option file will be used by the the manual execution to display options for configuration rather than user need to inputs configuration manually. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-10resulttool/manualexecution: Enable configuration options selectionYeoh Ee Peng
Current manualexecution required user to input configuration manually where there were inconsistent inputs and human typo issues. Enable manualexecution to have the optional feature where it will use pre-compiled configuration options file where user will be able to select configuration from the pre-compiled list instead of manual key-in the configuration. This will eliminate human error. Expect the pre-compiled configuration options file in json format below { "bsps-hw": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "beaglebone-yocto", "2": "edgerouter", "3": "mpc8315e-rdb", "4": "genericx86", "5": "genericx86-64" } }, "bsps-qemu": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "qemuarm", "2": "qemuarm64", "3": "qemumips", "4": "qemumips64", "5": "qemuppc", "6": "qemux86", "7": "qemux86-64" } } } Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-05devtool: standard: Handle exporting generated config fragmentsNathan Rossi
The cml1 and ccmake bbclasses generate configuration fragment source files that must be exported from the WORKDIR as a source file to be preserved across builds. This change adds detection of the current recipes inherited classes and for cml1 and ccmake classes checks for the specific generated configuration fragment files. These files are then exported by devtool and included as SRC_URI files from within the target layer. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Refactor and simplify codebaseYeoh Ee Peng
Simplify and removed unnecessary codes. Refactor to allow pythonic loop. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Fixed step sorted by integerYeoh Ee Peng
Currently the manual execution display step by sorting the step as string, where steps were not being sorted correctly when there are more than 9 steps. Fixed the step sorting by sorting step as integer. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Enable display full steps without press enterYeoh Ee Peng
Current manualexecution required pressing enter button to show each step information, where this was wasting execution time. Enable display full steps without needing to any press enter button. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Standardize input checkYeoh Ee Peng
Current input checking does not match the standard input practiced by QA team. Change the input checking to match the standard input practiced by the QA team. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool: Allow extraction of ptest dataRichard Purdie
Rather than simply discarding the ptest data, change the code to discard it when writing out the new testresult files, or optionally either preserve it, or write it as seperate discrete logs. This means the autobuilder should start writing out individual ptest log files as well as allowing ueers to extract these manually. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool: Allow store to work on single filesRichard Purdie
Store operations using a single file as a source weren't working as the os.walk command didn't like being given a single file. Fix the store operation to work for single files. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-03wic/engine: Add missing newlineRichard Purdie
On some hosts the wic.Wic2.test_wic_cp_ext selftest was failing as files weren't being copied into the rootfs with "wic cp". This was due to a bug added by: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=89d4a8df074598cfb3a76e41db7c45d845afd961 where there should be a second newline added at the end of the expression due to the difference in the way echo -e and printf behave. [YOCTO #13237] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-03resulttool: Enable report for single result fileYeoh Ee Peng
Current validation check function inside resulttool disallow the report for single result file although the underlying library was able to handle both directory and file as source input to report. Removed the validation check as it was no longer needed and to enable report for single result file. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-24devtool: prevent starting git gc in backgroundTaras Kondratiuk
Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "<snip>/openembedded-core/scripts/devtool", line 342, in <module> ret = main() File "<snip>/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] Signed-off-by: Taras Kondratiuk <takondra@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18resulttool/manualexecution: To output right test case idMazliana
We found that manualexecution does not capture test suite values correctly if there are more than one test suite in test cases. After verification has made we found out we should retrieved full test cases value <test_module.test_suite.test_case> from oeqa/manual/ json file rather than split it them into new variables test_suite and test_cases. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18resulttool/report: Enable roll-up report for a commitYeoh Ee Peng
Enable roll-up all test results belong to a commit and to provide a roll-up report. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-12correct do_patch for kernel bbappend in sdkYann CARDAILLAC
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python that was the case with Phytec's BSP, the fix was to replace the do_patch rule with : do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>