summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2019-06-14runqemu: Add the support to pass multi ports to tcpserial parameterKevin Hao
In some cases(such as the oeqa's qemurunner), we need to setup multi serial devices via the '-serial 127.0.0.1:xx" and the order of them is significant. The mixing use of "tcpserial" and "-serial 127.0.0.1:xx" cause ambiguous issues and we can't fix it by only adjusting the order of them. So add the support to pass multi ports to the tcpserial parameter, this will make sure that the order of setting up the serial is really what we want. [YOCTO Bug 13309] Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> [Fixes qemuppc hang in warrior] Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14resulttool: Remove prints if no tests occurJon Mason
Printing the lack of a test is not necessary (per feedback). Remove this from the template to quieten it. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14resulttool/manualexecution: Enable creation of test case configurationsangeeta jain
Allow the creation of test case configuration file based on user inputs. Where this testcase configuration file will be used by the the manual execution to run selected test cases for a module rather than compulsory run all test cases in manual json file. Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14resulttool: modify to be multi-machineJon Mason
Currently, the code will sum all of the different machine results into a single report of the tests results. This can lead to confusion as to which machine may be experiencing issues. Modify the code to store the results in a per machine basis and report them accordingly. Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14resulttool: add LTP compliance sectionArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14resulttool: add ltp test supportArmin Kuster
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-01wic/bootimg-efi: replace hardcoded volume name with labelChee Yang Lee
volume name should refer to --label in .wks. Replace the hardcoded volume name with label. set "ESP" as default name when no lable specified. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-08resulttool: Add option to dump all ptest logsJoshua Watt
Adds an option to dump all the ptest logs to individual files in a specified directory. If multiple test runs are present, the '--prepend-run' argument will create separate directories for each test run under the target directory and put the logs there to prevent each test run from clobbering the others. [YOCTO #13331] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-08wic: change expand behaviour to match docsRoss Burton
The documentation says that --expand takes a comma-separated list of partition:size pairs, but the code was splitting on hyphens. Hyphens are not a transitional separator for a list of items, so change the code to reflect the documentation. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-08recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan
If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02resulttool/manualexecution: Enable test case configuration optionsangeeta jain
Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual There are cases when all test cases all not required to run for a module on specific DUT. Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help in reducing testing time and reporting unneccesary skip or failures. Example pre-defined json format test case configuration file (for build-applince): { "testcases" : [ "build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance", "build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)" ] } Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02resultool/resultutils: Fix module import errorRichard Purdie
Fix AttributeError: module 'urllib' has no attribute 'request' when using remote http urls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02resulttool: Add log subcommandJoshua Watt
Adds a subcommand for dumping various logs from test results Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02resulttool: Load results from URLJoshua Watt
Adds support for resulttool to load JSON files directly from a http:// or https:// URL Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-12runqemu: do not check return code of tputChen Qi
The subprocess.run was replaced by subprocess.check_call because of compatibility support down to python 3.4. But we really don't care about whether that command succeeds. Some user reports that in some tmux environment, this command fails and gives some unpleasant traceback output. So we use 'call' instead of 'check_call' to avoid such problem. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11resulttool/manualexecution: Refactor and remove duplicate codeYeoh Ee Peng
Remove duplicate codes. Replace unnecessary class variables with local variables. Rename variables and arguments with simple and standard name. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-11recipetool: add missed moduleLei Yang
When I use recipetool to add a new recipe, it says: [snip] sys.exit(14) NameError: name 'sys' is not defined [snip] Signed-off-by: Lei Yang <Lei.Yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-10resulttool/manualexecution: Enable creation of configuration option fileYeoh Ee Peng
Allow the creation of configuration option file based on user inputs. Where this configuration option file will be used by the the manual execution to display options for configuration rather than user need to inputs configuration manually. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-10resulttool/manualexecution: Enable configuration options selectionYeoh Ee Peng
Current manualexecution required user to input configuration manually where there were inconsistent inputs and human typo issues. Enable manualexecution to have the optional feature where it will use pre-compiled configuration options file where user will be able to select configuration from the pre-compiled list instead of manual key-in the configuration. This will eliminate human error. Expect the pre-compiled configuration options file in json format below { "bsps-hw": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "beaglebone-yocto", "2": "edgerouter", "3": "mpc8315e-rdb", "4": "genericx86", "5": "genericx86-64" } }, "bsps-qemu": { "IMAGE_BASENAME": { "1": "core-image-sato-sdk" }, "MACHINE": { "1": "qemuarm", "2": "qemuarm64", "3": "qemumips", "4": "qemumips64", "5": "qemuppc", "6": "qemux86", "7": "qemux86-64" } } } Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-05devtool: standard: Handle exporting generated config fragmentsNathan Rossi
The cml1 and ccmake bbclasses generate configuration fragment source files that must be exported from the WORKDIR as a source file to be preserved across builds. This change adds detection of the current recipes inherited classes and for cml1 and ccmake classes checks for the specific generated configuration fragment files. These files are then exported by devtool and included as SRC_URI files from within the target layer. Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Refactor and simplify codebaseYeoh Ee Peng
Simplify and removed unnecessary codes. Refactor to allow pythonic loop. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Fixed step sorted by integerYeoh Ee Peng
Currently the manual execution display step by sorting the step as string, where steps were not being sorted correctly when there are more than 9 steps. Fixed the step sorting by sorting step as integer. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Enable display full steps without press enterYeoh Ee Peng
Current manualexecution required pressing enter button to show each step information, where this was wasting execution time. Enable display full steps without needing to any press enter button. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool/manualexecution: Standardize input checkYeoh Ee Peng
Current input checking does not match the standard input practiced by QA team. Change the input checking to match the standard input practiced by the QA team. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool: Allow extraction of ptest dataRichard Purdie
Rather than simply discarding the ptest data, change the code to discard it when writing out the new testresult files, or optionally either preserve it, or write it as seperate discrete logs. This means the autobuilder should start writing out individual ptest log files as well as allowing ueers to extract these manually. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04resulttool: Allow store to work on single filesRichard Purdie
Store operations using a single file as a source weren't working as the os.walk command didn't like being given a single file. Fix the store operation to work for single files. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-03wic/engine: Add missing newlineRichard Purdie
On some hosts the wic.Wic2.test_wic_cp_ext selftest was failing as files weren't being copied into the rootfs with "wic cp". This was due to a bug added by: http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=89d4a8df074598cfb3a76e41db7c45d845afd961 where there should be a second newline added at the end of the expression due to the difference in the way echo -e and printf behave. [YOCTO #13237] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-03resulttool: Enable report for single result fileYeoh Ee Peng
Current validation check function inside resulttool disallow the report for single result file although the underlying library was able to handle both directory and file as source input to report. Removed the validation check as it was no longer needed and to enable report for single result file. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-24devtool: prevent starting git gc in backgroundTaras Kondratiuk
Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "<snip>/openembedded-core/scripts/devtool", line 342, in <module> ret = main() File "<snip>/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] Signed-off-by: Taras Kondratiuk <takondra@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18resulttool/manualexecution: To output right test case idMazliana
We found that manualexecution does not capture test suite values correctly if there are more than one test suite in test cases. After verification has made we found out we should retrieved full test cases value <test_module.test_suite.test_case> from oeqa/manual/ json file rather than split it them into new variables test_suite and test_cases. Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-18resulttool/report: Enable roll-up report for a commitYeoh Ee Peng
Enable roll-up all test results belong to a commit and to provide a roll-up report. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-12correct do_patch for kernel bbappend in sdkYann CARDAILLAC
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python that was the case with Phytec's BSP, the fix was to replace the do_patch rule with : do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-07scripts/resulttool: Enable manual result store and regressionYeoh Ee Peng
To enable store for testresults.json file from manualexecution, add layers metadata to configuration and add "manual" map to resultutils.store_map. To enable regression for manual, add "manual" map to resultutils.regression_map. Also added compulsory configurations ('MACHINE', 'IMAGE_BASENAME') to manualexecution. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06scripts: Add oe-pylintRichard Purdie
Add an oe-pylint wrapper around pylint3 to run static analysis on our python codebase. For now display errors and exclude common warnings which are not applicable to our codebase. The aim is to start generating this report on the autobuilder and to improve our score over time. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-06send-error-report: Add --no-ssl to use http protocolRobert Yang
The script use https protocol by default, but the error-report-web server's https connection may not work (e.g., doesn't work with python 2.7.6), so add an option --no-ssl to make it use http. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-05oe.scriptutils: enable color in a more flexible wayChristopher Larson
Rather than recreating handlers and forcing them, iterate over the handlers and enable color on ones we can handle. This makes it easier to handle color properly when we introduce the bb.msg default log filters. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-04runqemu: make gl options gtk+-specificAlexander Kanavin
If in the future GL is enabled for SDL frontend, or some other frontend, it would be useful to be able to differentiate. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-04runqemu: add an option to choose the SDL frontend instead of Gtk+ defaultAlexander Kanavin
When Gtk+ and SDL are both enabled, qemu defaults to Gtk+. This option allows to revert to the 'classic' frontend. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28scripts/wic: Be consistent about how we call bitbakeRichard Purdie
Whilst its tempting to call bitbake_main(), the script also directly executes bitbake. By doing both this creates all kinds of races. Therefore change to use subprocess execution in all cases. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28wic: Ensure that sourceparams values can contain '='Jan Kiszka
This allows '--sourceparams key1=val1=val11,key2=val2'. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28resulttool/regression: Ensure regressoin results are sortedYeoh Ee Peng
Sorted regression results to provide friendly viewing of report. Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28resulttool/store: Fix missing variable causing testresult corruptionRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-27 resulttool/report: Ensure ptest results are sortedRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-27resulttool/report: Ensure test suites with no results show up on the reportRichard Purdie
ptest suites with no results don't show up on the reports even though we have a duration for them. Fix this so the fact they report no tests is visible. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-27resulttool/report: Handle missing metadata sections more cleanlyRichard Purdie
Currently some older results files cause the code to give tracebacks. Handle these missing sections more cleanly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-27resulttool/store: Handle results files for multiple revisionsRichard Purdie
Currently we cant store results if the results files span multiple different build revisons. Remove this limitation by iterating. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25runqemu: do not check for GL librariesAlexander Kanavin
qemu has been using libepoxy for a long time, and libepoxy loads GL via dlopen() only when instructed to. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25runqemu: add options for enabling virgl GL accelerationAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25devtool: deploy-target: allow use of alternate sshOlaf Mandel
Allow the use of an alternate SSH executable for deploy-target and undeploy-target. The command can be specified with -e / --ssh-exec. Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>