aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2019-02-21scripts/resulttool: enable manual execution and result creationMazliana
Integrated “manualexecution” operation to resulttool scripts. Manual execution script is a helper script to execute all manual test cases in baseline command, which consists of user guideline steps and the expected results. The last step will ask user to provide their input to execute result. The input options are passed/failed/blocked/skipped status. The result given will be written in testresults.json including log error from the user input and configuration if there is any.The output test result for json file is created by using OEQA library. The configuration part is manually key-in by the user. The system allow user to specify how many configuration they want to add and they need to define the required configuration name and value pair. In QA perspective, "configuration" means the test environments and parameters used during QA setup before testing can be carry out. Example of configurations: image used for boot up, host machine distro used, poky configurations, etc. The purpose of adding the configuration is to standardize the output test result format between automation and manual execution. To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To execute manual test cases, execute the below $ resulttool manualexecution <manualjsonfile> By default testresults.json store in <build_dir>/tmp/log/manual/ [YOCTO #12651] Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-21resulttool: enable merge, store, report and regression analysisYeoh Ee Peng
OEQA outputs test results into json files and these files were archived by Autobuilder during QA releases. Example: each oe-selftest run by Autobuilder for different host distro generate a testresults.json file. These scripts were developed as a test result tools to manage these testresults.json file. Using the "store" operation, user can store multiple testresults.json files as well as the pre-configured directories used to hold those files. Using the "merge" operation, user can merge multiple testresults.json files to a target file. Using the "report" operation, user can view the test result summary for all available testresults.json files inside a ordinary directory or a git repository. Using the "regression-file" operation, user can perform regression analysis on testresults.json files specified. Using the "regression-dir" and "regression-git" operations, user can perform regression analysis on directory and git accordingly. These resulttool operations expect the testresults.json file to use the json format below. { "<testresult_1>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, ... "<testresult_n>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, } To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To store test result from oeqa automated tests, execute the below $ resulttool store <source_dir> <git_branch> To merge multiple testresults.json files, execute the below $ resulttool merge <base_result_file> <target_result_file> To report test report, execute the below $ resulttool report <source_dir> To perform regression file analysis, execute the below $ resulttool regression-file <base_result_file> <target_result_file> To perform regression dir analysis, execute the below $ resulttool regression-dir <base_result_dir> <target_result_dir> To perform regression git analysis, execute the below $ resulttool regression-git <source_dir> <base_branch> <target_branch> [YOCTO# 13012] [YOCTO# 12654] Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-21oe-build-perf-report/gitarchive: Move common useful functions to libraryRichard Purdie
These functions can be reused by the resulttool code so move to the common function library for this purpose. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libdrm: Upgrade 2.4.96 -> 2.4.97Fabio Berton
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3-six: upgrade 1.11.0 -> 1.12.0Richard Purdie
The license checksum changed due to a change in copyright years only. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3-setuptools: upgrade 40.0.0 -> 40.8.0Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3-pip: upgrade 18.0 -> 19.0.2Richard Purdie
The license checksum changed due to copyright year changes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3-pbr: upgrade 5.1.1 -> 5.1.2Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3-gitdb: upgrade 2.0.4 -> 2.0.5Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20diffutils: upgrade to 3.7Chen Qi
The following two patches are removed as they are not needed anymore. 0001-Unset-need_charset_alias-when-building-for-musl.patch 0001-explicitly-disable-replacing-getopt.patch Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20sysstat: upgrade to 12.1.3Chen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20kmod: Fix PVRichard Purdie
Meant to fix this pre merge, the git revision is the PV so no need to add SRCPV. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20glib-networking: upgrade 2.54.1 -> 2.58.0Anuj Mittal
* Autotools support has been removed upstream, so migrate recipe to meson. For changes, see: https://gitlab.gnome.org/GNOME/glib-networking/blob/glib-2-58/NEWS * Remove unsupported configure options: pkcs11, ca-certificates. See: https://bugzilla.gnome.org/show_bug.cgi?id=793281 https://bugzilla.gnome.org/show_bug.cgi?id=753260 License-Update: Change to LGPLv2.1 Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libjpeg-turbo: upgrade 2.0.1 -> 2.0.2Anuj Mittal
For changes, see: https://github.com/libjpeg-turbo/libjpeg-turbo/releases Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20tar: upgrade to 1.31Chen Qi
Remove the musl specific do_install, as it's not suitable for this version. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20kmod: upgrade to 26Chen Qi
Refresh patches to avoid warning. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20opkg-utils: do not set mtime on data.tar.XAlejandro del Castillo
Suggested-by: Tristan Ramseyer <t@ramseyer.email> Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20logrotate.py: restore /etc/logrotate.d/wtmpMingli Yu
During the test logrotate.LogrotateTest.test_1_logrotate_setup, there is below logic: # mkdir $HOME/logrotate_dir # sed -i "s#wtmp {#wtmp {\n olddir $HOME/logrotate_dir#" /etc/logrotate.d/wtmp After all logrotate.LogrotateTest finished, only cleanup $HOME/logrotate_dir as below, but don't restore the config file /etc/logrotate.d/wtmp. [snip] def tearDownClass(cls): cls.tc.target.run('rm -rf $HOME/logrotate_dir') [snip] That's to say, there is one additional line added to /etc/logrotate.d/wtmp and will make the logrotate service start failed when run systemd.SystemdBasicTests.test_systemd_failed Take an example as below when run test as root: # cat /etc/logrotate.d/wtmp # no packages own wtmp -- we'll rotate it here /var/log/wtmp { olddir /root/logrotate_dir missingok monthly create 0664 root utmp minsize 1M rotate 1 } # ls /root/logrotate_dir ls: cannot access '/root/logrotate_dir': No such file or directory # systemctl start logrotate Job for logrotate.service failed because the control process exited with error code. See "systemctl status logrotate.service" and "journalctl -xe" for details. # systemctl status logrotate logrotate.service - Rotate log files Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset> Active: failed (Result: exit-code) since Wed 2019-02-13 03:35:19 UTC; 7s ago Docs: man:logrotate(8) man:logrotate.conf(5) Process: 540 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE) Main PID: 540 (code=exited, status=1/FAILURE) Feb 13 03:35:18 qemumips systemd[1]: Starting Rotate log files... Feb 13 03:35:19 qemumips logrotate[540]: error: wtmp:9 error verifying olddir path /root/logrotate_dir: No such file or directory Feb 13 03:35:19 qemumips logrotate[540]: error: found error in file wtmp, skipping Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Failed with result 'exit-code'. Feb 13 03:35:19 qemumips systemd[1]: Failed to start Rotate log files. Add the logic to restore /etc/logrotate.d/wtmp to make the cleanup complete to fix the above issue. Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20menu-cache: fix upstream version checkAlexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20ffmpeg: update to 4.1.1Alexander Kanavin
Drop backported patches. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20ca-certificates: upgrade 20180409 -> 20190110Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libwebp: upgrade 1.0.1 -> 1.0.2Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20dnf: upgrade 4.0.10 -> 4.1.0Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20btrfs-tools: upgrade 4.19.1 -> 4.20.1Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20librepo: upgrade 1.9.3 -> 1.9.4Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20webkitgtk: upgrade 2.22.5 -> 2.22.6Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20gcr: upgrade 3.28.0 -> 3.28.1Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20createrepo-c: upgrade 0.12.0 -> 0.12.1Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20lighttpd: upgrade 1.4.52 -> 1.4.53Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libcomps: upgrade 0.1.9 -> 0.1.10Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20iso-codes: upgrade 4.1 -> 4.2Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20p11-kit: upgrade 0.23.14 -> 0.23.15Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20meson: upgrade 0.49.0 -> 0.49.2Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libdnf: upgrade 0.24.1 -> 0.26.0Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20epiphany: upgrade 3.30.2 -> 3.30.3Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20vala: upgrade 0.42.4 -> 0.42.5Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libmodulemd: update to 2.1.0Alexander Kanavin
Add support for gobject-introspection. Add v2 API fixes to the gtkdoc/gi disabling patch. Add a dependency on glib-native to allow use of glib-mkenums. Disable v2 API/library for now as libdnf is not compatible with it, yet is erroneously linked against it (if it's enabled). Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20libpipeline: update to 1.5.1Alexander Kanavin
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20glibc: CVE-2019-7309Zhixiong Chi
Backport the CVE patch from the upstream commit 3f635fb43389b54f682fc9ed2acc0b2aaf4a923d Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20python3: fix INCLDIRSTOMAKE in _sysconfigdata.pyChangqing Li
recipe glade do_configure failed after python upgrade to 3.7.2, it will do runtime check of python headers with native python3 under recipe-sysroot-native, it will check under path INCLDIRSTOMAKE, but this value is '/usr/include /usr/include /usr/include/python3.7m /usr/include/python3.7m', which will cause below error: ERROR: This autoconf log indicates errors, it looked at host include and/or library paths while determining system capabilities. fix it by replace /usr/include to STAGING_INCDIR Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20sudo: upgrade to 1.8.27Chen Qi
License-Update: copyright years updated Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20dbus-test: upgrade to 1.12.12Chen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20dbus: upgrade to 1.12.12Chen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-20cups: upgrade to 2.2.10Chen Qi
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19mc: upgrade 4.8.21 -> 4.8.22Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19libinput: upgrade 1.12.3 -> 1.12.6Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19gtk+3: upgrade 3.24.1 -> 3.24.5Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19clutter-gst-3.0: upgrade 3.0.26 -> 3.0.27Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19adwaita-icon-theme: upgrade 3.28.0 -> 3.30.1Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-19grep: upgrade 3.1 -> 3.3Denys Dmytriyenko
License checksum changed due to cleanup, all URLs converted to https. need_charset_alias is no longer being used, drop the patch. Signed-off-by: Denys Dmytriyenko <denys@ti.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-21 07:23:34 +0000