aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib
AgeCommit message (Collapse)Author
2018-08-23oeqa/core/decorator: add skipIfInDataVarRichard Leitner
skipIfInDataVar will skip a test if a value is in a certain variable. Signed-off-by: Richard Leitner <richard.leitner@skidata.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/patch.py: Clean up getstatusoutput usageRobert Yang
We can't use subprocess.check_output() or subprocess.call() here since the one who invokes runcmd() needs handle CmdError() exception (error out or ignore it). Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/gpg_sign.py: Clean up getstatusoutput usageRobert Yang
Replace usage of oe.utils.getstatusoutput() with direct subprocess calls. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/utils: Fix get_multilib_datastore to work for original tuneRichard Purdie
Currently the original datastore returned by this function doesn't always work as the tune isn't set back to the original. Fix it to work like all_multilib_tune_list() in utils.bbclass and correct the data returned. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23lib/oe/package.py: use bb.utils.break_hardlinks helperRasmus Villemoes
This does the same thing, but is more efficient in case st_nlinks is (already) 1. Depends on bitbake commit 7ae93cf40ab91965147055100432961436bce46c . Signed-off-by: Rasmus Villemoes <rv@rasmusvillemoes.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23oeqa/runtime/ldd: Clean up testRichard Purdie
* Merge the two tests together as having them separate is pointless * Test that ldd runs correctly * Add in a dependency on the "ldd" package being installed instead of the sdk tools feature Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23oeqa/runner: Print any errors/failures earlyRichard Purdie
Its a pain to have to wait until oe-selftest finishes to see the failures for example. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23oeqa/runner: Use the proper logger functions instead of print()Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23oeqa/context: Only set buffer mode for non-concurrent testsRichard Purdie
Periodically we'd see: NOTE: core-image-sato-1.0-r0 do_testsdk: ====================================================================== NOTE: core-image-sato-1.0-r0 do_testsdk: ERROR: broken-runner NOTE: core-image-sato-1.0-r0 do_testsdk: ---------------------------------------------------------------------- NOTE: core-image-sato-1.0-r0 do_testsdk: testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/lib/oeqa/core/utils/concurrencytest.py", line 122, in _run_test test.run(process_result) File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 1194, in run protocol = TestProtocolServer(result, self._passthrough, self._forward) File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 514, in __init__ stream = stream.buffer AttributeError: '_io.StringIO' object has no attribute 'buffer' which seems to occur if a result arrives before all the runner threads have started. The runner's result handling changes sys.stdout to a buffer temporarily which can be seen in other threads and it can sometimes fail. Since the tests are running in a separate process we don't need this buffer handling in the concurrent case so only set when not parallelising. The concurrent class handle setting buffer mode internally. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23oeqa/concurrencytest: Ensure subunit streams are flushed at exitRichard Purdie
Without this, error output such as that in the teardown can be lost and processes may recieve signals they're not expecting causing other strange errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-21assimp.py: fix AttributeError in tearDownClassChen Qi
When running this test case, we will see the following error. AttributeError: type object 'BuildAssimp' has no attribute 'project' assimp.py test case does not make use of SDKBuildProject, so remove the import statement and the tearDownClass. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-19kernel-devsrc: restructure for out of tree (and on target) module buildsBruce Ashfield
The existing kernel-devsrc package starts with a full copy of the kernel source and then starts to strip out elements that are not required. This results in extra time (I/O) and extra space being taken up in the final package. The main purpose of the kernel-devsrc package has been to build modules against the running kernel, not to include a full copy of the source code for re-building the kernel. The end result was a 600M kernel-devsrc package. This restructuring of the package uses an approach similar to other distros, where the kernel-devsrc package is for building against the running kernel and uses a curated set of copied infrastructure, versus a mass copy of the entire kernel. The differences in this approach versus other is largely due to the architecture support and the split build/source directory of the kernel. The result is a kernel-devsrc package of about 10M, which is capable of running "make scripts" and compiling kernel modules against the running kernel. Along with the changes to the copying of the infrascture, we also have the following changes: - a better/more explicit listing of dependencies for on-target builds of "make scripts" or "make modules_prepare" - The kernel source is installed into /lib/modules/<version>/build and a symlink created from /usr/src/kernel to the new location. This aligns with the standard location for module support code - There is also a symlink from /lib/modules/<version>/source -> build to reserve a spot for a new package that is simply the kernel source. That package is not part of this update. Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16runtime selftest: limit kernel hw bp archesHongzhi.Song
1. So far, only qemux86[-64] support hw breakpoint, no matter whether or not with kvm. qemuppc: The oe-core configuration uses a PPC G4 system as the default cpu but qemu doesn't simulate the hw breakpoint register for G4. qemuarm: The arch more than v7 supports hw breakpoint, however arm use v5 as default. qemuarm64: We temporarily drop qemuarm64 for the moment. Normally it will print debug info once, but endlessly when we trigger the break point. Now it is hard to located the issue, but we will confirm it later. qemumips*: Kernel dosen't support hw bp for mips. 2. Syslog maybe not started, so we use dmesg to confirm. 3. Running 'ls' to trigger the hardware breakpoint test. Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16runtime selftest: test_trace_events_sample of ksample.pyHongzhi.Song
The debug info can not be written to trace buffer immediately by thread. So we should sleep some seconds. Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16oe.package_manager: support loading intercepts from multiple pathsChristopher Larson
- if POSTINST_INTERCEPTS is set, use the listed intercept files, or - if POSTINST_INTERCEPTS_PATH is set, load from the listed paths, or - if POSTINST_INTERCEPTS_DIR is set, load from it (for compatibility), or - load from ${COREBASE}/meta/postinst-intercepts Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager: Add definitions for riscv machinesRichard Purdie
Add definitions to the locale table for riscv architecture. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15selftest/package: Improve test to cover sparseness and hardlinking from sstateRichard Purdie
The sparseness test was sometimes working and sometimes failing depending on whether sstate was valid. This adds an explict test of sstate to the test for both hardlinking and sparseness. Tweak the test name to cover the fact its tests sparseness too. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15sstate/lib.oe.path: Ensure file sparseness is preservedRichard Purdie
Files when restored from sstate were missing their sparseness. Fix up various functions to preserve this and make things more deterministic. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15selftest: Replace bitbake -p with bitbake -eRichard Purdie
Parsing all the recipes is annoying when trying to re-execute oe-selftest and also unnecessary as its really just a sanity check. When the tests were originally being developed the guard was useful but less so now. Replace it with bitbake -e which is fast and checks the basic configuration is valid. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager/sdk: Use filtered copies of the deploy ipk/deb directoriesRichard Purdie
Similar to rpm, use copies of the ipk/deb directories for rootfs construction. This means the image creation code can no longer "see" recipes wich aren't in its dependency chain which is good for a variety of reasons including determinism, incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking performance. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15package_manager: Remove rpm specific pieces of create_packages_dir()Richard Purdie
This function is generic, tweak the variable names and move out the rpm specific directory name to make it truly generic and reusable for deb/ipk. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15image: Add locale archive optimisationRichard Purdie
Refactor the locale archive function from the SDK to also make it work during general image creation. This reduces the size of the locales from 900MB to 220MB in core-image-lsb-sdk. The exception handling around subprocess was dropped as the standard subprocess exception printing is better handled than the catchall exception. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15selftest/package: Add test to ensure sparse files are preservedRichard Purdie
Add a new element to the hardlink test to check we also preseve file sparseness during the packing process. This should ensure we don't regress this issue again. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-14parselogs.py: output correct log locationChen Qi
The log entry in results is altered to remove 'target_logs'. This causes wrong log location in output. e.g. AssertionError: 1 != 0 : Log: /path/to/image/1.0-r0/postinstall.log But when user wants to check the log, the user will find the log is not present. The actual log file is /path/to/image/1.0-r0/target_logs/postinstall.log. So fix to use the correct log location. Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-14oeqa/esdk/devtool: Drop OETestDepends usageRichard Purdie
OETestDepends doesn't work with parallelism and in this case we don't really need this dependency, it would just short out some tests quickly in the rare case the esdk environment was broken. Currently this is masking tests which is a much worse problem and we can't make OETestDepends work reliably with parallelism so drop the dependencies. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-10lib/oe: Fix collections ABCs DeprecationWarning in Python 3.7+Khem Raj
- Prefer collections.abc (new in Python 3.3) over collections for abstract base classes - In Python 3.8, the abstract base classes in collections.abc will no longer be exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes." - https://docs.python.org/3.7/whatsnew/3.7.html#deprecated - see https://github.com/python/cpython/commit/c66f9f8d3909f588c251957d499599a1680e2320 Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-10oeqa/sdk/buildgalculator: check for nativesdk-gettext-devRoss Burton
We don't need target gettext to build, but nativesdk-gettext-dev (for nls.m4). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-09package: skip strip on signed kernel modulesfoocampo
Executing strip action on kernel modules removes the signature. Is not possible to strip and keep the signature, therefore avoid strip signed kernel modules. Signed-off-by: Omar Ocampo <omar.ocampo.coronado@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-08context.py: Do not mask exceptions needlessly.Paulo Neves
There were a lot of assumptions in the controller import code of context.py which were not true anymore. These assumptions reflected themselves by catching exceptions and re-raising them with unhelpful error messages in other parts of the code. This commit does not fix the classes controller classes that became broken after the refactor but at least it allows for the exceptions to be thrown where the imports fail, thus actually showing what exactly went wrong with the import. An example of such an improvement is that before if the controller class failed during it's init contructor the controller would just be skipped and the task would just complain it could not find the controller. Now for example, if there is a NamerError due to a variable not being declared, the user will get that report. Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-08-08oe_syslog.py: fix for syslog-ngChen Qi
When using syslog-ng as the syslog provider, oe_syslog test case fails because it cannot find the syslog daemon. This is because it greps for 'syslogd' but syslog-ng's daemon is 'syslog-ng'. So fix it to check both 'syslogd' and 'syslog-ng'. Also, when the test case fails, what I get is: | AssertionError: 1 != 0 : No syslogd process; ps output: <empty here> This does not help user. The output is actually from the 'PS | GREP' command. And when the 'PS | GREP' command fails, the output is always empty. So also fix this problem. After the change, it looks like: | AssertionError: False is not true : No syslog daemon process; ps output: | PID USER VSZ STAT COMMAND | 1 root 16476 S {systemd} /sbin/init | 2 root 0 SW [kthreadd] | 3 root 0 IW [kworker/0:0] ... Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07selftest/package: Fix hardlink test when using sstateRichard Purdie
The build target won't cause the package task to run if things were already built and in sstate. Ensure we run the package task explicitly to ensure the test works as intended. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07oeqa/sdk: add test that CMake worksRoss Burton
Add a new SDK testcase that builds assimp, a project that uses cmake. Using TARGET_ARCH and TARGET_OS which is now exported into the environment, check that the generated binaries match the ELF headers we expect. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-01multiconfig: Enable multiconfig dependencies on oe-coreAlejandro Enedino Hernandez Samaniego
This patch enables multiconfig dependencies (mcdepends) to be used on recipes using the following format: task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on" For the sake of simplicity consider the following example: Assuming we have set up multiconfig builds, one for qemux86 and one for qemuarm, named x86 and arm respectively. Adding the following line to an image recipe (core-image-sato): do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs" Would state that core-image-sato:do_image from x86 will depend on core-image-minimal:do_rootfs from arm so it can be executed. This patch makes modifications to bitbake.conf to enable mcdepends, and to sstatesig and staging.bbclass to avoid conflicts between packages from different multiconfigs. [YOCTO #10681] Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31oe-selftest: devtool: avoid parallel races by using temporary copy of corePaul Eggleton
Some of the devtool tests make changes to files under meta/ - legitimately since we want these tests to be working with real recipes and associated files. Unfortunately with the new oe-selftest parallelisation this can break other tests if files go missing at the wrong time (among other scenarios). To avoid this issue, simply take a copy of the core repository and use that for these tests. (We copy the entire repository since changing the path of meta/ influences COREBASE and thus we need to have things like scripts/ alongside as well). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-07-31systemd.py: increase default bus timeoutChen Qi
Use SYSTEMD_BUS_TIMEOUT to set default timeout to 240s to avoid timeout problem on slow qemu machines. Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
2018-07-31parselogs.py: ignore network interface name changing failureChen Qi
The following error from systemd-udevd is not harmful. It's just because our qemu targets are using eth0. Error changing net interface name 'eth0' to 'enp0s3': Device or resource busy Note that systemd is using a different network interface naming scheme from traditional ethX naming scheme. To make this error message go away, we could symlink /etc/udev/rules.d/80-net-setup-link.rules to /dev/null to recover traditional naming scheme. But I'm not sure if this will cause regression in user experience for systemd users of OE. So just ignore this error message so that parselogs.py test case does not fail. Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
2018-07-31oe_syslog.py: fix settings in OEHasPackage decoratorChen Qi
1. For test_syslog_running, we should not restrict it to run for only busybox-syslog and sysklogd. So extend it to all syslog providers in oe-core and meta-openembedded. 2. For test_syslog_startup_config, fix to make it depend on the existence of busybox-syslog. The previous condition "!sysklogd && busybox" is incorrect, because busybox may be compiled without syslog support, and sysklogd and busybox-syslog conflict with each other. Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
2018-07-31oeqa/selftest: Add package hardlink testRichard Purdie
We keep breaking the preservation of hardlinks during the packaging process. Add a selftest which tests this to try and prevent this breaking again. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30oeqa/selftest: check if rm_work is enabledAnuj Mittal
rm_work if enabled leads to some tests failing that rely on artifacts being present. Check if rm_work.bbclass is included and show an error and exit if it is. Fixes [YOCTO #12694] Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-30oeqa/runtime/scanelf: removeRoss Burton
These tests are intended to search for bad RPATHs and text relocations, but we do these tests at buildtime and as pax-utils is never installed in any default images the tests are never executed. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26selftest/wic: Fix/cleanup class setup codeRichard Purdie
The code was using Wic instead of WicTestCase for image_is_ready and native_sysroot which was confusing some tests. Fix image_is_ready. For native_sysroot, convert the only user to just run the call itself as there is no point in executing this for all the other tests which don't use the value. Whilst here, move _get_image_env_path to the base class alongside its parent variabe and fix it to use WicTestCase. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26oeqa/loader: Ensure invalid test names don't trigger a tracebackRichard Purdie
oe-selftest -r <invalid test name> currently triggers a traceback. Ensure this doesn't happen and the user gets a sensible error message. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26qemurunner.py: fix is_alive() to avoid confusing with recycled pidRobert Yang
[YOCTO #12493] Fixed: - qemu started with pid 10000 - qemu exited unexpectedly - The pid 10000 is re-used by another different process. The is_alive() returned True in such a case because both qemu_pidfile and /proc/10000 exist, but it's another process, this patch fixed the problem. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26lib/oe: split out machine to ELF data dictionaryRoss Burton
2018-07-26oeqa/sdk/sdkbuildproject: use os.path.joinRoss Burton
Instead of mushing two paths together and hoping that the slashes line up correctly, use os.path.join. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26oeqa: rationalise Perl testsRoss Burton
As with the Python test, this can be both better and faster. No need to copy a file, just run a one-liner. Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-26logging: use warning instead warnChen Qi
The warn method is deprecated. We should use the documented warning instead. Quoting from the python's official doc: """ Note: There is an obsolete method warn which is functionally identical to warning. As warn is deprecated, please do not use it - use warning instead. """ Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-07-25selftest/bbtests: Add test for bitbake execution outside the build directoryRichard Purdie
Also fix a related test's error message to match what it does. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25selftest/bblayers: Place the test layer directory in builddirRichard Purdie
Placing the layer in meta means the directory is in an unclean state which may influence other tests. Use our build directory instead since we 'own' that. This helps keep oe-selftest parallelisation clean. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25selftest/wic: clean up runCmd error code handlingRichard Purdie
runCmd handles checking exit 0 internally unless specified otherwise and its error messages are much more useful including the failed command output. Clean up the unneeded 0 exit code checks in the wic tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>