aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib
AgeCommit message (Collapse)Author
2016-02-11oe/gpg_sign: fix incorrect variable nameMarkus Lehtonen
Prevents crash in signing if GPG_PATH is defined. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-11oeqa/prservice: Fix whitespace problemRichard Purdie
Recent bitbake changes mean the whitespace formatting around the way functions are defined, changed which broke the rather ugly construct this test uses to append to a python function. This really needs to be rewritten and improved but fix the whitespace so at least the tests work again and other regressions don't creep in. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10oeqa: Improve test failure messagesRichard Purdie
When the test fails, print the incorrect values so aid debugging. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10oeqa/sdkext/devtool.py: Add location test to ensure that devtool is the eSDK ↵Aníbal Limón
one. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10oeqa/sdkext: Add devtool basic tests for eSDK.Aníbal Limón
Add simple myapp application is a C app that prints hello world and exit. Add devtool test for that this app to the workspace, build and reset it. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10oeqa/oetest: Fix compatibility SDK tests using eSDK.Aníbal Limón
When run SDK tests over eSDK we need to use SDKExtTestContext instead of SDKTestContext because if we use SDKTestContext search for SDK manifest and depends on the SDK manifest generation so populate_sdk needs to be executed. Adds a compatibility mode flag to SDKExtTestContext for search tests over sdk module instead of sdkext module and change testsdk calls to comply with this new param. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10classes/populate_sdk_ext: Add SDK_EXT_TARGET_MANIFEST and SDK_EXT_HOST_MANIFESTAníbal Limón
Extensible SDK needs to point to the correct manifest so add SDK_EXT_TARGET_MANIFEST and SDK_EXT_HOST_MANIFEST variables. oeqa/oetest.py: Fix SDKExtTestContext for load the correct manifests. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10testsdkext: Add skeleton for support Extensible SDK tests.Aníbal Limón
oeqa/sdkext: Add module and __init__.py will contain eSDK tests. classes/testsdk: Add support for run eSDK tests. oeqa/oetest: Create oeSDKExtTest for now only inherit oeSDKTest, modified SDKExtTestContext now inherit SDKTestContext and set sdkext filesdir for store data fixtures. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10oeqa/oetest: oeSDKTest when run a command redirect env output to nullAníbal Limón
Some tests are failing because the eSDK env load script shows a help message on the output so redirect this to /dev/null. There was a discussion with Paul Eggleton [1], he don't want to have env variables to change the behaviour of what the env script needs to print so redirect the output in oeSDK.run() method. I didn't agree because it can hides another messages like error ones and with the variable you can control what prints or not. [1] http://lists.openembedded.org/pipermail/openembedded-core/2016-February/116744.html Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10oetest.py/TestContext: Move loadTests and runTests inside it.Aníbal Limón
Method's for loadTests and runTests make sense to define inside TestContext because it can be different around Image, SDK, SDKExt. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10testimage/testsdk: Move get test suites routine inside TestContext.Aníbal Limón
In order to provide better abstraction move functions to get the test suite inside the TestContext. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10testimage/testsdk: Modularize TestContext.Aníbal Limón
Move anonymous duplicated class TestContext from testimage/testsdk to oeqa/oetest now we have two new classes ImageTestContext and SDKTestContext with common code in TestContext class. Signed-off-by: Aníbal Limón <limon.anibal@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10get_test_suites: Add sdkext type for load test suites.Aníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10testimage: Modularize helper functions for get test lists.Aníbal Limón
Test lists functions can be used in other parts so modularize it and move to oeqa/oetest.py library. Testimage class was updated to meet the new sign of the functions. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-10oeqa: setup bitbake logger after tinfoil.shutdownEd Bartosh
Bitbake logger stops working after tinfoil.shutdown removes console handler from it. This makes bb.{error,warn,note,critical} messages disappear from the console. Adding console handler to bitbake logger again should fix this issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-07sstatetests: Fix after change to sstate populate_lic SWSPECRichard Purdie
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9fd214d2e7a931cfb46f40cb76d49aeb07af612e changed SSTATE_SWSPEC, correctly however it did mean that some sstate files now appear where the tests don't expect them. Filter the test results to ignore the correct behaviour. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06package_manager.py: fix python indentation bug (opkg)Ioan-Adrian Ratiu
This if branch is causing the following error during do_rootfs: "Exception: UnboundLocalError: local variable 'cfg_file' referenced before assignment". This happends because the cfg_file variable is defined at a deeper nesting level (just above the if branch). Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06copy_buildsystem.py: Pass the nativelsb argument to gen-lockedsig-cacheRandy Witt
If the nativelsb argument is not used, then create_locked_sstate_cache() can get collisions when moving the files from the input_sstate_cache to the output_sstate_cache. The specific case where this was encountered was when a "universal" nativelsb directory already existed in the input_sstate_cache. Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-04copy_buildsystem: add ability to exclude layersChen Qi
In some cases, we may have some kind of download layers in BBLAYERS, so that we can set BB_NO_NETWORK to "1". This results in extremely large extensible SDK. And we actually don't need these download layers in the SDK. Add a new variable, SDK_LAYERS_EXCLUDE, to enable users to explicitly exclude some layers when generating the extensible SDK. [YOCTO #8878] Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-02-04classes/lib: Add expand parameter to getVarFlagRichard Purdie
This sets the scene for removing the default False for expansion from getVarFlag. This would later allow True to become the default. On the most part this is an automatic translation with: sed -e 's:\(\.getVarFlag([^,()]*, [^,()]*\)):\1, True):g' -i `grep -ril getVar *` In this case, the default was False, but True was used since in most cases here expansion would be expected. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-31lib/oe/package_manager: remove package feed listsStefan Agner
Remove opkg package feed lists after generating the rootfs. The lists have been generated by the local feed but are no longer necessary after image generation. This still leaves the package management fully usable (and hence is different from omitting package-management from IMAGE_FEATURES). Signed-off-by: Stefan Agner <stefan.agner@toradex.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-29lib/oe/patch: Make GitApplyTree._applypatch() support read-only .git/hooksPeter Kjellerstedt
Rather than modifying files in .git/hooks, which can be read-only (e.g., if it is a link to a directory in /usr/share), move away the entire .git/hooks directory temporarily. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-29oeqa/runtime/rpm: be more verbose if test_rpm_query_nonroot failsRoss Burton
If some parts of this test fail put the failing output in the logs to help debugging. Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-26meta/lib: new module for handling GPG signingMarkus Lehtonen
Add a new Python module (oe.gpg_sign) for handling GPG signing operations, i.e. currently package and package feed signing. The purpose is to be able to more easily support various signing backends and to be able to centralise signing functionality into one place (e.g. package signing and sstate signing). Currently, only local signing with gpg is implemented. [YOCTO #8755] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-23lib/oe/sdk: Partially revert "sdk.py: fix conflicts of packages"Richard Purdie
OE-Core rev: f2b64f725803ad8be7c2876c531e057a4fe5ca7c (poky 1362986886cc96c8cc11fb60795f729b41770414) unintentionally broke opkg/dpkg multilib support within the SDK by making things not honour self.install_order. This reinstates that code for opkg/dpkg but not rpm where the original problem was. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22lib/oe/package_manager: prevent testing an undefined variableJoshua Lock
This prevents an error in do_populate_sdk when building buildtools-tarball with ipk as the package manager: Exception: UnboundLocalError: local variable 'pkg' referenced before assignment Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22gen-lockedsig-cache: copy correct native sstate into ext SDKPaul Eggleton
When constructing the sstate-cache directory for the extensible SDK, we were copying in any matching native sstate packages, and as the signature doesn't actually change when the distro changes (since NATIVELSBSTRING is just a path separator for the artifacts and is not part of the signature) we ended up copying duplicated packages when the distro changed e.g. upon host distro upgrade. Only search in the NATIVELSBSTRING-named subdirectory for native packages and the issue goes away. Fixes [YOCTO #8885]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22devtool: build: ensure pkgdata is written outPaul Eggleton
When you run devtool build, you need to have the pkgdata written out at the end, so that if what you're adding is a library and the next thing you add is something that depends on that library, the necessary information to map the dependency back to the recipe is present. In practical terms all this means is we need do_packagedata to run in addition to do_populate_sysroot. This does mean that do_package needs to run which wasn't running before, and that means that the few package QA tests that run within do_package such as installed-vs-shipped will now be run. This may be a bit bothersome, and prompted a fix for one of our oe-selftest tests as a result, but I don't see an easy way around it. Ultimately if you care about using the recipe in an image you'll need to fix any such errors anyway. Fixes [YOCTO #8887]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22classes/populate_sdk_ext: add option to bring in pkgdata for worldPaul Eggleton
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include pkgdata for all recipes in the world target. There are a couple of uses for this: 1) If you use "devtool add" to add a recipe that builds something which depends on anything in world, the dependency can then be correctly mapped to the recipe providing it and that recipe can be added to DEPENDS, since we have the pkg-config and shared library dependency data within pkgdata. 2) You'll be able to search for these recipes and any files they package for the target with "devtool search" since that also uses pkgdata This of course assumes you've tailored world through EXCLUDE_FROM_WORLD to only include recipes you'd want built in your distro, but I think that's a reasonable assumption; failing that there is a WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes you don't want. Note that this patch relies on functionality implemented in a recent BitBake patch and will not work without it. Implements [YOCTO #8600]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-22oeqa/selftest/signing: use temporary rpmdbMarkus Lehtonen
Use temporary rpmdb when importing gpg public key and checking rpm signature. This patch should fix a problem where test_signing_packages() sometimes fails with "pmdb: BDB1540 configured environment flags incompatible with existing environment" Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-20lib/oe/package_manager.py: Remove list() from PkgsList classMariano Lopez
Now that the method list() is not used anymore, remove it. [YOCTO #7427] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-20lib/oe/rootfs: Use list_pkgs() instead of list()Mariano Lopez
This patch changes the use list_pkgs() instead of list() from class RpmPkgsList. The change is in two functions, image_list_installed_packages from rootfs.py and sdk_list_installed_packages from sdk.py. With this change the functions calling the functions listed above, must format the output as they required. The formatting can be done using format_pkg_list() from oe.utils. The classes calling the afected functions are changed too with this patch, to keep the same functionality using the new data structure. [YOCTO #7427] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-20lib/oe/utils: Add function format_pkg_list()Mariano Lopez
The class PkgsList returns a dictionary with all the installed packages, because the data structure is a dictionary there is needed to format the data in order to write to a file. The function format_pkg_list returns a formated sting with all packages installed. The output will depend on the requested format when calling the function. [YOCTO #7427] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-20lib/oe/package_manager: Add list_pkgs() to PkgsList classMariano Lopez
Currently the class PkgList returns a formated string of the installed packages. It would be more clean to pass a standard data structure to the callers instead to format the output inside PkgsList class. This patch adds list_pkgs() method to PkgsList class to get the all the information for installed packages and return a dictionary with the info. [YOCTO #7427] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-19oeqa/runtime/parselogs: use -F to search fixed strings for grepYi Zhao
The grep output is not correct if the log message contains special characters. Use -F which tells grep to search for fixed strings, not expressions and pass lists instead of strings to subprocess. Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-19oe-selftest: devtool: fix test_devtool_add_library if python was built firstPaul Eggleton
If Python is in the sysroot then this test can fail due to some brokenness in libftdi's CMakeLists.txt file for its python bindings. Just disable it in order to have the test work more reliably. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19recipetool: create: add basic support for extracting dependencies from cmakePaul Eggleton
Add support for extracting dependencies from CMakeLists.txt. There's still a bunch of things missing that are outside the scope of OE-Core and we still lack a proper extension mechanism, but this is a good start. This also adds an oe-selftest test to exercise the new code a bit. Implements [YOCTO #7635]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19recipetool: create: improve extraction of pkg-config / lib depsPaul Eggleton
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict and thus we were skipping some macros. * Add support for PKG_CHECK_EXISTS * Avoid duplicates in warning on missing pkg-config dependencies * Ignore dependency on musl (since this may come up if it's the selected C library) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19selftest/buildhistory: Improve test to remove sources of errorRichard Purdie
When we add buildhistory, we expect do_rootfs to rerun, but depending on IMAGE_FSTYPES, the number of tasks which would execute after do_rootfs varies (e.g. live would add do_bootimg and we recently added do_image). Therefore limit the test to -c rootfs and then we're clear that only one task should re-run. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19image: Fix wic environment issuesRichard Purdie
The wic environment function needs to run after the rootfs size is setup. We move this code to a specific task, and depend on that task from the wic images and other places its needed. This fixes: ====================================================================== FAIL: test_image_env (oeqa.selftest.wic.Wic) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/utils/decorators.py", line 106, in wrapped_f return func(*args, **kwargs) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/wic.py", line 218, in test_image_env self.assertTrue(var in content, "%s is not in .env file" % var) AssertionError: False is not true : ROOTFS_SIZE is not in .env file Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19insane: Start to clean up do_configure_qa codeRichard Purdie
This cleans up the do_configure_qa code so that the exit status from package_qa_handle_error is handled correctly. It also converts package_qa_check_license to use our standard QA check configuration interface through package_qa_handle_error rather than a mix of return values, bb.fatal and bb.build.FuncFailed. Due to the merging of multiple lines into one message, we need to tweak a QA test to account for this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11DpkgRootfs: Fix logcheck_error false-positive when use multilibAníbal Limón
Rootfs with dpkg was failing due to false-positive in logcheck_error because current logic of DpkgPM handles missing dependencies failure using apt-get -f install [1][2]. This support was broken due to addition of logcheck and don't take into account dpkgpm cases, in order to fix add an attr for specify expected errors regex'es by package manager. [1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659 [2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038 Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Matt Madison <matt@madison.systems> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-11package_manager.py: fixes for multilib deb packaging buildsMatt Madison
* tmp/deploy/deb subdirectories do not get hyphens replaced with underscores, so don't do that translation when building the sources list. * Fix MULTILIB_VARIANTS handling to be more general and work for all architectures * Also include a fix for a warning generated by apt due to missing apt/preferences.d directory. Signed-off-by: Matt Madison <matt@madison.systems> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-11oeqa/runtime/logrotate: fix hardcoded root directoryYi Zhao
Use $HOME instead of /home/root in case user changes the default root directory by ROOT_HOME Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-11oeqa/runtime/smart: fix hardcoded root directoryYi Zhao
Use $HOME instead of /home/root in case user changes the default root directory by ROOT_HOME Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-01-11populate_sdk_ext: Change to include siginfo and non sstate task sigsRichard Purdie
Right now, the locked task hashes list for the extensible SDK locks down only the sstate tasks. Whilst asthetically pleasing, this gives two problems: * Half the task are left floating meaning checksum mismatches are a pain to debug * The later code which copies relavent data files out the sstate cache can't use any of this data. This patch modifies things so all the checksums are listed in the locked file. An exclusion of tasks probably makes more sense for the library function rather than an allowed list. The only sstate task being deliberaly excluded here was do_package so add in a function to explictly exclude those sstate object files. The net result of this that siginfo files for all tasks are included in the SDK, which means commands like "bitbake -S printdiff" now function. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11image: Create separate tasks for rootfs constructionRichard Purdie
This patch splits the code in lib/oe/image into separate tasks, one per image type. This removes the need for the simple task graph code and defers to the bitbake task management code to handle this instead. This is a good step forward in splitting up the monolithic code and starting to make it more accessible to people. It should also make it easier for people to hook in other tasks and processes into the rootfs code. Incidentally, the reason this code was all combined originally was due to limitations of fakeroot where if you exited the session, you lost permissions data. With pseudo this constraint was removed. We did start to rework the rootfs/image code previously and got so far with untangling it however we did prioritise some performance tweaks over splitting into separate tasks and in hindsight, this was a mistake and should have been done the other way around. That work was suspended due to changes in the people working on the project but this split has always been intended, now is the time to finish it IMO. There were some side effects of doing this: * The symlink for the manifest moves to the rootfs-postcommands class and into the manifest function. * There is no seperate "symlink removal" and "symlink creation", they are merged * The date/time stamps of the manifest and the built images can now be different since the tasks can be run separately and the datetime stamp will then be different between do_rootfs and the do_image_* tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11image: Move pre/post process commands to bbclassRichard Purdie
As the next step in splitting up do_image, move the pre and post processing commands to separate tasks. This also creates the do_image_complete task which acts as the end marker task for image generation. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11sstatesig: Handle special case of gcc-source shared-workdir for printdiffRichard Purdie
Often, bitbake -S printdiff would show that there was a checksum not found which would turn out to be from gcc-source. This is due to it being a shared-workdir recipe. For now, hardcode the special case into the sstatesig code to stop people (including me) puzzling over this. If/as/when we add any other shared workdir recipes, we'll need to rethink this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-07x11vnc: remove all references to moved packageIoan-Adrian Ratiu
Together with the move to meta-oe, all references to x11vnc should be removed from oe-core. There are three of these: a distro alias, a packagegroup rdepends and a runtime test. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Ross Burton <ross.burton@intel.com>