aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2016-04-05scripts/oe-selftest: avoid the creation of coverage file when coverage not ↵Humberto Ibarra
installed Coverage subprocessing file is being created even when coverage is not installed, which causes errors of "module not found" to be send to the oe-selftest output. This patch adds indent to the block of code creating this coverage file, so it can only be executed when coverage is actually installed. [Yocto #9334] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-05scripts/oe-selftest: remove coverage file if any coverage option is givenHumberto Ibarra
Coverage temporal file for sub-processing is being removed only when the --coverage option was found. This is wrong since the file is created when any coverage option (source, include or omit) is given, even if --coverage is not one of them. This patch makes sure to remove the file if any coverage option was given. Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-05scripts/oe-selftest: remove unneeded coverage warningHumberto Ibarra
There is a message that warns the user about enabling subprocessing for coverage to work. After the fix for Yocto #8930, this task is done automatically, so the warning is not needed anymore. Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-05scripts/oe-buildenv-internal: Fix regression in BB_ENV_EXTRAWHITE settingOtavio Salvador
The commit OE-Core:ada4639 (oe-buildenv-internal: simplify derivation of BB_ENV_EXTRAWHITE) changed the format of BB_ENV_EXTRAWHITE variable to: ,---- | BB_ENV_EXTRAWHITE='ALL_PROXY | BB_NO_NETWORK | BB_NUMBER_THREADS | ... | ' `---- Instead of: ,---- | BB_ENV_EXTRAWHITE='ALL_PROXY BB_NO_NETWORK BB_NUMBER_THREADS...' `---- The old format allow for external script parsing easier and there is no need to change the format as it has no benefit from usage perspective. Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-04classes/buildhistory: optimise getting package size listPaul Eggleton
Invoking oe-pkgdata-util in turn for every package in the list was slow with a large image. Modify oe-pkgdata-util's read-value command to take an option to read the list of packages from a file, as well as prefix the value with the package name; we can then use this to get all of the package sizes at once. This reduces the time to gather this information from minutes to just a second or two. Fixes [YOCTO #7339]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-03wic/isoimage-isohybrid.py: change cpio generated uid&gid to rootIoan-Adrian Ratiu
By default cpio preserves the uid&guid's of the original user which leads to host contamination and boot failures because commands like mount from initramfs expect to be run by root and the original host user might not even exist on the target. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-03wic/isoimage-isohybrid.py: use glob to find initramfs locationIoan-Adrian Ratiu
Some filenames can omit 'initramfs', or use other names. This makes detection more flexible by using only the image name, machine arch and image type in a glob wildcard. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31relocate_sdk: additional error checksJuro Bystricky
When installing SDK in a non-default location and the path length of the SDK install location is longer than the path length of the default SDK location, relocation of .ldsochache section will overwrite file location outside of the .ldsocache section size. In addition, additional checks were added to ensure that any path in sections .gccrelocprefix and .ldsochache will not exceed the space allocated for it within the file, which would also result in file corruption. [YOCTO #9268] Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31devtool: modify: call shutdown on tinfoil when donePaul Eggleton
Strictly speaking we ought to explicitly shut down a tinfoil instance when we're done with it. This doesn't affect modify's operation but is important if you want to be able to call into modify() from another plugin (though anyone doing so should be advised that the function is by no means a stable API and is subject to change in future releases). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31oe-publish-sdk: exclude sstate-cache if publishing minimal SDKPaul Eggleton
If SDK_EXT_TYPE is set to "minimal" then the SDK won't contain many sstate artifacts, and you're required to set up an sstate mirror in this case anyway so there's no point publishing the "stub" sstate-cache directory from within the SDK since it won't be useful for update purposes and may be confused with the real sstate-cache. There is however a possibility that people might publish the real sstate-cache directory under the same output directory provided to oe-publish-sdk, thus deleting it after extracting (as we were doing with other files we wanted to clean up at the end) would be problematic, besides which extracting it and then deleting it is wasteful. Thus, introduce a "-p" command line option to the SDK installer that we can use to tell tar not to extract the items we don't want when publishing. This has the added benefit of mostly keeping references to these in the place they belong i.e. in populate_sdk_ext.bbclass. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31oe-publish-sdk: prevent specifying a directory for the SDK argumentPaul Eggleton
The SDK argument is expected to be an installer .sh file; if a directory is specified we can get an ugly failure later on; best to check up front. Fixes [YOCTO #9065]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31scripts, lib: Don't limit traceback lengths to arbitrary valuesRichard Purdie
There appears to have been a lot of copy and pasting of the code which prints tracebacks upon failure and limits the stack trace to 5 entries. This obscures the real error and is very confusing to the user it look me an age to work out why some tracebacks weren't useful. This patch removes the limit, making tracebacks much more useful for debugging. [YOCTO #9230] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-29runqemu: fix for isoRobert Yang
It should be the similar type as hddimg, rather than ext234 or btrfs. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28scripts:/oe-selftest: Use timestamp instead of test names in coverage data fileHumberto Ibarra
This fixes the problem by changing the name to the coverage data file, using the timestamp as an identifier. The name for the coverage data file is constructed based on the tests ran; this has created a couple of issues so far, affecting coverage report. If --run-tests-by option is given, the data file name won't have any identifier, causing following runs to overwrite themselves. On the other hand, if too many tests are given, the file name exceeds linux limits and fails to store the coverage data all together. [Yocto #9253] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28relocate_sdk: fixed .gccrelocprefix section handlingWenlin Kang
When fixing paths for .gccrelocprefix section, it will corrupt the next entry during updating the current one if "new_prefix" length is more than "DEFAULT_INSTALL_DIR", this problem is obvious on the code, but it's only found when install sdk onto a net file system. Signed-off-by: Wenlin Kang <wenlin.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28gio-module-cache: Add class for Gio modulesJussi Kukkonen
The new class uses gio-querymodules tool on postinst and postrm. This regenerates the module cache which is useful to avoid loading modules that are not needed at runtime: If a Gio module is not listed in the cache file it will always get loaded. * Add a postinst-intercept 'gio-module-cache': it runs gio-querymodules using qemuwrapper. This is required because the tool actually loads the modules to generate the cache. * Add a gio-module-cache class that adds postinstall and postrm scripts. In the sysroot population case use the new intercept. * Inherit the new class in glib-2.0, glib-networking and gconf. Fixes [YOCTO #9241]. Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28oe-git-proxy: support username / password in http proxyAndré Draszik
We also make sure to correctly support usernames that contain spaces. For simplicity sed + regex has been replaced with shell parameter expansion, which works in both, bash and dash. Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28oe-git-proxy: also check all_proxy and http_proxy env variablesAndré Draszik
Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28wic: fix type of no-table optionEd Bartosh
Type of --no-table option was incorrectly set in new wks parser. It causes parser to require argument for this option, which makes wic to fail with wks files that use --no-table: Error: argument --no-table: expected one argument Changed action parameter to 'store_true' to fix the issue. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-26oe-buildenv-internal: simplify derivation of BB_ENV_EXTRAWHITEAndre McCurdy
Use 'sort --unique' to combine the two lists and remove duplicates. Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23runqemu-internal: split the code into functionsRobert Yang
Use config_<machine> to split the code into separate functions, so that different machines won't affect each other, and they will have a better structure. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu-internal: cleanup unsed codeRobert Yang
* remove akita and spitz related code They are not supported by runqemu anymore: $ runqemu spitz Error: unable to classify arg [spitz] So remove related code. * Remove checking of 256M for qemuarm, qemu can check it, for example: $ runqemu qemuarm qemuparams="-m 1024" [snip] qemu: Too much memory for this machine: 1024 MB, maximum 256 MB [snip] Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu: simplify checking for iso and ramfsRobert Yang
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu: add support for qcow2 and vdiRobert Yang
[YOCTO #9168] Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu: remove ISO and RAMFS from help textRobert Yang
They don't work, and the script can check the type correctly. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu: simplify the checking for vm imagesRobert Yang
* So that we can add more image support easliy. * I think that wic should be vm images. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23runqemu: fix ROOTFS for vmdkRobert Yang
* Make it can boot scsi and virtio block drive such as root=/dev/sdX and /dev/vdX. * Drop VM from help info, id doesn't work, and the script can check whether it is a vm disk or not. * Make it can be run by: $ runqemu tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.vmdk or: $ runqemu qemux86-64 vmdk [YOCTO #9170] Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2016-03-23oe-publish-sdk: fix remote publishingPaul Eggleton
A typo in the command prevented the last part of the publish from succeeding. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23oe-publish-sdk: improve help output slightlyPaul Eggleton
This was just copy-and-pasted from devtool - set a reasonable description and tweak the sdk parameter help text. Also add a copyright statement and drop the opening comment describing the command-line syntax that duplicated the help output (with at least one mistake in it). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23oe-publish-sdk: drop SDK installer file from published outputPaul Eggleton
We don't really need the SDK installer in the published output, for two reasons: 1) The directory produced is for the consumption of devtool sdk-update, and the installer isn't used by that at all 2) It wouldn't really make sense to point users at the update directory to download the SDK installer because it contains a bunch of things that aren't meant for manual download, so it wouldn't be very tidy. Leaving the file present can mislead you into thinking the opposite of both of the above. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23devtool: add: create git repository if URL specified as positional argumentPaul Eggleton
When we add from a fetched URL we are supposed to turn the resulting source tree into a git repository (if it isn't already one). However, we were using the older deprecated option name here instead of the positional argument, so "devtool add -f <url>" resulted in the repo being created but "devtool add <url>" didn't, which was wrong. Also update the oe-selftest tests to check that this worked. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23devtool: add: delete externalsrc files on npm recipe do_installPaul Eggleton
The npm class just installs whatever is in ${S}; if you're using externalsrc in conjunction with it the symlinks (oe-workdir and oe-logs by default) plus singletask.lock will end up in the final package, which isn't really right. Introduce a variable so we know the path the files will be installed into within npm.bbclass, and append to do_install within the workspace bbappend to delete the files. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23devtool: configure-help: fix error if do_configure not already runPaul Eggleton
The code here for running do_configure if it hadn't already been run was using the wrong string substitution parameters; fix it and test it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23devtool: sdk-install: add option to allow building from sourcePaul Eggleton
By default the sdk-install subcommand expects to restore the requested items from sstate and fails if it can't. If the user is OK with building from source, add a -s/--allow-build option to allow them to do that. In the process, ensure we show the status output while we're installing. Also add the missing header to the top of the file. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-21oe-buildenv-internal: Correct the sed expression which updates $PATHPeter Kjellerstedt
Without this, the code that adds paths to $PATH could cause it to end up with a trailing : which would then cause an error from the sanity checker. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20oe-buildenv-internal: Some clean upPeter Kjellerstedt
* Consistent indentation (four spaces) * Use [ -z ...] and [ -n ... ] where possible * Unset temporary variables * Use $(...) instead of `...` * Avoid an unnecessary call to expr Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20oe-buildenv-internal: Add variables individually to BB_ENV_EXTRAWHITEPeter Kjellerstedt
Instead of adding all variables to BB_ENV_EXTRAWHITE as one, treat them separately and add them one by one as needed. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20oe-buildenv-internal: Add paths to $PATH individuallyPeter Kjellerstedt
Instead of assuming that the path to the scripts directory always is in $PATH directly before the bitbake directory, treat them as separate paths and add them individually to $PATH. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20oe-selftest: Fixed --list-tests-by tag optionDaniel Istrate
Commit 35be67951305950ba797dc2efddbc7d88fc0556a broke the --list-tests-by tag option. This patch fixes that. Having a module in lib/oeqa/selftest named testmodule: class TestClass(oeSelfTest): @tag(feature='tag1') def test_func1(self): pass @tag(feature=('tag1', 'tag2')) def test_func2(self): pass @tag(feature=('tag2', 'tag3')) def test_func3(self): pass @tag(feature=('tag1', 'tag2', 'tag3')) def test_func4(self): pass $ oe-selftest --list-tests-by tag tag1 ID TAG(s) NAME CLASS MODULE ---- ---------------- ---------- --------- -------- tag1 test_func1 TestClass testmodule tag1, tag2 test_func2 TestClass testmodule tag1, tag2, tag3 test_func4 TestClass testmodule ______________________________ Filtering by: tag Looking for: tag1 Total found: 3 $ oe-selftest --list-tests-by tag tag1 tag2 ID TAG(s) NAME CLASS MODULE ---- ---------------- ---------- --------- -------- tag1 test_func1 TestClass testmodule tag1, tag2 test_func2 TestClass testmodule tag1, tag2, tag3 test_func4 TestClass testmodule tag2, tag3 test_func3 TestClass testmodule ______________________________ Filtering by: tag Looking for: tag1, tag2 Total found: 4 $ oe-selftest --list-tests-by tag tag* ID TAG(s) NAME CLASS MODULE ---- ---------------- ---------- --------- -------- tag1 test_func1 TestClass testmodule tag1, tag2 test_func2 TestClass testmodule tag1, tag2, tag3 test_func4 TestClass testmodule tag2, tag3 test_func3 TestClass testmodule ______________________________ Filtering by: tag Looking for: tag* Total found: 4 Signed-off-by: Daniel Istrate <daniel.alexandrux.istrate@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20devtool: change config symlink name to .config.newMarkus Lehtonen
Otherwise (if the symlink is named .config) kernel build considers source tree as dirty and fails. [YOCTO #9270] Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-20recipetool: create: don't create extra files directory unconditionallyPaul Eggleton
The extra directory next to the recipe should only be created if there are files to put into it; currently only the npm plugin does this. I didn't notice the issue earlier because the test was actually able to succeed under these circumstances if the recipe file came first in the directory listing, which was a fault in my original oe-selftest test; apparently on some YP autobuilder machines the order came out reversed. With this change we can put the oe-selftest test that highlighted the issue back to the way it was, with an extra check to reinforce that only a single file should be created. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-12scripts/oe-selftest: Add short names to most common optionsHumberto Ibarra
Add short names to most common options in oe-selftest. The options changed were --run-tests, --run-all-tests, --list-tests and --list-modules. [Yocto #9079] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-12python: move ast module into python-coreRoss Burton
The compile() method returns objects that inherit from ast.AST so it's best that python-core contains this class. [YOCTO #8684] Signed-off-by: Ross Burton <ross.burton@intel.com>
2016-03-12scripts/oe-selftest: Add search expression matching to run/list optionsHumberto Ibarra
The oe-selftest script required an exact matching for the parameters passed to its run-tests-by and list-tests-by options. Many tests can be retrieved here and filtering is a must. This patch add this filtering functionality by enabling the use of wildcards such as "*". [Yocto #8916] Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: shrinkwrap and lockdown npm modulesPaul Eggleton
"npm shrinkwrap" creates a file that ensures that the exact same versions get fetched the next time the recipe is built. lockdown is similar but also includes sha1sums of the modules thus validating they haven't changed between builds. These ensure that the build is reproducible. Fixes [YOCTO #9225]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: support creation of additional files by pluginsPaul Eggleton
Allow plugins to create additional files to go alongside the recipe. The plugins don't know what the output filename is going to be, so they need to put the files in a temporary location and add them to an "extrafiles" dict within extravalues where the destination filename is the key and the temporary path is the value. devtool add was also extended to ensure these files get moved in and preserved upon reset if they've been edited by the user. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: check if npm available if npm:// URL specifiedPaul Eggleton
If the user specifies an npm:// URL then the fetcher needs npm to be available to run, so check if it's available early rather than failing later. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: split npm module dependencies into packagesPaul Eggleton
Rather than rolling all of an npm module's dependencies into the same package, split them into one module per package, setting the SUMMARY and PKGV values from the package.json file for each package. Additionally, mark each package with the appropriate license using the license scanning we already do, falling back to the license stated in the package.json file for the module if unknown. All of this is mostly in aid of ensuring all modules and their licenses now show up in the manifests for the image. Additionally we set the main LICENSE value more concretely once we've calculated the per-package licenses, since we have more information at that point. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: add license file crunchingPaul Eggleton
Matching license texts directly to md5sums only goes so far. Some licenses make the copyright statement an intrinsic part of the license statement (e.g. MIT) which of course varies between projects. Also, people often seem to take standard license texts such as GPLv2 and reformat them cosmetically - re-wrapping lines at a different width or changing quoting styles are seemingly popular examples. In order to match license files to their actual licenses more effectively, "crunch" out these elements before comparing to an md5sum. (The existing plain md5sum matching has been left in since it's a shortcut, and our list of crunched md5sums isn't a complete replacement for it.) As always, this code isn't providing any guarantees (legal or otherwise) that it will always get the license correct - as indicated by the accompanying comments the LICENSE values it writes out to the recipe are indicative and you should verify them yourself by looking at the documentation supplied from upstream for the software being built if you have any concerns. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09recipetool: create: match *LICENSE* as a license filePaul Eggleton
For example, this picks up a file named MIT-LICENSE.txt. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>