aboutsummaryrefslogtreecommitdiff
path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* wic/imager/direct: Ensure correct ROOTFS_DIR is shownRichard Purdie2 days1-1/+6
| | | | | | | | | | | | | | | | | | This changes the output of "wic create mkefidisk -e core-image-sato" from: The following build artifacts were used to create the image(s): ROOTFS_DIR: /media/build1/poky/build/tmp.wic.ybraavmb/rootfs_copy to: The following build artifacts were used to create the image(s): ROOTFS_DIR: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs which s much less confusing for the user. [YOCTO #12564] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/test-dependencies.sh: removeMartin Jansa9 days1-286/+0
| | | | | | | | | | | | | | | * with RSS used in pyro this script isn't very useful anymore * RSS makes sure that the dependencies are almost always deterministic the only case known to me where dependencies are different based on what was already built in TMPDIR are runtime dependencies resolved by shlibs code in package.bbclass (which is using global pkgdata, not specific to given recipe and its RSS) as described here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=9217#c4 but for this case it's not worth running complete test-dependencies.sh runs Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* yocto-check-layer: add a test for correct setting of ↵Alexander Kanavin13 days2-0/+7
| | | | | | | | | LAYERSERIES_COMPAT_collection [YOCTO #12661] Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool/oeqa: Ensure added layers set LAYERSERIES_COMPATRichard Purdie2018-04-071-0/+1
| | | | | | | | Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated code from devtool/oeqa needs to set this to avoid warnings which break various tests. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* crosstap: replace script with new python based implementationVictor Kamensky2018-04-061-136/+450
| | | | | | | | | | | | | | | | New crosstap python implementation is total replacement for crosstap shell script, that has superseding capabilities. New script support cross compiling of SystemTap scripts for user-land, by using supplied image rootfs. Whereas old script could only deal with scripts against kernel. New script has more complex logic and additional capabilities. As invocation interface new script support old "legacy" mode and provides alternative new regular options interface to access additional functionality. Signed-off-by: Victor Kamensky <kamensky@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* recipetool: create: fix port number parsing issueMing Liu2018-04-031-2/+4
| | | | | | | | | | | | | | | A flaw was found when I run: $ recipetool create "ssh://git@xxx.xxx:7999/xxx.git" the url turned out to be: "git://git@xxx.xxx/7999/xxx.git;protocol=ssh" after parsing, the port number was parsed as part of the path, this is definitely wrong and lead to fetching failures. This issue could be fixed in reformat_git_uri, by filtering out port numbers when formatting ":". Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/oe-setup-rpmrepo: remove the scriptYi Zhao2018-03-291-103/+0
| | | | | | | | | After we switched to RSS, this script was not working for a long time. 'bitbake package-index' can do the same thing and works well. So remove this script. Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checklayer: remove reference to undefined classAnuj Mittal2018-03-191-2/+2
| | | | | | | | LayerError doesn't exist and will lead to an error when this failure code path is hit. Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* scripts/oe-build-perf-report: fix comparing arbitrary commitsMarkus Lehtonen2018-03-161-7/+4
| | | | | | | | | Fix a crash when generating a txt report and the two commits to be compared were not consecutive (but there were some tested commits between them). Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* build-recipe-list: improvementsRoss Burton2018-03-151-9/+21
|
* build-recipe-list: build universe instead of worldRoss Burton2018-03-121-1/+1
| | | | | | | | | Building world means recipes that are excluded from world build for whatever reason get skipped from the manifests, which isn't useful. Instead building universe and pass -k so that the expected dependency failures are not fatal. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: add --force-patch-refresh to 'modify' and 'finish' commandsAlexander Kanavin2018-03-111-9/+11
| | | | | | | | | | | | | | | This is very useful for updating patch context so that any fuzz is eliminated. Simply issue: devtool modify <recipe> devtool finish --force-patch-refresh <recipe> <layer_path> Without this flag, devtool will not deem the commits in the workspace different to patches in the layer, even if the commits have different, up-to-date context line in them. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: add wks file to make use of exclude-path functionalityCalifornia Sullivan2018-03-061-0/+3
| | | | | | | | | | Using --exclude-path and a wks.in file we can create an image that takes the /boot/ directory for the boot partition, has an empty /boot/ directory in the rootfs partition. The boot partition gets mounted to /boot/ after startup. Signed-off-by: California Sullivan <california.l.sullivan@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Set a miniumum FAT16 volume size.Paulo Neves2018-03-031-1/+1
| | | | | | | | | | | | Set a miniumum FAT16 volume size because images are not valid to parted if this minimum value is not respected. The value set is determined experimentally forr a logical sector size of 512. This fixed my local problem but, there may be better solutions. Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: partition plugin wrongly assumes it is rootfsPaulo Neves2018-03-032-3/+3
| | | | | | | | | | | | | | | | | | | | | | The partition plugin is used as the base for other plugins. One of the methods the plugins use, is the prepare_rootfs method. The prepare_rootfs method wrongly assumes that the value ROOTFS_SIZE from bitbake datastore is relevant to every invocation of prepare_rootfs, which it clearly is not, for example in the bootimg-partition case. This commit adds an optional argument to prepare_rootfs where a caller can tell prepare_rootfs if it is an actual rootfs and whether related rootfs information retrieved from bitbake is valid. The default behavior of this optional argument is to assume that the invocation is an actual rootfs, to maintain compatibility with previous implementations. Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: Add post operation hook for SourcePluginParthiban Nallathambi2018-03-033-0/+17
| | | | | | | | | | do_post_partition hook is needed if some operations like security signing the parition needs to be done. source plugins can make use of this to implement post operatiosn in do_post_partition. do_post_partition is called after do_prepare_partition if present. Signed-off-by: Parthiban Nallathambi <pn@denx.de> Signed-off-by: Ross Burton <ross.burton@intel.com>
* devtool: search: tweak help textPaul Eggleton2018-03-011-2/+2
| | | | | | | | | | | | * We now match on more than just target recipes, so don't specify that only target recipes are searched. * We're printing the SUMMARY value in addition to the name, so mention that so it's clear where that text is coming from. * Remind users that they should use quotes around the keyword to avoid shell expansion when using regular expressions. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: search: also look in recipe cachePaul Eggleton2018-03-011-33/+63
| | | | | | | | | | | | | | | | | | | | | If pkgdata isn't present or is incomplete, then you get either a traceback or you don't see the results you were hoping for. The recipe cache that bitbake collects during startup contains some useful information for each recipe that we could search through as well, and we can access it easily using tinfoil's all_recipes() API function, so add some code that does that. (We still show a warning if pkgdata isn't present, as there are certain dynamic packages that are generated at packaging time that won't show up in the cache). One side-effect of this is that we will start showing non-target recipes - that's actually a good thing, since seeing those is useful, however we exclude nativesdk recipes when in the eSDK to avoid confusion since nativesdk isn't directly applicable there. Fixes [YOCTO #12356]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: deploy-target: don't specify ssh/scp port unless user doesPaul Eggleton2018-03-011-8/+4
| | | | | | | | | | | If the user doesn't specify a port then we should avoid specifying one on the ssh/scp command line in case the user has configured one for the host they are connecting to, which was being overridden unnecessarily. Fixes [YOCTO #12381]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: finish: fix erroneously creating bbappend for relative pathsPaul Eggleton2018-03-011-3/+1
| | | | | | | | | | | | | After OE-Core rev 5e3fe00a0233d563781849a44f53885b4e924a9c we call os.path.abspath() on the original layer path, but we later compare that to the destination layer path. If that layer path isn't absolute but is effectively the same path, it should be writing to the original recipe but because we weren't making it absolute we were writing a bbappend instead. Call os.path.abspath() on the destination path as well to avoid that. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: reset: delete bbappend file if _check_preserve() doesn'tPaul Eggleton2018-03-011-0/+7
| | | | | | | | | | If the .devtool_md5 file doesn't contain a reference to the bbappend file (e.g. because devtool was interrupted before it could write that out) then _check_preserve() won't delete it, so we need to delete it separately because otherwise the recipe won't actually be reset. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: fix poor handling of upgraded BBCLASSEXTENDed recipesPaul Eggleton2018-03-012-3/+8
| | | | | | | | | | | | | Fix two aspects of handling BBCLASSEXTENDed targets (e.g. openssl-native) that have been run through "devtool upgrade": * Fix recipe name not showing up in "devtool status" * Fix "devtool reset" not deleting empty directories under the recipe directory within the workspace, which may lead to problems if you subsequently run "devtool upgrade" on the same target again Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-init-env: search 'bitbake' outside of oe-coreEnrico Scholz2018-03-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | atm, 'oe-init-build-env' expects 'bitbake' to be within the OE core git repository. This complicates the project setup because you have to manage the 'bitbake' directory or symlink manually (or specify the bitbake location explicitly). Looking for 'bitbake' outside the main git repository will ease project management significantly. Now, you can put everything into git submodules, clone the project with | git clone --recursive ... and continue immediately with | ..../oe-init-build-env E.g. when you had previously | . | |-- build/ | `-- sources/ | `-- org.openembedded.core/ | `-- bitbake -> ../bitbake (where 'bitbake' must be created manually after cloning the project), you can have now | . | |-- build/ | `-- sources/ | |-- bitbake/ | `-- org.openembedded.core/ which is completely managed by 'git'. Patch adds $OEROOT/.. to the search path for 'bitbake' Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* buildhistory-diff: honour report_all flagAnuj Mittal2018-02-241-1/+1
| | | | | | | | | Make sure that we're passing a bool value. Without this, buildhistory shows all the output for all the keys/fields when it shouldn't be by default. Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/contrib/build-perf-test: Fix fetchall reference to use runallRichard Purdie2018-02-201-1/+1
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* oe-pkgdata-util: add --long option to list-pkg-files to show sizesRoss Burton2018-02-161-6/+12
| | | | Signed-off-by: Ross Burton <ross.burton@intel.com>
* devtool: set up git repos so that singletask.lock is ignoredPaul Eggleton2018-02-061-0/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | singletask.lock is written out while certain tasks execute for recipes that have externalsrc.bbclass enabled - this includes recipes in devtool's workspace. It appears that there's a race where singletask.lock will be there one minute and then when we try to get the file checksum of it (since we want to know if anything in the source tree has changed) it will be gone, and git chokes. To fix that, add singletask.lock to .git/info/exclude in the repository, regardless of whether we created the repository or not. In any case singletask.lock should never be tracked by git, so this is a good thing to be doing for that reason as well. This fixes oe-selftest failures in test_devtool_modify that we've seen on the Yocto Project autobuilder: bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'add', '-A', '.']' returned non-zero exit status 128. Note that this only fixes this issue for devtool; if you are using externalsrc independently of devtool there's a chance this will still be an issue unless you add singletask.lock to your .gitignore. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* postinst_intercepts: add a delay_to_first_boot hookAlexander Kanavin2018-01-291-0/+2
| | | | | | | | | This allows postinsts scripts to explicitly request a deferral to first boot (by calling 'postinst_intercept delay_to_first_boot') instead of 'exit 1' which should be used only to indicate actual script failures. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* distrodata: add a utility script to compare list of recipesTan Shen Joon2018-01-292-0/+240
| | | | | | | | | | | | | | | | | | | | | distrocompare.sh is added to compare the added list of recipes between two releases. The output of the script will share the information of the licenses required and other distributions that are using the package. If a single input is provided, it will compare the current branch with the provided branch/commit-ish package list. To run : distrocompare.sh <older hash> <newer hash> E.g. distrocompare.sh morty 92aa0e7 E.g. distrocompare.sh morty pyro E.g. distrocompare.sh morty output : The script will produce a file ending with new_recipe_list.txt preceeded by the branch name from input Signed-off-by: Tan Shen Joon <shen.joon.tan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oe-pkgdata-util: Include colon when searching for variablesAmanda Brindle2018-01-291-8/+8
| | | | | | | | When searching for variables, include colon to ensure the script doesn't find a variable that starts with the same name. Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oe-pkgdata-util: Add support for RPROVIDESAmanda Brindle2018-01-221-61/+93
| | | | | | | | | | | | | In lookup_recipe, package_info, and list_pkg_files, check if the package name exists in runtime-rprovides. If so, and the provider package has a different name than the inputted package, print a note that says the specified package is in another package's RPROVIDES. If the provider package has the same name as the inputted package, continue as before. Fixes [YOCTO 11943] Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* oe-pkgdata-util: Refactor functions for consistencyAmanda Brindle2018-01-221-59/+48
| | | | | | | | | | Refactor functions lookup_recipe and package_info to be consistent with list_pkg_files. Print the appropriate information as soon as it's found, rather than storing it in a mappings variable and wait to print after searching all packages. Signed-off-by: Amanda Brindle <amanda.r.brindle@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* python3: Restructure python3 packaging and replace it with autopackagingAlejandro Hernandez2018-01-201-442/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | See previous commit (python2 version) for more info, since mostly everything applies here as well. Old manifest file had several issues: - Its unorganized and hard to read and understand it for an average human being. - When a new package needs to be added, the user actually has to modify the script that creates the manifest, then call the script to create a new manifest, and then submit a patch for both the script and the manifest, so its a little convoluted. - Git complains every single time a patch is submitted to the manifest, since it violates some of its guidelines. - It changes or may change with every release of python, its impossible to know if the required files for a certain package have changed (it could have more or less dependencies), the only way of doing so would be to install and test them all one by one on separate individual images, and even then we wouldnt know if they require less dependencies, we would just know if an extra dependency is required since it would complain, lets face it, this isnt feasible. - The same thing happens for new packages, if someone wants to add a new package, its dependencies need to be checked manually one by one. Features/Fixes: - A new manifest format is used (JSON), easy to read and understand. This file is parsed by the python recipe and python packages read from here are passed directly to bitbake during parsing time. - It provides an automatic manifest creation task (explained on previous commit), which automagically checks for every package dependencies and adds them to the new manifest, hence we will have on each package exactly what that package needs to be run, providing finer granularity. - Dependencies are also checked automagically for new packages (explained on previous commit). This patch has the same features as the python2 version but it differs in the following ways: - Python3 handles precompiled bytecode files (*.pyc) differently. for this reason and since we are cross compiling, wildcards couldnt be avoided on python3 (See PEP #3147 [1]). Both the manifest and the manifest creation script handle this differently, the manifest for python3 has an extra field for cached files, which is how it lets the user install the cached files or not via : INCLUDE_PYCS = "1" on their local.conf. - Shared libraries nomenclature also changed on python3, so again, we use wildcards to deal with this issue ( See PEP #3149 [2]): - Fixes python3 manifest, python3-core should be base and everything should depend on it, hence several packages were deleted: python3-enum, re, gdbm, subprocess, signal, readline. - When building python3-native it adds as symlink to it called nativepython3, which is then isued by the create_manifest task. - Fixes [YOCTO #11513] while were at it. References: [1] https://www.python.org/dev/peps/pep-3147/ [2] https://www.python.org/dev/peps/pep-3149/ Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* python: Restructure python packaging and replace it with autopackagingAlejandro Hernandez2018-01-201-426/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reason we have a manifest file for python is that our goal is to keep python-core as small as posible and add other python packages only when the user needs them, hence why we split upstream python into several packages. Although our manifest file has several issues: - Its unorganized and hard to read and understand it for an average human being. - When a new package needs to be added, the user actually has to modify the script that creates the manifest, then call the script to create a new manifest, and then submit a patch for both the script and the manifest, so its a little convoluted. - Git complains every single time a patch is submitted to the manifest, since it violates some of its guidelines. - It changes or may change with every release of python, its impossible to know if the required files for a certain package have changed (it could have more or less dependencies), the only way of doing so would be to install and test them all one by one on separate individual images, and even then we wouldnt know if they require less dependencies, we would just know if an extra dependency is required since it would complain, lets face it, this isnt feasible. - The same thing happens for new packages, if someone wants to add a new package, its dependencies need to be checked manually one by one. This patch fixes those issues, while adding some additional features. Features/Fixes: - A new manifest format is used (JSON), easy to read and understand. This file is parsed by the python recipe and python packages read from here are passed directly to bitbake during parsing time. - It provides an automatic manifest creation task (explained below), which automagically checks for every package dependencies and adds them to the new manifest, hence we will have on each package exactly what that package needs to be run, providing finer granularity. - Dependencies are also checked automagically for new packages (explained below). - Fixes the manifest in the following ways: * python-core should be base and all packages should depend on it, fixes lang, string, codecs, etc. * Fixes packages with repeated files (e.g. bssdb and db, or netclient and mime, and many others). - Sitecustomize was fixed since encoding was deprecated. - The JSON manifest file invalidates bitbake's cache, so if it changes the python package will be rebuilt. - It creates a solution for users that want precompiled bytecode files (*.pyc) INCLUDE_PYCS = "1" can be set by the user on their local.conf to include such files, some argument they get faster boot time, even when the files would be created on their first run?, but they also sometimes give a magic number error and take up space, so we leave it to the user to decide if they want them or not. - Fixes python-core dependencies, e.g. When python is run on an image, it TRIES to import everything it needs, but it doesnt necessarily fails when it doesnt find something, so even if we didnt know, we had errors like (trimmed on purpose): # trying /usr/lib/python2.7/_locale.so # trying /usr/lib/python2.7/lib-dynload/_locale.so # trying /usr/lib/python2.7/_sysconfigdata.so while it didnt complain about _locale it should have imported it, after creating a new manifest with the automated script we get: # trying /usr/lib/python2.7/lib-dynload/_locale.so dlopen("/usr/lib/python2.7/lib-dynload/_locale.so", 2); import _locale # dynamically loaded from /usr/lib/python2.7/lib-dynload/_locale.so How to use (after a new release of python, or maybe before every OE release): - A new task called create_manifest was added to the python package, which may be invoked via: $ bitbake python -c create_manifest This task runs a script on native python on our HOST system, and since the python and python-native packages come from the same source, we can use it to know the dependencies of each module as if we were doing it on an image, this script is called create_manifest.py and in a very simplistic way it does the following: 1. Reads the JSON manifest file and creates a dictionary data structure with all of our python packages, their FILES, RDEPENDS and SUMMARY. 2. Loops through all of them and runs every module listed on them asynchronously, determining every dependency that they have. 3. These module dependencies are then handled, to be able to know which packages contain those files and which should RDEPEND on one another. 4. The data structure that comes out of this, is then used to create a new manifest file which is automatically copied onto the user's python directory replacing the old one. Create_manifest script features: - Handles modules which dont exist anymore (new release for example). - Handles modules that are builtin. - Deals with modules which were not compiled (e.g. bsddb or ossaudiodev) - Deals with packages which include folders. - Deals with packages which include FILES with a wildcard. - The manifest can be constructed on a multilib environment as well. - This method works for both python modules and shared libraries used by python. How to add a new package: - If a user wants to add a new package all that has to be done is modify the python2-manifest.json file, and add the required file(s) to the FILES list, the script should handle all the rest. Real example: We want to add a web browser package, including the file webbrowser.py which at the moment is on python-misc. "webbrowser": { "files": ["${libdir}/python2.7/lib-dynload/webbrowser.py"], "rdepends": [], "summary": "Python Web Browser support"} Run bitbake python -c create_manifest and the resulting manifest should be completed after a few seconds, showing something like: "webbrowser": { "files": ["${libdir}/python2.7/webbrowser.py"], "rdepends": ["core","fcntl","io","pickle","shell","subprocess"], "summary": "Python Web Browser support"} Known errors/issues: - Some special packages are handled differently: core, misc, modules,dev, staticdev. All these should be handled manually, because they either include binaries, static libraries, include files, etc. (something that we cant import). Specifically static libraries are not not supported by this method and have to be handled by the user. - The change should be transparent to the user, other than the fact that now we CANT build python-foo (it was pretty dumb anyway, since what building python-foo actually did was building the whole python package anyway), but doing IMAGE_INSTALL_append = " python-foo" would create an image with the requested package with no issues. [YOCTO #11510] [YOCTO #11694] [YOCTO #11695] Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* scripts/oe-depends-dot: add it to handle dot filesRobert Yang2018-01-181-0/+121
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add it to handle recipe-depends.dot and task-depends.dot. E.g.: * Print why rpm is built $ oe-depends-dot -k rpm --why/-w recipe-depends.dot Because: core-image-sato libdnf libsolv dnf * Print bzip2-native's depends $ oe-depends-dot -k bzip2-native --depends/-d recipe-depends.dot Depends: automake-native gnu-config-native libtool-native quilt-native autoconf-native * Remove duplicated dependencies to reduce the size of the dot files. For example, A->B, B->C, A->C, then A->C can be removed. The dot files are too big, we nearly couldn't use 'dot -T' to generate pictcures for target recipes, remove the duplicated dependencies makes is it possible. $ bitbake core-image-sato -g $ oe-depends-dot -r recipe-depends.dot Saving reduced dot file to recipe-depends-reduced.dot $ du -sh recipe-depends*.dot 608K recipe-depends.dot 32K recipe-depends-reduced.dot It has been recuded from 608K to 32K, now we can generate a picture, otherwise, it is too big: $ dot -Tpng recipe-depends-reduced.dot -O It also can handle task-depends.dot. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* buildhistory-diff: add support for colourising the outputRoss Burton2018-01-181-2/+7
| | | | | | | | | | | | Colour value removes in red and additions in green, making it easier to scan the output for relevant changes. This adds a --colour option to specify whether colouring should be on, off, or detected. The default is detected, and depends on whether stdout is a TTY (same behaviour as git). Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* devtool: rename the suggested license change tag to 'License-Update:'Alexander Kanavin2018-01-181-2/+2
| | | | | Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: if we can't get from ioctl, try from os.stat()Dogukan Ergun2018-01-141-1/+9
| | | | | | | | | | | Under some conditions, ioctl FIGETBSZ can't return real value. We can try to use fallback via os.stat() to get block size. Source of patch: https://github.com/intel/bmap-tools/commit/17365f4fe9089df7ee9800a2a0ced177ec4798a4 Signed-off-by: Dogukan Ergun <dogukan.ergun@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* scripts/wic: explicitly set BUILDDIR within eSDKChang Rebecca Swee Fun2018-01-121-0/+2
| | | | | | | | | | | | | | | | | | | | | | | When we run wic within eSDK: $ wic create mkefidisk -e core-image-minimal ERROR: BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?) In order to figure out variable values, one must have sourced the OE build environment setup script. However, when we are in within the eSDK environment which isn't initialised like the normal OE build environment, we can't use wic utility with eSDK. Reference: https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#wic-requirements While wic ought to be fixed to be able to run without bitbake & native tools [YOCTO #11281], but this is a workaround to set BUILDDIR in the environment so that bitbake environment is setup for wic to build its required native tools. Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/wic: fix error of import wic module in eSDK environmentChang Rebecca Swee Fun2018-01-121-1/+1
| | | | | | | | | | | | | | | | | | wic modules in scripts/lib/ are needed for wic to work, but path to the python module is not exported in eSDK environment and we were using an absolutized path of wic script within the sysroots. We now changed to use real script path instead, where the wic modules are located. This will also resolved the tracebacks found when running wic from within the eSDK environment. Traceback (most recent call last): File "/tmp/deploy/sdk/poky_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/wic", line 58, in <module> from wic import WicError ImportError: No module named 'wic' Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/wic: append bitbake executable file path in eSDK environmentChang Rebecca Swee Fun2018-01-121-0/+12
| | | | | | | | | | | | wic needs a set of tools to be available from sysroots. wic will find bitbake executable within the environment, and wic was unable to locate bitbake executable within eSDK because it wasn't setup with the OE build environment script. Hence, we need to add bitbake file path into the environment PATH for wic to be able to discover it and import bb modules. Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* scripts/wic: use scriptpath module to find bitbake path and oe lib pathChang Rebecca Swee Fun2018-01-121-4/+3
| | | | | | | | Use the scriptpath module in order to standardize the adding of bitbake and meta/lib path to sys.path. Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86Richard Purdie2018-01-021-0/+5
| | | | | | | | | | On pre 4.15 host kernels, an APIC window emulation bug can cause qemu to hang. On 64 bit we can use the x2apic, for 32 bit, we just have to disable the other timer sources and rely on kvm-clock. [YOCTO #12301] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* wic: Fix a path to a psuedo state directory (PSEUDO_LOCALSTATEDIR).Adrian Fiergolski2018-01-021-1/+1
| | | | | | | | | | | In case of 'new_rootfs' the psuedo directory is not copied. Thus PSEUDO_LOCALSTATEDIR should still point to the dsa 'native_sysroot'/../pseudo. Otherwise PSEUDO_LOCALSTATEDIR points to a not existing director ('new_rootfs'/../pseudo) and UID and GUID attributes are not applied to files of the image. Signed-off-by: Adrian Fiergolski <adrian.fiergolski@cern.ch> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: Introduce --fsuuid and have --use-uuid make use of UUID tooTom Rini2017-12-186-19/+47
| | | | | | | | | | | First, allow for wic to be given a filesystem UUID to be used when creating a filesystem. When not provided, wic will generate the UUID to be used. Next, when --use-uuid is passed, we update the fstab to mount things via UUID (and if not found, then use PARTUUID) as UUID is more portable. Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: partition.py: Update comments slightlyTom Rini2017-12-181-3/+1
| | | | | | | | | First, we support squashfs as root, so mention that. Second, the btrfs rootfs creation function had a copy/paste of the previous function comment, remove the irrelevant line. Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* wic: kparser.py: Check for SquashFS and use-uuidTom Rini2017-12-181-0/+5
| | | | | | | | The SquashFS filesystem does not support UUIDs so make this combination be an error. Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* devtool: add license checksum change handling to 'devtool upgrade'Alexander Kanavin2017-12-181-2/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Specifically, 'devtool upgrade' will now do these things: 1) determine if any of the license checksums need updating; if so, write the new checksums into the LIC_FILES_CHKSUM value in the recipe that is written to the workspace; 2) print a notice to the standard output: NOTE: New recipe is /home/ak/development/poky/build/workspace/recipes/puzzles/puzzles_git.bb NOTE: License checksums have been updated in the new recipe; please refer to it for the difference between the old and the new license texts. 3) and the cool part: devtool will create a diff of the old and new licenses, and write the diff into the workspace recipe as a comment, like this: ====== FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'. The following is the difference between the old and the new license text. Please update the LICENSE value if needed, and summarize the changes in the commit message via 'License-checksum-change:' tag. (example: 'License-checksum-change: copyright years updated.') The changes: --- LICENCE +++ LICENCE @@ -1,8 +1,9 @@ This software is copyright (c) 2004-2014 Simon Tatham. Portions copyright Richard Boulton, James Harvey, Mike Pinna, Jonas -Klker, Dariusz Olszewski, Michael Schierl, Lambros Lambrou, Bernd -Schmidt, Steffen Bauer, Lennard Sprong and Rogier Goossens. +Kölker, Dariusz Olszewski, Michael Schierl, Lambros Lambrou, Bernd +Schmidt, Steffen Bauer, Lennard Sprong, Rogier Goossens and Michael +Quevillon. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files ====== Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
* devtool: provide useful defaults for version/commit when upgrading recipesAlexander Kanavin2017-12-181-4/+12
| | | | | | | Specifically, 'devtool upgrade' will use the latest upstream release if available or latest commit if upstream never makes releases. Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
* devtool: add a 'latest-version' commandAlexander Kanavin2017-12-181-0/+26
| | | | | | | | | | | | | | | | | | | | | | This command queries the upstream server for what the latest release is and prints the output; it is a much neater way to find out these things than fumbling with distrodata, 'bitbake -c checkpkg' and awkward to read csv output in a file. Examples: python3 (tarballs): NOTE: Current version: 3.5.3 NOTE: Latest version: 3.6.3 rpm (git): NOTE: Current version: 4.13.90 NOTE: Latest version: 4.14.0 NOTE: Latest version's commit: da3720f62e57648fb1dc2a632744d38866139971 puzzles (git without version tags): NOTE: Latest commit: ee8ea9b9785964694cb2b3ad77c3fb2460f49510 Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>