aboutsummaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2017-02-07devtool: improve parse failure handlingpaule/devtool25Paul Eggleton
With the move to tinfoil2, the behaviour when parsing failed has changed a bit - exceptions are now raised, so handle these appropriately. Specifically when if parsing the recipe created when running devtool add fails, rename it to .bb.parsefailed so that the user can run bitbake afterwards without parsing being interrupted. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-02-07recipetool: create: do not treat numbers in SCM URLs as versionsPaul Eggleton
Numbers within SCM (e.g. git) URLs are extremely unlikely to be valid version numbers - more likely they are just part of the name, thus don't try to extract them and use them as the version - doing so causes pretty bad behaviour within devtool: --------- snip --------- $ devtool add https://github.com/inhedron/libtr50 NOTE: Fetching git://github.com/inhedron/libtr50;protocol=https... ... NOTE: Using default source tree path .../build/workspace/sources/libtr ... RecursionError: maximum recursion depth exceeded while calling a Python object --------- snip --------- (This was because ${PV} was being substituted into the URL, but PV's value was being set to include ${SRCPV}, so there was a circular reference.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-02-07recipetool: create: properly handle npm optional dependenciesPaul Eggleton
npm's package.json supports two types of dependencies - optionalDependencies and dependencies; in the code for creating a recipe from a non-npm source (e.g. a git repository) we were not handling optionalDependencies and thus when pointed at a node.js application outside of npm we weren't taking care of all dependencies. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-02-05wic: fix call of serial_console_form_kargsEd Bartosh
As syslinux module has been recently removed and serial_consloe_form_kargs became local API in rootfs_pcbios_ext plugin it should be called without syslinux. prefix. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05wic: isoimage-isohybrid: use wic-tools to get syslinux pathEd Bartosh
wic-tools recipe specific sysroot contains syslinux as wic-tools depends on it. Used wic-tools target to get syslinux path should guarantee that syslinux is installed there and can be used. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05wic: flatten imager class hierarchyEd Bartosh
wic code is hard to follow due to deep and twiggy class inheritance tree. Flatten imager tree: wic -> wic_create -> Creator -> DirectPlugin -> DirectImageCreator to wic -> wic_create -> DirectPlugin by removing Creator class and creator module merging DirectImageCreator into DirectPlugin Changed APIs to use the same parameters names. Passed parsed command line options as an object down the stack. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: split out typingAnders Darander
This allows us to use typing.py without having to add the whole python3-misc package. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add ipaddress to python3-ioAnders Darander
This allows us to use ipaddress without requiring the add the whole python3-misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add _compat_pickle to python3-pickleAnders Darander
This allows us to depend on _compat_pickle.* wihtout having to add the whole python3-misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add socketserver to netserverAnders Darander
socketserver.* should be part of python3-netserver. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: add argparse to RDEPENDS for netclientAnders Darander
http/server.py requires argparse. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python3-manifest: move htlm.py to python3-htmlAnders Darander
This allows us to use html.py without importing misc. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-05python-3-manifest: fix adding imp to importlibAnders Darander
Commit: 512334f102a33833d39af53467894315f0715d07 "python-3.5-manifest: Add imp to importlib" added imp to importlib in the generated manifest, but not in the generator script. Signed-off-by: Anders Darander <anders@chargestorm.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: get rid of image_output_dir variableEd Bartosh
Used options.outdir instead of image_output_dir. There is no sense to use extra variable for this. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: direct: fix creation of work directoryEd Bartosh
It was a typo in current code: mktemp was used instead of mkdtemp to create work directory. This is fixed by using mkdtemp. Create work directory as a subdirectory of output directory to make sure both are on the same partition to make moving of result image faster. This also fixes possible disk space issues as mkdtemp uses TMPDIR, TEMP or TMP environment variables to get default value of its 'dir' parameter. Those variables are usually pointing to /tmp, which is not the best location to create huge images. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: engine: create output dirEd Bartosh
Make sure output directory exists before creating an image. Create it if it doesn't exist. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02scripts/runqemu: fix a typoMing Liu
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02build-perf-test-wrapper.sh: implement lockingMarkus Lehtonen
In order to prevent multiple instances of the script running at the same time. Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02wic: remove syslinux.pyEd Bartosh
This module contains singe function serial_console_form_kargs, which is used only by rootfs_pcbios_ext plugin. Moved it there and removed syslinux module to make it easy to find and mainain plugin code. [YOCTO #10619]
2017-02-02wic: code cleanupEd Bartosh
Fixed indentation, unused imports, trailing lines etc. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: msger.py: remove unused APIsEd Bartosh
Removed unused enable_logstderr and disable_logstderr APIs. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: removed code from __init__.pyEd Bartosh
The code deals with non-existing directory and can be removed. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: move oe/misc.py one level upEd Bartosh
Flattened directory structure: moved wic/utils/oe/misc.py -> wic/utils/misc.py [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: move 2 APIs to wic.engineEd Bartosh
Moved find_canned and get_custom_config APIs to engine module. Removed empty wic.utils.misc module. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wci: misc: removed build_name APIEd Bartosh
This API is not used in wic code. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: partition: simlify calling plugin methodsEd Bartosh
Replaced parse_sourceparams function with list comprehension. Used local variables instead of attributes. Moved global variable to the local scope. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: removed test fileEd Bartosh
This file is not used anywhere in the wic code. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: renamd direct_plugin.py -> direct.pyEd Bartosh
As this files is located in plugins/imager subdirectory it's obvious that it's an imager plugin. Renamed to direct.py to be consistent with plugin naming scheme. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: simplified code of direct_pluginEd Bartosh
Removed unused methods. Got rid of get_default_source_plugin and _full_name methods. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: pylinted direct_pluginEd Bartosh
Fixed wrong continued indentation, unused import and trailing new line pyling warnings. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: improve naming in direct_plugin classesEd Bartosh
Synchronized attribure names in DirectImageCreator and DirectPlugin for better readability. Simplified code, removed unneeded global variable disk_methods. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: get rid of __rootfs_dir_to_dict methodEd Bartosh
Replaced class method __rootfs_dir_to_dict with a list comprehension. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: moved content of direct.py to direct_pluginEd Bartosh
This move simplifies directory structure and makes further refactoring easier. The code from direct.py was used only in direct_plugin, so it's safe to move it there. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: removed conf.py and empty config file.Ed Bartosh
Removed as they're not used anymore in wic code. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: direct_plugin: stop using config managerEd Bartosh
This is a preparation to removing conf.py and config/wic.conf from the codebase. Got rid of using configmgr global object in direct_plugin and direct modules. It was used to implicitly parse kickstart file and set couple of variables. Replaced usage of configmgr by passing parameters directly to the DirectImageCreator. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-02-02wic: creator: stop using config managerEd Bartosh
This is a preparation to removing conf.py and config/wic.conf from the codebase. confmgr object is complicated for no reason and almost useless as all configuration info comes from command line and bitbake variables. It's used it creator.py to store information about output directory, logs and some never used functionality like tmpfs for future use, which doesn't actually happen. [YOCTO #10619] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31verify-bashisms: support warnings with more than one line of source codePatrick Ohly
All warnings start with "possible bashism in", followed by one or more (in the case of line continuation) lines of source code. To support more than one line, we now split by matching against the known intro text. Example: $ verify-bashisms guile ... /.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"): echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \ > ${B}/guile-config.cross Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: check scripts only once, include original file and linePatrick Ohly
Several scripts that are defined in .bbclass files end up in multiple different recipes. It's better (faster, less repetitive error reports) to check them only once. In addition, the real information for the developer is where he can find the script, not which recipe file uses it. verify-bashisms now prints the original file instead of the recipe whenever possible (i.e. 'filename' is set) and also bumps the line number so that it is relative to the file and not the script. Example with one real error and one added just for testing: $ verify-bashisms core-image-minimal core-image-sato Loading cache: 100% |#################################################################################| Time: 0:00:00 Loaded 2935 entries from dependency cache. Parsing recipes: 100% |###############################################################################| Time: 0:00:01 Parsing of 2137 .bb files complete (2101 cached, 36 parsed). 2935 targets, 412 skipped, 0 masked, 0 errors. Generating scripts... Scanning scripts... /.../openembedded-core/meta/classes/populate_sdk_ext.bbclass possible bashism in install_tools line 515 (should be 'b = a'): if [ "${SDK_INCLUDE_TOOLCHAIN}" == "1" -a ! -e $unfsd_path ] ; then possible bashism in install_tools line 521 (type): type fixme Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: revise update-rc.d whitelist entryPatrick Ohly
The actual code recently changed to: if ${@use_updatercd(d)} && type update-rc.d >/dev/null 2>/dev/null; then Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: fix problems with tinfoil2Patrick Ohly
tinfoil2 is based on a client/server architecture, which broke the verify-bashisms script: - The tinfoil instance and its data proxies can't be pickled, so all interaction with the bitbake server has to run in the main script process and only processing of the plain scripts can be done with multiprocessing: _pickle.PicklingError: Can't pickle <class 'bb.tinfoil.TinfoilCookerAdapter.TinfoilRecipeCacheAdapter'>: attribute lookup TinfoilRecipeCacheAdapter on bb.tinfoil failed - The multiprocessing pool has to be created before initializing tinfoil, otherwise the pool workers end up trying to communicate with the bitbake server during shutdown: ERROR: UI received SIGTERM Process ForkPoolWorker-2: Traceback (most recent call last): File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap util._exit_function() File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function _run_finalizers(0) ... File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive assert self._parent_pid == os.getpid(), 'can only test a child process' AssertionError: can only test a child process - func() needs to defined before creating the pool to avoid: AttributeError: Can't get attribute 'func' on <module '__main__' from '/work/openembedded-core/scripts/verify-bashisms'> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: explicitly shut down serverPatrick Ohly
Current tinfoil2 requires manually shutting down the server. Without that, the script hangs during exit. This might change in the future. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: point out where to get checkbashisms.plPatrick Ohly
The current SourceForge project seems to be unmaintained (last release 2.0.0.2 from 2015) while the copy used by Debian is quite active (last commit 2016-09-30). Ideally, checkbashisms.pl should get installed automatically via a recipe, but for now at least provide the link for manual installation. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31verify-bashisms: fix typoPatrick Ohly
Variable was renamed, it's now called "output". Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31runqemu: allow bypassing of network setupJuro Bystricky
At present it is silently assumed all QEMU machines support networking. As a consequence, one cannot run QEMUs without network emulation using "runqemu". This patch allows bypassing any network setup providing the qemuboot.conf file contains: qb_net = none [YOCTO#10661] Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31oe-publish-sdk: add pyshtables.py to .gitignoreAndrea Galbusera
pyshtables.py should be ignored by git as it is generated. If kept in the repo, causes subsequent runs of sdk-update to fail. [ YOCTO #10963 ] Signed-off-by: Andrea Galbusera <gizero@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31isoimage-isohybrid: renamed variable hdd_dirEd Bartosh
Renamed variable hdd_dir to deploy_dir as this variable is assigned to the value of DEPLOY_DIR_IMAGE. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCHEd Bartosh
isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image. It doesn't work for all machines, for example for quemux86-64 machine MACHINE_ARCH is quemux86_64 and initrd name is core-image-minimal-initramfs-qemux86-64.cpio.gz Used TRANSLATED_TARGET_ARCH variable to get the initrd image name. Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to make it available from <image>.env file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: Look for image artifacts in a common locationTom Zanussi
Rather than have each image type look for artifacts in image-specific locations, move towards having them look for artifacts in a common location, in this case DEPLOY_DIR_IMAGE Use the existing deploy.bbclass to have the bootloaders put their binaries in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2017-01-31wic: isoimage-isohybrid: stop using HDDDIREd Bartosh
Stop using HDDDIR in isoimage-isohybrid wic plugin. This variable is set by hddimg code, which is going to be removed soon. All required artifacts should be available from ISODIR. wic-image.bbclass has been modified to build iso artifacts, so it should be safe to remove usage of HDDDIR. [YOCTO #10835] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: use INITRD_LIVE in isoimage-isohybridEd Bartosh
INITRD variable is not set if hddimg is disabled. isoimage-isohybrid can't get correct name for initrd if INITRD variable is not set. Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>