aboutsummaryrefslogtreecommitdiffstats
path: root/scripts/lib
AgeCommit message (Collapse)Author
2017-09-11devtool: upgrade: check that user has configured git properlyPaul Eggleton
If user.name or user.email haven't been set then git rebase can't really work properly. Check that the user has set these and error out if not. (Elsewhere we are relying on OE's git patch functionality which forces a dummy OE value - that's OK there as it's completely under OE's control and therefore it's OK for a dummy OE user to be the committer, but here the rebase may require intervention so it's reasonable to have the user's actual name and email on the operation.) Fixes [YOCTO #11947]. (From OE-Core rev: 129a3be07e272013be2db17552c13b4d8cc2cf6e) Signed-off-by: paul <paul@peggleto-mobl.ger.corp.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11wic: run bmaptool with native Python3Ed Bartosh
Modified wic code to run bmaptool using native Python3 from wic-tools native sysroot. [YOCTO #11891] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-09-11wic: update help contentEd Bartosh
Added ext* partitions to the description of 'wic ls', 'wic cp' and 'wic rm' commands. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-09-11wic: implement ext fs support for 'wic rm'Ed Bartosh
Implemented removing files or directories from the ext partition using debugfs tool. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-09-11wic: implement ext fs support for 'wic cp'Ed Bartosh
Implemented copying files to the ext partition using debugfs tool. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-09-11wic: implement ext fs support for 'wic ls'Ed Bartosh
Implemented listing directory contents for ext file system using debugfs tool. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-09-11devtool: status: Sort entries before printingOla x Nilsson
Sorted entries are easier to read. Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-08-31scriptutils: fix fetch_url() to use lowercase dummy recipe namePaul Eggleton
recipetool create (and hence devtool add) and devtool upgrade use fetch_url() which creates a dummy recipe in order to fetch source. Previously the random part of the name was using uppercase characters, and this triggers a QA warning after OE-Core commit 4713f8b2c4f2c74239d284adcf1e59e61aa66576, so use lowercase instead as I really should have in the first place. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31recipetool: create: detect Eclipse licensesPaul Eggleton
Add detection of EPL 1.0 and EDL 1.0 license files. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31recipetool: create: suppress npm shrinkwrap/lockdown warnings againPaul Eggleton
Since OE-Core revision 9a47a6690052ef943c0d4760630ee630fb012153 the mechanism we were using to suppress the warnings about NPM_LOCKDOWN and NPM_SHRINKWRAP not being set on the first fetch of the source is no longer available since we are using the normal fetch/unpack tasks to do the job. Use the newly added noverify parameter to suppress the warnings again. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31recipetool: create: fix SRCPV prefix for non-git SCMsPaul Eggleton
If you're fetching from an SCM other than git (for example subversion or mercurial) then we need to use a different prefix for the SRCPV in PV instead of +git. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31recipetool: create: make recently added branch/tag handling git specificPaul Eggleton
The branch and tag handling code that was recently added in OE-Core revs ecca596b75cfda2f798a0bdde75f4f774e23a95b and 3afdcbdc9a3e65bc925ec61717784ffec67d529d is specific to git, so only apply it when we're fetching from a git URL. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31devtool: add: add explicit srcrev/branch optionsPaul Eggleton
At the moment when fetching source from a git repository you have to know that you can specify the revision and branch in the URL with ';rev=' and ';branch=' respectively, and you can also get thrown off by the shell splitting on the ; character if you forget to surround the URL in quotes. Add explicit -S/--srcrev and -B/--srcbranch options (consistent with devtool upgrade) to make this easier for the user to discover and use. (The rev and branch URL parameters will continue to work, however.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31devtool: edit-recipe: fix regressionPaul Eggleton
OE-Core commit 5a16b3c804c5eca331a1c08a7ce31a54909af105 attempted to use the same function to get the path to a recipe as the new "find-recipe" command it implemented, except that cannot work because (a) it didn't return anything and (b) event if it had tried, a command function can only return an exit code and we don't want that for find-recipe if it succeeded. Split out a separate reusable function for both commands. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31devtool: upgrade: workaround for recipes which apply patches conditional ↵Paul Eggleton
upon class If we're upgrading a recipe that appends additional patches for, say, class-native, and we're just upgrading the target variant, then when we copied the recipe into the workspace we skipped copying the additional patches for the native variant. This caused warnings because the workspace recipe is preferred. Look at SRC_URI for all variants when copying files to work around this. More work is needed to make it easier to work with recipes that use BBCLASSEXTEND where you need to build more than one variant at once, but this at least fixes the immediate ugliness. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31devtool: upgrade: fix handling of non-absolute pathsPaul Eggleton
If your BBLAYERS has non-absolute paths in it (e.g. "${COREBASE}/../something") then none of the paths matched in copy_recipe_files() with the result that no files got copied and you ended up with an error later on because the recipe file couldn't be found at the destination. Fix this as well as adding an explicit check to see if no files got copied - error out earlier if so. Fixes [YOCTO #10981]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31devtool: update-recipe: ensure patches get deleted in srcrev modePaul Eggleton
Patches that we identify as having been "deleted" (i.e. patches in SRC_URI that no longer appear in the git tree) need to be dropped even if we're updating in srcrev mode. This fixes the case where HEAD of the git tree is valid upstream (i.e. no extra commits), but there are patches left over in the recipe, e.g. when we do devtool upgrade and then all of the commits rebased on top of the new branch get skipped. Fixes [YOCTO #11972]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27devtool: deploy-target: Support stripped libs and execsTobias Hagelborn
New devtool deploy-target option --strip which enables deploying stripped binaries, saving some space on target. * Copies the files of ${D} into a new directory and strips them in place * Used oe.package.strip_execs for stripping directory * Added devtool.conf option "strip" for changing default behavior Config example: [Deploy] strip = true [YOCTO #11227] Signed-off-by: Tobias Hagelborn <tobiasha@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: always read image partitionsEd Bartosh
Got rid of lazy evaluation of self.partitions property. It's not needed because partitions of the source image should be always read. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: implement 'wic write' commandEd Bartosh
This command writes image to the media or another file with the possibility to expand partitions to fill free target space. [YOCTO #11278] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: extend list of used toolsEd Bartosh
Added sfdisk, e2fsck, mkswap, resize2fs, mkdosfs to the list of used tools in Disk class. They're going to be used in 'wic write' implementation. Added dependency to util-linux to wic-tools to ensure that sfdisk and mkswap are available from wic-tools native sysroot. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: added 'fstypes' parameter to Disk.__init__Ed Bartosh
This parameter specifies list of supported filesystems. So far only 'fat' is supported, but 'wic write' is going to support at least 'fat', 'ext' and 'swap'. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: convert partition number to intEd Bartosh
Converted partition number to int in order to use it as an index in the list of partitions. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: get more info from the 'parted print' outputEd Bartosh
Got partition type and sector sizes from the output of 'parted print'. This info may be used in the implementation of 'wic write' command. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27wic: reimplement getting paths of used toolsEd Bartosh
So far every used tool have to have separate property and private attribute in the Disk class. This is too verbose, considering that there will be much more tools used. Reimplemented getting tools paths using custom __getattr__ method. This is much more compact and readable. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23recipetool: create: fix npm license code regressionPaul Eggleton
OE-Core commit 1df60b09f7a60427795ec828c9c7180e4e52f98c caused a regression in npm handling since it still expected to be able to get the results of the license handling, but this no longer happens until after the npm plugin is called. Thus, call the license handling function ourselves here (which will record this as having been handled so it doesn't get done again later). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23recipetool: create: fix broken import in npm modulePaul Eggleton
With "import oe" in create_npm.py you get "AttributeError: module 'oe' has no attribute 'package'" when it tries to call oe.package.npm_split_package_dirs(). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: allow plugins to set LICENSE and LIC_FILES_CHKSUMPaul Eggleton
We were being a bit prescriptive in setting LICENSE and LIC_FILES_CHKSUM. We can't always trust what's in the metadata accompanying some source which plugins will almost always be pulling from, however we do want to allow plugins to set the LICENSE and LIC_FILES_CHKSUM values. Merge what we find in our license file scan with what the plugin sends back. Additionally, plugins can now add a "license" item to the handled list in order to inhibit the normal LICENSE / LIC_FILES_CHKSUM handling if they have already taken care of it completely. Thanks to Mark Horn <mark.d.horn@intel.com> for prompting, testing and fixing this patch. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: allow plugins to set PN / PV more easilyPaul Eggleton
Previously if we were able to auto-determine the name from the URL, that took precedence over any name that might be set in extravalues by a plugin. Some plugins might be able to get a better idea of the name and thus we should move defaulting of the name further down after the plugins have had a chance to set it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21devtool: import: new plugin to import the devtool workspaceLeonardo Sandoval
Takes a tar archive created by 'devtool export' and imports (untars) it into the workspace. Currently the whole tar archive is imported, there is no way to limit what is imported. https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510 [YOCTO #10510] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21devtool: append md5sum only if not already presentLeonardo Sandoval
In case the proposed md5sum to be appended to the .devtool_md5 file is already present, do not append it. Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21devtool: export: new plugin to export the devtool workspaceLeonardo Sandoval
By default, exports the whole workspace (all recipes) including the source code. User can also limit what is exported with --included/--excluded flags. As a result of this operation, a tar archive containing only workspace metadata and its corresponding source code is created, which can be properly imported with 'devtool import'. https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510 [YOCTO #10510] Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21devtool: upgrade: enable branch checking when revision is providedChang Rebecca Swee Fun
When devtool upgrade is run on a recipe with revision specified that is not on master branch, and branch isn't set by --srcbranch or -B, then we should get the correct branch and append the branch to the URL. If the revision was found on multiple branches, we will display error to inform user to provide a correct branch and exit. [YOCTO #11484] Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: create: replacing PV in SRCURIStanley Phoong
During recipe creation, it seems that the automation for replacing ${PV} at the SRCURI for tag, (e.g mbed-tls-${PV}) is causing some issue due to PV assuming it's a git source. A fix is implemented in this patch to resolve this issue. Signed-off-by: Stanley Phoong <stanley.cheong.kwan.phoong@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: create: handle git URLs specifying only a tagStanley Phoong
If a git URL is passed to recipetool create with a tag=, recipetool should handle it assuming that the tag is valid. [YOCTO #11393] Signed-off-by: Stanley Phoong <stanley.cheong.kwan.phoong@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: create: being able to set branch when revision is providedChang Rebecca Swee Fun
This change is to improve the buildability of the recipe created by recipetool and devtool. When recipetool create is run on a git URL and a revision specified that is not on master, and "branch=" isn't already in the URL, then we should get the correct branch and append the branch to the URL. If the revision was found on multiple branches and 'master' is not in the list, we will display error to inform user to provide a correct branch and exit. [YOCTO #11389] Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-21recipetool: create: disable PREMIRRORS and MIRRORS by defaultChang Rebecca Swee Fun
When creating new recipes, we are almost certainly fetching a new source rather that something that has already been fetched. I have disable PREMIRRORS and MIRRORS settings in the recipe that created by devtool while leaving an option for users to enable them manually if needed. Since devtool already has this options, we need to ensure that recipetool is able to handle the options passed from devtool. Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16recipetool: create: fix incorrect URL variable usagePaul Eggleton
We have two variables here, srcuri and fetchuri. srcuri is what eventually ends up in the recipe, whereas fetchuri is what we actually pass to the fetcher when we fetch the source within recipetool - sometimes these need to be different particularly for an upcoming patch to handle automatically setting the branch parameter. In OE-Core revision 9a47a6690052ef943c0d4760630ee630fb012153 I erroneously changed the call to scriptutils.fetch_url() to pass srcuri instead of fetchuri - this likely didn't have any ill effect, but change it back to passing fetchuri to match the original intent. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09devtool: sdk-update: fix pulling updates from gitAndrea Galbusera
Commit 4657bc9d165e51981e034e73e7b92552e873eef7 replaced the git pull logic with the git fetch + git reset --hard combo, but resetting to HEAD does not really pull in new commits from remote... Replace with resetting to the upstream branch instead. Signed-off-by: Andrea Galbusera <gizero@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-08-08devtool: fix handling of errors during task executionPaul Eggleton
* If an error is logged while executing a task, we need to ensure we exit instead of assuming everything went OK. * If we receive CookerExit, the server is shutting down and we need to stop waiting for events and probably exit (knotty does this). This will occur if an exception or bb.fatal() happens during an event handler. This fixes a couple of issues highlighted when using devtool upgrade or modify on a non-supported recipe with intel-iot-refkit together with bitbake master, but I'd be very surprised if it were hard to reproduce in other scenarios. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-03wic: improve generating disk system identifierJonathan Liu
This should reduce the chance of generating 0xffffffff as the disk system identifier. Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: Switch to using --use-uuid by defaultTom Rini
The most portable way to specifiy a root device in a disk image that we create is to use PARTUUID rather than /dev/sda2. As background, both GPT and MBR tables provide valid UUID values for each partition and the Linux Kernel contains the logic to parse this value. With this change we can now boot the default disk images when used as any valid block device that the included kernel uses. This for example means that VirtualBox can be used to run vmdk without changes as it uses IDE for the virtual disk controller. Cc: Matt Porter <mporter@konsulko.com> Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: ensure generated disk system identifier is non-zeroJonathan Liu
Zero may be interpreted as no MBR signature present and another partitioning program might install a new MBR signature. Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: add /boot mount point to fstab by defaultEd Bartosh
wic avoided adding /boot to fstab for no reason. This exception was hardcoded in the wic code. There is no need for this as mountpoint in .wks file is an optional field. It can be used only if user wants to have partitions automatically mounted on system boot. [YOCTO #11662] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: rootfs: make copied rootfs uniqueEd Bartosh
Used unique suffix (line number from .wks file) for the copied rootfs directory to avoid possible conflicts. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: rootfs: fix rootfs path reportingEd Bartosh
wic gets rootfs paths from partition object property 'rootfs_dir' and shows them in final report. rootfs plugin sets this property to the temporary path, which causes temporary paths appearing in the report. Changed the code to prevent storing temporary rootfs path in part.rootfs_dir. This should fix the report. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: use absolute paths in rootfs pluginEd Bartosh
Using relative paths can cause copyhardlinktree API to fail as it changes current directory when working. Converted all paths to absolute paths using os.path.realpath. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30wic: copy rootfs directory before changing fstabEd Bartosh
wic updates /etc/fstab on root partition if there are valid mount points in .wks When wic runs from bitbake this can cause incorrect results or even breakage of other tasks working with the same rootfs directory in parallel with do_image_wic. Implemented copying rootfs directory to a temporary location using copyhardlinktree before updating fstab to avoid conflicts with other tasks working with the same rootfs directory. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21wic: add description of --mkfs-extraoptsEd Bartosh
Updated help contents with the description of new wks option --mkfs-extraopts Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-07-21wic: implement wks option --mkfs-extraoptsEd Bartosh
This option specifies extra options to pass to mkfs.<fstype> utilities. [YOCTO #11709] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>