aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oe
AgeCommit message (Collapse)Author
2015-02-23sstatesig: Only squash dependencies for allarch packagegroupsRichard Purdie
The idea of squashing packagegroup dependencies was to avoid allarch packages rebuilding upon tune/arch changes. Now that the allarch class inclusion is conditional, we can narrow down the packagegroup squashing to be specifically applied to allarch recipes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit bd8018792155ae842952432900c8431feeaffe30) Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
2015-02-23sstatesig: Move saferecipedeps handling to be earlierRichard Purdie
We want to use the saferecipedeps handling code to allow gcc-cross-* to work on multiple different tunes. Its currently in target only code so it needs to be earlier to allow it to work on native-> target dependencies. This change has no effect on existing uses but makes gcc-cross become shared as desired. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 9e03db2dfab0b534b86fd48c9190b2d7d0d21238) Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
2015-02-23sstatesig: include native/cross/nativesdk deps in target signaturesMartin Jansa
* I don't have any real evidence or good statistics for this, but when comparing signature dumps from my big bitbake world builds I usually see a lot of rebuilds caused by changes in .bbclasses and only very rare would be the case where oe-core upgrade brings changes in -native recipes and no change in .bbclasses used from target recipes * changing the default to include them shouldn't cause significant increase in rebuilds and sstate reuse a bit safer * people working on toolchain (e.g. using gcc from AUTOREV) can easily extend sstate_rundepfilter to ignore them again (it's easier than removing existing filter), example how add own signature handler in your layer is here: https://github.com/openwebos/meta-webos/commit/9ac3a7c803e7793b3274e4998f167b6278db8042 Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 336a7897e39b9e42dcfcba9e2520ea96b0c6a8d6) Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
2015-02-23sstatesig: Correctly handle matches spanning stamps and sstatedirRichard Purdie
By resetting filesdates at this point, we lose matches from stamps which may not have been in sstatedir. When we don't have hashes specicifed, its better to return all matches and have the caller decide which are relavent and which are not since this function has no ability to decide. There will almost always be one match from stamps we need to keep and refer to. (From OE-Core rev: f4c1c9ad2c7e944d4926d0629611da97f9df6a9a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8629844640a5a4c6a9d9375e7b637bcbf3451c4c) Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
2015-02-23sstatesig: Ensure we return all matches for find_sigdataRichard Purdie
When the hashes to find isn't specified we need to return matches from both the sstate cache and the local stamps directory regardless of how many we've found so far. If we don't do this, we can miss stamps and the comparison is less accurate/incorrect. (From OE-Core rev: 08a074e11e2d517b81ca71fd9bda65297bb015a7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 04fc682af7c1657b1c9f37b66a78c4ffc8066e24) Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
2015-02-23lib/oe/sstatesig: Ensure packagegroups don't continually rebuildRichard Purdie
packagegroups are allarch and shouldn't change depending on the target or machine selected. In general they should have good stable namespaces for their dependencies. As such we can exclude them from rebuilding when dependency checksums change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Saul Wold <sgw@linux.intel.com> (cherry picked from commit 80b065ff46322ec0cad039dfd9eb2d010168dba6) Signed-off-by: Andreas Oberritter <obi@opendreambox.org> (merge fixes)
2014-04-11sstatesig: Anchor inherits class testsRichard Purdie
This avoids a nasty sstate hash corruption issue where the fact the testimage bbclass was inherited meant that the checksum changed due to testimage.bbclass being confused with image.bbclass. This patch anchors the bbclass names to avoid this confusion. (From OE-Core master rev: 943a75a4f3b6877e4092dae14b59b7afef8cad3d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2014-04-11sstatesig.py: Fix image regeneration issueRichard Purdie
With the "ABI safe" recipes, we've been excluding those from signatures. This is fine in the general case but in the specific case of image recipes it breaks. A good test case is the interfaces file. Editting this causes init-ifupdown to rebuild but not an image containing it (e.g. core-image-minimal). We need to ensure the checksums are added to the image recipes and this change does that. (From OE-Core master rev: fd085f15e7cd093953f974f69277e130174d551d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-04Revert "buildhistory_analysis: fix error when comparing image contents"Richard Purdie
This reverts commit 5b616aa7b618f6ed221d6fa9738220a2c2349f7d.
2014-01-26meta/lib/oe/terminal.py: Don't pass non-supported '--disable-factory' flag ↵Jacob Kroon
to gnome-terminal By default, all GNOME terminals share a single process, reducing memory usage. This can be disabled by starting gnome-terminal with the --disable-factory option However, gnome-terminal in Fedora 20 does no longer support the '--disable-factory' flag, so remove it. As the support for 'mate' terminals was added as a copy of the gnome code in 8cc078a9c679845464c59028f584d7aba098cc1f, remove the flag here aswell. (From OE-Core master rev: e8dca725ed8211a874472300a3ed50e494039ab9) Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2013-12-12buildhistory_analysis: fix error when comparing image contentsPaul Eggleton
OE-Core commit b7de1eaac9eed559b2d68058f5de67de74a6cb58 added an extra argument to the compare_dict_blobs() function but missed adding the argument to one call to compare two versions of the image-info.txt file. (From OE-Core master rev: 24a45d752c3e3d0d8b59c040355e4fe7de22b041) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2013-10-11lib/oe/path: Fix copytree functions for tar 1.27Richard Purdie
tar version 1.27 returns: tar: --same-order option cannot be used with -c with the commandlines we have been using. We can remove the -s option (which is --same-order) to remove the error. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-13bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific ↵Richard Purdie
directory Currently we have a hierarchy of pkgdata directories and the code has to put together a search path and look through each in turn until it finds the data it needs. This has lead to a number of hardcoded paths and file globing which is unpredictable and undesirable. Worse, certain tricks that should be easy like a GL specific package architecture become problematic with the curretn search paths. With the modern sstate code, we can do better and construct a single pkgdata directory for each machine in just the same way as we do for the sysroot. This is already tried and well tested. With such a single directory, all the code that iterated through multiple pkgdata directories and simply be removed and give a significant simplification of the code. Even existing build directories adapt to the change well since the package contents doesn't change, just the location they're installed to and the stamp for them. The only complication is the we need a different shlibs directory for each multilib. These are only used by package.bbclass and the simple fix is to add MLPREFIX to the shlib directory name. This means the multilib packages will repackage and the sstate checksum will change but an existing build directory will adapt to the changes safely. It is close to release however I believe the benefits this patch give us are worth consideration for inclusion and give us more options for dealing with problems like the GL one. It also sets the ground work well for shlibs improvements in 1.6. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01meta: Don't use deprecated bitbake APIRichard Purdie
These have been deprecated for a long time, convert the remaining references to the correct modules and prepare for removal of the compatibility support from bitbake. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26oe.types: add 'path' typeChristopher Larson
- path normalization ('normalize' flag, defaults to enabled) - existence verification for paths we know should exist ('mustexist' flag) - supports clean handling of relative paths ('relativeto' flag) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-07-30lib/oe/terminal.py: add support for MATE desktop terminalsAndre McCurdy
A simple clone of the corresponding Gnome class. Without this, devshell fails completely on a default installation of MATE desktop Linux Mint 15. Signed-off-by: Andre McCurdy <andre.mccurdy@entropic.com> Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-07-02TmuxRunning: handle multi-word commandsTyler Hall
Just as in f8ed7446755eeb88191e16749350efa1e7e6197c, tmux wants a single argument for its command. This applies to the "split-window" command as well as "new." Note that this alone is not enough to fix the TmuxRunning devshell when using pseudo because tmux does not preserve the environment that pseudo requires. Signed-off-by: Tyler Hall <tylerwhall@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-07-02lib/oe/lsb.py: fall back to /etc/os-release for host distro IDPaul Eggleton
The new standard for host distribution identification [1] is /etc/os-release, and a number of newer distributions provide this file, so add support for this in order to pick up more distributions. Additionally, handle "rolling release" style distributions that don't report a version number, e.g. Arch Linux. With this change we can identify the most common distributions, so this should satisfy [YOCTO #4271]. Note that this doesn't imply support for these distros as build hosts, just that we can identify them. [1] http://www.freedesktop.org/software/systemd/man/os-release.html Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-06-11handle two-word commands with tmuxPeter Seebach
Trying to make a devshell using tmux can fail because "tmux new" expects a single command, not a series of arguments. It does, however, split strings in a suitable way. So you can quote the command. The failure mode is particularly arcane, in that you end up with a message like: ERROR: Unable to spawn terminal auto: \ Execution of 'pseudo /bin/bash' failed with exit code 1: usage: new-session [-d] [-n window-name] [-s session-name] \ [-t target-session] [command] which is confusing because there's no "new-session" anywhere in sight (that's actually "tmux new"), and because what failed to execute wasn't either pseudo or bash. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07test_utils: import functions directly for concisenessRoss Burton
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-06-07test-utils: handle import bb failing and skip the testRoss Burton
Instead of reporting an error when bb cannot be imported, skip the test instead. This makes it a lot easier to iterate a test suite when we don't care about this particular test. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-05-30utils: add trim_version() functionRoss Burton
Add a helper function that returns just the first <num_parts> of <version>, split by periods. For example, trim_version("1.2.3", 2) will return "1.2". This should help reduce the spread of numerous copies of this idea across classes and recipes. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-15cachedpath.py: global name 'error' is not definedRobert Yang
The error is not global since we don't use "from os import *", so it should be os.error. [YOCTO #4489] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-10classes/buildhistory: track contents of selected files in imagesPaul Eggleton
Add a BUILDHISTORY_IMAGE_FILES variable specifying a space-separated list of files within an image to copy into buildhistory, so that changes to them can be tracked. Typically this would be used for configuration files, and by default this includes /etc/passwd and /etc/group, but the user is free to extend this list by setting the variable in local.conf. Implements [YOCTO #4154]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09class/lib: Fix up various file access methodsRichard Purdie
There are various bits of cruft that have built up around our file accesses. This patch cleans some of them up, specifically: * Remove pointless "from __builtin__ import file" * Use open(), not file() * Wrap file usage in a with container to ensure files are closed * Add missing .close() calls in some cases Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09classes/lib: Fix getcmdstatus breakageRichard Purdie
I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets add a wrapper and have this work in both worlds. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09image/kernel-module-split/eglibc-ld.inc: Remove has_key() usageRichard Purdie
The has_key() attribute has been removed in python 3 since there is better syntax available. Use the improved syntax. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09sanity/patch.py: Remove commands module usageRichard Purdie
The commands module is removed in python3. Use the subprocess module instead and the pipes module to replace the mkargs usage. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09meta/lib/oe: Replace StandardError with ExceptionRichard Purdie
StandardError is removed in python3, replace with Exception class instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09classes/lib: Update to use print as a function callRichard Purdie
In python 3 print is a function call. In some cases bb.note is a more appropriate call to make. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09classes/recipes/lib: Fix various python whitespace issuesRichard Purdie
There are some left over tab characters in the python functions. This removes them and resolves python 3 errors. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09clases/lib: Use modern exception syntaxRichard Purdie
Update older code to use modern exception handling syntax which is the form accepted by python 3. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-03path.py: Deal with race issueRichard Purdie
The change to use copyhardlinktree in some of the sstate code instead of copytree exposed a race condition. This is due to cp failing if it finds a directory doesn't exist yet some other process creates it while cp was trying to create it itself. tar doesn't error in this case. To fix this we need to create the directory structure with tar, then use cp to hardlink the files. Messy but probably worth doing. I also took the opportunity to remove src_bak since the code is neater without it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-01sstate.bbclass: make hard links for staging filesRobert Yang
Make hard links for staging files instead of copy to save the disk space (3G will be saved for a core-image-sato build), and it doesn't affect much on the build time. The following directories are affected: 1) The sysroot 2) The DEPLOY_DIR 3) The pkgdata [YOCTO #4372] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-04-15oe.terminal: add tmux classesChristopher Larson
This adds two new Terminal classes. It's separated into two, so that opening a split inside a tmux window is preferred to the other terminal types, but opening a tmux session is prioritized only slightly higher than screen. - tmuxrunning: Open a new pane in the current running tmux window. Requires that the TMUX variable be added to the env whitelist to use it. - tmux: Open a new tmux session Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-05lib/oe/classextend.py: avoid extending any kernel packagePaul Eggleton
For multilib and other uses of classextend, we don't want any dependencies on kernel packages to be extended since there should only be one kernel variant. Fixes [YOCTO #2918] (where kernel-dev was being extended.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-04utils: add helper to get all non-system packagesRoss Burton
For example if PACKAGES is "foo foo-data foo-dev foo-doc", this will return "foo-data". Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-04oe.terminal: add tmux classesChristopher Larson
This adds two new Terminal classes. It's separated into two, so that opening a split inside a tmux window is preferred to the other terminal types, but opening a tmux session is prioritized only slightly higher than screen. - tmuxrunning: Open a new pane in the current running tmux window. Requires that the TMUX variable be added to the env whitelist to use it. - tmux: Open a new tmux session Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-29meta/lib/oe/lsb.py: extract only the needed info from lsb-releaseCristian Iorga
Instead of running lsb_release -a, a lsb_release -ir will be run. This will prevent issue with distros that don't have all the needed info in /etc/lsb-release file, in which case lsb_release won't generate an error code. Partial fix for [YOCTO #4071] Signed-off-by: Cristian Iorga <cristian.iorga@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-29meta/lib/oe/lsb.py: fix data extraction from /etc/lsb-releaseCristian Iorga
In some cases, /etc/lsb-release file is used to extract info about poky build host machine. But the strings are not stripped of end of line special characters. As such, when this info is concatenated and used as a directory entry in sstate_cache, this is an issue. Usually, this issue is masked by the fact that distro related info is extracted from the output of lsb_release command. In case of Yocto Linux, running "lsb_release -a" will give an error code because CODENAME info is not present. As such, bitbake will extract the info from /etc/lsb-release, running into the above issue. Consequence is that building under BA will crash. Partial fix for [YOCTO #4071] Signed-off-by: Cristian Iorga <cristian.iorga@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-18package: Add cachedpath optimisationRichard Purdie
Currently, various standard library operations like os.walk(), os.path.isdir() and os.path.islink() each call stat or lstat which involves a syscall into the kernel. There is no caching since they could conceivably have changed on disk. The result is that for something like the do_package task of the kernel we're spending over two minutes making 868,000 individual stat calls for 23,000 files. This is suboptimal. This patch adds lib/oe/cachedpath.py which are a set of replacement functions for these operations which use cached stat data rather than hitting the kernel each time. It gives a nice performance improvement halving the build time of the kernel do_package. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-12lib/oe/lsb: enable getting distro ID when lsb_release is not installedPaul Eggleton
If lsb_release is not installed (as it may not be on headless/minimal installations on distros whose LSB package has a long list of dependencies) we need to gather the information directly from files in /etc. Fixes [YOCTO #4012]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-09base.bbclass: don't backfill features that already existRoss Burton
It's too easy to cause rebuilds because the DISTRO_FEATURES have changed in meaningless ways (such as re-ordering or duplicate items). Help stop this by checking if the feature to be back-filled is already present. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-12distro_check: Remove creation of empty Meego filelist.Anders Roxell
This fixes permissions errors when running distrodata. Signed-off-by: Anders Roxell <anders.roxell@enea.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2013-02-11lib/oe/path.py: support missing directory components in realpath()Enrico Scholz
Some use cases in OE operate on symlinks which dangling path components. Assume that these are directories instead of raising ENOENT. Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11lib: implemented oe.path.realpath()Enrico Scholz
Various parts of the buildsystem have to work with symlinks. Resolving them is not trivial because they are always relative to a sysroot directory. Patch adds a function which returns the destination of a symlink by assuming a given path as the / toplevel directory. A testsuite was added too. Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11buildhistory_analysis: handle more R* variablesPaul Eggleton
Report changes to RPROVIDES, RREPLACES, and RCONFLICTS. As RSUGGESTS isn't widely used and isn't of huge concern if it changes, it is not reported by default. Implements [YOCTO #3391]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06package: Process package stripping in parallelRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06package.bbclass: Multithread per file dependency generation codeRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06sstate/path.py: Add copyhardlinktree() function and use for performance ↵Richard Purdie
optimisation Add a function which copys a tree as a set of hardlinks to the original files, then use this in sstate to reduce some of the overhead of sstate package creation since the file isn't actually copied. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-08 04:17:31 +0000