Commit message (Collapse)AuthorAgeFilesLines
* toaster: disable git remote check to allow for firewallsDavid Reyna2018-10-041-2/+5
| | | | | | | | | | | | | | | | | Toaster users behind firewalls that block "git" access usually require developers to add remap rules in their "~/.gitconfig" to remap GIT accesses to HTTP* access. However, there is a "git remote" test in Toaster that is not aware of such remaps, resulting in a false error. For now, disable this nice-to-have check to support this release, and re-enable when we can add remapping accommodations for this test. [YOCTO #12944] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: add 'thud' release to ToasterDavid Reyna2018-10-042-11/+11
| | | | | | | | | Toaster needs to include new YP-2.6 "Thud" release. [YOCTO #12943] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: error logs missing for cli buildsDavid Reyna2018-10-041-4/+5
| | | | | | | | | | | The method 'store_log_event' in 'buildinfohelper.py' always puts log messages from CLI builds into the backlog but never takes them out. The "close" method now forces all backlogged CLI events to be registered. [YOCTO #12813] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/gitsm: fix a typoLaurent Bonnans2018-10-041-1/+1
| | | | | | | 'qbareclone' in place of 'bareclone' Signed-off-by: Laurent Bonnans <laurent.bonnans@here.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: use correct check to decide if the shallow tarball should be ↵Urs Fässler2018-10-041-1/+1
| | | | | | | | | | | unpacked The shallow_tarball check is never true due a check on the caller side. The tarball check is not related to the code on the caller side. Signed-off-by: Urs Fässler <urs.fassler@bbv.ch> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: prevent access to non-existing fullshallow tarballUrs Fässler2018-10-042-1/+15
| | | | | | Signed-off-by: Urs Fässler <urs.fassler@bbv.ch> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/git: improve readability of method need_updateUrs Fässler2018-10-041-4/+9
| | | | | | | | | To improve the readability we extract the different scenarios of why the clonedir needs an update. Signed-off-by: Urs Fässler <urs.fassler@bbv.ch> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-user-manual: Added section on multicong dependenciesScott Rifenbark2018-10-041-0/+76
| | | | | | | | Created a new section to show how to handle dependencies when you are doing a multiple configuration build. Put it in the "Examples" section. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
* bitbake-user-manual: Added BBMULTICONFIG supportScott Rifenbark2018-10-042-7/+58
| | | | | | | | | The BBMULTICONFIG variable is a variable used for BitBake and was not documented in the BitBake Manual glossary. I added the definition. I also added the variable to the example in the section describing how to execute builds for multiple configurations. Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
* bitbake-user-manual: Added new section on multiconfig buildsScott Rifenbark2018-10-042-0/+69
| | | | | | | Created a new example. Also added a figure for the separate configuration file hierarchy Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
* test/fetch: use canonical GitLab URLsRoss Burton2018-10-031-2/+2
| | | | | | | | | GitLab returns a redirect from /foo to /foo.git which older releases of Git don't handle when cloning. These tests don't clone so the redirect works, but let's be consistant with the structure of these URLs and add the .git suffix. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetch2/gitsm.py: Rework the git submodule fetcherMark Hatle2018-09-252-123/+159
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The prior fetcher did not know how to work with MIRRORS, and did not honor BB_NO_NETWORK and similar. The new fetcher approach recursively calls 'gitsm' download on each submodule detected. This ensures that it will go throug the standard download process. Each downloaded submodule is then 'attached' to the original download in the 'modules' directory. This mimics the behavior of: git submodule init but there is no chance it will contact the network without permission. It then corrects upstream reference URIs. The unpack steps simply copies the items from the downloads to the destdir. Once copied the submodules are connected and we then run: git submodule update According to the git documentation, git submodule init can and will modify the project configuration and may connect to the network. Doing the work manually prevents this. (This manual process is allowed based on my reading of the documentation.) See: https://git-scm.com/book/en/v2/Git-Tools-Submodules The small change to the existing test is due to this new code always assuming the code is from a remote system, and not a 'local' repository. If this assumption proves to be incorrect -- code will need to be added to deal with local repositories without an upstream URI. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/layerindexlib/tests/restapi.py: Fix test cases when BB_SKIP_NETTETS=yesMark Hatle2018-09-251-77/+87
| | | | | | | | | | | Change the way the network related tests are skipped, based on how the fetch tests are handled. We introduce a 'skipIfNoNetwork()' function that will report skipped tests through the standard unit test reporting system. [YOCTO #12928] Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/bb/tests/parse.py: Test case was changing chdirMark Hatle2018-09-251-0/+4
| | | | | | | | The test case was changing the current directory, but was never restoring it to the original location. This causes occasional failures in later test cases. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* lib/layerindexlib/tests/cooker.py: Fix topdir to use an absolute (real) pathMark Hatle2018-09-251-1/+1
| | | | | | | | | The test case needs to access test case files. Different versions of python may return absolute or relative locations in __file__. Use the same approach as other test cases in determining the location of the test files. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils: lockfile: Fix infinite loopIoan-Adrian Ratiu2018-09-251-1/+5
| | | | | | | | | | | | | | | | | | | | | | | A nasty corner case leads to a hang when utils.lockfile is called from oe-core's package-manager:deploy_dir_lock (in turn called from rootfs:_create further up the call stack) with "name" owned by root and the user running bitbake has no write access. Because this code runs under pseudo, the UID and EUID of the bitbake worker process are 0, so the os.access(dirname, os.W_OK) returns True i.e. it thinks the path is writable when in fact it's not writable. Only later when trying to open the file an Exception it thrown because the OS prohibits writing, but the Exception is ignored and the open is retried leading to an infinite loop. So this fix is to not ignore the "Permission Denied" exception. An alternative fix would be to replace the os.access() call with an try: open() except() at the beginning of the function. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* daemonize/build: Clean up /dev/null fd handlingRichard Purdie2018-09-212-7/+4
| | | | | | | | | | | | | | At the end of bitbake selftest we see: sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r+' encoding='UTF-8'> Clean up the /dev/null handling to drop the unused entry in build.by and ensure the other open() calls are cleaned up. NULL was unused since http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/build.py?id=4a081b5a52e3d27da8d4b062f3fda292e8d8fb0a back in 2012. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* COW: Fix StopIteration warningRichard Purdie2018-09-211-1/+1
| | | | | | | | | | | Fix the warning: WARNING: lib/bb/data_smart.py:235: DeprecationWarning: generator 'COWDictMeta.iter' raised StopIteration for k, v in self.variables.iteritems(): by using return from the generator, not raising StopIteration. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Ensure setscene tasks that aren't covered get builtRichard Purdie2018-09-071-2/+4
| | | | | | | | | | | | | | | | | | Running "bitbake gconf-native -c cleansstate; bitbake core-image-sato:do_populate_sdk" results in a build where it fails to find gconf-native and fails to build it, merrily trying to build the SDK without gconf being present. The issue is the missing setscene tasks are effectively ignored as the later code in runqueue thinks that since other sstate tasks are present, these 'cover' the missing one. In reality we need to call BB_SETSCENE_DEPVALID to make that decision. To do that we need a "reduced" setscene dependency graph which we don't have in main task graph context. Since that was already done in setscene, we should just assume anything in the non-covered list needs to be built. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker/runqueue: Turn universe warnings into verbnotesRichard Purdie2018-09-072-2/+2
| | | | | | | If the user puts universe on the commandline, they don't really want warnings so use the new verbnote level instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* msg: Add explicit verbnote log levelRichard Purdie2018-09-072-0/+19
| | | | | | | | It has become apparant we need a log level which reaches the console but isn't a warning/error. Add "verbnote" as a way of doing this, behaves as a note but with a higher priority. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* server/process: Various server startup logging fixesRichard Purdie2018-09-043-13/+22
| | | | | | | | | | | | | | | | | | | | | | | There were various problems in the server startup loggin: a) stdout/stderr were not being flushed before forking which could potentially duplicate output b) there were separate buffers for stdout/stderr leading to confusing logs where the entries could be reordered. This was particularly confusing due to the separator the logs use to idendify new messages c) an fd wasn't being closed during server startup meaning if the server failed to start, the closed fd wasn't detected as it was held open by the other reference d) If the pipe was detected as being closed, the code incorrectly retried server startup e) The event code would remap stdout/stderr without flushing them, leading to lose log messages Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* cooker.py: Fix incorrect bb files matched warningMark Hatle2018-09-043-1/+88
| | | | | | | | | | | | | | | | | | In the case of a sublayer of an existing layer, where the sublayer and main layer share a path, the system may not match the paths properly resulting in: No bb files matched BBFILE_PATTERN_sublayer '^/path/main/sublayer' because it has already matched the main layer. Fix this issue by sorting the collection items based on the pattern, using longest to shortest. Obviously regex wildcards could still be an issue but these are typically not used, so this simply fix should work in the existing cases. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* tests/fetch: Update gnome.org urls after upstream changesRichard Purdie2018-08-281-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* fetcher: Fixed remote removal not throwing exception.Paulo Neves2018-08-281-4/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this fix it is assumed that the removal of the remote can only fail because there is not remote to remove. This is a false assumption. Example error which would be ignored: git -c core.fsyncobjectfiles=0 remote rm origin failed with exit code 1, output: Note: A branch outside the refs/remotes/ hierarchy was not removed; to delete it, use: git branch -d master error: could not lock config file config error: Could not remove config section 'remote.origin' Due to the masking of this error a stranger error will be presented to the user, because this time we do not mask the exception: git -c core.fsyncobjectfiles=0 remote add --mirror=fetch origin https://github.com/ptsneves/tl-wn722.git failed with exit code 128, output: fatal: remote origin already exists. The most likely reason that the remote cannot be removed nor modified is that the DL_DIR/git2 does not have permissions compatible with the user running bitbake. This commit fixes: https://bugzilla.yoctoproject.org/show_bug.cgi?id=12728 Signed-off-by: Paulo Neves <ptsneves@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: enable project import and merged Toaster settingsDavid Reyna2018-08-284-42/+48
| | | | | | | | | | | | | | | | Enable the ability to select an existing build directory into Toaster. This opens to the user the backend features of 12823, for command line user compatibility. Enable the ability to select saving Toaster settings in the regular "bblayers.conf" and "local.conf" instead of the default "toaster_bblayers.conf" and "toaster.conf". This opens to the user the backend features of 12821, for command line user compatibility. [YOCTO #12902] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/layerdetails.js: don't hide local layer infoAwais Belal2018-08-281-1/+2
| | | | | | | | | | | | | | | The local layer info (provided through custom fixtures) should not be hidden. It is better to handle it in the same manner as an imported layer, otherwise the layer path and dependency info is not shown. The layer editing fields are handled in the html side of things appropriately so this does not harm that implementation. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/models.py: allow local paths for custom recipe's baseAwais Belal2018-08-281-0/+7
| | | | | | | | | | | | | In a case where the layer source is local only and the recipe is not yet built, we can search for the path with layer's local_source_dir, and if available that should be used rather than just skipping the scenario. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/checksettings: allow CUSTOM_XML_ONLY setting through envAwais Belal2018-08-281-1/+2
| | | | | | | | | | | | | This change allows the CUSTOM_XML_ONLY toaster setting to be provided through the environment so the user can do this without mingling with the settings.xml, for scenarios where modifying settings.xml is not achievable. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toastergui/newproject.html: fix release divsAwais Belal2018-08-281-2/+2
| | | | | | | | | | | | | | | The release drop down divs are not being closed appropriately, which showed adverse reactions on the UI that aligned the "Create project" button with the left edge of the screen without any margins. This fixes these divs which in turn aligns the button appropriately. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/widgets.py: avoid divide by zero issuesAwais Belal2018-08-281-7/+12
| | | | | | | | | | | | | | | There can be cases where the variables being used to divide in build percentage expressions can be zero. For example, a setup consisting of only local repos will have repos_to_clone=0 and will generate a divide by zero scenario. Fix this by checking the divisor in such cases. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: allow TOASTER_DIR to be overridden from cmdlineAwais Belal2018-08-281-2/+7
| | | | | | | | | | | | | | | | | TOASTER_DIR is used for higher level toaster artifacts such the SQL DB and creating toaster internal build directories for projects. Prior to this change it was evaluated as `dirname $BUILDDIR` and user had no control over it. This change allows to override this variable from the command line for more flexibility. The variable defaults to its original setting if the optional argument is not passed. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: use a more flexible way to find bitbakeAwais Belal2018-08-281-1/+12
| | | | | | | | | | | | | | | | | | | | | | | The current mechanism for finding the bitbake binary assumes a directory structure which is identical to poky, where oe-core's meta and bitbake directories are at the same level. There can be a case where bitbake is used from elsewhere and in such cases the above mentioned assumption fails to hold, whereas this is totally allowed by the oe-init-build-env script which can take bitbakedir as an argument. The better approach is to allow bitbake to be derived from PATH, while keeping the older mechanism in place so it can be removed after tests are done in various environments. This makes more sense as toaster has also been launched from the same bitbake instance that is the one in PATH. [YOCTO #12891] Signed-off-by: Awais Belal <awais_belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: allow pokydirname to be evaluated when all layers are localAwais Belal2018-08-281-3/+15
| | | | | | | | | | | | | | | | | | Toaster depends on pokydirname for identifying the location of the oe-init-build-env script (and there might be other purposes in the future). The problem with current approach is that it only checks/sets the variable with git based repos, whereas toaster provides mechanisms to allow having layers that are all locally available. The evaluation of the variable fails in such scenarios, so use a more flexible mechanism in this case and try to locate poky in the local layers as well, if not already set. [YOCTO #12891] Signed-off-by: Awais Belal <awais.belal@mentor.com> Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* parse/ast: ensure saved event handlers really do get restoredPaul Eggleton2018-08-241-18/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In finalize() we save event handlers, register the ones relevant to the recipe being finalised, trigger events, and then restore the handlers so that one recipe's custom handlers (actually implemented within a class inherited by the recipe) do not affect other recipes. However, if an exception occurs during parsing, the saved handlers were not being restored. Use a try...finally block to ensure that the handlers are always restored. This issue became apparent since in OpenEmbedded-Core we have recently introduced a find_intercepts() handler for the bb.event.RecipePreFinalise event in image-postinst-intercepts.bbclass that images and old-style SDK recipes will end up inheriting. So far it doesn't seem that the the error has manifested itself in normal builds, but when parsing OE-Core recipes in the OE layer index it has: core-image-rt-* image recipes were parsed which in the default configuration raise SkipRecipe. The next non-image recipe that is parsed will trigger a real exception, because the find_intercepts() handler is still registered and gets fired, but in the context of the new recipe the POSTINST_INTERCEPTS_PATHS variable is not set, and the code in find_intercepts() is written with the reasonable assumption that that isn't possible given that the class itself sets a default, and thus it fails. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake: Bump version 1.39.0 -> 1.39.1Richard Purdie2018-08-232-2/+2
| | | | Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* daemonize: Flush stdio on exitJoshua Watt2018-08-211-2/+7
| | | | | | | | | | | | | In spite of a comment suggesting otherwise, os._exit() does not flush buffered output from file descriptors before exiting the process like os.exit() does. This means that any un-flushed output is lost in the daemon process, in particular the traceback from any thrown exceptions, making debugging exceptions in the daemon quite difficult. The solution is to flush stdout and stderr before exiting. Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster: Fix comparison in recipe templateKarsten Strand2018-08-211-1/+1
| | | | | | | | | Use == instead of = when comparing task outcome to OUTCOME_FAILED. Prior to this fix the recipe template would cause a TemplateSyntaxError exception. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Toaster: Implement the project-specific feature and releated enhancements ↵David Reyna2018-08-2029-55/+1900
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and defects. Here is the primary driving enhancement: * Bug 12785 - Support Project Specific configuration for external tools (e.g. ISS, Eclipse) - Isolated project-specific configuration page (full Toaster context hidden) - Support for new project, reconfigure existing project, and import existing command line project - Ability to define variables (e.g. image recipe) and pass them back to external GUI - Ability to execute the cloning phase, so that external GUI receive a buildable project - Ability to call back to the external GUI when updates are completed and ready - Compatibility of above projects with the normal full Toaster interface - Ability to pass to a 'complete' or 'cancel' web page so that the external GUI can immediately stop that Toaster instance, and not leave dangling servers nor edit sessions open Here are the supporting enhancements, where at least the back end is implemented: * Bug 12821 - Make Toaster conf changes compatible with command line usage * Bug 12822 - Support importing user changes to conf files into Toaster * Bug 12823 - Support importing user build directories into Toaster * Bug 12824 - Scan imported layers for content so that they are immediately available * Bug 12825 - show layer clone item in progress bar Here are defects fixed: * Bug 12817 - builddelete.py requires explicit 'add_arguments' * Bug 12818 - Remove orphaned imported layers when project is deleted * Bug 12826 - fix imported layer management * Bug 12819 - build using selected bitbake env, not Toaster's env * Bug 12820 - Toaster randomizes the layer order in toaster_bblayers.conf [YOCTO #12785] Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bb/utils.py: add break_hardlinks helperRasmus Villemoes2018-08-161-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | bb.utils.copyfile is called in a few places with identical src and dst in order to create an st_nlinks==1 version of the file. That that even works relies on an implementation detail of copyfile (namely, that it creates a temporary file and then does a rename). Moreover, it's a waste of time if the file already has st_nlinks==1. So create a helper that optimizes away the copy in the st_nlinks==1 case. Of course, this helper relies on the same implementation detail, but that's now contained within bb.utils itself. To test that we do at least sometimes hit the no-copy path, I tested locally with if sstat[stat.ST_NLINK] == 1: + bb.note("Woohoo, 2*%d bytes I/O avoided" % sstat[stat.ST_SIZE]) return True (and the obvious places in oe-core patched), and the do_package log files are indeed filled with woohoo notes. Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Move decision if a task can be started to one common placeAndreas Müller2018-08-151-4/+8
| | | | | Signed-off-by: Andreas Müller <schnitzeltony@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* utils/md5_file: don't iterate line-by-lineRoss Burton2018-08-151-4/+9
| | | | | | | | | | | | | Opening a file in binary mode and iterating it seems like the simple solution but will still break on newlines, which for binary files isn't really useful as the size of the chunks could be huge or tiny. Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't want to fill up memory: use mmap() to open the file and read the file in 8k blocks. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* checksum: sanity check path when recursively checksummingRoss Burton2018-08-141-0/+2
| | | | | | | | In case something goes tragically wrong, catch a request to checksum / and refuse. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* event.py: fix typoKai Kang2018-08-141-1/+1
| | | | | Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* runqueue: Allow the hash validate function to have an idea of build completionRichard Purdie2018-08-071-0/+5
| | | | | | | | | | | | There is an oversight in the current hash validation API in that the function can't know how many setscene tasks already completed. Rather than trying to add additional parameters to the function, causing incompatibilities, store the value in the datastore. This is useful to allow build status reporting to the user for figures on sstate reusage and build completion. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* toaster/orm/management/commands/lsupdates.py: Use new layerindexlib moduleMark Hatle2018-08-011-131/+97
| | | | | | | | | | | | Change lsupdates.py to use the new layerindexlib module to load the data from the public layer index. It still does all of the manual parsing. This is intended to be a stop gap until the toaster can use the module itself to manage the data. Everything else is functionally equivalent to the prior version. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bitbake-layers: disable parsing for layerindex commandsMark Hatle2018-08-011-2/+2
| | | | | | | | | These don't need to access recipe information, so let's not waste the user's time parsing all recipes. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bblayers/layerindex.py: Switch to use the new layerindexlib classMark Hatle2018-08-011-185/+119
| | | | | | | | | | | | | | | | | | | Display changes: The output will now include references to the layers that the user already has on their system. It does this by querying the cooker derived index. The code that enables this behavior is labeled as 'TODO' currently. As part of the work we need to make a final determination if this is the desired output. Also changed the default branch to no longer define itself as 'master'. When the user does NOT set a branch, the default is now the 'LAYERSERIES_CORENAMES', and if that doesn't exist 'master'. This is subtly different in behavior, but more consistent with user expectations. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* layerindexlib: Initial layer index processing module implementationMark Hatle2018-08-0117-1/+2866
| | | | | | | | | | | | | | | | | | | | The layer index module is expected to be used by various parts of the system in order to access a layerindex-web (such as layers.openembedded.org) and perform basic processing on the information, such as dependency scanning. Along with the layerindex implementation are associated tests. The tests properly honor BB_SKIP_NETTESTS='yes' to prevent test failures. Tests Implemented: - Branch, LayerItem, LayerBranch, LayerDependency, Recipe, Machine and Distro objects - LayerIndex setup using the layers.openembedded.org restapi - LayerIndex storing and retrieving from a file - LayerIndex verify dependency resolution ordering - LayerIndex setup using simulated cooker data Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* bblayers/layerindex.py: Fix addition of layersMark Hatle2018-08-011-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | When a layer is added it needs to be in a list, otherwise the system will error such as: Specified layer directory / doesn't contain a conf/layer.conf file Additionally, instead of calling the add layer function over and over, it is better to add all of the new content in one command. Otherwise the order is important as the system now checks if the layer can be added. For instance, trying to add meta-python: Layer Required by Git repository Subdirectory =================================================================================================================== meta-python - git://git.openembedded.org/meta-openembedded meta-python meta-oe meta-python git://git.openembedded.org/meta-openembedded meta-oe openembedded-core meta-python git://git.openembedded.org/openembedded-core meta Adding layer "meta-python" (.../oe-core/meta-openembedded/meta-python) to conf/bblayers.conf ERROR: Layer 'meta-python' depends on layer 'openembedded-layer', but this layer is not enabled in your configuration The system would try to add meta-python before the dependent meta-oe. Adding them both at the same time resolves this issue. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>