update.py: improve fetch failure handling

* Report layer which failed to fetch in error message
* Don't retry fetching a repo if it already failed for another layer
  (where more than one layer is in the same repository)
* Exit immediately if all fetches failed

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>

2 files changed, 6 insertions, 4 deletions

diff --git a/TODO b/TODO
index 43e5fe2cd1..70b8526e7a 100644
--- a/TODO
+++ b/TODO
@@ -3,7 +3,6 @@ TODO:
 * Duplication of first maintainer when editing to add a second?
 * Try to re-use existing recipe record with same PN instead of deleting and re-creating (if within same layer)
 * meta-arago-extras is preferred over meta-networking e.g. for crda; probably need an explicit field for priority order
-* Update script does not report which layer failed with -q when fetch fails
 * Document macros for URL fields
 
 Later:
@@ -12,7 +11,6 @@ Later:
 * Ability for reviewers to comment before publishing a layer?
 * Add link to the all layers and all recipes tables from the layer details page?
 * Prevent SMTP failures from breaking submission process
-* Update script still retries a fetch when repo has already failed
 * All-branch search/results so you can see version availability of recipes in all branches at once?
 * Rawrecipes branch support
 * Display no-results found message when search does not return any results (all tables)
diff --git a/layerindex/update.py b/layerindex/update.py
index c32d8cc684..60baa92ff4 100755
--- a/layerindex/update.py
+++ b/layerindex/update.py
@@ -233,7 +233,7 @@ def main():
             # Handle multiple layers in a single repo
             urldir = layer.get_fetch_dir()
             repodir = os.path.join(fetchdir, urldir)
-            if not layer.vcs_url in fetchedrepos:
+            if not (layer.vcs_url in fetchedrepos or layer.vcs_url in failedrepos):
                 logger.info("Fetching remote repository %s" % layer.vcs_url)
                 out = None
                 try:
@@ -242,11 +242,15 @@ def main():
                     else:
                         out = runcmd("git fetch", repodir)
                 except Exception as e:
-                    logger.error("fetch failed: %s" % str(e))
+                    logger.error("Fetch of layer %s failed: %s" % (layer.name, str(e)))
                     failedrepos.append(layer.vcs_url)
                     continue
                 fetchedrepos.append(layer.vcs_url)
 
+        if not fetchedrepos:
+            logger.error("No repositories could be fetched, exiting")
+            sys.exit(1)
+
         logger.info("Fetching bitbake from remote repository %s" % settings.BITBAKE_REPO_URL)
         if not os.path.exists(bitbakepath):
             out = runcmd("git clone %s %s" % (settings.BITBAKE_REPO_URL, 'bitbake'), fetchdir)