bitbake/lib/bb/fetch/*: Implement special 'now' handling for BitBake

    Patch courtsey Justin Patrin to teach BitBake SRCDATE="now" handling.
    If SRCDATE is set to now we will not look at the temporary DL_DIR
    for a saved tarball as we want to download a new version. OpenEmbedded
    should consider stamping the do fetch phase.

    Justin thanks for the patch and sorry for all the trouble


bitbake/doc/manual/usermanual.xml: Document the 'now' method for svn,svk and cvs

5 files changed, 34 insertions, 9 deletions

diff --git a/doc/manual/usermanual.xml b/doc/manual/usermanual.xml
index 756f6aca9..97a7bf66e 100644
--- a/doc/manual/usermanual.xml
+++ b/doc/manual/usermanual.xml
@@ -251,9 +251,9 @@ a per URI parameters separated by a <quote>;</quote> consisting of a key and a v
 
         <section>
             <title>CVS File Fetcher</title>
-            <para>The URN for the CVS Fetcher is <emphasis>cvs</emphasis>. This Fetcher honors the variables <varname>DL_DIR</varname>, <varname>SRCDATE</varname>, <varname>FETCHCOMMAND_cvs</varname>, <varname>UPDATECOMMAND_cvs</varname>. <varname>DL_DIRS</varname> specifies where a temporary checkout is saved, <varname>SRCDATE</varname> specifies which date to use when doing the fetching, <varname>FETCHCOMMAND</varname> and <varname>UPDATECOMMAND</varname> specify which executables should be used when doing the CVS checkout or update.
+            <para>The URN for the CVS Fetcher is <emphasis>cvs</emphasis>. This Fetcher honors the variables <varname>DL_DIR</varname>, <varname>SRCDATE</varname>, <varname>FETCHCOMMAND_cvs</varname>, <varname>UPDATECOMMAND_cvs</varname>. <varname>DL_DIRS</varname> specifies where a temporary checkout is saved, <varname>SRCDATE</varname> specifies which date to use when doing the fetching (the special value of "now" will cause the checkout to be updated on every build), <varname>FETCHCOMMAND</varname> and <varname>UPDATECOMMAND</varname> specify which executables should be used when doing the CVS checkout or update.
             </para>
-            <para>The supported Parameters are <varname>module</varname>, <varname>tag</varname>, <varname>date</varname>, <varname>method</varname>, <varname>localdir</varname>, <varname>rsh</varname>. The <varname>module</varname> specifies which module to check out, the <varname>tag</varname> describes which CVS TAG should be used for the checkout by default the TAG is empty. A <varname>date</varname> can be specified to override the SRCDATE of the configuration to checkout a specific date. <varname>method</varname> is by default <emphasis>pserver</emphasis>, if <emphasis>ext</emphasis> is used the <varname>rsh</varname> parameter will be evaluated and <varname>CVS_RSH</varname> will be set. Finally <varname>localdir</varname> is used to checkout into a special directory relative to <varname>CVSDIR></varname>.
+            <para>The supported Parameters are <varname>module</varname>, <varname>tag</varname>, <varname>date</varname>, <varname>method</varname>, <varname>localdir</varname>, <varname>rsh</varname>. The <varname>module</varname> specifies which module to check out, the <varname>tag</varname> describes which CVS TAG should be used for the checkout by default the TAG is empty. A <varname>date</varname> can be specified to override the SRCDATE of the configuration to checkout a specific date.  The special value of "now" will cause the checkout to be updated on every build.<varname>method</varname> is by default <emphasis>pserver</emphasis>, if <emphasis>ext</emphasis> is used the <varname>rsh</varname> parameter will be evaluated and <varname>CVS_RSH</varname> will be set. Finally <varname>localdir</varname> is used to checkout into a special directory relative to <varname>CVSDIR></varname>.
 <screen><varname>SRC_URI</varname> = "cvs://CVSROOT;module=mymodule;tag=some-version;method=ext"
 <varname>SRC_URI</varname> = "cvs://CVSROOT;module=mymodule;date=20060126;localdir=usethat"
 </screen>
@@ -276,7 +276,7 @@ will be tried first when fetching a file if that fails the actual file will be t
         <section>
             <title>SVK Fetcher</title>
             <para>
-            <emphasis>Currently NOT suppoered</emphasis>
+            <emphasis>Currently NOT supported</emphasis>
             </para>
         </section>
 
@@ -284,7 +284,7 @@ will be tried first when fetching a file if that fails the actual file will be t
             <title>SVN Fetcher</title>
             <para>The URN for the SVN Fetcher is <emphasis>svn</emphasis>.
             </para>
-            <para>The Variables <varname>FETCHCOMMAND_svn</varname>, <varname>DL_DIR</varname> are used by the SVN Fetcher. <varname>FETCHCOMMAND</varname> contains the subversion command, <varname>DL_DIR</varname> is the directory where tarballs will be saved.
+            <para>This Fetcher honors the variables <varname>FETCHCOMMAND_svn</varname>, <varname>DL_DIR</varname>, <varname>SRCDATE</varname>. <varname>FETCHCOMMAND</varname> contains the subversion command, <varname>DL_DIR</varname> is the directory where tarballs will be saved, <varname>SRCDATE</varname> specifies which date to use when doing the fetching (the special value of "now" will cause the checkout to be updated on every build).
             </para>
             <para>The supported Parameters are <varname>proto</varname>, <varname>rev</varname>. <varname>proto</varname> is the subversion prototype, <varname>rev</varname> is the subversions revision.
             </para>
diff --git a/lib/bb/fetch/__init__.py b/lib/bb/fetch/__init__.py
index a1d7396ce..c69de2bb0 100644
--- a/lib/bb/fetch/__init__.py
+++ b/lib/bb/fetch/__init__.py
@@ -173,6 +173,7 @@ class Fetch(object):
         """
         tarpath = os.path.join(data.getVar("DL_DIR", d, 1), tarfn)
         if os.access(tarpath, os.R_OK):
+            bb.debug(1, "%s already exists, skipping checkout." % tarfn)
             return True
 
         pn = data.getVar('PN', d, True)
@@ -192,6 +193,28 @@ class Fetch(object):
         return False
     try_mirror = staticmethod(try_mirror)
 
+    def check_for_tarball(d, tarfn, dldir, date):
+        """
+        Check for a local copy then check the tarball stash.
+        Both checks are skipped if date == 'now'.
+
+        d Is a bb.data instance
+        tarfn is the name of the tarball
+        date is the SRCDATE
+        """
+        if "now" != date:
+            dl = os.path.join(dldir, tarfn)
+            if os.access(dl, os.R_OK):
+                bb.debug(1, "%s already exists, skipping checkout." % tarfn)
+                return True
+
+            # try to use the tarball stash
+            if Fetch.try_mirror(d, tarfn):
+                return True
+        return False
+    check_for_tarball = staticmethod(check_for_tarball)
+
+
 import cvs
 import git
 import local
diff --git a/lib/bb/fetch/cvs.py b/lib/bb/fetch/cvs.py
index b2a65de24..17952ed41 100644
--- a/lib/bb/fetch/cvs.py
+++ b/lib/bb/fetch/cvs.py
@@ -129,7 +129,7 @@ class Cvs(Fetch):
             data.setVar('TARFN', tarfn, localdata)
 
             # try to use the tarball stash
-            if Fetch.try_mirror(d, tarfn):
+            if Fetch.check_for_tarball(d, tarfn, dldir, date):
                 bb.debug(1, "%s already exists or was mirrored, skipping cvs checkout." % tarfn)
                 continue
 
diff --git a/lib/bb/fetch/svk.py b/lib/bb/fetch/svk.py
index c0819da3d..19103213c 100644
--- a/lib/bb/fetch/svk.py
+++ b/lib/bb/fetch/svk.py
@@ -101,9 +101,7 @@ class Svk(Fetch):
             data.setVar('TARFILES', dlfile, localdata)
             data.setVar('TARFN', tarfn, localdata)
 
-            dl = os.path.join(dldir, tarfn)
-            if os.access(dl, os.R_OK):
-                bb.debug(1, "%s already exists, skipping svk checkout." % tarfn)
+            if Fetch.check_for_tarball(d, tarfn, dldir, date):
                 continue
 
             olddir = os.path.abspath(os.getcwd())
diff --git a/lib/bb/fetch/svn.py b/lib/bb/fetch/svn.py
index ee57ea61d..04dd4e08d 100644
--- a/lib/bb/fetch/svn.py
+++ b/lib/bb/fetch/svn.py
@@ -112,7 +112,7 @@ class Svn(Fetch):
             data.setVar('TARFN', tarfn, localdata)
 
             # try to use the tarball stash
-            if Fetch.try_mirror(d, tarfn):
+            if Fetch.check_for_tarball(d, tarfn, dldir, date):
                 bb.debug(1, "%s already exists or was mirrored, skipping svn checkout." % tarfn)
                 continue
 
@@ -127,8 +127,12 @@ class Svn(Fetch):
             svncmd = data.getVar('FETCHCOMMAND', localdata, 1)
             svncmd = "svn co -r {%s} %s://%s/%s" % (date, proto, svnroot, module)
 
+            # either use the revision or if SRCDATE is now no braces
             if revision:
                 svncmd = "svn co -r %s %s://%s/%s" % (revision, proto, svnroot, module)
+            elif date == "now":
+                svncmd = "svn co %s://%s/%s" % (proto, svnroot, module)
+
             if svn_rsh:
                 svncmd = "svn_RSH=\"%s\" %s" % (svn_rsh, svncmd)