bitbake/logging: Overhaul internal logging process

At the moment it bugs me a lot that we only have one effective logging
level for bitbake, despite the logging module having provision to do
more advanced things. This patch:

* Changes the core log level to the lowest level we have messages of
  (DEBUG-2) so messages always flow through the core logger
* Allows build.py's task logging code to log all the output regardless
  of what output is on the console and sets this so log files now
  always contain debug level messages even if these don't appear
  on the console
* Moves the verbose/debug/debug-domains code to be a UI side setting
* Adds a filter to the UI to only print the user requested output.

The result is more complete logfiles on disk but the usual output to the
console.

There are some behaviour changes intentionally made by this patch:

a) the -v option now controls whether output is tee'd to the console.

Ultimately, we likely want to output a message to the user about where the
log file is and avoid placing output directly onto the console for every
executing task.

b) The functions get_debug_levels, the debug_levels variable, the
set_debug_levels, the set_verbosity and set_debug_domains functions are
removed from bb.msg.

c) The "logging" init function changes format.

d) All messages get fired to all handlers all the time leading to an
increase in inter-process traffic. This could likely be hacked around
short term with a function for a UI to only request events greater than
level X. Longer term, having masks for event handlers would be better.

e) logger.getEffectiveLevel() is no longer a reliable guide to what
will/won't get logged so for now we look at the default log levels instead.

[YOCTO #304]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

1 files changed, 45 insertions, 29 deletions

diff --git a/lib/bb/msg.py b/lib/bb/msg.py
index 12d19ff8e..77a7a0fed 100644
--- a/lib/bb/msg.py
+++ b/lib/bb/msg.py
@@ -75,6 +75,25 @@ class BBLogFormatter(logging.Formatter):
             msg += '\n' + ''.join(formatted)
         return msg
 
+class BBLogFilter(object):
+    def __init__(self, handler, level, debug_domains):
+        self.stdlevel = level
+        self.debug_domains = debug_domains
+        loglevel = level
+        for domain in debug_domains:
+            if debug_domains[domain] < loglevel:
+                loglevel = debug_domains[domain]
+        handler.setLevel(loglevel)
+        handler.addFilter(self)
+
+    def filter(self, record):
+        if record.levelno >= self.stdlevel:
+            return True
+        if record.name in self.debug_domains and record.levelno >= self.debug_domains[record.name]:
+            return True
+        return False
+
+
 class Loggers(dict):
     def __getitem__(self, key):
         if key in self:
@@ -84,12 +103,6 @@ class Loggers(dict):
             dict.__setitem__(self, key, log)
             return log
 
-class DebugLevel(dict):
-    def __getitem__(self, key):
-        if key == "default":
-            key = domain.Default
-        return get_debug_level(key)
-
 def _NamedTuple(name, fields):
     Tuple = collections.namedtuple(name, " ".join(fields))
     return Tuple(*range(len(fields)))
@@ -110,44 +123,47 @@ domain = _NamedTuple("Domain", (
     "Util"))
 logger = logging.getLogger("BitBake")
 loggers = Loggers()
-debug_level = DebugLevel()
 
 # Message control functions
 #
 
-def set_debug_level(level):
-    for log in loggers.itervalues():
-        log.setLevel(logging.NOTSET)
+loggerDefaultDebugLevel = 0
+loggerDefaultVerbose = False
+loggerDefaultDomains = []
 
-    if level:
-        logger.setLevel(logging.DEBUG - level + 1)
-    else:
-        logger.setLevel(logging.INFO)
+def init_msgconfig(verbose, debug, debug_domains = []):
+    """
+    Set default verbosity and debug levels config the logger
+    """
+    bb.msg.loggerDebugLevel = debug
+    bb.msg.loggerVerbose = verbose
+    bb.msg.loggerDefaultDomains = debug_domains
 
-def get_debug_level(msgdomain = domain.Default):
-    if not msgdomain:
-        level = logger.getEffectiveLevel()
-    else:
-        level = loggers[msgdomain].getEffectiveLevel()
-    return max(0, logging.DEBUG - level + 1)
+def addDefaultlogFilter(handler):
 
-def set_verbose(level):
-    if level:
-        logger.setLevel(BBLogFormatter.VERBOSE)
+    debug = loggerDefaultDebugLevel
+    verbose = loggerDefaultVerbose
+    domains = loggerDefaultDomains
+
+    if debug:
+        level = BBLogFormatter.DEBUG - debug + 1
+    elif verbose:
+        level = BBLogFormatter.VERBOSE
     else:
-        logger.setLevel(BBLogFormatter.INFO)
+        level = BBLogFormatter.NOTE
 
-def set_debug_domains(domainargs):
-    for (domainarg, iterator) in groupby(domainargs):
+    debug_domains = {}
+    for (domainarg, iterator) in groupby(domains):
+        dlevel = len(tuple(iterator))
+        debug_domains["BitBake.%s" % domainarg] = logging.DEBUG - dlevel + 1
         for index, msgdomain in enumerate(domain._fields):
             if msgdomain == domainarg:
-                level = len(tuple(iterator))
-                if level:
-                    loggers[index].setLevel(logging.DEBUG - level + 1)
                 break
         else:
             warn(None, "Logging domain %s is not valid, ignoring" % domainarg)
 
+    BBLogFilter(handler, level, debug_domains)
+
 #
 # Message handling functions
 #