context.py: fix skipping function

The current codes to skip test cases are logically correct, but they
do not work correctly in reality. It does skip the tests as the command
line argument specifies, but the related information is wrong.
e.g.
$ oe-selftest -R archiver bblayers runtime_test
2019-06-17 09:24:53,764 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it
2019-06-17 09:25:06,309 - oe-selftest - INFO - Adding layer libraries:
2019-06-17 09:25:06,310 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta/lib
2019-06-17 09:25:06,310 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta-yocto-bsp/lib
2019-06-17 09:25:06,310 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta-selftest/lib
2019-06-17 09:25:06,312 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable
2019-06-17 09:25:10,521 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea5/chenqi/SWAT/poky/build-selftest/conf/local.conf
2019-06-17 09:25:10,521 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2019-06-17 09:25:10,522 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver)
2019-06-17 09:25:10,522 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "runtime_test"'
2019-06-17 09:25:10,522 - oe-selftest - INFO - Skip by the command line argument "runtime_test"
2019-06-17 09:25:10,523 - oe-selftest - INFO - test_archiver_filters_by_type (archiver.Archiver)
2019-06-17 09:25:10,523 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "runtime_test"'
2019-06-17 09:25:10,523 - oe-selftest - INFO - Skip by the command line argument "runtime_test"

The archiver.Archiver.xxx tests should be skipped by 'archiver' command line
argument, not 'runtime_test'.

Change to use a function generator to achieve the desired effect. After the change,
the effect is as follows.

$ oe-selftest -R archiver bblayers runtime_test
2019-06-17 09:19:06,223 - oe-selftest - WARNING - meta-selftest layer not found in BBLAYERS, adding it
2019-06-17 09:19:19,598 - oe-selftest - INFO - Adding layer libraries:
2019-06-17 09:19:19,599 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta/lib
2019-06-17 09:19:19,599 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta-yocto-bsp/lib
2019-06-17 09:19:19,599 - oe-selftest - INFO - 	      /buildarea5/chenqi/SWAT/poky/meta-selftest/lib
2019-06-17 09:19:19,602 - oe-selftest - INFO - Running bitbake -e to test the configuration is valid/parsable
2019-06-17 09:19:24,368 - oe-selftest - INFO - Adding: "include selftest.inc" in /buildarea5/chenqi/SWAT/poky/build-selftest/conf/local.conf
2019-06-17 09:19:24,368 - oe-selftest - INFO - Adding: "include bblayers.inc" in bblayers.conf
2019-06-17 09:19:24,369 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver)
2019-06-17 09:19:24,369 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "archiver"'
2019-06-17 09:19:24,369 - oe-selftest - INFO - Skip by the command line argument "archiver"
2019-06-17 09:19:24,369 - oe-selftest - INFO - test_archiver_filters_by_type (archiver.Archiver)
2019-06-17 09:19:24,370 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "archiver"'
2019-06-17 09:19:24,370 - oe-selftest - INFO - Skip by the command line argument "archiver"
2019-06-17 09:19:24,370 - oe-selftest - INFO - test_archiver_filters_by_type_and_name (archiver.Archiver)
2019-06-17 09:19:24,370 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "archiver"'
2019-06-17 09:19:24,371 - oe-selftest - INFO - Skip by the command line argument "archiver"
2019-06-17 09:19:24,371 - oe-selftest - INFO - test_archiver_srpm_mode (archiver.Archiver)
2019-06-17 09:19:24,371 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "archiver"'
2019-06-17 09:19:24,372 - oe-selftest - INFO - Skip by the command line argument "archiver"
2019-06-17 09:19:24,372 - oe-selftest - INFO - test_bitbakelayers_add_remove (bblayers.BitbakeLayers)
2019-06-17 09:19:24,373 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "bblayers"'
2019-06-17 09:19:24,373 - oe-selftest - INFO - Skip by the command line argument "bblayers"
2019-06-17 09:19:24,373 - oe-selftest - INFO - test_bitbakelayers_createlayer (bblayers.BitbakeLayers)
2019-06-17 09:19:24,373 - oe-selftest - INFO -  ... skipped 'Skip by the command line argument "bblayers"'
2019-06-17 09:19:24,374 - oe-selftest - INFO - Skip by the command line argument "bblayers"
[snip]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>

1 files changed, 6 insertions, 1 deletions

diff --git a/meta/lib/oeqa/core/context.py b/meta/lib/oeqa/core/context.py
index 09627044c8..58244895af 100644
--- a/meta/lib/oeqa/core/context.py
+++ b/meta/lib/oeqa/core/context.py
@@ -9,6 +9,7 @@ import json
 import time
 import logging
 import collections
+import unittest
 
 from oeqa.core.loader import OETestLoader
 from oeqa.core.runner import OETestRunner
@@ -45,10 +46,14 @@ class OETestContext(object):
     def skipTests(self, skips):
         if not skips:
             return
+        def skipfuncgen(skipmsg):
+            def func():
+                raise unittest.SkipTest(skipmsg)
+            return func
         for test in self.suites:
             for skip in skips:
                 if test.id().startswith(skip):
-                    setattr(test, 'setUp', lambda: test.skipTest('Skip by the command line argument "%s"' % skip))
+                    setattr(test, 'setUp', skipfuncgen('Skip by the command line argument "%s"' % skip))
 
     def loadTests(self, module_paths, modules=[], tests=[],
             modules_manifest="", modules_required=[], filters={}):