python3: Add target and native recipes

3.3.3 is latest bugfix release in 3.3 series

License text is changed to reflect year and prev releases

+++ ../Python-3.3.2/LICENSE     2013-05-15 09:32:54.000000000 -0700
@@ -74,7 +74,9 @@
     3.2.1           3.2         2011        PSF         yes
     3.2.2           3.2.1       2011        PSF         yes
     3.2.3           3.2.2       2012        PSF         yes
+    3.2.4           3.2.3       2013        PSF         yes
     3.3.0           3.2         2012        PSF         yes
+    3.3.1           3.3.0       2013        PSF         yes

 Footnotes:

@@ -110,8 +112,8 @@
 distribute, and otherwise use Python alone or in any derivative
version,
 provided, however, that PSF's License Agreement and PSF's notice of
copyright,
 i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008,
2009, 2010,
-2011, 2012 Python Software Foundation; All Rights Reserved" are
retained in Python
-alone or in any derivative version prepared by Licensee.
+2011, 2012, 2013 Python Software Foundation; All Rights Reserved" are
retained
+in Python alone or in any derivative version prepared by Licensee.

We use native python in disutils bbclass and this needs
to peek into target sysroot when building plugins in cross
environment. Otherwise anything that inherits distutils3.bbclass
will not build.

Fix host include contamination issue

Let compiler append sysroot to include path if it can

Fix the compiler invocation and linker flags when cross compiling

The details are in patch header. This should fix the QA errors about
host contamination

Add virtual/libintl to dep list

There is use of libintl.h but we dont
express the dependency

Add lzma dependency for python3-misc

Fixes

and from test-dependencies.sh run there is undeterministic lzma
dependency:

WARN: packages/armv5te-oe-linux-gnueabi/python3/python3-misc/latest lost
dependency on  liblzma

Fix staging warnings due to duplicate install

WARNING: The recipe python3 is trying to install files into a shared
area when those files already exist. Those files and their manifest
location are:

builds/world/tmp/sysroots/qemux86-64/usr/include/python3.3m/pyconfig.h
builds/world/tmp/sysroots/qemux86-64/usr/lib/libpython3.3m.so.1.0
builds/world/tmp/sysroots/qemux86-64/usr/lib/libpython3.3m.so
builds/world/tmp/sysroots/qemux86-64/usr/lib/python3.3/config/Makefile

Let compiler search includes relative to sysroot

This makes it cross compilable and assumption on hardcoded
paths is fixed.

Errors like below would appear if ncursesw is installed on build host

cc1: warning: include location "/usr/include/ncursesw" is unsafe for
cross-compilation [-Wpoison-system-directories]

Signed-off-by: Khem Raj <raj.khem@gmail.com>

Fix linking not happening issue in modules

We need to tweak CONFIGURE_LDFLAGS for python3 instead of LDFLAGS
directly

Add second compile stage which uses original makefile

Signed-off-by: Khem Raj <raj.khem@gmail.com>

1 files changed, 336 insertions, 0 deletions

diff --git a/meta/recipes-devtools/python/python3/python-3.3-multilib.patch b/meta/recipes-devtools/python/python3/python-3.3-multilib.patch
new file mode 100644
index 0000000000..7d66a0e00d
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/python-3.3-multilib.patch
@@ -0,0 +1,336 @@
+get the sys.lib from python itself and do not use hardcoded value of 'lib'
+
+-Khem
+
+Upstream-Status: Pending
+
+Index: Python-3.3.2/Include/pythonrun.h
+===================================================================
+--- Python-3.3.2.orig/Include/pythonrun.h	2013-05-15 09:32:54.000000000 -0700
++++ Python-3.3.2/Include/pythonrun.h	2013-07-27 16:19:54.099877246 -0700
+@@ -181,6 +181,8 @@
+ /* In their own files */
+ PyAPI_FUNC(const char *) Py_GetVersion(void);
+ PyAPI_FUNC(const char *) Py_GetPlatform(void);
++PyAPI_FUNC(const char *) Py_GetArch(void);
++PyAPI_FUNC(const char *) Py_GetLib(void);
+ PyAPI_FUNC(const char *) Py_GetCopyright(void);
+ PyAPI_FUNC(const char *) Py_GetCompiler(void);
+ PyAPI_FUNC(const char *) Py_GetBuildInfo(void);
+Index: Python-3.3.2/Lib/distutils/command/install.py
+===================================================================
+--- Python-3.3.2.orig/Lib/distutils/command/install.py	2013-05-15 09:32:54.000000000 -0700
++++ Python-3.3.2/Lib/distutils/command/install.py	2013-07-27 16:19:54.099877246 -0700
+@@ -25,6 +25,8 @@
+     from site import USER_SITE
+     HAS_USER_SITE = True
+ 
++libname = sys.lib
++
+ if sys.version < "2.2":
+     WINDOWS_SCHEME = {
+         'purelib': '$base',
+@@ -45,7 +47,7 @@
+ INSTALL_SCHEMES = {
+     'unix_prefix': {
+         'purelib': '$base/lib/python$py_version_short/site-packages',
+-        'platlib': '$platbase/lib/python$py_version_short/site-packages',
++        'platlib': '$platbase/'+libname+'/python$py_version_short/site-packages',
+         'headers': '$base/include/python$py_version_short$abiflags/$dist_name',
+         'scripts': '$base/bin',
+         'data'   : '$base',
+Index: Python-3.3.2/Lib/pydoc.py
+===================================================================
+--- Python-3.3.2.orig/Lib/pydoc.py	2013-05-15 09:32:55.000000000 -0700
++++ Python-3.3.2/Lib/pydoc.py	2013-07-27 16:19:54.103877246 -0700
+@@ -372,7 +372,7 @@
+ 
+         docloc = os.environ.get("PYTHONDOCS", self.PYTHONDOCS)
+ 
+-        basedir = os.path.join(sys.base_exec_prefix, "lib",
++        basedir = os.path.join(sys.base_exec_prefix, sys.lib,
+                                "python%d.%d" %  sys.version_info[:2])
+         if (isinstance(object, type(os)) and
+             (object.__name__ in ('errno', 'exceptions', 'gc', 'imp',
+Index: Python-3.3.2/Lib/site.py
+===================================================================
+--- Python-3.3.2.orig/Lib/site.py	2013-05-15 09:32:55.000000000 -0700
++++ Python-3.3.2/Lib/site.py	2013-07-27 16:19:54.103877246 -0700
+@@ -303,13 +303,19 @@
+         if sys.platform in ('os2emx', 'riscos'):
+             sitepackages.append(os.path.join(prefix, "Lib", "site-packages"))
+         elif os.sep == '/':
+-            sitepackages.append(os.path.join(prefix, "lib",
++            sitepackages.append(os.path.join(prefix, sys.lib,
+                                         "python" + sys.version[:3],
+                                         "site-packages"))
+-            sitepackages.append(os.path.join(prefix, "lib", "site-python"))
++            if sys.lib != "lib":
++                sitepackages.append(os.path.join(prefix, "lib",
++                                        "python" + sys.version[:3],
++                                        "site-packages"))
++            sitepackages.append(os.path.join(prefix, sys.lib, "site-python"))
++            if sys.lib != "lib":
++                sitepackages.append(os.path.join(prefix, "lib", "site-python"))
+         else:
+             sitepackages.append(prefix)
+-            sitepackages.append(os.path.join(prefix, "lib", "site-packages"))
++            sitepackages.append(os.path.join(prefix, sys.lib, "site-packages"))
+         if sys.platform == "darwin":
+             # for framework builds *only* we add the standard Apple
+             # locations.
+Index: Python-3.3.2/Lib/trace.py
+===================================================================
+--- Python-3.3.2.orig/Lib/trace.py	2013-05-15 09:32:56.000000000 -0700
++++ Python-3.3.2/Lib/trace.py	2013-07-27 16:19:54.103877246 -0700
+@@ -751,10 +751,10 @@
+                 # should I also call expanduser? (after all, could use $HOME)
+ 
+                 s = s.replace("$prefix",
+-                              os.path.join(sys.base_prefix, "lib",
++                              os.path.join(sys.base_prefix, sys.lib,
+                                            "python" + sys.version[:3]))
+                 s = s.replace("$exec_prefix",
+-                              os.path.join(sys.base_exec_prefix, "lib",
++                              os.path.join(sys.base_exec_prefix, sys.lib,
+                                            "python" + sys.version[:3]))
+                 s = os.path.normpath(s)
+                 ignore_dirs.append(s)
+Index: Python-3.3.2/Makefile.pre.in
+===================================================================
+--- Python-3.3.2.orig/Makefile.pre.in	2013-07-27 16:19:16.000000000 -0700
++++ Python-3.3.2/Makefile.pre.in	2013-07-27 16:19:54.103877246 -0700
+@@ -96,6 +96,8 @@
+ 
+ # Machine-dependent subdirectories
+ MACHDEP=	@MACHDEP@
++LIB=		@LIB@
++ARCH=		@ARCH@
+ 
+ # Multiarch directory (may be empty)
+ MULTIARCH=	@MULTIARCH@
+@@ -115,7 +117,7 @@
+ MANDIR=		@mandir@
+ INCLUDEDIR=	@includedir@
+ CONFINCLUDEDIR=	$(exec_prefix)/include
+-SCRIPTDIR=	$(prefix)/lib
++SCRIPTDIR=	@libdir@
+ ABIFLAGS=	@ABIFLAGS@
+ 
+ # Detailed destination directories
+@@ -635,6 +637,7 @@
+ 		-DEXEC_PREFIX='"$(exec_prefix)"' \
+ 		-DVERSION='"$(VERSION)"' \
+ 		-DVPATH='"$(VPATH)"' \
++		-DARCH='"$(ARCH)"' -DLIB='"$(LIB)"' \
+ 		-o $@ $(srcdir)/Modules/getpath.c
+ 
+ Modules/python.o: $(srcdir)/Modules/python.c
+@@ -701,7 +704,7 @@
+ Python/compile.o Python/symtable.o Python/ast.o: $(GRAMMAR_H) $(AST_H)
+ 
+ Python/getplatform.o: $(srcdir)/Python/getplatform.c
+-		$(CC) -c $(PY_CORE_CFLAGS) -DPLATFORM='"$(MACHDEP)"' -o $@ $(srcdir)/Python/getplatform.c
++		$(CC) -c $(PY_CORE_CFLAGS) -DPLATFORM='"$(MACHDEP)"' -DARCH='"$(ARCH)"' -DLIB='"$(LIB)"' -o $@ $(srcdir)/Python/getplatform.c
+ 
+ Python/importdl.o: $(srcdir)/Python/importdl.c
+ 		$(CC) -c $(PY_CORE_CFLAGS) -I$(DLINCLDIR) -o $@ $(srcdir)/Python/importdl.c
+Index: Python-3.3.2/Modules/getpath.c
+===================================================================
+--- Python-3.3.2.orig/Modules/getpath.c	2013-05-15 09:32:59.000000000 -0700
++++ Python-3.3.2/Modules/getpath.c	2013-07-27 16:19:54.107877246 -0700
+@@ -121,9 +121,11 @@
+ #define EXEC_PREFIX PREFIX
+ #endif
+ 
++#define LIB_PYTHON LIB "/python" VERSION
++
+ #ifndef PYTHONPATH
+-#define PYTHONPATH PREFIX "/lib/python" VERSION ":" \
+-              EXEC_PREFIX "/lib/python" VERSION "/lib-dynload"
++#define PYTHONPATH PREFIX "/" LIB_PYTHON ":" \
++              EXEC_PREFIX "/" LIB_PYTHON "/lib-dynload"
+ #endif
+ 
+ #ifndef LANDMARK
+@@ -135,7 +137,7 @@
+ static wchar_t progpath[MAXPATHLEN+1];
+ static wchar_t *module_search_path = NULL;
+ static int module_search_path_malloced = 0;
+-static wchar_t *lib_python = L"lib/python" VERSION;
++static wchar_t *lib_python = L"" LIB_PYTHON;
+ 
+ static void
+ reduce(wchar_t *dir)
+Index: Python-3.3.2/Python/getplatform.c
+===================================================================
+--- Python-3.3.2.orig/Python/getplatform.c	2013-05-15 09:33:00.000000000 -0700
++++ Python-3.3.2/Python/getplatform.c	2013-07-27 16:19:54.107877246 -0700
+@@ -10,3 +10,23 @@
+ {
+ 	return PLATFORM;
+ }
++
++#ifndef ARCH
++#define ARCH "unknown"
++#endif
++
++const char *
++Py_GetArch(void)
++{
++	return ARCH;
++}
++
++#ifndef LIB
++#define LIB "lib"
++#endif
++
++const char *
++Py_GetLib(void)
++{
++	return LIB;
++}
+Index: Python-3.3.2/Python/sysmodule.c
+===================================================================
+--- Python-3.3.2.orig/Python/sysmodule.c	2013-05-15 09:33:00.000000000 -0700
++++ Python-3.3.2/Python/sysmodule.c	2013-07-27 16:19:54.107877246 -0700
+@@ -1612,6 +1612,10 @@
+                         PyUnicode_FromString(Py_GetCopyright()));
+     SET_SYS_FROM_STRING("platform",
+                         PyUnicode_FromString(Py_GetPlatform()));
++    SET_SYS_FROM_STRING("arch",
++                        PyUnicode_FromString(Py_GetArch()));
++    SET_SYS_FROM_STRING("lib",
++                        PyUnicode_FromString(Py_GetLib()));
+     SET_SYS_FROM_STRING("executable",
+                         PyUnicode_FromWideChar(
+                                Py_GetProgramFullPath(), -1));
+Index: Python-3.3.2/setup.py
+===================================================================
+--- Python-3.3.2.orig/setup.py	2013-07-27 16:19:17.000000000 -0700
++++ Python-3.3.2/setup.py	2013-07-27 16:19:54.107877246 -0700
+@@ -439,7 +439,7 @@
+         # directories (i.e. '.' and 'Include') must be first.  See issue
+         # 10520.
+         if not cross_compiling:
+-            add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
++            add_dir_to_list(self.compiler.library_dirs, os.path.join('/usr/local', sys.lib))
+             add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
+         # only change this for cross builds for 3.3, issues on Mageia
+         if cross_compiling:
+@@ -497,8 +497,7 @@
+         # be assumed that no additional -I,-L directives are needed.
+         if not cross_compiling:
+             lib_dirs = self.compiler.library_dirs + [
+-                '/lib64', '/usr/lib64',
+-                '/lib', '/usr/lib',
++                '/' + sys.lib, '/usr/' + sys.lib,
+                 ]
+             inc_dirs = self.compiler.include_dirs + ['/usr/include']
+         else:
+@@ -675,11 +674,11 @@
+             elif curses_library:
+                 readline_libs.append(curses_library)
+             elif self.compiler.find_library_file(lib_dirs +
+-                                                     ['/usr/lib/termcap'],
++                                                     ['/usr/'+sys.lib+'/termcap'],
+                                                      'termcap'):
+                 readline_libs.append('termcap')
+             exts.append( Extension('readline', ['readline.c'],
+-                                   library_dirs=['/usr/lib/termcap'],
++                                   library_dirs=['/usr/'+sys.lib+'/termcap'],
+                                    extra_link_args=readline_extra_link_args,
+                                    libraries=readline_libs) )
+         else:
+Index: Python-3.3.2/Lib/sysconfig.py
+===================================================================
+--- Python-3.3.2.orig/Lib/sysconfig.py	2013-05-15 09:32:55.000000000 -0700
++++ Python-3.3.2/Lib/sysconfig.py	2013-07-27 16:19:54.111877246 -0700
+@@ -21,10 +21,10 @@
+ 
+ _INSTALL_SCHEMES = {
+     'posix_prefix': {
+-        'stdlib': '{installed_base}/lib/python{py_version_short}',
+-        'platstdlib': '{platbase}/lib/python{py_version_short}',
++        'stdlib': '{installed_base}/'+sys.lib+'/python{py_version_short}',
++        'platstdlib': '{platbase}/'+sys.lib+'/python{py_version_short}',
+         'purelib': '{base}/lib/python{py_version_short}/site-packages',
+-        'platlib': '{platbase}/lib/python{py_version_short}/site-packages',
++        'platlib': '{platbase}/'+sys.lib+'/python{py_version_short}/site-packages',
+         'include':
+             '{installed_base}/include/python{py_version_short}{abiflags}',
+         'platinclude':
+@@ -33,10 +33,10 @@
+         'data': '{base}',
+         },
+     'posix_home': {
+-        'stdlib': '{installed_base}/lib/python',
+-        'platstdlib': '{base}/lib/python',
++        'stdlib': '{installed_base}/'+sys.lib+'/python',
++        'platstdlib': '{base}/'+sys.lib+'/python',
+         'purelib': '{base}/lib/python',
+-        'platlib': '{base}/lib/python',
++        'platlib': '{base}/'+sys.lib+'/python',
+         'include': '{installed_base}/include/python',
+         'platinclude': '{installed_base}/include/python',
+         'scripts': '{base}/bin',
+@@ -81,10 +81,10 @@
+         'data': '{userbase}',
+         },
+     'posix_user': {
+-        'stdlib': '{userbase}/lib/python{py_version_short}',
+-        'platstdlib': '{userbase}/lib/python{py_version_short}',
++        'stdlib': '{userbase}/'+sys.lib+'/python{py_version_short}',
++        'platstdlib': '{userbase}/'+sys.lib+'/python{py_version_short}',
+         'purelib': '{userbase}/lib/python{py_version_short}/site-packages',
+-        'platlib': '{userbase}/lib/python{py_version_short}/site-packages',
++        'platlib': '{userbase}/'+sys.lib+'/python{py_version_short}/site-packages',
+         'include': '{userbase}/include/python{py_version_short}',
+         'scripts': '{userbase}/bin',
+         'data': '{userbase}',
+Index: Python-3.3.2/configure.ac
+===================================================================
+--- Python-3.3.2.orig/configure.ac	2013-05-15 09:33:00.000000000 -0700
++++ Python-3.3.2/configure.ac	2013-07-27 16:19:54.111877246 -0700
+@@ -769,6 +769,41 @@
+ MULTIARCH=$($CC --print-multiarch 2>/dev/null)
+ AC_SUBST(MULTIARCH)
+ 
++AC_SUBST(ARCH)
++AC_MSG_CHECKING(ARCH)
++ARCH=`uname -m`
++case $ARCH in
++i?86) ARCH=i386;;
++esac
++AC_MSG_RESULT($ARCH)
++
++AC_SUBST(LIB)
++AC_MSG_CHECKING(LIB)
++case $ac_sys_system in
++Linux*)
++  # Test if the compiler is 64bit
++  echo 'int i;' > conftest.$ac_ext
++  python_cv_cc_64bit_output=no
++  if AC_TRY_EVAL(ac_compile); then
++    case `/usr/bin/file conftest.$ac_objext` in
++    *"ELF 64"*)
++      python_cv_cc_64bit_output=yes
++      ;;
++    esac
++  fi
++  rm -rf conftest*
++  ;;
++esac
++
++case $ARCH:$python_cv_cc_64bit_output in
++ppc64:yes | powerpc64:yes | s390x:yes | sparc64:yes | x86_64:yes)
++  LIB="lib64"
++  ;;
++*:*)
++  LIB="lib"
++  ;;
++esac
++AC_MSG_RESULT($LIB)
+ 
+ AC_SUBST(LIBRARY)
+ AC_MSG_CHECKING(LIBRARY)