summaryrefslogtreecommitdiffstats
path: root/meta/lib/oe
AgeCommit message (Collapse)Author
2020-05-27prservice.py: fix do_package with newer Python in Ubuntu 20.04Martin Jansa
* with Ubuntu 20.04 which is using python 3.8 I'm seeing a lot of errors like: ERROR: libxml2-2.9.10-r0 do_package: Can NOT get PRAUTO, exception No module named '_sysconfigdata' not sure what caused this from python 3.8, but this seems to work * PRserv is enabled with: PRSERV_HOST = "localhost:0" (From OE-Core rev: 4b26eaf7152fb712aba47a0c746333578f58ee8d) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25sdk: Install nativesdk locales for all TCLIBC variantsKhem Raj
install_locales() here is actually operating on nativesdk and only glibc is the default library for nativesdk, since thats what most of desktop/server distros use, therefore bailing out based on TCLIBC is not needed here, since nativesdk-glibc would be required for all non-glibc targetting SDKs as well. Fixes SDK install time error ERROR: OE-core's config sanity checker detected a potential misconfiguration. Either fix the cause of this error or at your own risk disable the checker (see sanity.conf). Following is the list of potential problems / advisories: Your system needs to support the en_US.UTF-8 locale. ERROR: SDK preparation failed Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-11-10lib/oe/lsb: Make sure the distro ID is always lowercasedPeter Kjellerstedt
In commit 8689e561 (lib/oe/lsb: attempt to ensure consistent distro id regardless of source), the distro ID returned by oe.lsb.distro_identifier() was lowercased, but only if a release version is also present. This changes the code to always lowercase the distro ID, including the default distro ID "unknown", which is used if no other ID can be identified. Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-09-04package: Improve determinismRichard Purdie
Its possible in cases with multiple shlib providers we were not being deterministic. Add in a couple of sorted() calls to fix the shlibs and pkgconfig cases with this potential issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-27gpg_sign/selftest: Fix secmem parameter handlingRichard Purdie
We keep seeing "cannot allocate memory" errors from rpm when signing packages on the autobuilder. The following were tried: * checking locked memory use (isn't hitting limits) * Restricting RPM_GPG_SIGN_CHUNK to 1 * Limiting to 10 parallel do_package_write_rpm tasks * Allowing unlimied memory overcommit * Disabling rpm parallel compression and the test still failed. Further invetigation showed that the --auto-expand-secmem wasn't being passed to gpg-agent which meant the secmem couldn't be expanded hence the errors when there was pressure on the agent. The reason this happens is that some of the early gpg commands can start the agent without the option and it sticks around in memory so a version with the correct option may or may not get started. We therefore add the option to all the key gpg calls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-27rootfs: Fix dependency for every dpkg runRicardo Ribalda Delgado
Avoid getting a warning on do_rootfs. Fixes: NOTE: Installing the following packages: libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb WARNING: Unable to install packages. Command '/workdir/build/tmp/work/qt5122-poky-linux/bottlecam-image/1.0-r0/recipe-sysroot-native/usr/bin/apt-get install --force-yes --allow-unauthenticated libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb' returned 100: Reading package lists... Building dependency tree... Reading state information... Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-06-14lib/oe/reciputils.py: support character `+' in git pvHongxu Jia
While pv contains char `+' (such as ncurses 6.1+20181013), it was incorrectly converted to `6.1' In this commit: - Convert [^\+]* to .*? Support pv to contain char `+' and not greedy match (Such as ncurses 6.1+20181013) - Add [^\+] to sfx match Support sfx contains extra chars between `+' and `git' (such as asciidoc "8.6.9+py3-gitAUTOINC+618f6e6f6b") - Make sfx and rev greedy match Run `devtool check-upgrade-status --all' in poky and compare results, only one difference on ncurses version: Without the commit: INFO: ncurses 6.1 6.1+20181013 Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 With the commit: INFO: ncurses 6.1+20181013 MATCH Hongxu Jia <hongxu.jia@windriver.com> 7a97a7f937762ba342d5b2fd7cd090885a809835 Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02utils/multiprocess_launch: Improve failing subprocess outputRichard Purdie
Output before this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1.: Traceback (most recent call last): File "/media/build1/poky/meta/lib/oe/utils.py", line 272, in run ret = self._target(*self._args, **self._kwargs) File "/media/build1/poky/meta/classes/package_ipk.bbclass", line 230, in ipk_write_pkg shell=True) File "/usr/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Note how stdout/stderr from the failing command isn't shown. After this patch: ERROR: bash-4.4.18-r0 do_package_write_ipk: Fatal errors occurred in subprocesses: Command 'PATH="X" opkg-build -Z xz -a "--memlimit=50% --threads=88" Foobar /media/build1/poky/build/nodistro-glibc/work/core2-64-oe-linux/bash/4.4.18-r0/deploy-ipks/core2-64' returned non-zero exit status 1. Subprocess output:Foobar *** Error: Package name Foobar contains illegal characters, (other than [a-z0-9.+-]) opkg-build: Please fix the above errors and try again. We suddenly get a much more usable error message. The traceback is supressed as its distracting from the real problem in this case. Ideally python itself would handle this but it doesn't so we have to wrap the exception. We already do this in bitbake itself for the same reason. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-05-02buildhistory: call a dependency parser only on actual dependency listsAlexander Kanavin
Previously it was also called on filelists and possibly other items which broke the parser. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com>
2019-04-09patch/insane: Rework patch fuzz handlingAndreas Müller
Currently there are three issues which can be enhanced: 1. Fuzz warnings cannot be configured as errors for hardening. It happened often to me that these warnings were overseen and detected after commits were already out. 2. The output is too verbose - particularly when more than one file is affected. Meanwhile all users should know why patch fuzz check is performed. So move links with background information to insane.bbclass. 3. Reduce copy & paste effort slightly by printing PN (nit: <recipe> was not a correct suggestion e.g for native extended recipe - see example below) To achieve patch.py drops patch-fuzz info encapsulated by a header- and footer- string into log.do_patch. With this insane.bbclass can drop warnings/errors depending on 'patch-fuzz' in ERROR_QA or WARN_QA. Default remains unchanged: Spit out warnings only. A message for two fuzzed patches and 'pact-fuzz' in ERROR_QA now looks like: | ERROR: autoconf-native-2.69-r11 do_patch: Fuzz detected: | | Applying patch autoreconf-exclude.patch | patching file bin/autoreconf.in | Hunk #1 succeeded at 73 with fuzz 1 (offset -3 lines). | Hunk #2 succeeded at 143 (offset 6 lines). | Hunk #3 succeeded at 167 (offset 6 lines). | Hunk #4 succeeded at 177 (offset 6 lines). | Hunk #5 succeeded at 281 (offset 15 lines). | Hunk #6 succeeded at 399 (offset 15 lines). | Hunk #7 succeeded at 571 (offset 20 lines). | Hunk #8 succeeded at 612 (offset 20 lines). | Hunk #9 succeeded at 636 (offset 20 lines). | Hunk #10 succeeded at 656 (offset 20 lines). | Hunk #11 succeeded at 683 (offset 20 lines). | | Applying patch autoreconf-gnuconfigize.patch | patching file bin/autoreconf.in | Hunk #1 succeeded at 55 with fuzz 1 (offset -3 lines). | Hunk #3 succeeded at 663 (offset 18 lines). | | The context lines in the patches can be updated with devtool: | | devtool modify autoconf-native | devtool finish --force-patch-refresh autoconf-native <layer_path> | | Don't forget to review changes done by devtool! | | ERROR: autoconf-native-2.69-r11 do_patch: QA Issue: Patch log indicates that patches do not apply cleanly. [patch-fuzz] Signed-off-by: Andreas Müller <schnitzeltony@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09lib/oe/utils: Make prune_suffix prune a suffixAndre Rosa
... instead of replacing a substring that could happen more than once and not only when it ends with it. Do the same for the prefix. See related https://github.com/openembedded/bitbake/pull/24 . There it stops replacing sufixes once first one is matched but not here. Signed-off-by: Andre Rosa <andre.rosa@lge.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-04utils.py: added sh_quote() functionEnrico Scholz via Openembedded-core
This function is a wrapper around "shlex.quote()" and can be used in "${@...}" context where shlex (or pipes, which provides similar functionality) is unavailable. Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-05devtool: Support kmeta directory usage with devtool modify/finishJaewon Lee
When using Kmeta directories, devtool finish will add every single file in the directory to the bbappend. This is because in the current implementation, the get_recipe_local_files function treats the kmeta directory like a file. Modifying the function to loop through the provided directories and return all included files instead of just the top level directory. This will enable correct file to file comparison when determing which files are new/changed and need to be added to the bbappend. Adding an extra check in devtool-source.bbclass to not copy the cfg file if its already included somewhere in the kmeta directory Also during 'modify', when moving necessary files in the kmeta directory from the workdir to oe-local-files, the dangling parent directories are left behind. This in itself is not an issue as the temporary devtool workspace is automatically deleted, but this causes an incorrect include directory to be added in kernel-yocto.bbclass. Changing the order of the if statements to catch the correct conditional. This is safe to do as when not in the devtool context, there will be no oe-local-files directory. Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25lib/oe/package_manager.py: turn nativesdk postinst warnings into notesAlexander Kanavin
The warnings deal with two specific cases of failure: 1) unable to execute nativesdk postinsts for mingw SDKs because they need to be run under wine 2) unable to execute target postinsts when there is no qemu usermode support for the target Neither of these should be a big problem as mingw issue was never found to problematic, and target postinsts deal with things that are needed at runtime and not at build time which is the purpose of SDKs. The specific reason to do this is to reduce the amount of warnings shown by the Yocto autobuilder, to zero eventually. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-17lib/oe/utils: Fix hang in multiprocess_launch()Richard Purdie
If large results values are returned by the subprocesses, we can hit a deadlock where the subprocess is trying to write data back to the parent, the pipe is full and the parent is waiting for the child to exit. Avoid this by calling the update() method which would trigger reading a result from the child, avoiding the deadlock. The issue is described in https://bugs.python.org/issue8426 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-16lib/oe,oeqa/selftest: Fix DeprecationWarning: invalid escape sequenceRichard Purdie
Fix another load of regex escape sequence warnings for newer python versions. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15utils: add -src to system package name blacklistRoss Burton
oe.utils.packages_filter_out_system() returns PACKAGES after removing "system" packages but it doesn't handle ${PN}-src as generated by PACKAGE_DEBUG_SPLIT_STYLE=debug-with-srcpkg. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONSAlejandro del Castillo
Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to deinstalled and pinned). This is brittle, and not consistent across the different solver backends. Use new --add-ignore-recommends flag instead. Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06lib/oe/rootfs: prelink only when image-prelink is inheritedAndrej Valek
Use prelink only when image-prelink class is inherited. Inheriting this class adds prelink-native as dependency to do_rootfs, which makes prelink binary and configuration available. |Exception: FileNotFoundError: [Errno 2] No such file or directory: |'/home/.../1.0-r0/recipe-sysroot-native/etc/prelink.conf' | | if not os.path.exists(prelink_cfg): | shutil.copy(self.d.expand('${STAGING_DIR_NATIVE}${sysconfdir_native}/prelink.conf'),prelink_cfg) Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06package_manager.py: Use data.tar.xz for ipkg tooAngus Lees
Commit b95b6ba1a2959e2294a8848fa35f20163388eb06 changed package_ipk.bbclass to xz when building packages. This updates OpkgDpkgPM.extract() accordingly. Signed-off-by: Angus Lees <gus@inodes.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-04terminal: Fix working directory in tmux terminalsJacob Kroon
Recent version of tmux seems to use PWD from the environment instead of the actual working directory. Help it use the correct path. Fixes setting the correct working directory when running devshell and menuconfig tasks. Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26lib/oe/patch.py: Fix applying a directory as a patchTomasz Dziendzielski
If a SRC_URI content ends with '.patch' bitbake is trying to apply it as it's a patch file. It causes that if we use git repository for 'patch' package the bare clone is extracted to a directory (i.e. build/downloads/git2/git.mirror.org.patch/) which is considered to be a patch file, so patch.py tries to apply that directory as a patch which ends up with a failure. Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26lib/oe/patch.py: Fix the function commentTomasz Dziendzielski
If it's not a patch the function returns nothing. Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24lib/oe/reciputils.py: parallelize upstream version checksAlexander Kanavin
Previously this was done via bitbake tasks, and when this was rewritten to a for loop, performance sufered significantly: from 90 seconds to about 12 minutes for oe-core. This change restores the previous run time, and makes it possible to perform such checks with command line utilities in an interactive way. Implementation note: we have to create a copy of the recipe data, as Tinfoil API can't be used from multiple threads and only allows one process to access the data at a time. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24lib/oe/package_manager: turn postinst_intercept warnings into failures for ↵Alexander Kanavin
nativesdk The few cases where they failed should be now all fixed. The only allowed exception is when building mingw32 SDKs, as there is currently no support for running postinst_intercepts through wine. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-21classes/sstate: Update output hashJoshua Watt
Updates the output hash calculation for determining if tasks are equivalent. The new algorithm does the following based on feedback: 1) The output hash function was moved to the OE library. 2) All files are printed in a single line tabular format 3) Prints the file type and mode in a user-friendly ls-like format 4) Includes the file owner and group (by name, not ID). These are only included if the task is run under pseudo since that is the only time they can be consistently determined. 5) File size is included for regular files Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14meta: Fix Deprecated warnings from regexsRichard Purdie
Fix handling of escape characters in regexs and hence fix python Deprecation warnings which will be problematic in python 3.8. Note that some show up as: """ meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.   """ where the problem isn't on 1293 in package.bbclass but in some _prepend to a package.bbclass function in a different file like mesa.inc, often from do_package_split() calls. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14lib/sstatesig: Update to longer sha256 hash lengthRichard Purdie
Update the code to match the recent switch to sha256 hashes Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14rpm: produce sane amount of logging when creating a rootfsAlexander Kanavin
Previously there was lots of irrelevant noise in the logs, because we also wanted to be able to debug postinst issues easily. I have adjusted the logging levels so that postinst info is still written to the logs, but other things are not. [YOCTO #13119] Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14ccache.bbclass: Refactor it to make it more reliableRobert Yang
The previous ccache.bbclass has the following problems: - It uses host's ccache for native recipes, but this may not work on some hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always build failures (m4-native failed at do_configure, and others will also be failed if I disable CCACHE for m4-native) - native/nativesdk/cross/crosssdk recipes use host's ccache, but target uses ccache-native, this may confuse user. - The target recipes may use both host's ccache and ccache-native, this may cause unexpected problems and be hard to debug. This is because ccache-native is in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when rebuild target recipes, and then it would use hosttools/ccache, but the previous ccache files were generated by ccache-native. - Target recipes can't use ccache when no ccache is installed on the host: CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}" After refactored: All types recipes (native, target and others) will use ccache-native except ccache-native itself, host's cache won't be used any more. It is more reliable now, which will work everywhere when ccache-native can be built. And now we need use "CCACHE_DISABLE = '1'" to disable ccache for the recipe rather than "CCACHE = ''" since we set CCACHE in anonymous function, and d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we can't check whether CCACHE is set or not in anonymous function since it is always set. Use CCACHE_DISABLE to disable it would be more clear. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2019-01-11sdk: Fix SDKIMAGE_LINGUAS handlingRichard Weinberger
Currently SDKIMAGE_LINGUAS is broken for any inputs except "all". In the non-"all" case, each enabled language package is installed via pm.install("nativesdk-glibc-binary-localedata-%s.utf-8" % lang) This will throw a python exception since pm.install() expects a list of strings and not a string. Fix the problem by constructing a list. That way it is now also possible to call the package installer just once. Cc: "Burton, Ross" <ross.burton@intel.com> Fixes: 67615e01751b ("rootfs_rpm.bbclass: migrate image creation to dnf") Signed-off-by: Richard Weinberger <richard@nod.at> Signed-off-by: Ross Burton <ross.burton@intel.com>
2019-01-07oeqa: Fix for QEMU_USE_KVMRobert Yang
Fixed: MACHINE = "qemux86" QEMU_USE_KVM = "qemux86" IMAGE_CLASSES += "testimage" $ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs [snip] File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean raise ValueError("Invalid boolean value '%s'" % value) ValueError: Invalid boolean value 'qemux86' Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm will be enabled if target_arch == build_arch or both of them are x86 archs. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07sstate: Implement hash equivalence sstateJoshua Watt
Converts sstate so that it can use a hash equivalence server to determine if a task really needs to be rebuilt, or if it can be restored from a different (equivalent) sstate object. The unique hashes are cached persistently using persist_data. This has a number of advantages: 1) Unique hashes can be cached between invocations of bitbake to prevent needing to contact the server every time (which is slow) 2) The value of each tasks unique hash can easily be synchronized between different threads, which will be useful if bitbake is updated to do on the fly task re-hashing. [YOCTO #13030] Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26base.bbclass, classextend.py: Drop catering to gcc-initialKhem Raj
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15oe: Fix opkg status list parse - Missing postinstRaul Martins
While parsing opkg package status, last package status was not properly handled, resulting in final image without postinst and pkg depends Signed-off-by: Raul Martins <raul.martins@alta-rt.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15lib/oe/elf.py: Add powerpc64 architecture definition for muslSerhey Popovych
Add the ELF definition for the powerpc64 architecture when building with musl as libc. Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15lib/oe/recipeutils: Add a new function to mimic do_checkpkgRichard Purdie
The code in distrodata.bbclass related to the do_checkpkg task is rather dated, has holes in it (ignoring some recipes) and has horrible locking and csv related issues. We should use modern APIs such as tinfoil to make the calls we need directly against bitbake, cutting out the middleman and clarifing the code. This change imports the bits of distrodata.bbclass that are needed by the automated upgrade helper (AUH) into a standalone function which uses the tinfoil API. This can then be used by AUH and by the tests in oeqa/selftest/distrodata as well as by any other standalone script that needs this functionality. Its likely it can be further improved from here but this is a good start and appears to function as before, with slightly wider recipe coverage as some things skipped by distrodata are not skipped here (images, pieces of gcc, nativesdk only recipes). Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14lib/oe/utils: Set stderr for host_gcc_version()Robert Yang
Fixed: $ ln -s /usr/bin/ccache /folk/lyang1/bin/gcc $ rm -fr tmp/hosttools/ && bitbake -p [snip] ERROR: Error running gcc --version: It didn't print the error message, now it is: ERROR: Error running gcc --version: ccache: error: Could not find compiler "gcc" in PATH For the error itself, it is because ccache is not in my HOSTTOOLS, so this is an expected error. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05lib/oe/utils: Improve multiprocess_lauch exception handlingRichard Purdie
We've seen a cryptic: "ERROR: Fatal errors occurred in subprocesses, tracebacks printed above" message from oe-selftest with no other traceback information. Improve the traceback logging to try and give a better indication of any errors that is ocurring. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03lib/oe/recipeutils: drop obsolete functionsPaul Eggleton
These date from the time before Tinfoil's API covered this functionality (back when you could actually access cooker from a tinfoil-based script). Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03lib/oe/recipeutils: patch_recipe(): fix replacing varflag valuesPaul Eggleton
The code here wasn't correctly getting the variable history for varflags, so for example if you did a devtool upgrade on a recipe where the SRC_URI checksums were in the .inc file (typical for python recipes in order to support both python 2 and 3) then after the upgrade the new values would be set in the recipe and the old values were left in the .inc, which is not right. Teach the code here how to get the history for varflags so it works properly. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03lib/oe/recipeutils: patch_recipe(): fix handling of values across ↵Paul Eggleton
includes/classes If we were setting a variable and part of the variable's value was being set in a class or a .inc file, we were still just setting the value outright on the assumption that it was too hard to do otherwise. With some careful use of the variable history we can do better for certain situations i.e. when the recipe does not currently set the value outright. Additionally, correctly remove _appends for variables we are changing if we're trying to remove the value added in the _append. Fixes [YOCTO #12623] and partially fixes [YOCTO #9360]. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-22terminal: Cope with unreleased versions of tmuxMike Crowe
When tmux is built from a non-release Git version, its version number is "next-X" where X appears to be the expected version number for the next release. For example, when built from the current state of master, running "tmux -V" yields: tmux next-2.9 Currently check_tmux_pane_size only checks for the version being less than 1.9, so it seems unfair to fail with an obscure Python error in this case. Let's just use the version number after the "next-" prefix if it is present. Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21package_manager.py: fix the message used to catch failing postinsts from dnfAlexander Kanavin
Latest dnf versions have tweaked it. Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16lib/oe/package_manager: Avoid race problems when calling list_pkgs()Richard Purdie
list_pkgs() for rpm calls RpmPM() which would try and create a copy of the package feed. This can be called for example from buildhistory whilst some other task may be working on an SDK or image construction, causing the package feed to disappear part way through an operation. Avoid the need to copy the package index just to list the installed packages, avoiding the race. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-13locale: Allow usage of cross-localedef for ARCAlexey Brodkin
With this it's possible to build locale data for ARC and not do it instead on the first boot. Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-28package_manager.py: correct the deploydir when packagefeed-stability inheritedMingli Yu
After create_packages_dir added in below commit: 85e72e1 package_manager: Filter to only rpms we depend upon When add below line into conf/local.conf INHERIT += "packagefeed-stability" There comes below error when do_rootfs Exception: FileExistsError: [Errno 17] File exists: '/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm' -> '/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm' def create_packages_dir(d, subrepo_dir, deploydir, taskname, filterbydependencies): [snip] bb.utils.remove(subrepo_dir, recurse=True) [snip] In create_packages_dir function, there is a logic as bb.utils.remove(subrepo_dir, recurse=True) to clean subrepo_dir which is actually as example is /$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm. But currently when inherit packagefeed-stability class, the deploydir should be /$Prj/tmp/deploy/rpm-prediff, not the default /$Prj/tmp/deploy/rpm. If use /$Prj/tmp/deploy/rpm, then result in the logic as below: os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm") Update to the actual deploydir to guarantee the logic as below: os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm/i586/initscripts-1.0-r155.i586.rpm") Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18package_manager: handle remove() with no packagesRoss Burton
If remove() is called with an empty package list, ensure we do nothing instead of calling the underlying package manager with an invalid command line. [ YOCTO #12900 ] Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-10-18package_manager: implement PACKAGE_EXCLUDE for opkgRoss Burton
opkg has supported the --add-exclude option to install since 0.3.0, so use it to implement support for PACKAGE_EXCLUDE. Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>