aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oe/package_manager.py
AgeCommit message (Collapse)Author
2014-05-08package_manager: Fix NoneType Object on do_populate_sdkRicardo Ribalda Delgado
PACKAGE_EXCLUDE can be not defined or empty, leading to a build error. File: '/var/lib/jenkins/jobs/qt5022-cesium/workspace/repo/yocto/meta/lib/oe/package_manager.py', lineno: 1649, function: _create_configs 1645: "Pin-Priority: %d\n\n" % (arch, priority)) 1646: 1647: priority += 5 1648: *** 1649: for pkg in self.d.getVar('PACKAGE_EXCLUDE', True).split(): 1650: prefs_file.write( 1651: "Package: %s\n" 1652: "Pin: release *\n" 1653: "Pin-Priority: -1\n\n" % pkg) Exception: AttributeError: 'NoneType' object has no attribute 'split' ERROR: Function failed: do_populate_sdk ERROR: Logfile of failure stored in: /var/lib/jenkins/jobs/qt5022-cesium/workspace/build/tmp/work/bobcat_64-poky-linux/meta-toolchain/1.0-r7/temp/log.do_populate_sdk.21363 NOTE: recipe meta-toolchain-1.0-r7: task do_populate_sdk: Failed Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-05-08package_manager: Fix Argument list too longRicardo Ribalda Delgado
Function buildhistory_list_installed_image fails with error "Argument list too long". This patch uses a temporal file to pass the package list to opkg-query-helper.py File: '/var/lib/jenkins/jobs/qt5022-cesium/workspace/repo/yocto/meta/lib/oe/package_manager.py', lineno: 421, function: list 0417: try: 0418: output = subprocess.check_output("echo -e '%s' | %s" % 0419: (output, opkg_query_cmd), 0420: stderr=subprocess.STDOUT, *** 0421: shell=True) 0422: except subprocess.CalledProcessError as e: 0423: bb.fatal("Cannot compute packages dependencies. Command '%s' " 0424: "returned %d:\n%s" % (e.cmd, e.returncode, e.output)) 0425: Exception: OSError: [Errno 7] Argument list too long ERROR: Function failed: buildhistory_list_installed_image ERROR: Logfile of failure stored in: /var/lib/jenkins/jobs/qt5022-cesium/workspace/build/tmp/work/qt5022-poky-linux/qimage-dev/1.0-r0/temp/log.do_rootfs.16747 NOTE: recipe qimage-dev-1.0-r0: task do_rootfs: Failed ERROR: Task 7 (/var/lib/jenkins/jobs/qt5022-cesium/workspace/repo/yocto/../qtec/meta-qt5022/recipes-core/images/qimage-dev.bb, do_rootfs) failed with exit code '1' NOTE: Tasks Summary: Attempted 4999 tasks of which 30 didn't need to be rerun and 1 failed. NOTE: Writing buildhistory Auto packing the repository for optimum performance. Summary: 1 task failed: /var/lib/jenkins/jobs/qt5022-cesium/workspace/repo/yocto/../qtec/meta-qt5022/recipes-core/images/qimage-dev.bb, do_rootfs Summary: There were 74 WARNING messages shown. Summary: There were 2 ERROR messages shown, returning a non-zero exit code. Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-03-28package_manager.py: leave the __db.00* files in placeLaurentiu Palcu
Do not delete the __db.00* files in the PackageManager class. Leave this operation up to the client classes. One side effect of this deletion was the following message appearing in the output of the next rpm command executed: rpmdb: BDB1540 configured environment flags incompatible with existing environment We might also gain some time here by not deleting/creating those files very often. [YOCTO #6049] Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-26apt/package_mamager: Ensure WORKDIR is used for lists directoryRichard Purdie
The native sysroot should not be used as a store for the lists files since multiple images running at once would conflict over this. Instead redirect this to WORKDIR. This means some extra directories need to be created. Also create apt.conf.d to silence some warnings. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-26package_manager: Fix apt-ftparchive index creation problemsRichard Purdie
a) There were missing parameters to the release and package commands (".") b) The commands need to be executed as one block since they build upon each other Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-26package_manager: Ensure we don't process directories twiceRichard Purdie
Processing directories twice is both pointless and introduces a race condition. When building the list, ensure duplicates (like "all" and "noarch") are handled correctly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-21package_manager.py: create separate class for installed packages listingLaurentiu Palcu
This commit creates a new class that has the only purpose to generate various listings of installed packages in the rootfs. Basically, the methods involved in listing the installed packages, that were part of each backend PM class implementation, were moved to this new class. This change avoids instantiating a new PM object just to get the list of installed packages in a certain rootfs. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-19package_manager: Use apt-ftparchive for deb packagesRicardo Ribalda Delgado
Use apt-ftparchive to create a Release file compatible with SecureApt. apt-ftparchive is also a more efficient replacement of dpkg-scanpackages: root@neopili:~/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/bobcat _64# time PSEUDO_UNLOAD=1 apt-ftparchive packages . >/tmp/kkk real 0m26.873s user 0m20.968s sys 0m1.212s root@neopili:~/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/bobcat _64# time PSEUDO_UNLOAD=1 dpkg-scanpackages . >/tmp/kkk dpkg-scanpackages: info: Wrote 6022 entries to output Packages file. real 0m59.721s user 0m16.668s sys 0m11.164s apt-ftparchive is not compatible with libpseudo. The calls to ftw() returns the path in absolute format instead of relative. This produces wrong Packages and Release files. ie: MD5Sum: d20227a958f6870137ce0e41b7b84307 1453 /home/ricardo/curro/qtec/qt5022/build-qt5022-cesium/build/tmp/deploy/deb/all/Release This is why it is called with PSEUDO_UNLOAD. Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-07do_rootfs: Added PACKAGE_FEED_URIS functionalityDavid Nyström
Adding a common interface to add predefined package manager channels to prebuilt rootfs:es. Adding PACKAGE_FEED_URIS = "http://myre.po/repo/, will assume repo directories named (rpm,ipk,deb) as subdirectories and statically add them to the rootfs, using the same PKG_ARCHs as the build which produced the images. Tested with RPM, IPK and DEB. deb feed functionality seem broken, is anyone using this ? Signed-off-by: David Nyström <david.c.nystrom@gmail.com> Signed-off-by: David Nyström <david.nystrom@enea.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-03-07package_manager.py: make list_installed() list pkg dependencies tooLaurentiu Palcu
list_installed("deps") will now return the package dependencies. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-03-07image.py, rootfs.py, package_manager.py: redirect stderr to stdout when ↵Laurentiu Palcu
calling check_output() If a command executed with subprocess.check_output() fails, the subprocess.CalledProcessError.output contains only STDOUT and the user needs to check the log.do_rootfs to see any other details. This commit forwards stderr to stdout so that, in case of failure, the entire error output will be displayed in terminal. [YOCTO #5902] Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-03-05package_manager.py: RpmPM: don't add smart channel if already addedLaurentiu Palcu
Make sure the channel was not already added, before attempting to add. [YOCTO #5890] Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-05Revert "lib/oe/package_manager.py: RpmPM: fix issue with multilib builds"Laurentiu Palcu
Converting a list to a set does not keep the order of the items, as they were added, which might result in the wrong packages being installed in the final image... This reverts commit 12f47c23df8c109676f66d580d666a3147f3b046. [YOCTO #5890] Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25package_manager.py: create index files for all backends in PACKAGE_CLASSESLaurentiu Palcu
The previous implementation was checking IMAGE_PKGTYPE and created the index files just for the backend used to create the image. Apparently, 'bitbake package-index' should attempt to create the index files for all backends specified in PACKAGE_CLASSES. [YOCTO #5827] Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25package_manager.py: correctly handle empty opkg-query-helper.py outputJonathan Liu
If the output from opkg-query-helper.py is empty, output.split('\n') would result in a list containing one element which is an empty string while iterating over each line in the output. An exception is then thrown by the line: pkg, pkg_file, pkg_arch = line.split() with the message: Exception: ValueError: need more than 0 values to unpack To avoid this, we add a condition to only split the output if it isn't empty. Signed-off-by: Jonathan Liu <net147@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-21package_manager.py: support ipk incremental image generationHongxu Jia
Add the following three functions to OpkgPM class: - The 'dummy_install' is used to dummy install pkgs, and returns the log of output; - The 'backup_packaging_data' is used to back up the current opkg database; - The 'recover_packaging_data' is used to recover the opkg database which backed up by the previous image creation; Tweak 'remove' function in OpkgPM class, which the options for remove with dependencies was incorrect. Tweak 'handle_bad_recommendations' function in OpkgPM class: - Fix none value check; - Add the existance check of opkg status file; [YOCTO #1894] Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
2014-02-20package_manager.py: Fix $i -> %s in bb.noteMartin Jansa
* change the text a bit, because I have trouble parsing two verbs at the beginning of sentence. Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-02-20package_manager.py: fix installed package list creation for Opkg/DpkgLaurentiu Palcu
Small error in the package list creation routine. Buildhistory was supposed to use this but was never called. Hence, it escaped tests... Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2014-02-16lib/oe/package_manager: Make empty package directories non-fatalRichard Purdie
Its perfectly reasonable to call -c package-index with empty package directories. This ensures the builds don't fail in such cases, resolving autobuilder failures. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-14package_manager.py: move multilib prefix list computation function to RpmIndexerLaurentiu Palcu
Since the code from anonymous function in rootfs_rpm.bbclass has been removed, MULTILIB_PREFIX_LIST variable was never set. Hence not all directories got indexed. This commit will move the multilib prefix list computation function from RpmPM class to RpmIndexer, since the indexer needs it too. I was hoping to avoid this but, unfortunately, I couldn't. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-14package_manager.py, rootfs.py, sdk.py: add Indexer classLaurentiu Palcu
Because the package-index.bb needs to create package indexes outside do_rootfs environment, move the indexing capability out of PackageManager class to a smaller Indexer class. This commit: * simply moves the indexing functions for ipk/deb with no changes; * rewrites the RPM indexing function so that it can be easily moved out of the PackageManager class; * removes some RPM duplicate code, moves it into a method inside RpmPM class and changes the RpmPM constructor so that the new method is effective; Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11image.py, package_manager.py, rootfs.py: dump command output on errorLaurentiu Palcu
Print the entire command output in case of errors. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: RpmPM: fix issue with multilib buildsLaurentiu Palcu
Use python sets instead of lists, to avoid duplicates. When doing a multilib build, "smart channel --add" fails because it tries to add 'all' channel twice. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: check that package list is not emptyLaurentiu Palcu
If we're just attempting to install packages and the package list is empty, just return. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11rootfs.py, package_manager.py, sdk.py: Fix building from feeds feature for opkgLaurentiu Palcu
When using opkg as the PM backend, one has the option to provide custom feeds to create the rootfs from. This commit: * fixes this in the refactored code; * moves the custom config creation code to python; * clean up the package-ipk.bbclass; Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: OpkgPM, bad recommendation handling changeLaurentiu Palcu
The following commit: rootfs_ipk: Ensure that BAD_RECOMMENDATIONS are honoured for all architectures changed the way BAD_RECOMMENDATIONS are handled. Make the change in the new code too. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: do not use stampsLaurentiu Palcu
Using stamps shortens rootfs/image creation because the package indexes are not created again if no new package was installed in the deploy dir. Unfortunately, there are some syncronization issues with do_package_write_*() and the indexes are not properly created. Will have to revisit this issue in the near future. For now, lose it. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: OpkgPM/DpkgPM fixesLaurentiu Palcu
This commit: * adds missing mark_packages() function for OpkgPM. This is needed to * manually change the packages installation status in the status file; * fix OpkgPM remove_packaging_data() issue; * fix OpkgPM handle_bad_recommendations() issue; * improve OpkgPM/DpkgPM mark_packages() regex pattern; * fix DpkgPM list_installed() problem; * fix DpkgPM _create_configs() problem: status file should not be truncated if it already exists/ Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: DpkgPM, use lock on deploy dirLaurentiu Palcu
Lock deploy directory against concurrent index creation. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: RpmPM fixesLaurentiu Palcu
This commit: * fixes a crash when handling interecept hook failures which happened when MULTILIB_GLOBAL_VARIANTS was not set; * convert dashes to underscores and use sets (so that we make sure the items are unique) when creating RPM repos; * uses a regex pattern to search for packages in the feeds list. The old method could match also strings in the middle. For example: 'rpm' matched 'kernel-module-lttng-probe-rpm" in qemux86_64 feeds; * issue a bb.fatal if smart returns error while installing packages. Otherwise we might end up with an incomplete image... * fixes the /etc/rpm/platform file creation; Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: moving global SCRIPTLET_FORMAT and ↵Hongxu Jia
DB_CONFIG_CONTENT to RpmPM The tweak made the code looks better. Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11package_manger.py/rootfs.py: rename save_rpmpostinist with save_rpmpostinstHongxu Jia
The tweak made the code looks better. Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: fix invoking rpm save_rpmpostinist failedHongxu Jia
- Use self.target_rootfs instead of self.image_rootfs as saved dir; - Remove redundant comment line; Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: support RpmPMHongxu Jia
- Implementation RpmPM class - Support rpm incremental image generation Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: fixes for dpkg backendLaurentiu Palcu
This commit contains the following fixes: * pass the apt config directory to the DpkgPM constructor, so one can instantiate this class multiple times and give it different config files (like for creating SDK); * change constructor argument name from 'dpkg_archs' to 'base_archs'; * export APT_CONFIG environment variable before calling apt-get, not in constructor. If done in constructor, the last class instantiation, sets the environment, which is note desireable; Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: add support for opkg backendLaurentiu Palcu
Additionaly, the commit contains a couple of minor changes (comments, error printing, etc). Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: use the bitbake APT_ARGS variableLaurentiu Palcu
This commit will revert on using the bitbake APT_ARGS variable, so users can alter the way apt is called without needing to change it in code. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: add deploy dir locking mechanismLaurentiu Palcu
This is needed in order to serialize the index file creation when multiple do_rootfs tasks are running in the same time. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
2014-02-11lib/oe/package_manager.py: add new package management libraryLaurentiu Palcu
This new library is intended to be used by the new python rootfs creation code. It implements the rpm/dpkg/opkg package management backends: RpmPM, DpkgPM and OpkgPM. The base API is this: update() install() install_complementary() remove() write_index() remove_packaging_data() list_installed() All implementations have to provide these functions. Some backends may need to implement additional functions though. Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-18 05:48:46 +0000