summaryrefslogtreecommitdiffstats
path: root/lib
AgeCommit message (Collapse)Author
2014-01-10toaster: Toaster GUI, generic search, filter and orderAlexandru DAMIAN
This patch implements table searching, filtering and ordering, in a generic mode reusable for all tables. The search operates list of fields defined in the corresponding class for each model, search_allowed_fields. The search expression and filters are sent through GET requests using a QuerySet-like input. The inputs are filtered and validated before usage to prevent inadvertent or malicious use. Filters and table headers are defined in the views for each table, and rendered by generic code which is easily modified for various tables. The Build table and Configuration table are implemented using this framework as an example of how it should be used. [YOCTO #4249] [YOCTO #4254] [YOCTO #4255] [YOCTO #4256] [YOCTO #4257] [YOCTO #4259] [YOCTO #4260] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2014-01-10toaster: Build dashboard implementationAlexandru DAMIAN
This patch adds the build dashboard page implementation, which is the landing page for the Toaster GUI. Also adds correct links from the main build page to the various parts of the dashboard. [YOCTO #4258] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2014-01-10toaster: Create the base page navigation structureAlexandru DAMIAN
Updating the general container pages to use the graphical design and features from the design phase. In the process of adapting the Simple UI to the designed interface, we create all the pages and the navigation structure for the Toaster GUI. Views for each page have been added, and the url mapping has been updated to reflect newly added pages. The table page has been refactored to be component-oriented instead of class-oriented in order to facilitate reusage. Changes are made in different layers of the template (base, basetable) in order to maximize code reuse among different pages in the build. Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2014-01-10toaster: update static media for Toaster GUIAlexandru DAMIAN
In the process of making the GUI more usable, we bring in the static assets used during the design phase into the application. Licenses used: Bootstrap 2.3.2 under Apache License 2.0 Font Awesome under SIL OFL 1.1 (GPL compatible) and MIT licence GLYPHICONS Halflings released with Bootstrap, CC BY 3.0 jQuery 2.0.3 under MIT licence prettify.js under Apache Licence 2.0 Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2014-01-10toaster: clone Simple UI as base for Toaster GUIAlexandru DAMIAN
This patch clones the Simple UI to provide the base code for the development of the Toaster GUI. The clone takes the place of the application that was reserved for Javascript MVC code. The templates used for Simple UI are renamed to start with an "simple_" to prevent name resolution conflict with the Toaster GUI templates. Minor changes are made to the settings.py and urls.py in the toaster main section to account for the newly enabled application. Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2014-01-08fetch2/gitannex: Fix function arguments to match bitbake masterRichard Purdie
This code clearly uses an earlier fetcher API. Update it to match master. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08gitannex: Add missing file from previous commitRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08bitbake: fetch2/git: add description for nobranchZhenhua Luo
Signed-off-by: Zhenhua Luo <zhenhua.luo@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08bitbake: gitannex.py: Add Git Annex supportOtavio Salvador
This add a Git Annex backend which reuses the Git fetcher code; it allows managing files with git, without checking the file contents into git, being useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space. Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07bitbake: fetch2/git: add nobranch option for SRC_URI to skip SHA validating ↵Zhenhua Luo
for branch For rebased git tree, some commits doesn't exist in any branch, and such commits are valid in tag, the change is useful for such case. Signed-off-by: Zhenhua Luo <zhenhua.luo@freescale.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-07bitbake/lib/bb/build.py: fix the task flags cleandirsRobert Yang
The user manual said: 'cleandirs' - directories which should created before the task runs but should be empty But it only removes the dir, doesn't create it [YOCTO #5703] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-03bitbake/lib/bb/cooker.py: remove a duplicated self.dataRobert Yang
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02bitbake: fetcher2: clean(): remove the .patch.doneRobert Yang
There was a problem: $ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall Everything should be removed, but the 0006-omapfb-port-to-new-xserver-video-API.patch.done still exists in the DL_DIR, this is because the clean() in the fetch2/__init__.py skips removing the local file, so that it will skip removing the .done. The local file (file://) isn't needed to be removed since it is not downloaded into DL_DIR, but the .done should be removed, this patch will remove the .done, and it doesn't remove anything else since the clean() in local.py does nothing. [YOCTO #5687] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02bitbake: fetcher2: git.py: clean(): remove the .tar.gz.doneRobert Yang
There was a problem: $ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall The git2_git.pingu.fi.xf86-video-omapfb.tar.gz has been removed from the DL_DIR, but the git2_git.pingu.fi.xf86-video-omapfb.tar.gz.done still exists, this is because the "open(ud.donestamp, 'w').close()" in try_mirror_url() will create the git2_git.xxx.tar.gz.done, but no one removes it (the clean() in fetch2/__init__.py removes the DL_DIR/git2/pkg.done) This only happens on the git fetcher AFAIK. [YOCTO #5688] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02bitbake: replace 3 spaces with 4Robert Yang
A part of fetch2/__init__.py uses 3 spaces as the indent, I think that they should be typos. Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22fetch2: avoid printing "no checksum" error message twicePaul Eggleton
Because of the way we were handling this error, it was printed twice - once via logger.error() (to avoid the log being printed) and a second time when the exception gets wrapped in a FuncFailed at a higher level. Call logger.error() earlier and change the text we send in the exception to be more brief, so it more closely resembles the behaviour when there is an invalid checksum. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22fetch2: fail checksum validation if SRC_URI checksums set to ""Paul Eggleton
We were checking SRC_URI md5sum/sha256sum values against None here, so if they were set to "" then no error was produced. Since the value is still effectively unset in this case, this is not the right behaviour; just check if the value doesn't evaluate to False instead. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20Update to version 1.21.1 for masterRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20siggen: Fix reversed difference outputRichard Purdie
The output when comparing siginfo files for dict_diff is reversed and shows additions when things were removed and vice versa. This patch reverses the operation so the changes are shown correctly and makes the output less confusing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20runqueue: Further extend bitbake -S output to view signature differencesRichard Purdie
Based upon the list of difference starting points, we can use the siggen.find_siginfo() function call and the difference printing code to provide a list of differences between the current build target and whatever can be obtained from the sstate cache. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-19siggen: When printing signatures recursively, limit the outputRichard Purdie
Currently the code prints all differences. If the task dependencies have changed hash, we recurse into those and print those differences as well. This leads to a lot of output. The reality is if the parents changed signature, we might as well just say that and recurse with no other output since we're much more interested in how the parents changed in nearly all cases. The changes in the parent are probably the same ones we'd have printed at each level anyway. By doing this we focus the output more carefully on the thing the user wants/needs to see. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-19runqueue: Fix data being written into siginfo/sigdata filesRichard Purdie
The way hash_deps was being generated was different to the way siggen generated the data internally which lead to seemingly different sigdata/siginfo files for the same checksum. The -S output correct but the files written during builds contained superflous data which would look like a difference. This patch removes the badly duplicated data and uses it from the source which ensures its consistent. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18runqueue: Add output for -S option for listing the changepoints compared ↵Richard Purdie
with an sstate cache Its useful to understand where the delta starts against an existing sstate cache for a given target. Adding this to the output of the -S option seems like a natural fit. We use the hashvalidate function to figure this out and assume it can find siginfo files for more than just the setscene tasks. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18bb.fetch2.git: reuse basecmd attributeOlof Johansson
The basecmd is initialized in urldata_init; there's no need redoing that work. Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18fetch2/git: Improve handling of unresolved names verses branchesRichard Purdie
Currently the fetcher doesn't distinguish between names that the fetcher needs to resolve verses branch names that the user specified. This meant that if you specify a tag and a branch, the fetcher broke. This separates the two so that the branch name is preserved and can be used in appropriate places. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18build/ast: Create strong task add/del API in bb.buildRichard Purdie
Currently its near impossible to control task addition/deletion from metadata context. This adds stong add/deltask API to bb.build which is traditionally where it resided. The rather broken remove_tasks function was removed, it didn't appear to do anything useful or have any users. This allows us to clean up hacks currently in use in metadata and use standard API for it instead. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18ast/BBHandler/build: Add support for removing tasks (deltask)Richard Purdie
Back in the depths of time we did support task removal. In the pre AST days it was nearly impossible to continue supporting it, it wasn't used so it was dropped. With the modern codebase we can easily now support deltask and it would be very useful within the metadata since it can massively simplify dependency trees. As an example, a core-image-sato had 47703 inter task dependencies before this patch and a patch to native.bbclass, afterwards with the noexec tasks deleted, we had 29883. Such a significant simplification is worthwhile and justifies adding a deltask operation to the system. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-13data_smart: Fix hash corruption issueRichard Purdie
We were accidentally using references to sets in the contains functionality instead of creating a copy. This could cause data corruption and corruption of the resulting sstate checksums. This patch fixes this to make a copy of the set and resolved the corruption issue. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10process: Add timeout to select callRichard Purdie
On some machines, a hang has been noticed where the system sits in the select call despite the task having completed. The exact reasons for this as unknown but adding a timeout unblocked the builds and resolved the hangs in question. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10hob: after adding a layer, hob do not parse the configurationCristiana Voicu
Moved the parseConfiguration method before obtaining the machines and other configurations from bitbake. If not, Hob doesn't see the new machines added by the new layer. [YOCTO #5632] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-09toaster: Add more information for tasks in Simple UIAlexandru DAMIAN
This patch adds more information about Tasks in the Simple UI: * all local file system information is not listed in a single column, showing the common relationship * adding the display for the location of the task source * we display the work directory for each task Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: add pagination to the Simple UIAlexandru DAMIAN
In an effort to make the Simple UI more usable and reponsive, this patch adds pagination support for the pages with lots of entries: Builds, Configuration and Tasks. Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: migrate orm models to SouthAlexandru DAMIAN
As Toaster advances, database schema alteration will force users to delete old versions and lose all data collected. In order to prevent this, and to allow database updates to happen without having to delete old data, we use South to handle migrations for the ORM application which stores the Toaster data. [YOCTO #5559] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: update to Django 1.5Alexandru DAMIAN
In order to remain up to date with the relevant technologies, Toaster is updated with this patch to Django 1.5. This also makes headways to allow usage of emerging Django-related technologies. Changes include the startup script Django version check, usage of TemplateView instead of deprecated simple function to do redirects, and update to the new form of the _url_ template tag. Support for Django 1.4.5 is now deprecated. [YOCTO #5558] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toasterui: do not link non-image targetsAlexandru DAMIAN
In the Simple UI, builds table, targets that are images have link to the list of installed packages. There is no point in having links enabled for the non-image targets, so we don't link in this case. [YOCTO #5366] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toasterui: fix version numbers for no PE specifiedAlexandru DAMIAN
PE is an optional field in recipes specifiying the epoch for the recipe. The canonical form for the full recipe version string is: PE:PV-PR If no PE is specified, we shouldn't store the initial ":" character, as it leads to inconsistency with how the version string is used elsewhere. This patch drops the leading ":" [YOCTO #5459] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: change package storage modelAlexandru DAMIAN
Up until this patch, package information lived in two places - one table for build packages and one table for target installed packaged. This situation leads to two problems: there is no direct link between a build package and a installed package, and a lot of data is duplicated. This change unifies all package types in a single table. The SimpleUI remains the same for continuity sake, but the REST API will be changed in a future patch. The package dependencies and package files are now kept in a single table. Since we collect target installed package information at all times, we need to expand it to supplement missing information if a package is not actually built in the current build. Small changes to the Simple UI reflect the updated database schema. [YOCTO #5565] [YOCTO #5269] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: update build stats readingAlexandru DAMIAN
In the processes of removing local system accesses from toaster UI, we remove the build stats code that was moved to toaster.bbclass, and adapt the database writing code to read the data from BuildStatsList event sent by the toaster.bbclass [YOCTO #5604] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: move layer information reading codeAlexandru DAMIAN
In the processes of removing local system accesses from toaster UI, we remove the layer data reading code that was moved to toaster.bbclass, and adapt the database writing code to read the data from event sent by the toaster.bbclass [YOCTO #5604] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: remove Author field in SimpleUIAlexandru DAMIAN
Removes a left-over Author field in the Simple UI recipe page. [YOCTO #5449] Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09toaster: Analysis API variablehistory updateAlexandru DAMIAN
This patch updates the Analysis REST API to expose the variablehistory information. A UI tool can query this API to get a picture on how a variable value got to its final form. The documentation for VariableHistory is updated on Wiki https://wiki.yoctoproject.org/wiki/Analysis_REST_API_Contracts Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
2013-12-09data: Ensure we add the contains keys in a particular orderRichard Purdie
If we don't sort the data, the values can reorder changing the signatures meaning we get confused builds and significant cache misses. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-06fetch2/git: Ensure check is made in the correct directoryRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05toaster: exclude "set in documentation.conf" from variable historyCristiana Voicu
That has no impact on the builds themselves, so we should just remove that line from the variable history. [YOCTO #5561] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05hob: changes to INHERIT var should use "append" operationCristiana Voicu
Setting INHERIT var means to remove other operations made on INHERIT. This is too intrusive, so we decided to use append for this case. [YOCTO #5448] Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05fetch2/git: Add sanity check to ensure we really did fetch the correct revisionsRichard Purdie
The fetcher made the rather bold assumption that if it fetched from the upstream, the revisions were present and correct. These checks are fast and ensure that really is the case. The avoids accidental network accessed and missing branch configuration problems. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03Hob: check if warnings are related to package buildValentin Popa
If warnings come from recipes parsing and not from package build, 'parent' object will be None; so don't update the color for it. [YOCTO #5621] Signed-off-by: Valentin Popa <valentin.popa@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03hob: fix package property dialog for changes to FILES_INFOPaul Eggleton
The FILES_INFO structure is now much simpler, so remove all of the horrible mangling we had to do here in order to read it. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03hob: name package files list variables appropriatelyPaul Eggleton
This treeview code was obviously copy-pasted from one of the config dialogs and the variables were never renamed. Rename them now to improve readability. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03lib/bb/ui: handle PKGSIZE change to bytesPaul Eggleton
PKGSIZE is now in bytes in pkgdata, so we need to treat it as such in the UI code for Hob / Toaster. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>