aboutsummaryrefslogtreecommitdiffstats
path: root/bitbake/lib/toaster/orm
AgeCommit message (Collapse)Author
2016-11-30bitbake: toaster: orm models Handle CustomImageRecipe BRLayer hereMichael Wood
The schedule_build function on the project object is where the BRLayers are created for the build. Instead of creating the BRLayer for the CustomImageRecipe in the localhostbbcontroller create it here so that all that mechanism is in one place. Also fix a number of pyflake errors. (Bitbake rev: f8d3ea784937b6e416d3e5a4feb1283c478e4caa) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: toaster: orm models Project class Fix pyflake errorsMichael Wood
(Bitbake rev: 69f33397083f54f977fa0cd4dd621731f32fd034) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: toaster: buildinfohelper toaster-custom-images layerMichael Wood
This fixes the unidentified layers issue by making the toaster-custom-images layer a local layer. By doing this we also fix the git assumptions made for the local layers which stop recipes and other meta data being associated with them. This also removed some of the special casing previously needed when we didn't have the concept of a local (non git) layer. Also rename created flag var to a have a different var for each returned value so that the same value isn't used multiple times. [YOCTO #10220] (Bitbake rev: ba5332d4960d7f4f79aef63136796e2fa67284e3) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: toaster: orm gen_layerdeps Protect against circular Layer dependenciesReyna, David
Limit the recursion (to say 20 levels) when processing layer dependencies so that circular dependecies do not cause infinite decent and an out-of-memory failure. The duplicate found layers are already immediately filtered in the code. [YOCTO #10630] (Bitbake rev: e9efef0bdb8068984c3013b87aac9e872ffb38ae) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30bitbake: toaster: runbuilds Write the pidfile in python rather than shell scriptMichael Wood
Write the pid file out in the start up of this management command. This ensures this has happened instead of relying on the shell command having been run which may or may not be the case. This also makes it simpler for testing. Couple of clean ups of runbuilds as identified by pyflake (Bitbake rev: 999e980ee1a58d16f33ef6c0e41aecdcd0206f39) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23bitbake: toaster: layerindex updater Take into account layers being predefinedMichael Wood
As we can now provide layer definitions through fixtures we need to be more clever how we update the metadata in the database to avoid duplicate metadata being created. To do this we make more effort to match existing data in the database and update only the fields which will be better provided by the layer index. This removes the need for us to special case layers which are provided as part of poky such as openembedded-core or meta-poky which exist on the layerindex but with different git urls. (Bitbake rev: f981b68f66718d5b196684f4e378a5f195ff0337) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23bitbake: toaster: orm/fixtures Add the master release and correct morty releaseMichael Wood
Add the master release option to base your project on and correct the morty release so that for poky based setups we use the poky provided version of the layer rather than checking out the layer from its own git repository. [YOCTO #10497] (Bitbake rev: c83ab92362378b22d3f4d6119bf362f704577ca2) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23bitbake: toaster: settings fixture Set default release to masterMichael Wood
Now that morty has been released we now set the DEFAULT_RELEASE back to master. (Bitbake rev: 115cee16a9aecfcd1061bb106ebca4e861d9b296) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-04bitbake: toaster: use current directory if BUILDDIR is not setEd Bartosh
If BUILDDIR environment variable is not set signal_runbuilds function throws TypeError as os.getenv('BUILDDIR') returns None: ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str' Traceback (most recent call last): File "bitbake/lib/bb/ui/toasterui.py", line 391, in main buildinfohelper.update_build_information(event, errors, warnings, taskfailures) File "bitbake/lib/bb/ui/buildinfohelper.py", line 1184, in update_build_information self.internal_state['build'], errors, warnings, taskfailures) File "bitbake/lib/bb/ui/buildinfohelper.py", line 238, in update_build_stats_and_outcome signal_runbuilds() File "bitbake/lib/toaster/orm/models.py", line 1746, in signal_runbuilds '.runbuilds.pid')) as pidf: File "/usr/lib64/python3.4/posixpath.py", line 82, in join path += b TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str' Used os.getenv('BUILDIR', '.') to make it always return meaningful directory path. Current directory '.' will be used if BUILDDIR is not set. (Bitbake rev: da631152a0db3f432709a05ff15a268d784ca3ab) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-04bitbake: toaster: orm models Handle run builds process not yet runningMichael Wood
During tests we may want to call the runbuilds process manually for example when doing a "one shot" approach rather than a long running process during tests. (Bitbake rev: 60d3f93836da5523705b0b2e25567d1c9040ec89) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15bitbake: toaster: Update default release to Mortybrian avery
Set Morty to be the default release in toaster for the Morty release when creating new projects. (Bitbake rev: 00f79096f639ce3a9c0b7c72cfb36f14e264733d) Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15bitbake: toaster: Update poky fixture for Morty releasebrian avery
Update the poky fixture to the Morty release. This removes the master branch from the release and limits it to the morty branch. Normally, we would also support at least one past branch but the change from Python 2.7 -> Python 3 makes that infeasible. (Bitbake rev: 2674ca33b900f4f3f16be504d7c67d0fc69c1c2d) Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-15bitbake: toaster: Update oe-core fixture for Morty releasebrian avery
Update the oe-core fixture to the Morty release. This removes the master branch from the release and limits it to the morty branch. Normally, we would also support at least one past branch but the change from Python 2.7 -> Python 3 makes that infeasible. (Bitbake rev: 93f1e6c3d022b1f12a230879160efa941cb1e250) Signed-off-by: brian avery <brian.avery@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21bitbake: toaster: notify runbuilds when build status changesEd Bartosh
Called signal_runbuilds API when build is scheduled, cancelled or finished to notify runbuilds process about builds status change. [YOCTO #8918] (Bitbake rev: fe08f0fa4b328908e73695ebbceca87bc86a49f9) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-21bitbake: toaster: implement signal_runbuilds functionEd Bartosh
This function reads pid of runbuilds process from BUILDDIR/.runbuilds.pid and sends SIGUSR1 to it. signal_runbuilds function will be used in Toaster code to notify runbuilds when build is scheduled, finished or cancelled. (Bitbake rev: 62955224a6d99e9f581d2bef924058070bfa4c43) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08bitbake: toaster: orm Update IMAGE_FSTYPES valuesBelen Barros Pena
This patch fixes a typo in one of the IMAGE_FSTYPES values listed in Toaster. It also updates the hardcoded list of values to match the latest list in meta/classes/image_types.bbclass [YOCTO #9447] (Bitbake rev: 46db3279cb81b3ca6ce047204aee620f5ee51220) Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-02bitbake: toaster: Fix oe-core fixtureMichael Wood
Due to a copy paste error we managed to get some of the wrong information in the oe fixture that provides a suggested default settings for Toaster. This meant it tested correctly when it shouldn't have. Fix: - The use of local bitbake - An incorrect call to realpath which didn't include its parent module. - The field used for the local_dir of an existing openembedded-core (Bitbake rev: d57a9124650e5367919668dfccf6aad4962a77f1) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17bitbake: toaster: update web urls for openembedded-core's special caseDavid Reyna
The layer index update command has a special case for the updating 'openembedded-core' layer, and it was missing reading and updating the git web URL fields. [YOCTO #8037] (Bitbake rev: ce2f990a366d2d939e93e01f67688f12740c5fee) Signed-off-by: David Reyna <david.reyna@windriver.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: bldcontrol model BRLayer Add corresponding local_source_dirMichael Wood
Sync the BRLayer object with the new field added to the Layer object. The BRLayer (BuildRequest Layers) are snapshots of the layers in the project at build time and therefore need to mirror the required fields of the layer object. (Bitbake rev: a3112c922f036425977abffa0137b9133f61fcd6) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: add local_source_dir field to modelSujith H
Add a new field local_source_dir to model. This will clearly differentiate us from the vcs_url which is for git path. Adding migration file 0010_layer_local_source_dir_path.py along with this patch. [YOCO #9911] (Bitbake rev: 83763d89f9d0bc535e930a2094ba8201675d40be) Signed-off-by: Sujith H <sujith.h@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: fix unhandled exceptionEd Bartosh
manage.py lsupdates throws DataError exception if the recipe can't be saved to the MySQL database: django.db.utils.DataError: (1406, "Data too long for column 'license' at row 1" Adding DataError exception to the list of exceptions should make lsupdates to print a warning message and skip the recipe. (Bitbake rev: 01891c13ae8dcba64e5cf79956f1b64c2afdaae3) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: add started property to BuildElliot Smith
Add a property to the Build model which records whether the BuildStarted event has occurred for the build. The proxy for this event is the presence of variables recorded against the Build: as the buildinfohelper only saves variables when the BuildStarted event occurs (as the variables aren't available on the bitbake server before that point), we can tell whether BuildStarted has happened by counting Variable objects on the Build. This can then be used to determine whether a Build "properly" started, enabling a different dashboard display (left-hand menu hidden) if the build didn't record any useful information (e.g. if it had a bad target). [YOCTO #8443] (Bitbake rev: aa151a4d2de4a54fe3075a8c56a4935158398a18) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: show "Tasks starting..." until the first task completesElliot Smith
To prevent showing a "0% of tasks complete" message for a long time, don't show the progress bar until the first task has finished. While waiting for that first task, show a message about tasks starting instead. [YOCTO #9631] (Bitbake rev: 5529bcd860d2932b967a064ae28690ac5a725342) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: show progress of recipe parsing in recent builds areaElliot Smith
Modify buildinfohelper and toasterui so that they record the recipe parse progress (from ParseProgress events in bitbake) on the Build object. Note that because the Build object is now created at the point when ParseStarted occurs, it is necessary to set the build name to the empty string initially (hence the migration). The build name can be set when the build properly starts, i.e. at the BuildStarted event. Then use this additional data to determine whether a Build is in a "Parsing" state, and report this in the JSON API. This enables the most recent builds area to show the recipe parse progress. Add additional logic to update the progress bar if the progress for a build object changes. [YOCTO #9631] (Bitbake rev: f33d51d46d70e73e04e325807c1bc4eb68462f7b) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: move most recent builds templating to clientElliot Smith
The most recent builds area of the all builds and project builds table needs to update as a build progresses. It also needs additional functionality to show other states (e.g. recipe parsing, queued) which again needs to update on the client side. Rather than add to the existing mix of server-side templating with client-side DOM updating, translate all of the server-side templates to client-side ones (jsrender), and add logic which updates the most recent builds area as the state of a build changes. Add a JSON API for mostrecentbuilds, which returns the state of all "recent" builds. Fetch this via Ajax from the build dashboard (rather than fetching the ad hoc API as in the previous version). Then, as new states for builds are fetched via Ajax, determine whether the build state has changed completely, or whether the progress has just updated. If the state completely changed, re-render the template on the client side for that build. If only the progress changed, just update the progress bar. (NB this fixes the task progress bar so it works for the project builds and all builds pages.) In cases where the builds table needs to update as the result of a build finishing, reload the whole page. This work highlighted a variety of other issues, such as build requests not being able to change state as necessary. This was one part of the cause of the "cancelling build..." state being fragile and disappearing entirely when the page refreshed. The cancelling state now persists between page reloads, as the logic for determining whether a build is cancelling is now on the Build object itself. Note that jsrender is redistributed as part of Toaster, so a note was added to LICENSE to that effect. [YOCTO #9631] (Bitbake rev: c868ea036aa34b387a72ec5116a66b2cd863995b) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: fixtures Add READMEMichael Wood
Add README to explain fixtures directory (Bitbake rev: 4060776e408ec6baec3bafba79ac5db1bc380f01) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: lsupdates Add spinner for parsing/http fetchMichael Wood
Adds a spinner so that you know that the parse and http fetch from the layerindex is in progress. (Bitbake rev: e1c1c8827f3892551084bf1c0909c1b33f0dca83) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-11bitbake: toaster: Add poky and openembedded configuration fixturesMichael Wood
These fixtures provide a recommended default configuration of toaster for either using bitbake and oe-core or as part of poky. They can be used as the sample configuration for writing custom configurations. (Bitbake rev: d2f4ffbf061e874a7731441f8346fdc568e9f07f) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-01bitbake: lib/toaster: Fix missing new files from previous commitsRichard Purdie
(Bitbake rev: f77e6f21a2cc57a3fcb5970437e55cfae39849a3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: admin Add Layer_Version to the admin-able modelsMichael Wood
If the migration didn't get the release conversion right for say, a local or imported layer it would be handy to be able to edit this in the django admin page. Also useful for developers to be able to tweak layers on the fly. (Bitbake rev: 0b23b6919ef1162a0c7fb5b5a961a24653c51eb2) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: orm Remove the layerindex specific up_branch fieldsMichael Wood
We don't need to keep track of layerindex data in our database. And using branch==release is very confusing in the schema. Instead use the existing Release definition to keep track of which release a layer_version is for. Remove the Branch model and all references to it. Create a migration path to convert from up_branches to their corresponding releases. (Bitbake rev: f8f4cffe6fd371f3a7e63690c68f3fcb5dc1f297) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: lsupdates Add progress information and clean up loggingMichael Wood
Adds basic progress % information and provides better description of what is happening. (Bitbake rev: 6393dbf97d450d2521c0bc9429da0987bb7720ec) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: tests: Remove references to LayerSource modelMichael Wood
Remove and replace layersource model references in the tests and test data. Remove the orm/test as this only tested LayerSource interactions which have now been removed. (Bitbake rev: 61a47cbc92c856690cb0e8da7102b2e669eaee0a) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: Replace references to LayerSource modelsMichael Wood
Replace references to the now deprecated layersource models across Toaster with the new enums for layer source types. (Bitbake rev: 48c09c62eb979d840132e58144f0d81ffee675b1) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: lsupdates Add layerindex fetcherMichael Wood
Move and refactor the layerindex layer source update mechanism so that we don't have to track the layerindex objects in the toaster database. Move this out of the orm and into the management command. Paves the way for future improvement to allow you to specify a layer index server as an argument to the command. (Bitbake rev: f83527edc6d52a34cd73a9c3650ee484407e2e0c) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: toaster: models Remove LayerSource models and replace with enumMichael Wood
We had a lot of complexity around different layer sources to allow for multiple sources and different priorities for these source. This was implemented using rigged abstract classes which represented the different layer sources when in fact just an enum/flag on the layer_version object is sufficient for our needs. Remove the LayerSourcePriority object as this is not needed. We no longer have a problem of multiple layers coming from multiple sources so this is not needed. Two migrations are added to first remove the child models which represented layersources. Then a second migration is needed to remove the LayerSource model it's self as Django can't understand the non-standard base class dependency. Triggering this issue: https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies Clean up a number of flake8 warnings in classes which were modified. [YOCTO #9853] (Bitbake rev: 26624740418de95adb1f4a970a30a5f1149ebb79) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: the customise image button shouldn't rely on targetsElliot Smith
The build dashboard customise image button (for creating a new custom image based on an image recipe used by a build) shouldn't rely on targets: whether a new custom image can be created or not depends on whether any of the recipes used by the build are image recipes. Modify the method used to determine whether a build has customisable images to look at the image recipes used during the build, rather than whether the targets run by the build refer to image recipes. (Bitbake rev: 6648876c91134bda8498b4f8d7ace9147ec0d985) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: add package manifest path to Target objectsElliot Smith
Store the path to the *.rootfs.manifest file for targets which generate images. A link to the package manifest is displayed in the build dashboard for targets which produce image files. Like the license manifest path, if a target would have produced the package manifest (but didn't, because it already existed), that path is copied from the target which did produce the package manifest. (Bitbake rev: 79b8e349a0da2ea6b97ad82daa5837e6dfffe0af) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: only show "New custom image" button for builds with image ↵Elliot Smith
targets Add a has_image_targets() method to Build, and use that to hide the "New custom image" button on the build dashboard if a build has no targets which build images. [YOCTO #9514] (Bitbake rev: 3c4b053e44ea512ef2ced67289a7b0161db6ce9b) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: use has_images() methods to display images correctlyElliot Smith
In the build dashboard, we had issues with showing images correctly, as we were using the is_image property of targets to determine whether a target would have image files. This property can be set to True if a target refers to an image recipe (e.g. "core-image-minimal"), even if the task used in the build didn't produce any image files. By adding has_images() methods to the Target and Build objects, which count associated Target_Image_File objects, we can correctly determine whether a target has image files associated with it, and if any of the targets for a build has image files. This means that we can screen out the left-hand "Images" menu options for builds which contained image-related targets (e.g. "core-image-minimal") but which didn't produce any images (e.g. "rootfs" task). [YOCTO #9500] [YOCTO #9784] (Bitbake rev: f6bba0ff254d5ed3163151d4b938f3a43c9acb0a) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: improve scan for SDK artifactsElliot Smith
SDK artifacts were previously picked up by toaster.bbclass and notified to buildinfohelper (via toasterui). The artifacts were then added to the Build object, so that it wasn't clear which artifact went with which target; we were also unable to attach SDK artifacts to a Build if they had already been attached to a previous build. Now, toaster.bbclass just notifies the TOOLCHAIN_OUTPUTNAME when a populate_sdk* target completes. The scan is moved to buildinfohelper, where we search the SDK deploy directory for files matching TOOLCHAIN_OUTPUTNAME and attach them to targets (not builds). If an SDK file is not produced by a target, we now look for a similar, previously-run target which did produce artifacts. If there is one, we clone the SDK artifacts from that target onto the current one. This all means that we can show SDK artifacts by target, and should always get artifacts associated with a target, regardless of whether it really build them. This requires an additional model, TargetSDKFile, which tracks the size and path of SDK artifact files with respect to Target objects. [YOCTO #8556] (Bitbake rev: 5e650c611605507e1e0d1588cd5eb6535c2d34fc) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: attach kernel artifacts to targetsElliot Smith
The bzImage and modules files were previously attached to a build, rather than to the target which produced them. This meant it was not possible to determine which kernel artifact produced by a build came from which target; which in turn made it difficult to associate existing kernel artifact with targets when those targets didn't produce artifacts (e.g. if the same machine + target combination was built again and didn't produce a bzImage or modules file because those files already existed). By associating kernel artifacts with the target (via a new TargetArtifactFile model), we make it possible to find all the artifacts for a given machine + target combination. Then, in cases where a build is completed but its targets don't produce any artifacts, we can find a previous Target object with the same machine + target and copy its artifacts to the targets for a just-completed build. Note that this doesn't cover SDK artifacts yet, which are still retrieved in toaster.bbclass and show up as "Other artifacts", lumped together for the whole build rather than by target. [YOCTO #8556] (Bitbake rev: 9b151416e428c2565a27d89116439f9a8d578e3d) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-19bitbake: toaster: improve image file suffix retrievalElliot Smith
Refactor retrieval of suffix from image file path, making it a a method on Target_Image_File. This makes it easier to use this in the build dashboard for individual images, plus reduces the complexity of the code required to get all of the image file suffixes for a build. (Bitbake rev: 9c38de3dec74c122c2060cad37331bdafc6858ec) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15bitbake: toaster: fix wrong usage of print_exc and format_excEd Bartosh
First parameter of traceback.print_exc and traceback.format_exc APIs is a 'limit' - a number of stracktraces to print. Passing exception object to print_exc or format_exc is incorrect, but it works in Python 2 and causes printing only one line of traceback. In Python 3 comparison of integer and exception object throws exception: TypeError: unorderable types: int() < <Exception type>() As these APIs are usually used in except block of handling another exception this can cause hard to find and debug bugs. (Bitbake rev: c5a48931ac8db9e56f978c50861c19d0d0c808e3) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15bitbake: toaster: fix migrationsEd Bartosh
(Bitbake rev: bfb44886c3bec8320e4f845c9ff24ae2a795aa9f) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15bitbake: toaster: Rework displaying package dependencies across ToasterMichael Wood
After porting the build table to a unified mechanism for showing dependencies in tables it highlighted that the dependencies selected to be shown were un-filtered. i.e. all dependencies from all contexts were shown. The context for a package's dependencies is based on the target that they were installed onto, or if not installed then a "None" target. Depending on where the template for the dependencies are show we need to switch this target which is why a filter and utility function on the model is added. Additionally to use the same templates in the build analysis we also need to optionally add links to the build data for the packages being displayed as dependencies. Customising a Custom image recipes may or may not have a target depending on whether they have been built or not, if not we do a best effort at getting the dependencies by using the last known target on that package to get the dependency information. [YOCTO #9676] (Bitbake rev: 31e7c26cc31a7c8c78c1464fa01581683bfd2965) Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: toaster: use // operator instead of /Ed Bartosh
Division operator works differently in Python 3. It results in float unlike in Python 2, where it results in int. Explicitly used "floor division" operator instead of 'division' operator. This should make the code to result in integer under both pythons. (Bitbake rev: 0c38441ed99b49dae8ef9613e320f0760853d6aa) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: toaster: replace map with list comprehensionsEd Bartosh
map returns map object in python 3. Replaced calls of map to equivalent list comprehensions in the code which requires lists. [YOCTO #9584] (Bitbake rev: d41eb6d6c061cb35ec1ecd899f856876f9077167) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: toaster: use items and range instead of old APIsEd Bartosh
Used items() and range() APIs instead of iteritems() and xrange() as latter don't exist in python 3 [YOCTO #9584] (Bitbake rev: 372dd3abcb201bd9ac2c3189c5505d3578ce0dd0) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: toaster: use decode('utf-8') for binary dataEd Bartosh
Decoded binary data to utf-8 to fix the following error when runnign on python 3: TypeError: Type str doesn't support the buffer API [YOCTO #9584] (Bitbake rev: 752ea00919ef054622a51ce097923309934eff2b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>