| Age | Commit message (Collapse) | Author |
|---|
|
Fixes "2efb14648 toaster: Monitoring - implement Django logging system" when
running in a container.
When running in a container, the previous approach of using BASE_DIR
is not a writable path. Also, we really do not want to be writing logs into
the source tree, as the BASE_DIR was resolving to bitbake/lib/toaster/logs
Since Toaster is only ever running in an environment where oe-init-buildenv
or similar has been sourced, we should instead write the logs to BUILDDIR.
Using BUILDDIR to logs make path writable but django-log-viewer does'nt manage
to write logs using an absolute path as BUILDDIR, where the existing toaster_ui.log
was already being written.
Also drop the /logs/ directory, as it has not been created which also breaks
in a container environment
To handle the constraints linked to django-log-viewer and /logs/, we've updated
bitbake/bin/toaster to create a toaster_logs/ directory in BUILDDIR if it doesn't exist,
when toaster starts up.
Also manage to set BUILDDIR/toaster_logs/ as default location for toaster logs.
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Tested-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In line with the inclusive language migration defined at:
https://wiki.yoctoproject.org/wiki/Inclusive_language
replace the remaining usage of "abort" in documentation, error
messages, and comments with halt/fail/exit as appropriate.
A couple of external Javascript API calls in Toaster remain, as
they cannot currently be changed.
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In line with the inclusive language migration defined at:
https://wiki.yoctoproject.org/wiki/Inclusive_language
rename:
BB_ENV_WHITELIST -> BB_ENV_PASSTHROUGH
BB_ENV_EXTRAWHITE -> BB_ENV_PASSTHROUGH_ADDITIONS
(Bitbake rev: fe60627839d4280cf0117ed1afbfccdff1181b6a)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Directly support the various 'manage' commands from the Toaster
executable, so that users do not have to manually set up the required
environment and paths.
Examples:
$ . toaster manage createsuperuser
$ . toaster manage lsupdates
[YOCTO #13170]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Updates for YOCTO #12891 allowed a user to have a directory
structure different to that of yocto (bitbake isn't inside
oe-core) whereas the definition of OE_ROOT in the main toaster
binary still assumes the same while checking for .templateconf
and hence we see an error on the cmdline in such cases:
bash: <repo-path>/bitbake/bin/../../.templateconf: No such file or directory
The change here now allows the user to provide OE_ROOT through
the environment in such cases and otherwise defaults to the older
mechanism to fix this issue.
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
TOASTER_DIR is used for higher level toaster artifacts
such the SQL DB and creating toaster internal build
directories for projects. Prior to this change it was
evaluated as `dirname $BUILDDIR` and user had no control
over it. This change allows to override this variable
from the command line for more flexibility. The variable
defaults to its original setting if the optional argument
is not passed.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix typo in shutdown code to kill threads when "kill -0" is not enough.
Use the '--noreload' flag for 'runserver' so that there are no extra
and unaccounted threads.
[YOCTO #12555]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a 'nobuild' option for starting Toaster without the project
and hosted builds support. This allows a Toaster host to provide
local build statistics without opening the host to external users
building projects.
[YOCTO #12315]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster needs the ability to allow custom extensions to execute when
Toaster is started and stopped. Toaster will look for a custom
extension script in the fixtures directory and execute any applicable
hooks.
[YOCTO #11938]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are four main API deprecations in Django-1.10:
(a) String view arguments to url() must be replaced by
the explicit class reference
(b) New TEMPLATES stucture in settings.py consolidates
TEMPLATE_DIRS, TEMPLATE_CONTEXT_PROCESSORS,
TEMPLATE_LOADERS, TEMPLATE_STRING_IF_INVALID, and
TEMPLATE_DEBUG
(c) patterns() wrapper in url() is removed, with
urlpatterns now a simple list
(d) NoArgsCommand in commands() must be replace by
BaseCommand, and handle_noargs() changed to
handle()
Also, the Django version checker must be updated to accept
two digit sub-version numbers (e.g. "1.8" < "1.10")
[YOCTO #11684]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the 'noweb' option for Toaster is used, perform the database
check/create if the Toaster database does not yet exist.
This will allow Toaster to not fail if the first use is with 'noweb'.
This avoids potentially clashing database updates if there are
multiple overlaping 'noweb' sessions (for example with a CI system).
If the user wished to update the database, they can either use the
explicit "lsupdate" command or (re)start a web hosted Toaster session
(which is gated by the webserver's PID).
[YOCTO #11378]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a superfluous section in toaster start/stop script that was
setting the TOASTER_DIR and had a set of comments around that setting.
This was done in two places and only the last one was effective. This
patch removes the spurious section to make it clearer what was
happening and what TOASTER_DIR was actually getting set to.
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a superfluous section in toaster start/stop script that was
setting the TOASTER_DIR and had a set of comments around that setting.
This was done in two places and only the last one was effective. This
patch removes the spurious section to make it clearer what was
happening and what TOASTER_DIR was actually getting set to.
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some of the tinfoil2 changes resulted in TOASTER_DIR being cleared by
the memory resident bitbake server toaster starts up. This prevented
toaster from being able to connect to its sqlite database. Adding
TOASTER_DIR to the BB_ENV_EXTRAWHITE list stops the cooker from clearing
out the TOASTER_DIR variable.
[YOCTO #9252]
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that we use python3, we should refer to pip3, not pip.
[YOCTO #10774]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Write the pid file out in the start up of this management command. This
ensures this has happened instead of relying on the shell command having
been run which may or may not be the case. This also makes it simpler for
testing.
Couple of clean ups of runbuilds as identified by pyflake
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Setting OEROOT in toaster script makes oe-init-build-env to
break with error:
bash: ../bitbake/bin/../../scripts/oe-buildenv-internal: No such file or directory
This happens because OEROOT contains path relative to build
directory.
Renamed OEROOT to OE_ROOT and unset it after it's used.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster script kills runserver process 2 ways:
- sending signal to pid from .toastermain.pid.
- sending signal to pids found by grepping ps output:
ps fux | grep "python.*manage.py runserver"
Second approach is redundant and harmfull as it kills all django
development server running on the machine.
[YOCTO #7973]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This file is not created anywhere, but handled in toaster
script code.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is no point of trying to kill django development server
when toaster starts because 'manage.py checksocket' command is already
used in the script code to check if development server port is occupied.
Even if Toaster is listening on another port, killing previous instance
looks quite implicit and doesn't solve anything as there are other
processes that might be still running.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This has been replaced using django's inbuilt loaddata.
Django command documented at:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#django-admin-loaddata
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Call django's inbuilt loaddata command to load the appropriate fixtures.
We also attempt to load a fixture called "custom" and fail silently if
we don't have one. This is where initial customisations can be done to
load particular settings or data into Toaster (for example layers or
default values for variables)
Make sure the value for TEMPLATECONF is available to checksettings so
that we can have a go a working out which default data to load.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
If the user explicitly passes in "--help" then it should return 0. This
is the convention follow by the typical application. This allows the
user to check for options without triggering an error.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Explicitly use python3 so that the modules for python3 are checked.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
|
|
Explicitly used python3 as default python for oe builds
will continue to be python2.
[YOCTO #9584]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We don't need to pass the DATABASE_URL around and read it back if we
setup the django framework in the correct way.
We make the default sqlite database path a full path so that the
database isn't being assumed to be in CWD.
Also add some more useful comments on the database settings.
This is preparation work to migrate the build tests and be able to
trigger builds on differently configured databases.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Used print() function instead of print statement
to make toaster script to work with both python 2 and python 3
[YOCTO #9584]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For the noweb command we don't need to check if a socket is in use or
not as we're not starting the django development server. We're just
setting up the environment and running the runbuilds scheduler.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Allow binding to a specified address and port to start the django
development server on.
Remove the assumption that you want to bind to 0.0.0.0 and set the
default to localhost.
Add some additional help text after Toaster webserver has started
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add help text for unrecognised or missing command for toaster script
Remove assumption that no command is 'start' as the default.
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This variable will be used in localhost controller code
to determine correct bitbake location, so it has to be
exported.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Added 'INHERIT+="toaster buildhistory"' line to the conf/local conf
when Toaster starts. It should make commandline builds to provide
all required information to Toaster backend.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Set BITBAKE_UI variable to 'toastergui' for command line builds
to use toasterui as a default ui module for bitbake.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
noui command line option doesn't makes sense anymore as toaster doesn't
run bitbake. It should be safe to to remove it.
The purpose of this option was to skip running bitbake observer process.
This was never used before as it's not possible to run toaster build
without running observer.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
>From now on toaster script will not run bitbake server.
It will be started by runbuilds and stopped after the build.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This cleanup fixes a few issues:
1) Removes the superflous code to set toasterconf.json
- this isn't used and referenced meta-yocto
2) Changes exit to return so we don't surprise the user by exiting their shell
- this is necessary because it is being sourced
3) Removes the last references to the old TOASTER_MANAGED variable
- this is historical and no longer used.
4) Adds -t parameter to lsof
- This stops it from dying on odd filesystems and is much
faster since all we are using are processes anyway
5) Handles start and stop as params
- it was easy to confuse the script especially
if we were calling it with parameters.
- if start/stop isn't specified, it will still toggle
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster script run bitbake with --postread conf/toaster.conf.
It turned out that variables set this way don't influence the
build. Changing --postread to --read makes bitbake to read
variables earlier and should fix this issue.
[YOCTO #8781]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Sometimes Toaster connection to the bitbake server fails with
this error in the log:
ERROR: Could not connect to server 0.0.0.0:46572
: Could not register UI event handler
It happens more often on slow machines.
Increasing timeout after restarting bitbake should fix this issue.
[YOCTO #8776]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Removed nobrowser and brbe script parameters as both
are confusing and nobrowser is not used anywhere.
brbe parameter usage can only be justified if toaster
doesn't work properly and user has to manually connect
toaster to running bitbake server. Even in this scenario
it's very unlikely to achieve as toaster script is not
designed for this kind of usage.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Set default values of script parameters just before
they are parsed to increase readability.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Moved addr:port, bitbake.lock and toastermain.pid checks
to the place where the rest of Toaster starting happens.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Removed check_pidbyfile and notify_chldexit functions from
toaster script as they're not called in the script.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This function is useless as it's called just once
and makes code less readable.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Updated GPL information, years of development and
usage information. Removed outdated information about
2 ways of starting Toaster.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Used MANAGE variable to avoid repeating path to
manage.py in many places in toaster script.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Removed unused variable RUNNING from the toaster script.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Made toaster script more readable by splitting long lines and
removing and adding whitespace.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Used new management command checksocket to check if
Toaster can listen on address:port.
[YOCTO #8775]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Alassane Yattara
Scott Murray
Reyna, David
Richard Purdie
Awais Belal
brian avery
Michael Wood
Ed Bartosh
Randy Witt
brian avery