| Age | Commit message (Collapse) | Author |
|---|
|
Fixed:
Assume there is no master branch in hello layer:
$ update.py -l hello -b master
INFO: Skipping update of layer hello - branch master doesn't exist
This is correct since hello layer doesn't have master branch, but when --nocheckout:
$ update.py -l hello -b master --nocheckout
[snip]
INFO: Sorting layers for branch mater:
WARNING: Cannot find required collections on branch master:
WARNING: hello: LAYERDEPENDS: <snip>
This is incorrect, this patch fixed the problem, now it skips it since the
branch doesn't exists when --nocheckout.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
When layer_a RECOMMENDS layer_b, try to add layer_b before layer_a, but if
layer_b is not found, still add layer_a.
And print summary error mesage:
$ update.py -b master
ERROR: Issues found on branch master:
openembedded-core: Added without LAYERRECOMMENDS
meta-secure-env: Failed to add since LAYERDEPENDS is not satisfied
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
Fixed:
$ git clone <url>
warning: remote HEAD refers to nonexistent ref, unable to checkout.
$ git rev-parse HEAD
HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Catch the error and avoid that.
And use "git reset --hard" to replace of "git reset --hard HEAD", HEAD is
default for git reset, so they are the same, but the later one reports error
when remote HEAD doesn't exist:
$ git reset --hard HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
[snip]
$ git reset --hard
No errors.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
Fixed:
$ update.py -b <new_branch>
[snip]
NOTE: Starting bitbake server...
Traceback (most recent call last):
File "update_layer.py", line 471, in main
utils.setup_core_layer_sys_path(settings, branch.name)
File "/buildarea1/lyang1/layerindex-web/layerindex/utils.py", line 376, in setup_core_layer_sys_path
core_layerdir = os.path.join(core_repodir, core_layerbranch.vcs_subdir)
AttributeError: 'NoneType' object has no attribute 'vcs_subdir'
[snip]
This is because core_layerbranch is not in database yet for completely new
branch, so it is None and we will get the error. Avoid calling
setup_core_layer_sys_path() when "update_layer.py --initial" will fix the
problem.
And also only add core layer's sys path when it is present, since core layer
may not be added yet for completely new branch.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
|
|
Add the ability to export the recipe listing for a layer to a CSV file
for importing into external tools. At the moment we include name,
version and license, but there is a parameter that lets you specify the
fields to include in the URL if desired.
Implements [YOCTO #12722].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
We're about to replace this with a proper CSV export function, so we
don't need this dead code hanging around anymore.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
It's a bit of a pain to have to set the two python environment fields on
every record in order to have things set correctly, and it can easily
get forgotten, so try to set them automatically by default (assuming
reasonable naming).
Note that this does introduce an annoying behaviour whereby if you click
"Add another Maintenance plan layer branch" and then decide you don't
want it, the admin form will insist you fill in the fields unless you
clear out the python2/3 environment fields. I'm not sure how to fix
that, so I'm leaving it as-is for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Collect information about patches applied by a recipe, and record each
patch along with the upstream status, presenting them in the recipe
detail.
Implements [YOCTO #7909].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Checking out a revision in the bitbake/layer repositories is something
we are doing in a few places, so add a checkout_repo() function that
does this, ensuring that we don't get tripped up by any junk files,
and avoids checking out if the repository is already at the desired
revision (thus avoiding the clean operation and e.g. preserving any
.pyc files that aren't stale and would speed things up a little). Note
that we do the clean before checking out in case there are untracked
files that are tracked in the commit we are checking out.
In addition to adding this function, change the existing code that we
use in the update script to check out a layer use the new function.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
It would be useful in some scenarios to get the complete list of
recursive dependencies for a layer, so add a function to do that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The Recipe Reporting System needs to be able to provide links to commits
in the web interface for the repository, but we can only do this if we
have a custom template URL just like we do for file/tree links, since
it's different for different git web interfaces. Add support in all the
various places for such a URL and make use of it in the RRS.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Save each remote SRC_URI so we can use these for the recipe reporting
system. This replaces an earlier implementation in the rrs branch where
we simply stored SRC_URI verbatim.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Every time we add something that links to Recipe we had to add it to the
exclusions list in the readonly_fields line for RecipeAdmin (and
ClassicRecipeAdmin), which is tedious and easily forgotten. We can avoid
this by looking at each field and excluding it by its attributes rather
than having a hardcoded list of names.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If the RRS is enabled, then add a link to it in the tools menu. I don't
expect this to be used a lot, but otherwise the only way you'd get there
would be either externally or via the link from the layer detail.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Make this function more easily reusable from the RRS code.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The old RRS branch had its own addition of dependency support, but in the
mean time we added that to the layer index in the master branch using a
different structure. Adapt the RRS recipe detail page to that structure.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If the RRS is enabled, then add a way to get from the layer detail page
to any maintenance plans in which the layer is included.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Sometimes we get back UTF-8 characters (particularly when picking up
names from git commits), and the ascii codec will error out if that
happens, so switch over to utf-8.
We can as a result remove the decode() calls from the bulk change view.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Give some space to the tabs,
Add some top margin to the nav-pills class so we
create some breathing space between the milestone
overview and the tabs.
Change 'No Maintainer' to 'No maintainer'
Just to keep the capitalisation style consistent
across the interface.
Apply the muted class to all not-sortable table headings
The class was missing from the Recipe and Maintainer
columns in the recipes table; and from the Not updated
and % done column in the maintainers table.
Remove the strong tag from the recipe status information
This is just to match the presentation of the milestone
overview information in the base_toplevel.html template.
Separate the footer from the bottom of the viewport,
It's hard to see the footer on click on its links when
they are so close to the bottom of the veiwport, so
add some margin at the bottom of the footer <div>.
Change the label of the recipes tab,
From 'Recipes status' to 'Recipes upstream status' to
match the label of the 'Upstream status' filter in the
recipes table.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
|
|
In e902b67bccddf34d8bbd65ad6c81d078e945ebb8 I missed a couple of Context
usages in the layer publish view and the result was that it broke
publishing a layer (and apparently I didn't run a final test on that,
shame on me).
Thanks to Yi Zhao <yi.zhao@windriver.com> for pointing this out.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
There were several issues with this code, including that it used
SortedDict which was removed in Django 1.9 and that it seemed not to
have been fully updated to accommodate changes in bitbake's recipe
parsing API. In the end I decided the simplest thing would be to move it
over to using oe.recipeutils.patch_recipe() which is actually a now much
more mature version of the code that originally started life here. With
that we can get the bulk change functionality working again and gain
some of the improvements in behaviour that we've developed in
oe.recipeutils.patch_recipe(), as well as avoiding effectively
duplicated code.
Implements [YOCTO #9730].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
It seems that in df492b127723973e5e2c18c74b1fee853d41bc69 the original
intention was to move setup_layer to the utils module, but that didn't
actually get done in the final patch - however the change was made here
to accommodate the move, meaning it's been broken since then. Revert
that so we're actually calling the function in the place it exists.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add a function that lets us import OE python modules easily.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
With Django 1.10+, if you use get_template() to retrieve a template,
then you can't pass a context when calling .render() on it, you need to
pass a dict instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Django 1.9+ requires an exception parameter for the page_not_found view.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
We use django-reversion to track history of admin/maintainer changes to
the site, and part of our extension on top of django-reversion involves
annotating each "revision" with a description of the changes that were
made. In django-reversion 2.0.0+ the pre_revision_commit signal that we
were using to do this annotation is gone in favour of just using
Django's standard pre-save signal. This was a little challenging to
adapt to for our purposes but not impossible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
resolve_variable isn't available with Django 1.10+, and it's not even
used here, so just drop it from the import line.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The patterns() function is deprecated in Django 1.8 and gone in 1.10, so
we should switch over to the new list format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Rename publish() to publish_view() and call it directly from the view
rather than using a string name (which is not possible in Django 1.10+).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
This is deprecated in Django 1.8, removed in later Django releases.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
This change happened sometime in Django 1.7 and we missed it until now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The utils.setup_django() costs a lot of time, but both update.py and
update_layer.py call it, so move layer validation from update_layer.py
to update.py to avoid calling update_layer.py when possible can save a
lot of time.
Now we don't have to call update_layer.py in the following cases:
* The branch doesn't exist
* The layer is already update to date on specified branch (when no
reload)
* The layer dir or conf/layer.layer doesn't exist
We can save up to 98% time in my testing:
$ update.py -b master --nofetch [--fullreload]
Before Now Reduced
No update: 276s 3.6s 98%
Partial update: 312s 87s 72%
Full reload: 1016s 980s 3%
Note:
* All of the testing are based on --nofetch
* "No update" means all layers on the branch is up-to-date, for
example, when we run it twice, there is no update in the second run,
so we only need about 3s now, which is the most common case when we
use cron to run it per half an hour.
* "Partial update" means part of the layers have been updated.
* "Full reload" means all of the layers have been updated.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
This makes it easy to see which layers failed. For example:
ERROR: Failed layers on branch master: openembedded-core meta-python
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
We have an update.py running periodically in background, but we also
need to run it manually, for example, run it to update actual_branch,
the manually run usually failed because can't get lockfile, we have to
run it again and again. A timeout option helps a lot in such a case. Now
the following command can make sure we can run the command successfully:
$ update.py -b master -a actual_branch -t 2000
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The previous commit broke the layer order, e.g.:
A -> B -> C -> D
The algorithm is checking the dependencies one by one, and until we find D, add
D to layerquery_sorted, and add it "collections", the one in "collections"
means it's dependencies are OK, then C, B and A will check against collections,
so that update_layer.py will update them one by one. The previous commit added
A/B/C/D to collections directly, so that when check against it, all the
dependencies are met, thus broke the layer sorting, and then there would be
failures if we pass layer A to update_layer.py before B, C and D (suppose they
are newly to database). This commit fix the problem.
BTW., why I use collections to record the one whose dependencies are matched,
but not directly use layerquery_sorted, it is because collections contains both
the ones to be added/updated and the ones in database, but layerquery_sorted
only contains the ones to be updated/added.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Default site name should be Layer Index, not metadata index as it was
before.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add a page with basic statistics for the index - number of layers,
recipes, classes, machines and distros on an overall basis (distinct
names) and per branch, since I've been asked a few times for this kind
of information. It's currently only linked from the Tools menu for
logged-in users, but the URL will work for anyone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If the user hit Ctrl+C during the initial info gathering then it didn't
break out of the loop in update.py, so you had to hit Ctrl+C for as many
layers as were involved in the update. Look for exit code 254 from
update_layer.py and stop if it is returned since that indicates Ctrl+C
has been used.
Additionally, ensure we return exit code 254 and print a message from
the main update script when it is interrupted in this way.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
For reasons that are not immediately clear to me, if you don't specify a
name when adding the ViewSet it takes one from the model used in the
queryset in the ViewSet. The new layers view uses LayerBranch and so it
silently replaced the layerBranches URL in the router, even though the
URL itself was still present. Unfortunately that's broken Toaster.
Specify a name to restore the old URL.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
It was a bit awkward to query layers externally - you had to do multiple
queries and you couldn't get the YP Compatible version info at all. Add
an additional LayerBranch-based view that exposes the branch name,
layer fields, YP Compatible Version and active maintainer information
with just one call.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Django REST Framework now requires a field specification for every
ModelSerializer, so specify '__all__' to retain the current behaviour.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Layer maintainer and layer note information wasn't available through the
REST API since it wasn't needed for Toaster, but for other uses it is
useful.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The entire API is meant to be read-only, so we should be using this
class instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If you query on a boolean field you can use the string "False" to match
False in the database; however if you try the same with __isnull then
the query will match every record which is obviously undesirable. If
__isnull is being used, then convert the value to a boolean so that the
query works properly.
An example of this type of query:
http://127.0.0.1:8000/layerindex/api/layerBranches/?filter=yp_compatible_version__isnull:false
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
tinfoil is not needed in cases like the layer is already up-to-date or the
layer is invalid, so only init it when needed.
This can save about 1min when run "update.py -b <branch>" (124 layers).
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
This can save a lot of time, here is my testing data when PARALLEL_JOBS is 10,
this is the fetch time only, I hacked it to stop when the fetch is done to get
the data (124 layers):
$ update.py -b <branch>
Before: 2m30
Now: 16s
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Fixed:
$ ./update.py -l foo -b new_branch
INFO: Fetching remote repository foo
DEBUG: run cmd 'git fetch -p' in 'foo'
[snip]
DEBUG: run cmd 'git checkout origin/new_branch' in oe-core
ERROR: error: pathspec 'origin/new_branch' did not match any file(s) known to git.
The "new_branch" is newly created, it doesn't exist in local repo since it
isn't fetched, it only fetches the layer specified by -l, so only the foo layer
is fetched. This patch fixes problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The obsolete dependency is the one which is in database but not in
conf/layer.conf anymore. The old code had a problem for newly created
layerbranch, the new layerbranch has no dependencies, so no need remove. And it
had a side effect was that when need_remove was cleaned up, it would be set
again in the next for loop, thus might wrongly remove dependencies. This patch
can fix the problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
If your classic recipe search returned a single result, then since
commit c8c25fb641c500354cf36c3c59abb6f9fe96d223 you got a standard
recipe page instead of the proper page with fields you can edit. To fix
it, just drop the redirection functionality from the classic recipe
search, since we don't really want it here.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
We were using the layerbranch id to search for the specified layer,
which is most likely to return either no results or results for the
wrong layer. We can also avoid specifying the id field at all here as
the filter() function can handle real objects.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
Robert Yang
Paul Eggleton
Belen Barros Pena