| Age | Commit message (Collapse) | Author |
|---|
|
It looks like we're about to see a lot of changes in branch names in repos. If
we have the prune option here, those old names are lost, the changes propagate
to our source mirrors and our old releases break.
We have the force option so any replaced references should be replaced, its only
orphaned branches which will now be preserved.
I believe this behaviour will cause us fewer problems given the changes that
look likely to happen.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The bitbake-worker child on the SIGTERM signal handling send the SIGTERM to all
processes in it's process group. In cases when the bitbake-worker child got
SIGTERM after registering own SIGTERM handler and before the os.setsid() call
it can send SIGTERM to unwanted processes.
In the worst case during SIGTERM processing the bitbake-worker child can be in
the group of the process that started BitBake itself. As a result it can kill
processes that not related to BitBake at all.
Signed-off-by: Ivan Efimov <i.efimov@inango-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring
self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0])
AssertionError: '' is not true : Could not find upstream version for db
[YOCTO #13496]
The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget.
Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the
recipe. Also change the version being checked.
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix the warning:
WARNING: lib/bb/data_smart.py:235: DeprecationWarning: generator 'COWDictMeta.iter' raised StopIteration
for k, v in self.variables.iteritems():
by using return from the generator, not raising StopIteration.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 407d6e07b09123c12c382b4a92107f002c314b05)
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a callback that lets you modify or remove items in addition to the
current scheme where you can only add or remove. This enables you to for
example replace a layer with a temporary copy (which is what we will use
this for first in OE's oe-selftest).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the bitbake server recieved multiple connections, it currently closes
ones it can't handle (while its dealing with another). This is rather
antisocial behaviour which causes clients to quickly run through their
retries and abort.
Instead, queue any other connections until the current one is closed. This
way the client can decide when it wants to stop waiting for the server. If the
client is gone by the time we handle it, we handle that gracefully.
This also fixes a number of bugs in the connection handling where connections
which did drop early were badly handled causing tracebacks in the logs.
Also, handle queue incomming connections in a loop to ensure that the main
client handling doesn't starve that piece of the system.
This code was stress tested by running 50 connection attempts in parallel at
once, ensuring the code correctly handled them.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that we set a timeout for the socket, it can return EWOULDBLOCK
if a signal or other event happens to wake up even if we don't timeout.
If this happens, retry the connection, else we simply see it quickly
loop through the retries and abort the connection in a very short
interval.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current value of 2 seconds has shown to be short in
wider testing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing hangs in oe-selftest where server startup and shutdown are
racing. The assumption was a connect would timeout however no timeout is
set which can leave processes hanging. Set a short timeout for
the connection to avoid this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing slow startup in bitbake, add some timeing debug messages so
the logs are more useful for debugging when its slow.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On loaded production systems we've seen bitbake server take over
40s to start up. Increase the timeout to 90s which tries to avoid
failures in selftests.
The delays come from setting up the inotify watches (31s) so can't
really be avoided.
After 5s delay we now warn the user we're waiting for 90s so the
interactive exeperience shouldn't be much changed and its very
unlikely the user would see that anyway.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Follow dependent hash changes recursively also when specifying two
signature files explicitly. Previously this was only done when using the
--task option.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The functionalities of bitbake-diffsigs and bitbake-dumpsig are so
similar that they can be merged into one. Add an option --dump to make
bitbake-diffsigs dump the last signature data instead of comparing it.
Keep bitbake-dumpsig as a symbolic link to bitbake-diffsigs. When it is
called as bitbake-dumpsig, it behaves as if --dump was specified.
Also make -D the short option for --debug again (the way it used to be,
and still was for bitbake-dumpsig), so that -d can be used as the short
option for --dump.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing issues where the server doesn't start with no logs as to why. Allow
the server to print the last 60 log lines just in case this shows us something useful
about what is failing.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
10 log lines may not capture any full traceback, increase the number of
lines to 60 which covers most tracebacks.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #12898]
There might be no bitbake-cookerdaemon.log, print a message for debugging.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This simplifies the code and makes it easier to read but has the
same functionality.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Unseting -> Unsetting
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid failures like:
2018-12-04 03:30:56,033 - oe-selftest - INFO - ======================================================================
2018-12-04 03:30:56,033 - oe-selftest - INFO - FAIL: pkgdata.OePkgdataUtilTests.test_find_path (subunit.RemotedTestCase)
2018-12-04 03:30:56,033 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-12-04 03:30:56,033 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
[...]
AssertionError: 'Previous bitbake instance shutting down?,[77 chars]xist' != 'ERROR: Unable to find any package produci[14 chars]xist'
- Previous bitbake instance shutting down?, waiting to retry...
ERROR: Unable to find any package producing path /not/exist
We need to use the logger so output is correctly handled in such cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Small tweak to ensure these items are printed into the log since there
is other logging code which looks for the header and this makes it clearer
the server did start but is slow somewhere in startup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if there are no setscene tasks, the disk monitor isn't started.
Move the startup code to somewhere to ensure it always is started. This
issue would partially explain occasional selftest failures.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Improve connetion refused error handling:
NOTE: Retrying server connection...
NOTE: Reconnecting to bitbake server...
NOTE: Retrying server connection... (Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 471, in connectProcessServer
sock.connect(os.path.basename(sockname))
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/server/process.py", line 502, in connectProcessServer
os.close(i)
TypeError: an integer is required (got type NoneType)
)
WARNING: /home/pokybuild/yocto-worker/oe-selftest-opensuse/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
logger.info("Retrying server connection... (%s)" % traceback.format_exc())
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 490, in connectProcessServer
if command_chan_recv:
UnboundLocalError: local variable 'command_chan_recv' referenced before assignment
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Extend the server error handling to avoid:
Reconnecting to bitbake server...
NOTE: Retrying server connection... (Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 457, in connectProcessServer
sock.connect(os.path.basename(sockname))
ConnectionRefusedError: [Errno 111] Connection refused
)
WARNING: /home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py:481: ResourceWarning: unclosed <socket.socket fd=20, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
logger.info("Retrying server connection... (%s)" % traceback.format_exc())
NOTE: Starting bitbake server...
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Updates for YOCTO #12891 allowed a user to have a directory
structure different to that of yocto (bitbake isn't inside
oe-core) whereas the definition of OE_ROOT in the main toaster
binary still assumes the same while checking for .templateconf
and hence we see an error on the cmdline in such cases:
bash: <repo-path>/bitbake/bin/../../.templateconf: No such file or directory
The change here now allows the user to provide OE_ROOT through
the environment in such cases and otherwise defaults to the older
mechanism to fix this issue.
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before this fix it is assumed that the removal of the
remote can only fail because there is not remote to remove. This
is a false assumption. Example error which would be ignored:
git -c core.fsyncobjectfiles=0 remote rm origin failed with exit code 1, output:
Note: A branch outside the refs/remotes/ hierarchy was not removed;
to delete it, use:
git branch -d master
error: could not lock config file config
error: Could not remove config section 'remote.origin'
Due to the masking of this error a stranger error will be
presented to the user, because this time we do not mask the
exception:
git -c core.fsyncobjectfiles=0 remote add --mirror=fetch origin https://github.com/ptsneves/tl-wn722.git failed with exit code 128, output:
fatal: remote origin already exists.
The most likely reason that the remote cannot be removed nor
modified is that the DL_DIR/git2 does not have permissions
compatible with the user running bitbake.
This commit fixes:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12728
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The environment was being cleared before the UI imports occurred which
caused problems for graphical UIs like taskexp. The full environment was
intended to be available to UI clients and it was only meant to be cleared
for the server/cooker, so tweak the code order so this is the case.
This fixes problems reported for taskexp.
[YOCTO #12670]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Commit 15340edce23e63b060c75114d508e1f76757239c introduced a way which
allowed bitbake to be found from the PATH variable when directory
structures different from poky are used. This just drops a leftover
bitbake definition which made the earlier implementation meaningless
by reassigning the path relative to oe-core/meta.
[YOCTO #12942]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use == instead of = when comparing task outcome to OUTCOME_FAILED.
Prior to this fix the recipe template would cause a TemplateSyntaxError
exception.
(Bitbake rev: a53ffec4ed3d0f9221bca398e20e8f480fb2b325)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The local layer info (provided through custom fixtures) should
not be hidden. It is better to handle it in the same manner
as an imported layer, otherwise the layer path and dependency
info is not shown. The layer editing fields are handled in the
html side of things appropriately so this does not harm that
implementation.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In a case where the layer source is local only and the recipe
is not yet built, we can search for the path with layer's
local_source_dir, and if available that should be used rather
than just skipping the scenario.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This change allows the CUSTOM_XML_ONLY toaster setting to be
provided through the environment so the user can do this without
mingling with the settings.xml, for scenarios where modifying
settings.xml is not achievable.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The release drop down divs are not being closed
appropriately, which showed adverse reactions on
the UI that aligned the "Create project" button
with the left edge of the screen without any
margins. This fixes these divs which in turn
aligns the button appropriately.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There can be cases where the variables being used
to divide in build percentage expressions can be
zero. For example, a setup consisting of only local
repos will have repos_to_clone=0 and will generate
a divide by zero scenario.
Fix this by checking the divisor in such cases.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
TOASTER_DIR is used for higher level toaster artifacts
such the SQL DB and creating toaster internal build
directories for projects. Prior to this change it was
evaluated as `dirname $BUILDDIR` and user had no control
over it. This change allows to override this variable
from the command line for more flexibility. The variable
defaults to its original setting if the optional argument
is not passed.
[YOCTO #12891]
(Bitbake rev: e073775d3b6980fc8004ae28a3ccc3c5bbf50fb2)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current mechanism for finding the bitbake binary
assumes a directory structure which is identical to
poky, where oe-core's meta and bitbake directories are
at the same level. There can be a case where bitbake
is used from elsewhere and in such cases the above
mentioned assumption fails to hold, whereas this is
totally allowed by the oe-init-build-env script which
can take bitbakedir as an argument.
The better approach is to allow bitbake to be derived
from PATH, while keeping the older mechanism in place so
it can be removed after tests are done in various
environments. This makes more sense as toaster has
also been launched from the same bitbake instance
that is the one in PATH.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Toaster depends on pokydirname for identifying the location of
the oe-init-build-env script (and there might be other purposes
in the future). The problem with current approach is that it
only checks/sets the variable with git based repos, whereas
toaster provides mechanisms to allow having layers that are all
locally available. The evaluation of the variable fails in such
scenarios, so use a more flexible mechanism in this case and
try to locate poky in the local layers as well, if not already
set.
[YOCTO #12891]
Signed-off-by: Awais Belal <awais.belal@mentor.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In case something goes tragically wrong, catch a request to checksum / and
refuse.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Opening a file in binary mode and iterating it seems like the simple solution
but will still break on newlines, which for binary files isn't really useful as
the size of the chunks could be huge or tiny.
Instead, let's be a bit more clever: we'll be MD5ing lots of files, but we don't
want to fill up memory: use mmap() to open the file and read the file in 8k
blocks.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running bitbake command with Python 3.6.5 always result in
import error causing by the change of distutils module.
This patch replaces the method to search executable in PATH by
"/usr/bin/env <command>".
Signed-off-by: Tzu Hsiang Lin <t9360341@ntut.org.tw>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add Sumo (YP-2.5) to the release selection for new projects.
[YOCTO #12713]
Signed-off-by: David Reyna <david.reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Prevent movefile from falsely setting the source file's owner and
permissions on the destination directory instead of the destination
file when moving between devices.
This bug caused the last file moved into a directory to dictate the
directory's owner and permissions.
Signed-off-by: Mattias Hansson <mattias.hansson@axis.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes [#YOCTO 12030]
Updated the "Task Checksums and Setscene" section to provide a bit
of user information around the bitbake-dumpsigs use that lets a
user examine signatures and inputs that determine if a do_compile
task is indeed supposed to be run.
Added more explanation of how a user can examine signatures used
to determine if a do_compile task is indeed supposed to be run.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Made sure that the terms "OpenEmbedded-Core" and "OE-Core"
are used as such throughout the manual.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Ivan Efimov
Armin Kuster
Paul Eggleton
Peter Kjellerstedt
Robert Yang
Ross Burton
Scott Rifenbark
Yong, Jonathan
Awais Belal
Paulo Neves
Karsten Strand
Tzu Hsiang Lin
David Reyna
Mattias Hansson