| Age | Commit message (Collapse) | Author |
|---|
|
As you know, layer order in BBLAYERS can affect in parsing recipes process,
in some cases, users want to add a layer on the top of BBLAYERS variable
So, add "--prepend" option for bitbake-layers to support add-layer to prepend
Signed-off-by: Hieu Van Nguyen <hieu2.nguyen@lge.com>
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
ERROR: npmsw-test-1.0-r0 do_fetch: Error executing a python function in exec_func_python() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:base_do_fetch(d)
0003:
File: '/OE/build/oe-core/openembedded-core/meta/classes-global/base.bbclass', lineno: 150, function: base_do_fetch
0146: return
0147:
0148: try:
0149: fetcher = bb.fetch2.Fetch(src_uri, d)
*** 0150: fetcher.download()
0151: except bb.fetch2.BBFetchException as e:
0152: bb.fatal("Bitbake Fetcher Error: " + repr(e))
0153:}
0154:
File: '/OE/build/oe-core/bitbake/lib/bb/fetch2/__init__.py', lineno: 1835, function: download
1831: lf = bb.utils.lockfile(ud.lockfile)
1832:
1833: try:
1834: self.d.setVar("BB_NO_NETWORK", network)
*** 1835: if m.verify_donestamp(ud, self.d) and not m.need_update(ud, self.d):
1836: done = True
1837: elif m.try_premirror(ud, self.d):
1838: logger.debug("Trying PREMIRRORS")
1839: mirrors = mirror_from_string(self.d.getVar('PREMIRRORS'))
File: '/OE/build/oe-core/bitbake/lib/bb/fetch2/npmsw.py', lineno: 255, function: verify_donestamp
0251: def verify_donestamp(self, ud, d):
0252: """Verify the donestamp file"""
0253: def _handle(m, ud, d):
0254: return m.verify_donestamp(ud, d)
*** 0255: return all(self._foreach_proxy_method(ud, _handle))
0256:
0257: def update_donestamp(self, ud, d):
0258: """Update the donestamp file"""
0259: def _handle(m, ud, d):
File: '/OE/build/oe-core/bitbake/lib/bb/fetch2/npmsw.py', lineno: 242, function: _foreach_proxy_method
0238: def _foreach_proxy_method(ud, handle):
0239: returns = []
0240: #Check if there are dependencies before try to fetch them
0241: if len(ud.deps) > 0:
*** 0242: for proxy_url in ud.proxy.urls:
0243: proxy_ud = ud.proxy.ud[proxy_url]
0244: proxy_d = ud.proxy.d
0245: proxy_ud.setup_localpath(proxy_d)
0246: lf = lockfile(proxy_ud.lockfile)
Exception: AttributeError: 'FetchData' object has no attribute 'proxy'
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
local link sources
* https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json
shows 3 examples:
http tarball sources: This is the URL of the tarball. (eg, https://example.com/example-1.3.0.tgz)
local tarball sources: This is the file URL of the tarball. (eg file:///opt/storage/example-1.3.0.tgz)
local link sources: This is the file URL of the link. (eg file:libs/our-module)
it's not clear if the last one should start with file://, but in:
https://git.openembedded.org/bitbake/commit/lib/bb/fetch2/npmsw.py?id=4f983dc419a1a6f635a5d333f253d49244cec374
we strip 5 chars in:
localpath = version[5:]
so file:// wouldn't work in such case.
It works well when parsing the dependency in:
"@npm-local-link-sources/upper": "file:../"
but then when parsing the npm-shrinkwrap.json package like:
"..": {
"name": "@local-link-sources/upper",
"version": "1.0.0"
},
"node_modules/@local-link-sources/upper": {
"resolved": "..",
"link": true
}
it fails with:
bb.data_smart.ExpansionError: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception ParameterError: URL: 'npmsw://npm-shrinkwrap.json' has invalid parameters. Unsupported dependency: ..
because it no longer has the "file:" prefix in the version to recognize local link.
https://git.openembedded.org/bitbake/commit/lib/bb/fetch2/npmsw.py?id=19b9f7f0f451a636f3fdcdc1bb283ab431ede612
switched to walk "packages" instead of "dependencies" to support
lockfileVersion 2 (and 3) as in:
No version provided: an "ancient" shrinkwrap file from a version of npm prior to npm v5.
1: The lockfile version used by npm v5 and v6.
2: The lockfile version used by npm v7 and v8. Backwards compatible to v1 lockfiles.
3: The lockfile version used by npm v9 and above. Backwards compatible to npm v7.
Add the npm source with npm-shrinkwrap files generated for all 3
versions to better document the differences in this case and also
easily compare what might change in some future npm with another
lockfileVersion:
npm-shrinkwrap-v1.json generated with npm-6.14.18
npm-shrinkwrap-v2.json generated with npm-8.19.4
npm-shrinkwrap-v3.json generated with npm-10.1.0
There is only exception for "" package name used for root package
and for all other packages it expects "node_modules/" prefix, which
is stripped from the name
* I don't know how common this case is in npm ecosystem, but this is what
we have in some component and what was failing with current npmsw://
implementation, the SRC_URI is git repository with some libraries
and a "sampler", S points to the "sampler" directory and npmsw:// uses
the shrinkwrap file from this "sampler" which refers to the libraries
to test in the directories above (so that it tests the libraries from
the same checkout, not some other released version from npm registry)
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
parent directory
* parse only the packages starting with node_modules/
it was already stripping the 'node_modules/' prefix from package name
and had exception for root package with empty name, but for other local
sources like '..' it failed to parse the url and triggered ParamterError
* also allow missing version in cases which are resolved and link, e.g.:
"node_modules/@npm-local-link-sources/upper": {
"resolved": "..",
"link": true
}
* we need to use symlink instead of copy, because otherwise cp will fail
when trying to copy parent directory from relative path over itself
this seems much more fragile than I would prefer, but haven't found
easier way yet
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
* be a bit more strict when searching for file: prefix, but it's not clear
if file:// prefix is allowed as well, the 3 examples in:
https://docs.npmjs.com/cli/v10/configuring-npm/package-lock-json
currently show:
http tarball sources: This is the URL of the tarball. (eg, https://example.com/example-1.3.0.tgz)
local tarball sources: This is the file URL of the tarball. (eg file:///opt/storage/example-1.3.0.tgz)
local link sources: This is the file URL of the link. (eg file:libs/our-module)
for absolute path it doesn't matter much as
///opt/storage/example-1.3.0.tgz
will still resolve correctly but if someone uses file://../ then
it won't be resolved as relative path (npm itself resolved "file:../",
"file://../", "../" and just ".." the same).
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
* instead of:
bb.fetch2.ParameterError: URL: 'npmsw:///tmp/bitbake-fetch-pqaqvbau/data/npm-shrinkwrap.json' has invalid parameters. Unsupported dependency: ..
show:
bb.fetch2.ParameterError: URL: 'npmsw:///tmp/bitbake-fetch-0h0bgkfn/data/npm-shrinkwrap.json' has invalid parameters. Unsupported dependency: name '..', depname '@npm-local-link-sources/upper', version '1.0.0', resolved 'None'
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
|
|
This adds the missing import. This bug was introduced with 1ab1d36c.
Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Careful reading of the code can contrive cases where poorly timed
ConnectionError's will result in the client mode being incorrectly reset
to MODE_NORMAL when it should actual be a stream mode for the current
command. Fix this by no longer attempting to restore the mode when the
connection is setup. Instead, attempt to set the stream mode inside the
send wrapper for the stream data, which means that it should always end
up in the correct mode before continuing.
Also, factor out the transition to normal mode into a invoke() override
so it doesn't need to be specified over and over again.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that the bitbake hash server supports SSL connections, we need to
capture a few environment variables which can affect the ability to
connect via SSL. Note that the variables are only put in place to affect
the environment while actually invoking the server
[RP: Tweak to use BB_ORIGENV as well]
[RP: Tweak to handle os.environ restore correctly]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A function accidentally defined as:
somefunction() {
:
}
which is unclosed due to the space at the end, would currently silently
cause breakage. Have the parser throw and error for this.
[YOCTO #15470]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In db.py, the ifnull() statement guarantees that the SQL request will
return a value. It's therefore unnecessary to test the case when no
value is found.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
according to db.py, prserv.NotFoundError is returned here when
adding a new value to the database failed
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This exception handler is already present in db.py's get_value() code.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Useful for connecting a PR server to an upstream one
- "test-package" checks whether the specified package
version and arch is known in the database.
- "test-pr" checks a specified output hash is found in the database.
Otherwise it returns 'None' instead of a new value.
- "max-package-pr" returns the highest PR number for
(version, arch) entries in the database, and None if not found
Add new DB functions supporting the above, plus test_value()
which tells whether a given value is available for the specified
package and architecture.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Choosing only one style of capitalization
Add extra space after some commas too
Remove idle spaces
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Moving the code and related definitions from
hashserv/__init__.py to asyncrpc/client.py,
allowing this function to be used in other asyncrpc clients.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Suggested-by: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In both the PRServerClient and PRClient objects.
This aligns with what is done in hashserv/server.py and makes it
possible to benefit from possible specializations of the logger
in the corresponding super classes, instead of using
always the global logger.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
optparse is deprecated since Python 2.7
Note that this is neither supposed to change the options
supported by bitbake-prserv nor the way they are interpreted.
Note that in the "--help" output, long options are now reported
for example as "--host HOST" instead of "--host=HOST" but
both are equivalent anyway, as they already were with optparse.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To aligh with the hashserv code
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
serv.py: simplify the PRServerClient() interface by passing the
server object instead of multiple arguments, and then retrieving
the data through this object.
This replicates what is done for ServerClient() in hashserv/server.py
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Tim Orling <ticotimo@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Document BB_LOADFACTOR_MAX which was recently added.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is inspired by the same section in the yocto-docs. It aims to provide
information in what contexts(recipes, .conf, bbclass,...) a variable is usually
used. For that I tried to group similar variables, so that a short overview is
given. This was inspired by [YOCTO #14072], but of course does not implement a
warning if a variable is used in an unintended context.
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fedora 40 introduces wget2 as a drop-in replacement for wget. This
rewrite does not currently have support for FTP. This causes
the wget fetcher to fail complaining about an unrecognized option.
Making --passive-ftp conditional based on the protocol used in
the SRC_URI limits the scope of the problem. It also gives us
an opportunity to build the older wget as a host tool.
Signed-off-by: Rob Woolley <rob.woolley@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there is an error parsing .netrc, warn the user on stderr
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds support for hashserver credentials to be specified in the
SignatureGenerator
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Git uses a lock file to prevent concurrent modifications to the global
config, so if unpack tasks for different recipes try to run "git lfs
install" simultaneously the operation can fail:
error: could not lock config file /home/build/.gitconfig: File exists exit status 255
Run `git lfs install --force` to reset Git configuration.
Adding "--local" sets the smudge and clean filters in the local
repository's config instead of modifying the user's global config.
Signed-off-by: Derek Erdmann <derek.erdmann@sonos.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch addresses an issue in bitbake-worker where stdout,
reserved for status reporting, is improperly accessed by child processes.
The problem occurs during the execution of parseRecipe,
which calls anonymous functions. If these functions use print-like operations,
they can inadvertently output data to stdout. This unexpected data can cause
the runqueue to hang silently, if the stdout buffer is flushed
before exec_task is executed.
To prevent this, the patch redirects stdout to /dev/null and ensures it is
flushed prior to the execution of exec_task.
Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running in a cgroup which is limited to a subset of cpus (via
cpuset.cpus), cpu_count() should return the number of cpus that can be
used instead of the number of cpus the system has.
This also aligns the semantics with nproc.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a command is run with a non-null log, errors are only output to the
log and are not returned in the exception. In that case direct users to
that logfile instead of telling the command had no output.
Signed-off-by: Viswanath Kraleti <quic_vkraleti@quicinc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Whilst typically the URI query is a list of key-value pairs, that's not
actually required by the URI specification.
For example: http://example.com/foo?bar is a valid query, but this will
result in the fetcher raising an exception:
File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__
self.query = self._param_str_split(urlp.query, "&")
File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split
for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]:
ValueError: not enough values to unpack (expected 2, got 1)
In this case the query is just "bar", but the fetcher is trying to split
it into a key-value pair.
The URI object exposes the parsed query explicitly as a dictionary of
key-value pairs, so we have to be a little creative here: if a value is
None then it isn't a key-value pair, but a bare key.
Fix this by handling elements without the deliminator in _param_str_split()
(by assigning the value to None), and handle a None value when formatting
the query in _param_str_join().
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Red text on a black background can make it difficult for people with
visual impairments to read the text of error messages. Respect the
presence of a non-empty NO_COLOR environment variable as an indication
the user doesn't want colorization to be automatically enabled.
See: https://no-color.org/
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
FIXES [YOCTO #15404]
When using git fetcher on a repo with parentheses in its URL, the
invocation of the git clone command will fail. The clone directory
is not quoted thus the shell will return an error and won't execute
the command.
(Bitbake rev: b5624ee5643d881afa004571a096a189ab5389b5)
Cc: Philippe Rivest <privest@genetec.com>
Signed-off-by: Philippe Rivest <technophil98@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Limit the split to key & value (2 items) instead of the n items one
can get if there are '=' characters in the value.
Fixes [YOCTO #15447]
Signed-off-by: david d zuhn <david.zuhn@sonos.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* In the current implementation, "umask" variable is initially set to
None and overwritten with user-specified value. However, in the worker
implementation, a faulty if clause would only check whether the
variable contains a value that evaluates to True, and not whether
the variable is defined, so the value of 0 would lead to umask not
being changed.
This bug makes it impossible to have a task set its umask to value 0,
for any possible reason it may want to.
Fix this bug by extending the condition checked in the worker
implementation.
Signed-off-by: Sava Jakovljev <sjakovljev@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
preserved_envvars() is used when the BB_ENV_PASSTHROUGH
environment variable is not set. Therefore, its code shouldn't
return this variable.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reported-by: Robert P. J. Day <rpjday@crashcourse.ca>
Tested-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A race condition has happened where the exiting server removed the PID
file between the existence check and the removal, resulting in a
FileNotFoundError exception.
The fix is to ignore the FileNotFoundError exception, the existence
check is now redundant so remove it to simplify.
Fixes [YOCTO #14341]
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently when git fetches fail, it destroys all the existing local clone data.
For large repositories this can introduce long build delays when for example,
you just typo'd the git revision hash.
The git fetcher should be able to recover most directories so when the fetch is
for a git repo, avoid removing things unless clean is explicitly called
(e.g. a -c cleanall task).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Drop the ".py" file extension from the "-u" option's argument that has been
overlooked while applying the original patch (see [1]) to make the example work.
While at it sort the recipes' names consistently with respect to what is found
in the self-test examples below.
[1] https://lore.kernel.org/bitbake-devel/6f2645a7c4db2ae149d387544d2b94209cfed3f4.camel@linuxfoundation.org/
Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds support for asyncrpc servers to send connection headers to clients
on connection. Since this is a breaking protocol change, clients must
opt-in to expect headers from the server, corresponding to a version
bump in the client protocol.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When a LayerBranch (a specific layer at a specific release) does not
exist in the layerindex database ("Layerindex Metadata"), the dependency
would throw a backtrace. Instead fail early and provide an error message.
Since layerindexlib will also check the local layers, inform the user that
a local checkout might resolve the situation. Recommend that they reach
out to the layer maintainers and layer index admins to properly fix it for
everyone.
While we are here, remove some trailing whitespace.
[YOCTO #15365]
[YOCTO #13954]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current implementation only performs a git lfs fetch alongside of a
regular git fetch. This causes issues when the downloaded revision is
already part of the fetched repository (e.g. because of moving back in
history or the updated revision already being part of the repository at
the time of the initial clone).
Fix this by explicitly checking whether the required LFS objects are
available in the downloade directory before confirming that a downloaded
repository is up-to-date.
This issue previously went unnoticed as git lfs would silently fetch the
missing objects during the `unpack` task. With network isolation turned
on, this no longer works, and unpacking fails.
Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a very rare case where the maxval is improperly computed
initially for cache loading progress, and the value will go over.
Explanation from bitbake/lib/bb/cache.py:736 in MulticonfigCache:__init__:progress()
# we might have calculated incorrect total size because a file
# might've been written out just after we checked its size
In that case, progressbar will receive a value over the initial maxval.
This results in a ValueError stack trace as well as bitbake returning 1.
Traceback (most recent call last):
File ".../poky/bitbake/lib/bb/ui/knotty.py", line 736, in main
cacheprogress.update(event.current)
File ".../poky/bitbake/lib/progressbar/progressbar.py", line 256, in update
raise ValueError('Value out of range')
ValueError: Value out of range
This fix mirrors the behavior of MulticonfigCache and accepts the new
value as the new maxval. This is also what the percentage printout
is doing in bitbake/lib/progressbar/progressbar.py:191 in ProgressBar:percentage()
I encountered this issue randomly while working on a project with
VSCode saving files while commands where fired.
Note: This file is a fork from python-progressbar. It hasn't been
refreshed in 8 years. We did only two commits, 5 years ago with minor
modifications. This new change is also not how the upstream project is
behaving.
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some ditros don't enable /proc/pressure and it tends to be those which we
see bitbake timeout issues on, seemingly as load gets too high and the bitbake
processes don't get scheduled in for minutes at a time.
Add support for stopping running extra tasks if the system load average goes
above a certain threshold by setting BB_LOADFACTOR_MAX.
The value used is scaled by CPU number, so a value of 1 would be when
the load average equals the number of cpu cores of the system, under one
only starts tasks when the load average is below the number of cores.
This means you can centrally set a value such as 1.5 which will then
scale correctly to different sized machines with differing numbers
of CPUs.
The pressure regulation is probably more accurate and responsive, however
our graphs do show singificant load spikes on some workers and this
patch is aimed at trying to avoid those.
Pressure regulation is used where available in preference to this load
factor regulation when both are set.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Re-enable connection pooling in case `postgresql+psygopg` driver
is used. Async connection pooling is supported in psycopg 3 [psycopg]
driver in SQLAlchemy. Allow the connection pool to grow to
arbitrary size.
Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 987ab2a446aab235d6e86e97c05f25fb800d7acc.
There's been a report that this breaks downloads from Jfrog Artifactory
as self.user_agent is set to 'Mozilla Firefox', and when Artifactory
sees that, it sends a response tailored for showing in an interactive browser
(which in my opinion it has every right to).
If we're using wget, we should say so via wget's default; handling uncooperative
servers should be done on per-recipe basis, and ideally with tickets
to admins of those servers.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Before, everything up to the last slash was removed when extracting the
names of the tags. This would lead to that a tag such as "agent/11.0.0"
would be incorrectly identified as "11.0.0", which would then be treated
as a correct version matching "^(?P<pver>\d+(\.\d+)+)".
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is mostly preparations for the next commit.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Hieu Van Nguyen
Martin Jansa
Sven Schwermer
Richard Purdie
Joshua Watt
Michael Opdenacker
Simone Weiß
Rob Woolley
Derek Erdmann
Yang Xu
Felix Moessbauer
Viswanath Kraleti
Ross Burton
Peter A. Bigot
Philippe Rivest
david d zuhn
Sava Jakovljev
Alexander Kanavin
Yoann Congal
Ulrich Ölmann
Tim Orling
Philip Lorenz
Enguerrand de Ribaucourt
Tobias Hagelborn
Peter Kjellerstedt