Age | Commit message (Collapse) | Author |
|
Update Toaster to support Django 4.2, to match current
hosts and to address CVEs.
[YOCTO #15152]
Signed-off-by: Kieran McNulty <Kieran.McNulty@windriver.com>
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
If the clone target directory exists and is a valid git repo, but the
git directory is not a child, it needs to be erased and re-cloned. One
example of how this can happen is if a clone creates the directory, but
then fails to actual clone and make it a git repository. This left-over
directory can be particularly problematic if the download directory is a
descent of some top layer git repo (e.g. the default with poky), as the
commands that operate on the remote later will then mangle the layers
git repository instead of the download git repo.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
|
|
Explicitly pass down the lfs parameter from the source repo URI to
submodules. Add smudge skip to final checkout phase to make sure we
don't access the network here. Everything should have been fetched and
setup from the lfs logic in the git fetcher at this point.
Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Error occured while running bitbake on cephfs:
WARNING: The free inode of path is running low (-0.001K left)
ERROR: Immediately halt since the disk space monitor action is "HALT"!
Signed-off-by: Samantha Jalabert <samantha.jalabert@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds Python wrappers around the xattr API from libc and the ACL API from
libacl.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* with latest bitbake I'm seeing very low number of bitbake tasks
executed in parallel, probably due to pressure regulation
show the values this is based on in the note
* also simplify a bit by counting the pressure and exceeds signs
only once
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add new functions to return some of the get_srcrev data in new and different
ways. We need two different forms of the data, one is a string to inject into
PKGV, the other is the full revisions as a string to include in hash computations
so that the hash changes when the input revisions change.
This allows us to clean up and simplify the code in OE-Core and move the
version information from PV to PKGV.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
file checksums are part of the data checksummed
to generate the task hash. The list of file checksums
was not ordered.
In this commit we make sure the task hash checksum takes
a list of checksum data that is ordered by unique file name
thus guaranteeing reproducibility.
Signed-off-by: Paulo Neves <paulo@myneves.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"Not" should be "No"
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adds a --mc argument to `bitbake-layers show-appends` so that users can
filter appends for a specific multiconfig (instead of always showing the
default configuration)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The debug in the comments was out of date. It is still useful so update the
code sample to the new code needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current calculation is not correct because if tdiff is less than
1.0, it's not taken into consideration when calculating the current
pressure.
Also, make it clear that the 1.0s is the psi accumulation cycle,
which might be changed in the future. We have this cycle because it
could largely avoid the 0 result issue, that is, if the interval
between checks are too small, the result might be 0. With this
accumulation logic, which has been there but let's make it clear,
this 0 result problem could be mitigated.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This fetcher allows BitBake to fetch from a Google Cloud Storage
bucket. The fetcher expects a gs:// URI of the following form:
SSTATE_MIRRORS = "file://.* gs://<bucket name>/PATH"
The fetcher uses the Google Cloud Storage Python Client, and
expects it to be installed, configured, and authenticated prior
to use.
If accepted, this patch should merge in with the corresponding oe-core
patch titled "Add GCP fetcher to list of supported protocols".
Some comments on the patch:
There is also documentation for the fetcher added to the User
Manual. I'm still not completely sure about the recommends_checksum()
being set to True. As I've noted in the mailing list, it will throw
warnings if the fetcher is used in recipes without specifying a
checksum. Please let me know if this is intended behavior or if it
should be modified.
Here is how this fetcher conforms to the fetcher expectations
described at this link:
https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/fetch2/README
a) Yes, network fetching only happens in the fetcher
b) The fetcher has nothing to do with the unpack phase so there is no
network access there
c) This change doesn't affect the behavior of DL_DIR. The GCP fetcher
only downloads to the DL_DIR in the same way that other fetchers,
namely the S3 and Azure fetchers do.
d) The fetcher is identical to the S3 and Azure fetchers in this
context
e) Yes, the fetcher output is deterministic because it is downloading
tarballs from a bucket and not modifying them in any way.
f) I set up a local proxy using tinyproxy and set the http_proxy
variable to test whether the Python API respected the proxy. It
appears that it did as I could see traffic passing through the
proxy. I also did some searching online and found posts indicating
that the Google Cloud Python APIs supported the classic Linux proxy
variables, namely:
- https://github.com/googleapis/google-api-python-client/issues/1260
g) Access is minimal, only checking if the file exists and downloading
it if it does.
h) Not applicable, BitBake already knows which version it wants and
the version infomation is encoded in the filename. The fetcher has no
concept of versions.
i) Not applicable
j) Not applicable
k) No tests were added as part of this change. I didn't see any tests
for the S3 or Azure changes either, is that OK?
l) I'm not 100% familiar but I don't believe this fetcher is using any
tools during parse time. Please correct me if I'm wrong.
Signed-off-by: Emil Ekmečić <eekmecic@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
process.signal_received is a list for signum and not iterable,
change a suitable method to handle sig.
Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The runtaskdep data in siginfo files was written out with full paths to
the bb files, matching bitbake's internal "unique key" ID for recipes/tasks.
When originally implemented this made sense.
Over time, the main use for the data in siginfo files has become to match
against other siginfo files to debug changes of hash calcuations. The
recipename data is not useful for this as the siginfo filenames use PN
instead which can often be derived from the recipe filename but not always.
It is time to throw away the 'tid' data format and switch over the use a
hybrid PN form which includes the multiconfig. That can be easily stripped
off in the find_siginfo code in oe-core.
The other purpose of having a sortable dependency ID is retained and the
multiconfig needs to be included to allow the taskhashes to be processed
and calculated correctly. PN is meant to be unique between recipes, only
one would ever be built so using PN in this location is fine.
The one risk of this change is there isn't any compatibility to the old
format. I'm not convinced we should spend time complicating the code
with it. This change will change the taskhashes everywhere so the only
mixing of old and new siginfo files will be either through hash equivalence
or through users using the tool against old and new info files manually
which will give some weird output but it should be clear they're in
different formats as there would be large paths from the old files not
present in the new ones.
We have options to add backwards compatibility if some issue is found
to need that.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add one more verification that checks if localpath is None, because
we can't compute checksum of a None.
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fetch them
When there are no dependencies, _foreach_proxy_method does not verify that there are
dependencies to fetch before fetching them.
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Update test_python_snippet_function_reference to not require the 'func'
flag, now that we know the real function will be returned for context
functions without the flag.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
We previously checked for the existence of the 'func' flag to determine
if we should avoid looking up in the metadata. This was done to ensure
the user gets the function for 'def' python functions rather than their
string contents. We can sidestep the metadata lookup and check our
function context directly, instead.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
This test makes sure that '${@eval()}' calls the eval builtin, even if an 'eval' variable is defined in the metadata.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
This avoids the need to hardcode a list of python builtins. This also slightly changes behavior, in a case like `${@eval("3")}`, this will ensure we always call the builtin, even if the metadata has an 'eval' variable defined.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
This ensures that any change to the presence of builtins in inline
python execs will be noticed.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
This is a test for an issue seen long ago, to avoid regressions, where a
reference to a def'd function in the metadata would return the string
value from the metadata rather than the function in inline python.
Signed-off-by: Christopher Larson <kergoth@gmail.com>
|
|
Fixes a few errors when the bitbake indent plugin is reloaded:
1) Define functions with "!" so that vim doens't issue a warning when
they are replaced
2) Rename GetPythonIndent -> GetBBPythonIndent to prevent potential
conflict with other plugins
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #13550]
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Set the maxsplit value to match the expected number of variables.
This also avoids an unnecessary split as the parameters are in the form 'key=value'
and the 'value' could contain the '=' character.
Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
|
|
Using multiconfig to target baremetal pieces of the system and building
corresponding toolchains for them results in hundreds and hundreds of
"Deferring %s after %s" and "Deferred task %s now buildable".
To clean up the output and to reduce risk of missing important warnings,
convert these notice messages to debug messages.
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Denys Dmytriyenko <denys@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is currently hard to tell when bitbake is throttling task execution
due to system pressure changes. Add notes to the console output to make
this clearer, only generating output when the values change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is unclear from the server logs when parsing starts. We know that timeouts
sometimes happen when parsing but it is unclear where in the code the delays
are from. Adding this debug message to the server log should help narrow
that down.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To learn more about the server timeout issues, be clear in the error
message about which command is showing the timeout. It is currently
unclear if this is the original command or a ping to the server.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"fetch2/npmsw: Add support for the new format of the shrinkwrap file"
added support for the new format shrinkwrap files but this regressed
our tests which still use the old format.
Similar to how npm handles this, support both for now until we can
migrate our tests.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Packages like @(._.)/execute causes problems because they generate names
that are not supported by yocto
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Dev dependencies should not be fetched only if it is specified in the
recipe.
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Npm is a package manager that has its own manner to handle installation of packages.
But it is not yocto friendly, for instance NPM fetch dependencies in the middle of compilation.
The shrinkwrap file changed its format over npm versions, but npm does not version
this file, so we can use it properly.
The actual changes make NPM depencies work with the actual shrinkwrap format.
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there are several multiconfigs in play for example a non-multiconfig with
a task with one hash and then three multiconfigs for the same task, different
architectures but the same hash (different to the non-mc), the three mcs
will be deferred until after the non-mc task but then will all run together
and race against each other.
Change the code to re-enable deferred tasks one at a time. This way, if they do
race, they won't run in parallel against each other.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To support running the bitbake self-tests using pytest, add a
pytest.ini This enables these tests to be run using the following
command from the bitbake directory.
pytest
For now, testpaths only includes the bitbake selftests.
Add a note to the README.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
By default, pytest captures all stdout and exposes it using its built-in
fixtures (capsys, caplog etc), so stdout does not support getvalue().
To support running tests using both unittest and pytest, use assertLogs
to capture logging and assert on the log output instead.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
git config returns an error code if user.email or user.name are not set,
so bb.process.run raises an ExecutionError, failing the test.
Improve the logic by catching the ExectionError and using this to set a
default value.
Change a direct invocation of 'git init' to use self.git_init so that
it follows the code path above.
Remove the related comment from the README now that git sets up the user
details automatically.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
TestCase.assertRaisesRegexp was renamed to assertRaisesRegex in Python
3.2, so rename to fix a warning during test execution.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pytest test collection attempts to collect all classes containing
"Test". TestTimeout is not a test class so add the __test__ = False
attribute to indicate this.
Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
highlights for variables
Some variables are described in the glossary so add a term role to the
references. For the others, highlight them by surrounding them with
two backticks.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Partially reverting https://git.openembedded.org/bitbake/commit/?id=d1779e2f7789e516efae434b3781c981d0ef4ded
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Reported-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Document the new variable.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I'd tested target configurations but in real world use, it became clear
the layername functionality in worker context was failing for virtual
class extensions. Fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #15128]
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This addresses [YOCTO #15127]
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
- Move "langdale" link to outdated section
- Add link to "mickledore" manual
- Fix typo in "kirkstone" name
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Include the hashfn (the value of BB_HASHFILENAME) in the task dependency
data. This allows tasks to get a specific unique hash for dependent
tasks when one is available.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes a crash when the server process attempts to check the PID of the
lock file that resulted because an integer (os.getpid()) was attempting
to be concatenated to a string
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|