diff options
author | Richard Purdie <richard.purdie@linuxfoundation.org> | 2020-01-03 18:03:35 +0000 |
---|---|---|
committer | Richard Purdie <richard.purdie@linuxfoundation.org> | 2020-01-06 23:53:54 +0000 |
commit | d05bde16bdad761ed8f4c0a48de60c649aa33e85 (patch) | |
tree | ad872406f4fc1163a0e1f81ec7d4fe49f71ff38c /scripts/gen-lockedsig-cache | |
parent | 5115d2616b0847d521ccbecf1fbbf1f695a04fea (diff) | |
download | openembedded-core-contrib-d05bde16bdad761ed8f4c0a48de60c649aa33e85.tar.gz |
sstate: Add extra directory level
We're having speed issues on the autobuilder due to the numbers of files in sstate
directories. We previously split these by the first two characters of the hash.
This change extends this to split by the next two characters as well, creating
more layers of directories.
This should signifiantly speed up eSDK builds on the autobuilder as the current
sstate layout simply isn't scaling there but addresses a general complaint.
gen-lockedsig-cache needed to be updated for the new split level sstate.
Also update tests for new layout.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffstat (limited to 'scripts/gen-lockedsig-cache')
-rwxr-xr-x | scripts/gen-lockedsig-cache | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/scripts/gen-lockedsig-cache b/scripts/gen-lockedsig-cache index 9bfae9d8323..6a7d2859104 100755 --- a/scripts/gen-lockedsig-cache +++ b/scripts/gen-lockedsig-cache @@ -78,11 +78,18 @@ files = set() sstate_content_cache = {} for s in sigs: prefix = s[:2] + prefix2 = s[2:4] if prefix not in sstate_content_cache: sstate_content_cache[prefix] = build_sha_cache(prefix) - - for f in sstate_content_cache[prefix][s]: - files.add(f) + if prefix2 not in sstate_content_cache[prefix]: + sstate_content_cache[prefix][prefix2] = build_sha_cache(prefix + "/" + prefix2) + + if s in sstate_content_cache[prefix]: + for f in sstate_content_cache[prefix][s]: + files.add(f) + if s in sstate_content_cache[prefix][prefix2]: + for f in sstate_content_cache[prefix][prefix2][s]: + files.add(f) elapsed = time.perf_counter() - start_time print("Gathering file list took %.1fs" % elapsed) |