You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Uses error_codes found in test_tools/error_codes to provide more specific error indication.
Before/After Comparison
Before: Returned a 0 or 1, making determination of having to rerun very course
After: Error codes are now bases on test_tools/error_codes, making it easier to limit when we do a rerun of the test.
Also, changed how we load in test_tools, we try curl, git, and wget, to avoid a failure due to the program not
being installed.
Update return error codes to be more specific (stop returning only 0/1).
Update the way test_tools-wrappers are loaded.
Non-compliant requirements:
Requires further human verification:
Validate that the new error codes (E_GENERAL, E_USAGE, and 101) are correctly interpreted by downstream automation/rerun logic in real runs (end-to-end behavior).
Validate tool-loading behavior across environments lacking curl/wget/git and that failures are surfaced with the intended exit codes/messages.
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 Security concerns
Supply chain / integrity: The new logic downloads main.zip from https://github.com/redhat-performance/test_tools-wrappers/archive/refs/heads/main.zip via curl/wget and unzips it without any checksum/signature verification or pinning to a specific commit/tag. This can allow unexpected code changes (or tampering in transit/environment) to be executed when sourced later (e.g., source ${TOOLS_BIN}/general_setup). Consider pinning to a commit/tag and verifying a known checksum (or using git clone of a pinned ref) before sourcing/executing.
The new download paths (curl/wget fetching a GitHub zip) do not verify integrity (checksum/signature) and unzip content into place. Consider adding a verification step (pinned commit/tag + checksum) and/or restricting to git clone with a pinned ref to reduce tampering risk.
attempt_tools_wget()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
wget ${tools_git}/archive/refs/heads/main.zip
if [[ $? -eq 0 ]]; then
unzip -q main.zip
mv test_tools-wrappers-main ${TOOLS_BIN}
rm main.zip
fi
fi
}
attempt_tools_curl()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
curl -L -O ${tools_git}/archive/refs/heads/main.zip
if [[ $? -eq 0 ]]; then
unzip -q main.zip
mv test_tools-wrappers-main ${TOOLS_BIN}
rm main.zip
fi
fi
}
attempt_tools_git()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
git clone $tools_git "$TOOLS_BIN"
if [ $? -ne 0 ]; then
exit_out "Error: pulling git $tools_git failed." 101
fi
fi
}
attempt_tools_wget
attempt_tools_curl
attempt_tools_git
The wget/curl attempts only proceed to git if TOOLS_BIN still doesn't exist, but they don't explicitly handle missing binaries or partial failures (e.g., wget downloads HTML/error page, unzip fails, or leftover main.zip). Consider checking command availability, validating unzip success, and cleaning up on failure to avoid confusing subsequent runs.
attempt_tools_wget()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
wget ${tools_git}/archive/refs/heads/main.zip
if [[ $? -eq 0 ]]; then
unzip -q main.zip
mv test_tools-wrappers-main ${TOOLS_BIN}
rm main.zip
fi
fi
}
attempt_tools_curl()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
curl -L -O ${tools_git}/archive/refs/heads/main.zip
if [[ $? -eq 0 ]]; then
unzip -q main.zip
mv test_tools-wrappers-main ${TOOLS_BIN}
rm main.zip
fi
fi
}
attempt_tools_git()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
git clone $tools_git "$TOOLS_BIN"
if [ $? -ne 0 ]; then
exit_out "Error: pulling git $tools_git failed." 101
fi
fi
}
attempt_tools_wget
attempt_tools_curl
attempt_tools_git
Some failure paths now use E_GENERAL/E_USAGE, but the test_tools clone failure uses a hard-coded 101. Ensure 101 aligns with the shared test_tools/error_codes taxonomy (or replace it with a named constant) so callers can consistently interpret failures.
attempt_tools_git()
{
if [[ ! -d "$TOOLS_BIN" ]]; then
git clone $tools_git "$TOOLS_BIN"
if [ $? -ne 0 ]; then
exit_out "Error: pulling git $tools_git failed." 101
fi
fi
}
The reason will be displayed to describe this comment to others. Learn more.
Variable "test_name_run" is undefined, looks like in most places it was updated to "test_name". Let's make it consistent one way or the other and then LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Uses error_codes found in test_tools/error_codes to provide more specific error indication.
Before/After Comparison
Before: Returned a 0 or 1, making determination of having to rerun very course
After: Error codes are now bases on test_tools/error_codes, making it easier to limit when we do a rerun of the test.
Also, changed how we load in test_tools, we try curl, git, and wget, to avoid a failure due to the program not
being installed.
Clerical Stuff
This closes #57
Mention the JIRA ticket of the issue, this helps keep
everything together so we can find this PR easily.
Relates to JIRA: RPOPC-871
Testing Done
Ran scenario file
global:
ssh_key_file: replace_your_ssh_key
terminate_cloud: 1
os_vendor: rhel
results_prefix: documentation
os_vendor: rhel
system_type: aws
cloud_os_id: ami-035032ea878eca201
systems:
system1:
tests: "coremark_pro"
host_config: "m5.xlarge"
csv results file
Test,Multi_iterations,Single_iterations,Scaling,Start_Date,End_Date
cjpeg-rose7-preset,384.62,147.06,2.62,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
core,4.20,1.70,2.47,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
linear_alg-mid-100x100-sp,314.47,146.20,2.15,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
loops-all-mid-10k-sp,16.68,6.48,2.57,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
nnet_test,19.31,8.51,2.27,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
parser-125k,121.21,34.48,3.52,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
radix2-big-64k,1602.56,601.68,2.66,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
sha-test,384.62,192.31,2.00,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
zip-test,285.71,125.00,2.29,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
Score,13219.99,5343.15,2.47,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
Returned 0 as expected. Testing on rerun verification handled in the appropriate zathras pr.