Skip to content

Update error codes to be more specific#58

Merged
dvalinrh merged 5 commits intomainfrom
rerun
Mar 24, 2026
Merged

Update error codes to be more specific#58
dvalinrh merged 5 commits intomainfrom
rerun

Conversation

@dvalinrh
Copy link
Copy Markdown
Contributor

Description

Uses error_codes found in test_tools/error_codes to provide more specific error indication.

Before/After Comparison

Before: Returned a 0 or 1, making determination of having to rerun very course
After: Error codes are now bases on test_tools/error_codes, making it easier to limit when we do a rerun of the test.
Also, changed how we load in test_tools, we try curl, git, and wget, to avoid a failure due to the program not
being installed.

Clerical Stuff

This closes #57

Mention the JIRA ticket of the issue, this helps keep
everything together so we can find this PR easily.

Relates to JIRA: RPOPC-871

Testing Done
Ran scenario file
global:
ssh_key_file: replace_your_ssh_key
terminate_cloud: 1
os_vendor: rhel
results_prefix: documentation
os_vendor: rhel
system_type: aws
cloud_os_id: ami-035032ea878eca201
systems:
system1:
tests: "coremark_pro"
host_config: "m5.xlarge"

csv results file
Test,Multi_iterations,Single_iterations,Scaling,Start_Date,End_Date
cjpeg-rose7-preset,384.62,147.06,2.62,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
core,4.20,1.70,2.47,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
linear_alg-mid-100x100-sp,314.47,146.20,2.15,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
loops-all-mid-10k-sp,16.68,6.48,2.57,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
nnet_test,19.31,8.51,2.27,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
parser-125k,121.21,34.48,3.52,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
radix2-big-64k,1602.56,601.68,2.66,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
sha-test,384.62,192.31,2.00,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
zip-test,285.71,125.00,2.29,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z
Score,13219.99,5343.15,2.47,2026-03-13T14:47:59Z,2026-03-13T14:48:59Z

Returned 0 as expected. Testing on rerun verification handled in the appropriate zathras pr.

@dvalinrh dvalinrh requested a review from grdumas March 13, 2026 15:09
@github-actions
Copy link
Copy Markdown

This relates to RPOPC-871

@kdvalin
Copy link
Copy Markdown
Member

kdvalin commented Mar 24, 2026

/review

@qodo-code-review
Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

57 - Partially compliant

Compliant requirements:

  • Update return error codes to be more specific (stop returning only 0/1).
  • Update the way test_tools-wrappers are loaded.

Non-compliant requirements:

Requires further human verification:

  • Validate that the new error codes (E_GENERAL, E_USAGE, and 101) are correctly interpreted by downstream automation/rerun logic in real runs (end-to-end behavior).
  • Validate tool-loading behavior across environments lacking curl/wget/git and that failures are surfaced with the intended exit codes/messages.
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 Security concerns

Supply chain / integrity:
The new logic downloads main.zip from https://github.com/redhat-performance/test_tools-wrappers/archive/refs/heads/main.zip via curl/wget and unzips it without any checksum/signature verification or pinning to a specific commit/tag. This can allow unexpected code changes (or tampering in transit/environment) to be executed when sourced later (e.g., source ${TOOLS_BIN}/general_setup). Consider pinning to a commit/tag and verifying a known checksum (or using git clone of a pinned ref) before sourcing/executing.

⚡ Recommended focus areas for review

Supply Chain

The new download paths (curl/wget fetching a GitHub zip) do not verify integrity (checksum/signature) and unzip content into place. Consider adding a verification step (pinned commit/tag + checksum) and/or restricting to git clone with a pinned ref to reduce tampering risk.

attempt_tools_wget()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                wget ${tools_git}/archive/refs/heads/main.zip
                if [[ $? -eq 0 ]]; then
                        unzip -q main.zip
                        mv test_tools-wrappers-main ${TOOLS_BIN}
                        rm main.zip
                fi
        fi
}

attempt_tools_curl()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                curl -L -O ${tools_git}/archive/refs/heads/main.zip
                if [[ $? -eq 0 ]]; then
                        unzip -q main.zip
                        mv test_tools-wrappers-main ${TOOLS_BIN}
                        rm main.zip
                fi
        fi
}

attempt_tools_git()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                git clone $tools_git "$TOOLS_BIN"
                if [ $? -ne 0 ]; then
                        exit_out "Error: pulling git $tools_git failed." 101
                fi
        fi
}

attempt_tools_wget
attempt_tools_curl
attempt_tools_git
Error Handling

The wget/curl attempts only proceed to git if TOOLS_BIN still doesn't exist, but they don't explicitly handle missing binaries or partial failures (e.g., wget downloads HTML/error page, unzip fails, or leftover main.zip). Consider checking command availability, validating unzip success, and cleaning up on failure to avoid confusing subsequent runs.

attempt_tools_wget()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                wget ${tools_git}/archive/refs/heads/main.zip
                if [[ $? -eq 0 ]]; then
                        unzip -q main.zip
                        mv test_tools-wrappers-main ${TOOLS_BIN}
                        rm main.zip
                fi
        fi
}

attempt_tools_curl()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                curl -L -O ${tools_git}/archive/refs/heads/main.zip
                if [[ $? -eq 0 ]]; then
                        unzip -q main.zip
                        mv test_tools-wrappers-main ${TOOLS_BIN}
                        rm main.zip
                fi
        fi
}

attempt_tools_git()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                git clone $tools_git "$TOOLS_BIN"
                if [ $? -ne 0 ]; then
                        exit_out "Error: pulling git $tools_git failed." 101
                fi
        fi
}

attempt_tools_wget
attempt_tools_curl
attempt_tools_git
Exit Codes

Some failure paths now use E_GENERAL/E_USAGE, but the test_tools clone failure uses a hard-coded 101. Ensure 101 aligns with the shared test_tools/error_codes taxonomy (or replace it with a named constant) so callers can consistently interpret failures.

attempt_tools_git()
{
        if [[ ! -d "$TOOLS_BIN" ]]; then
                git clone $tools_git "$TOOLS_BIN"
                if [ $? -ne 0 ]; then
                        exit_out "Error: pulling git $tools_git failed." 101
                fi
        fi
}

Copy link
Copy Markdown
Contributor

@grdumas grdumas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable "test_name_run" is undefined, looks like in most places it was updated to "test_name". Let's make it consistent one way or the other and then LGTM.

Copy link
Copy Markdown
Contributor

@grdumas grdumas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@grdumas grdumas added the group_review_lgtm Indicates approval after a group review meeting label Mar 24, 2026
@dvalinrh dvalinrh merged commit e49d026 into main Mar 24, 2026
2 checks passed
@grdumas grdumas deleted the rerun branch March 25, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

group_review_lgtm Indicates approval after a group review meeting Possible security concern Review effort 3/5

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update to be more specfic on error codes.

3 participants