Fix tests silently skipped by Docker BuildKit

[welteki]

Description

Remove the separate FROM build AS test stage from all template Dockerfiles and move the test commands directly into the build stage. Change the default value of TEST_ENABLED from true to false, making testing opt-in. Update the README and handler_test.py comments to reflect the new default.

Motivation and Context

Docker BuildKit (the default builder since Docker 23.0) optimizes builds by skipping stages that are not referenced by the final target. The test stage was not a dependency of the ship stage, so BuildKit silently skipped it — meaning tests never ran during faas-cli build.

By moving the test commands into the build stage (which ship depends on), tests are guaranteed to execute when enabled.

• I have raised an issue to propose this change (required)
• My issue has received approval from the maintainers or lead with the design/approved label

How Has This Been Tested?

Built a function using faas-cli build with --build-arg TEST_ENABLED=true and verified that tests are executed during the build. Also confirmed that without the build arg, tests are skipped as expected.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I've read the CONTRIBUTION guide
• I have signed-off my commits with git commit -s
• I have added tests to cover my changes.
• All new and existing tests passed.

[reviewfn]

AI Pull Request Overview

Summary

• Removed separate FROM build AS test stage from all template Dockerfiles to prevent Docker BuildKit from skipping test execution.
• Moved test commands directly into the build stage, ensuring tests run when TEST_ENABLED=true since ship depends on build.
• Changed default value of TEST_ENABLED from true to false across all templates, making testing opt-in.
• Updated README.md and handler_test.py comments in all templates to reflect the new opt-in testing behavior.

Approval rating (1-10)

9

Summary per file

Summary per file

File path	Summary
README.md	Updated testing documentation to indicate testing is disabled by default and must be explicitly enabled.
template/python3-flask-debian/Dockerfile	Removed test stage, moved test execution to build stage, changed TEST_ENABLED default to false.
template/python3-flask-debian/function/handler_test.py	Updated comment to instruct enabling tests via TEST_ENABLED=true.
template/python3-flask/Dockerfile	Removed test stage, moved test execution to build stage, changed TEST_ENABLED default to false.
template/python3-flask/function/handler_test.py	Updated comment to instruct enabling tests via TEST_ENABLED=true.
template/python3-http-debian/Dockerfile	Removed test stage, moved test execution to build stage, changed TEST_ENABLED default to false.
template/python3-http-debian/function/handler_test.py	Updated comment to instruct enabling tests via TEST_ENABLED=true.
template/python3-http/Dockerfile	Removed test stage, moved test execution to build stage, changed TEST_ENABLED default to false.
template/python3-http/function/handler_test.py	Updated comment to instruct enabling tests via TEST_ENABLED=true.

Overall Assessment

The changes correctly address the issue with Docker BuildKit silently skipping tests by integrating test execution into the build stage, which is a prerequisite for the final ship stage. Making testing opt-in by default is a sensible performance optimization. The modifications are consistent across all four template variants and align with the PR description. No regressions or security concerns identified.

Detailed Review

Detailed Review

Dockerfile Changes

All four Dockerfiles (python3-flask, python3-flask-debian, python3-http, python3-http-debian) have identical changes:

• Removal of FROM build AS test eliminates the unused stage that BuildKit optimizes away.
• Relocation of the conditional test run (RUN [ "$TEST_ENABLED" = "false" ] && echo "skipping tests" || eval "$TEST_COMMAND") from the test stage to the build stage ensures execution when enabled.
• Changing ARG TEST_ENABLED=true to ARG TEST_ENABLED=false enforces opt-in testing, preventing unintended build delays.

This approach resolves the core issue without introducing new dependencies or altering the build process for users who disable tests.

Test File Comment Updates

Each handler_test.py file updated the comment from "To disable testing..." to "To enable testing...", with the corresponding build_arg change from TEST_ENABLED=false to TEST_ENABLED=true. This accurately reflects the new default behavior and provides clear guidance for users.

README.md Update

The testing section now states "Testing is disabled by default and must be explicitly enabled," which is accurate and helps users understand the change. The examples for enabling tests via CLI or YAML remain unchanged and correct.

Potential Considerations

• No impact on existing functions that explicitly set TEST_ENABLED=true in their configuration.
• The change may surprise users who relied on default test execution, but the opt-in approach is preferable for CI/CD performance.
• All templates maintain consistency, reducing confusion.

No code issues, security risks, or performance regressions observed.

AI agent details.

Agent processing time: 1m6.693s
Environment preparation time: 5.726s
Total time from webhook: 1m14.978s