refactor: version PyTorch directory structure for multi-version support

[Eren-Jeager123]

Summary

Restructure PyTorch to support maintaining multiple versions concurrently (1-year support window). Versioned directories isolate each version's build artifacts. Workflows are version-agnostic — no new workflow files needed when adding a future version.

1. Versioned directory structure

docker/pytorch/2.11/
├── Dockerfile.cuda
├── Dockerfile.cpu
├── versions-cuda.env      # Build-time dependency versions (sourced by workflows + Dockerfiles)
├── versions-cpu.env
├── cuda/pyproject.toml
├── cuda/uv.lock
├── cpu/pyproject.toml
└── cpu/uv.lock

.github/config/image/
├── pytorch-2.11-ec2-cuda.yml       # Workflow metadata (framework, labels, prod_image)
├── pytorch-2.11-ec2-cpu.yml
├── pytorch-2.11-sagemaker-cuda.yml
└── pytorch-2.11-sagemaker-cpu.yml

Shared scripts (scripts/pytorch/) and tests (test/pytorch/) stay in place.

2. PR workflows — configs matrix (zero changes to reusable workflows)

gatekeeper → detect-versions → build-images / unit-test / sanity / security / telemetry / single-gpu / efa

Key design: detect-versions reads each version's config YAML and outputs a rich configs matrix where each entry carries all metadata fields. Downstream test jobs iterate over this matrix and pass matrix.* fields directly as individual inputs to unmodified reusable workflows.

# detect-versions outputs:
#   configs: [{"version":"2.11", "framework":"pytorch_runtime", ..., "prod_image":"pytorch:2.11-cu130-amzn2023"}]

sanity-test:
  strategy:
    matrix:
      include: ${{ fromJson(needs.detect-versions.outputs.configs) }}
  uses: ./.github/workflows/reusable-sanity-tests.yml
  with:
    image-uri: ${{ needs.build-images.result == 'success' && CI_IMAGE || PROD_IMAGE }}
    framework: ${{ matrix.framework }}
    framework-version: ${{ matrix.framework_version }}
    ...

This solves the GitHub Actions limitation where matrix job outputs only expose the last-finishing leg's values — metadata travels with the matrix, not as job outputs.

Version detection combines both path patterns in a single grep:

git diff --name-only origin/main...HEAD \
  | grep -oP '(?:docker/pytorch/|pytorch-)\K[0-9]+\.[0-9]+'

Prod image fallback: When only test files change (build-change: false), build-images is skipped and test jobs automatically fall back to the prod image from ECR:

image-uri: ${{ needs.build-images.result == 'success' && CI_IMAGE || PROD_IMAGE }}
aws-account-id: ${{ needs.build-images.result == 'success' && CI_ACCOUNT || PROD_ACCOUNT }}

3. Autorelease workflows — simplified

• Single load-config job (merged determine-config + load-config to save an instance)
• version dispatch input instead of full config file path (e.g., 2.11)
• framework-short-version output (e.g., "2.11") parsed once, used everywhere — no repeated cut -d. -f1,2 in downstream jobs
• Cron schedules map version → config file internally

4. Dockerfile ARG for version paths

Dockerfiles use ARG DLC_PYTORCH_VERSION=2.11 instead of hardcoding:

COPY docker/pytorch/${DLC_PYTORCH_VERSION}/cuda/pyproject.toml /tmp/build/

5. Additional improvements

• Unit tests as separate job (parallel with sanity/security/telemetry)
• No unused matrix outputs — removed dead runtime-image-uri / sagemaker-image-uri outputs from build jobs
• Trailing backslash fix in wheel fetch/upload commands
• upload_cached_wheels.sh accepts Dockerfile path as parameter
• test_versions.py reads DLC_PYTORCH_VERSION env var to locate versioned env files
• libcap security patch (CVE-2026-33814, CVE-2026-39820, CVE-2026-33811, CVE-2026-42499)
• Zero changes to reusable workflows — no blast radius to other frameworks

Adding PyTorch 2.12

1. Copy docker/pytorch/2.11/ → docker/pytorch/2.12/, update version pins
2. Create 4 config files (pytorch-2.12-{ec2,sagemaker}-{cuda,cpu}.yml)
3. Autorelease: add one cron line + one case entry per workflow
4. PR workflows: auto-detect from diff — no changes needed

Test plan

• PR workflow detects version from docker/pytorch/2.11/** and pytorch-2.11-* changes
• Configs matrix passes correct metadata per version to reusable workflows
• Unit tests run as separate job per version
• Reusable workflows (vLLM, SGLang, Ray, Base) work unchanged
• DLC_PYTORCH_VERSION build-arg resolves correctly in Dockerfile COPY paths
• Security scan passes (libcap patched)
• Prod image fallback works when build is skipped (test-only change PR)
• Autorelease workflow_dispatch with version input works
• Multi-version PR builds all versions in parallel

[sirutBuasai]

I'm actually curious about this. What's the reason for not combining this with the config file? That way we only have one file to track all the package versions, etc. You dont necessarily to change this in this PR cuz it's out of scope, but I'm curious

[Eren-Jeager123]

This file was added before I implemented the PyTorch 2.11 currency & migration by Junpu. I think maybe those are mainly being used for Dockerfile args, but I don't see any blockers to merge them to config file. Let's discuss this in the future