chore: reorganize evals/ into domain-based folder structure

[adnanrhussain]

Summary

Reorganizes the evals/ folder from a flat, Python-centric structure into a scalable, language-agnostic domain taxonomy. This is a structural change only — no prompt content was modified.

New structure:

evals/
  literacy/
    qualitative-text-complexity/
      examples/        # Python notebooks + prompt modules (temporary)
      grade-level-appropriateness/
      sentence-structure/
      vocabulary/
      conventionality/
      subject-matter-knowledge/
  math/               # placeholder
  utils/              # Makefile, install.sh, run.sh, requirements.txt, .env.example
  CHANGELOG.md
  README.md

📂 View branch

Commits

Commit	Description
f488d16	Reorganize evals/ into domain-based folder structure
43cc6f1	Update READMEs to reflect new evals/ structure
fb872b0	Migrate release-please component from evals/prompts to evals
226b570	Update TypeScript SDK prompt import paths to new evals/ structure
736ff1b	Update CI workflow to trigger on all evals prompt file changes

TODO

• Before merging — push the release-please anchor tag:

  git tag evals-v1.4.0 b17cfb728723d27277fcc4a986d95c6189010cd4
  git push origin evals-v1.4.0

  Without this, release-please treats evals as a brand new component and may walk the full commit history on the next release.

• After the first release PR under the new evals component merges — remove last-release-sha from release-please-config.json in a follow-up commit. If left in, it permanently anchors all future releases to the same SHA.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

Reorganizes the evals/ directory into a domain-based taxonomy (starting with literacy → qualitative text complexity), and updates the TypeScript SDK + release automation to reference the new prompt locations.

Changes:

• Move evaluator prompt .txt files into evals/literacy/qualitative-text-complexity/** and add supporting READMEs/examples.
• Update TypeScript SDK prompt import paths and test docs to use the new evals/ structure.
• Migrate release-please component key from evals/prompts → evals and adjust CI workflow path filters for prompt changes.

Reviewed changes

Copilot reviewed 24 out of 53 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
sdks/typescript/tests/README.md	Updates local-dev test docs to point at new prompt folder path.
sdks/typescript/src/prompts/vocabulary/user.ts	Updates vocabulary user prompt import paths to new evals/ structure.
sdks/typescript/src/prompts/vocabulary/system.ts	Updates vocabulary system prompt import paths to new evals/ structure.
sdks/typescript/src/prompts/vocabulary/background-knowledge.ts	Updates background-knowledge prompt import path.
sdks/typescript/src/prompts/subject-matter-knowledge/index.ts	Updates SMK prompt import paths.
sdks/typescript/src/prompts/sentence-structure/complexity.ts	Updates sentence-structure complexity prompt + rubric import paths.
sdks/typescript/src/prompts/sentence-structure/analysis.ts	Updates sentence-structure analysis prompt import paths.
sdks/typescript/src/prompts/grade-level-appropriateness/index.ts	Updates GLA prompt import paths.
sdks/typescript/src/prompts/conventionality/index.ts	Updates conventionality prompt import paths.
release-please-config.json	Moves release-please package key to evals and adds last-release-sha.
.release-please-manifest.json	Renames manifest entry from evals/prompts to evals.
.github/workflows/test-sdk-typescript.yml	Adjusts workflow path filters to trigger on evals/**/*.txt.
evals/utils/run.sh	Adds a helper script to run the utils Makefile jupyter target.
evals/utils/install.sh	Adds a helper script to run the utils Makefile install target.
evals/utils/Makefile	Adds Makefile targets for venv setup, install, and launching Jupyter.
evals/utils/requirements.txt	Adds Python dependencies for evaluator notebooks/utilities.
evals/utils/.env.example	Adds example env file for API keys.
evals/math/README.md	Adds placeholder README for future math evaluators.
evals/README.md	Updates evals-level README to describe the new structure.
evals/CHANGELOG.md	Renames changelog header and documents the reorg under Unreleased.
evals/init.py	Removes package marker (shifts away from Python-package framing).
evals/prompts/init.py	Removes old prompts package marker.
evals/literacy/qualitative-text-complexity/README.md	Adds README describing prompt modules in the new QTC folder.
evals/literacy/qualitative-text-complexity/vocabulary/background-knowledge.txt	Adds/moves vocabulary background-knowledge prompt.
evals/literacy/qualitative-text-complexity/vocabulary/grades-3-4-system.txt	Adds/moves grades 3–4 vocab system prompt.
evals/literacy/qualitative-text-complexity/vocabulary/grades-3-4-user.txt	Adds/moves grades 3–4 vocab user prompt.
evals/literacy/qualitative-text-complexity/vocabulary/other-grades-system.txt	Adds/moves other-grades vocab system prompt.
evals/literacy/qualitative-text-complexity/vocabulary/other-grades-user.txt	Adds/moves other-grades vocab user prompt (with examples/rubric).
evals/literacy/qualitative-text-complexity/sentence-structure/analysis-system.txt	Adds/moves sentence-structure analysis system prompt.
evals/literacy/qualitative-text-complexity/sentence-structure/analysis-user.txt	Adds/moves sentence-structure analysis user prompt.
evals/literacy/qualitative-text-complexity/sentence-structure/complexity-system.txt	Adds/moves sentence-structure complexity system prompt.
evals/literacy/qualitative-text-complexity/sentence-structure/complexity-user.txt	Adds/moves sentence-structure complexity user prompt.
evals/literacy/qualitative-text-complexity/sentence-structure/rubric-grade-3.txt	Adds/moves grade 3 sentence-structure rubric.
evals/literacy/qualitative-text-complexity/sentence-structure/rubric-grade-4.txt	Adds/moves grade 4 sentence-structure rubric.
evals/literacy/qualitative-text-complexity/sentence-structure/rubric-grades-5-12.txt	Adds/moves grades 5–12 sentence-structure rubric.
evals/literacy/qualitative-text-complexity/grade-level-appropriateness/system.txt	Adds/moves GLA system prompt.
evals/literacy/qualitative-text-complexity/grade-level-appropriateness/user.txt	Adds/moves GLA user prompt.
evals/literacy/qualitative-text-complexity/subject-matter-knowledge/system.txt	Adds/moves SMK system prompt.
evals/literacy/qualitative-text-complexity/subject-matter-knowledge/user.txt	Adds/moves SMK user prompt.
evals/literacy/qualitative-text-complexity/conventionality/system.txt	Adds/moves conventionality system prompt.
evals/literacy/qualitative-text-complexity/conventionality/user.txt	Adds/moves conventionality user prompt.
evals/literacy/qualitative-text-complexity/examples/vocabulary_evaluator.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/text_complexity_combo.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/smk_evaluator.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/sentence_structure_evaluator.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/grade_level_evaluator.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/conventionality_evaluator.ipynb	Updates notebook imports for new examples module layout.
evals/literacy/qualitative-text-complexity/examples/vocab_prompts.py	Adds Python prompt-module for vocab evaluator examples.
evals/literacy/qualitative-text-complexity/examples/smk_prompts.py	Adds Python prompt-module for SMK evaluator examples.
evals/literacy/qualitative-text-complexity/examples/sent_str_prompts.py	Adds Python prompt-module for sentence-structure evaluator examples.
evals/literacy/qualitative-text-complexity/examples/gla_prompts.py	Adds Python prompt-module for GLA evaluator examples.
evals/literacy/qualitative-text-complexity/examples/conventionality_prompts.py	Adds Python prompt-module for conventionality evaluator examples.
README.md	Updates root docs to point at new evals paths and requirements file location.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[czi-fsisenda]

Nice!