Skip to content

Speed up E2E test cases#117

Merged
anna-singleton-resolver merged 7 commits intomainfrom
feature/fast-e2e
Mar 9, 2026
Merged

Speed up E2E test cases#117
anna-singleton-resolver merged 7 commits intomainfrom
feature/fast-e2e

Conversation

@anna-singleton-resolver
Copy link
Collaborator

@anna-singleton-resolver anna-singleton-resolver commented Mar 6, 2026

Use a fixture that holds open a streaming connection to speed up the e2e test cases by 4x! (on the benign_model set)

Current Speed of benign_model: ~160s
New Speed of benign_model: ~38s

We could actually go faster (all cases running in ~6s) by using subtests, but then we lose the ability to run single cases.

@anna-singleton-resolver
Copy link
Collaborator Author

#116 shoudl get merged first

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR speeds up the E2E functional classification tests by reusing a single long-lived streaming gRPC connection across cases, and adds an env-var-based selector for which testcase directory to run.

Changes:

  • Added StreamingSender + streaming_sender session fixture to reuse a persistent classify_images stream for E2E runs.
  • Switched the E2E test to use streaming_sender.send(...) and made testcase selection configurable via ATHENA_E2E_TESTCASE_DIR.
  • Documented the new E2E configuration in the README.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File Description
tests/functional/e2e/testcases/parser.py Adds env-var driven testcase loading (ATHENA_E2E_TESTCASE_DIR).
tests/functional/e2e/test_classify_single.py Uses the new session-scoped streaming sender to classify each E2E case faster.
tests/functional/conftest.py Introduces the StreamingSender helper and a session-scoped streaming_sender fixture; modifies athena_options scope and compression quality.
README.md Documents E2E behavior and the ATHENA_E2E_TESTCASE_DIR environment variable.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

@anna-singleton-resolver anna-singleton-resolver merged commit 4cd3aef into main Mar 9, 2026
17 checks passed
@anna-singleton-resolver anna-singleton-resolver deleted the feature/fast-e2e branch March 9, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants