feat: Third-party eval metrics adapter (DeepEval + Autoevals) with strands-evals mappers#568

Open

stone-coding wants to merge 20 commits into

stone-coding:feature/third-party-eval-adapter

Commits on Jun 30, 2026

Add DeepEvalHandler integration with unit tests
haomiao037
committed
Fix span extraction to use real AgentCore _eval_log_records structure
haomiao037
committed
Set context field from tool messages for HallucinationMetric support
haomiao037
committed
Use metric.success for label instead of manual threshold comparison
haomiao037
committed
Add model override and timeout enforcement to DeepEvalHandler
haomiao037
committed
Add model override, timeout enforcement, use metric.success, fix SingleTurnParams deprecation
haomiao037
committed
Fix _get_required_params to handle GEval unmappable typing params
haomiao037
committed
Add .deepeval/ to gitignore
haomiao037
committed
Move model override to init to avoid per-call mutation
haomiao037
committed
Refactor to BaseAdapter framework with DeepEval/Autoevals adapters and EvaluatorInput support
haomiao037
committed
Major refactor: move to custom_code_based_evaluators, add span parser layer, simplify per TJ/Irene feedback
haomiao037
committed
Fix review items: add reference_inputs to model, tighten error detection, add validate_fields to DeepEvalAdapter
haomiao037
committed
Adapt to upstream ReferenceInput model: remove duplicate field, use expected_response_text property
haomiao037
committed

Commits on Jul 6, 2026

feat: Add third-party eval metrics adapter (DeepEval + Autoevals)
haomiao037
committed

Commits on Jul 15, 2026

Commits on Jul 16, 2026

Commits on Jul 20, 2026

fix: Address TJ's PR review feedback
haomiao037
committed

Commits on Jul 22, 2026

feat: Add ConversationalTestCase support + remove amazon distro scope
haomiao037
committed