Conversation
| # Whether the target natively supports JSON output (e.g., via a "json" response format). | ||
| supports_json_response: bool = False | ||
|
|
||
| # The input modalities supported by the target (e.g., "text", "image"). |
There was a problem hiding this comment.
We also likely want editable_history; This let's us set the system history that are not actually responses from the target. We use PromptChatTarget for this currently.
There was a problem hiding this comment.
you mean targets that inherit from PromptChatTarget have editable history ?
| from the actual model. If not provided, `model_name` will be used for the identifier. | ||
| Defaults to None. | ||
| capabilities (TargetCapabilities, Optional): Override the default capabilities for | ||
| custom_capabilities (TargetCapabilities, Optional): Override the default capabilities for |
There was a problem hiding this comment.
I wonder if we could have a mapping of known default capabilities here, or potentially retrieved from a list in target_capabilites class.
if underlying_model == "gpt-5.1":
_default = X
elif ...
There was a problem hiding this comment.
We also talked about a method to discover these which I think will be useful. But the defaults we know could go a long way early on :)
| SUPPORTED_DATA_TYPES = {"text", "image_path"} | ||
| _DEFAULT_CAPABILITIES: TargetCapabilities = TargetCapabilities(supports_multi_turn=True) | ||
| _DEFAULT_CAPABILITIES: TargetCapabilities = TargetCapabilities( | ||
| supports_multi_turn=True, |
There was a problem hiding this comment.
maybe only minimal since it's generic?
There was a problem hiding this comment.
you mean it only has minimal multi turn support ? as in we should keep the default as false for this one ?
5 participants
Description
This PR builds on #1433 to expand the TargetCapabilities class and consolidate the logic in Targets to use the TargetCapabilities class rather than misc variables. This is the first of at least 2 more PRs which will allow users to query target capabilities and add validation to attacks, converters, scorers, etc which have requirements for targets.
TargetCapabilitiesdataclass — expanded fields:supports_multi_turn(existing)supports_multi_message_pieces— rejects messages with >1 piece whenFalsesupports_json_response— whether JSON response format is supportedinput_modalities— allowed input data types (text, image_path, audio_path, …)output_modalities— produced output data typesAdded
assert_satisfies()to validate oneTargetCapabilitiesagainst another. This will be useful later when validating whether a target satisfies the requirements of an attack / scorer / converterPromptTarget._validate_request()— converted from abstract to a concrete baseimplementation that auto-enforces capabilities:
supports_multi_message_pieces=Falseconverted_value_data_typeagainstinput_modalitiessupports_multi_turn=FalseAll ad-hoc inline validation in individual targets was deleted.
Renamed constructor param
capabilities→custom_capabilities.Added
is_json_response_supported()delegating tocapabilities.supports_json_response.Per-target
_DEFAULT_CAPABILITIESdeclarations added/updated:OpenAIChatTargetOpenAICompletionTargetsupports_multi_message_pieces=FalseOpenAITTSTargetOpenAIImageTargetOpenAIVideoTargetRealtimeTargetPlaywrightTargetPlaywrightCopilotTargetWebSocketCopilotTargetPromptShieldTargetHuggingFaceChatTargetCrucibleTargetTests and Documentation
test_target_capabilities.pycovering modality declarations,assert_satisfies, and per-target defaultstest_supports_multi_turn.pyextended with constructor override testspytest.raiseserror message patterns updated to the new unified formatget_conversation→get_message_piecespatch_central_databasefixture added to HTTP target tests that were missing it