Add token credential param to groundedness (#4668)

ela-ine · ccozianu · svaruag · web-flow · commit de3b50d52e65 · 2025-12-09T15:00:09.000-08:00
* Ccozianu/rm bug fix (#2247) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/rm bug fix (#2248) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/rm bug fix (#2259) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2273) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2275) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2292) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation * Add SystemLog: prefix for scorer * Fixed type error on timeout --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2308) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation * Add SystemLog: prefix for scorer * Fixed type error on timeout * Go to a different endpoint for connections listkeys call in batch_benchmark_score --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2316) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation * Add SystemLog: prefix for scorer * Fixed type error on timeout * Go to a different endpoint for connections listkeys call in batch_benchmark_score * Synced all batch-benchmark-inference patch version --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2409) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation * Add SystemLog: prefix for scorer * Fixed type error on timeout * Go to a different endpoint for connections listkeys call in batch_benchmark_score * Synced all batch-benchmark-inference patch version * use default cred * replaced with managed cred * add more logs * add compute target * remove logs, handle errors better --------- Co-authored-by: svaruag <gasi@microsoft.com> * Ccozianu/4office debug (#2428) * add more logs * fix stdout logs * Make sure we recover details * Make sure we recover details (#2238) * SystemLog: prefix logging * Ccozi/temp logging fix (#2246) * Make sure we recover details * SystemLog: prefix logging * Adding more detailed logging * Fixing typo * fix NPE in logging * Cleaned up somehwat dependence on status definitions * handle folder * Added finetuned_start_timeout_seconds * Fix typo in main invocation * Add SystemLog: prefix for scorer * Fixed type error on timeout * Go to a different endpoint for connections listkeys call in batch_benchmark_score * Synced all batch-benchmark-inference patch version * use default cred * replaced with managed cred * add more logs * add compute target * remove logs, handle errors better * Add notebook to list aoai connections * cleanup notebook test values --------- Co-authored-by: svaruag <gasi@microsoft.com> * add token credential param to groundedness * update ver * fix merge * fix merge * typo --------- Co-authored-by: ccozianu <ccozianu@microsoft.com> Co-authored-by: svaruag <gasi@microsoft.com> Co-authored-by: Elaine Wan <elainewan@microsoft.com>
diff --git a/assets/evaluators/builtin/groundedness/evaluator/_groundedness.py b/assets/evaluators/builtin/groundedness/evaluator/_groundedness.py
@@ -193,6 +193,7 @@ def __init__(self, model_config, *, threshold=3, credential=None, **kwargs):
         )
         self._model_config = model_config
         self.threshold = threshold
+        self._credential = credential
         # Needs to be set because it's used in call method to re-validate prompt if `query` is provided
 
     @overload
@@ -299,7 +300,8 @@ def _ensure_query_prompty_loaded(self):
         self._flow = AsyncPrompty.load(
             source=self._prompty_file,
             model=prompty_model_config,
-            is_reasoning_model=self._is_reasoning_model)
+            is_reasoning_model=self._is_reasoning_model,
+            token_credential=self._credential)
 
     def has_context(self, eval_input: dict) -> bool:
         """
diff --git a/assets/evaluators/builtin/groundedness/spec.yaml b/assets/evaluators/builtin/groundedness/spec.yaml
@@ -1,6 +1,6 @@
 type: "evaluator"
 name: "builtin.groundedness"
-version: 3
+version: 4
 displayName: "Groundedness-Evaluator"
 description: "Assesses whether the response stays true to the given context in a retrieval-augmented generation scenario. It’s best used for retrieval-augmented generation (RAG) scenarios, including question and answering and summarization. Use the groundedness metric when you need to verify that ai-generated responses align with and are validated by the provided context."
 evaluatorType: "builtin"
