core: cache the tool search handler per session

[mchen-oai]

Why

Tool router construction rebuilds the deferred-tool BM25 index during session initialization and before each sampling continuation, even when the searchable tool metadata is unchanged. Local profiling measured append_tool_search_executor at roughly 113 ms per continuation, making repeated index construction the largest measured router-building cost.

What changed

• Add a session-scoped ToolSearchHandlerCache so continuations and user turns can reuse the existing handler.
• Key reuse on the complete ordered Vec<ToolSearchInfo>, rebuilding when searchable text, loadable tool specs, source metadata, or ordering changes.
• Build handlers outside the cache lock and recheck before publishing them, avoiding holding the mutex during index construction.

Verification

• cache_reuses_identical_search_infos_and_rebuilds_changed_inputs covers exact cache reuse and invalidation when the ordered search metadata changes.
• Local rollout profiling showed the initial router build populating the cache and unchanged later continuations reusing it:
  • uncached: 118 ms median across 14 spans from 3 rollouts
  • cached: 4 ms median across 12 spans from 3 rollouts

[mzeng-openai]

Can we use a fingerprint instead of storing a whole copy of the original search_infos?

[mchen-oai]

@mzeng-openai - To my understanding, you will drive broader tool caching (Slack). The broader approach may eliminate the need for this PR, unless it caches at multiple levels. What do you think though and what is your timeline? Are you thinking I proceed with this PR for initial improvement?

[mzeng-openai]

I'll do some research and let you know. In the meanwhile this seems like a viable interim solution to unblock the launch since basically half of the latency comes from BM25 index. But would love to get thoughts from core agent folks

[mchen-oai]

Updated this so ToolSearchHandler owns the original ordered search_infos, and the cache compares directly against the handler's actual inputs. This removes the search_infos copy and avoids a separate derived cache key. Still open to the fingerprint approach if others feel strongly though.