Skip to content

Add standalone prediction and evaluation references - #3

Open
wzh20188 wants to merge 7 commits into
ldbc:mainfrom
wzh20188:agent/add-standalone-prediction-evaluation-reference
Open

Add standalone prediction and evaluation references#3
wzh20188 wants to merge 7 commits into
ldbc:mainfrom
wzh20188:agent/add-standalone-prediction-evaluation-reference

Conversation

@wzh20188

Copy link
Copy Markdown
Contributor

What changed

  • add a standalone OpenAI-compatible prediction script for Cypher, GQL, and SQL
  • add reference execution-accuracy evaluators for Spanner GQL and TuGraph Cypher
  • add a reference grammar, similarity, and Google BLEU wrapper
  • add setup, input-format, security, and usage documentation

Why

These scripts make the experiment workflow easier to reproduce while keeping environment-specific evaluation implementations clearly separated as references.

Security and portability

  • API keys and database passwords are read from environment variables or runtime arguments
  • cloud resource identifiers are supplied at runtime
  • machine-specific absolute paths were removed
  • generated data, credentials, and service-account files are not included

Validation

  • compiled all four Python scripts with python -m py_compile
  • verified --help for every script
  • verified the repository-relative grammar/similarity evaluator path
  • scanned the staged contribution for credential patterns and local absolute paths
  • ran git diff --cached --check

@wzh20188
wzh20188 marked this pull request as ready for review July 15, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant