Use the Python client to evaluate test cases, stream events, and run CI gates.
Python SDK
The Python client provides a straightforward way to evaluate test cases, stream
events, and gate builds from Python-based workflows.
Install
pip install quorum-eval
Quick start
from quorum_eval import QuorumClientclient = QuorumClient(api_key="qrm_...")result = client.evaluate( test_cases=[{ "input": "What is the boiling point of water?", "actualOutput": "Water boils at 100 C at sea level.", "retrievalContext": [ "The boiling point of water at standard pressure is 100 C (212 F)." ], }], strategy="auto",)print(f"Pass rate: {result.pass_rate:.1%}")print(f"Cost: ${result.summary.total_cost:.4f}")
Streaming
for event in client.evaluate_stream(test_cases): print(event["event"], event["data"])