TASK 01
Relevance
Does the retrieved passage address the user's query, or is it unrelated to the information needed?
relevant · irrelevant
Research model · RAG evaluation
Evaluate the context.
Not just the answer.
RAGOne is a focused evaluation model for judging whether retrieved context is relevant, sufficient, evidentially strong, or in conflict with a claim.
Four evaluation tasks. One consistent interface. Designed for research and experimentation.
01 / What it evaluates
Four questions that matter in a RAG pipeline.
RAGOne treats retrieved-context evaluation as distinct decisions. Each task has its own label set and can be called directly through the Python package.
TASK 02
Answerability
Can the query be answered using the supplied context, without filling gaps from outside knowledge?
yes · no
TASK 03
Evidence quality
How directly and strongly does the context support the proposition or answer being assessed?
strong · medium · weak
TASK 04
Contradiction
Does a claim conflict with what the supplied context explicitly states?
yes · no
02 / RAGOne examples
RAGOne examples
Choose a task to inspect its query, retrieved context, predicted label, and full JSON score breakdown across every candidate label. RAGOne helps assess whether retrieved evidence supports a generated answer before that answer is accepted.
Answerability
Can the query be answered from the retrieved context?
Query / claim
What is the refund period?
Retrieved context
Customers can request a refund within 30 days.
Predicted label
YESThe context directly states the refund period.
JSON result
{
"task": "answerability",
"prediction": "yes",
"scores": [
{"label": "yes", "score": 0.893767},
{"label": "no", "score": 0.106233}
]
}The answerability scores shown are from one recorded model run. Score values for the other task examples are illustrative JSON examples, not verified inference results. Run the package to obtain scores for your own inputs. Scores rank candidate labels and should not be treated as calibrated probabilities.
03 / Reported results
Promising early results. Clearly scoped.
The figures below summarize the experiments reported for this model. Small evaluation sets are useful for initial checks, but do not establish generalization to every dataset or production setting.
92.5%Separate evaluation set accuracy37 correct / 40 examples
94.591%Recorded validation accuracyAt epoch 5
19.03 msObserved answerability latencySingle recorded inference run
The 40-example evaluation contained 10 examples per task: answerability 10/10, evidence quality 9/10, contradiction 10/10, and relevance 8/10. Latency is environment-dependent; it is not a universal speed guarantee.
04 / Workflow
A small evaluation step between retrieval and generation.
Use RAGOne to inspect retrieved material before a downstream answer is accepted, or as one component in a larger evaluation experiment.
STEP 01
Provide the evidence
Pass the user's query and a retrieved context passage to the model. Keep the source text available for audit.
STEP 02
Choose the task
Select relevance, answerability, evidence quality, or contradiction and provide that task's valid options.
STEP 03
Inspect the result
Review the predicted label and returned scores. Combine the result with your pipeline's own thresholds and validation rules.
05 / Try RAGOne
Run the model in Python.
Install the published package, load the model, and evaluate a query against its retrieved context.
Python · ragone-ai
from ragone import RAGOne
model = RAGOne.from_pretrained("Hariharan05/ragone")
result = model.predict(
query="What is the refund period?",
context="Customers can request a refund within 30 days.",
task="answerability",
options=["yes", "no"],
)
print(result)For reproducible latency measurements, warm up the model and measure multiple calls on the target hardware. Record the device, batch size, input length, and timing method alongside results.