Tag
#Evaluation
3 pieces filed under Evaluation.
Why we do not extract facts by default
A 500-question memory experiment cut reader input tenfold but lost 6.4 points of answer accuracy. The missing details were gone before retrieval.
How we took LongMemEval from 80.0 to 94.2 without touching retrieval
LongMemEval-S climbed from 80.0 to 94.2 while retrieval recall@10 stayed at 0.99. The gains came from evidence budget, reader generation, and instructions.
We retrieved the memory, then dropped the answer
A 200-question conflict run scored 0.800 at the parent-Memory boundary and 0.635 at the evidence boundary. The gap was in what reached the reader.