Cedar quality

For: Cypress Managers and platform admins (Admin › Cedar quality)

Grove, Cypress's team app, runs Cedar for each organization that connects it. Two things come back to Cypress from Grove: how Cedar scores in its benchmarks, and the answers that were flagged for a person to look at. This page is where you read the first and judge the second. What you decide goes back to Grove, which teaches Cedar.

Benchmarks

Each time Grove's team runs the benchmark (ten to twenty real questions for each model setting), the run lands here. A judge model grades every answer: faithful to what Cedar read, correct, and useful, and Grove times each one.

• The latest run at the top: the setting, when it ran, the share of answers that were right, the time where the slowest tenth of answers starts, and whether it passed its gates (the bars a setting has to clear).
• Two trends: the share right, run after run, and the slowest-tenth time in seconds. Faster and more right is better; a setting that gets faster by getting less right is not.
• Settings compared: the latest run of each setting side by side, the most right first and, when two tie, the faster one.
• Click a run to see every question: what Cedar answered, its marks, any error and how long it took.

Benchmarks usually run in a sandbox, so the organization shown is a sandbox.

Audit

Grove sends an answer here when one of its live checks fails (the answer was too long, said something it had not read, or made a claim it should not), when a person corrected Cedar, when the judge scored it low, or at random. Each card shows the organization, the question, Cedar's answer and what it read to answer, in plain words.

For each one, decide:

• Right: the answer was right.
• Wrong: the answer was wrong. Say what the right answer was in the note: it is required, and it is what Cedar learns.
• Dismiss: nothing to learn (a test, a question that made no sense).

And, for Right or Wrong:

• Teach this org: Grove turns it into a lesson for that organization's Cedar.
• Add to benchmark: Grove adds the question to the benchmark, so every future setting is measured against it.

A verdict is final, and it is recorded in the organization's audit trail with your name. Decided answers are listed below the queue. Filter the queue by organization or by why it was flagged.

Business rules

• Only Grove (an app Cypress made) sends benchmarks and flagged answers, and only for the organization it is connected to. Only Cypress staff give verdicts. Grove reads back the verdicts for that organization alone.
• An answer already decided never reopens, even if Grove flags it again.
• Developers: the endpoints are in Agent quality.