Evaluations

Fetch evaluations

Fetch evaluations after a webhook, request includes, and reconcile completed evaluations daily.

When to use this

Use the evaluation routes to bring completed evaluations into your own system. Listing them at least daily by completion time brings in every evaluation. To receive each one within minutes, also subscribe to the evaluation.completed webhook and fetch the evaluation it names.

GET /external/v1/evaluations/{evaluation_id} reads one evaluation, usually after an evaluation.completed webhook.
GET /external/v1/evaluations lists evaluations by completion time, to reconcile or to import history.
GET /external/v1/evaluation-definitions reads the definitions and rubrics that describe evaluations. See Evaluation definitions.

The daily reconcile alone brings in every evaluation. The dashed webhook path is optional: it gets each evaluation to you within minutes. Both paths apply the same keep-newest rule.

Read How evaluations work first for the resource and the rule for keeping the newest evaluation per conversation.

Required scopes

Route	Required scopes
`GET /external/v1/evaluations`	`encounters:read`, `patients:read`, `evaluations:read`
`GET /external/v1/evaluations/{evaluation_id}`	`encounters:read`, `patients:read`, `evaluations:read`
`GET /external/v1/evaluation-definitions`	`evaluations:read`

The API key must be organization-scoped. Existing keys don't gain evaluations:read, and keys can't be edited, so create a new key with these three scopes; Get an API key explains how. Evaluation routes don't require documents:read. Add it only if you also fetch transcripts or read encounters and actors directly.

Evaluation routes return route_not_allowed until ChartHero releases them in your environment. Successful evaluation reads are audited as public API disclosures. The definitions route holds no PHI.

Fetch an evaluation after a webhook

The webhook is optional. ChartHero sends evaluation.completed when an evaluation completes, to endpoints your organization admin subscribes to it, as Receive webhooks describes. Like every ChartHero webhook, it is a thin pointer:

{
  "id": "evt_evaluation_completed_01",
  "type": "evaluation.completed",
  "api_version": "2026-05-01",
  "occurred_at": "2026-09-14T15:09:40Z",
  "organization_id": "org_synthetic_001",
  "resources": {
    "evaluation_id": "eval_synthetic_001",
    "encounter_id": "enc_synthetic_001",
    "document_id": "doc_synthetic_transcript_001"
  }
}

occurred_at equals the evaluation's completed_at. To handle it:

Verify the signature and dedupe by event id.
Fetch the evaluation named by resources.evaluation_id, with the includes you need to map it to your records.
Keep the newest evaluation per (document_id, evaluated_actor_id, definition_id). An older evaluation can complete after the one that replaced it, and it still sends an event.

curl "https://api.charthero.ai/external/v1/evaluations/eval_synthetic_001?include=encounters,patients,actors,evaluation_definitions" \
  -H "Authorization: Bearer <api_key>"

Leave out evaluation_definitions if you cache definitions by definition_id. Webhook delivery has one attempt and no retries, so also reconcile daily.

Includes

Evaluation routes accept include with any of these values, comma-separated:

`include` value	Returns
`encounters`	The evaluations' encounters, with `external_ids` such as your visit ID.
`patients`	The encounters' patients, with `external_ids` such as your MRN.
`actors`	The evaluated people, with their provider `external_ids`, while they have a membership in the organization. With `encounters`, also the people those encounters reference.
`evaluation_definitions`	Each definition the evaluations reference, with its rubrics, exactly as `GET /external/v1/evaluation-definitions` lists it.

included holds only the families you request, de-duplicated, and is omitted when empty. On the list route, encounters, patients or actors lowers the default and maximum limit to 25. evaluation_definitions reads no records, so on its own it keeps the standard limits.

There is no documents include. Fetch the conversation's transcript from document detail with a key that has documents:read.

Response

Evaluation detail with record includes:

{
  "evaluation": {
    "evaluation_id": "eval_synthetic_001",
    "evaluation_type": "care_coordination_conversation",
    "definition_id": "6f102fa4-ad28-4f20-b86d-b944ee32d7eb",
    "definition_version": 1,
    "rubric_id": "81cb28c6-2c30-4473-acf4-1ff258030470",
    "rubric_version": 1,
    "evaluated_role": "care_coordinator",
    "result_schema": "conversation_evaluation.v1",
    "organization_id": "org_synthetic_001",
    "encounter_id": "enc_synthetic_001",
    "activity_id": "act_synthetic_call_001",
    "document_id": "doc_synthetic_transcript_001",
    "workflow_variant": "phone_call",
    "evaluated_actor_id": "11111111-1111-4111-8111-111111111111",
    "conversation_occurred_at": "2026-09-14T15:02:11Z",
    "conversation_duration_seconds": 312.4,
    "completed_at": "2026-09-14T15:09:40Z",
    "created_at": "2026-09-14T15:08:05Z",
    "updated_at": "2026-09-14T15:09:40Z",
    "result": {
      "summary": "Synthetic post-discharge follow-up call.",
      "profile": {
        "remote_party_role": "patient",
        "call_context": "post_discharge_follow_up",
        "dominant_intent": "clinical_status_check",
        "clinical_complexity_level": "moderate",
        "care_navigation_complexity_level": "low",
        "interaction_complexity_level": "low",
        "overall_complexity_level": "moderate",
        "risk_level": "active_signal",
        "transcript_extractability_quality": "good"
      },
      "total_points": 1,
      "scored_dimension_count": 1,
      "minimum_possible_points": -2,
      "maximum_possible_points": 2,
      "dimensions": [
        {
          "key": "safety_risk_recognition_and_escalation",
          "points": 1,
          "applicability": "highly_applicable",
          "rationale": "Synthetic rationale.",
          "finding_ids": ["f0000000-0000-4000-8000-000000000001"]
        }
      ],
      "findings": [
        {
          "finding_id": "f0000000-0000-4000-8000-000000000001",
          "level": "positive_strong",
          "primary_category": "clinical_attentiveness_symptom_drill_down_risk_recognition",
          "attribution": "evaluated_participant_behavior",
          "review_markers": ["safety"],
          "practice_element": "acuity_and_red_flag_screening",
          "focus_area": "health_status_symptoms_and_clinical_risk",
          "issue_type": "no_issue_described",
          "description": "Synthetic description.",
          "rationale": "Synthetic rationale.",
          "evidence": "Synthetic transcript excerpt.",
          "primary_evidence_range": {"start_seconds": 41.2, "end_seconds": 55.9},
          "supporting_evidence_ranges": []
        }
      ]
    }
  },
  "included": {
    "encounters": [
      {
        "encounter_id": "enc_synthetic_001",
        "organization_id": "org_synthetic_001",
        "external_ids": [
          {"source": "partner_ehr", "type": "appointment_id", "value": "visit-12345"}
        ],
        "patient_id": "pat_synthetic_001",
        "activity_ids": ["act_synthetic_call_001"],
        "document_ids": ["doc_synthetic_transcript_001", "doc_synthetic_note_001"],
        "date_of_service": "2026-09-14T15:00:00Z",
        "status": "completed",
        "encounter_type": "phone_call",
        "created_at": "2026-09-14T14:58:00Z",
        "updated_at": "2026-09-14T15:08:30Z"
      }
    ],
    "patients": [
      {
        "patient_id": "pat_synthetic_001",
        "external_ids": [
          {"source": "partner_ehr", "type": "patient_id", "value": "patient-98765"}
        ],
        "first_name": "Synthetic",
        "last_name": "Patient",
        "date_of_birth": "1900-01-01"
      }
    ],
    "actors": [
      {
        "actor_type": "user",
        "actor_id": "11111111-1111-4111-8111-111111111111",
        "external_ids": [
          {"source": "partner_ehr", "type": "provider_id", "value": "cc-778"}
        ],
        "display_name": "Sam Coordinator",
        "first_name": "Sam",
        "last_name": "Coordinator",
        "active": true
      }
    ]
  }
}

The example is abbreviated to one dimension and one finding. A care-coordination evaluation lists all seven dimensions and can have any number of findings.

List evaluations

GET /external/v1/evaluations returns your organization's evaluations by completion time, including any that a newer evaluation of the same conversation replaced. Its only filter is a completion-time window.

Query parameter	Default	Contract
`limit`	`50`, or `25` with a record include	Canonical decimal integer from `1` through `100`, or through `25` when `include` names `encounters`, `patients` or `actors`.
`offset`	`0`	Canonical decimal integer from `0` through `10000`.
`sort_order`	`desc`	`asc` or `desc`, by `completed_at`, then `evaluation_id`.
`start_at`	none	Timezone-aware ISO datetime. Returns evaluations with `completed_at` at or after it.
`end_at`	none	Timezone-aware ISO datetime. Returns evaluations with `completed_at` before it. Must be later than `start_at`.
`include`	none	`encounters`, `patients`, `actors`, `evaluation_definitions`, comma-separated.

Each parameter can appear at most once. The window filters completion time, not conversation time; to report by when conversations happened, select by conversation_occurred_at from the evaluations you've stored. Any other query key, including encounter_id, returns unsupported_filter.

An evaluation never changes after it completes, so the completion-time window works like an updated-since cursor: each evaluation appears in the window its completion time falls in.

The response is { "items": [...], "limit", "offset", "included" }. There is no total: keep requesting pages until one has fewer than limit items. Offset pages are not a snapshot. With desc, an evaluation completing between requests shifts later pages. With asc and a closed window, new completions can't, so reconcile with asc and dedupe by evaluation_id. A rare data correction made while you page can still shift later pages and skip an evaluation; the periodic re-list catches it.

A list page with an unscored evaluation, abbreviated to one dimension:

{
  "items": [
    {
      "evaluation_id": "eval_synthetic_002",
      "evaluation_type": "care_coordination_conversation",
      "definition_id": "6f102fa4-ad28-4f20-b86d-b944ee32d7eb",
      "definition_version": 1,
      "rubric_id": "81cb28c6-2c30-4473-acf4-1ff258030470",
      "rubric_version": 1,
      "evaluated_role": "care_coordinator",
      "result_schema": "conversation_evaluation.v1",
      "organization_id": "org_synthetic_001",
      "encounter_id": "enc_synthetic_002",
      "activity_id": "act_synthetic_call_002",
      "document_id": "doc_synthetic_transcript_002",
      "workflow_variant": "phone_call",
      "evaluated_actor_id": "11111111-1111-4111-8111-111111111111",
      "conversation_occurred_at": "2026-09-14T16:20:00Z",
      "conversation_duration_seconds": null,
      "completed_at": "2026-09-14T16:24:12Z",
      "created_at": "2026-09-14T16:22:40Z",
      "updated_at": "2026-09-14T16:24:12Z",
      "result": {
        "summary": "Synthetic call that reached a voicemail system.",
        "profile": {
          "remote_party_role": "unclear",
          "call_context": "unclear",
          "dominant_intent": "unclear",
          "clinical_complexity_level": "unclear",
          "care_navigation_complexity_level": "unclear",
          "interaction_complexity_level": "unclear",
          "overall_complexity_level": "unclear",
          "risk_level": "unclear",
          "transcript_extractability_quality": "machine_or_ivr"
        },
        "total_points": null,
        "scored_dimension_count": 0,
        "minimum_possible_points": null,
        "maximum_possible_points": null,
        "dimensions": [
          {
            "key": "safety_risk_recognition_and_escalation",
            "points": null,
            "applicability": "insufficient_evidence",
            "rationale": "Synthetic rationale.",
            "finding_ids": []
          }
        ],
        "findings": []
      }
    }
  ],
  "limit": 50,
  "offset": 0
}

Reconcile daily

Reconcile at least daily. Without the webhook, this is how evaluations reach you. With it, reconciling catches any delivery the webhook missed, because delivery has one attempt and no retries:

Request GET /external/v1/evaluations?sort_order=asc&start_at=<previous window's end minus 1 hour>&end_at=<now minus 5 minutes>, with the includes you need to map evaluations to your records.
Page by offset until a page has fewer than limit items.
Dedupe by evaluation_id, then keep the newest per (document_id, evaluated_actor_id, definition_id).
Update your copies of the included records.
Store the window's end_at as the next window's starting point.

On the first run there is no previous window: import history up to now minus 5 minutes, and use that as the first starting point.

An evaluation's completion time is recorded just before it becomes readable, occasionally up to a couple of minutes before. The 5-minute lag and the one-hour overlap leave room for that. Refresh your cached definitions when you reconcile, too.

curl "https://api.charthero.ai/external/v1/evaluations?sort_order=asc&start_at=2026-09-14T23:00:00Z&end_at=2026-09-15T23:55:00Z&include=encounters,patients,actors&limit=25&offset=0" \
  -H "Authorization: Bearer <api_key>"

Catch late and removed evaluations

Rarely, a ChartHero data correction changes whether an evaluation is returned, as How evaluations work describes: it stops being returned, or it is first returned after you reconciled the window its completion time falls in. A correction is the only way an evaluation arrives later than the 5-minute lag allows, and it sends no event. A correction made while you are listing can also shift later pages, so a listing can skip an evaluation that is still returned. To catch corrections, list a longer past period again on a slower schedule, such as the last 90 days once a week:

List the period with the windows and paging described in Import history.
Only if every page of the period came back, fetch each stored evaluation that completed in the period but wasn't returned, with GET /external/v1/evaluations/{evaluation_id}. Mark it removed only if that returns 404 with evaluation_not_found, and leave it out of your counts. If it returns the evaluation, the listing skipped it, so keep it. On any other error, leave it unmarked and check it at the next re-list. Mark evaluations rather than delete them: if the correction is reversed, the evaluation is returned again, unchanged.
Add the evaluations you don't have, clear the mark on any returned again, and keep the newest per (document_id, evaluated_actor_id, definition_id) again.

Corrections have no time limit, so a correction to an evaluation older than the period is caught only when you list that period again. To catch every one, list your full history again occasionally, such as quarterly.

Import history

New webhook subscriptions receive only new events. To bring existing evaluations into your system, list them in closed completion-time windows; monthly windows are a starting size. Choose windows that each hold fewer than 10,000 evaluations, such as weeks for a large organization.

offset can't exceed 10000. If the next page of a window would need a larger offset, split that window into smaller adjacent windows, restart each at offset 0, and dedupe what you already fetched. Keep each boundary inclusive at the start and exclusive at the end. Mark a window complete only when you reach a short page. If a window's timestamps can't be divided further, report the pagination limit instead of treating the import as complete. The code below splits windows for you; with windows sized as above, it rarely needs to.

Send one request at a time, at most about one per second, as Errors describes. The code below pauses between requests, retries a 503, and waits out a firewall block once before giving up.

import time
from datetime import datetime, timedelta, timezone

import requests

BASE_URL = "https://api.charthero.ai/external/v1"
API_KEY = "<api_key>"
INCLUDE = "encounters,patients,actors"
LIMIT = 25  # The maximum with a record include.
MAX_OFFSET = 10_000
SMALLEST_WINDOW = timedelta(milliseconds=1)
RECORD_IDS = {"encounters": "encounter_id", "patients": "patient_id", "actors": "actor_id"}


def is_firewall_block(response):
    """A 403 without the standard error body comes from the firewall, usually the rate limit."""
    if response.status_code != 403:
        return False
    try:
        return "reason_code" not in response.json()
    except ValueError:
        return True


def get_page(start_at, end_at, offset):
    waited_for_firewall = False
    for attempt in range(5):
        time.sleep(1)  # At most about one request per second.
        response = requests.get(
            f"{BASE_URL}/evaluations",
            params={
                "sort_order": "asc",
                "start_at": start_at.isoformat(),
                "end_at": end_at.isoformat(),
                "include": INCLUDE,
                "limit": LIMIT,
                "offset": offset,
            },
            headers={"Authorization": f"Bearer {API_KEY}"},
            timeout=30,
        )
        if is_firewall_block(response) and not waited_for_firewall:
            waited_for_firewall = True
            time.sleep(300)  # A rate limit clears; a block that continues needs support.
        elif response.status_code == 503:
            time.sleep(10 * 2**attempt)
        else:
            break
    response.raise_for_status()
    return response.json()


def fetch_window(start_at, end_at, found, records):
    """Add every evaluation completed in [start_at, end_at) to found, keyed by evaluation_id,
    and the included records to records, keyed by family and ID."""
    offset = 0
    while True:
        page = get_page(start_at, end_at, offset)
        for evaluation in page["items"]:
            found[evaluation["evaluation_id"]] = evaluation
        for family, id_field in RECORD_IDS.items():
            for record in page.get("included", {}).get(family, []):
                records[family][record[id_field]] = record  # The latest copy wins.
        if len(page["items"]) < LIMIT:
            return  # A short page completes the window.
        offset += LIMIT
        if offset > MAX_OFFSET:
            if end_at - start_at <= SMALLEST_WINDOW:
                raise RuntimeError(f"pagination limit reached for [{start_at}, {end_at})")
            middle = start_at + (end_at - start_at) / 2
            fetch_window(start_at, middle, found, records)
            fetch_window(middle, end_at, found, records)
            return


def reconcile(previous_end):
    end_at = datetime.now(timezone.utc) - timedelta(minutes=5)
    found = {}
    records = {family: {} for family in RECORD_IDS}
    fetch_window(previous_end - timedelta(hours=1), end_at, found, records)
    # Apply keep-newest to found, update your copies from records, and store end_at.
    return found, records, end_at

A 503 with evaluation_read_unavailable means the list query timed out or a record changed while includes were loading. ChartHero never returns a partial page; retry the same request with backoff, as the code does.

Common errors

HTTP status	Reason code	What it means
`400`	`unsupported_filter`	The request has a query key or `include` value the route doesn't support.
`401`	`missing_bearer`, `api_keys_disabled`, `test_prefix_not_allowed`, `malformed_token`, `invalid_key`, `disabled_key`, `revoked_key`, or `expired_key`	API-key authentication failed. Branch on the exact `reason_code`.
`403`	`org_scope_only_required`	Organization-scoped API keys are required.
`403`	`missing_scope`	The API key does not include every required scope. Evaluation routes need `encounters:read`, `patients:read` and `evaluations:read`; a key with only `evaluations:read` can read definitions but not evaluations.
`403`	`route_not_allowed`	Evaluation routes are not enabled.
`403`	`cross_organization_denied` or `demo_training_denied`	The evaluation belongs to another organization or is a demo or training record.
`404`	`evaluation_not_found`	No completed evaluation with that ID is available, including one whose encounter was removed or moved to another organization.
`422`	`invalid_limit`, `invalid_offset`, `invalid_sort`, or `invalid_query`	A parameter is invalid, a datetime is malformed or timezone-naive, the window is reversed, or a parameter is repeated.
`503`	`digest_key_unavailable` or `evaluation_read_unavailable`	Retry with backoff.

Authentication, route availability, organization-scope and scope checks run before query validation, and query validation runs before the evaluation is looked up, so a bad query reveals nothing about the evaluation it names.

Fetch evaluations

When to use this

Required scopes

Fetch an evaluation after a webhook

Includes

Response

List evaluations

Reconcile daily

Catch late and removed evaluations

Import history

Common errors

Next steps

.css-1hnz6hu{position:static;}.css-1hnz6hu::before{content:'';cursor:inherit;display:block;position:absolute;top:0px;left:0px;z-index:0;width:100%;height:100%;}Verify webhook signatures

Evaluation definitions

Review errors

Verify webhook signatures