> ## Documentation Index
> Fetch the complete documentation index at: https://docs.medera.info/llms.txt
> Use this file to discover all available pages before exploring further.

# Co-Therapy Agent

> Silent multimodal in-session clinical intelligence.

<div style={{ display: "inline-flex", alignItems: "center", justifyContent: "center", width: 76, height: 76, border: "1px solid rgba(20,20,19,0.08)", background: "#ffffff", marginBottom: 16 }}>
  <svg width="42" height="42" viewBox="0 0 16 16" shapeRendering="crispEdges" xmlns="http://www.w3.org/2000/svg" aria-hidden="true">
    <rect x="3" y="4" width="2" height="1" fill="#141413" />

    <rect x="2" y="5" width="1" height="5" fill="#141413" />

    <rect x="3" y="10" width="1" height="1" fill="#141413" />

    <rect x="4" y="11" width="2" height="1" fill="#141413" />

    <rect x="11" y="4" width="2" height="1" fill="#141413" />

    <rect x="13" y="5" width="1" height="5" fill="#141413" />

    <rect x="12" y="10" width="1" height="1" fill="#141413" />

    <rect x="10" y="11" width="2" height="1" fill="#141413" />

    <rect x="5" y="3" width="2" height="1" fill="#141413" />

    <rect x="9" y="3" width="2" height="1" fill="#141413" />

    <rect x="7" y="2" width="2" height="1" fill="#141413" />

    <rect x="7" y="4" width="2" height="7" fill="#76a1fc" />

    <rect x="6" y="12" width="4" height="1" fill="#141413" />

    <rect x="7" y="13" width="2" height="1" fill="#141413" />
  </svg>
</div>

<Info>
  **Category:** Clinical Operations  ·  **Channels:** Vocal acoustics · Facial physiology · Linguistic content · Assessments
</Info>

The Co-Therapy Agent is a silent multimodal partner — it **never** speaks to the patient. It runs the [Vocal Acoustic Engine](/multimodal/vocal/overview) and [Facial Physiological Engine](/multimodal/facial/overview), fuses outputs through the [Neurobehavioral Construct Computer](/multimodal/rdoc/overview) into 15 named construct activations, and surfaces the resulting canonical payload to the clinician on a private channel.

## Pipeline

<Frame>
  <svg viewBox="0 0 720 480" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="medera/cotherapy.svg" style={{ width: '100%', height: 'auto', display: 'block', maxWidth: '780px', margin: '24px auto' }}><defs><pattern id="mdr-dots" x="0" y="0" width="10" height="10" patternUnits="userSpaceOnUse"><circle cx="1" cy="1" r="1" fill="rgba(0,0,0,0.08)" /></pattern></defs><rect x="0" y="0" width="720" height="480" fill="#ffffff" /><rect x="0" y="0" width="720" height="32" fill="#141413" /><circle cx="14" cy="16" r="3" fill="#ff5f57" /><circle cx="26" cy="16" r="3" fill="#febc2e" /><circle cx="38" cy="16" r="3" fill="#28c840" /><text x="54" y="20" fill="#9a9893" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="500">medera/cotherapy.svg</text><text x="706" y="20" textAnchor="end" fill="#9a9893" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">multimodal fusion</text><rect x="0" y="32" width="720" height="448" fill="#fafaf6" /><rect x="0" y="32" width="720" height="448" fill="url(#mdr-dots)" /><rect x="0.5" y="0.5" width="719" height="479" fill="none" stroke="#141413" strokeOpacity="0.18" /><text x="360.0" y="56" textAnchor="middle" fill="#141413" fontFamily="Inter, sans-serif" fontSize="15" fontWeight="600">Multimodal co-therapy pipeline</text><rect x="48.000000000000014" y="96" width="197.33333333333334" height="130" fill="#ffffff" stroke="rgba(0,0,0,0.08)" strokeWidth="1" /><rect x="143.07" y="111.60" width="7.20" height="1.80" fill="#141413" /><rect x="141.27" y="113.40" width="1.80" height="9.00" fill="#141413" /><rect x="150.27" y="113.40" width="1.80" height="9.00" fill="#141413" /><rect x="143.07" y="122.40" width="7.20" height="1.80" fill="#141413" /><rect x="144.87" y="115.20" width="3.60" height="5.40" fill="#76a1fc" /><rect x="139.47" y="118.80" width="1.80" height="3.60" fill="#141413" /><rect x="152.07" y="118.80" width="1.80" height="3.60" fill="#141413" /><rect x="141.27" y="124.20" width="10.80" height="1.80" fill="#141413" /><rect x="144.87" y="126.00" width="3.60" height="3.60" fill="#141413" /><rect x="141.27" y="129.60" width="10.80" height="1.80" fill="#141413" /><rect x="155.67" y="117.00" width="1.80" height="1.80" fill="#76a1fc" /><rect x="157.47" y="115.20" width="1.80" height="5.40" fill="#76a1fc" /><text x="146.66666666666669" y="154" textAnchor="middle" fill="#141413" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Audio · 16 kHz</text><text x="146.66666666666669" y="174" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">F0 · jitter · shimmer</text><text x="146.66666666666669" y="190" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">HNR · prosodic flatness</text><line x1="146.66666666666669" y1="230" x2="146.66666666666669" y2="246" stroke="rgba(20,20,19,0.40)" strokeWidth="1.5" /><polygon points="142.66666666666669,246 150.66666666666669,246 146.66666666666669,252" fill="rgba(20,20,19,0.25)" /><rect x="261.3333333333333" y="96" width="197.33333333333334" height="130" fill="#ffffff" stroke="rgba(0,0,0,0.08)" strokeWidth="1" /><rect x="354.60" y="113.40" width="3.60" height="1.80" fill="#141413" /><rect x="361.80" y="113.40" width="3.60" height="1.80" fill="#141413" /><rect x="351.00" y="115.20" width="3.60" height="1.80" fill="#141413" /><rect x="365.40" y="115.20" width="3.60" height="1.80" fill="#141413" /><rect x="349.20" y="117.00" width="1.80" height="9.00" fill="#141413" /><rect x="369.00" y="117.00" width="1.80" height="9.00" fill="#141413" /><rect x="358.20" y="111.60" width="3.60" height="1.80" fill="#141413" /><rect x="358.20" y="115.20" width="3.60" height="1.80" fill="#76a1fc" /><rect x="358.20" y="117.00" width="3.60" height="1.80" fill="#76a1fc" /><rect x="358.20" y="118.80" width="3.60" height="1.80" fill="#76a1fc" /><rect x="358.20" y="120.60" width="3.60" height="1.80" fill="#76a1fc" /><rect x="358.20" y="122.40" width="3.60" height="1.80" fill="#76a1fc" /><rect x="358.20" y="124.20" width="3.60" height="1.80" fill="#76a1fc" /><rect x="351.00" y="126.00" width="1.80" height="1.80" fill="#141413" /><rect x="367.20" y="126.00" width="1.80" height="1.80" fill="#141413" /><rect x="352.80" y="127.80" width="3.60" height="1.80" fill="#141413" /><rect x="363.60" y="127.80" width="3.60" height="1.80" fill="#141413" /><rect x="356.40" y="129.60" width="7.20" height="1.80" fill="#141413" /><rect x="358.20" y="131.40" width="3.60" height="1.80" fill="#141413" /><text x="360.0" y="154" textAnchor="middle" fill="#141413" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Video · 15 fps</text><text x="360.0" y="174" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">MediaPipe 468-point</text><text x="360.0" y="190" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">rPPG · HR · HRV · RR</text><line x1="360.0" y1="230" x2="360.0" y2="246" stroke="rgba(20,20,19,0.40)" strokeWidth="1.5" /><polygon points="356.0,246 364.0,246 360.0,252" fill="rgba(20,20,19,0.25)" /><rect x="474.6666666666667" y="96" width="197.33333333333334" height="130" fill="#ffffff" stroke="rgba(0,0,0,0.08)" strokeWidth="1" /><rect x="564.33" y="111.60" width="14.40" height="1.80" fill="#141413" /><rect x="564.33" y="111.60" width="1.80" height="19.80" fill="#141413" /><rect x="564.33" y="131.40" width="16.20" height="1.80" fill="#141413" /><rect x="578.73" y="115.20" width="1.80" height="16.20" fill="#141413" /><rect x="576.93" y="111.60" width="1.80" height="1.80" fill="#141413" /><rect x="578.73" y="113.40" width="1.80" height="3.60" fill="#141413" /><rect x="567.93" y="118.80" width="9.00" height="1.80" fill="#76a1fc" /><rect x="567.93" y="122.40" width="9.00" height="1.80" fill="#76a1fc" /><rect x="567.93" y="126.00" width="5.40" height="1.80" fill="#76a1fc" /><text x="573.3333333333334" y="154" textAnchor="middle" fill="#141413" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Language</text><text x="573.3333333333334" y="174" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">Transcript · facts</text><text x="573.3333333333334" y="190" textAnchor="middle" fill="#5c5c5a" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">linguistic features</text><line x1="573.3333333333334" y1="230" x2="573.3333333333334" y2="246" stroke="rgba(20,20,19,0.40)" strokeWidth="1.5" /><polygon points="569.3333333333334,246 577.3333333333334,246 573.3333333333334,252" fill="rgba(20,20,19,0.25)" /><rect x="60" y="246" width="600" height="50" fill="#141413" stroke="none" strokeWidth="1" /><text x="360.0" y="268" textAnchor="middle" fill="#ffffff" fontFamily="Inter, sans-serif" fontSize="13" fontWeight="600">Neurobehavioral Construct Computer</text><text x="360.0" y="286" textAnchor="middle" fill="#dbe5ff" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="10">15 constructs · 5 domains</text><line x1="360.0" y1="300" x2="360.0" y2="316" stroke="rgba(20,20,19,0.40)" strokeWidth="1.5" /><polygon points="356.0,316 364.0,316 360.0,322" fill="rgba(20,20,19,0.25)" /><rect x="99.5" y="332.0" width="121" height="32" fill="#ffffff" stroke="rgba(0,0,0,0.08)" strokeWidth="1" /><text x="160.0" y="352.8" textAnchor="middle" fill="#141413" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Domain scores</text><rect x="296.0" y="332.0" width="128" height="32" fill="#76a1fc" stroke="none" strokeWidth="1" /><text x="360.0" y="352.8" textAnchor="middle" fill="#ffffff" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Crisis markers</text><rect x="492.5" y="332.0" width="135" height="32" fill="#ffffff" stroke="rgba(0,0,0,0.08)" strokeWidth="1" /><text x="560.0" y="352.8" textAnchor="middle" fill="#141413" fontFamily="ui-monospace, SFMono-Regular, Menlo, monospace" fontSize="11" fontWeight="600">Structured emit</text></svg>
</Frame>

## Endpoints

| Method | Path                                         | Purpose                     |
| :----- | :------------------------------------------- | :-------------------------- |
| `POST` | `/api/co-therapy-sessions`                   | Create session (backend)    |
| `POST` | `/api/co-therapy-sessions/{id}/start`        | Start session + invitations |
| `POST` | `/api/co-therapy-sessions/{id}/join`         | Record participant joining  |
| `POST` | `/api/co-therapy-sessions/{id}/leave`        | Record participant leaving  |
| `POST` | `/api/co-therapy-sessions/{id}/transition`   | Manually transition state   |
| `POST` | `/api/multimodal-therapy/analyze-session`    | Full multimodal analysis    |
| `POST` | `/api/multimodal-therapy/analyze-audio-only` | Audio-only analysis         |

## Configuration

| Field                            | Description                                               |
| :------------------------------- | :-------------------------------------------------------- |
| `audio_enabled`, `video_enabled` | Channel toggles                                           |
| `rdoc_constructs`                | Optional subset of the 15 named constructs                |
| `clinical_context`               | Chief complaint, active diagnoses, recent screener scores |
| `consent_recorded`               | Per-session consent flag (required)                       |

## Composable experts

| Expert                               | Role                                                         |
| :----------------------------------- | :----------------------------------------------------------- |
| **Vocal Prosody Expert**             | F0, jitter, shimmer, HNR, speaking rate, prosodic flatness   |
| **Affect Dynamics Expert**           | Facial affect intensity, valence, congruence                 |
| **Neurobehavioral Construct Expert** | Neurobehavioral Construct activation, severity, contributors |
| **Linguistic Content Expert**        | Topic, valence, certainty, pronoun shift                     |

## Quality gates

* **Audio**: ≥ 1.0 s, valid `voiced_fraction`

* **Video**: ≥ 30 s for HR, ≥ 60 s for HRV / BP, SNR > 3.0

* **Analyzer**: minimum 20 words of transcript + 30 s of session signal On quality failure, the engine returns `confidence: 0.0` and the analyzer falls back to a Medera reasoning transcript-inferred analysis marked `requires_human_review: true` — **never fabricated values**.

## Compliance guarantees

<Warning>
  * The Co-Therapy Agent **never** addresses or interacts with the patient. - Recording requires explicit per-session consent (`consent_recorded: true`). - Construct activations report `contributors[]` so clinicians can audit feature-level influence.
</Warning>

***

## Related

<CardGroup cols={2}>
  <Card title="Quickstart" icon="https://mintcdn.com/medera-357fd587/n1mgcc3nR2sq_PoL/icons/ui/zap.svg?fit=max&auto=format&n=n1mgcc3nR2sq_PoL&q=85&s=279c57d41af922abbba25c4afddfc3fe" href="/quickstart/co-therapy" width="40" height="40" data-path="icons/ui/zap.svg">
    Run a multimodal session.
  </Card>

  <Card title="Multimodal Overview" icon="https://mintcdn.com/medera-357fd587/iC-6rAuwl-0aQSw_/icons/ui/audio-lines.svg?fit=max&auto=format&n=iC-6rAuwl-0aQSw_&q=85&s=a4f3136aa8df6fe2d044778c71ba2bbc" href="/multimodal/overview" width="40" height="40" data-path="icons/ui/audio-lines.svg">
    Engine deep dive.
  </Card>

  <Card title="Neurobehavioral Construct Constructs" icon="https://mintcdn.com/medera-357fd587/iC-6rAuwl-0aQSw_/icons/ui/brain.svg?fit=max&auto=format&n=iC-6rAuwl-0aQSw_&q=85&s=0fdc19884888b6cfb111dd1490535243" href="/multimodal/rdoc/overview" width="40" height="40" data-path="icons/ui/brain.svg">
    15 named constructs.
  </Card>

  <Card title="Multimodal API" icon="https://mintcdn.com/medera-357fd587/n1mgcc3nR2sq_PoL/icons/ui/terminal.svg?fit=max&auto=format&n=n1mgcc3nR2sq_PoL&q=85&s=a54c23f663622840fa7b900ec04462a5" href="/api-reference/multimodal/analyze-video" width="40" height="40" data-path="icons/ui/terminal.svg">
    REST surface.
  </Card>
</CardGroup>
