> ## Documentation Index
> Fetch the complete documentation index at: https://larkup.de/documentation/llms.txt
> Use this file to discover all available pages before exploring further.

# Video Intelligence

> Index video into timestamped evidence for accurate chat answers.

Video Intelligence turns a video into searchable evidence. It transcribes speech, reads visible text, detects objects and anonymous tracks, and preserves timestamps so chat can point back to the relevant moment.

Its planner is genre-neutral: it adapts to the question, duration, and measured
source signals rather than selecting a hardcoded workflow for lectures, sports,
films, entertainment, meetings, or other content. Visual verification records
generic subject–relation–value bindings when a clip directly establishes them.

For each chat question, the model first supplies a typed investigation
directive: a local moment, a temporal relationship, or the whole source; then
an answer, comparison, trace, enumeration, or synthesis operation. The
evidence engine validates and executes that directive without routing from a
language-specific keyword list. Whole-source enumeration reads chronological
evidence rather than treating one highly ranked moment as the entire video.
During indexing, vision works in chronological waves: identities become
continuity candidates only when source evidence grounds them, and similarity of
appearance alone is never treated as identity proof.

## Install the tool

Open **Settings → Marketplace**, install **Video Intelligence (New)**, then open **Data → Media → Video**. Before indexing, the guide can collect an optional goal, known entities, and questions you expect to ask.

The optional goal is a focus, not a filter: the index still keeps source-wide coverage, while the planner and semantic reader spend additional attention on the details the user named. Larkup stores that hint with the media asset, sends it in the ephemeral indexing brief for local or Cloud work, and records it with the resulting knowledge revision so a later reindex remains auditable. The hint text is never placed in usage analytics.

## What the index contains

* Timestamped multilingual speech from faster-whisper.
* Visible text from slides, code, scoreboards, signs, and captions.
* Objects and anonymous motion tracks detected across the video.
* Source-language story notes that connect speech, named participants, visual layout, and events across scenes.
* Coverage metadata and an answering guide used to ground chat responses.

Balanced is the normal choice. Use Fast for a quick overview and Thorough for denser visual changes or short-lived details.

Before any runtime starts, choose an audio provider and add its API key in the
tool settings. Its transcription model is selected automatically. Agent and
vision controls default to their providers, models, and keys from **Settings →
AI Models**, while allowing independent per-tool overrides. When the agent uses
a text-only provider such as DeepSeek, select a separate vision provider. These accounts pay for model calls;
Larkup Cloud provides only the GPU compute that decodes, samples, reads, and
structures the video.

## Choose where it runs

**Local runtime**

Use this when you want processing to remain on your machine. Select **Local runtime** in the tool settings. Larkup detects Docker automatically and uses it when available; otherwise it installs and runs the native CPU runtime with `uv`. The runtime starts on `127.0.0.1` and the source copy inside the runtime is deleted after successful indexing by default.

The install confirmation shows the selected runtime, download size, available RAM, and model configuration. Saved **AI Models** settings supply the initial brain and vision models and keys; the tool settings can override them. The selected provider keys are injected only into the local process or container.

**Larkup Cloud**

Use this when you want Larkup to run the GPU video job for you. Select **Larkup Cloud** and connect your Cloud access. Your audio key and effective brain/vision settings—AI Models defaults or tool overrides—remain required. Their keys are forwarded only in the active worker payload, are never written to the Cloud job record, and are removed from a warm worker when the job ends.

Larkup Cloud has no platform-owned audio, brain, vision, or embedding provider credential. It does not expose the underlying GPU provider to your browser. You can ask chat questions while preserving the video timestamps in each answer.

**Custom deployment**

Use this when your team operates its own compatible runtime. Deploy the Video Intelligence Docker runtime behind your authentication boundary, then select **Custom endpoint** and enter its URL and API key in the tool settings.

The custom runtime must implement the same `/v1/uploads`, `/v1/jobs`, and `/v1/jobs/{id}` protocol, including the transient model-configuration bundle. The bundled runtime removes this bundle before persisting a job.

## Ask questions in chat

After indexing, ask direct or indirect questions such as “What changed near the end?”, “Which steps were shown before the error?”, or “Summarize the argument and the evidence on screen.” Larkup retrieves the timestamped video evidence first and reads compatible observations as one chronological trail. It performs a bounded source inspection only when that indexed trail is incomplete, conflicting, or cannot ground the requested identity. If an answer uses general knowledge beyond the video, it labels that part as an inference.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.