# About Wowza Video Intelligence Framework (VIF)

VIF adds real-time AI analysis to live streams in Wowza Streaming Engine — object detection, scene
understanding, and vision-language model (VLM) analysis — without a separate video pipeline. It's built as
two components: **VIC**, a plugin running inside Streaming Engine, and **VIS**, a standalone GPU inference
service. VIC forwards frames to VIS over a WebSocket connection and receives detection results back.

## Three ways to analyze a frame

- **Object detection** — RF-DETR models (four size/speed variants), with tracking handled by ByteTrack.
- **Scene understanding** — whole-frame classification, so results describe the scene rather than
individual boxes.
- **VLM analysis** — frames go to a vision-language model instead of a fixed classifier. Three modes:
**Detect** (open-vocabulary class list), **Describe** (free-text description), and **Custom** (your own
prompt and, optionally, your own output schema). VIF ships a ready-to-run OpenAI-compatible model
sidecar, or you can point it at a hosted endpoint.


## Built to be extended

Detection results reach the outside world through event listeners — four ship built-in (ID3 metadata,
webhooks, log files, overlays), and you can write your own. `IVifEventListener` is a real, versioned Java
interface with five methods (`onInit`, `onShutdown`, `immediate`, `batch`, `rollup`), all declared as
`default`, so an implementation only overrides what it actually uses. Detection results are delivered as
typed classes — `ObjectDetectionResponse`, `SceneDetectionResponse`, `VlmDetectionResponse` — not just raw
JSON.

The plugin API is published on Maven Central (`com.wowza.wms.plugin.videointelligence:wse-plugin-video-intelligence`),
so it's a normal `compileOnly` Gradle/Maven dependency — no separate SDK download required to build a
listener against it.

VIF also exposes a REST API for managing stream configuration and querying results programmatically,
alongside the WSE Manager UI.

## Deployment

VIF runs via Docker Compose (the most common path), as a manually installed plugin on an existing WSE
instance, or as a self-contained native installer for the standalone VIS service on Windows or Linux —
useful when Docker isn't an option in your environment.

## See also

- [Create a custom VIF event listener](https://www.wowza.com/docs/create-a-video-intelligence-event-listener)
- [VIF Custom Event Listener API Reference](/docs/wowza-video-intelligence-framework/vif-custom-event-listener-api-reference)
- [Use VLM analysis in VIF](https://www.wowza.com/docs/use-vlm-analysis-in-vif)