VIF adds real-time AI analysis to live streams in Wowza Streaming Engine — object detection, scene understanding, and vision-language model (VLM) analysis — without a separate video pipeline. It's built as two components: VIC, a plugin running inside Streaming Engine, and VIS, a standalone GPU inference service. VIC forwards frames to VIS over a WebSocket connection and receives detection results back.
- Object detection — RF-DETR models (four size/speed variants), with tracking handled by ByteTrack.
- Scene understanding — whole-frame classification, so results describe the scene rather than individual boxes.
- VLM analysis — frames go to a vision-language model instead of a fixed classifier. Three modes: Detect (open-vocabulary class list), Describe (free-text description), and Custom (your own prompt and, optionally, your own output schema). VIF ships a ready-to-run OpenAI-compatible model sidecar, or you can point it at a hosted endpoint.
Detection results reach the outside world through event listeners — four ship built-in (ID3 metadata, webhooks, log files, overlays), and you can write your own. IVifEventListener is a real, versioned Java interface with five methods (onInit, onShutdown, immediate, batch, rollup), all declared as default, so an implementation only overrides what it actually uses. Detection results are delivered as typed classes — ObjectDetectionResponse, SceneDetectionResponse, VlmDetectionResponse — not just raw JSON.
The plugin API is published on Maven Central (com.wowza.wms.plugin.videointelligence:wse-plugin-video-intelligence), so it's a normal compileOnly Gradle/Maven dependency — no separate SDK download required to build a listener against it.
VIF also exposes a REST API for managing stream configuration and querying results programmatically, alongside the WSE Manager UI.
VIF runs via Docker Compose (the most common path), as a manually installed plugin on an existing WSE instance, or as a self-contained native installer for the standalone VIS service on Windows or Linux — useful when Docker isn't an option in your environment.