Lumeo announced the launch of new Generative AI capabilities. These enable users to extract advanced insights from video and search through their footage.

With the growth of video content across various industries, Lumeo said the ability to efficiently analyze and derive insights from this data is crucial. The company’s new AI search capability makes it a compelling choice for enterprises, integrators and managed service providers (MSPs) seeking unmatched flexibility in accessing and deploying customized video analytics solutions.

Lumeo now supports running and integrating with large vision models such as OpenAI GPT-4, Google Gemini, Nvidia NIMs via APIs, as well as Generic Detectors, Generic Classifiers, and Caption Generators running on-device. These integrations enable:

1. Training-free Analytics: These Generative AI models are pre-trained on large datasets and allow users to use natural language prompts to extract scene and object attributes similar to human observation. This enables new analytics use cases that were previously too difficult or required new models. For instance, prompts like “describe if this is a possible case of vandalism” or “is there a fight going on?” can be answered accurately.
2. Smarter Captions and Alerts: Utilize large vision models to caption videos or scenes for faster retrieval and make alerts more actionable. For instance, instead of sending a generic “person detected” alert, users can now receive “two people detected wearing masks and attempting to open the door” or “person detected, looks like they are passing by the door.”
3. Hybrid Compute: Large vision models tend to be compute-heavy. Lumeo enables users to run Large Vision Models conditionally (say when a group of people are detected, or just on a person vs the entire scene), and on-device or via Lumeo’s integrations with cloud APIs from OpenAI, Google, and Nvidia. This reduces bandwidth and cost when using cloud APIs and enables more efficient hardware utilization when running on-device.

Lumeo’s new AI Search capability enables users to search through video using natural language, making it faster to find the relevant footage from anywhere. The salient features of Lumeo’s AI search are:

• Natural Language Queries: Users can perform searches using everyday language, such as “a person falling down” or “a red car entering the parking lot.”
• Index Objects, Attributes, and Descriptions: AI Search indexes scenes or specific objects, their attributes, captions and descriptions generated by classic vision AI models or large vision models.
• Index Custom Events: Define and search for custom events tailored to specific requirements, enhancing the relevance and precision of the search results. For instance, by creating an analytic rule that looks for illegal right turns, users can index “illegal right turn” events and search for them.
• Hybrid Deployment: AI Search indexes video content on the AI Gateway that is performing the processing, and then uploads the video and metadata for triggered events to Lumeo Cloud. This enables users to search from anywhere while keeping heavy processing load on-device.

Through these capabilities, AI Search helps with:

• Customized Search Queries: Tailor specific search queries to meet unique video analytics requirements, such as detecting particular objects, actions, or patterns within footage.
• Seamless Integration: Integrate AI search capability with Lumeo's existing suite of analytics tools for a unified and comprehensive video data management experience.
• Natural Language Search: Utilize natural language for queries, allowing users to explore video content using descriptive phrases or questions, regardless of predefined analytics categories.
• Enhanced Efficiency: Streamline the search process, significantly reducing the time and effort required to identify specific insights within large video datasets, enabling swift, informed decisions.
• Unrivaled Customization: Empower users to customize search parameters according to their industry-specific needs, unlocking new possibilities for data-driven decision-making in various fields, including security monitoring, retail analytics, and industrial optimization.

“We are thrilled to bring Generative AI, AI Search and Large Vision Models to video analytics, a testament to our commitment to empowering users with cutting-edge tools to unlock the full potential of their video data,” said Devarshi Shah, CEO of Lumeo. “With this groundbreaking feature, we are redefining the way organizations interact with their video analytics, enabling them to extract actionable insights with unparalleled efficiency and precision.”