Despite a false start many years ago, video analytics have finally emerged from under the shadow of disappointment and frustration that has long plagued the technology.

“With all of the ‘Minority Report’ and ‘CSI’ stuff from the past, there were a lot of misperceptions about what analytics can do and how they’re supposed to work. I think the industry [has] done a pretty good job of cleaning those things up,” says Alex Walthers, business development manager, application developer partner program, Axis Communications, Chelmsford, Mass.

Between adequately setting expectations and improvements in technologies, video analytics are on the verge of a renaissance of sorts that could transform security.

“We saw a first wave 10 or 15 years ago, and then people started to realize what analytics actually could do and what they could not do,” says Florian Matusek, product group director – video analytics, Genetec, Montreal. “Now with AI (artificial intelligence) and so on, there is a new wave of video analytics coming and also reaching new heights.”

When it comes to large-scale changes in analytics in general, there is not much to report, says Robert Wegner, vice president, system application engineering and technical support, Hanwha Techwin America, Ridgefield Park, N.J. 

“Technically, there is not that much new with video analytics; they’re just a lot better and are less expensive,” he says. “For example, line-crossing analytics have improved dramatically over the years, so you have fewer false alarms and more options for utilizing video analytics.”

What is new, he says, is manufacturers’ willingness to include analytics with their cameras, which contributes to the cost-effectiveness of deploying these technologies.

“A lot of manufacturers, including us, include video analytics where two or three years ago there weren’t that many,” Wegner says. “It’s kind of new that these are included with the cameras now, and that is quite a big step.”

Advances in processing power have enabled the latest generation of cameras to host at least a basic level of on-board analytics. While these real-time camera-based analytics may be limited in function, they are typically sufficient for many applications.

“These improvements have helped grow this segment of the camera market and increase their value compared to low-cost models,” says Bret McGowan, senior vice president of sales and marketing, Vicon Industries, Hauppage, N.Y. “Server-based solutions, however, still offer extreme performance, reliability and flexibility compared to the limited performance of camera-based analytics.”

Increased reliability has helped analytics improve security for organizations, while providing them with valuable intelligence that goes beyond security by using video and the metadata captured by the camera. Some examples include monitoring presence to reduce utility bills, identifying patterns in customer activity to improve sales by recognizing hot spots in retail environments.

“When integrators provide their customers with tools that can be used to make smarter business decisions, reduce costs, and increase efficiency, the end user gains a return on investment for security systems that can be measured through tangible business results,” says Chris Johnston, regional marketing manager, Bosch Security and Safety Systems, Fairport, N.Y.

Another growth area for video analytics can be seen in a shift from the technology used primarily for forensic security purposes to a more proactive approach. 

“We’re seeing this combination of video analytics with other technologies,” Walthers says. “Whether it’s microphones to add audio analytics, speakers to give audio alerts or lights to flash and respond in ways, more of these technologies are being connected, which means things are getting a little more proactive.”

By harnessing deep learning, parallel processing and data science technologies, video content analytics is transforming how organizations utilize video surveillance footage to be more productive, proactive and predictive.

“The AI-driven, comprehensive platform approach to video analytics allows for crimes to be solved faster, cities to be safer and for companies to serve customers better,” says Stephanie Weagle, chief marketing officer, BriefCam, Newton, Mass.

One of the largest trends in security is the notion of big data. Organizations capture as much data as possible for analysis, but the sheer volume of data captured by video surveillance can be both a blessing and a curse, says Willem Ryan, vice president of marketing and communications, Avigilon, a Motorola Solutions Company, Vancouver, Canada. On one hand, when an event occurs, the system will capture a great deal of data about what has happened, which enables appropriate actions to be taken to resolve the situation or provide critical post-event evidence. 

“On the other hand, finding those important needles in the haystack can be labor-intensive, frustrating and fraught with error,” he says.

Often, Ryan says, opportunities are missed because it is not viable to keep searching for relevant footage. 

“When a search is abandoned because it is too time-consuming, the system has failed for the customer,” he says. “Guiding your customers [to] adopt video analytic technologies that can help take care of the watching part of their job will provide incredible value to their business by allowing them to focus on proactive response.”

Analytics With Intelligence

The idea of artificial intelligence is not new, but the term has meant very different things through the years.            

“People were talking about AI in the ‘50s and they were talking about it in the ‘80s, and each time it meant something different,” Matusek says. 

Today, advanced AI algorithms allow video analytics technologies to learn on their own, performing hierarchical data extraction and continually adapting over time even in a complicated environment. As the system learns the environment, the more accurate it will become when an abnormal activity occurs. 

“Based on advanced deep learning algorithms, one type of AI is when a security device triggers an alarm to alert personnel when an abnormal behavior is detected,” explains Jennifer Hackenburg, senior product marketing manager, Dahua Technology USA, Irvine, Calif.

Accuracy and ease of use are two of the main benefits of AI in video analytics.

“Video analytics that effectively implement AI will provide solutions that are significantly more accurate and easy-to-use than non-AI based solutions,” says Zvika Ashani, chief technology officer, AgentVi, New York. “In addition, it will be possible to implement many use cases that were not possible before because of technological limitations. For example, detecting people with very specific attributes — wearing a hat, carrying a backpack, etc. — or detecting many different kinds of objects such as garbage piles, very specific kinds of vehicles and more.”

Perhaps the most significant application of AI in video analytics is in automating what are best described as dull tasks, such as monitoring live video and conducting post-event investigations. 

“Operators and investigators just aren’t made to watch many, many hours of video as their attention and accuracy significantly degrades after 30 minutes,” says Brent Boekestein, CEO, Vintra, San Jose, Calif. “Technology like ours has been used to save hundreds or even thousands of hours on challenging investigations.”

By taking over the task of watching security video, AI can help focus operators’ attention on events that may require response or further investigation. Video analytics technologies can use AI to continuously learn what typical activity in a scene looks like and then detect and flag events outside of that norm, adding automation to surveillance. 

“This helps save time during an investigation by allowing operators to quickly search through large amounts of recorded video faster, automatically focusing their attention on the atypical events that may need further investigation, enabling them to more effectively answer the critical questions of who, what, where and when,” Ryan says.

Increased computing power and lower hardware costs have contributed to making AI become an affordable option for the security market. The increased use of AI will increase detection accuracy and provide users with new detection and search features for security and non-security applications. The results, however, will not be immediate.

“Although these technologies are available today, due to their ‘learning’ nature, it will take time to fully realize their capability,” says Eric Olson, vice president of marketing, PureTech Systems, Phoenix.

At a high level, AI can be divided into two basic categories: general and narrow. General AI features the characteristics of human intelligence such as planning, understanding language, recognizing objects and sounds, and learning to solve problems. Narrow AI focuses only on a few or even a single facet of human intelligence like facial or speech recognition. Machine learning — a form of narrow AI — is a way of training an algorithm to adjust itself to improve its performance.

“With respect to video analytics, it can take many hours to many days to initially train the algorithm to detect a specific object,” Johnston says. “With machine learning, video analytics can be taught about new objects or situations within a particular scene and retain this information and call on it when processing subsequent scenes. We are looking at these capabilities to enhance our offerings in the near future.”

This training can be used to reduce or eliminate one of the main challenges of many security installations. 

“AI is a big step forward in video analytics due to its ability to be trained to reject false positives,” says Robert Beliles, chief marketing officer and vice president of marketing, Pelco by Schneider Electric, Fresno, Calif.

In fact, this training is key to the success of analytics with AI capabilities, and providers can play a major role in the process.

“Even if two vendors use the same analytics employing identical algorithms, the training can make a big difference,” Beliles says. “So how — and how much — training was done by the vendor, integrator and/or end user will become one of the distinguishing or differentiating features between vendors and integrator solutions. Assuming the vendor offers the ability to further train the analytics to filter out false positives in the field, these analytics may become even more effective.”

Many solutions built on out-of-the-box algorithms have been trained for things like autonomous driving or finding pictures of cats on the Internet, Boekestein says. Therefore, it is important that solutions be built from the ground up with their own deep-learning platform. This not only allows for future customization but the solution will most likely be trained for the type of video it will be encountering in the field.

“If your customer is routinely surveying video from slightly grainy surveillance footage, who cares if it can spot George Clooney in a sea of people on the red carpet?” Boekestein says.

The extreme forefront of AI and analytics is geared toward deeper identification of people, objects and events without human intervention, which are certainly on the horizon.

Looking to the Future

Whether it’s referred to as deep learning, expert systems, knowledge engineering or any of the other terms, AI is still in its early days and may not be quite ready to deliver on some of the promises that have been made.

“AI’s impact to this point has largely been more about marketing and branding than any actual impact on technology,” says Jeff Whitney, vice president of marketing, Arecont Vision, Glendale, Calif. “That will change as the cost of the technology and standard chipsets supporting useful AI-based analytics become more common and standards are adopted for the technology.”

However, this will be a gradual process. 

“Unlike traditional video analytics, where you set the rules and the algorithms work right out of the box, AI systems must be trained,” Olson says. “New views, new targets, different nuisance items must all be ‘taught’ to the neural network via hundreds or thousands of videos or images. This takes time, but eventually the neural networks will have enough data and have learned enough to make the deployment fairly seamless.”

With regard to video analytics, the primary applications of AI today are machine learning or deep learning, which offer a lot of promise to make analytics more accurate and easier to configure. However, despite claims to the contrary, much of that promise just isn’t being used today, Matusek says.

“There’s a lot of potential there and not many companies are actually doing it even though they’re saying they’re doing it,” he says. “If you walk around the exhibition floor, then you see a lot of deep learning and AI but if you go into conversations you find out that it will be ready in the next year but it’s not in the product right now.”

All this being said, AI is definitely the future of video analytics and will change the market in significant ways, 

“Video analytics and AI are the foundation stones to the next movement in security, which is automation — not only automated monitoring and alarming, but also automated response,” Olson says. “Much like computers are now controlling vehicles without the need for a driver, security systems will quickly be moving in the same direction.”