printer friendly version

Video analytics: the road ahead

Video analytics has come a long way to where it is a reliable and trusted addition to video surveillance today. Many organisations rely on analytics today to make the jobs of monitoring cameras easier and more accurate, leaving human operators to do what technology can’t and to add real value – which excludes sitting and watching a screen all the time.

Most camera manufacturers these days offer some form of analytics with their hardware, such as motion detection, tripwires and so forth. But what happens in the real world. When vendors demonstrate video analytics they have the benefit of controlling the environment and ensuring their systems work well. In the real world there are many factors that can alter the environment, sometimes in seconds. Do analytic applications work as well in these environments?

Hi-Tech Security Solutions asked three vendors who are making waves in the world of video analytics a few questions about their products and experience out in the real world. Our interviewees are:

• Gus Brecher from local VMS developer, Cathexis,

• Roy Alves from Axis Communications, and

• Dr Mahesh Saptharishi, CTO of Avigilon.

Hi-Tech Security Solutions: How have analytics improved over time? What do you see as the main driver in the technology advancing to the stage where people can rely on it?

Gus Brecher.

Gus Brecher: I think the main reasons for the improvements in video analytics has been twofold:

a) Processing power: With the exponential increase in available processing power, you can perform far more sophisticated algorithms on the available hardware, be it on recording servers or on the edge (on IP cameras).

b) IP systems: With the growth of IP, the ability of third-party systems (e.g. Ipsotek, AgentVI) to perform analytics separately from the VMS has become a reality. So companies can perform analytics on the same video that is being streamed to the VMS in a separate system and then feed analytics information or alarms back to the VMS. The ability to perform analytics on the edge on IP cameras has also provided a platform for these third-party development specialists. This has also had the effect of pushing the VMS vendors to up their game and improve their own analytic offerings.

Dr. Mahesh Saptharishi, CTO of Avigilon.

Dr Mahesh Saptharishi: Video analytics accuracy and installation ease has improved considerably over the past five years. There have been significant advancements in the science and algorithms making analytics more suitable for real-world applications. Processor technology has also improved to the point where previously inaccessible sophisticated computer vision algorithms can now be employed at similar or lower costs.

Roy Alves.

Roy Alves: Analytics has changed dramatically over the last 10 years. The main drivers were on one side the end-user requirements which have become more complex; on the other side the rapidly advancing hardware environment has driven improvements. Where a few years ago basic tripwire perimeter protection solutions were sufficient for end users, they now request more reliable solutions with low false alarm rates. Traditionally, standardisation tends to increase the credibility of solutions and market development in a positive sense both for developers and customers – there are numerous examples of that (like ONVIF and Bluetooth) – it is probably just a matter of time before it will also encompass analytics.

Hi-Tech Security Solutions: What analytical functionality can people rely on today?

Gus Brecher: There are many algorithms that do a good job. If you look at vector-based analytics (rather than the basic VMD), the first step is to detect the desired objects and then lock onto them. Once you get this right, then there are many things that you can do with the information. For instance: people counting, entering or leaving an area, loitering, direction, speed, etc. On the more sophisticated side of things, there is facial recognition which is really a specialised application in its own right that requires a lot of processing power and is very expensive. The jury is still out on this for pervasive use. Licence plate recognition is also very specialised, but the good solutions are very reliable.

Dr Mahesh Saptharishi: Capabilities such as detecting loitering, going in the wrong direction, detecting stopped vehicles, counting the number of people or vehicles in the scene, being able to distinguish between people, vehicles and other objects in the scene are all functionalities that customers can rely on today. In addition, analytics such as LPR and face detection and recognition are also maturing to a point where they can be used in many applications.

Roy Alves: There are several possibilities with analytics; to mention one vertical, we can take retail where the end-user can get additional value from their surveillance and loss prevention investment and “sweat the asset” by doing people counting/footfall at the establishment, queue measurement, dwell time measurement, analysis of hotspot and bottle necks within the store by heat maps, and analyse demographics (age/gender).

Another vertical is transportation where you can calculate the number of people entering, for example, a bus or a train. We also see maturity in number plate recognition, with a much higher accuracy in reading plates even when cars are moving at higher speeds. Better image quality in cameras has really had a positive benefit in reducing the number false alarms and providing better analytics.

Hi-Tech Security Solutions: To get reliable analytics, users obviously need more than just some software on their servers or on their cameras. What goes into an effective analytical solution, for example, HD cameras and effective lighting etc?

Gus Brecher: Environment is obviously the most critical variable when it comes to analytics. Camera shake from unsatisfactory camera mounting, poor lighting, noisy environments, attempts to detect objects too far away etc., all lead to poor results and the inevitable false alarms. Analytics configuration is an empirical process and as such, the person configuring this should take due care and consideration of the times of day, position of the sun, shadows etc. It is important to choose products that provide the ability to configure analytics on both live and recorded footage with the ability to change parameters and see the results of these changes immediately. I don’t believe that resolution plays a very big role in analytics right now as most analytics products still work on lower resolution video. This is because there is still a demand on processing power.

Dr Mahesh Saptharishi: Analytics requires camera angles to be positioned correctly. Most analytics available need the camera not to be positioned looking almost directly downward or where the horizon is not level in the field of view. Most analytics require that it also be calibrated properly (Avigilon’s self-learning analytics doesn’t require calibration). In addition, image quality is paramount. Majority of analytics process at CIF or 4CIF resolutions. Thus, HD resolutions don’t benefit most analytics solutions in the market (Avigilon’s analytics operates on HD video and does benefit from higher resolution). Typically analytics require about 100 pixels on target at a minimum for detect and classify. Detection range measurement should be done at night or when light levels are low to make sure that it is consistent with expectations.

Suitable visible or IR lighting should be used to achieve the desired range and accuracy. Avigilon’s solution also has a unique feature called “Teach-By-Example” wherein the operator can provide feedback on false alarms allowing the system to learn and get better over time.

Roy Alves: In order for analytics to work properly, the following questions or facts need to be addressed:

• What are the security or project requirements? So which analytics would be applicable?

• What is the apparent scene? Does it contain levelled or unlevelled topology? Is it an absolute open area or is there a physical barrier (building, fence etc)?

• Based on the above, the amount and layout of cameras and the infrastructure has to be established: for instance, analytics applied for perimeter protection requires that the camera image provides a perspective. This means the camera must be installed high enough (4-5 metres) and have an angle (i.e. so that a person walking towards the camera is changing in size – giving dimension).

• In addition, the maximum distance between the cameras needs to be kept in mind.

• Based on the above, will you need camera masts or not?

• The lighting is also important: Is there a light source, yes/no? Do you install IR cameras, thermal cameras or add an additional light?

• Regarding the type of camera, the first question is again important: What do you want to detect? Do you just want to see that it is a human being and that it triggers an alarm? Then a thermal camera would be perfect and very reliable. However, if you want to identify the person in such detail as even identifying the face, a colour camera with high resolution will be applicable.

• With all that defined you can then adjust and configure the analytics with the appropriate sensitivity levels and filters according to the environment.

Hi-Tech Security Solutions: Can you tell our readers how your analytical solutions have evolved over the past few years? Do you offer server-based or edge solutions and why? In which environments are you seeing the best results from your products?

Gus Brecher: We historically provided motion detection algorithms which detected motion with the ability to change sensitivity associated with the number of pixels that had been triggered. We took this further by enabling the system to learn the environment and analyse detection times in order to reject false alarms from things like passing clouds, trees blowing in the wind, etc. Add to this the ability to setup multiple zones within a camera view with different sensitivities for each zone, and you end up with a more than satisfactory solution.

Our new analytics suite uses vector based algorithms, which enables us to identify an object according to the size required, lock onto it and then track it within the camera view. The most critical aspect of using the vector based algorithms is that we can automatically take perspective into account. This means that once we know the size of a person, for example, at two points in the camera view, we can extrapolate what size a person is in other areas of the view. This dramatically reduces false alarms and enables us to add rules to the system associated with the objects that are being tracked.

We only offer server based analytics, but do enable third-party analytics solutions running on the edge to send us triggers. Our experience has shown, however, that due to the processing limitations on the cameras, when you do perform any form of reasonably sophisticated analytics, there is some form of sacrifice which may take the form of reduced frame rates, reduced number of video streams or some other area.

Dr Mahesh Saptharishi: Avigilon’s analytics are edge solutions (on the camera, or in an appliance). Analytics has access to uncompressed high quality, high resolution video at the camera. Thus, analytics algorithms can achieve better accuracy in most detection tasks. Avigilon has been a pioneer in artificial neural network technology and has made some significant leaps over the years. The primary use for Avigilon’s analytics is in outdoor perimeter protection – where a high detection rate and a low false alarm rate is desired.

Roy Alves: Axis development partners offer both server-based and embedded solutions. It is a fair assessment that the most complex and sophisticated solutions will continue to come from server-based analytics also going forward, but keep in mind that what is embedded today was server based 10 years go (or less), that is the way the technical development goes. One could view server-based solutions as the development platform for future embedded analytics.

Customers are definitely buying into edge analytics which come with the advantage of being infinitely scalable and decentralised, cost-effective – requiring no additional hardware and normally operate with very low bandwidth requirements and installation and maintenance costs. Some of the analytics in retail, for example, have developed to better understand their customer behaviour and to better design stores based on that knowledge.

Today we find that many customers look to analytics to get the most from an infrastructure perspective, if they are moving from analogue cameras to IP and today have a door beam counting solution, they may look to see if the new infrastructure can solve this task too (which Axis can). So if alarm verification is what they need, they will seek a solution for that.

In addition, video analytics has become more mainstream and accepted by the general public. For this we have seen many of the analytic companies focused on making video analytics more intuitive in handling and configuring. Furthermore, the requirement of providing integration to third-party systems has drastically increased. In order to serve this need, we are starting to see analytics manager, a platform allowing our server-based or camera based analytics to be integrated in any third-party video management system.

Hi-Tech Security Solutions: What’s next in terms of analytics?

Gus Brecher: We are seeing the introduction of some smart situational awareness software, which learns from the analytics environment over time and automatically generates triggers on what is deemed to be unusual behaviour. I think we will see facial recognition prices driven downwards as processing power becomes even better and more competitors enter the fray. I also see analytics being used more for business information like people counting and heat-mapping for marketing purposes. And, of course, the algorithms will continuously improve.

Dr Mahesh Saptharishi: Identity analytics (the ability to recognise a vehicle uniquely or recognise a person uniquely) is reaching a point where it will become standard. The types of objects that analytics can recognise in the scene is also getting richer. Analytics is also capable of recognising different types of activities in the scene (such as queue lengths being long, two people talking/interacting, fighting, collisions between vehicles, etc.). Many of these analytics will become standard offering in the next few years.

Roy Alves: Many of the analytics we see today in terms of audio detection, improved video motion detection, people/vehicle counting, number plate recognition, heat maps, wrong direction, perimeter protection are all, in our opinion, at a mature state. We will see tracking analytics embedded with the possibility to follow individual person’s movement in an area/store. Where have they been walking, stopping, for how long, what they have been looking at, touching and finally buying? How have they visited the store? What percentage of the people walking outside the store will enter, what is the age profile, gender, and mood? I think there is much work to be done here, e.g. having a camera telling a retailer who their customer is and their behaviour.

Also, I predict that, like surveillance industry where manufacturers are producing cameras for a particular vertical segment, we will see the same coming from software development. It’s not a case of one size fits all.

Within the security segment and perimeter protection I see increasing demand on not only receiving real-time alerts, but to gather as much data as possible about alarm objects, i.e. for further forensic reasons. Hence, making as much information as possible about an object available to the operator.

Hi-Tech Security Solutions: We hear a lot about using security solutions to add value in other areas of the business. Are you seeing this in the real world?

Gus Brecher: Absolutely. Surveillance is moving beyond the security domain. Customers are becoming more educated on what information and benefits can be obtained from their security systems. We have a large retailer in the UK, who is using our VMS for training purposes by recording customer interaction with audio and using the recordings to assist their staff in improving their customer interaction.

Shopping malls are using people counting algorithms to give them valuable information about the footfall in certain areas of the centres, and some are even charging rentals based on this foot traffic. And the marketing guys are looking at the “flow” of people around their stores via directional analysis and heat-mapping to assist them with their “point-of-purchase” advertising planning. Of course, this requires a mindset change for a lot of companies where people often work in silos, but it is happening.

Dr Mahesh Saptharishi: Retail analytics is one significant example of a non-security application of security technology. Understanding what products attracted the attention of customers, how many customers came into a store vs. purchased a product, how long is the checkout line, how many customers were helped by an employee, which shelves are currently empty, etc., are information that can help a retail business improve operational efficiencies and increase profit. The security/loss prevention department acts as a service provider to other departments within a business. Often, additional cameras or sensors are required to offer a valuable solution.

Much like the operating model of IT departments, we expect security departments to offer this information as a subscription service to the marketing, merchandising and operations department of a business. Often, just the act of seeing the insights that video provides is enough for a department to want to try it. If the marketing department was using third-party audit firms to help gather in-store data, video analytics can provide the same at a much cheaper cost, in real-time.

Roy Alves: Yes, I am definitely seeing this in the real world, if a camera can have a dual use, for example surveillance and queue measurement, then this is definitely of interest to departments beyond loss prevention. I think generating buy-in comes back to how well the loss prevention manager understands the challenges and KPIs (key performance indicators) of other departments. If he/she can present a convincing business case to the store manager or operations manager that he can deliver predictive queue analysis and proactive queue measurement with the same camera, I am sure there will be cooperation on budgets and investments.

For more information contact:

Axis Communications, +27 (0)11 548 6780, [email protected], www.axis.com

Avigilon, +27 (0)72 584 4992, [email protected], www.avigilon.com

Cathexis Africa, +27(0)31 240 0800, [email protected], www.cathexisvideo.com

Share this article:

Further reading: