printer friendly version

The value of computer-assisted vision

Sight is probably the most valuable sense humans possess and modern computer-assisted vision dramatically increases safety and security. Nowhere is this more evident than in driving and in CCTV surveillance, two areas where it is vitally important to minimise the effects of human fallibility.

In both areas, humans are often bombarded with a massive amount of visual information that needs to be digested and interpreted very quickly, often in conditions of severely impaired visibility. Both activities also demand undivided attention. Even small distractions behind the wheel of a car can result in potentially disastrous situations. And in surveillance, experience has shown that the fallibility of the human brain resulting from imperfect memory recall, biases, momentary distractions and boredom places severe limits on the reliability of conventional human-based video surveillance.

Interestingly, whereas the challenges of driving and CCTV surveillance appear to have a lot in common, so too do the solutions. In particular, many of the advanced algorithms developed for intelligent video content analysis are generic in nature, which means they can be applied to equal advantage in either application area.

Shared development

It was recognition of this fact that prompted Bosch to establish its research centre for video-based surveillance systems within its Competence Centre for Surround Sensing Systems (CCS) in Hildesheim, Germany. CCS has a reputation for research in computer vision systems. More recently, the research being performed at CCS has also greatly benefited Bosch Security Systems in the development of the company’s advanced intelligent video analysis (IVA) software that has now become an important feature of its network IP video solutions.

A recent example of developments in the automotive area is the night vision system for the Mercedes-Benz S Class based on an infrared-sensitive video camera. Infrared high-beam headlights illuminate an area 150 metres in front of the vehicle. The infrared image is picked up by the video camera whose high-performance electronics converts the signals into an image visible to the human eye. A central display enables drivers to identify dangerous situations more quickly, giving them more time to react.

The next stage in this development is the creation of driver-assistance systems that recognise defined characteristics and give the driver information about them. New technology now ready for series production includes highly advanced processing technology that is able to use the image data delivered by the video camera to recognise whether pedestrians are standing or moving. This means it can highlight pedestrians in colour on the display, for example, and draw the driver’s attention to them.

The advanced intelligent video-analysis algorithms used in developments like these are also the perfect basis for intelligent CCTV surveillance.

Meeting security’s special challenges

Bosch Security Systems has benefited from the extensive expertise in video content analysis of the company’s Competence Centre for Surround Sensing Systems since the late 1990s. Initially the products concentrated on simple video motion detection. The VMD01 video motion detector, introduced by Bosch in 2003 was based on simple tracking of objects. Developments accelerated from 2006 with the introduction of the company’s IVA (intelligent video analytics) software for its range of network IP video products. The introduction of intelligence to video motion detection significantly increased the possibilities for guard assisted CCTV surveillance and reduced demands on bandwidth and storage capacity. The new software introduced a more event-based surveillance regime, ensuring that only scenes in which important changes occur would be captured, transmitted and stored.

Changes in environmental conditions are one of the major causes of false alarms in video motion detection systems. To minimise these, Bosch’s IVA software includes a background learning algorithm developed by CCS that allows for changes in background and saves computational power by suppressing unwanted notification from, for example, moving trees, branches, leaves, clouds, shadows and falling rain and snow.

Moreover, traditional video motion detection systems are implemented as standalone software systems running on an industrial PC platform. These PCs are installed in a central location (eg, in the control room), and continuous streams of monitored video are transmitted over the network for analysis and archiving. This places a huge strain on network bandwidth, and the use of PC-based platforms for video analysis adds unnecessary costs for hardware, operating system, etc, and increases the risks of virus attacks. Bosch overcame this problem by embedding the VCA functionality in the encoders and cameras themselves. Video content is analysed, compared against known rules, and events are generated at the edge of the network – ie, in the cameras. Only video footage of interest (eg, abnormal events) is transmitted to the control centre.

This smart camera approach was a departure from traditional video over IP systems as it eliminated the single-point-of-failure (SPOF), reduced network traffic and eliminated the overhead of a separate PC for running the VCA software.

In later developments, the range of alarm criteria was extended to include object identification on the basis of aspect ratio, idle-object detection for detecting items left at a scene or cars parked in sensitive locations, object-removal detection for monitoring displays in, for example, museums and retail stores, and trajectory mapping for detecting suspicious behaviour such as loitering. An image stabilisation feature for pole-mounted cameras was also incorporated. This provides stabilisation on both the vertical and horizontal axes before the image is processed by the detection algorithm, ensuring reliable tracking even when the cameras are mounted on unstable supports.

In IVA version 3.5, the latest release of its IVA software, Bosch has built on the features of the earlier versions with further enhanced detection possibilities. These include new colour filtering capabilities that allow object colour or even a combination of colours to be detected. This is embodied in a colour histogram function that allows object colour or colours, saturation and precision to be set as monitoring criteria. The filter set has also been extended with new powerful features such as object trajectories, line crossing alerts and aspect ratio filtering. Triggers can be set to transmit alerts if, for example, objects cross a pre-defined line or multiple lines, or change speed (running), shape (crouching) or aspect ratio (falling).

Bosch’s IVA 3.5 also includes a new forensic search function. Content analysis information, in the form of metadata, is automatically generated by the IVA software and stored with the video images. The recorded metadata, comprising simple text strings describing specific image details, is much smaller and easier to search through than the recorded video. Searches, which may take days or even weeks when done manually, can now be completed within seconds just by searching the metadata with smart search facilities like those provided by an Internet search engine. The forensic search function in IVA 3.5 also allows extra detection criteria to be set even after the live video has been recorded. For example, even if the live system had not been configured to say detect a vehicle of a particular colour, it is possible to configure it to do this later during a forensic search of the recorded video.

Developments in the pipeline

While the latest version of Bosch’s IVA software contains filters for detecting suspicious behaviour such as loitering, more powerful algorithms for analysing crowd scenes are in development at CCS. These include algorithms capable of measuring every pixel in every frame of streaming video to track individuals in highly crowded scenes (eg, in sports stadiums) or detect counter-flows in crowds, incidents of crowd panic or people congregating at a single point (eg, where someone has collapsed). With their high computational complexity, however, these algorithms can generally only run offline on special hardware or a dedicated ASIC.

New exciting developments are also in the pipeline in video management, specifically in Bosch’s Video Management System (BVMS). Today the system is 2D map-based but future developments are expected to lead to a 3D interactive map-based system utilising the video input from multiple cameras. A greater level of interaction between cameras is also expected allowing powerful extensions to the IVA functionality. This will include tools such as 'camera handover of information', where object identification and descriptors are automatically handed over from one camera to another, enabling objects to be recognised and reliably tracked over multiple scenes. This will have a major impact on the effectiveness of guard-assisted surveillance, enabling security personnel to draw more confident conclusions about suspicious behaviour. In the case of an object left at a scene, for example, they would be able to conclude whether the owner is still in the neighbourhood or has really left the building. This function also opens up tremendous potential for more powerful forensic searching over several cameras.

Share this article:

Further reading: