printer friendly version

Big data operational and management requirements

Big data offers huge advantages in enhancing or ‘augmenting’ data, giving it added meaning and context, creating a broader perspective in which to view things, and adding to the value of data and enhancing the potential depth of evaluation. We are increasingly seeing the implementation of big data integration and synthesis to assist in decision making in a range of control room environments, and this is just the start of a process that is likely to be much more sophisticated in the future. Processes such as face recognition, number plate recognition, access and personal information, movement tracking, and various dimensions of video processing can all be combined to assist us to make decisions in the security area.

The foundation of big data is, by nature, shared and linked data. In many cases this may be live data from sources such as weather, traffic, number plate recognition, positioning or status from electronic tags, etc. In others, you may access short- to long-term stored data where delays of data transfer and processing can lead to a few seconds of latency in the process of the transmission and integration.

For example, scams using delays measured in seconds in live feeds for betting abuse have been found in a number of crime syndicate related activities. However, users are also likely to access systems that contain days, weeks or even years of recorded data. In all of these, there is a quality factor that should be evaluated very carefully. Risk implications include having inaccurate information, out of date information, faked information, or delayed updates that can have a major impact on the integrity of judgement or conclusions that arise from the data.

Data quality

Data is only as good as the capture process and ongoing management of the information in the database. One would consider the data inflow process, how consistently is data treated across the capture process, and the considerations regarding accuracy and possible deviations or inconsistencies. In addition, how timeous is the data or does it reflect conditions that may have already changed or become obsolete, how regularly is it updated and is there a way to establish the most recent update, and what happens with redundant data – is it cleaned, does it clog up the systems, when does it become redundant, and who has responsibility for this?

Obviously the design of the big data system interaction and integration would consider some of these factors, but the danger is in taking these kinds of conditions for granted and ending up with a domino effect where small errors can lead to major implications for valid decision making. Where parties responsible for data management of some of the systems are not efficient in their capture, management and maintenance it has implications for the validity of the data.

Before you start using big data sources, you need to have your own evaluations of the risk implications of using the various sources. This should include trustworthiness, confidence levels, and measures that are built in for verification. Further, it is not just the data itself. There may be various algorithms that have been set up to process the data which may have their own implications. The issue of algorithm bias and discrimination that may result from perspectives of those setting up the system came up as a prominent issue in a behaviour analysis conference in Minneapolis recently, with comments from an ACLU representative commenting about the potential of this.

Protecting yourself

Ultimately the use of big data is a social process and not just an IT one. Where data can get captured, criminals may be working actively to combat identification or labelling by systems such as number plate recognition, or face recognition. This may go to active strategies such as cloning number plates and even vehicles, not just hiding identifiers. Hacking and manipulation of such systems, or paying to have data compromised or changed need to be taking into consideration.

Users should conduct a thorough risk analysis of the systems they may be drawing from, and have procedures in place to ensure verification, legal accountability, and be in a position where they can evaluate the cost of working decisions or outcomes on the business or organisation. Use of powerful tools needs to be balanced with appropriate caution.

Craig Donald.

Relying on AI for decision making should also be part of any risk evaluation. Thinking and verifying before one acts on data should ultimately be a human decision which incorporates verification and situational awareness. I continually am surprised by the amount of social and psychological consideration that needs to go into security systems that are supposedly ‘intelligent’ and can think and act on their own. How much do you trust the people behind the data?

Craig Donald

Dr Craig Donald is a human factors specialist in security and CCTV. He is a director of Leaderware which provides instruments for the selection of CCTV operators, X-ray screeners and other security personnel in major operations around the world. He also runs CCTV Surveillance Skills and Body Language, and Advanced Surveillance Body Language courses for CCTV operators, supervisors and managers internationally, and consults on CCTV management. He can be contacted on +27 11 787 7811 or craig.donald@leaderware.com

Share this article:

Further reading: