Limitations Of Surveillance Camera Information Technology Essay

Face recognition for CCTV is basically the study of face recognition by using the images capture from the surveillance cameras. The surveillance camera face database use in this project is SCFace database which provided by University of Zagreb. There are three face recognition algorithms; Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and 2D-Fisherface algorithm being studies in this project. Besides that, there are also four similarity measurement method; Manhattan distance (L1), Euclidean distance (L2), Cosine angle (Cos) and Mahalanobis distance (MAH) being studies in this project. All the images in the database will be preprocessed before apply to the face recognition algorithms. Performance of this three face recognition algorithms and similarity measurement methods will be evaluated. All the algorithms will be coded in Matlab© software.

CHAPTER 1 : Introduction

1.1 Problem Statements

Over the past few years, face recognition have gained more focus by many of the researchers in field of Computer Vision. As we know that, the use of face recognition applications have largely increase in our daily life(exp). Face recognition for surveillance cameras have gained more interest from researchers recently. This is due to the problem of terrorism activities over the world. Face recognition for surveillance cameras can help to identify people who look suspicious.

Basically, there are a lot of algorithms that have been proposed by the researchers in Computer Vision. But for most algorithms, the limitations come from the fact that the face images must be in frontal view, illumination problem and poses variation problem.

There are large part of the work that using FERET database which include some works involved surveillance cameras for security applications. FERET database contains 1564 sets of face images. However, SCFace database is available for research in January 2010[1]. This database is a new surveillance camera face database that contains images from several types of CCTV camera. Since it is very new, we only manage to find one research work that using this database for face recognition applications[2].

For the context of our work, we would like to evaluate the performance of several baseline algorithms applied to the SCFace database. Depending on the result and time constraints, we might proposed an improve algorithm that might produce better performance and overcome the limitations of the work reported in SCFace database[2].

1.2 Project Scope

This project can be divided into three phases. The first phase of the project is the data gathering phase. A surveillance camera database is needed in order to perform a face recognition process. The database that is used for this project is the SCface database[2] which is provided by the University of Zagreb. This database was mainly designed for the means of testing face recognition algorithms in a real world situation. This database contains of 4,160 images which are taken in uncontrolled lighting and pose with five different quality surveillance cameras. Besides that, the images also taken in three different distance between user and cameras.

The second phase of the project is the data preprocessing phase. In the proposed method, all the input images need to be preprocessed before it can be used for the recognition phase. This is due to the variation of the illumination, pose and etc.

The third phase is the recognition phase. In this phase, the propose algorithm will be applied to the input image and then it will be compared with the images in the SCface database. Experimental results, recognition rates and will be analyzed after the recognition is done.

1.3 Research Objectives

The objectives of this project are:

To implement the existing baseline algorithm to the SCFace database.

To identify/propose a suitable novel algorithm for face recognition that can improve performance.

To conduct literary study on face recognition.

To implement and develop a simple GUI in Matlab©.

To test and evaluate the performance of the algorithms.

If possible, to integrate this module into the project Wireless Surveillance and Monitoring System with Face Recognition (FYP180).

CHAPTER 2 : Background and Literature Review

2.1 Background

2.1.1 Face Recognition Definition

Face recognition is a biometric identification by scanning a person’s face and matching the face image with the known face images in the databases or library. According to the Face Recognition [1], face recognition can be defined as given a still images or video frames of a scene, the system will identify or verity one of more persons in the scene using a stored database of faces. Face recognition research can be divided into several directions:

Face Recognition from outdoor facial images.

Face Recognition from non-frontal face images.

Video-based Face Recognition.

Integration of several methodologies to improve the face recognition performance and etc.

2.1.2 Face Recognition

Recently, researches which are related to the face recognition area have gained more interest and have become popular among the researchers in the field of computer vision. There are many ongoing researches in the area of face recognition. Face recognition is not just studied by the researchers from computer vision but there are also psychologist and neuroscientist. Besides that, areas of the usage of face recognition include entertainment (human-computer-interaction), smart cards (national ID or immigration), information security (desktop logon) and law enforcement and surveillance (CCTV control)[3].

According to the Prime Minister of Australia, John Howard, the extraordinary value of surveillance cameras is not just to reduce the incidence of assault and property damage but it is able to help identify and discover the terrorism activities[4]. It can also help to prevent unauthorized access of personal computers or workstations. Nowadays, most laptops or personal computers have the function of face recognition as a biometric login method. This has largely increased the use of face recognition in our daily life.

Algorithm for face recognition can be divide into two types ; Image-based face recognition algorithms and video-based face recognition algorithms. Some common algorithms can be used by both types to perform face recognition process. The algorithm uses for image-based face recognition are Principal Component Analysis (PCA), Independent Component Analysis (ICA), Bayesian Framework and etc. On the other hand, video-based face recognition algorithms are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), 2D-Fisherface algorithm and etc.

A summary of algorithms used for face recognition is given in the table below:

Algorithm

Similarity Measurement

Advantages / Limitations

L1

L2

Cosine Angle

Mahalanobis Distance

Principal Component Analysis (PCA)

Better result

Worse result

Worse result

Best result

Face image need to be normalized.

Face image need to be frontal-view.

Hard to decide the suitable threshold value for each face image.

Linear Discriminant Analysis (LDA)

Better result

Best result

Worse result

–

LDA images cannot perform 2D principal component analysis.

LDA discards components with poor discriminative capabilities.

LDA implicitly assumes Gaussian distribution of data

LDA may overfit the data.

Fisherface

–

–

Better result

–

More better recognition rate than other method.

Use less computing time to recognize images.

Removal of null space which has been shown to contain useful information.

Table Comparison of three Face recognition algorithms

2.1.3 Limitations of Face Recognition

Automatic face recognition is complex tasks which involve face detection and localization of the face features to recognize a face. Although all the human face have the same structure, but at the same time there are a lot of environmental and personal factors that will affecting the facial appearance. The main problem that occurs in automatic face recognition is large variability of recorded images due to pose, illumination conditions, facial appearance, frontal image and etc. Image of the same individual taken at different times might exhibit some differences in facial expression and intrapersonal variability. One way to overcome this problem is to include the intrapersonal variation in the training set images.

Besides that, another crucial problem occur in face recognition is illumination problem. As the image taken is different place, it will having different illumination. For example, when the image is taken in a darker place then the image produce will be darker and it is hard to recognize the face in a darker image. There are several techniques that have been propose recently to overcome this problem. One of the techniques is utilize a large number of example images of the same person under different illumination conditions to reconstruct the novel images. The most effective techniques is based on computer graphics techniques for relighting the probe images so that it resembles the illumination in the gallery images.

Other than that, pose variation also will be a problem in face recognition. When the camera just able to capture the side face of the person, it is hard to determine who is the person. This problem usually can be seen in our daily life. Most of the face recognition system unable to identify a person due to the pose variation. Recently, there are a few approach have been propose to overcome this problem. One of the approach is automatic generate of a novel views to resembling the pose in the probe image. This can be done by using a face model or a deformable 3D model or warping frontal images using the estimated optical flow between the probe image and database image.

2.1.4 Limitations of Surveillance Camera (CCTV)

The first limitation of surveillance camera (CCTV) is some of the surveillance cameras only able to observe the people who are within the camera view[5]. Due to this problem, it allowed the criminals know where is the areas or places that is un-monitored. This have make the surveillance cameras become useless in term of preventing and monitoring.

Besides that, low resolution of CCTV has also become an issues. As we know that, each images or video images contains pixels. In low resolution CCTV, it mean that the pixels of the images are small and less compare to those high resolution CCTV. We can see this clearly when we zoom in an image which is low resolution. There will have small squares appear in the whole image.

Another limitation that is related to low resolution CCTV is the illumination problem. When a video or image captured in a darker place or low illumination, we are hard to see clearly those information in the video or image. Due to this, illumination problem has become a concern to the researchers in the area of face recognition. In face recognition, low illumination images will affect the recognition rates and performance of the algorithms.

2.2 Literature Review

For the scope of this project, three types of commonly used algorithms for face recognition will be studied i.e PCA, LDA and 2D-Fisherface.

2.2.1 Principal Component Analysis (PCA)

According to the researches about face recognition, the earliest and successful algorithm developed was Principal Component Analysis also known as PCA. In the year of 1901, Karl Pearson had invented PCA algorithms [6] . Principal Component Analysis involves mathematical process which transform the large dimensional of an image into a smaller dimensional image.

A 2D image can be represented in 1D image vector by concatenating each row or column into a long thin vector [7]. PCA can be perform in the following steps[8]:

Step 1: Get image data.

Step 2: Subtract the mean from each of the data dimensions.

Step 3: Calculate the covariance matrix.

Step 4: Calculate the eigenvectors and eigenvalues of the covariance matrix.

Step 5: Choosing components and forming a feature vector. The eigenvector with the highest eigenvalues will be chosen as the principal component of the data set.

Step 6 : Deriving the new data set. It transpose the vector and multiply it on the left of the original data set. (Final data = RowFeatureVector * RowDataAdjust)

2.2.2 Linear Discriminant Analysis

Linear discriminant analysis (LDA) is a powerful tool used to reduce the data dimension and feature extraction[9]. LDA algorithm is widely use in the area of face recognition to extract the face features from an image. LDA will used between-class scatter (SB) and within-class scatter (Sw) matrix to define the face features. Between-class scatter matrix defines the scatter features around the overall mean for all face classes. Else, within-class scatter matrix defines the scatter features around the mean of each face classes. The main goal for LDA algorithm is maximize SB while minimizing SW. PCA algorithm will be used as a preprocessing step to prevent the SW become singular.

2.2.3 2D-Fisherface Algorithm

2D-Fisherface algorithm can be divided into 2 stages; 2D-Principal Component Analysis (Stage 1) and 2D-Linear Discriminant Analysis (Stage 2)[10]. The combination of this two algorithms makes 2D discriminant analysis preformed faster than 1D discriminant analysis. Besides that, 2D-Fisherface algorithm gives an automatic strategy to select the 2D principal components and discrimination vectors. First, 2D-PCA will transform X by selected 2D principal components. Then, the result of the previous step will be used in LDA process. LDA will perform a 2D discrimination transform Y by selected 2D discriminant vectors. In order to get the recognition result, the algorithm will use the nearest neighbor classifier to compare between the training images and test sample images.

2.2.4 SCFace Surveillance Cameras Face Database

In this face database, all the images were taken in an uncontrolled of indoor environment using five video surveillance cameras of different quality. This database contain 4,160 images which taken in different illumination (lighting) and poses. It fulfills the condition in real world situation where the lighting and poses are uncontrolled.

2.2.4.1 Surveillance Cameras Setup

All the images captured in the Video Communications Laboratory at the Faculty of Electrical Engineering and Computing, University of Zagreb, Croatia[2]. There are five types of surveillance cameras:

Type

Bosch

LTC0495/51

Shany

WTC-8342

J&S

JCC-915D

Alarmcom

VFD400- 12B

Shany

MTC-L1438

CCD Type

1/3″ IT

1/3″ Sony Super HAD

1/3″ Color

1/3″ IT

1/3″ Sony Super HAD

Active pixels

752×582

795×596

597×537

752×582

795×596

Resolution

540 TVL

480 TVL

350 TVL

460 TVL

480 TVL

Minimum Illumination

0,24/0,038

lux (IR)

0,15 lux

0,3 lux

1,5 lux

0 lux

(IR LED on at 4 lux)

SNR

> 50 dB

> 50 dB

> 48 dB

> 46 dB

> 50 dB

Video Output

1 Vpp, 75 Ω

1 Vpp, 75 Ω

1 Vpp, 75 Ω

1 Vpp, 75 Ω

1 Vpp, 75 Ω

Comment

IR night vision

Dome camera

Dome camera

IR night vision

Table Surveillance Cameras Specifications[2].

All the surveillance cameras are installed at the height of 2.25m. The only source of the illumination was the outdoor light which came through from a window on one side[2]. The cam8 was installed in a separate darker room for capture the IR mug shots. All of this six cameras were connected to a professional digital video surveillance recorder[2].

2.2.4.2 Face Images Capturing

The face images capturing process was conducted over 5 days period[2]. All the images are taken in 3 different distances which are 4.20m, 2.60m and 1.00m. There are nine images taken from left to right in the equal steps of 22.5 degrees (from -90 to -90 degrees).

2.2.4.3 Performance Evaluation

SCFace database using PCA algorithm as a face recognition algorithm and using it to evaluate the performance of this database. All the images in the database are preprocessing in a few steps. First, all the color images will be converted to grayscale images. Then, the images will scale and rotated using the eye coordinates information so that the eyes lie on a straight line. Image will be cropped into 64×64 pixels and masked with an elliptical mask[2]. After that, PCA algorithm will be apply to execute the recognition process.

2.2.4.3.1 DayTime Test

DayTime test is a straightforward test. Frontal mug shots represent the gallery of known images and cam1-5 images and three distances are probe sets[2].

Table Rank 1 performance results for DayTime Test[2].

Gallery: mug shot frontal images.

Distance metric: cosine angle

2.2.4.3.2 NightTime Test

NightTime test can used two possible galleries which are cam6-7 images can be compared to mug shot images or can be compared to cam8 images[2].

Table Rank 1 performance result for NightTime Test[2].

Gallery: mug shot frontal / night vision (cam8) images.

Distance metric: cosine angle

2.2.5 Similarity Measurement

2.2.5.1 City Block / Manhattan Distance (L1)

Equation Manhattan Distance (L1)

The Manhattan distance between two vectors is the sum of the lengths of the projections of the line segment between the points onto the coordinate axes[11].

2.2.5.2 Euclidean Distance (L2)

Equation Euclidean Distance (L2)

Euclidean distance is the distance between two points that one would be measured. The distance between points x and y is the length of line segment[12].

2.2.5.3 Cosine Distance (Cos)

Equation Cosine Angle (Cos)

Cosine angle similarity is a measure of similarity between two vectors of n-dimensions by finding the cosine of the angle between them[13].

2.2.5.4 Mahalanobis Distance (MAH)

Equation Mahalanobis Distance (MAH)

V: covariance matrix

Mahalanobis distance is based on correlations between variables by which different patterns can be identified[14].

CHAPTER 3: Methodology

In this project, there are three face recognition algorithms will be discussing which are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and 2D-Fisherface algorithm. The reason that this three algorithms are chosen because this are the most widely used algorithms all over the world. Besides that, these three algorithms are more easy to be implemented compare to the other algorithms. The details of these three algorithms will be discussed in the below:

3.1 Images Preprocessing

All the images in the database need to be normalized before used it for the recognition process. First, all the images of the SCFace database will be convert into grayscale images. The purpose for doing this is that it will be more convenient and easier than preprocessing color images. After that, the images will be mask with an ellipse mask where we called it “face mask”. This “face-mask” will used to find the face region only in the images. After the preprocessing steps, different algorithms will be used to get the recognition result.

3.2 Face Recognition Algorithm

3.2.1 Principal Component Analysis (PCA)

First of all, the PCA algorithm will be apply to the normalized database images. Eigenvectors of the normalized database images will be calculate and store it into a matrix. Then, PCA will be apply to the input image at the same time in order to get the eigenvectors of the input image. For the recognition process, nearest similarity measurement will be used to compare between the input image and the databases images. Four different similarity measurement performance will be studied and compare.

3.2.2 Linear Discriminant Analysis (LDA)

For LDA algorithm, it will calculate the eigenvectors of the normalized database images with the non-zero eigenvalues. After that, it will discard those eigenvectors with the largest eigenvalues. Then, the optimal discriminant feature of the faces can be obtain. At this stage, eigenvectors of the input images will be calculated and will be used it to compare with the SCFace database images. The nearest similarity between the input image and database images will be the recognized image.

3.2.3 2D- Fisherface Algorithm

As mention in the earlier, 2D-Fisherface algorithm is the combination of 2D-Principal Component Analysis and 2D-Linear Discriminant Analysis. First of all, 2D-Fisherface algorithm will perform a 2D-PCA transform X composed by the selected 2D principal components. After that, it will perform a 2D-LDA where transform Y composed by selected 2D discriminant vectors. The same process will be go through by the input image. After that, the similarity of the images will measure. The nearest similarity between the input image and database images will the recognition result.

CHAPTER 4: Proposed Solution and Implementation Plan

4.1 Implementation Plan

4.1.1 Software — Matlab©

The software that we uses for this project is Matlab©. Matlab© is a powerful tool in the area of image processing.

There is a toolbox called image processing toolbox. We will use this toolbox to implement and test the algorithms to recognize a face. In image processing toolbox, there are a lot of function had been integrate in the software for example like morphological operations[15].

Besides that, we will also use Matlab© to develop a graphical user interfaces (GUI) by using GUIDE. The initial design of the system interface can refer to page 30 in this report.

4.1.2 Context Diagram

Figure Context Diagram of Face Recognition System.

4.1.3 Data Flow Diagram

Figure Data Flow Diagram of Face Recognition System.

4.1.3.1 Data Flow Diagram Explanations

Process 1.0 (System Option)

In this process, the system will prompt the user with several option.

The system option are :

Option 1 : Choose Face Recognition Algorithms (PCA, LDA or 2D-Fisherface).

Option 2 : Input Recognition Image.

Option 3 : Exit System

Process 2.0 (Image Preprocessing)

At this stage, all the images in the SCFace database will be preprocess before applied to the face recognition algorithms. The preprocessing steps include normalized, thresholding and etc.

Process 3.0 (Choose Face Recognition Algorithms)

In this process, the user will choose the face recognition algorithm and applied it to the database images. Similarity of the database images will be calculate.

Process 4.0 (Input Recognition Image)

At this stage, the user need to input the recognition image in order to run the recognition process.

Process 5.0 (Face Recognition Algorithms)

This stage is about the same with process 3.0. The algorithms choose must be the same for two process. The algorithms will applied to the input images to find the similarity.

Process 6.0 (Compare Nearest Similarity)

This process will compare the similarity between input image and database images.

Process 7.0 (Recognition Result)

If the similarity is the nearest between two images then it will output the result and recognition details of the person in the image. If the similarity is not the nearest then it will compare again until it find the nearest similarity.

4.1.4 Use Case Diagram

Figure Use Case Diagram of Face Recognition System.

4.1.5 Flow Chart

Figure Flow Chart of Face recognition system.

4.2 Prototype

Figure Interface of the Face Recognition System.

This is the initial interface design of the face recognition application. It is created by using GUI function in the Matlab© software.

The input image will be placed at the left panel where else the right panel will placed the recognize image. At the right of the applications, the user needs to choose the face recognition algorithms from the drop down menu. The algorithms will be Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and 2D-Fisherface algorithm.

There are two buttons below the drop down menu. One of the buttons is called “Run Recognition Process”. When the user click this button, recognition process will be run according to the algorithm that chosen at the drop down menu. The next button is exit button. When the user clicks on this button, it will exit the face recognition application.

The panel below of the input image and recognition image is used to output the recognition details. It will show the personal details of the person in the image when the application recognize the image.

4.3 Milestone

Table Milestone for FYP part I.

Table Milestone for FYP part II.

4.4 Variation from Original Plan

Originally, the project needs to propose a novel algorithm to solve the problem of face recognition for CCTV. After we go through the research and data gathering phase, we found out that there are a few limitations need to me minimize. Other than that, SCFace database is a new database which introduce in the year of 2010[1]. Since the algorithm uses in SCFace database is baseline PCA. So it is suitable that we compare the performance of different baseline algorithms applied to this database.

We will applied PCA, LDA and 2D-Fisherface algorithm onto the SCFace database. The performance will be evaluate and we might get a better recognition rates by using different algorithms. Besides that, we will also compare several similarity measurements which applied to the algorithms and find the optimum or nearest rates for input image and database images.

CHAPTER 5 : Conclusions

For the scope of this project, there are three face recognition algorithms; Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA) and 2D-Fisherface algorithm will be studied. Data gathering according to this three algorithms had been researched and understood. On the first place, we need to clearly understand all the three algorithms and research that had been done by other researcher in the area of Computer Vision. Besides that, we also need to focus on the recognition rates and other important issues of the algorithms. This is critical that at the end of this project, we are require to improve the performance or recognition rates of the algorithms.

Other than that, we are required to propose a novel algorithm of face recognition on CCTV. This proposed solution will improve the performance and efficiencies of the algorithms. Besides that, the propose solution will also eliminate the some limitations which are now faces in the area of face recognition. The performance of the propose algorithm will be compare to the performance of the earlier three face recognition algorithms.

For the future works, the proposed algorithm will be developing with a wireless surveillance and monitoring system. With the integration of the algorithm to the system, it can increase the uses of face recognition in our daily life. This system can be installing at our house or work place. This can increase the safety of our family member although we are working at the work place.