DEVELOPMENT OF A ROOM RECOGNITION SYSTEM USING A
CATADIOPTRIC SENSOR AND ARTIFICIAL NEURAL NETWORK
EILEEN SU LEE MING
UNIVERSITI TEKNOLOGI MALAYSIA
DEVELOPMENT OF A ROOM RECOGNITION SYSTEM USING A
CATADIOPTRIC SENSOR AND ARTIFICIAL NEURAL NETWORK
EILEEN SU LEE MING
A thesis submitted in fulfilment of the
requirements for the award of the degree of
Master of Engineering (Electrical)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JANUARY 2006
iv
ACKNOWLEDGEMENT
First and foremost, my most sincere appreciation to my project supervisor,
Professor Dr. Shamsudin Hj. Mohd. Amin, who sets a noble example of a devoted
educator. His encouragement and constant support will always be treasured.
A big thank you to my co-supervisor, Associate Professor Dr. Rosbi Mamat, for
his kind guidance and invaluable advice.
My deepest gratitude to Yeong Che Fai, for his continuous motivation, technical
assistance and most importantly, his insightful counsel throughout the entire period of
this project. I can never thank him enough for the inspiration, for his patience and for
always being there as my pillar of strength.
To my dearest family members for their love, care and prayers. Thank you for
persevering with me.
To everyone who assisted me in one way or another, be it through advice or
words of encouragement - my dedicated friends, thanks for making this possible.
A special thanks to Universiti Teknologi Malaysia, especially the Department of
Mechatronics and Robotics, for providing financial support and excellent facilities to
complete my research.
Above all, I thank God for everything that He has given and planned for me.
v
ABSTRACT
Robot localization has been a challenging issue in robot navigation. In recent
years, there has been increasing interest in topological localization. One popular
approach for vision based topological localization is the appearance based method,
where the image can be used for recognition in its basic form without extracting local
features. The general aim of this work is to develop a room recognition system using
appearance-based method for topological localization. In this work, the room
recognition is achieved by matching color histogram of image using the Artificial
Neural Network. A hardware module and a software module have been developed
for this project. The hardware module consists of a catadioptric sensor system
implemented on a mobile platform. The software module encompasses several sub
modules namely image acquisition; image pre-processing; histogram plotting;
histogram filtering, sampling and normalization; neural network for offline training
and testing, and finally real time room recognition. A few experiments have been
conducted to evaluate the performance of the system and the results have been
favorable. Testing for suitable network setting was also carried out and a
recommendable setting was proposed.
vi
ABSTRAK
Lokalisasi robot merupakan satu isu yang mencabar dalam navigasi robot.
Sejak kebelakangan ini, terdapat perhatian yang meningkat terhadap lokalisasi secara
topologikal. Salah satu kaedah yang popular bagi lokalisasi topologikal
menggunakan ‘penglihatan’ adalah melalui pendekatan berasaskan penampilan, di
mana imej dapat digunakan untuk pengenalan dalam bentuk asas tanpa mengekstrak
ciri-ciri tempatan. Objektif keseluruhan projek ini adalah untuk menghasilkan satu
sistem pengenalan bilik yang menggunakan kaedah berasaskan penampilan bagi
lokalisasi topologikal. Dalam projek ini, pengenalan bilik dicapai dengan
memadankan histogram warna menggunakan Jaringan Neural Buatan (Artificial
Neural Network). Satu modul perkakasan dan satu modul perisian telah dihasilkan
bagi projek ini. Modul perkakasan merangkumi sistem penderia catadioptric yang
diimplementasikan atas platform bergerak. Modul perisian pula merangkumi
beberapa sub-modul iaitu pengambilan imej; pra-pemprosesan imej; pemplotan
histogram; penapisan histogram; pensampelan; normalisasi; jaringan neural untuk
latihan dan pengujian secara offline; dan akhir sekali, pengenalan bilik secara nyata.
Beberapa eksperimen telah dijalankan untuk mengevaluasi pencapaian sistem ini dan
keputusan yang diperoleh adalah memuaskan. Pengujian turut dilaksanakan untuk
mendapatkan aturan yang bersesuaian dan satu aturan saranan telah dicadangkan.
vii
TABLE OF CONTENT
CHAPTER TITLE PAGE
DECLARATION
DEDICATION
ACKNOWLEDGEMENT
ABSTRACT
ABSTRAK
CONTENTS
LIST OF TABLES
LIST OF FIGURES
LIST OF ABBREVIATIONS
ii
iii
iv
v
vi
vii
xiii
xiv
xviii
1 INTRODUCTION 1
1.1 Challenges in Mobile Robot Navigation
1.2 An Overview of Robot Localization
1.3 Problem Background
1.4 Project Objectives
1.5 Outline of the Proposed Approach
1.6 Outline of Thesis
1
2
3
5
5
7
viii
2 LITERATURE REVIEW ON ROBOT
LOCALIZATION 9
2.1 Types of localization
2.1.1 Position tracking or local techniques
2.1.2 Global localization
2.1.3 Place Recognition
2.2 Reviews on Sensors and Techniques
2.2.1 Odometry
2.2.2 Inertial Navigation
2.2.3 Magnetic Compasses
2.2.4 Active Beacons
2.2.5 Global Positioning System
2.2.6 Landmarks
2.2.7 Map Based Positioning
2.3 Vision Based Localization
2.3.1 Appearance Based Methods
2.3.2 Related Works
2.4 Features of the Proposed Technique
2.5 Summary
9
10
12
12
13
14
14
14
15
15
16
17
18
19
20
22
22
3 SYSTEM DEVELOPMENT METHODOLOGY AND
DESIGN
24
3.1 Design Methodology
3.2 Hardware requirement and specifications
3.3 Software Architecture
3.3.1 Image Acquisition
3.3.2 Pre-processing
3.3.3 Image Classification
25
27
28
29
30
30
ix
3.3.4 Data Retrieval and Matching
3.4 System Integration
3.5 Performance Analysis
3.6 Summary
30
31
31
32
4 APPEARANCE BASED ROOM RECOGNITION
SYSTEM DEVELOPMENT
33
4.1 Input Images
4.2 Color Extraction for Appearance Based
Room Recognition
4.2.1 RGB color space
4.2.2 RGB Components
4.3 Plotting Histogram
4.4 Histogram for Single Color Component
Image
4.5 Filtering
4.6 Rotation Invariant Property of Color
Histogram
4.7 Histogram Patterns To Represent Rooms
4.8 Data Sampling
4.9 Normalization
4.10 Artificial Neural Network for Room
Recognition
4.10.1 Multi Layer Perceptron
4.10.2 MLP Network Architecture
4.10.3 Activation Function
4.10.4 Learning Algorithm
4.11 Summary
34
35
35
36
43
45
50
52
54
55
56
57
58
58
60
61
67
x
5 SYSTEM IMPLEMENTATION
68
5.1 Hardware Implementation
5.1.1 The Mobile Robot
5.1.2 Main Controller
5.1.3 Catadioptric Sensor
5.1.4 Design Specifications
5.2 Software Implementation
5.2.1 Image Acquisition
5.2.2 Image Conversion To Single R,G, or B Color
Component
5.2.3 Color Histogram
5.2.4 Single Color Component Histogram
5.2.5 Histogram Filtering, Sampling and
Normalization
5.2.6 Artificial Neural Network
5.3 Summary
68
69
70
71
76
80
82
84
85
86
87
90
99
6 EXPERIMENTAL RESULTS AND DISCUSSION 100
6.1 Objectives of Experiment
6.1.1 Performance Comparison Based On
Different Color Component For Appearance
Based Room Recognition
6.1.2 Performance Comparison Based On
Different Sampling Rates For Appearance
Based Room Recognition
6.1.3 Performance Comparison Based On
Different Normalization Denominators For
101
101
101
xi
Appearance Based Room Recognition
6.1.4 Performance Comparison Based On
Different Network Settings For Appearance
Based Room Recognition
6.1.5 Real Time Recognition Performance
6.2 Experimental Setup
6.2.1 Experimental field
6.2.2 The Mobile Platform
6.2.3 Datasets
6.3 Experiment Procedure
6.3.1 Experiment 1: Performance Comparison
Based On Different Color Component For
Appearance Based Room Recognition
6.3.2 Experiment 2: Performance Comparison
Based On Different Sampling Rates For
Appearance Based Room Recognition
6.3.3 Experiment 3: Performance Comparison
Based On Different Normalization
Denominators For Appearance Based Room
Recognition
6.3.4 Experiment 4: Performance Comparison
Based On Different Network Settings For
Appearance Based Room Recognition
6.3.5 Experiment 5: Real Time Recognition
Performance
6.4 Experimental Results
6.4.1 Experiment 1: Performance Comparison
Based On Different Color Component For
Appearance Based Room Recognition
102
102
102
102
103
104
104
106
107
108
110
113
114
116
117
xii
6.4.2 Experiment 2: Performance Comparison
Based On Different Sampling Rates For
Appearance Based Room Recognition
6.4.3 Experiment 3: Performance Comparison
Based On Different Normalization
Denominators For Appearance Based Room
Recognition
6.4.4 Experiment 4: Performance Comparison
Based On Different Network Settings For
Appearance Based Room Recognition
6.4.5 Experiment 5: Real Time Recognition
Performance
6.5 Summary
119
121
124
126
130
7 CONCLUSION 132
7.1 Research Summary
7.2 Contributions
7.3 Limitations and Future Work
132
135
136
REFERENCES
138
xiii
LIST OF TABLES
TABLE NO. TITLE PAGE
5.1
6.1
6.2
6.3
6.4
6.5
6.6
6.7
Software environments for the system
Performance comparison based on different color
components
Performance comparison based on different sampling rates
Performance comparison of different normalization
denominators
Performance comparison based on different network
settings
Results for random real time testing
Detailed result for continuous real time testing
Result summary for continuous real time testing
81
117
119
122
124
127
129
130
xiv
LIST OF FIGURES
FIGURE NO. TITLE PAGE
1.1
1.2
3.1
3.2
3.3
Topological map of the indoor environment
Sample image of fish-eye field of view
Design Process of a Room Recognition System
Hardware requirements for the room recognition system
Software architecture for overall system
6
7
26
28
29
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
4.13
Overview of the room recognition development process
A Catadioptric Image of Room A
Converting from RGB color values to single color
component values
Flowchart showing how image in RGB color space is
converted into single color component image
Sample image of (a) original RGB color space (b) Red
color component (c) Blue color component (d) Green color
component
A 100-pixel image with different colors
The color histogram for image in Figure 4.6
The image in Figure 4.6 with only the R color component
Color intensity histogram for image in Figure 4.8
(a) A room image (b) The image with only its Red
component and (c) its corresponding color histogram
Flowchart showing the process of plotting the color
intensity histogram after the image has been reduced to an
8-bit color space.
A typical room image
A spike at the left side in all the histograms plotted for the
34
35
41
42
43
44
45
46
47
48
49
50
xv
4.14
4.15
4.16
4.17
4.18
4.19
4.20
4.21
4.22
4.23
4.24
room images
Thresholding histogram at a certain color intensity
(a) Before Thresholding (b) After thresholding
The rotation invariant property of a color histogram
Similar histogram pattern for Room A is obtained for
images taken in the same room irrespective of camera
location or image rotation.
Histogram for three different room images
Histogram patterns that represent different rooms
Histogram patterns sampled at different rates (a) at 200
bins per image (b) 100 bins per image (c) 50 bins per
image (d) 25 bins per image
Color histogram (a) before normalization and (b) after
normalization
A two-layer Multi Layer Perceptron
Binary sigmoid graph, range (0,1)
Local minimum and global minimum
Flowchart of the Backpropagation learning algorithm
50
51
52
53
54
54
55
56
59
60
62
66
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
BeMR : The robot used as test platform for the vision
system
Different catadioptric sensor designs
Catadioptric sensor design
Image taken with the catadioptric sensor
The Logitech Quick Cam Pro after modifications
Shiny ornament as reflector ball
The black cardboard used as shade
The sequence of mounting the reflector ball, webcam and
shade onto the aluminium rod and then mounting the rod
onto the BeMR.
The catadioptric sensor mounted onto the BeMR
Recommended distance between reflector ball and floor
(a) Camera fixed more than 3cm away from the reflector
(b) Image taken when the camera is more than 3cm away.
70
71
72
72
73
74
74
75
76
77
78
xvi
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
5.20
5.21
5.22
5.23
5.24
5.25
5.26
5.27
5.28
5.29
5.30
5.31
5.32
5.33
5.34
5.35
5.36
(a) Camera fixed less than 3cm away from the reflector
(b) Image taken when the camera is less than 3cm away.
(a) Camera fixed at 3cm away from the reflector (b) Image
taken when the camera is at the 3cm position.
Images taken without shade
Images taken with shade
Sub modules in the appearance based recognition software
GUI for the Save Picture algorithm
Original images are saved into folders
The last image saved can also be displayed
Original image conversion to single color component
image
Room A data for three different color components
Histogram of different color components for a room
Histograms plotted for different rooms
A histogram (a) before filtering and (b) after filtering
GUI for filtering, sampling and normalization
Histogram values saved as text files after filtering,
sampling and normalization
Histogram values (a) original (b) after filtering (c) after
sampling and normalization
An example shape of a network using Method A
An example shape of a network using Method B
An example shape of a network using Method C
GUI with function for network training, save net and load
net
Network is saved into their respective folders
Network can be retrieved using the ‘Load Net’ function
The network information available from the files
GUI with function for single image testing and result
display
GUI with automatic testing to recognize Training and Test
Set images
78
79
80
80
81
82
83
83
84
85
86
86
87
88
89
89
90
91
91
92
93
93
94
95
96
xvii
5.37 GUI for Real Time Room Recognition 98
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
Layout of the experiment field
Random path taken when capturing training images
Random path taken when capturing testing images
Illustration of the random locations for real time
recognition
Illustration of the locations for robot continuous real time
recognition
Number of correctly recognized images using inputs of
different color components.
Number of correctly recognized images using inputs
sampled with different sampling rates
Number of correctly recognized images using inputs
normalized with different denominator values
Number of correctly recognized images using different
network settings
Number of correct recognitions during random real time
testing
103
105
106
115
116
118
120
123
125
128
xviii
LIST OF ABBREVIATIONS
2D - Two dimensional
3D - Three dimensional
ANN - Artificial Neural Network
B - Blue
BeMR - Bluetooth enabled Mobile Robot
BP - Back Propagation
CAD - Computer Aided Design
CCD - Charge-coupled Device
DGPS - Differential Global Positioning System
G - Green
GPS - Global Positioning System
GUI - Graphical User Interface
MLP - Multi Layer Perceptron
MSE - Mean Square Error
NASA - National Aeronautics and Space Administrations
PCA - Principal Components Analysis
PDA - Personal Digital Assistant
R - Red
RGB - Red, Green, Blue
SA - Selective Availability
ZPR - Zero Phase Representation
1
CHAPTER 1
INTRODUCTION
Autonomous mobile robots designed to move freely in the world have the
same problems as humans when navigating. The world is a complex environment
and if robots only move around without ‘looking’ at where their action takes them,
they might get lost due to the imperfections in their moving mechanisms and the
environment. The research here therefore, focuses on a topological localization
strategy that employs vision and neural network for the robot to ‘see’ its
surroundings and then estimate its own location.
1.1 Challenges in Mobile Robot Navigation
In order for a mobile robot to perform its assigned tasks, it often requires a
representation of its environment, a knowledge of how to navigate in its
environment, and a method for determining its position in the environment. These
problems have been characterized by the three fundamental questions of mobile
robotics, which are “Where am I?”, “Where am I going?” and “How can I get there?’
(Leonard and Durrant-Whyte, 1991).
The first question is one of localization. The robot has to know where it is in
a given environment based on what it sees and what it was previously told. The
second and third questions are essentially those of specifying a goal and being able to
plan a path to achieve that goal. Therefore, finding a robust and reliable solution to
2
the first question of localization is a precursor to answering the remaining two
questions. This can be related with the example of going into a bookshop. You will
firstly need to know where you are standing (localization) and where is your
destination relative to your current position (identifying goal) so that you can plan
which direction you should take in order to reach your destination (path planning).
This is why many researchers (Duckett and Nehmzow , 2001; Cox and Wilfong,
1990; Fox, 1998; Borenstein et al.,1996) maintained that localization is one of the
most fundamental problems in mobile robotics.
1.2 An Overview of Robot Localization
Robot localization is the problem of estimating a robot’s pose relative to a
map of its environment (Fox et al., 1999). A wide variety of localization methods
have been proposed and a number of successful laboratory prototypes have been
developed. Some of these systems have been validated in larger environments,
generally consisting of enclosed areas within public buildings (Yamauchi and
Langley, 1997; Weiss and von Puttkamer, 1995; Burgard et al., 1998; Thrun et al.,
2000; Duckett and Nehmzow, 2000) and some attempts have been made for
localization in outdoor environments (Kweon and Kanade, 1991; Takeuchi and
Herbert 1998).
The localization issue can usually be categorized as being geometric or
topological (Andreasson and Duckett, 2004, Ulrich and Nourbakhsh, 2000).
Geometric approaches attempt to estimate the position of the robot as accurately as
possible (x, y, θ) with respect to the map’s coordinate system. Topological
localization gives a more abstract position estimate, for example “This is the coffee
room”.
The task of localization can again be divided into two sub-problems: position
tracking and global localization. In position tracking, a robot knows its initial
position and only has to accommodate small errors in its odometry as it moves (Fox
3
et al., 1999). The global localization is the ability to estimate the position of the robot
without knowledge of its initial location and the ability to relocalize if its position is
lost (Fox, 1998). Global localization hence, has to solve a much more difficult
localization problem, that of estimating its position from scratch. This includes the
kidnapped robot problem where the robot is all of a sudden transferred or
‘kidnapped’ to another location without the robot being aware of this.
Some researchers, however, categorize the localization tasks using different
terms. Yamauchi et al. (1998) divided the localization tasks to that of continuous
localization and place recognition. Continuous localization is like driving downtown
without getting lost which is similar to position tracking. In contrast, place
recognition is like waking up in a hotel room and trying to determine which city one
is in. In place recognition approaches, accurate coordinates are not needed and thus,
place recognition approaches are normally used in topological localization.
In determining its location, a robot needs access to two kinds of information.
First is the information or map, either gathered by the robot itself or supplied by an
external source during the training or initialization phase. This map specifies certain
features of the environment that are time-invariant and thus can be used to determine
a location. The second kind of information is the navigational information which the
robot gathers from its sensors during navigation. Generally, when a map or
information of the environment is available, the robot position is computed thanks to
a matching technique applied between currently observed part of the environment
and the global map.
1.3 Problem Background
Over the past few years, there has been tremendous scientific interest in
algorithms for estimating a robot’s location from sensor data. Many different
approaches have been introduced to handle various challenges in the robot
4
localization problem. A few challenges have been identified as dynamism of
environment, noise and errors, computational cost and ease of use.
A robot may be confronted with the problem of dynamic environment. Due to
the changes of furniture arrangement, the environment may look different from the
representation of a robot’s initial map. This raises the question of how to make a
robot localization method robust against such dynamic effects.
A general factor that complicates the robot localization is the existence of
noise and errors, particularly in the sensor readings. To perceive changes in the
environment, the robot has to sense repeatedly and often. However, knowledge
gained via sensing is incomplete, inaccurate and uncertain. One example is the use of
odometry sensors which count the revolutions that the wheels make while moving
and turning. The readings can be used to help in estimating the displacement over the
floor to give an indication of the location of the robot. Due to wheel slippage and
irregularities of the floor texture, the odometer readings may give inaccurate results.
With more revolutions, the cumulative error increases.
Another issue in localization system for robot is computational power. For a
robot to accurately determine its location, a detailed metric map will be a good input.
However, this type of maps requires extremely large memory and for the matching
algorithms to quickly determine a robot’s location in such a detailed map, a very fast
processing is needed. The computational cost is even higher if the robot is to do real
time image processing. Often, a two-dimensional map is used to avoid the
computation and space explosion that a three-dimensional representation may entail
(Kaelbling et al., 1998; Nourbakhsh et al., 1995; Schultz et al., 1999; Simmons and
Koenig, 1995). Statistical sampling is also used to alleviate this computational
burden at the cost of completeness (Dellaert et al., 1999).
Some approaches for robot localization face the problems in terms of ease of
use when a robot is transferred to a new domain. It will require a period of
initialization or retraining. Methods of localization which involve artificial
landmarks or beacons need a lot of additional engineering effort and modification of
5
the environment when a robot has to change or expand its environment. Localization
method which employs metric mapping will even need a new set of representation.
1.4 Project Objectives
The objective of this work is therefore to design a topological localization
strategy which can potentially help a robot recognize its surroundings despite the
dynamism, errors and computational limitations. Strategically, the system should
require no or little alteration when used in a new domain.
The objectives of this project are:
1. To develop a room recognition system using vision sensor and artificial
neural network
a. To develop hardware combination for color based sensing.
b. To develop an algorithm for room recognition using color histogram.
c. To implement Multi Layer Perceptron (MLP) network as recognition
engine.
2. To evaluate the performance of the room recognition system.
3. To propose a suitable setting for the room recognition system.
1.5 Outline of the Proposed Approach
The interest of this work is primarily in the place recognition aspect of the
topological localization problem. Putting together the issues to be solved, a room
recognition method employing vision and neural network for robot topological
localization is introduced. The proposed approach can be outlined as follows:
6
1. Adopting topological map as the world representation.
2. Room recognition (global localization) using visual information obtained
from camera.
3. Image matching using Artificial Neural Network.
4. Testing and evaluating this combination of sensor and techniques in an
unmodified indoor environment.
In this approach, the topological representation of the world is chosen
because creating such map takes little effort as there is no need to measure the
dimensions of the environment. By nature of their compactness, it has the potential
for representing environments which are several magnitudes larger than those which
can be tractably navigated using metric maps. Topological localization uses a graph
representation that captures the connectivity of a set of features in the environment.
Nodes of the graph represent locations while arcs represent the connectivity between
the locations. The topological map of a section of the P08 Robotics Building is
shown in Figure 1. This map and section of the building were used during testing.
Figure 1.1 Topological map of the indoor environment
A robust localization system requires a sensor that provides rich information
in order to allow the system to reliably distinguish between adjacent locations. For
this reason, a color web camera is used as the sensor. The images captured during the
training stage need to be representative of the environment. In order to keep the
number of necessary reference images low, the ‘fish-eye’ camera set-up which
provides a 360° circular image of its surroundings is implemented (Greiner and
Isukapalli, 1994; Betke and Gurvits, 1997). An example of fish-eye field of view
image is shown in Figure 1.2.
Research Room Pantry Robotic Lab
7
Figure 1.2 Sample image of fish-eye field of view
This vision-based localization system must be trained before it can be used in
an environment. During training, representative images are captured from the
environment and associated with corresponding locations. Classification of images is
done through the Multi Layer back propagation technique. At runtime, the acquired
image is compared to the map’s reference images. The location whose reference
image best matches the input image is then considered to be currently visible
location. Image matching is achieved through the same neural network.
For validation, this localization system is tested in an indoor environment. All
tests were performed in unmodified environments.
1.6 Outline of Thesis
The remainder of this thesis is organized in four main chapters. Chapter 2
reviews related works on robot localization and existing methods for vision-based
system focusing on appearance based method. Chapter 3 describes the development
methodology and guidelines in designing an appearance based room recognition
system for robot localization. Chapter 4 explains the theory and concept of the
8
appearance based room recognition system design. Chapter 5 describes the hardware
and software implementation of the system. Chapter 6 presents the experimental
results of the system performance analysis. Finally, Chapter 7 concludes the thesis
with summary of contributions and suggestions for future development.
Designing a robot that can navigate and operate in a real world environment
is a challenging task. The most fundamental competence it should have is the
localization capability. To localize, a robot needs to have external sensor information
and be able to give an estimate of its location. Localization is made complicated due
to a few factors such dynamism of environment, noise and errors, limitations in
computational resources and the ease of use of the system. Various techniques and
sensors have been introduced to tackle these issues. In this work, a vision-based
topological localization system is developed and investigated for an unmodified,
indoor environment.
137
enhancement, the system can be made to return several fuzzy states of
answers for Room A – for example, CONFIDENT RIGHT for (0.9, 0.01, 0.0)
; NOT CONFIDENT RIGHT for (0.4, 0.0,0.0) ; CONFUSED for (0.4, 0.4,
0.0) or WRONG for (0.0,0.9,0.0). When in NOT CONFIDENT or
CONFUSED states, a dynamic voting system can be designed to vote for a
confident answer to hopefully increase confidence and recognition rate.
4. The aluminium rod which serves as a mounting platform for the webcam,
reflector ball and shade is generating some occlusion factor in the system.
The rod can be replaced with a transparent glass cylinder. Ideally, the items
can be mounted within the transparent glass cylinder to avoid any occlusion
in the view. Then, the system can be fused with orientation tracking sensors
such as magnetic compass to determine a robot’s current heading position.
5. The software module can be modified to incorporate multiple color spaces for
future investigations in color and recognition.
6. Currently, the laptop used for this room recognition work is different from the
laptop used to control the BeMR in previous work by Yeong (2005). The
integration of both systems on the same laptop will be excellent to present a
multi purpose service robot with global localization capability that can be
controlled with a portable device.
The appearance based room recognition system successfully developed in this
work offers a partial solution to the problem of mobile robot localization in an
unmodified indoor environment. Many improvements can still be carried out to
improve the system with more capabilities and higher accuracy.
138
REFERENCES
Abe, Y., Shikano, M., Fukuda, T., Arai, F. and Tanaka, Y. (1999). Vision Based
Navigation System for Autonomous Mobile Robot with Global Matching.
Proceedings of the 1999 IEEE International Conference on Robotics and
Automation. May 10-15. Detroit, Michigan : IEEE, 1299-1304.
Andreasson, H. and Duckett, T. (2004). Topological Localization for Mobile Robots
Using Omni-directional Vision and Local Features. Proceedings of IAV 2004, the
5th IFAC Symposium on Intelligent Autonomous Vehicles. July 5-7. Lisbon,
Portugal.
Ascensio, J. R., Montiel, J. M. M. and Montano, L. (1999). Goal Directed Reactive
Robot Navigation with Relocation Using Laser and Vision. Proceedings of the
1999 IEEE International Conference on Robotics and Automation. May 10-15.
Detroit, Michigan : IEEE, 2905-2910.
Betke, M. and Gurvits, L. (1997). Mobile Robot Localization Using Landmarks.
IEEE Transactions on Robotics and Automation. 13(2): 251-263.
Borenstein, J., Everett, H.R. and Feng, L. (1996 ). “Where am I?" Systems and
Methods for Mobile Robot Positioning, University of Michigan : Technical
Report UM-MEAM-94-21.
Borenstein, J., Everett, H.R. , Feng, L., and Wehe, D. (1997). Mobile Robot
Positioning: Sensors and Techniques. Journal of Robotic Systems, Special Issue
on Mobile Robots. 14(4) : 231-249.
139
Brown, R. G. and Donald, B. R. (2000). Mobile Robot Self Localization Without
Explicit Landmarks. Algorithmica. 26 (3-4): 515-559.
Burgard, W., Fox, D., Hennig, D. and Schmidt, T. (1996). Estimating the Absolute
Position of a Mobile Robot Using Position Probability Grids. Proceedings of the
13th National Conference on Artificial Intelligence (AAAI ’96). August 4-8.
Portland, Oregon: AAAI, 896-901.
Burgard, W., Derr, A., Fox, D. and Cremers, A. B. (1998). Integrating Global
Position Estimation and Position Tracking for Mobile Robots: The Dynamic
Markov Localization Approach. Proceedings of the 1998 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS ’98). October 13-17.
Victoria BC : IEEE, 730-735.
Byrne, R. H., Klarer, P. R. and Pletta, J. B. (1992). Techniques for Autonomous
Navigation. Sandia National Laboratories (Albuquerque, N.M.): Sandia Report
SAND92-0457.
Cassinis, R., Grana, D. and Rizzi, A. (1996). Proceedings of the 1st Euromicro
Workshop on Advanced Mobile Robots (Eurobot'96). October 9-11.
Kaiserslautern, Germany : IEEE, 172-176.
Chan, W. K. (2003). Detection of Arching Fault In Underground Distribution Cable
Using Artificial Neural Network. Universiti Teknologi Malaysia : M.Eng Thesis.
Cox, I. J. (1989). Blanche: Position Estimation For An Autonomous Robot Vehicle.
Proceedings of the 1989 IEEE/RSJ International Workshop on Intelligent Robots
and Systems (IROS ’89). September 4-6. Tsukuba, Japan : IEEE, 432 – 439.
Cox, I. J. and Wilfong, G. ed. (1990). Autonomous Robot Vehicles. New York :
Springer-Verlag.
140
Dellaert, F., Fox, D., Burgard, W. and Thrun, S. (1999). Monte Carlo Localization
for Mobile Robots. Proceedings of the 1999 IEEE Conference on Robotics and
Automation. May 10-15. Detroit, Michigan : IEEE, 1322-1328.
Dellaert, F., Burgard, W., Fox, D. and Thrun, S. (2001). Using the
CONDENSATION Algorithm for Robust, Vision-Based Robot Localization.
State-of-the-Art in Content-Based Image and Video Retrieval, Kluwer Academic
Publishers.
Duckett, T. and Nehmzow, U. (2000). Performance Comparison of Landmark
Recognition Systems for Navigating Mobile Robots. Proceedings of the 17th
National Conference on Artificial Intelligence (AAAI 2000). July 30- August 3.
Austin, TX: AAAI.
Duckett, T. and Nehmzow, U. (2001). Mobile Robot Self Localization using
Occupancy Histograms and A Mixture of Gaussian Location Hyphotheses.
Robotics and Autonomous Systems. 34: 117-129.
Fausett, L. (1994). Fundamentals of Neural Networks : Architechture, Algorithm and
Applications. New Jersey: Prentice Hall, Inc.
Fox, D. (1998). Markov Localization: A Probabilistic Framework for Mobile Robot
Localization and Navigation. University of Bonn : Ph.D. Dissertation.
Fox, D., Burgard, W., Dellaert, F. and Thrun, S. (1999). Monte Carlo Localization:
Efficient Position Estimation for Mobile Robots. Proceedings of the 16th
National Conference on Artificial Intelligence. July 18-22. Orlando, Florida :
AAAI Press / The MIT Press, 343 - 349.
Gothard, B. M., Etersky, R. D. and Ewing, R.E. (1993). Lessons Learned on a Low-
Cost Global Navigation System for the Surrogate Semi-Autonomous Vehicle.
Proceedings of the SPIE Conference on Mobile Robots VIII. September 9-10.
Boston MA : SPIE, 2058: 258-269.
141
Greiner, R. and Isukapalli, R. (1994). Learning to Select Useful Landmarks.
Proceedings of the 12th National Conference on Artificial Intelligence (AAA1
’94). August 1-4. Seattle, Washington: AAAI, 1251-1256.
Gutmann, J.-S. and Schlegel, C. (1996). AMOS: Comparison of Scan Matching
Approaches for Self-Localization in Indoor Environments. Proceedings of the 1st
Euromicro Workshop on Advanced Mobile Robots (Eurobot'96). October 9-11.
Kaiserslautern, Germany : IEEE, 61-67.
Haykin, S. (1994). Neural Networks – A Comprehensive Foundation. New York :
Macmillan College Publishing Company, Inc.
Howard, D. and Beale, M. (2002).Neural Network Toolbox User’s Guide. Version 4.
Natick, M. A. : The Mathworks, Inc.
Ishiguro, H. and Tsuji, S. (1996). Image-based Memory of Environment.
Proceedings of the 1996 IEEE/RSJ International Conference on Intelligent
Robots and Systems. November 4-8. Osaka, Japan: IEEE, 634–639.
Jennings, C., Murray, D. and Little, J. J. (1999). Cooperative Robot Localization
with Vision-based Mapping. Proceedings of the 1999 IEEE International
Conference on Robotics and Automation. May 10-15. Detroit, Michigan : IEEE,
2659-2665.
Jogan, M. and Leonardis, A. (2003). Robust localization using an omnidirectional
appearance-based subspace model of environment. Robotics and Autonomous
Systems. 45 : 51–72.
Kaelbling, L., Cassandra, A. and Kurien, J. (1996). Acting Under Uncertainty:
Discrete Bayesian Models for Mobile Robot Navigation. Proceedings of the 1996
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
’96). November 4-8. Osaka, Japan : IEEE, 963-972.
Kaelbling, L., Littman, M. and Cassandra, A. (1998). Planning and Acting in
Partially Stochastic Domains. Artificial Intelligence. 101(1-2): 99-134.
142
Kak, A., Andress, K., Lopez-Abadia and Carroll, M. (1990). Hierarchical Evidence
Accumulation in the PSEIKI System and Experiments in Model-driven Mobile
Robot Navigation. Uncertainty in Artificial Intelligence. 5 : 353-369.
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems.
Transaction of the ASME – Journal of Basic Engineering. 82 : 35-45.
Kröse, B. J. A., Vlassis, N, Bunschoten, R. and Motomura, Y. (2001). A probabilistic
model for appearance-based robot localization. Image and Vision Computing.
6(19): 381–391.
Kweon, I. S. and Kanade, T. (1991). Extracting Topographic Features for Outdoor
Mobile Robotics. Proceedings of the 1991 IEEE International Conference on
Robotics and Automation. April 9-11. Sacramento CA : IEEE, pp 1992-1997.
Langley, P., Pfleger, K. and Sahami, M. (1997). Lazy Acquisition of Place
Knowledge. Artificial Intelligence Review on Lazy Learning. 11(1-5) : 315-342.
Leonard, J. and Durrant-Whyte, H. (1991). Mobile Robot Localization by Tracking
Geometric Beacons. IEEE Transactions on Robotics and Automation. 7(3) : 376-
382.
Leonardis, A. and Bischof, H. (2000). Robust recognition using eigenimages.
Computer Vision and Image Understanding— Special Issue on Robust Statistical
Techniques in Image Understanding. 78 (1): 99–118.
Lippmann, R. P. (1987). An introduction to computing with neural nets. IEEE
Acoustics,
Speech and Signal Processing Magazine. 4(2): 4-22.
Masters, T. (1993). Practical Neural Network Recipes in C++. San Diego, C.A.:
Academic Press, Inc.
143
Moravec, H. P. (1988). Sensor Fusion in certainty grids for mobile robots. AI
Magazine. 9 (2) : 61-74.
Muller, B. and Reindhart, J. (1990). Neural Networks – An Introduction. Berlin :
Springer-Verlag.
Murase, H. and Nayar, S. K. (1997). Detection of 3D objects in cluttered scenes
using hierarchical eigenspace. Pattern Recognition Letters. 18 (4): 375–384.
Nayar, S. K., Nene, S. A. and Murase, H. (1996). Subspace methods for robot vision.
IEEE Transactions on Robotics and Automation. 12(5) : 750–758.
Negenborn, R. (2003). Robot Localization and Kalman Filters On finding your
position in a noisy world. Utrecht University : M.Sc Thesis.
Nourbakhsh, I., Powers, R. and Birchfield, S. (1995). Dervish An Office Navigating
Robot. AI Magazine. 16(2) : 53-60.
Ramos, F. T., Upcroft, B., Kumar, S. and Durrant-Whyte, H., F. (2005). A Bayesian
Approach for Place Recognition. Proceedings of the IJCAI Workshop on
Reasoning with Uncertainty in Robotics. July 30. Edinburgh, Scotland.
Schiele, B. and Crowley, J. L. (1994). A Comparison of Position Estimation
Techniques Using Occupancy Grids. Proceedings of the 1994 IEEE International
Conference on Robotics and Automation. May 8-13. San Diego, USA : IEEE,
1628-1634.
Schultz, A., Adams, W. and Yamauchi, B. (1999). Integrating Exploration,
Localization, Navigation and Planning with a Common Representation.
Autonomous Robots. 6(3): 293 - 308.
Shaffer, G., Gonzalez, J. and Stentz, A. (1992). Comparison of Two Range-based
Estimators for a Mobile Robot. Proceedings of the SPIE Conference on Mobile
Robots VII. November 18. Boston, USA : SPIE, 1831: 661-667.
144
Simmons, R. and Koenig, S. (1995). Probabilistic Robot Navigation in Partially
Observable Environments. Proceedings of the 14th International Joint
Conference on Artificial Intelligence. July . San Mateo, California : Morgan
Kaufmann, 1080 -1087.
Takeuchi, Y. and Hebert, M. (1998). Evaluation of Image Based Landmark
Recognition Techniques. Carnegie Mellon University (Pittsburgh, P.A.): CMU
Technical Report CMU-RI-TR-98-20.
Thrun, S. (1998). Finding Landmarks for Mobile Robot Navigation. Proceedings of
the 1998 IEEE International Conference on Robotics and Automation. May 16-
20. Leuven, Belgium: IEEE, 958-963.
Thrun, S., Beetz, M., Bennewitz, M., Burgard, W., Cremers, A., Dellaert, F., Fox, D.,
Hahnel, D., Rosenberg, C., Roy, N., Schulte, J. and Schulz, D. (2000).
Probabilistic Algorithms and the interactive Musuem Tour Guide Robot Minerva.
International Journal of Robotics Research. 19 : 972-999.
Ting, H. N. (2002). Speech Analysis and Classification Using Neural-Networks for
Computer-Based Malay Speech Theraphy. Universiti Teknologi Malaysia :
M.Eng Thesis.
Ulrich, I. and Nourbakhsh, I. (2000). Appearance-Based Place Recognition for
Topological Localization. Proceedings of the 2000 IEEE International
Conference on Robotics and Automation. April 24-28. , San Francisco, CA :
IEEE, 1023-1029.
Weiss, G., Wetzler, C. and von Puttkamer, E. (1994). Keeping Track of Position and
Orientation of Moving Indoor Systems by Correlation of Rangefinder Scans.
Proceedings of the 1994 IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS ’94). September 12-16. Munich, Germany: IEEE,
595 – 601.
145
Weiss, G. and von Puttkamer, E. (1995). A Map Based On Laser Scans Without
Geometric Interpretation. Proceedings of the 4th International Conference on
Intelligent Autonomous Systems. March 27 - 30. Karlsruhe, Germany : IOS, 4:
403-407.
Yamauchi, B. (1996). Mobile Robot Localization in Dynamic Environments Using
Dead Reckoning and Evidence Grids. Proceedings of the 1996 IEEE
International Conference on Robotics and Automation (ICRA ’96). April 22-28.
Minneapolis, MN : IEEE, 1401-1406.
Yamauchi, B. and Langley, P. (1997). Place Recognition in Dynamic Environments.
Journal of Robotic Systems, Special Issue on Mobile Robots Special Issue. 14 (2):
107-120.
Yamauchi, B., Langley, P., Schultz, A. C., Grefenstette, J., and Adams, W. (1998).
Magellan: An integrated adaptive architecture for mobile robotics. Institute for
the Study of Learning and Expertise (Palo Alto, CA) : Technical Report 98-2.
Yeong, C. F. (2005). Development and Evaluation of Various Modes of Human
Robot Interface for Mobile Robot. . Universiti Teknologi Malaysia : M.Eng
Thesis.
Top Related