Prof. Marco Diani

University of Pisa , Italy
Email: m.diani@iet.unipi.it

Image processing methods in image fusion

Abstract

Image fusion represents a powerful means for integrating information from different sources. In the past decades it has been applied to different fields such as pattern recognition, visual enhancement, object detection, area surveillance and so forth. Existing image fusion techniques rely on different approaches, the most widely encountered being based on multi-resolution decomposition or on appropriate pixel-level weighted averaging. Fusion techniques apply advanced image processing concepts that are used to define or to improve the fusion strategy. For example, images to be fused can be preliminarily filtered for noise reduction; image co-registration is often needed to spatially superimpose corresponding pixels in the two images; multiresolution analysis is required in many fusion methods; optimal weights must be determined in pixel-level average fusion. As one can easily argue, lot of material could be presented in this context, but only a few aspects will be expanded in more detail to fit the time devoted to this course . The material, which will focus on image processing aspects encompassing different image fusion approaches, will be presented with reference to two specific image fusion applications: human vision enhancement in bad visibility and video frame fusion for object detection. The first application will be used to introduce the important topic of multiresolution decomposition and to illustrate its possible application to a case study. Multi-resolution decomposition is the basic tool for most of the existing image fusion schemes and includes, for example, the widely used pyramid transform and wavelet transform. In the second application, the problem of detecting moving objects within a block of consecutive frames is considered as an example of pixel level weighted averaging. The strategy to derive the optimal weights and to integrate the different frames in a video sequence will be presented, and the extension to the fusion of results from more sensors will be illustrated.

Biography

Marco Diani was born in Grosseto , Italy , in 1961. He received his Laurea degree (cum Laude) in Electronic Engineering from the University of Pisa, Italy, in 1988. In the period 1989-1991 he was with the SMA ( Florence ) where he was involved in the project of airborne pulsed Doppler radars. In 1991 he joined the Department of Information Engineering of the University of Pisa where he is currently an Associate Professor. He teaches the courses of Random Signals and Design and Simulation of Remote Sensing Systems at the Faculty of Engineering of the University of Pisa . His research interests are in the fields of signal processing, image processing and remote sensing. He has published over 50 papers in these areas. In the last few years his research studies focused on hyperspectral image analysis for classification and object detection, on IR video processing for object detection and tracking and on image fusion for enhancing object perception in bad visibility conditions. Since 1991 he has been a member of IEEE.

Dr. Alfonso Farina

SELEX-Sistemi Integrati, Roma , Italy
Email: afarina@selex-si.com

Prof. Luigi Chisci

University of Firenze , Italy
Email: chisci@dsi.unifi.it

Prof. Antonio De Maio

University of Napoli , Italy
Email: ademaio@unina.it

Eng. Alessio Benavoli

University of Firenze, Italy

Knowledge-Based Radar Signal and Data Processing

Abstract

Knowledge-Based (KB) techniques extensively resort to a-priori information to restore the radar performance in very hostile conditions [1-6]. They are based on the real-time exploitation of a-priori knowledge concerning the environment surrounding the radar. In fact environmental context is the key to efficient adaptation: sensors like humans might benefit from the context. Examples of a-priori knowledge are Digital Terrain Elevation Models (DTEM), previous look data, Geographic Information Systems (GIS's), roadways (to highlight sectors of surveillance where moving cars or vehicles might be present), background of air/surface traffic, system calibration information, et cetera. The ultimate goal is to make the radar an intelligent device, such that it is capable of developing cognition of the surrounding environment [7-9]. Otherwise stated the environment, in which the radar system operates, acts as a teacher and the radar can become more expert with time by learning from the environment.

In this tutorial some aspects of KB radar signal and data processing will be illustrated. A particular emphasis will be given to KB Constant False Alarm Rate (CFAR) processors, KB Space Time Adaptive Processing (STAP) and KB target tracking.

Part I – Introduction to KB systems for radar applications 

An historical background concerning the evolution and the taxonomy of modern surveillance radars will be given. Then the use of artificial intelligence and KB systems in radar applications will be deeply explored and motivated.

Part II – KB Radar Signal Processing 

Particular attention will be devoted to the design and the analysis of CFAR detectors exploiting KB processing techniques [10,11]. CFAR algorithms composed of two stages will be presented [11]. The former is a KB data selector which, exploiting the a-priori information provided by a GIS, chooses the training samples for threshold adaptation. The latter is a conventional CFAR processor. The performance of these KB schemes will be shown in the presence of real radar data, collected by the McMaster IPIX radar, also in comparison with other common CFAR detectors such as the Cell Averaging CFAR (CA-CFAR) and the Ordered Statistics CFAR (OS-CFAR). The noticeable performance improvements, obtained suitably exploiting the a-priori information available about the sensed environment, will be also discussed.

It follows an overview on the combined use of a-priori information and adaptive signal processing techniques for the design and the analysis of a Knowledge-Aided (KA) radar receiver for Doppler processing [12]. Precisely it will be described a KA radar detector composed of three elements: a geographic map based training data selector, which exploits some a-priori information concerning the topography of the observed scene, a data-adaptive selector which removes dynamic outliers from the training data, and an adaptive radar detector which performs the final decision about the target presence. The performance of this KA algorithm will be shown both on simulated as well as on real radar data.

Some insights will be given on the problem of KB adaptive filtering algorithms for the cancellation of heterogeneous clutter [13]. After a short introduction concerning the application of the Recursive Least Squares (RLS) technique [14] for the rejection of unwanted clutter, the design of modified RLS filtering procedures, accounting for the spatial variation of the clutter power as well as of the disturbance covariance persymmetry property, will be presented. Then the concept of Knowledge-Based RLS will be introduced and its potentiality stressed [14].

An in-depth overview on the challenging KA airborne STAP will be provided [1-6,15-17] . Indeed it is well known that conventional STAP schemes suffer significant performance degradations when confronted with non-stationary terrain clutter and dense target backgrounds. KB is a viable mean to operate in these hostile scenarios both in a direct as well as in an indirect fashion. As to the former approach Bayesian filters will be presented. As to the latter it will be introduced a KA processor exploiting an hybrid training selection algorithm, which pre-screens secondary data through the use of terrain information from the United States Geological Survey (USGS) [18]. The performance of this last approach will be shown on measured airborne radar data, obtained from the Multi-Channel Airborne Radar Measurements (MCARM) program.

Part III - KB radar data processing 

First a brief overview of the architecture of a traditional (non KB) tracking system will be given [19]. This architecture consists of data processing blocks for filtering and estimation, data association, track formation-maintenance and target classification. To improve the performance of the traditional tracking system, various types of a-priori information (e.g. DTEM, GIS, meteorological maps; target characteristics; location of discretes etc.) can be exploited in the different blocks of the tracking system. In particular, all the information from the various maps can be fused together in order to build, and possibly update in real-time, a new map which provides a criticality index (which measures the local clutter density) for each cell in the radar surveillance region. As far as filtering and estimation [20] is concerned, equality and/or inequality constraints arising from environmental/target information (e.g. aircrafts in airports, vehicles on roads, ships along the coast, etc.) can be employed in order to improve the estimation accuracy. In this context, a brief overview on constrained estimation and filtering (e.g. constrained batch estimation [24,25], particle filtering [21,22], projection methods, receding-horizon estimation [23] etc.) will be presented. For the purpose of data association, it is important to exploit the criticality index of the track location to tune parameters of the data association algorithm (e.g. gate size and shape, clutter density etc.) for best performance. In the algorithms for track formation and maintenance, the criticality index can be used to appropriately tune the related parameters (e.g. M and N for the M/N logic) as well as to apply suitably modified promotion and deletion strategies for tracks located in high clutter density regions. Another relevant task of the tracking system is target classification, i.e. determination of the type of target (e.g. civil aircraft, military aircraft, helicopter, ship, raft etc.). To this end, a-priori information on the target characteristics (speed, maneuverability, etc.), on the terrain characteristics (topography) as well as on the coupling between target and terrain characteristics can be employed by means of suitable rules to help classification.

Finally, some applications of KB systems will be overviewed in order to show, both via simulation and experimental results, the performance improvement with respect to traditional systems. More specifically, the attention will be devoted to:

•  tracking ground targets with road constraints [ 26,27]

•  A-SMGCS (tracking aircrafts in airports) [28]

•  Ship-borne radar tracking [29]

•  tracking of ballistic targets.

LECTURE LIST OF CONTENTS

Introduction to KB systems for radar applications

1. Historical Background
2. Evolution and taxonomy of modern surveillance radars
3. Artificial intelligence and KB systems
4. Why KB in radar signal processing ?
5. Why KB in radar data processing ?

PART II

KB Radar Signal Processing

1. KB-CFAR detection for radar systems
   • Conventional CFAR algorithms for range processing
   • Exploiting a-priori information: GIS-based CFAR detectors
   • Performance analysis on real radar data
   • Joint use of KB and data-dependent criteria for outliers excision
   • Design and analysis of a KB detector for Doppler processing
   • Analysis on measured McMaster IPIX data
2. KB Clutter Filtering
   • Preliminaries on Recursive Least Squares (RLS) algorithms for clutter suppression
   • Normalized RLS algorithms
   • KB-RLS clutter filtering
3. KB-STAP
   • Direct and indirect KB-STAP
   • Exploiting knowledge of the clutter ridge in airborne radar: KB-GLRT
   • Demonstration of knowledge aided STAP using measured airborne data from MCARM project
4. Conclusions

KB radar data processing

1. 1. Traditional Tracking Systems
   • Filtering and estimation
   • Data association
   • Track formation
   • Target classification
2. Knowledge-based Tracking Systems
   • A-priori information
   • Constrained filtering and estimation
   • KB data association and gating
   • KB track formation and maintenance
   • KB target classification
3. Applications
   • Ground target tracking
   • A-SMGCS
   • Ship-borne radar tracking
   • Ballistic target tracking
4. Conclusions

PART IV

REFERENCES

[1] W. L. Melvin, M. Wicks, P. Antonik, Y. Salama, P. Li, and H. Schuman, “Knowledge-Based Space-Time Adaptive Processing for Airborne Early Warning Radar,” IEEE AES Systems Magazine, Vol. 13, No. 4, pp. 37-42, April 1998.

[2] R. Adve, P. Antonik, W. Baldygo, C. Capraro, G. Capraro, T. Hale, R. Schneible, and M. Wicks, “Knowledge-base Application to Ground Moving Target Detection, ” AFRL-SN-TR-2001-185, In-House Technical Report, September 2001.

[3] A. Farina, G. Capraro, H. Griffiths, and M. Wicks, “Knowledge-Based Radar Signal & Data Processing,” Research and Technology Organisation (RTO) Lecture Series 233, NATO, Rome, Italy, November 2003.

[4] J. R. Guerci, “Knowledge-Aided Sensor Signal Processing and Expert Reasoning,” Proc. of the 2002 Workshop on Knowledge-Aided Sensor Signal Processing and Expert Reasoning (KASSPER), Washington D.C. , April 2002, USA .

[5] D. J. Zywicki, W. L. Melvin, G. A. Showman, and J. R.. Guerci, “STAP Performance in Site-Specific Clutter Environments,” Proc. of the 2003 IEEE Aerospace Conference, Vol. 4, pp. 1-16, March 2003.

[6] “KASSPER: Knowledge-Aided Sensor Signal Processing and Expert Reasoning,” KASSPER Workshops, Conference Proceedings and Data, Airforce Research Laboratory, DARPA, April 2004, USA .

[7] V. C. Vannicola and J. A. Mineo, “Applications of Knowledge-Based Systems to Surveillance,” Proc. of the 1988 IEEE National Radar Conference, pp. 157-164, April 1988.

[8] V. C. Vannicola, L. K. Slaski, and G. J. Genello, “Knowledge Based Resource Allocation for Multifunction Radars,” Proc. of the 1993 SPIE Conference on Signal and Data Processing of Small Targets, Vol. 1954, pp. 410-425, April 1993.

[9] S. Haykin, “Radar vision,” Proc. of the Second International Specialist Seminar on the Design and Application of Parallel Digital Processors, pp. 75-78, April 1991.

[10] W. Baldygo, R. Brown, M. Wicks, P. Antonik, G. Capraro, and L. Hennington, “ Artificial Intelligence Applications to Constant False Alarm Rate (CFAR) Processing ,” Proceedings of the 1993 National Radar Conference, pp. 275-280, April 1993.

[11] A. De Maio, A. Farina, and G. Foglia, “Design and Experimental Validation of Knowledge-Based CFAR Detectors ,” submitted to IEE Proc. on Radar and Sonar Navig.

[12] E. Conte, A. De Maio, A. Farina, and G. Foglia, “Design and Analysis of a Knowledge-Aided Radar Detector for Doppler Processing ,” in press on IEEE Trans. on Aerospace and Electronic Systems.

[13] H. L. Van Trees, “Optimum Array Processing,”, Part IV of Detection, Estimation, and Modulation Theory, John Wiley & Sons, Inc., New York, 2002.

[14] A. De Maio, A. Farina, and G. Foglia, “Knowledge-Based Recursive Least Squares Techniques for Heterogeneous Clutter Suppression,” submitted to IEE Proc. on Radar and Sonar Navig.

[15] D. D. Weiner, G. T. Capraro, and M. C. Wicks, “An Approach for Utilizing Known Terrain and Land Feature Data in Estimation of the Clutter Covariance Matrix,” Proc. of the 1998 IEEE Radar Conference, pp. 381-386, May 1998.

[16] C. T. Capraro, G. T. Capraro, D. D. Wiener, M. C. Wicks, “Knowledge Based Map Space Time Adaptive Processing (KBMapSTAP),” Proc. of the International Conference on Imaging Science, Systems, and Technology, pp. 533-538, Las Vegas, NV, June 2001.

[17] C. T. Capraro, G. T. Capraro, D. D. Wiener, M. C. Wicks, W.J. Baldygo, “Improved STAP Performance using Knowledge-Aided Secondary Data Selection,” Proc. of the 2004 IEEE National Radar Conference, pp. 361-365, Philadelphia, PA, April 2004.

[18] C. Capraro, G. T. Capraro, A. De Maio, A. Farina, and M. Wicks, “Demonstration of Knowledge Aided STAP using Measured Airborne Data,” accepted for publication on IEE Proc. on Radar and Sonar Navig.

[19] Y. Bar-Shalom, X. Rong Li, “Multitarget-Multisensor Tracking: Principles and Techniques,” YBS Publishing, Storrs , CT , 1995.

[20] Y. Bar-Shalom, X. Rong Li, T. Kirubarajan,, “Estimation with applications to tracking and navigation,” John Wiley & Sons, 2001.

[21] A Doucet, J. F. G. De Freitas, and N. J. Gordon, “Sequential Monte Carlo Methods in Practice,” New York : Springer-Verlag, 2001.

[22] B. Ristic, S. Arulampalam , and N. Gordon, “Beyond the Kalman filter: particle filters for tracking applications,” Artech House, 2004.

[23] C.V. Rao, J.B. Rawlings, D.Q. Mayne, “ Constrained State Estimation for Nonlinear Discrete-Time Systems: Stability and Moving Horizon Approximations, ” IEEE Trans. on Automatic Control, Vol. 48, No 2, February 2003.

[24] A. Benavoli, L. Chisci, A. Farina, L. Ortenzi, G. Zappa, “Hard-constrained vs. Soft-constrained Parameter Estimation”, accepted to IEEE Trans. On Aerospace and Electronic Systems, 2006.

http://www.dsi.unifi.it/users/chisci/recent_publ.htm

[25] A. Benavoli, L. Chisci, A. Farina, “Estimation of Constrained Parameters with Guaranteed MSE Improvement,” submitted to IEEE Trans. on Signal Processing, July 2005. http://www.dsi.unifi.it/users/chisci/recent_publ.htm

[26] T. Kirubarajan, Y. Bar-Shalom, K.R. Pattipati, I. Kadar, “ Ground target tracking with variable structure IMM estimator,” IEEE Trans. on Aerospace and Electronic Systems Vol. 36,  No. 1, pp.26-46,  Jan. 2000

[27] M. S. Arulampalam, N. Gordon, Orton, B. Ristic, “A variable structure multiple model particle filter for GMTI tracking”. Proc. of Fusion 2002, Vol. 2, pp. 927-934, Sunnyvale, CA, USA, 2002.

[28] A. Farina, L. Ferranti, G. Golino,“ Constrained tracking filters for A-SMGCS,” Proc. of Fusion 2003, Vol. 1,  pp. 414 – 421, Cairns , Australia , 2003.

[29] A. Benavoli, L. Chisci, A. Farina, S. Immediata , L. Timmoneri, “Knowledge-Based System for Multi-Target Tracking in a Littoral Environment,” accepted for publication on IEEE Trans. on Aerospace and Electronic Systems, September 2004.
http://www.dsi.unifi.it/users/chisci/recent_publ.htm

Biography

Alfonso FARINA (Fellow of IEEE, Fellow of Royal Academy of Engineering, and Fellow of IEE) received his doctor degree in electronic engineering from the University of Rome (I) in 1973. In 1974 he joined Selenia, now SELEX-Sistemi Integrati, where he is a manager (since May 1988). He was Scientific Director in the Chief Technical Office. Today he is director of the Analysis of Integrated Systems Group. In his professional life Alfonso has provided technical contributions to detection, signal, data & fusion, image processing for radar systems. He has provided leadership in many projects – also conducted in the international arena – in surveillance for ground and naval applications, in airborne early warning and in imaging radar. Since 1979, he has also been Professore Incaricato of Radar Techniques at the University of Naples ; in 1985 he was appointed Associate Professor. He is the author of more than 350 peer reviewed technical publications and the author of books and monographs: Radar Data Processing (Vol. 1 and 2) (translated in Russian and Chinese) , 1985-1986  ; Optimised Radar Processors, 1987 ; Antenna Based Signal Processing Techniques for Radar Systems, 1992 . He has written the Chapter 9 on “ECCM techniques” in the Radar Handbook (2 nd Edition 1990), edited by Dr. M. I. Skolnik of Naval Research Laboratory. He has been session chairman at many international radar conferences. He uses to lecture at universities and research centres in Italy and abroad; He also frequently gives tutorials at the Intl. Radar Conferences on signal, data and image processing for radar; in particular on multi-sensor fusion, adaptive signal processing, space time adaptive processing (STAP) and detection . In the 1987 He received the Radar Systems Panel Award of IEEE-AESS for development of radar data processing techniques . He is the Italian representative at the International Radar Systems Panel of IEEE-AESS. He is the Italian industrial representative ( Panel Member at Large) at the SET (Sensor and Electronic Technology) of RTO (Research Technology Organisation) of NATO. He has been in the BoD of the International Society for Information Fusion (ISIF). He is the executive chair of the International Conference on Information Fusion, Fusion 2006 ( Florence , 10-13 July 2006). He has been nominated Fellow of IEEE with the following citation: "For development and application of adaptive signal processing methods for radar systems." Recently he has been nominated international fellow of the Royal Academy of Engineering ( UK ). He is a referee of numerous publications submitted to several Journals of IEEE, IEE, Elsevier, etc., He has also cooperated with the editorial board of ECEJ (Electronics & Communication Engineering Journal) of IEE. More recently, Alfonso has served as a member in the Editorial Board of Signal Processing (Elsevier). Also he has been the co-guest editor of the Signal Processing (Elsevier) special issue on “New trends and findings in antenna array processing for radar”, September 2004. He is the co-recipient of the following best paper awards: entitled to Mr. B. Carlton, of IEEE Trans. on Aerospace and Electronic Systems for the years 2001 and 2003; also of the International Conference on Fusion 2005. Alfonso has been the leader of the team that received the 2002 AMS CEO award for Innovation Technology. Alfonso has been the co-recipient of the AMS Radar Division award for Innovation Technology in 2003. Moreover, Alfonso has been the co-recipient of the 2004 AMS CEO award for Innovation Technology. Recently, He has been the leader of the team that has won in 2004 the 1 st prize award for Innovation Technology of Finmeccanica ( Italy ). This award context has seen the submission of more than 320 projects. This award has been set for the first time in 2004.

Luigi Chisci was born in Florence , Italy , in 1959. He received the degree in Electrical Engineering in 1984 from the University of Florence and the Ph.D. in Systems Engineering in 1989 from the University of Bologna . In 1990 he joined the Department of Systems and Computer Science of the University of Florence as a research associate. From 1992 to 1993, he was associate professor at the University of Pisa . Since 1993, he has been at the University of Florence as associate professor first and then as full professor since 2004. His educational and research career have been in the area of control and systems enginering. His research interests have spanned over adaptive control and signal processing, algorithms and architectures for real-time control and signal processing, recursive identification, filtering and estimation, predictive control. His current interests concern control of constrained systems, constrained estimation and modeling/control of telecommunication networks. He has co-authored about 90 papers of which over 30 on international journals. Since 2000 he has served the Conference Editorial Board of the IEEE Control Systems Society as an Associate Editor.

Antonio De Maio was born in Sorrento , Italy , on June 20, 1974. He received the Dr. Eng. degree (with honors) in 1998 and the Ph.D. degree in information engineering in 2002, both from the University of Naples Federico II, Naples , Italy . Currently he is an Assistant Professor at the University of Naples Federico II .Accomplishments/Awards/Interests: Dr. De Maio's current research lies in the field of statistical signal processing, with emphasis on radar detection and multiple access communications.

Alessio Benavoli was born in Arezzo, Italy, in 1979. He received the MS degree in Computer and Control Engineering in 2004 from the University of Firenze, Italy. Since 2005 he has been pursuing the Ph.D. in Control Engineering at the University of Firenze and currently works on constrained estimation, data fusion, belief theory and tracking.

Prof. Mieczyslaw M. Kokar

Northeastern University , Boston , MA , USA
Email: mkokar@ece.neu.edu

Ontology Based Situation Awareness and High Level Fusion: Methods and Tools

Abstract

In this tutorial the participants will learn:

• how “situation” can be formalized
• what an ontology is
• how to represent ontologies in the OWL language using an OWL tool
• how to check consistency of an ontology
• how to build rules on top of an ontology
• how to use a general reasoning tool to monitor or query situations
• what kind of situation-related reasoning can be performed and how

Higher-level fusion involves estimation of abstract entities - sometimes called “situations” - that can be represented as relations among objects, both physical and conceptual. Unlike features of physical objects, features of relations are not directly measured by sensors. Instead, the existence of a relation is derived from a domain theory relevant to a specific scenario.

This tutorial will cover both theoretical and practical aspects of situation awareness and high-level information fusion. First, a motivational example will be given to demonstrate the importance of relations and to introduce the concept of situation. This will be followed by a presentation of some methodological techniques and some technologies that are needed for establishing an ontological approach to higher level processing. The notion of ontology will be introduced in theoretical, computational and practical terms. Examples of specific ontologies will be discussed using both a graphical representation language (UML – the Unified Modeling Language) and an evolving standard language used for communicating ontologies and annotations as well as for processing and fusion of semantic annotations (OWL – Web Ontology Language). An overview of OWL constructs will be provided using Protégé, the most popular tool for editing ontologies, and the ezOWL graphical plugin. Other tools will also be demonstrated in the context of a methodology for ontology engineering. Situation awareness and high-level information fusion will be discussed using an illustrative example.

Outline:

Hour 1: From Level 1 to High Level fusion. The notions of “situation” and “situation awareness.” A motivational example.

Hour 2: Ontologies and Web Ontology Language (OWL). Ontology engineering and ontology tools.

Biography

Professor Mieczyslaw M. Kokar is with the Department of Electrical and Computer Engineering at Northeastern University in Boston . His technical research interests include Ontology-Based Computing, Information Fusion, Self-Controlling Software and Modeling Languages. In Information Fusion Dr. Kokar's primary interests are in higher-level fusion, situation awareness, synthesis of information fusion algorithms from specifications within the framework of formal methods, the use of symbolic information in the process of designing fusion systems. In Self-Controlling Software Dr. Kokar's research interest is focused around the specification and design of self-controlling software using the control theory metaphor. In the area of modeling languages, Dr. Kokar works on developing formal specifications of the OWL language, ontology development, annotation of information, logical reasoning about OWL annotated information, consistency checking, formalization of the UML language, consistency checking of UML models vs. UML Metamodel and of UML Metamodel vs. MOF (the Meta-Object Facility). Dr. Kokar teaches various graduate courses in software engineering, formal methods and artificial intelligence. His research has been supported by DARPA, NSF, AFOSR and other agencies. Dr. Kokar is a member of the Editorial Board of Information Fusion: An International Journal on Multi-Sensor, Multi-Source Information Fusion. He has an M.S. and a Ph.D. in computer systems engineering from Wroclaw Institute of Technology, Poland . He is a senior member of the IEEE and member of the ACM.

More information about Professor Kokar can be found at his web site: http://www.coe.neu.edu/~kokar

Prof. Yaakov Bar-Shalom

Distinguished IEEE AESS Lecturer, Univ. of Connecticut , USA
Email: ybs@ee.uconn.edu

Prof. T. Kirubarajan

McMaster Univ. , Canada
Email: kiruba@mcmaster.ca

Multitarget Tracking and Multisensor Fusion, PART II

Abstract

(See AM1)

[mttvsiapf06: 8.3,8.4] Common origin testing and fusion of local tracks.

[286vCDC04] Multisensor track-to-track association for tracks with dependent errors.

[267] Tracking and data association with kinematic and classification information.

Software demo for NetCenTrack : Network-Centric Multisensor-Multitarget Tracking, Fusion and Sensor Management Testbed. This presentation will demonstrate the distributed (network-centric) tracking and fusion testbed that is under development at McMaster University .

The testbed, which is capable of running on multiple computers across a TCP/IP network and fusing raw data and tracks from multiple heterogeneous sources, consists of modules for scenario generation, data/signal processing, tracking, fusion, sensor management and performance evaluation. The testbed is designed in a modular fashion so that an end-to-end system can be built by combining different components from different stages -- one needs only the data interface between any two consecutive stages of the system.

For example, the tracker module (e.g., based on the IMM estimator, particle filter and the MHT algorithm) can be specified by changing a library file that meets the data interface requirements.

The software demo will show examples of scenario definition (with active as well as passive sensors), tracker integration (using the Kalman filter, IMM estimator and the particle filter), distributed network definition and fusion and performance evaluation. Sensor resource management and signal processing modules are under development and a working version of them will be demonstrated as well.

The course is based on the book Multitarget-Multisensor Tracking: Principles & Techniques by Y. Bar-Shalom and X.R. Li (YBS Publishing, 1995) and additional notes.

Background text:

Y. Bar-Shalom, X. R. Li and T. Kirubarajan, Estimation with Applications to Tracking and Navigation: Algorithms and Software for Information Extraction , Wiley, 2001.

Biography

(See also AM1)

Dr. Subrata Das

Charles River Analytics, Inc., Cambridge , MA , USA
Email: sdas@cra.com

An Integrated Approach to Data Fusion and Decision Support

Abstract

This tutorial is intended to provide a detailed understanding of both the cutting-edge and the most commonly used technologies for low-level (level 1) data fusion, situation assessment (level 2 fusion), and the associated generation of appropriate response recommendations for decision making under uncertainty. More emphasis will be placed on level 2 fusion and response recommendations; we assume that level 1 fusion topics have already been covered in various talks and past tutorials. The discussion of the tutorial will be grounded on an architecture (see the left half of the figure below) whose cognitive framework is based on Rasmussen's integrated theory of human information processing, to provide key decision-support functionalities at multiple levels: 1) low-level fusion of data into information supporting skill-based processing; 2) aggregation of fused information into high-level abstraction of task-relevant knowledge supporting rule-based processing; and 3) aggregation of task-relevant current and prior knowledge into decision recommendations supporting knowledge-based processing. The tutorial will cover relevant technology, algorithms and tools (see the right half of the figure below) needed for implementing the architecture.

For the situation assessment part of the tutorial, State Space Models (SSM), Hidden Markov Models (HMM), and static/dynamic Bayesian belief networks will be presented in detail. Various exact and approximate algorithms for inferencing in these models and networks will be covered, including the speaker's own formalism for handling dynamic networks. To illustrate interactions between the level 1 and 2 fusion processes, only the necessary aspects of level 1 fusion will be presented in this part, but details on the other aspects of level 1 fusion will be omitted. For level 1 fusion, various versions of the particle filtering algorithm for multitarget tracking will be covered, and, for completeness, the Kalman filtering approach will be introduced.

For the response recommendations part of the tutorial, traditional Expected Utility Theory (EUT), rule-based expert systems, and influence diagram based decision-making processes will be described. Then a symbolic argumentation technique using first-order and non-classical modal logics will be presented. Various techniques for aggregating arguments including probability, possibility, and Dempster-Shafer theories will be covered. The argumentation technique and probabilistic aggregation are the major focus of the speaker's recent book on symbolic decision-making and a forthcoming book on the foundations of intelligent agents.

As for software tools, an in-house 5 th generation application development platform E5 (Prolog and Lisp) and Argumentation Building Engine (REASON), and an in-house belief network engine (BNet@Builder, with its temporal extension) will be used for illustrating response recommendations and situation assessment respectively. The commercial-off-the-shelf tools MATLAB and Hugin will be used for illustrating Kalman/particle filtering for level 1 fusion and influence diagrams for decision-making respectively.

To provide the tutorial with a flavor of practical decision-making processes, the participants will analyze a small military scenario, build belief networks to assess enemy situations in the first part of the tutorial, and develop an argumentation model for appropriate response recommendations in the second part of the tutorial.

Tutorial Outlines

Lesson 1: Architectures – Rasmussen's Hierarchy of Human Information Processing, JDL levels 1-5

Lesson 2: Military and Homeland Security Scenarios – Conventional, MOUT, OOTW, Bioterrorism

Lesson 3: Foundational Technologies – Probability and Statistics, First-Order and Modal Logics

Lesson 4: Brief Introduction to Level 1 Fusion –Multitarget Tracking, Kalman Filtering and Extensions, Particle Filtering, Rao-Blackwellised Filtering

Lesson 5: Situation Assessment – State Space Model (SSM), Hidden Markov Model (HMM), Static and Dynamic Bayesian Belief Networks, Message Passing Algorithm, Junction Tree Algorithm, Approximate Inferencing via Particle Filtering

Lesson 6: Decision Making – Expected Utility Theory (EUT), Rule-based Expert Systems, Influence Diagrams, Dempster-Shafer Theory of Belief Functions, Certainty Factor, Symbolic Argumentation

Lesson 7: Foundational Tools – Bayesian Belief Network Engine, 5th Generation Application Development Environment E5 (Prolog and Lisp), Argumentation Building Engine (REASON)

Lesson 8: Applications – Problem Modeling using Foundational Tools

Lesson 9: Selected References

Biography

Dr. Subrata Das is currently the Chief Scientist at Charles River Analytics, Inc. ( http://www.cra.com/ ), Cambridge , MA . Subrata leads research projects in the areas of intelligent agents, high-level information fusion, decision-making under uncertainty, planning and scheduling, and machine learning, for various organizations, including DoD, DARPA, and NASA. His technical expertise includes mathematical logics, probabilistic reasoning including Bayesian belief networks, symbolic argumentation, and a broad range of computational artificial intelligence techniques. Subrata held research positions at Imperial College and Queen Mary and Westfield College , both part of the University of London . He received his PhD in Computer Science from Heriot-Watt University in Scotland and M.Tech from Indian Statistical Institute. Subrata has published many journal and conference articles. Subrata is a member of the editorial board of the Information Fusion journal published by Elsevier Science. He is a technical committee (higher level fusion) member of the 9 th International Conference on Information Fusion to be held in Florence , Italy this year. Subrata is the author of the book entitled “Deductive Databases and Logic Programming” published by Addison-Wesley, and co-authored the book entitled “Safe and Sound: Artificial Intelligence in Hazardous Applications” published by the MIT Press.

Dr. Georgios Haralabus

NATO Undersea Research Centre, La Spezia , Italy
Email: Haralabus@nurc.nato.int

On SONAR systems and SONAR signal analysis

Abstract

Systems that use acoustic energy to probe the undersea environment or to detect submerged targets are called SONAR (SOund, NAvigation and Ranging). This tutorial offers a high-level description of  SONAR systems and signal processing techniques associated with passive and active detection analysis. Real-life examples that demonstrate the effectiveness of these systems in the complex shallow water environment are included.

Biography

Georgios Haralabus received his B.S. degree in mathematics from Aristotle University , Thessaloniki , Greece and the M.S. and Ph.D. degrees in signal processing and underwater acoustics from Duke University , USA . He is with the NATO Undersea Research Centre where he works on the development and the utilization of SONAR systems through application of passive multi-frequency matched field processing detection techniques in conjunction with genetic algorithms, broadband low frequency active detection problems, and detection enhancement through environmental adaptation. He is the Program Manager of the Environmental Adaptation program and the leader of the Broadband Environmentally Adaptive Sonar project at the Centre. In 2004 he was accepted as a fellow of the Hellenic Institute of Acoustics, a member of the European Acoustics Association.

Prof. Katia Sycara

Carnegie Mellon University
Email: katia@cs.cmu.edu

Dr. Bin Yu

Carnegie Mellon University
Email: byu@cs.cmu.edu

Decentralized Coordination of Autonomous Sensor Systems

Abstract

Autonomous sensor systems of the near future are envisioned to consist of hundreds of unmanned vehicles such as UAVs and UGVs. These networked autonomous and geographically distributed sensors play strong roles in military and civilian operations, e.g., battlefield surveillance and disaster rescue. At the same time, sensor systems offer many exciting research challenges due to their real-world constraints such as imperfect sensor data, real-time execution, and scarce wireless communication bandwidth. Many algorithms have been developed in the context of sensor systems, however, most of them are centralized and not scalable due to their optimistic assumption of unlimited communication bandwidth.

In this tutorial we will discuss various distributed coordination algorithms that scale well to large numbers of autonomous mobile sensors that may enter and leave the system dynamically. Specifically, we will focus on approximate algorithms for data delivery and fusion, task and resource allocation, and cooperative path planning. Moreover, we will present some advanced techniques that can effectively reason about spatial and temporal sensor data in the presence of noise and uncertainty. We will describe a novel approach for force aggregation and classification using Dempster-Shafer theory that has the potential of improving commanders' decision making.

This tutorial is intended for any delegates who have some basic knowledge of AI. Familiarity with basic concepts of sensor systems is desirable but not essential.

Biography

Dr. Katia Sycara is a Professor in the School of Computer Science at Carnegie Mellon University . She holds BS in Applied Math from Brown University, MS Electrical Engineering from University of Wisconsin , and a Ph.D. in Computer Science from Georgia Institute of Technology. Her research work lies in the intersection of Operations Research, Artificial Intelligence and Software Engineering. Dr. Sycara has authored over 300 technical papers and book chapters in multiagent/multirobot systems, negotiation, auctions, agent teams, and human-agent interaction. Prof. Sycara is a Fellow of the AAAI, Fellow of the IEEE and the recipient of the 2002 ACM/SIGART Agents Research Award. She is a founding Editor-in-Chief of the International Journal of Autonomous Agents and Multi-Agent Systems and on the editorial board of 5 additional journals. She has served as the Program Chair for the Second International Semantic Web Conference (ISWC-03) and the General Chair of the Second International Conference on Autonomous Agents (Agents-98).

Dr. Bin Yu is a Postdoctoral Fellow in the School of Computer Science at CMU. He received his Ph.D. in Computer Science from North Carolina State University in 2002. His research interests lie in the areas of artificial intelligence and distributed sensor systems. Currently, he is leading the research effort of the Information Fusion project under the AFOSR PRET Program. Dr. Yu has authored more than 20 technical papers in the areas of artificial intelligence, peer-to-peer systems, and distributed sensor systems. One of his papers appeared at the Fourth International Conference on Agents and Multiagent Systems (AAMAS-05) and was nominated for the best paper award. He has served as the Proceedings Chair of the Fourth International Conference on Autonomous Agents and Multiagent Systems (AAMAS-05).