Over the past several years, much progress has been made towards the development of small-scale aircraft, known broadly as Micro Air Vehicles or MAVs. As these systems are maturing, interest in MAVs has accelerated substantially for applications ranging from battlefield surveillance, smart munitions and real-time bomb damage assessment, to forest-fire reconnaissance, surveys of natural disaster areas and inexpensive traffic and accident monitoring. At the University of Florida, researchers in aerospace and computer engineering have established a long track record in designing, building and test-flying innovative and rugged MAV and small UAV flight vehicles. Up until recently, these platforms were exclusively remotely piloted, with no autonomous or intelligent capabilities. This is at least partially due to the fact that sensors that are available for larger platforms are not currently practical for use on our smaller MAVs. The one sensor that is critical to most conceivable MAV missions, such as remote surveillance, is an on-board video camera with a transmitter that streams the video to a nearby ground station. Exploitation of this rich and important sensor is therefore desirable, since no additional on-board hardware (and weight) is required. As such, we develop a general and unified computer vision framework for MAVs that not only addresses basic flight stability and control, but enables more intelligent missions, such as moving-object tracking and localization, as well.  The proposed system defines a framework for real-time image feature extraction, horizon detection and sky/ground segmentation, and contextual ground object recognition. Multiscale Linear Discriminant Analysis (MLDA) defines the first stage of the vision system and generates a multiscale description of images, incorporating both color and texture through a dynamic representation of image details. This representation is ideally suited for horizon detection and sky/ground segmentation of images, which we accomplish through the probabilistic representation of tree-structured belief networks (TSBNs). In the last stage of the vision processing, we seamlessly extend this probabilistic framework to perform computationally efficient detection and recognition of objects in the segmented ground region, through the idea of visual contexts. By exploiting the concept of visual contexts, we can quickly focus on candidate regions, where objects of interest may be found, and then compute additional features through the Complex Wavelet Transform (CWT) and HSI color space for those regions only. These additional features, while not necessary for global regions, are useful in accurate detection and recognition of smaller objects. Throughout, our approach is heavily influenced by real-time constraints and robustness to transient video noise.

Vision-Based Flight Stability and Control

Horizon Detection and Tracking

Sky/Ground Segmentation

Object Recognition