The last decade has seen a dramatic expansion in the deployment of Unmanned Airborne Vehicles (UAVs), as witnessed by deployments to Bosnia, Afghanistan and Iraq. Despite their widespread use in military applications, there are only a handful of operational commercial UAV systems in place, as current systems cannot be certified for use in civilian airspace. In order for UAVs to be integrated into civil airspace, it must be shown that the UAV system has at least an equivalent level of safety to human-piloted platforms.
One of the current barriers of certification to overcome is the inability for a UAV system to autonomously perform a safe forced landing in the event of an emergency. Damien Dusha of the Airborne Avionics Research Group (AARG) at QUT is part of a research group developing a forced landing system for a UAV using machine vision techniques. In order for a UAV to plan and execute a safe forced landing, it must first develop situational awareness. One technique the group is focussing on is horizon detection.
Horizon Detection
Successful detection of the horizon enables estimates of the pitch and roll of the aircraft to be calculated. For instance, many authors trivially define the roll angle as the arctan of the gradient of the line, and derive the pitch angle from the percentage of sky and ground in the image. However, the system must first be able to recognize the horizon from on-board video.
Existing techniques of horizon detection build upon a number of assumptions:
- An image may be segmented into sky and non-sky classes.
- The horizon falls on an edge between the sky and non-sky class.
- The horizon is approximated as a straight line.
- The horizon is coplanar to the local tangent plane.
However, there are many cases where the assumptions are invalid, especially where the vehicle is flown at low altitudes, or where the terrain is non-uniform. AARG take a similar approach to previous methods, seeking to explicitly locate candidate edges in an image and perform additional processing to extract the likeliest line corresponding to the horizon.
In order to collect data for use in the project, the group must run flight tests, during which the experiments collect 1024x768 uncompressed data from two cameras at 30Hz, along with synchronised inertial and GPS data. The data is collected as raw bayer code to save disk space, bandwidth and processing power. These flights capture roughly 1GB/min, and so two hours of flying represents an enormous amount of data.
Before image processing can occur, the images must first be converted to a standard RGB format, and losslessly compressed. This is a computationally expensive process, and so the group utilises the multiple processors of the Sirius supercomputer, located in QUT's HPC unit. This greatly increased the speed of the processing. For instance, the time required to convert a group of images was reduced from 6-8 hours on a desktop machine, to around 15 minutes on the supercomputer.
Image Processing
The images captured from the flight data are first broken up into red, green and blue channels. Each channel undergoes morphological smoothing, followed by edge detection. Although the positions of the horizon in each of the edge maps are reasonably well correlated, image noise usually causes few edge pixels to occupy the same position in each of the planes. In addition, the horizon will not be perfectly straight in all but the most benign cases. Therefore, before combining the edge maps, dilation is performed on each map to increase the overlap over the horizon.
|
|
|
Figure 1: Image from test flight
(left), along with edge map from blue channel (centre) and red
channel (right). |
Once the three edge maps are combined, the combined image is masked using a variety of methods, in order to remove spurious responses in the individual channel edge maps. One of the methods, known as the "Vote 2 from 3" scheme, involves marking an edge pixel if it appears in 2 of the 3 edge maps. Another method involves performing an ADD operation on each pixel in all three edge maps. Each method has its own benefits. For instance, the "Vote 2 from 3" scheme encounters problems when conditions involve a blue/green haze.
After masking, the combined edge map is then dilated - this serves to both eliminate some of the natural variation in the horizon, and to smooth the noisy response of the edge detection. Finally, the standard Hough transform is performed on the masked image, in order to detect the strongest line response in the image. The strongest response is assumed to be the horizon.
The algorithm devised by AARG has met with some success. It has been tested in a number of different conditions, such as changing light situations and glare (see figure 3). It has also had some success with noisy video links. Problems have been encountered in situations in which there is a strong line in view, such as the runway seen in figure 4, or in very hazy conditions.
 |
 |
 |
| Figure 2: Successful detection of the horizon in flight. | Figure 3: Successful detection of the horizon under glary conditions, while taxiing. | Figure 4: Unsuccessful detection - runway mistaken for horizon. |
Other Work
The group is involved in a number of other research projects. One of these is focused on developing a high-altitude vision-based forced landing site identification system for UAVs. A number of algorithms have been proven to identify appropriate landing areas - large open spaces, free of obstructions, suitable for landing purposes. This research is necessary for UAVs to fly in civilian airspace above populated areas in case of emergencies.
Contacts
Damien Dusha, Associate Professor Rodney A. Walker
Airborne Avionics
Research Group, Queensland University of Technology
Written by D. Dusha and T. Curtis, November 2006.