Live Video and UAVs: Opportunities and Challenges
Live Video and UAVs: Opportunities and Challenges
Lately, UAVs have become very popular both as professional tools and for recreation. In this blog, we will focus on how wireless video systems are implemented to give a First Person View (FPV) to the pilot remote controlling the UAV, or how to achieve live video streaming for broadcast.
Live UAV video streaming to anywhere in the world, e.g. direct to the station of a broadcaster for immediate playout, is not a simple task. You need good quality video, low latency and strong reliability.
UAV is a generic term describing several types of unmanned, remote controlled or autonomous aerial vehicles, such as fixed wing planes, helicopters and multi-rotors. Professional UAVs have a wide range of applications, e.g. mapping, aerial photography of live events, real-estate documentation, search and rescue, nature conservation, surveillance, and inspection of infrastructure. Several companies also plan on delivering goods by UAV. Many of these use cases requiring Beyond Visual Line of Sight (BVLOS) flying (read more about BVLOS challenges here).
Wireless video for UAVs
UAVs can be piloted in different ways, e.g. autonomously, remotely in line of sight by visually observing the UAV, or remotely by FPV. In an FPV system the video image from an onboard camera is transmitted by radio link to a personal video display on the ground. Systems span from simpler consumer grade to advanced professional grade with high-power video transmitters and ground receivers with directional tracking antennas.
Challenges and antenna solutions for wireless video
The range of the wireless video link is limited by several factors. The signal strength itself will diminish rapidly when distance increases, and obstacles in the line of sight can give additional attenuation. However, there are some other, naturally occurring, challenges to the radio link.
External sources of radio transmission in the environment can interfere with the main signal. If the interfering signals occur within the same frequency band as the wireless video link it will act as inband noise. This will reduce the signal to noise ratio.
If the source of interference is powerful, but outside the band of the wireless link, it is called a blocker. A blocking signal can penetrate insufficient front-end channel filtering, and decrease the dynamics of the Low Noise Amplifier (LNA).
2. Multipath fading
Even with a strong, noise free signal, a radio link can get sudden dropouts, especially in cluttered environments. This is due to the reflected propagation path cancelling the direct propagation path. The cancellation occurs because of phase shift associated with different propagation delay.
Directional antennas with a narrow beam and high directional gain will increase the receive strength of the signal from the UAV. The antenna can even be equipped with a tracker that automatically directs the antenna at the UAV.
The easiest way to avoid reflected signals is by having a directional antenna focus at the line of sight. Another, less intuitive yet simple, way is by adding antenna polarization. The most effective type for this purpose is circular polarization, where the radio wave due to antenna shape will have a twisting propagation.
Furthermore, if the antenna could be elevated to better maintain line of sight with the UAV, it will again increase the receive strength.
4. Analog FPV transmission
Most consumer, and even professional, FPV systems today use analog wireless video transmission, which are very compact and low-cost. This results in near zero latency between the image captured by the camera and the one viewed by the UAV pilot. Another favorable property of analog video is the gradually increasing picture noise when the video link starts to break down, as opposed to the sudden complete loss of imagery if using digital FPV transmission.
5. Digital FPV transmission
We are starting to see more and more of compact and low latency HD video links. The digital radio transmission system does however need robustness to preserve an acceptable framerate even under deteriorating radio conditions.
6. Future development
Wireless video for UAVs is still fairly immature technology, and there is need for compact and cost efficient FPV systems to be developed. With integration of System-on-a-Chip (SoC) solutions, and the next generation of cellular telecommunication, 5G, we will see lowered costs, increased transmission bandwidth and better noise reduction. This will result in higher performance, extended range, better image quality and stronger reliability.
Apart from the optics, the image sensor is the most critical part for high quality video (read more about it here). There are currently two main types of image sensors available: Complementary Metal Oxide Semiconductor (CMOS) or Charge Coupled Device (CCD). CMOS sensors are most commonly used but have some shortcomings. Images are scanned pixel by pixel, line by line, and there is a limit to the scanning speed. CCD cameras have a different way of collecting the image. The light-induced charge of each pixel is collected from all lines simultaneously. This is referred to as global shuttering and is the reason why CCD cameras can handle faster movement and vibration without image distortions.
There are different types of image stabilization, the most common being the use of a mechanical gimbal or the use of Digital Image Stabilization (DIS), but there is also the option of post–production stabilization software on the receiving end.
With today’s UAVs, many of them incorporate the use of a mechanical gimbal for image stabilization. The biggest reason being that it doesn’t interfere with image quality. The most efficient gimbal would be the 3-axis brushless gimbal. 3 axis refers to the camera being able to adjust in all directions, left/right, up/down and back/forth. Three brushless motors are what is used to make these adjustments. Accelerometers and gyroscopes react to which way and how fast the camera moves due to outside influence and a SoC decides how to counter-react to stabilize the camera.
Digital (or Electronic) Imaging Stabilization
This is a technology where the image is cropped by onboard software with image stabilization algorithms applied. There are pros and cons with this type of stabilization. The obvious con is that when you crop the image sensor digitally, you lose image information. There are also higher probability to get jello effects within the images using digital image stabilization. The advantage of DIS is that it does not add the extra weight of a gimbal.
DIS relies on an accurate motion sensor for tracking the source of jitter, which may be e.g. vibration or motion. The motion information is then used to compensate for it by cropping the viewable image from a stream of video frames, keeping the same objects centered.
Gimbaled stabilization seems to be the preference for more serious aerial video.
Top potential uses for FPV UAV technology
When combined with UAVs, FPV technology offers a lot of opportunity in several areas.
1. Search & rescue
The nature of a FPV system is that it feeds video back to the pilot in real time. This capacity has the potential to preserve lives. Search and rescue missions often rely on highly trained teams putting their lives on the line to help people in danger. In most cases the most dangerous and difficult part of the operation is locating the people in need of help. A UAV with e.g. thermal live feed can assist with localizing those in need.
2. Disaster management
Disaster management is the creation of plans through which communities reduce vulnerability to hazards and cope with disasters. Disaster management does not avert or eliminate the threats; instead, it focuses on creating plans to decrease the effect of disasters. Events covered by disaster management include acts of terrorism, industrial sabotage, fire, natural disasters (such as earthquakes, hurricanes, etc.), public disorder, industrial accidents, and communication failures.
This is where FPV technology can save the day. Not only can UAVs be more willingly put at risk in dangerous situations, but they can feed back live footage to management and rescue teams and prepare them for what lies ahead.
Just like in many search and rescue situations, FPV UAVs can help prevent people being put at risk for the sake of dangerous inspections. Structures such as dams, bridges and cell towers need regular checks to make sure that they are in sound condition.
Also in remote areas, lacking transportation infrastructure, access to inspection information by UAV is made more accessible. Here inspection of pipelines and power grids comes to mind.
4. Nature conservation
Nature conservation encompasses many topics, from habitat to species. Some of which are not only difficult to address from the ground but also dangerous. Imagine if you will, looking for poachers while counting elephants and rhinos, with the immediate danger of conflict with groups of armed criminals. A UAV, flying far BVLOS, feeding back live images letting you perform your work in relative safety (read more about it here).
Silent and energy efficient electric motors allows for long distance and discreet surveillance. The live video stream broadcasting imagery so the surveillance operation can be coordinated in real time. Alongside conventional video cameras, there is the possibility of using thermal camera payloads, so that e.g. living beings can be more easily detected in darkness or in dense vegetation.
At SmartPlanes, we believe that we will see even more of FPV technology in the future, since it despite its challenges also offers a lot of opportunities.
Roger Öhlund, CMO SmartPlanes