Integrated ISR model using drones

by

ISR with airborne Thermal and Optical sensors

Drones (UAVs)  are invaluable tools in surveillance and security as, along with various sensor payloads, it can play a role in ALL phases of ISR (Intelligence, Surveillance and Reconnaissance) from initial Detection to the final Response.

Two of the most commonly used sensors are thermal and optical cameras, which are very effectively deployed on mobile platforms such as UAVs.

UAV Categories

UAVs have been categorized in terms of range/altitude as follows: 

Hand-held600 m altitudeabout 2 km range
Close 1,500 m altitudeup to 10 km range
NATO type3,000 m altitudeup to 50 km range
Tactical5,500 m altitudeabout 160 km range
MALE (Med Altitude, Long Endurance)< 9,000 m range over 200 km
HALE  (High Altitude, Long Endurance)> 9,100 m indefinite range

The type of UAV you will use depends on your application, eg the USA southern border is patrolled by MALE UAVs (Predator) for Detection, but this is supplemented by smaller drones for Identification as well along with Situational Awareness (SA) during Response actions

Most Authorities have set 400ft as the maximum altitude for sUAS (small Unmanned Aerial Systems). The flexibility of these small UAV/S make them ideal to support the full range of  ISR phases.

ISR (Intelligence, Surveillance and Reconnaissance)

ISR is defined as the coordinated and integrated acquisition, processing and provision of timely, accurate, relevant, coherent and assured information and intelligence to the site commanding officer’s conduct of activities. 

Examples of ISR systems include surveillance and reconnaissance systems ranging from satellites, to manned aircraft to unmanned aircraft systems fixed wing or multi-copter.  This includes DRI, SA and Response (or ReACT).

Integrated ISR model
Integrated ISR model

More detailed levels of DRI lead to higher levels of situational awareness (SA), better decision-making and more effective Action. Thermal and Optical cameras differ in how they support these processes.

Situational Awareness (SA)

SA is the perception of environmental elements and events with respect to time or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time, or some other variable, such as a predetermined event.

Situational Awareness

Understanding of the environment is critical to decision-makers in complex, dynamic areas from aviation, air traffic control, ship navigation, power plant operations, military command and control, and emergency services such as firefighting and policing.

The SA framework based on a theoretical framework by (Endsley, 1995b), describes three stages or steps of SA formation: 

Level 1. Perception – involves the process of monitoring, detection and simple recognition.

Level 2. Comprehension – integrating this information too understand its like;y impact

Level 3. Projection – the ability to project the future actions of the elements in the environment

Different levels of SA

Sensors are used in different ways to achieve higher levels of Situational Awareness.

DRI (Detection, Recognition, Identification)

Resolution required for (DRI) Detection, Recognition, and Identification depends on the type of camera.

Using thermal cameras in drones in security and surveillance
Thermal cameras in DRI

Resolution for thermal cameras and optical IP cameras are measured differently.

 For example, when defining the performance of a thermal camera we use the Johnson Criteria of “detection”, “recognition” and “identification” (DRI).  On the other hand, IP camera resolution performance is usually defined by the number of pixels in the sensor, and we are usually interested in the ability to identify a person.

How much resolution? 

The following sections compare how resolution for DRI is defined using thermal and optical technologies.

Thermal Cameras

In military circles, “identification” is used along with “detection”, and “recognition” as part of the DRI criteria established by John Johnson (1958). 

Detection:                 Ability to distinguish an object from the background

Recognition:             Ability to classify the object class (animal, human, vehicle, boat …)

Identification:          Ability to describe the object in detail (a man with a hat, a deer, a Jeep …)

The following pictures illustrate these definitions, thermal cameras can detect, recognise and identify:

Thermal cameras in DRI
Detection At several kms, 2 targets are detected out of the background Recognition a human is walking along the fence Identification 2 males with trousers and jackets are identified – one is smoking

Optical IP Cameras

Optical IP cameras provide color images, and this can help identify or recognize the object. They will switch to monochrome or black-white when it gets very dark.  We use the terms detection, recognition and identification differently in surveillance applications.  Sometimes we have different meanings depending on the application.  Here are some examples: 

  • Detection: We sometimes detect or observe something moving in the distance.  We require different resolution for each of these applications, so we have to define what we mean before calculating the resolution 
  • Recognition: Sometimes we use this term when we want to recognize the object as a man or a woman.  In this case we are interested in how people or vehicles are moving, rather than the details (identity) of the objects. 
  • Identification: In surveillance applications we usually mean identifying who a person is, or the license plate number, or maybe the numbers on the tail of a plane.
More insight leads to better situational awareness

Comparison of Thermal and Optical Technology

Thermal cameras and optical cameras use totally different technology.  The thermal cameras provide images based on the heat energy (Infra-Red) emitted by the object, while the optical camera provides an image from the light reflecting off the object.

The optical IP cameras provide better detail than thermal cameras because they provide color video rather than the false color provided by monochrome thermal cameras.  Color provides much more information and makes it easier to identify things.   Since, the thermal cameras use heat energy from the target, they operate in total darkness, while optical cameras require some light source.

 To “see” at night with an optical IP camera, we use standard white light or IR light that reflects off the target.  Optical cameras are sensitive to infra-red (IR) light, but at a different frequency than the thermal cameras.   IR illuminators that operate at 850 nm or 940 nm can be used with IP cameras for night vision.

 We usually measure thermal resolution in line pairs, and we measure IP camera resolution in pixels.

Resolution Criteria for Thermal Cameras

Note that the Criteria was defined in cycles on target (or line pairs across a target).

Table 1: Summary of Johnson Criteria

Discrimination Level             Cycles on Target                        Description
Detection    1.0±0.25Object is of threat level significance
Orientation1.4±0.35Object aspect
Recognition    4.0±0.8Class of object (Jeep, tank, etc.)
Identification6.4±1.5Member of class

Resolution Criteria for Optical Cameras

IP cameras use different criteria for detection, recognition and identification.  Instead of line pairs, we use the pixel count to measure resolution.  This provides a more defined way of determining what detail we can see.  The resolution of IP Cameras is defined by how many megapixels they provide (1.2 megapixels, 2.0 megapixels, etc.).  This is further defined by the horizontal and vertical pixel count (i.e. 1280 x 720, or 1920 x 1080, etc.).   We also use pixels /ft. (or pixels/m) rather than pixels on target to determine the resolution we need for specific applications.

 Here is a summary of the criteria, with a description of what we can discern at a certain resolution (defined in pixels/ft.).   

 Table 2: Summary of IP Camera resolution terms

Discrimination Level(Pixels/ft)Description
DETECTION(1.6pixels/ft) at least 9 pixels across a target, for example, a 5’ 7” person
RECOGNITION(2.7pixels/ft) at least 15 pixels across the target (that’s 5’ 7”), to determine if the target is male/female
IDENTIFICATION(80pixels/ft) able to identify a person you know, requires about 40 pixels across face.


Comparison Summary

Cycles vs Pixels

Thermal camera resolution is measured differently than Optical IP camera resolution.  They use different technology.  Thermal cameras are measured using line pairs and the DRI criteria, while optical cameras generally use pixels/ft. (or pixels/m).