Beyond Embedded AI: Optimizing the Thermal Perception Pipeline from Sensor Data to Edge Deployment

ABSTRACT

The rapid advances in deep learning, computational imaging, and embedded processors have made it possible to run sophisticated perception workloads at the edge. For thermal and multispectral systems, the competitive benefit goes beyond placing an artificial intelligence (AI) model on an embedded processor. The greater advantage comes from optimizing the complete perception pipeline, from sensor data and image signal processing to object detection, tracking, runtime optimization, thermal management, and field deployment, under real-world constraints for size, weight, power, latency, bandwidth, and environmental conditions. 

This paper describes how Teledyne FLIR OEM integrates Prism™ thermal and multispectral image signal processing, application-specific detection and tracking libraries, and modern embedded AI platforms to help customers build intelligent imaging systems that deliver actionable information at the point of sensing.

.

DOWNLOAD PDF

Watch Presentation

TABLE OF CONTENTS

1. Acroynms
2. Introduction
3. Advances in Lightweight Models and Mobile Processors
4. Designing a Neural Network for Embedded Platforms - Introducing Pelee^TM
5. Optimization
6. The Rapid Development Power of Mobile Processors
7. Frameworks
8. Video Processing Pipeline
9. Prism ISP Functions
10. Conclusions

1. Acryonms

2. Introduction

Over the past decade, advances in deep learning have transformed computer vision from systems built around handcrafted feature extraction into data-driven models trained on large datasets and optimized for specialized perception tasks. This shift fundamentally changed the challenges of object detection: instead of designing fixed visual features by hand, developers now focus on collecting representative training data, selecting appropriate model architectures, and adapting increasingly sophisticated neural networks to real-world deployment conditions. As is often the case in early technology development, less attention was initially paid to the computing power needed to train and run these models. 

This lack of concern limited how the new technology could be used in vision-based systems, especially video-centric applications where the need for real-time object classification, metadata generation, and alerts is critical. At the same time, an arms race was being waged in the mobile phone industry around camera performance. This led to the rapid development of highly advanced image signal processors (ISPs) that enabled ever-improving camera performance using algorithms and what is now referred to as computational imaging (CI). More recently, the rapid adoption of large language models, vision-language models, and multimodal AI has increased the demand for heterogeneous processors that can run both traditional vision pipelines and newer foundation-model-inspired workloads closer to the sensor.

Today’s processors feature heterogeneous architectures with multiple central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), neural processing units (NPUs), and ISP compute blocks. In addition, the open-source community has contributed many of the building blocks used to create perception systems, including YOLO-family object detectors, transformer-based detection models, platform-agnostic file formats such as Open Neural Network Exchange (ONNX), development frameworks including PyTorch and TensorFlow, and production runtimes such as TensorRT, ONNX Runtime, OpenVINO, LiteRT/TensorFlow Lite, and hardware-specific software development kits (SDKs). It’s become a common workflow to train in PyTorch or TensorFlow, export to ONNX, optimize and quantize for the target runtime, and then validate latency, power, memory use, and accuracy directly on the deployed hardware.

Teledyne FLIR OEM is focused on integrating perception and computational imaging or image signal processing onto highly size, weight, and power- (SWaP) optimized processors for a wide range of applications. Leveraging new silicon designs fabricated on the most advanced nodes enables system developers to increase battery life and simplify system thermal management.

This is the central design challenge for intelligent thermal systems. Visible-camera AI techniques cannot simply be transferred to thermal imaging without accounting for sensor physics, lower texture content, environmental variability, atmospheric effects, target size, optics, frame rate, calibration, and the image-processing steps that shape what the detector sees. In practice, detection and tracking performance depend on the full chain: raw sensor data, radiometric or non-radiometric processing, tone mapping, denoising, stabilization, super-resolution, model architecture, quantization, runtime mapping, memory bandwidth, and power management. Teledyne FLIR OEM’s advantage lies in its ability to optimize across that chain rather than treating AI inference as a standalone function.

3. Advances in Lightweight Models 
and Mobile Processors

Once the impact of deep learning on computer vision was recognized, work commenced on developing neural networks and models that could run at the edge on low-cost mobile processors, enabling object detection for video real-time applications including Automatic Target Recognition (ATR), vehicle autonomy, Advanced Driver Assistance Systems (ADAS), security, and for both ground and airborne intelligence, surveillance, and reconnaissance (ISR). Other software packages incorporated into vision-based products include Simultaneous Localization and Mapping (SLAM), collision avoidance, sensor fusion, optical flow, and object tracking. These computationally demanding features, when compiled for a specific hardware architecture, are mapped across CPU, GPU, DSP, NPU, ISP, and memory subsystems within the target system on chip (SoC). Managing those tasks across modern heterogeneous processors is a significant engineering challenge, but enables major power, latency, and thermal-management advantages compared with cloud-dependent architectures. In just a few processor generations, advanced SoCs have moved from 10 nm-class nodes to 7 nm, 5 nm, and 4 nm-class designs, resulting in substantially improved performance per watt.

Table 1 - Representative Embedded Edge AI Processors and Modules

Image signal processing and AI object detection algorithms are computationally expensive. For example, Teledyne FLIR OEM’s advanced denoising and super-resolution algorithms perform up to 100,000 operations per pixel per second (approximately 2 trillion operations per second (TOPS) for a 640x512 resolution input at 60 frames per second (fps)) and run on a GPU core. Running large object detection models also consumes a lot of processing power and runs on the DSP subsystem (Qualcomm®) or GPU (NVIDIA®). 

For thermal systems, raw TOPS is a useful but incomplete metric. Real performance depends on how efficiently each stage of the pipeline is mapped to the appropriate compute resource, how many times video data must move through memory, whether image enhancement improves or degrades detector confidence, and whether the complete system can sustain required frame rates without exceeding power or thermal limits. The most effective embedded AI designs therefore begin with the sensing problem and the deployment environment, not with the processor specification alone.

There are many variables to consider when developing and deploying an object detector, including processor architecture, desired accuracy, acceptable latency or frame rate, memory bandwidth, software runtime, model size, and power/thermal budget. In response to demands to run object detectors on embedded processors, several low-power neural networks were developed, including You Only Look Once (YOLO) by Joseph Redmon from the University of Washington in 2015 and MobileNet by Google in 2017. Since then, the detector landscape has expanded substantially to include newer YOLO-family models, transformer-based detectors, non-maximum suppression (NMS)-free architectures, and models designed for multi-task perception and open-vocabulary detection. For embedded thermal and multispectral systems, the most important question is not simply which network is newest, but which model delivers the required detection performance, latency, power consumption, and integration reliability on the selected processor.

YOLO revolutionized object detection with its single-pass approach. Unlike region-based convolutional neural networks (RCNN) involving multiple stages such as region proposals followed by classification, YOLO accomplishes detection and localization in a single forward pass of the network. By dividing an input image into a grid and predicting multiple bounding boxes and class probabilities simultaneously for each grid cell, the YOLO algorithm offers both speed and accuracy. One-stage detectors use a single feed-forward network to directly predict object classes and locations. Single Shot Detector (SSD) improves YOLO in several aspects, including using multi-scales of features for prediction and using default boxes and aspect ratios for adjusting to varying object shapes. While two-stage approaches generally produce higher accuracy, one-stage approaches often operate at higher efficiency. Newer detectors continue to improve this trade space through better feature aggregation, quantization-aware design, transformer-inspired attention mechanisms, and deployment-friendly architectures that reduce post-processing overhead.

4. Designing a Neural Network for Embedded Platforms  - Introducing Pelee

Figure 1 - Architecture of Object Detector (Juan R. Terven, 2023)

Designing object detection for thermal cameras requires a different starting point than selecting a general-purpose vision model. Thermal imagery often contains fewer visual textures than visible imagery; targets may occupy only a small number of pixels; and environmental conditions can change contrast, apparent shape, and background clutter. These factors make it important to co-design the detector with the sensor, optics, image-processing pipeline, and deployment hardware.

Teledyne FLIR OEM extended the single-pass approach with the development of its Pelee network (patent pending). Convolutional neural networks (CNNs) are comprised of three main blocks: the feature extractor or backbone; a feature aggregator or neck; and the head or object detector. Pelee features a new backbone design that is built with a dense connectivity pattern to encourage feature reuse, and a two-way dense layer to get different scales of receptive fields. Pelee incorporates a new feature fusion block (FFB) based on the Path Aggregation Feature Pyramid Network (PANet). Designed with small target detection in mind, the new FFB adopts a late fusion strategy, which aims to ensure that each feature map of the output retains the detailed information learned from the shallow layers and acquires high-level semantic features. 

Table  2 - Backbone Resources

The backbone network directly affects the memory, speed, and performance of the object detector. In SSD CNN, over 90% of the computation cost is consumed by the backbone network. Teledyne FLIR OEM’s CNN incorporates a new backbone design that avoids using a squeeze-and-excitation block, carefully selects activation functions according to the hardware, and adjusts the network’s width and depth according to the hardware.

Pelee should therefore be understood as an application-specific, deployment-oriented detector architecture rather than as a general-purpose benchmark winner against every new model family. Its value is in the engineering trade space that matters for smart thermal cameras: small-target performance, predictable latency, memory efficiency, hardware-aware activation choices, and the ability to run alongside Prism ISP functions, video stabilization, tracking, and other perception workloads within a constrained SWaP envelope.

Figure 2 - Pelee Backbone Based Model Performance

Figure 3 – Pelee versus YOLO Networks Performance Comparison

5. Optimization

Quantization

Quantization is the compression of floating-point values in neural network parameters to improve model latency, reduce memory footprint, and lower power consumption when running inference. Model training is generally performed using 32-bit floating-point data on powerful GPU-based systems. Inference on embedded processors is then optimized using FP16, FP8, INT8, INT4, or other runtime-supported precisions depending on the processor, model architecture, calibration data, and acceptable accuracy trade-offs.

Teledyne FLIR OEM uses INT8 to reduce memory footprint, compute time, and energy consumption because it remains a practical compromise between precision and speed for many embedded vision workloads. However, deployment now depends increasingly on explicit quantization workflows, representative calibration data, quantization-aware training when needed, and target-specific engine generation. Developers should validate FP16, INT8, and lower-precision options on the actual hardware because performance, accuracy, memory use, and thermal behavior can vary significantly between GPU, DSP, NPU, and vendor-specific runtime implementations. Gaining back compute resources gives developers the flexibility to add more capabilities, including real-time inference on two or more camera feeds.

Pruning

Pruning is a method used to reduce unused data paths in the neural network by determining the importance of each unit and removing passive paths. After training, many of the network neurons have a weight value of 0, so these neurons can be eliminated through a secondary network training process after quantization. Current pruning methods include weight pruning, channel pruning, and neuron pruning, each with its advantages and disadvantages. 

The criteria for determining the importance of the unit can impact the accuracy. The use of constraint learning methods can improve it, although the implementation is complex. The random search method of pruning is simple to implement, but the network model’s performance is limited compared to other methods that improve it. Therefore, each method needs to be selected in conjunction with the actual application scenario. There is no single method that can synthesize complexity and model compression efficiency.[i]

Thermal Management

For many product categories, including drones and weapon sights, power consumption is a critical design constraint. High consumption limits the operating time, adds support logistics challenges, and adds complexity and costs for thermal management. For some battery-operated devices, the choice is to use a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) processor to keep power to an absolute minimum, but third-party analytics and ISP software are not available, as these libraries are typically compiled to run on Advanced RISC Machine (ARM) processors and other cores using open-source components and frameworks like OpenCV. While this is not the case for the most power-constrained products, the power demands of new-generation embedded processors fabricated on 4 nm to 7 nm nodes offer systems designers many options for adding high-value capabilities to products.

For a system running multiple cameras and running multiple feature routines like an object detector, target tracker, and local contrast enhancement, the developer will reach the upper limit of the processor’s throughput and generate a lot of heat. To help manage this, software developers can build in configuration parameters that can be set or adjusted dynamically. For example, for many applications it is not necessary to do inference at frame rate. By reducing the inference output to 10 fps or less, the power profile of the entire software stack may be managed to fit the power budget.

6. The Rapid Development Power 
of Mobile Processors

Running CNN models and CI algorithms is extremely computationally demanding. Until recently, embedded processors did not have the power to meet AI and CI workloads. AI requires massive parallelism of multiply-accumulate functions. Traditional GPUs were able to do parallelism in a similar way for graphics, so they were reused for AI applications, particularly for model training. As a result, NVIDIA has established itself as the dominant player in model training.

Today’s mobile and embedded processors are amazing feats of engineering. For example, Qualcomm’s Dragonwing QCS8550 SoC is a 4-nm design that features CPU, GPU, DSP, NPU, computer vision, and dedicated ISP resources for demanding IoT and edge AI applications. The integrated ISP and computer vision resources support functions such as video denoising, stabilization, image correction, white balance, color processing, automatic gain control, focus, and other camera functions. This class of chip requires enormous development investment and depends on advanced foundry capability, software tooling, and long-term ecosystem support.

Figure 5 - QCS 8550 Processor Cores

Teledyne FLIR OEM wanted to create a platform that customers could build their products around and would allow creation of highly integrated software stacks and minimize cost, latency, and power by using a single processor where practical. This has been the goal over several platform generations, including the selection of the Intel® Myriad™ processor for Teledyne FLIR OEM Boson® thermal infrared (IR) camera module. The Myriad chip architecture could not meet the heavier compute needs for computational imaging or CNN-based object detectors. With the introduction of NVIDIA Xavier, NVIDIA Jetson Orin-class modules, Qualcomm RB5/QCS5165, and Qualcomm QCS8550-class processors, it became practical to run multiple software stacks distributed across specific processor cores and accelerators.

System on Chip (SoC) Overview

Today’s embedded processors leverage the latest in foundry nodes, enabling multi-core designs with unified memory. Processor suppliers offer very comprehensive development support, and developers can deploy their code on these multicore processors to maximize performance while meeting power and thermal budgets.

CPU

The core functionality of the CPU is to fetch instructions from random access memory (RAM) and then decode and execute the instructions. CPUs are generalist processors designed to execute routines that handle sequential workloads. Many CPUs have built-in vector processors to speed up some operations (ex. NEON on ARM-based CPUs). Several of Teledyne FLIR OEM’s digital product capabilities run on CPU cores, including Prism ISP correlation video tracker and electronic video stabilization. Given the large number of CPU cores available on current generation SoCs, developers have tremendous flexibility to optimize performance and power. CPUs are not a good fit for tasks requiring large numbers of mathematical operations per pixel. CPUs have throughput limitations and consume relatively high power per operation.

GPU

AI and CI workloads are massive, demanding a significant amount of bandwidth and processing power. Processor designers have created powerful chip architectures integrating memory, security, and real-time data connectivity. Traditional CPUs typically lack the processing performance needed but are ideal for performing sequential tasks. GPUs can handle the parallelism of AI’s multiply-accumulate functions and can be applied to AI applications. In fact, GPUs can serve as AI accelerators, enhancing performance for neural networks and similar workloads. GPUs can have thousands of compute cores and are used for many parallel processing tasks. GPUs are especially useful for training models. However, GPUs consume significant power and have higher on-chip memory requirements. For this reason, modern mobile processors employ dedicated AI co-processors that use digital signal processors (DSPs) for running inference at the edge.

DSP

DSPs are specialized and power-efficient SoC subsystems optimized to execute mathematical operations. They utilize wide instruction words to help maximize processing per clock cycle. However, they are not as easy to program and require familiarity with the features of the DSP hardware, programming environment, and optimization of DSP software to achieve the best performance. The Qualcomm Hexagon™ Tensor Processor (HTP) is an AI accelerator that is suited for running computationally intensive AI workloads. To get improved performance and run an AI model on HTP, a model must be quantized to one of the supported precisions: INT4, INT8, INT16, or FP16. 

ISP

Because digital photography is the front line of the competitive battle in the mobile handset industry, mobile processor suppliers incorporate dedicated silicon intellectual property (IP) blocks to perform image enhancement functions including noise reduction, lens distortion correction, white balancing, auto exposure, auto-focus, de-mosaicing, and image compression. At the system level, ISP tuning is typically needed to optimize the parameters and various filters to a specific image sensor and lens. ISP fine-tuning is an industry unto itself. Many SoC suppliers offer ISP libraries for popular image sensors, potentially saving systems developers significant investment. 

There is little justification for mobile processor suppliers to design and fabricate a thermal camera ISP. Still, the benefits of image signal processing are valuable, so Teledyne FLIR OEM has developed a comprehensive set of ISP features including 16- to 8-bit tone mapping (automatic gain control), spatiotemporal de-noising, super-resolution, turbulence mitigation, and electronic image stabilization. Most of these algorithms run on GPUs given the significant processing demands.

Memory

Given the high data rates in digital video streams and the fact that data may be read repeatedly by multiple algorithms, memory capacity and memory bandwidth are critical to efficient signal processing. Through a combination of increased clock speeds, higher memory bandwidth, dedicated AI acceleration, and improved software optimization, Teledyne FLIR OEM’s largest existing AI model (768x768 pixel input resolution) is executed in 6 ms on the Qualcomm QCS8550 versus 33 ms on the QRB5165. This represents a more than 5X increase in inference speed and gives a safe margin for achieving inference at frame rate for next-generation SXGA (1280x1024) resolution cameras.

7. Frameworks

AI frameworks are foundational building blocks that shape how intelligent systems are created, optimized, and deployed. These frameworks, equipped with libraries and pre-built functions, enable developers to craft sophisticated AI algorithms without having to design every function from scratch. In production edge AI systems, the framework discussion now extends beyond model training to include export formats, runtime optimization, quantization tooling, hardware-specific compilers, regression testing, over-the-air update strategies, and fleet-level performance monitoring.

PyTorch remains widely used for research, experimentation, and model development because of its flexibility and user-friendly execution model. Its dynamic computation graph allows developers to change the behavior of models on the fly and use Python control flow operations, making it well suited for complex model architectures, rapid prototyping, and iterative experimentation.

TensorFlow remains important in production environments and mobile deployment workflows, particularly where TensorFlow Lite or LiteRT is already integrated. However, modern embedded deployment is rarely a simple PyTorch-versus-TensorFlow choice. A common production path is to train in PyTorch or TensorFlow, export to ONNX or another portable representation, optimize the graph, quantize the model, compile to a runtime such as TensorRT, ONNX Runtime, OpenVINO, LiteRT/TensorFlow Lite, or Qualcomm AI Engine Direct, and then validate latency, accuracy, power, and thermal behavior on the target device. The best framework is therefore the one that supports the full deployment pipeline for the selected processor and application, not simply the one that is most convenient during training.

8. Video Processing Pipeline

The video processing pipeline is where Teledyne FLIR OEM’s approach becomes most differentiated. Rather than treating the camera, ISP, detector, tracker, and embedded processor as independent components, the pipeline is engineered as a connected system. Each stage affects the next: denoising changes target texture, contrast enhancement changes model input distribution, stabilization improves temporal consistency, super-resolution can improve effective detail, and tracking converts frame-by-frame detections into persistent information. Optimizing these interactions is essential for reliable edge perception.

Teledyne FLIR OEM developed Prism ISP, a collection of image processing software libraries for thermal images that include local contrast enhancement (LCE), spatial and temporal noise reduction, electronic image stabilization, turbulence mitigation, super-resolution, and several video fusion techniques. These functions improve image quality for human interpretation while also shaping the information presented to downstream detection and tracking algorithms. The objective is both to make video look better and to improve the reliability, efficiency, and deployability of the complete thermal perception pipeline while minimizing power requirements and cost. Thermal camera modules from Teledyne FLIR OEM do not include integrated processors to run Prism ISP. With the availability of modern mobile and embedded processors, it is now practical to add an accessory electronics module with the power to run ISP, detection, and tracking algorithms at useful video frame rates while meeting real-world power and thermal constraints.

Figure 6 - Prism Video Processing Pipeline

Thermal imaging is used across many use cases to visualize objects: at meters distances for inspection and firefighting; at multiple kilometers in intelligence, surveillance, and reconnaissance (ISR); and from aerial, tower-based, and ground-level perspectives. The combination of range, perspective, and day-, night-, and weather-based environmental conditions creates real-world imaging challenges that can be mitigated with Prism ISP algorithm functions. 

9. Prism ISP Functions

Turbulence Mitigation

Figure 7 - comparing without (left) and with (right) Turbulence Mitigation

Atmospheric turbulence refers to the irregular and chaotic motion of fluid or air characterized by velocity, pressure, and density fluctuations. Turbulence affects photon transmission because the solar gain of the surface causes the atmosphere to mix. It churns the air and causes water vapor, smoke, and other substances to alter the refraction of the infrared signal. Advanced techniques remove the distortion effects while maintaining good temporal image element quality, meaning objects in motion are not transformed into indistinguishable blobs or streaks. This is critical when detecting targets, including people and vehicles, using AI. 

For AI detection and tracking, turbulence mitigation is more than an image-quality feature. Distortion, shimmer, and apparent motion can reduce detector confidence and make tracks less stable. By improving temporal consistency before inference and tracking, the pipeline can deliver more reliable target information without depending solely on larger models or higher compute budgets.

The primary goals of turbulence mitigation algorithms are to enhance signal quality, reduce noise, and restore clarity and stability in turbulent conditions. The Prism ISP algorithm employs various mathematical and signal-processing techniques to analyze and process the data affected by turbulence. 

1. Data Acquisition: The first step is to acquire the data affected by turbulence. This is presented in the form of image frames.
2. Pre-processing: The acquired data is pre-processed to remove noise or artifacts that are not directly related to turbulence. This involves filtering, noise reduction, and calibration techniques to improve the signal quality.
3. Turbulence Estimation: It is essential to estimate the turbulence characteristics present in the data. This involves analyzing the statistical properties of the fluctuations and identifying relevant parameters such as turbulence intensity, correlation length, and time scales.
4. Deconvolution or Reconstruction: Deconvolution techniques attempt to reverse the blurring effects caused by turbulence. Deconvolution algorithms often utilize mathematical models of turbulence or empirical knowledge to restore the original signal.
5. Adaptive Filtering: Turbulence conditions can vary over time and space, so many turbulence mitigation algorithms employ adaptive filtering techniques. These methods dynamically adjust the filtering parameters based on estimated turbulence parameters to optimize the trade-off between noise reduction and the preservation of essential signal features.
6. Post-processing: After the turbulence mitigation steps, additional post-processing may be performed to enhance the quality further and reduce artifacts in the final output. Depending on the specific application, this may involve techniques such as denoising, sharpening, contrast enhancement, or feature extraction.

Super Resolution

Figure 8 – comparing without (left) and with (right) Super Resolution

Multi-frame super-resolution is a powerful technique that aims to reconstruct a high-resolution image from one or more low-resolution images. The optical resolution of a system is limited by the diffraction limit set by the system aperture and the sampling resolution of the detector. A small aperture and a low-resolution sensor will result in a cheaper and smaller system that will result in poor-quality images with highly aliased and/or blurred edges. This is especially true when long-range targets are captured using a long focal length optic that scales the effects of diffraction when combined with a finite aperture size. By employing sophisticated algorithms, super-resolution reconstructs high-resolution images using data from the degraded inputs. While traditional methods have predominantly focused on exploiting the inherent spatial correlations within low-resolution images, recent advancements have demonstrated the efficacy of leveraging aliasing techniques to achieve superior results. 

In a perception pipeline, super-resolution can also affect detection range, small-target classification, and operator confidence. The benefit must be evaluated at the system level because sharpening or reconstruction artifacts may help visual interpretation but can also shift the input distribution seen by an AI model. This reinforces the need to tune ISP and detection together rather than as isolated functions.

Image Stabilization

Figure 9 - Typical Applications for Electronic Stabilization

Cameras are often mounted on moving platforms such as ground vehicles, marine craft, aircraft, drones, or towers. Platform motion transfers directly onto the video image, which can lead to viewer fatigue and poor image quality, impacting situational awareness and evidence fidelity. Electronic stabilization eliminates most motion-induced image distortion, particularly when imaging through a narrow field of view zoom optic used for long-range imaging. 

For automated perception, stabilization improves temporal consistency and can reduce false track initiation, missed detections, and operator fatigue. It also helps downstream algorithms use motion cues more effectively because the image motion caused by the platform is reduced before detection and tracking are performed.

MSX - Multi-Spectral Dynamic Imaging

Figure 10 - MSX Embosses the Visible Edges on Thermal Images in Real Time

There are many different approaches to image fusion or blending thermal and visible imagery into a single fused video stream. The objective is to bring out each spectrum's unique and valuable information while not requiring the transmission of two or more streams for downstream human operators or systems to analyze. This can be complex in practice due to the need for perfect image registration (alignment) of two or more channels and issues due to the different integration times for each sensor. 

Teledyne FLIR OEM has patented a unique technology called MSX (Multi-Spectral Dynamic Imaging). Unlike image blending (mixing a visible light and thermal image), MSX does not remove thermal details or decrease thermal transparency. Instead, it applies a high-frequency filter to extract sharp visible details like symbology, including outlines, words, numbers, and other high-contrast edge details, and overlays them on the thermal image. This helps give image definition and has proven to be a beneficial technology in fields like predictive maintenance, security, and aerial ISR applications. 

AGC – Automatic Gain Control

Figure 11 -  Local Contrast Enhancement (Boson® 640 source video)

In the context of images, contrast refers to the difference in intensity between the brightest and darkest regions. It plays a crucial role in defining the overall visual impact of an image. Local contrast enhancement, also known as local contrast stretching, operates on the principle of expanding the dynamic range of contrast within specific regions of an image rather than globally altering the entire image's contrast.

Local contrast enhancement brings out intricate details, enhances textures, and accentuates features. Converting raw 16-bit monochrome sensor data into the 8-bit color depth we see on displays relies on algorithms to map the extensive distribution of pixel values into 1 of 256 shades of color or gray. Because the distribution of pixel values in an image is rarely linear, maximizing contrast is highly complex and dynamic from scenario to scenario. Unlike global contrast enhancement, which adjusts the contrast across the entire image uniformly, local contrast enhancement focuses on preserving local details and fine textures. It targets smaller regions within the image and amplifies the contrast within those regions while maintaining the relative contrast relationships between neighboring regions. 

For AI pipelines, tone mapping and local contrast enhancement must be managed carefully because they shape the numerical input to the detector. A setting that looks visually pleasing may not always maximize model confidence, reduce false alarms, or preserve small-target detail. Teledyne FLIR OEM’s pipeline-level approach enables these image-processing choices to be evaluated against the final system objectives of reliable detection, tracking, and decision support within the available SWaP envelope.

10. Conclusion

There have been remarkable advances in deep learning, embedded processors, and production AI deployment tools. For thermal and multispectral systems, these advances are important, but they are not sufficient by themselves. The market is moving beyond the question of whether AI can run at the edge. The more important question is whether the complete perception pipeline can be engineered to deliver reliable, actionable information in the field while meeting demanding SWaP, latency, bandwidth, and thermal constraints.

Teledyne FLIR OEM’s advantage goes beyond simply running AI on embedded processors. It is the ability to optimize the full thermal perception pipeline of sensor data, image signal processing, model design, quantization, runtime deployment, tracking, memory use, power management, and environmental robustness as an integrated system. In practical applications across defense, autonomy, robotics, industrial automation, security, and automotive systems, differentiation will come from this full-stack integration rather than from headline TOPS or generic model selection alone.

The next generation of intelligent thermal cameras will therefore be defined by how well sensing physics, computational imaging, embedded AI, and deployment engineering are brought together. By combining thermal sensor expertise, Prism ISP, application-specific detection and tracking libraries, and experience optimizing software on modern heterogeneous processors, Teledyne FLIR OEM helps customers move from edge AI demonstrations to deployed perception systems that perform in real-world operating environments.

[i] https://patentimages.storage.googleapis.com/f5/e3/05/021253fa9a6921/US20220019843A1.pdf

[ii] https://patents.google.com/patent/US20220019843A1/en?oq=17%2f374909

[iii] A Review of Artificial Intelligence in Embedded Systems https://doi.org/10.3390/mi14050897

[iv] https://opencv.org/blog/pytorch-vs-tensorflow/