Cloud detection remains a fundamental challenge in satellite-based Earth Observation (EO). With up to two-thirds of the Earth’s surface covered in clouds at any time, a significant portion of remote sensing imagery (RSI) becomes contaminated, reducing its utility and increasing data transmission loads. Traditionally, cloud coverage estimation has relied on thresholding, time-differentiation, and statistical analysis, but these methods suffer from sensitivity to background noise, subjective thresholds, and low adaptability. The rise of CubeSats and microsatellites, coupled with limited downlink capacity, has amplified the need for on-board, real-time image filtering to optimize bandwidth and prioritize high-quality data transmission.
Recent developments in machine learning, especially convolutional neural networks (CNNs), have opened a new path for automated cloud detection directly onboard satellites. By combining quantized neural networks with low-power field-programmable gate arrays (FPGAs), it is now feasible to deploy advanced image classification systems in resource-constrained space environments.
Deploying AI algorithms in space is constrained by size, weight, power, and radiation tolerance. Traditional AI models running on GPUs or CPUs may offer high performance but are unsuitable for the power budgets and form factors of most nanosatellite platforms. Field Programmable Gate Arrays (FPGAs) offer a compelling alternative. They allow flexible deployment of neural network architectures while maintaining low power consumption.
For example, Xilinx Zynq-7020-based development boards have demonstrated the ability to host CNNs that operate within a 2.5 W power budget and achieve up to 15 frames per second (FPS) in cloud detection tasks. These boards include embedded ARM processors that can run Linux-based systems and integrate tightly with programmable logic components to optimize compute-intensive tasks. Compared to Vision Processing Units (VPUs) like Intel’s Myriad-2, FPGA-based systems can offer better throughput and avoid some limitations related to hardware inflexibility and radiation sensitivity.
Resource utilization is a crucial factor in selecting and optimizing FPGAs for cloud detection. Parameters such as LUT (Look-Up Table) usage, flip-flop count, block RAM (BRAM), and DSP slice allocation determine how well the network can be mapped onto the hardware. Fine-tuning parallelization using folding techniques helps achieve the desired throughput without exceeding hardware limits.
The effectiveness of a CNN depends heavily on the training dataset and its preprocessing pipeline. For this application, the Landsat 8 Cloud Cover Assessment Validation dataset (L8 biome) provides a rich source of annotated imagery spanning eight biomes: Barren, Forest, Grass/Crops, Shrubland, Snow/Ice, Urban, Water, and Wetlands. Each scene measures 185 km by 180 km and includes natural color composites from red, green, and blue bands.
Preprocessing steps involve correcting for satellite orbit-induced rotations, removing no-data regions, and segmenting full scenes into manageable 512×512 pixel tiles. These tiles are then categorized based on cloud coverage percentages, with those exceeding 70% labeled as “cloudy.” To ensure robust model generalization, the dataset is divided across training, validation, and testing sets, stratified by cloud cover levels and biome types.
In-depth tile-based evaluation reveals that certain biomes, particularly snow/ice, introduce significant false positives due to the visual similarity between clouds and snow or ice. Excluding such biomes from training can materially improve model accuracy and reduce false-positive rates (FPR).
A proposed CNN architecture, CloudSatNet-1, consists of ten convolutional layers and two fully connected layers, optimized for binary classification, cloudy or not. A key feature is the use of quantization-aware training (QAT) to reduce bit widths for weights and activations while preserving accuracy. Bit widths of 4 and 3 have demonstrated comparable performance to full 32-bit models, with the 4-bit version achieving an accuracy of 94.4% and an FPR of just 2.23% when snow/ice data is excluded.
The architecture benefits from Brevitas and FINN frameworks, which enable quantized model training and hardware synthesis, respectively. These frameworks streamline the export of PyTorch models into FPGA-compatible bitstreams and help tailor the design for specific performance targets like 10 FPS.
Model parameters, such as convolutional kernel size, stride, and padding, are carefully balanced to maintain spatial resolution and computational efficiency. The use of batch normalization and ReLU activations contributes to convergence stability and inference speed.
Evaluation metrics include accuracy, precision, recall, F1 score, and FPR. When using the full L8 dataset, the 4-bit model demonstrated nearly 90% accuracy, while performance improved significantly with the exclusion of the snow/ice biome. This suggests that RGB-only models struggle with high-albedo regions where clouds are visually indistinguishable from snow or ice.
Biome-specific analysis shows top performance in grass/crops and forest regions, with minimal FPR. However, performance drops sharply in snow/ice environments. A visual analysis of misclassified tiles confirms that features like fog, snow, and water reflections commonly cause false positives.
Quantization also introduces marginal accuracy loss, but benefits such as reduced memory footprint and faster inference outweigh the degradation. Statistical tests such as the t-test were used to validate that quantized inference on hardware did not materially distort model outputs.
Deployment on FPGAs involves mapping CNN layers to parallel compute units using techniques like dataflow architecture acceleration (DFA). The first layer often represents a computational bottleneck and receives priority for DSP resource allocation. Experiments have shown that models can be tuned to meet specific FPS targets (e.g., 10 FPS) by adjusting the number of processing elements (PEs) and SIMD lanes.
Power consumption remains stable around 2.5 W, even under maximum load, demonstrating favorable throughput-to-power ratios. Flexibility in parallelization allows adaptation to mission-specific requirements, such as imaging cadence or power budgets.
Batch inference also increases throughput, achieving up to 20 FPS with batch sizes of 120, though real-time scenarios may favor single-frame inference. These deployment characteristics make the system well-suited for on-orbit operations where bandwidth, power, and reliability are paramount.
The integration of quantized CNNs into FPGA-based platforms represents a shift toward smarter, more autonomous satellites. This evolution reduces dependency on ground-based post-processing and prioritizes actionable data for transmission. With advancements in model quantization and edge AI deployment frameworks, future missions can expect more efficient onboard processing.
Nevertheless, limitations remain. The inability to distinguish between snow and cloud using RGB alone highlights the need for multi-spectral or hyperspectral inputs. Additional channels like near-infrared (NIR) could resolve many false positives. Furthermore, current classification approaches simplify the problem to binary labels, which may not fully capture the nuances of cloud structure or type.
Future work may include expanding the architecture to handle semantic segmentation, incorporating spectral diversity, and exploring binary neural networks (BNNs) for even greater resource efficiency. Moreover, resilience to space radiation and model redundancy for fault tolerance remain critical areas for development.
Satellite-based cloud detection has entered a new era through the fusion of AI, edge computing, and optimized hardware acceleration. Systems like CloudSatNet-1 demonstrate that it is possible to deploy accurate, power-efficient cloud classification models onboard CubeSats using only RGB inputs and minimal FPGA resources. These systems can significantly reduce bandwidth usage by filtering irrelevant imagery and provide near-instant feedback for mission-critical operations.
With continued refinement in model architecture, quantization techniques, and deployment strategies, onboard AI will play a central role in next-generation satellite missions. By learning from and adapting to diverse operational environments, these systems will drive smarter EO workflows and enable satellites to serve as intelligent agents in orbit.
Discover more about on-board cloud detection software in the Software category of the SmallSat Catalog. The SmallSat Catalog is a curated digital portal for the smallsat industry, showcasing hundreds of products and services from across the industry. As a one-stop shop for nanosatellite and small satellite missions, the SmallSat Catalog provides everything a mission builder needs to plan a successful smallsat mission.
To learn more about cloud detection using satellite onboard image processing, please explore the following research works on this topic:
