{"id":7,"date":"2025-05-07T17:37:57","date_gmt":"2025-05-07T17:37:57","guid":{"rendered":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-2\/?page_id=7"},"modified":"2025-12-13T00:38:42","modified_gmt":"2025-12-13T00:38:42","slug":"methodology","status":"publish","type":"page","link":"https:\/\/mscvprojects.ri.cmu.edu\/2025team16-2\/methodology\/","title":{"rendered":"Methodology"},"content":{"rendered":"\n

We present a physics-guided, two-stage framework for dynamic thermal scene reconstruction from UAV\n infrared imagery using 3D Gaussian Splatting. The core design principle is to decouple\n static scene geometry<\/strong> from time-varying thermal dynamics<\/strong>:\n Stage 1 learns a stable 3D thermal representation at discrete timestamps, and Stage 2 models\n temperature evolution over time while keeping geometry fixed.<\/p>\n \n

\n Given calibrated UAV thermal frames captured across a mission, our goal is to reconstruct a thermal scene that\n supports novel-view synthesis<\/strong> and time-dependent temperature estimation<\/strong>.\n Thermal imagery introduces challenges not present in RGB\u2014view-dependent attenuation, conduction-induced edge\n blurring, and temporal thermal variation\u2014which we address with physics-aware components integrated into a\n Gaussian splatting pipeline.\n <\/p>\n\n\n\n

Thermal Image Formation Revisited<\/strong><\/h3>\n\n\n\n

In contrast to visible light reconstruction, thermal image formation is heavily influenced by the temperature, which in a sense relies on dynamic atmospheric and material conditions. We model the observed thermal frame xt,\u03b8<\/em><\/sub> at time t<\/em> and viewpoint \u03b8<\/em> as: <\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

where <\/em>f(X) is the canonical projection of an ideal, unperturbed thermal 3D scene, and <\/em>Pt<\/sub>,\u03b8<\/em><\/sub> encodes viewpoint and time dependent physical distortions. To generate more accurate reconstructions, we seek to estimate the underlying 3D thermal state X~<\/sup> such that:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

thereby correcting for the effects of environmental attenuation and surface level thermal diffusion.<\/p>\n\n\n\n

Stage 1: Static Physics Guided Thermal 3D Reconstruction<\/strong><\/h2>\n\n\n\n

1. Viewpoint Aware Attenuation Estimator (VAE)<\/strong><\/h3>\n\n\n\n

The VAE module<\/strong> models the degradation of thermal intensity due to atmospheric interference. Inspired by classical radiative transfer principles, we represent thermal attenuation as:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

where k denotes the attenuation factor, and d<\/em> is the propagation path length.<\/p>\n\n\n\n

We design a lightweight MLP that predicts the attenuation parameters (k(abs), k(sca), d) for each 3D Gaussian, conditioned on its spatial position and timestamp. This learned correction is applied directly to the Gaussian\u2019s appearance encoding (e.g., SH coefficients), yielding:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

This step compensates for the viewpoint specific drop in thermal radiance and significantly reduces ghosting and floaters.<\/p>\n\n\n\n

2. Thermal Diffusion Correction Unit (TDCU)<\/strong><\/h3>\n\n\n\n

To address the edge blurring caused by heat transfer between adjacent surfaces, we introduce the Thermal Diffusion Correction Unit (TDCU)<\/strong>.\n In thermal imaging, heat is fundamentally a form of energy that naturally diffuses from hotter to cooler regions. As this energy propagates through conduction, it leads to smoothed temperature gradients across object boundaries. In practice, this manifests in thermal images as softened edges, where the temperature difference between adjacent surfaces becomes visually indistinct particularly problematic in scenes where fine boundaries carry semantic importance.\nTo mitigate that, we draw from the heat equation to simulate spatial temperature smoothing:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

where u<\/em> represents the surface temperature field and \u03b1<\/em> captures local diffusivity.<\/p>\n\n\n\n

Instead of relying on fixed physical coefficients, the TDCU is implemented as a deep residual network that ingests both the original image and its second order spatial gradients. By learning a spatially adaptive correction mask, the TDCU selectively sharpens edges that have been degraded by thermal conduction while preserving coherent heat flow patterns.<\/p>\n\n\n\n

3. Surface Smoothness Regularizer (SSR)<\/strong><\/h3>\n\n\n\n

While real world thermal distributions tend to change smoothly, abrupt transitions or sharp corners often arise from modeling errors. We introduce the Surface Smoothness Regularizer (SSR)<\/strong> to penalize spurious discontinuities. Drawing on the Harris corner metric, we compute a spatial map of irregular regions and weight their contribution to the loss:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

where C<\/em> denotes the corner response, i<\/em> is the current iteration, and L1 is the reconstruction error. This adaptive term progressively focuses the model on difficult regions during early training, encouraging structural coherence.<\/p>\n\n\n\n

4. Full Optimization Objective<\/strong><\/h3>\n\n\n\n

The total training loss combines three components: image fidelity, perceptual consistency, and structural smoothness:<\/p>\n\n\n

\n
\"\"<\/figure>\n<\/div>\n\n\n

where lambda_s and lambda_p are empirically set to 0.2. The model is optimized end-to-end using Adam, with separate learning rate schedules for the Gaussian parameters and the physics based modules.<\/p>\n\n\n

\n
\"\"

Figure 5:<\/strong> Stage 1: Physics guided static thermal 3D reconstruction.<\/strong>.<\/em> The system builds on 3D Gaussian Splatting (3DGS) and integrates three physics aware modules: the Viewpoint Aware Attenuation Estimator (VAE)<\/strong> which models atmospheric effects to correct view dependent attenuation. The Thermal Diffusion Correction Unit (TDCU)<\/strong> refines edge clarity by compensating for heat conduction induced blurring, and finally the Surface Smoothness Regularizer (SSR)<\/strong> enforces physically consistent temperature continuity by penalizing discontinuities. Together, these components improve fidelity and realism in novel view synthesis of thermal infrared scenes.<\/figcaption><\/figure>\n<\/div>\n\n\n


Stage 2: Dynamic Thermal Modeling via Physics Guided Temporal Evolution<\/strong><\/h2>\n\n\n\n
\n

\nStage 2 extends the static reconstruction into a dynamic thermal model by predicting how temperature evolves through time while keeping geometry fixed. This matches the physical assumption that scene structure and appearance are static over short mission windows, while temperature changes<\/strong>.<\/p>\n\n

Inputs<\/h4>\n