Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Human-AI Landscape Visualization Through 3D Gaussian Point Clouds: Investigating Interactive Environmental Reimagination Technology

Hong Kong Metropolitan University ↗
Publication year
Digital Object Identifier10.54941/ahfe1007152

Abstract

This research investigates 3D Gaussian Splatting (3DGS) not merely as a rendering technique, but as a probabilistic medium for environmental visualization and "organological" imagination. As emerging technologies mediate our perception of natural environments, a rupture occurs between the logical rigidity of traditional data and the fluid, "alien" logic of neural networks. Drawing on Luciana Parisi’s theory of the "incomputable" and the "aleatory" in automated thinking (2013, 2015, 2016), this paper argues that the inherent artifacts and fragmentations of 3DGS offer a new aesthetic language for spatial cognition—one that moves from deductive reconstruction to inductive, probabilistic generation. The methodology focuses on the artistic workflow of the project Phantom Terrains. It unfolds in three phases: First, Hong Kong’s high-density public housing estates are documented via "defective" mobile scanning, generating fragmented video footage rather than solid geometry. Second, these data "splats"—translucent ellipsoids encoding spatial and temporal uncertainty—are processed as raw video signals within a TouchDesigner network. Third, an immersive interface utilizes MediaPipe hand tracking to map somatic gestures to signal degradation, empowering participants to co-create speculative "phantom terrains." By foregrounding human agency within a responsive AI-assisted medium, this work demonstrates how Gaussian point clouds can serve as a generative canvas for collaborative environmental storytelling and offers design principles for future interactive art systems.

INTRODUCTION

The visualization of landscape has historically oscillated between the poles of objective documentation and subjective interpretation. From the precise topographical sketches of 19th-century geographers to the hyper-realistic digital twins of the 21st century, our tools for representing landscape shape our cognitive relationship with it. Today, as we enter an era where environmental perception is increasingly mediated by computational systems, a critical tension emerges: does the drive for photorealistic simulation displace human creative agency?

The emergence of 3D Gaussian Splatting (3DGS) marks a significant ontological shift in this domain. Unlike the solid, continuous surfaces of Received December 22, 2025; Revised December 31, 2025; Accepted January 10, 2026; Available online February 11, 2026

photogrammetry (Meshes) or the opaque, hidden layers of Neural Radiance Fields (NeRFs), 3DGS represents the world as a cloud of discrete, probabilistic entities—millions of translucent “splats” that coalesce to form an image but never quite form a solid object (Kerbl et al., 2023). This “point-based” nature presents a unique affordance for Computational Modeling and Simulation, it allows the environment to be treated not as a static artifact, but as a dynamic, fluid medium responsive to human interaction.​ This paper introduces a framework for “Interactive Environmental Reimagination,” a novel human-AI workflow that leverages the specific mathematical properties of 3D Gaussian Splatting to transform static environmental scans into “phantom terrains” (see Figure 1)—speculative landscapes that exist between the real and the imagined. The contribution of this work is twofold. First, it provides a technical methodology for the real-time semantic manipulation of Gaussian Splats, moving beyond simple scene reconstruction to active scene re-simulation. Second, it frames this interaction within the context of environmental storytelling, arguing that the ability to collaboratively alter the material properties of a digitized landscape fosters a deeper form of spatial cognition and engagement than passive observation.

Figure 1: Screenshot of the project phantom terrains.
Original PDF Figure 1:

THEORETICAL FRAMEWORK From Photogrammetry to Neural Rendering

Traditional environmental digitization has relied heavily on Structure-from-Motion (SfM) pipelines to generate textured polygonal meshes. While effective for diverse applications in cultural heritage, mesh- based approaches struggle with the “incomputable” complexity of organic decay and atmospheric density.

Kerbl et al. (2023) introduced 3D Gaussian Splatting as a hybrid alternative. By representing a scene as a collection of 3D Gaussians, this

Human-AI Landscape Visualization Through 3D Gaussian Point Clouds 913 method combines the explicit control of point clouds with the continuous differentiability of neural rendering.

The “Alien Logic” of the Splat Traditional 3D modeling relies on logical geometry: points connect to lines, lines to faces. It is a deductive system. Luciana Parisi (2013), in her analysis of “soft(ware) thought,” suggests that neural computation introduces a “heterogeneous logic”. This logic is not based on pre-defined axioms but on the inductive processing of massive contingency. The machine “thinks” by ingesting millions of data points (photos) and producing a probabilistic best-guess of reality.

In 3D Gaussian Splatting, this manifests as the “splat.” A splat is not a point of truth; it is a region of statistical likelihood, defined by a covariance matrix that describes its stretch and rotation. When the AI fails to perfectly reconstruct a scene—when it creates floating artifacts or “fog”—it is not failing; it is revealing the “incomputable” nature of the data. These artifacts are the “randomness” (aleatory) that Parisi (2013) identifies as central to modern algorithmic aesthetics. They represent the machine’s uncertainty, a “digital hallucination” that parallels the gaps in human memory.

Organological Extension of Imagination Li et al. (2025) frame 3D Gaussian Splatting as an “organological extension” of artistic imagination. Drawing on Stiegler’s concept of organology, they argue that the tool does not just serve the artist; it co-constitutes the creative process. The specific affordances of the Gaussian medium—its ability to be discrete yet continuous, solid yet transparent—allow the artist to externalize internal cognitive processes (like memory or imagination) into a spatial form. In Phantom Terrains, the machine’s “hallucinations” become the artist’s raw material, creating a feedback loop between human intent and algorithmic contingency.

METHODOLOGY: THE POETICS OF FRAGMENTATION AND SIGNAL FLOW Our methodology moves away from the standard engineering pipeline of geometric reconstruction, adopting instead a workflow of “transcoding” where spatial environments are treated as fluid video signals (see Figure 2).

Figure 2: The ‘transcoding’ pipeline.
Original PDF Figure 2:

Capture: The Mobile Gaze and Defective Scanning The data acquisition for Phantom Terrains focused on the dense, vertical topography of Hong Kong’s public housing estates and their adjacent urban parks. To capture the fleeting, subjective nature of memory, we utilized mobile-based 3D Gaussian Splatting applications rather than industrial scanners.

Crucially, rather than exporting the standard volumetric data (.ply files) for geometric reconstruction, we deliberately exported the captured “splats” directly as MP4 video footage. By rendering the incomplete, floating ellipsoids into flat video files, we treat the Gaussian artifacts not as 3D geometry, but as “temporal footage”—a documentary recording of the AI’s hallucination. The scanning strategy was intentionally “defective,” leaving gaps in the trajectory to ensure the footage contained loose, vibrating splats where the algorithm struggled to interpolate the complex density of the housing estates (Figure 3).

Figure 3: The captured “splats” directly as MP4 video footage.
Original PDF Figure 3:

Synthesis: The TOP Architecture of Memory The core synthesis occurs within TouchDesigner, shifting the logic from spatial coordinates to texture processing.

1. Ingestion via TOPs: The MP4 fragments are imported using the Movie File In TOP (Texture Operator) family. This transforms the “splats” from mathematical points into pixel-based signal streams. 2. Layering and Hierarchy: Within the TouchDesigner environment, these video fragments are organized into a complex compositing network. We utilize the logic of “Input/Output” to assign semantic meaning to

Human-AI Landscape Visualization Through 3D Gaussian Point Clouds 915 the data: foreground layers represent sharp, visceral recollections, while background layers are treated as subconscious noise. 3. The Logic of Signal Processing: The manipulation of these textures— adjusting luminance, alpha channels, and compositing order—provides a layer of meta-cognition. The technical act of processing video data becomes a metaphor for cognitive sorting: deciding which “signal” (memory) is amplified and which is suppressed.

Networked Reintegration Finally, the fragments undergo Networked Reintegration. They are not simply played back; they are passed through a generative architecture of noise operators and visual feedback loops.

  • The Cognitive Architecture: The “network” in TouchDesigner acts as the brain’s sorting mechanism. It creates real-time distortions where the video signals bleed into one another.
  • Feedback Loops: By feeding the output of the TOPs back into the input (Video Feedback), we create “ghosting” trails where past frames linger over present ones. The network decides which fragments are displayed with clarity and which are distorted into abstraction. This simulates the memory paradox: as the signal loops and repeats, it becomes both more intense and less accurate, embodying the “incomputable” drift of remembrance.

CASE STUDY: PHANTOM TERRAINS Phantom Terrains is an interactive digital artwork that visualizes the paradox of memory: “The harder you try to forget, the clearer it becomes; the more you try to remember, the blurrier it gets.”

The Visual Language of the Phantom Visually, the work appears as a spectral landscape. It does not look like a photograph, nor does it look like a polygon video game. It looks like a painting in flux. The Gaussian splats are rendered as distinct brushstrokes that hang in the void. Because the data was captured incompletely, the landscape is full of holes—vast negative spaces where the algorithm had no data, and dense, hyper-real clusters where the “memory” is intense.

Implementing the “Incomputable” Logic The artwork focuses on the “Incomputable” elements—the artifacts. In a standard engineering context, the floating “noise” points caused by the algorithm’s uncertainty would be pruned (deleted). In Phantom Terrains, these are isolated and amplified.

We use Luciana Parisi’s (2013) concept of the “aleatory” to drive the animation. The movement of the splats is not determined by Newtonian physics (gravity/collision), but by the “alien logic” of the covariance matrices. We map the uncertainty of the splat (its scale/transparency

ratio) to its volatility. Small, opaque splats (high certainty) remain static, representing “grounded facts.” Large, transparent splats (high uncertainty) are subjected to turbulent noise fields, representing “false memories” or “hallucinations.”

The Memory Paradox via Network Design The core interaction is driven by a hybrid system that combines MediaPipe hand tracking with the TouchDesigner TOP (Texture Operator) network. MediaPipe detects the viewer’s hand landmarks in real-time (21 3D keypoints per hand), converting embodied gesture into parametric data that modulates the signal flow of the video fragments. This creates a feedback loop where the viewer’s attempts to “grasp” or “manipulate” memory physically trigger the very signal degradation that renders memory incomputable (see Figure 4).

Figure 4: Conceptual diagram of embodied interaction and media channel mapping, generated by Nano Banana AI. “The more you try to forget, the clearer it becomes” (Suppression - Clarity): When MediaPipe detects a resting hand state (low motion velocity, open palm facing away from the camera), the system interprets this as “non- engagement” or peripheral attention. The network responds by: Disabling all feedback loops within the TOP network. Maximizing contrast and saturation of the raw MP4 footage using Level TOP and limit TOP operators. Forcing the unprocessed Hong Kong housing estate footage to the foreground via the compositing hierarchy.
Original PDF Figure 4:

Result: The image snaps into aggressive, photorealistic clarity. The raw, unfiltered environmental capture—with all its architectural details intact— forces itself upon the viewer with unwanted vividness. The “phantom” is suppressed, and the material reality dominates.

“The more you think, the blurrier it gets” (Focus - Dissolution): When MediaPipe detects active hand engagement—specifically, when the viewer’s

Human-AI Landscape Visualization Through 3D Gaussian Point Clouds 917 hand moves toward the screen or performs a “grasping” gesture (detected via pinch distance between thumb and index finger)—the system escalates signal degradation: MediaPipe hand position and pinch confidence feed directly into a Noise CHOP and Feedback TOP. The pinch intensity (distance between finger landmarks) modulates the feedback loop opacity, creating “ghosting” where previous frames bleed into the present. A Displace TOP driven by Perlin noise warps pixel coordinates proportionally to hand velocity. The system applies cumulative color shifts via HSV Adjust TOP, cycling through hues as the gesture persists.

Result: The more vigorously the viewer attempts to “grab” or inspect a detail, the more the video signal self-destructs. The housing estate dissolves into a chaotic wash of color and temporal trails. The act of looking—of reaching toward memory—actively destroys the clarity of the image, embodying the “entropy” that Parisi identifies as intrinsic to computation. The “incomputable” is not a failure; it is the true nature of memory under scrutiny.

Technical Integration The MediaPipe outputs (hand landmarks, gesture recognition, pinch confidence) are streamed from Python into TouchDesigner via OSC (Open Sound Control) messages or CHOP Export, allowing real-time parameter binding. Each hand landmark can be mapped to TOP operations:

Thumb-Index pinch distance → Feedback loop intensity Wrist vertical position → Displacement noise scale Palm facing angle → Hue rotation speed Hand velocity → Temporal blur kernel size This embodied interface transforms the philosophical paradox of memory into a somatic experience, where the viewer’s own reaching gesture becomes the mechanism of forgetting.

DISCUSSION: THE RUPTURE OF THE DIGITAL OBJECT Beyond Deduction: The Probabilistic Image

Phantom Terrains serves as an aesthetic critique of the “Digital Twin.” A traditional Digital Twin implies a linear, deductive relationship: the Physical Reality (Input) logically determines the Digital Representation (Output). In this paradigm, the goal is a perfect correspondence where the output is a faithful mirror of the input.

However, as Parisi (2019) notes, AI introduces a rupture—a “heterogeneous” gap where the algorithm generates something new that is neither the original reality nor a predictable representation. The neural network does not simply copy; it processes massive amounts of contingency to produce a probabilistic “hallucination.”

The “Phantom” in the title refers to this gap. The landscape we see is not the housing estate we photographed, nor is it a pure invention. It is

a probabilistic distribution of light and signal. The viewer is not looking at a “wall” or a “tree”; they are looking at a mathematical prediction of where a wall or tree might be. This shift from “object” to “probability” is crucial for contemporary spatial cognition. It suggests that in an AI-mediated world, reality is negotiable—an “alien” logic that exists between the physical territory and its digital map.

The Aesthetic of the Organological Following Li et al. (2025), we observe that the “organological” nature of 3DGS lies in its structural similarity to the brain’s visual cortex—building a scene from discrete, overlapping signals rather than continuous surfaces. Phantom Terrains exploits this. By breaking the Gaussian cloud into fragments and reassembling them via the TouchDesigner network, we essentially build an externalized model of “forgetting.” The artwork becomes an apparatus for thinking with the machine, accepting its glitches and “incomputable” artifacts not as errors, but as the poetic substance of a new, shared reality.

CONCLUSION

This research repositions 3D Gaussian Splatting from a tool of geometric reproduction to a medium of “organological” reimagination. Through the development of Phantom Terrains, we have demonstrated how the specific technical affordances of the medium—its capacity to be captured as fragmented mobile footage and processed as a probabilistic video signal—can be harnessed to explore the “alien logic” of contemporary spatial memory.

By integrating MediaPipe hand tracking with TouchDesigner’s texture operators, the system creates a somatic feedback loop where the viewer’s own physical engagement dictates the stability of the digital image. This interaction design validates Luciana Parisi’s concept of the “incomputable”: the harder the viewer tries to “grasp” the memory (via active gesture and focus), the more the algorithmic entropy dissolves the image into abstraction. Conversely, it is only through “forgetting” (non-engagement) that the hyper- real clarity of the Hong Kong housing estates reveals itself.

Ultimately, Phantom Terrains suggests that in an era of automated thinking, the “Digital Twin” is a myth. What remains is a “Phantom”—a heterogeneous landscape generated from the friction between human intent and machine hallucination. The artwork does not seek to repair the glitches of the AI or fill the gaps of the scan; instead, it inhabits them, proposing that these “digital ruins” are the most accurate representations we have of a world increasingly mediated by algorithmic probability.

REFERENCES

Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3D Gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4), 139-1. Li, Xiaoqiao, Renjie Li, Cheuk-Kit Chung, Ho-Yin Ma, and Chi-Fu Lai. “Reimagining

Creativity: Gaussian Splatting as an Organological Extension of Imagination in Artistic Processes.” Leonardo 58.3 (2025): 241–249. Print.​

Human-AI Landscape Visualization Through 3D Gaussian Point Clouds 919 Parisi, L. (2013). Contagious Architecture: Computation, Aesthetics, and Space. The

MIT Press.​ Parisi, L. (2015). Instrumental reason, algorithmic capitalism, and the incomputable (pp. 125–137). Meson press. Parisi, L. (2016). Automated Thinking and the Limits of Reason. Cultural Studies ↔

Critical Methodologies, 16(5), 471–481.​ Parisi, L. (2019). Critical Computation: Digital Automata and General Artificial Thinking. Theory, Culture & Society, 36(2), 89–121.​

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2026): Disruptive and Innovative Technologies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-76-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →