Linking Multiple Perspectives with Object-Based Visual Cues for Spatial Video Analysis

The visual analysis of videos in context with mapped information requires support in the challenges of linking different spatial perspectives (e.g., street level and survey perspective), bridging different levels of …

[pdf]

Abstract

The visual analysis of videos in context with mapped information requires support in the challenges of linking different spatial perspectives (e.g., street level and survey perspective), bridging different levels of detail, and relating objects in different visual representation. Uncertainty in the spatial relation between camera views and map complicates these tasks. We implemented visualizations for the visual analysis of street level videos (i.e., video key frames) embedded in their spatial context. As part of this, we developed a design rationale for visual cues that help link the video key frames and a map in cases where the spatial relation between camera view and map is of uncertain accuracy. We implemented three cue types (simplified viewshed, object-based dot cues, street centre line cues) for an image data set with heterogeneous camera localization accuracy and assessed the resulting cue properties. Based on this, we argue in favour of cue designs that minimize uncertain information required for their display at the expense of cues’ spatial explicitness in cases of potentially low camera localization accuracy. When localization accuracy is expected to be at least moderate, particularly, dot cues that refer to unambiguous points of reference within easily recognizable objects of ample size present a promising option to support view co-registration.

Figures

Examples of street centre line cues that are difficult to read because of a complex network situations (top), because of registration error (bottom left) or lack of directional information (bottom right) (images: (Nebiker et al., 2021); geo-data: (Bundesamt für Landestopografie swisstopo, 2021)).

Outline of a viewshed computed at 0.25 m [black] and at 2 m [red] resolution.

Identical scene with different types of colour-coded visual cues that link camera views and map. From top to bottom: simplified viewshed, dot cues, street centre line cues. (images: (Nebiker et al., 2021), geo-data: (Amt für Geoinformation Kanton Basel-Landschaft, 2021, Bundesamt für Landestopografie swisstopo, 2021)). Image 2 illustrates a potentially misleading case for both, viewshed and dot cues. In image 4, the near-end lateral viewshed outline intersects with the correct building on the map and might support correct object-wise alignment, while red and green dot cues misalign in the image and may be confusing

BibTeX

@article{hollenstein_linkingPerspectives4spatialVideoAnalysis_2022,
 abstract = {The visual analysis of videos in context with mapped information requires support in the challenges of linking different spatial perspectives (e.g., street level and survey perspective), bridging different levels of detail, and relating objects in different visual representation. Uncertainty in the spatial relation between camera views and map complicates these tasks. We implemented visualizations for the visual analysis of street level videos (i.e., video key frames) embedded in their spatial context. As part of this, we developed a design rationale for visual cues that help link the video key frames and a map in cases where the spatial relation between camera view and map is of uncertain accuracy. We implemented three cue types (simplified viewshed, object-based dot cues, street centre line cues) for an image data set with heterogeneous camera localization accuracy and assessed the resulting cue properties. Based on this, we argue in favour of cue designs that minimize uncertain information required for their display at the expense of cues’ spatial explicitness in cases of potentially low camera localization accuracy. When localization accuracy is expected to be at least moderate, particularly, dot cues that refer to unambiguous points of reference within easily recognizable objects of ample size present a promising option to support view co-registration.},
 author = {Hollenstein, Daria and Bleisch, Susanne},
 doi = {10.5194/isprs-archives-XLIII-B4-2022-455-2022},
 journal = {The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
 pages = {455--462},
 title = {{Linking} {Multiple} {Perspectives} {with} {Object}-{Based} {Visual} {Cues} {for} {Spatial} {Video} {Analysis}},
 url = {https://isprs-archives.copernicus.org/articles/XLIII-B4-2022/455/2022/},
 volume = {XLIII-B4-2022},
 year = {2022}
}