CIDER: Collaborative Interactive Dynamic Environments for eXtended Reality
Hung-Jui Guo, Hiranya Garbha Kumar, Yung-Jen Lin Guo, Balakrishnan Prabhakaran · 2026 · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3796237
Summary
CIDER (Collaborative Interactive Dynamic Environments for eXtended Reality) is an XR platform designed to enable real-time remote collaboration by automatically replicating a physical environment as an interactive virtual space. The authors address a core problem in collaborative XR research: existing systems either suffer from high latency, rely on static scenes that cannot be manipulated, or require costly specialized hardware that restricts who can participate. The system uses a Leader-Follower paradigm. A single Leader wears a Microsoft HoloLens 2 and scans their physical environment using the device's scene understanding and depth sensing capabilities. CIDER automatically reconstructs that space using a pipeline that converts the raw scene mesh into a point cloud, runs 3D instance segmentation to detect furniture and objects, retrieves matching CAD models from a local database, and aligns them spatially. The resulting virtual environment is transmitted to a Unity Cloud Server and shared with Followers, who can join using HoloLens 2, Meta Quest 2/3 headsets, or ordinary Windows/Mac/Linux computers running a Unity emulator. A key design decision is data efficiency: rather than transmitting full 3D meshes, CIDER sends only lightweight JSON records containing model identifiers and pose data. Followers reconstruct the scene locally from their on-device CAD library. This approach dramatically reduces bandwidth requirements and keeps latency low even over intercontinental connections. The system also undergoes a Prefab Conversion pre-processing step that converts CAD models into Unity prefab format, achieving a 10x improvement in object loading times on the HoloLens 2. The paper evaluates CIDER across short-, medium-, and long-distance scenarios (up to 12,000 km) and conducts a 17-participant user study assessing usability and sense of presence.
Key findings
CIDER achieves an end-to-end reconstruction latency of approximately 5.8 seconds for a room-sized environment (55.2 square metres with three pieces of furniture), with interaction latency consistently around 0.22 seconds across all tested distances — more than 10 times lower than the closest comparable AR/MR collaboration system, PLANWELL (2.4 seconds). Unnavigated area update latency averaged around 0.32 seconds, and spatial perception accuracy (the positional consistency between physical and virtual objects as experienced by Followers) was approximately 0.2 metres. The 17-participant user study used the validated Spatial Presence Experience Scale (SPES) and Presence Questionnaire (PQ). Mean scores across all eight SPES items exceeded baseline scores from the original SPES validation study, indicating stronger-than-expected usability and presence. Questions assessing interactivity (Q2, Q5, Q6, Q7) received the highest ratings. Only Q10 — "Did you experience any delay between your actions and expected outcomes?" — received predominantly negative responses, suggesting participants noticed some perceptible latency despite the system's strong technical performance. The illumination-invariant 3D instance segmentation approach (trained on spatial features only, without colour data) achieved a mean IoU of 70.7 across categories, compared to 71.8 for the colour-trained baseline — a marginal trade-off that greatly improves robustness under real-world lighting conditions. Medium mesh detail with upsampling was identified as the optimal configuration, balancing reconstruction accuracy and processing speed on resource-constrained HMDs.
Relevance
CIDER is relevant to accessibility practice in two ways. First, the paper explicitly frames hardware accessibility as a design goal: by requiring only one MR headset (for the Leader) and enabling all Followers to participate via ordinary computers or common consumer VR headsets, CIDER lowers the cost barrier for remote XR collaboration substantially. This is a practically important finding for organizations — including rehabilitation services, educational institutions, and social services — considering XR for remote engagement with clients who cannot access specialist equipment. Second, CIDER demonstrates that immersive collaborative environments can be built without deep technical expertise. The automated pipeline and non-expert setup process are specifically cited as features that broaden the potential user base, including for cross-disciplinary research. For accessibility practitioners, the platform's use cases — remote education, virtual walkthroughs, collaborative spatial planning — have direct applicability to supporting people with disabilities who cannot physically travel to training or consultation spaces. Limitations include the Leader-only paradigm (a single physical environment is shared, limiting multi-source scenarios) and the current dependency on the discontinued HoloLens 2 for the Leader role, though the authors report migration to other HMDs is underway.
Tags: extended reality · mixed reality · virtual reality · remote collaboration · 3D reconstruction · accessibility · head-mounted displays · human-computer interaction