Portals into the past: a VR app makes memories accessible

Wist transforms iPhone videos into space-filling scenes called Wist CEO Andrew Mchugh “spatial memories”. The AR constructs can be displayed in the real space and entered with mixed reality glasses such as Meta Quest or Apple Vision Pro. The technology is intended to keep memories alive in a way and make it tangible how it was not possible so far.

For the 3D reconstruction of the videos, Wist uses various data sources, including a depth image generated by the lidar sensor and the position and orientation of the iPhone in the room, determines the motion sensors. Since Wist stores the raw data, the spatial memories can be continuously improved by updates to the reconstruction pipeline. The app supports all PRO and PRO-MAX models from the iPhone 13.

No technical know-how required

We made the self -experiment and tried for an afternoon. Access to the early access version via test flight is currently required. The use is simple and intuitive: the app starts directly in the recording mode, so that the desired moment can be captured immediately with an ordinary camera app. After that, processing of the video begins, a process that can currently take several hours. As soon as the 3D reconstruction is completed, you get a notification. The maximum video length is currently 40 seconds.

The “spatial” videos can be projected directly into the room by AR, optionally on the iPhone display or significantly more immersive with a headset like Meta Quest 3 or Apple Vision Pro. A separate mixed reality app is available for the devices. Since the stored memories are uploaded to the cloud from the smartphone, no cumbersome data transfer is necessary: the scenes are available immediately when the application starts.

Wist tried with quest 3: a little holodeck

These scenes only develop their full charm in mixed reality glasses. Thanks to the wide range of visibility and stereoscopic depth, the motifs look alive and present: in our case a dog that has made itself comfortable in the summer heat on one piece of lawn. The fact that the pet and the surrounding area in original size directly in front of a appearance increases the reality impression.

In the evening we returned to the same place and took the recording with the help of the Passhrough mode exactly over the original scene to test the spatial accuracy of the 3D reconstruction. She blends into the real geometry surprisingly seamlessly. We stood in the same place and looked at the mixed reality projection of a dog that was there a few hours ago.

The illusion collapses as soon as you are too far from the original camera center. Objects fray and throw translucent shadows: empty spaces that the camera could not grasp and therefore cannot reconstruct. In the future, AI could fill these gaps. At the moment, these artifacts still irritate or, depending on the point of view, apply to the dystopian charm of spatial memories.

Immersive Memories: There are these solutions

Apple is one of the few large companies that sees potential in the idea of immersive memories. It advertises Apple Vision Pro as a device with which memories can be captured and experienced.

With Visionos 26, Apple introduced “Spatial Scenes”, a function that at least partially goes in this direction: it transforms 2D photos with the help of generative AI into multi-perspective backdrops. This function is not available for videos.

With his spatial memories, Wist goes beyond Apple’s solutions and accepts that the result has an imperfect effect. However, Wist cannot offer an all -round view in which people or events would be fully visible from every perspective.

A technique called Gaussian Splitting is suitable for snapshots of this kind, which is now also finding its way into Meta Quest. However, the recording of moving scenes remains a significant technical hurdle.

Reality inspires sci-fi and vice versa

The emotional effect of the spatial memories we recorded one day cannot yet be estimated. The time interval is missing. In a greater connection and with a view to long -term developments, the question arises as to how immersive memory techniques will affect human identity and whether nostalgia could become addicted from a certain degree of technical perfection. A topic that science fiction has often taken up.

The recent Black Mirror relay gives a surprisingly optimistic answer to this: In the episode “Eulogy”, a man wins a new, healing view of his past through technically reconstructed memories. The technology shown in it is strongly reminiscent of Wist and related techniques.