Summary of Key Points
This article explores the century-old challenge of "new perspectives synthesis" – the ability to create images from angles that were never captured in the first place – by tracing the technological developments from the film era to the AI age. It covers the multi-camera 3D camera scams and pioneering stories of Nimslo/Nishika in the 1980s, the innovation and failure of the Lytro light field camera in the 2010s, and Apple's "Spatial Reconstruction" feature in iOS 27, which uses AI algorithms to generate depth and new perspectives from a single photo. Essentially, all these technologies are trying to solve the same problem: how to create an unlimited number of viewing angles with limited visual information.
1. Apple’s “Spatial Reconstruction”: The Secret Behind Making Photos Come Alive with AI
Apple's "Spatial Reconstruction" feature in iOS 27 turns static photos into three-dimensional images. When you swipe through the photos, AI automatically generates the obscured parts, as if you could view the scene from the side or back. This is achieved through two key technologies:
1. Depth Estimation: Using AI to determine the relative positions of objects in a single flat photo (for example, which person is in front and which tree is behind).
2. Detail Generation: Applying 3D Gaussian Blurring technology to break down the scene into millions of tiny "light points" and quickly create new perspectives, all in just one second.
This technology is the result of years of AI research by Apple (such as NeRF, or Neural Radiance Field). It doesn't require multiple cameras; a single lens is enough, allowing the phone to "imagine" perspectives that don't actually exist.
2. The “4-Lens Camera” from the Film Era: The Twisted Story of Pioneers and Scams
In the 1980s, two companies tried to solve the problem of creating new perspectives with multiple lenses, but their fates were very different:
- Nimslo: A technological pioneer with a tragic end
Hong Kong engineer Lu Guohua and American businessman Jerry Nims invented a 4-lens 3D camera that took four photos with slight differences using high-quality glass lenses, which could then be printed as a 3D effect without special glasses. However, the project failed due to factory strikes during production, the need to send photos to the United States for processing (which was expensive and time-consuming), and a price of $200 (equivalent to $700 today), leading to bankruptcy.
- Nishika: A scam that ultimately failed
Nishika acquired Nimslo's technology and scaled down the camera significantly, using plastic lenses, fake LCD screens, and a hollow body with added weight to give it a more substantial appearance. They also used a pyramid scheme to sell the cameras. The company went bankrupt within a year due to regulatory investigations in the US, and the cameras became worthless.
This shows that technological innovation without a viable business model or reliance on fraud cannot sustain itself for long.
3. The Lytro Light Field Camera: A Flash in the Pan
In 2011, Lytro introduced the world's first consumer-grade light field camera, which could capture information from all directions of light (effectively taking countless photos from different angles), allowing for "post-shot focusing." However, it also failed:
- The first generation had low resolution, resembling a blurry mosaic.
- The second generation was priced at $1599, which was too expensive.
Mobile phone manufacturers later achieved similar "refocusing" effects with dual cameras and simple algorithms (e.g., the HTC One M8), making it more affordable and convenient.
Lytro was eventually acquired by Google, and its technology became a foundation for VR/AR and Pixel phones. Nevertheless, it demonstrated the feasibility of capturing light fields, paving the way for subsequent AI advancements in photography.
4. The AI Era: How NeRF and Computational Photography Are Revolutionizing Photography
In 2020, the emergence of NeRF (Neural Radiance Field) technology changed the way 3D scenes are processed:
- Previously, creating 3D images required point clouds and models; NeRF uses AI to turn scenes into "calculable fog," allowing for the generation of images from any perspective without defining specific surfaces.
Subsequent 3D Gaussian Blurring technologies solved NeRF's speed issues, enabling real-time rendering (60 frames per second), which is what Apple's Spatial Reconstruction feature utilizes.
Google's computational photography techniques (e.g., Night Sight) have shown that the limits of mobile phone photography lie not in hardware but in algorithms. By combining multiple photos and using AI to generate details, phones can capture clearer images in low-light conditions than professional cameras. Both Apple and Samsung have adopted these technologies, marking the beginning of a new era of "computational photography."
5. The Common Goal of a Century-Long Technological History: Creating Unlimited Perspectives with Limited Information
From Lippmann's theory of "integral photography" in the 19th century, to Nimslo's multi-lens cameras, Lytro's light field technology, and Apple's Spatial Reconstruction, all these efforts aim at the same ancient goal: to capture a scene from every possible angle using just one or a few photos.
In the past, this was achieved through hardware (multiple lenses, microlenses); today, it's done with AI algorithms (depth estimation, 3D generation). The technology has evolved, but the goal remains the same: to transform photography from capturing moments into reconstructing spaces. In the future, perhaps taking a single photo will allow us to observe a scene as freely as if we were actually there. This is the allure of technological progress.
This article is essentially a micro-history of technology, illustrating that innovation doesn't always succeed, but each attempt lays the foundation for the future. True progress often comes from using smarter methods (like AI) to solve old problems. Apple's Spatial Reconstruction feature didn't appear out of nowhere; it builds on the work of countless pioneers and those who failed along the way.