Nvidia has publicly released the code for its groundbreaking framework, Neuralangelo, to create ultra-high definition 3D models from regular video clips.
Now available on GitHub, the tool allows users to take a high-quality video clip and convert it into a 3D model, which can then be used in game development, movie effects, architecture, engineering, or even entire metaverses.
The Nvidia research website states the Neuralangelo framework “facilitates the creation of detailed digital twins of object-centric and large-scale real-world scenes with the aid of ubiquitous mobile devices.”
A video by Nvidia’s research team explained that Neuralangelo uses multi-resolution 3D hash grids and surface rendering to make detailed 3D models. These methods improve the creation process using numerical gradients and coarse-to-fine optimization techniques, particularly helpful when modeling large, smooth areas. Without these techniques, these areas often show up with lots of noise and gaps in the model.
In the study, Neuralangelo is shown to effectively use RGB video captures to make detailed 3D models, including large models of the Nvidia headquarters park and Johns Hopkins University.
The footage uses a consumer drone to fly around the Nvidia park, which is then converted into a 3D model with highly detailed surfaces.
Source: Nvidia Research
Also, Neuralangelo performs well on the DTU benchmark, a standard test in this field, verifying that it can make detailed 3D models while keeping smooth surfaces intact. Angel Investor Javi Lopez praised Neuralangelo on X, quoting Arthur C Clarke, “Any sufficiently advanced technology is indistinguishable from magic.”
Compared to Nvidia’s previous method called Instant NeRF, Neuralangelo is a significant improvement. It makes smoother 3D models by using numerical gradients to work out the smoothing operation and by controlling the amount of detail at different stages of the modeling process.
From a broader perspective, the release of Neuralangelo is a significant advancement in the field of 3D modeling at large, potentially revolutionizing and democratizing applications in video games, cinema, virtual environments, and more.
By providing a reliable and efficient means to achieve high-fidelity 3D surface reconstruction, Neuralangelo underscores the continuous evolution of technology in bridging the gap between a virtual ‘metaverse’ and the real world.
The post Turn drone footage into 3D metaverses as Nvidia launches new AI tool appeared first on CryptoSlate.
