The latest projection mapping IS Yabe.Everything in this world can be displayed

How to apply light and measure.Just this is so amazing.

The super -high -speed projection mapping, which was interviewed at SIGGRAPH ASIA three years ago, has evolved.Even though it is evolution, it is a ridiculous evolution as if Pikachu became the god of electricity itself, rather than a corresponding evolution that a lizard became a lizard!

First of all, look at this GIF video for a small hand.

Yes, it is a magic ball that disappears.Even though the red hockey pack is flying from above, the pack is gone on the way, right?The magic -like idea has been realized by Watanabe Lab / Tokyo Tech, a dynamic projection mapping based on depth information using high -speed RGB and IR projectors.

What is on display is a unique projection mapping technology.But I don't use existing software or trackers.So how are you tracking?

I use this custom set.High -speed infrared projector, high -speed RGB projector, high -speed infrared camera from above.All high speed.

Technically (this technology is amazing), the infrared projector that was custom -made is irradiated with a striped pattern, and the pattern is read with a camera to create a depth map.The tracking of the camera at this time is 500Hz.We review the depth 500 times per second.In addition, there are six stripes pattern patterns to read the depth, and the camera captures a small gap of each six, recognizing the accurate depth (it seems to be a phase shift).

The striped pattern at the lower left is the image of the infrared camera (one of the six stripes).The image on the upper left is the depth of the depth based on that, and the image on the right is actually visible.

We can see such a face in our eyes (Shimashima, which looks like a black shadow border, is the shadow of infrared rays).Did you think it was a beautiful Kabuki face?

It's really a pure white face.The shape of the modeling object is accurately recognized, and no matter how much it moves, the image is masked in the appropriate position.Not only the face but also the three -dimensional effect of the throat Buddha and the contour expression were very real, and when I saw it for the first time, I thought it was such a figurine.I will be deceived so much.

By the way, this "Dynamic projection mapping based on depth information using high -speed RGB / IR projector" is a long -term dynamic projection, but the approach is different from general projections in various ways.

First of all, there is no need for trackers.By irradiating and reading infrared rays, the shape of the projected target is grasped as an accurate three -dimensional product.The accuracy of tracking is also high (0.Tracking at 4ms).It's about following the hockey pack.

Oh yeah, it is the principle of the disappearing magic ball at the beginning, but it is realized by hiding the pack with black light (striped pattern) → illuminating the entire surface alternately.It seems that the movement of the pack is invisible by alternately flashing in black and white so that we cannot recognize the flashing of fluorescent lights.Proof that tracking accuracy is high enough to accurately recognize the pack that moves at high speed!What a god's idea!

In addition, it seems that the software was written in C language to realize this high -speed processing.It is generally said that the time of writing the program and the time required for processing are inversely proportional.Easy -to -understand programming languages such as Python are easy to write for humans, but they tend to be a little more wasteful.The C language is quite a language -like language, so it's hard for humans to understand and it takes time to write, but the processing is crispy.Shave the latency even over time!This is the engineer soul!!

Infrared tracking and C language (high -level language).These approaches have achieved a super -low latency of 8 ms for the entire dynamic projection.Even if the projection is moved, it will follow at 8/1000 seconds.No more unconscious perception speed.

I was also taught the breakdown of this 8MS.4ms + calculation processing by imaging processing such as infrared rays 0.8ms + 2ms = about 8 ms for projector projectors projected to data transmission to projectors.After that, it seems that these processes can be driven quickly, but the GPU is a bottleneck in the image processing part.If the data band is thicker, the number of tracking can be raised from 500Hz to 3000Hz.

So how can this high -speed and high -precision dynamic projections be used?Well, you can use it for anything.Rather, the challenge is that the application is too wide and you have to squeeze it.For example, like this rabbit, it can deceive the texture of the object.Since trackers are unnecessary, if you decide the focal length between the projector and the screen, the freedom of installation is high.Extreme story, you can make it look like you are dressed even if you stand naked, or you can make your angry face look like a laughing face.

And this is no longer a visual hack that does not require goggles.In the first place, I'm looking at the light with a sight, so if you control the light more accurately, you'll be able to deceive as much as you can.I was actually deceived, so I looked at it!

Sony displayed the background video on a large LED and used it for video production, but if you look at it, it is more important to see how you look at it.I feel that the scale of the future video technology looks like a flicker.Even if there is no cyber brain, our five senses are easily hacked.I'm sorry.

Source: Siggraph ASIA 2021