Artificial intelligence techniques applied to the filming of the Lumière brothers, considered the parents of cinema, gave rise to an unprecedented restoration of content made available on YouTube last Sunday (3). Denis Shiryaev, responsible for the initiative, used machine learning methods to color and optimize a collection of 21 works by the pioneers of the seventh art, dating from 1895 to 1902.
To make the project possible, Shiryaev first used an algorithm developed by his company, Neural Love, with it, removing duplicate frames and restoring the original rate of the films – a crucial step for maintaining the fluid effect after the transition to 60 frames per second, according to him.
After that, he adds, it was necessary to stabilize the catches and remove noise at the necessary points, in addition to correcting eventual damages. Neural networks, in turn, increased each frame to 4K resolution, generated more frames until everything reached 60 fps, and restored a realistic playback speed.
In the final stages, Denis colored the materials, added details to the faces with an algorithm based on a generative adversary network and inserted new sounds, which “are not real”, he indicates.
“At certain points, I did a terrible job. Still, I really like, for example, looking at a dog on the screen and, when it starts to bark, feeling like I’m really in the room.”
Art & Entertainment
Not everyone approves of remastering. Researchers fear that such actions will create distorted images of the past and harm the choices of the original authors, even removing elements characteristic of the times of the works.
Shiryaev, however, defends himself, saying that his goal is only artistic and directed to entertainment, although historically mistaken. It also highlights technological limitations, even with so many advances.
“Neural networks can only guess colors when we are doing colorization, generate new pixels when we are doing an upscale and assemble faces when we work with some facial restoration”, he concludes, revealing the process step by step.