Tremendo
Deep learning powered music video
   

---

Video clip realizado utilizando material de archivo de la Filmoteca Española, en particular de la excelente colección Sagarminaga que contiene escenas costumbristas y documentales de finales del siglo XIX y principios del XX. El fragmento más antiguo que hemos utilizado data de 1896 y el más moderno de 1904.

Todo el material ha sido remasterizado y procesado con inteligencia artificial. Se utilizaron algoritmos open-source de Deep learning para colorear, reescalar y aumentar el frame rate de los vídeos para darles un look contemporáneo. A las imágenes documentales le hemos añadido una serie de deepfakes de imágenes del archivo de la Biblioteca Nacional de la misma época.

El vídeo clip es una colaboración entre Espadaysantacruz Studio & NYSU ( https://nysuforever.net/) para La juventud ( https://www.instagram.com/lajuventud.divinotesoro)

Tenemos que agradecer profundamente a toda la comunicad científica que publica sus librerías de deep learning y que nos permite integrarlas en procesos creativos. En particular hemos utilizado los siguientes procesos.

Remasterizado y limpiado de los vídeos originales on DeepRemaster basado en Temporal convolutional neural networks.

Conversión a 75fps con DAIN: Depth-Aware video frame Interpolation

Coloreado con DeOldify

Escalado a 4k con ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

DeepFakes con Deep Learning First Order Motion Model for Image Animation

Programado en Python con PyTorch corriendo en una GPU Tesla P100 en Google Cloud

 

Colorizing tests

---

Music made using archive material from the National Spanish Film Archive, in particular from the excellent Sagarminaga collection that contains traditional and documentary scenes from the late 19th and early 20th centuries. The oldest fragment we have used dates from 1896 and the most modern from 1904.

All the material has been remastered and processed using artificial intelligence. Open-source Deep learning algorithms were used to color, rescale and increase the frame rate of the videos to give them a contemporary look. To the documentary images we have added a series of deepfakes of pictures of the archive of the National Library of the same period.

The music video is a collaboration between Espadaysantacruz Studio & NYSU ( https://nysuforever.net/ ) for La Juventud ( https://www.instagram.com/lajuventud.divinotesoro/ ).

We have to deeply thank all the scientific community that publishes the deep learning libraries, allowing to integrate them into creative processes. In particular we have used the following processes.

Video Enhancement using DeepRemaster based on temporal convolutional neural networks.

FPS Boosting using de Deep Learning with DAIN: Depth-Aware video frame Interpolation

Deep Learning colorizing using DeOldify

4K Upscaling using ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

DeepFakes using the Deep Learning First Order Motion Model for Image Animation

Running in a single GPU Tesla P100 on Google Cloud

Coded in Python with PyTorch

 

Deepfake tests

A Project by: Espadaysantacruz Studio & NYSU films
Client: La Juventud
Powered by: Python
May - 2020