David W. Romero

David W. Romero

PhD. Student in Efficient Deep Learning

Vrije Universiteit Amsterdam

Biography

I am a final-year PhD Student at the Vrije Universiteit Amsterdam supervised by Erik Bekkers (UvA), Jakub Tomczak and Mark Hoogendoorn. I have spent time at Mitsubishi Electric Research Laboratories, Qualcomm AI Research and Google Research.

My research interests include all aspects of efficiency in Deep Learning, such as data efficiency, computational efficiency, and parameter efficiency. My specific focus is on continuous relaxations and parameterizations of Deep Learning methods, such as Continuous Kernel Convolutions. Continuous Kernel Convolutions are a new family of neural parameterizations with interesting efficiency properties for which I received the Qualcomm Innovation Fellowship.

In my free time, I enjoy learning new things, such as coffee making and carpentry, and doing sports like fitness and basketball.

Interests

  • Efficient Deep Learning
  • Continuous parameterizations
  • Symmetry equivariance

Education

  • PhD. in Efficient Deep Learning

    Vrije Universiteit Amsterdam

  • MSc. Computational Engineering, 2018

    Technische Universität Berlin

  • BSc. Mechatronic Engineering, 2016

    Universidad Nacional de Colombia

Publications

Towards a General Purpose CNN for Long Range Dependencies in $N$D

We demonstrate that Continuous Convolutional Kernels allow creating a single neural architecture that achieves state of the art on several different tasks in 1D and 2D.
PDF Code Slides

Learning Partial Equivariances From Data

We provide a method with which equivariant networks can adapt to the equivariance levels exhibited in data via backpropagation.
PDF

FlexConv: Continuous Kernel Convolutions With Differentiable Kernel Sizes

We demonstrate that the kernel size of CNNs can be learned via backpropagation by using continuous kernel parameterizations.
PDF Code Poster Slides Demos

CKConv: Continuous Kernel Convolution For Sequential Data

We provide a way to model long-term interactions, without vanishing gradients, in parallel and within a single layer. It naturally handles irregularly-sampled data and sampling rate discrepancies.
PDF Code Slides Demos

Group Equivariant Stand-Alone Self-Attention For Vision

We provide a general self-attention formulation to impose group equivariance to arbitrary symmetry groups.
PDF Code Slides Demos

Wavelet Networks: Scale Equivariant Learning From Raw Waveforms

We utilize scale-translation equivariance for learning on raw time-series, e.g., audio.
PDF Code

Attentive Group Equivariant Convolutional Networks

A generalization of equivariant visual attention to arbitrary groups.
PDF Code Slides Video

Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring in Data

We utilize attention to focus learning on the set of co-occurring transformations in data.
PDF Slides Video

▾   Education

▾   Experience

▾   Honors and Awards

Languages

Dutch

Fully Professional Proficiency (C1)

German

Fully Professional Proficiency (C1)

English

Fully Professional Proficiency (C1)

Spanish

Native Proficiency (C2)

Contact