Garrett Bingham

Garrett Bingham

About Me


I am a first-year Ph.D. student in the Neural Networks Research Group at The University of Texas at Austin. I am interested in the automatic design of neural network architectures, and more broadly in techniques for automated machine learning.

 

Research


Evolutionary Optimization of Deep Learning Activation Functions

Garrett Bingham*, William Macke*, and Risto Miikkulainen
GECCO 2020

LEAF

The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains the most commonly-used in practice. This paper shows that evolutionary algorithms can discover novel activation functions that outperform ReLU. A tree-based search space of candidate activation functions is defined and explored with mutation, crossover, and exhaustive search. Experiments on training wide residual networks on the CIFAR-10 and CIFAR-100 image datasets show that this approach is effective. Replacing ReLU with evolved activation functions results in statistically significant increases in network accuracy. Optimal performance is achieved when evolution is allowed to customize activation functions to a particular task; however, these novel activation functions are shown to generalize, achieving high performance across tasks. Evolutionary optimization of activation functions is therefore a promising new dimension of metalearning in neural networks.

 

Improving Low-Resource Cross-lingual Document Retrieval by Reranking with Deep Bilingual Representations

Rui Zhang, Caitlin Westerfield, Sungrok Shim, Garrett Bingham, Alexander Fabbri, William Hu, Neha Verma, Dragomir Radev
ACL 2019

ACL

Low-resource cross-lingual document retrieval performance is improved with deep bilingual query-document representations. Experimental results on the MATERIAL dataset show that our model outperforms the competitive translation-based baselines on English-Swahili, English-Tagalog, and English-Somali cross-lingual information retrieval tasks.

 

Part of Speech Tagging with Neural Architecture Search

Garrett Bingham

DARTS Neural Architecture Search

An automatically discovered bidirectional recurrent architecture nearly matches state-of-the-art accuracy for part of speech tagging across 60 treebanks.
Image credit: https://github.com/quark0/darts

 

Ineffectiveness of Gradient-based Neural Architecture Search

Garrett Bingham

Ineffectiveness of Gradient-based Neural Architecture Search

Most neural architecture search approaches utilize reinforcement learning or neuroevolutionary methods. Architecture optimization by gradient descent has been considered as a possible alternative. However, by training language models on Penn treebank, we demonstrate that gradient descent explores the search space ineffectively, and find that randomly initialized architectures are often able to outperform those discovered after extensive searching. We argue that gradient descent simply serves as a proxy for arbitrarily modifying the architecture, and show that gradient descent does not discover more capable architectures with each iteration of architecture search.

 

Preliminary Studies on a Large Face Database

Benjamin Yip, Garrett Bingham, Katherine Kempfert, Jonathan Fabish, Troy Kling, Cuixian Chen, and Yishi Wang
2018 IEEE International Conference on Big Data

MORPH-II

I discovered thousands of gender, race, and birthdate inconsistencies in the MORPH-II face image dataset that previously published research had missed. In this paper we discuss our strategy to fix these errors and release these corrections in the hope that future research utilizing MORPH-II will be more accurate.

 

Random Subspace Two-dimensional LDA for Face Recognition

Garrett Bingham

RS-2DLDA

Random Subspace Two-dimensional LDA (RS-2DLDA) improves upon a 2D generalization of LDA in which the input data is left in matrix form instead of being vectorized. RS-2DLDA builds an ensemble of classifiers by performing k-nearest neighbor classification in subspaces defined by random selections of the feature vectors learned during training. This gives high accuracy and prevents overfitting. Applied to face recognition, RS-2DLDA outperformed similar approaches on the MORPH-II and ORL datasets.

 

News


UT Austin

I began my Ph.D. studies at The University of Texas at Austin.

Amazon

In Amazon's Robotics AI group I developed an automated and parallelized solution for a video classification workflow.

Yale University

I graduated cum laude from Yale University with a B.S. in Computer Science & Mathematics.

LILY Lab

In the Language, Information, and Learning at Yale (LILY) Lab, I worked on cross-lingual information retrieval for the IARPA MATERIAL Program under the supervision of Prof. Dragomir Radev.

Reservoir Labs

At Reservoir Labs, I used a polyhedral compiler to speed up neural network computations by improving a tool called R-Stream.TF. I was supervised by Dr. Benoit Meister.

AIT-Budapest

I studied computer science at the Aquincum Institute of Technology in Budapest, Hungary.

University of North Carolina Wilmington

Supervised by Dr. Cuixian Chen and Dr. Yishi Wang at UNC Wilmington, I developed a new algorithm for face recognition called RS-2DLDA. I presented a poster of my work at the Council on Undergraduate Research REU Symposium at The Westin Alexandria and delivered an oral presentation at the NES Mathematical Association of America Meeting hosted by Sacred Heart University.