Weight Banding
Weights in the final layer of common visual models appear as horizontal bands. We investigate how and why.
Papers 5613 stories
Branch Specialization
When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings.
Multimodal Neurons in Artificial Neural Networks
We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.
Self-Organising Textures
Neural Cellular Automata learn to generate textures, exhibiting surprising properties.
Visualizing Weights
We present techniques for visualizing, contextualizing, and understanding neural network weights.
Curve Circuits
Reverse engineering the curve detection algorithm from InceptionV1 and reimplementing it from scratch.
High-Low Frequency Detectors
A family of early-vision neurons reacting to directional transitions from high to low spatial frequency.
Naturally Occurring Equivariance in Neural Networks
Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights.
Understanding RL Vision
With diverse environments, we can analyze, diagnose and edit deep reinforcement learning models using attribution.
Communicating with Interactive Articles
Examining the design of interactive articles by synthesizing theory from disciplines such as education, journalism, and visualization.
Self-classifying MNIST Digits
Training an end-to-end differentiable, self-organising cellular automata for classifying MNIST digits.
Thread: Differentiable Self-organizing Systems
A collection of articles and comments with the goal of understanding how to design robust and general purpose self-organizing systems.
Curve Detectors
Part one of a three part deep dive into the curve neuron family.
Exploring Bayesian Optimization
How to tune hyperparameters for your machine learning model using Bayesian optimization.
An Overview of Early Vision in InceptionV1
An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of 'neuron groups.'
Visualizing Neural Networks with the Grand Tour
By focusing on linear dimensionality reduction, we show how to visualize many dynamic phenomena in neural networks.
Zoom In: An Introduction to Circuits
By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks.
Thread: Circuits
What can we learn if we invest heavily in reverse engineering a single neural network?
Growing Neural Cellular Automata
Training an end-to-end differentiable, self-organising cellular automata model of morphogenesis, able to both grow and regenerate specific patterns.
Visualizing the Impact of Feature Attribution Baselines
Exploring the baseline input hyperparameter, and how it impacts interpretations of neural network behavior.
Computing Receptive Fields of Convolutional Neural Networks
Detailed derivations and open-source code to analyze the receptive fields of convnets.