Exploring Bayesian Optimization
How to tune hyperparameters for your machine learning model using Bayesian optimization.
Latest
The Transformer Family
[Updated on 2023-01-27: After almost three years, I did a big refactoring update of this post to incorporate a bunch of new Transformer models since 2020.…
An Overview of Early Vision in InceptionV1
An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of 'neuron groups.'
Visualizing Neural Networks with the Grand Tour
By focusing on linear dimensionality reduction, we show how to visualize many dynamic phenomena in neural networks.
Thread: Circuits
What can we learn if we invest heavily in reverse engineering a single neural network?
Zoom In: An Introduction to Circuits
By studying the connections between neurons, we can find meaningful algorithms in the weights of neural networks.
Growing Neural Cellular Automata
Training an end-to-end differentiable, self-organising cellular automata model of morphogenesis, able to both grow and regenerate specific patterns.
Curriculum for Reinforcement Learning
[Updated on 2020-02-03: mentioning PCG in the “Task-Specific Curriculum” section. [Updated on 2020-02-04: Add a new “curriculum through distillation” section.
Visualizing the Impact of Feature Attribution Baselines
Exploring the baseline input hyperparameter, and how it impacts interpretations of neural network behavior.
Computing Receptive Fields of Convolutional Neural Networks
Detailed derivations and open-source code to analyze the receptive fields of convnets.