FIELD NOTES01
What happens to a neural network when you round its weights
Round a real detector’s weights to 8, 4 or 2 bits and see which layers, and which boxes, break first.
Essays, notebooks, and field notes connected by optimization.
Round a real detector’s weights to 8, 4 or 2 bits and see which layers, and which boxes, break first.
Retrain a linear classifier under class weighting, oversampling, undersampling, and SMOTE, and watch accuracy diverge from minority recall.
Step through boosting rounds on noisy 1D data and see shrinkage and tree depth trade convergence speed for stability.
Drag a starting point across a bowl, a ravine, and a Rosenbrock valley and watch gradient descent, momentum, RMSProp, and Adam disagree.