https://cs.lozic.me/areas/machine-learning/Recent content in Machine-Learning on CS TheoremsHugoen-usFri, 16 Apr 2027 12:00:00 +0100https://cs.lozic.me/posts/t025-minimum-description-length-and-occams-razor/Fri, 16 Apr 2027 12:00:00 +0100https://cs.lozic.me/posts/t025-minimum-description-length-and-occams-razor/<h2 id="symptom">Symptom</h2> <p>You fit a model. It scores 94% on training data and 71% on held-out data.</p> <p>So you simplify: fewer parameters, more regularization. Training drops to 88%, held-out rises to 84%. You simplify further and both drop. Somewhere in there was an optimum, and you found it by trial and error, with a validation set and patience.</p>