How to Shrink a Language Model Without Making it Too Dumb

Models have grown roughly 100-fold in a few years, while consumer graphics memory has roughly doubled. It’s not just a matter of tightening things up to make them fit. ✦ AI generated