Not enough people talking about this obvious fact that most people shouldn't bother training models (I very cursoryily skimmed mostly the title). It was the TLDR of my talk on "post training for applications" https://www.youtube.com/watch?v=grpc-Wyy-Zg
But isn't anyone who is doing fine-tuning really using lora adapters? Who's got the compute to be doing full fine-tuning of big models? So the capacity argument is less clear.
This post is hilarious. People like this author are the ones vetting start-ups? Please. The idea that alignment leads to a degradation in model utility is hardly news.
But let’s be clear: fine-tuning an LLM to specialize in a task isn’t just about minimizing utility loss. It’s about trade-offs. You have to weigh what you gain against what you lose.
Not enough people talking about this obvious fact that most people shouldn't bother training models (I very cursoryily skimmed mostly the title). It was the TLDR of my talk on "post training for applications" https://www.youtube.com/watch?v=grpc-Wyy-Zg
Most people should just not do it!
lol that comment on british and french was cold
Thank you
Isn't everyone doing LoRA nowadays when they say they are fine tunning?
But isn't anyone who is doing fine-tuning really using lora adapters? Who's got the compute to be doing full fine-tuning of big models? So the capacity argument is less clear.
This post is hilarious. People like this author are the ones vetting start-ups? Please. The idea that alignment leads to a degradation in model utility is hardly news.
But let’s be clear: fine-tuning an LLM to specialize in a task isn’t just about minimizing utility loss. It’s about trade-offs. You have to weigh what you gain against what you lose.
Great post! Adapters are still underappreciated. You may be interested in this novel adapter method:
https://stenruediger.substack.com/p/supercharge-your-llms-introducing
Fascinating read! I’m Harrison, an ex fine dining line cook. My stack "The Secret Ingredient" adapts hit restaurant recipes for easy home cooking.
check us out:
https://thesecretingredient.substack.com