It takes $2-$3M to train a custom model from scratch using OpenAI!!GPT-4 finetuning service cost starts at "$2-3 million" and requires "billions of tokens at minimum", It sounds terrifying, but could actually be a good deal to medium-sized companies. Think about how much resources you need to set up the pipeline in-house:
- Pay big salaries to top AI engineers ($300k+/yr). At least 5 of them.
- Pay eye-watering cloud bills or buy GPUs and rent facilities.
- Set up training infrastructure - really good distributed systems engineer required.
- Iterate lots of times on open-source models. You won't get it right in the first few tries.
- Scale up deployment pipelines.
- Monitor reliability.
- Worry about efficient serving.
- And even after all this: your finetuned Llama-2 will still trail far, far behind a finetuned GPT-4.
The amount of work that goes into a reliable LLM in production is mind-boggling.
