Primarily lack of fine tuning options. Even 32k tokens is hilariously inadequate for something that could realistically cover our business domain.
If things don't clear up soon, I might go a DIY path with a more open model. At least on this path, I don't have a dark cloud of deprecation over my head.
We develop, configure and manage software products for US financial institutions.
The amount of information describing just one of our customers' needs would easily overrun any practical context window. Also, the cost of running maxed-out prompts 24/7 is definitely untenable at our scale per current OpenAI pricing models.
It's not like we need a ridiculous amount of context, but 32k tokens is definitely not enough. We are currently experimenting with fine-tunes of davinci & curie. Already seeing good progress with training runs as small as 200k tokens. We are going to take this up 2 orders of magnitude over the next few weeks.
I still have zero clue how the embed API is actually meant to be used. The documentation leaves a lot to be desired.
The lack of definition around use cases is making me think I got distracted with shiny bullshit again. The python notebooks with the olympic use case was difficult to follow without some abstract description/diagram/overview of how it all fits together.
