I would like to know the cost of enabling this type of selective specialization of the models.

If it’s particularly not intensive, I wouldn’t be surprised if model architecture moves towards self-specialization or topic-selection with some effective function calling e.g. model used for a while -> automatically call model specializer after a few queries on the topic -> now use the newly returned specialized LLM

I wonder at what magnitude this could improve model efficacy