I’ve been experimenting with routing between models, and the biggest headache has definitely been managing caching and token costs. It is tricky to balance performance versus price when you are switching back and forth between different providers LLM synthesis step
