Right here’s how to not waste your finances on evaluating fashions and methods
You possibly can construct a fortress in two methods: Begin stacking bricks one above the opposite, or draw an image of the fortress you’re about to construct and plan its execution; then, maintain evaluating it towards your plan.
Everyone knows the second is the one method we will probably construct a fortress.
Generally, I’m the worst follower of my recommendation. I’m speaking about leaping straight right into a pocket book to construct an LLM app. It’s the worst factor we will do to spoil our challenge.
Earlier than we start something, we’d like a mechanism to inform us we’re transferring in the suitable path — to say that the very last thing we tried was higher than earlier than (or in any other case.)
In software program engineering, it’s referred to as test-driven improvement. For machine studying, it’s analysis.
Step one and probably the most priceless talent in growing LLM-powered functions is to outline the way you’ll consider your challenge.
Evaluating LLM functions is nowhere like software program testing. I don’t undermine the challenges in software program testing, however evaluating LLMs isn’t as easy as testing.
