Taming the generative AI back end
The novel power of today’s AI is in its ability to deal with intent. This is a superpower, no doubt, but it creates a huge imperative for app developers: the need to map between the anything-is-possible large language model (LLM) and the strict capabilities of code. Unrestrained, LLM endpoints will let your user create unicorns and leprechauns while your back end can handle only purchase orders and customer profiles. You must harness the LLM’s ability to understand intent to what the app is logically capable of, meanwhile keeping context (and therefore spend) under control. Here I’ll discuss some practical, realistic techniques for doing that today. Between what the user wants to do and what your app is capable of is you. Or, more specifically, the mediation layer you build. This layer can sit anywhere on a broad spectrum, from using incredibly lightweight inline strings to using a massive retrieval-augmented generation (RAG) system backed by a vector database. Somewhere in there is t