Large language models don’t actually follow a format. They generate text by sampling probabilities, one token at a time. You can write the best prompt imaginable—« Respond only with valid JSON. This is extremely important. »—and the model will still occasionally get it wrong. Not every time, but often enough to break a production application. As long…
