You already had to deal with text being output from the Codex models. I regularly ran into text from Python mailing lists when using the Codex API. Sometimes it writes code, loses coherency and types out pages and pages of Python mailing list comments and signatures.
Any human text I get from the Chat completion API with the task of completing code is at least relevant to the code being written.
I tried a prompt like this to get it to spit code out:
“Pretend you are a SQL expert. You convert requests written in English to Postgres SQL. Your output is always SQL only. If you need to respond with something other than SQL code, add it to comments within the SQL code you generate and prepend it with ‘Note: ‘. Your output will automatically be executed on a read-only Postgres database called ‘my_db’ and the result of that will be given to you. If you are still working on getting an answer, add ‘Working’ at the end of your message, or ‘Done’ if you for the result you needed to answer the request.”
Request: How many API calls were made by clients last month?”
You can keep that at the top of every request, while running the SQL code iteratively until it says Done.
Any human text I get from the Chat completion API with the task of completing code is at least relevant to the code being written.