Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

LLMs can count words (and letters) just fine if you train them to do so.

Consider the fact that GPT-4 can generate valid XML (meaning balanced tags, quotes etc) in base64-encoded form. Without CoT, just direct output.



That's GPT-4, which you wouldn't use for in-line suggestions because it's too slow.

I don't know what model Copilot uses these days, but it constantly makes bracket mistakes in Python.


You don't need a GPT-4-sized model to count brackets. You just need to make sure that your training data includes enough cases like that for NN to learn it. My point is that GPT-4 can do much more complicated things than that, so there's nothing specific about LMs that preclude them from doing this kind of stuff right.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: