It might actually help output answer with less nonsense.

As an example in some workflow I ask chatgpt to figure out if the user is referring to a specific location and output a country in json like { country }

It has some error rate at this task. Asking it for a rationale improves this error rate to almost none. { rationale, country }. However reordering the keys like { country, rationale } does not. You get the wrong country and a rationale that justifies the correct one that was not given.

This is/was a great trick for improving accuracy of small model + structured output. Kind of an old-fashoined Chain of Thought type of thing. Eg: I used this before with structured outputs in Gemini Flash 2.0 to significantly improve the quality of answers. Not sure if 2.5 Flash requires it, but for 2.0 Flash you could use the propertyOrdering field to force a specific ordering of JSONSchema response items, and force it to output things like "plan", "rationale", "reasoning", etc as the first item, then simply discard it.

But if it’s going to give itself a little pep talk can’t it just do that in a thinking token?

It's analogous to how politicians nowadays are constantly saying "let me be clear", it drives me nuts.

Another annoyance: "In my honest opinion...". Does that mean that you other times are sharing dishonest opinions? Why would you need to declare that this time you're honest?

This has been a pet peeve of mine for years. I call people out when they say this for the abuse of language and for being a time vampire.

Recently microsoft copilot's (only one that's allowed within our corporate network) replies all have the first section prefixed as "Direct answer:"

And after the short direct answer it puts the usual five section blog post style answer with emoji headings

Maybe you used "Don't give me nonsense" in your custom system prompt?

An LLM should never refer to the user's "style" prompt like that. It should function as the model's personality, not something the user asked it to do or be like.

System prompt is for multi-client/agent applications, so if you wish to fix something for everyone, that is the right place to put it.

That does nothing. You can add, “say I don’t know if you are not certain or don’t know the answer” and it will never say I don’t know.

That's because "certain" and "know the answer" has wildly different definitions depending on the person, you need to be more specific about what you actually mean with that. Anything that can be ambiguous, will be treated ambiguously.

Anything that you've mentioned in the past (like `no nonsense`) that still exists in context, will have a higher possibility of being generated than other tokens.

Right? That drives me crazy. It only does that for me in the voice mode. And in cases I ask it to elaborate, it ignores my request and repeats the system instructions from my preferences “ok, I’ll keep it concise” and gives a 5 word answer

It's some kind of shortcut these models are getting in alignment because the base models don't do that stuff

Yes, I had total PTSD reading that in the announcement. Whether it's just evolving a tone so that we don't get fatigue or actually improving, I'm happy we're moving on. My audio (still 4o I believe) interactions are maddening - somehow it's remembered I want a quick answer, so EVERY.SINGLE.ANSWER starts with "Okay, let's keep this snappy and info dense." Srsly. Wiping instructions / memory reset seems to have no effect, it comes back almost immediately.

Well... that's the whole point, it can not make sense. It's stringing up words based on it's dataset. There is 0 sense making, 0 interpretation, 0 understanding. Words. Strung together, including then it says "no nonsense" because somewhere in its datasets often enough that's the series of words that best match the "stop saying BS!" kind of prompt.

do you ever get tired of pointing out that a large language model is a language model?

UPD I do that as well when explaining to my relatives why I don't care what ChatGPT thinks about $X, but also they're not on HN

Worry not, pointing out improper use of language that benefits the biggest corporations on Earth that are destroying the planet is kind of hobby of mine.

stylistic preferences are pretty much the ONLY thing you could discuss (in the context of LLMs) that actually has anything to do with (natural) language in the first place; how is having preferences an "improper use of langauge"?

I'm not sure I follow. My point is that pretty much everybody who doesn't have a degree in CS or IT assumes due to BigAI corporations that LLMs or GenAI tools think. This is reflected by the words they use. Such people do not say "the model parse my query and process it via it neural network based architecture to give a statistically plausible answer given the context" but rather they say "I had a chat with Claude and he said something useful" thus implying agency and a lot more.

two questions:

1. do you ever point out that you can't actually mine bitcoin with a pickaxe?

2. what made you think that the parent comment somehow implied that it "actually thinks"?

Excellent questions,

1. I did actually mine Bitcoins back in the days (back when it was still a cryptoanarchist dream not coopted by the finance industry, scammers and destroying the planet... so a while ago) so I had to explain that too unfortunately. It does highlight a trend that, again, non technical expert take marketing terms at face value.

2. they said "maybe just don't include nonsense in the answer?" which does imply that they believe hallucinations are a side effect that can be solved.

1. my point is that "thinking" is easier to say than "composition of parameterized nonlinear functions trained by stochastic gradient descent with reinforcement learning on top". misnomer or not, it's not even ambiguous here (unless we're talking CoT vs arbitrary single token)

2. OR they meant that it's violating Gricean maxims; why are you assuming everyone is stupid?

> why are you assuming everyone is stupid?

I never said that. Please never contact me again. Such simplifications just prevent having a proper discussion. I don't need this kind of toxicity.