> it's great that you can return 402 payment required, but where is the browser supposed to go from there?

Don't forget the browser is allowed to return a HTML page with the 402 error, telling the browser, user, or other user-agent exactly what to do.

For example the 402 error page could contain a payment form to be filled in, or other instructions to the user.

Or even a button: "Yes I agree to pay $10 to see the article".

The entire point of returning a status code, at least in theory, would be to not simply be practically-semantically indistinguishable from a 200.

In practice, if you try to return a 402 with an HTML page explaining how to pay, you'll quickly revert that because you'll break some percentage of HTTP clients (not browsers, but other ones) that will try to do "something" with 402, even if it's just assuming "well, it's a 4xx so it's basically a 404", and you'll return a 200 for that page. On the one hand, it isn't necessarily that many clients that will blow up, but on the other hand, you'll observe that no clients will be confused by the 200, so the engineering pressures are pretty consistent here, which is why everybody ends up in the same place.

In theory, if you read the standard, you ought to be able to return a 402, and use content-negotiation to decide whether to return it in a human-readable format, or some rich selection of machine-readable formats, and you ought to be able to annotate it with several dimensions of somewhat overlapping information about exactly how to cache it and exactly what conditions should cause the cache to be expired, including some specification of the HTTP-standard authorization techniques available, etc. etc.... in practice, pretty much every individual element in that collection only sorta works, and trying to do all of them at once essentially impossible at scale, and you'll return a 200 page with HTML and expect the client to pick up the pieces, and if you do offer a JSON view, while you may honor the negotiation stuff you'll probably also allow a querystring or form parameter to ask for JSON too because you can't count on the people writing clients to even know what headers are.... and libraries often make them somewhat less convenient to work with than form parameters, since everybody needs those. And you probably won't "content negotiate" at all, but just have entirely separate URLs for your human stuff and your official "REST API", because most people who try to get clever and make your human website URLs also be the "REST API" pretty quickly discover it's not worth the effort.

I'm sorry, but this is a misunderstanding of the web at a very basic level.

    In theory, if you read the standard, you ought 
    to be able to return a 402 [snip]

Every "problem" you just described applies to 404 as well, and everything handles that just fine.

There is absolutely no problem with servers returning 4xx/5xx codes along with some HTML to render.

All the other things you just described, the content negotiation and so on, are highly debatable at best but the important thing to understand is that they're completely orthogonal to whether or not the server has returned some HTML along with the error code.

You are correct in that human readers don't really know/care whether it's a 4xx/5xx code. But that's fine. They're humans, so it's OK for the HTML returned to give them a choice of what to do.

Non-200 response codes are, of course, already massively useful for other bits of code, for RESTful services and otherwise.

> Every "problem" you just described applies to 404 as well, and everything handles that just fine.

I'm not sure what value of everything you're using, but I remember Internet Explorer's 'friendly error messages'. If your html isn't long enough, and that abomination still exists, you'll get Microsoft's 404 instead of your nice and short 404.

I have worked with many embedded http clients that can't reliable return content unless it's from an HTTP 200. I wouldn't write one like that, but you don't always get to pick your platform, and some platforms give you HTTP or nothing, so you can't use TCP and do a better job (or more often, you have much more important platform issues to work around than to roll your own http when the platform one works enough and it's trivial to just return 200 in case the server had something to return to the client)

> If your html isn't long enough, and that abomination still exists, you'll get Microsoft's 404 instead of your nice and short 404.

The most annoying part is that this knowledge was abused to make even the most basic and otherwise-useless 404 pages show up in IE.

Back when this was a thing, I literally saw this happen. I have one of these pages saved, which is how I know - it was a page which just had "Not Found - The requested document was not found on this server." on it (along with the site name), and then a long HTML comment saying 'Unfortunately, Microsoft has added a clever new "feature" to Internet Explorer.' and explaining the comment's existence.

All for a 404 page that gave far less useful information than IE's own 404 page did. If the page had any brand styling on it I might understand, but this was literally a bare-bones, no-styling page with nothing useful on it.

Also many libraries are built by throwing exceptions for non-200 error codes.

No conflict there.

For SOAP stuff, an exception on 4xx/5xx is generally a sensible design.

For human user-agents (ie, web browsers) there's still no conflict. Deciding whether or not to render some HTML is orthogonal to whether or not some other action might be taken. Generally, you do want to present a choice to the user at this point.

Developers are always doing a if curl status > 400 throw error. I keep running into this pattern in external libruaries. Then they throw the error in place. Please stop this and bubble up the error so the user of the lib can decide.

You do know browsers render 404 error pages, right?

Yes. 200, 301, and 404 are top-tier status codes that even the simplest system can't help but handle in some sensible manner. In the second tier you have the necessary support for partial content and basic content negotiation. After that is when it gets a lot more complicated.

The fact that "browsers render 404 pages" won't save you when you try to get "clever", err, I mean, "standards compliant" and serve a complicated HTTP page out on 402. It probably won't confuse browsers, which will probably just render the page and leave the user oblivious a 402 occurred, which is precisely why I included "(not browsers, but other ones)".

Browsers are forced to march together in some semblance of lock step. When you step out into the heterogeneous world programming language APIs, "convenience" code written on top of those APIs, frameworks written on top of that convenience code, internal libraries written on top of those frameworks... yeah... you can't get very many complicated semantics through that sort of pipeline. And that's not a terribly unrealistic pipeline, unfortunately.

If 402 had actually become part of a means to get paid it would have quickly joined that list of top tier codes right behind 200.

Don't forget these popular code. You were probably 301/302ed at least once today. 201 is popular too. In fact you probably have to do something that will give you a 201 before you get your 402.

301 Moved Permanently 302 Found 400 Bad Request 401 Unauthorized 403 Forbidden 404 Not Found 410 Gone 500 Internal Server Error 501 Not Implemented 503 Service Unavailable 550 Permission denied

What you are saying is true; what I was keeping myself confined to are the codes that you have reasonable assurance that the client will actually do something with. A lot of those codes will not cause the client to do anything terribly specific, like "follow a redirect"; it'll just return the content as if it was a 200, and if it's an API, maybe return the request as failed, but that's about it. In the vast majority of cases, clients don't particularly care if they get 400, 401, 403, 404, or indeed, 4xx.

I wonder if there's even a single client in the world that will get a standards-compliant 412 Precondition Failed and try a second request with a different set of conditions. Or indeed do anything sensibly different at all vs. having received a 404. (I specify "standards-compliant" because I'm sure there's plenty of APIs that use 412 for something non-standard.)

Most modern browsers do.

> The HTTP 418 I'm a teapot client error response code indicates that the server refuses to brew coffee because it is a teapot.

I wonder how FF/Chrome would react to this!

By just rendering it: https://httpstat.us/418

I saw dozens of websites with elaborative 404 pages, like including search form, or even returning site search results with context of requested page, splitting URI to keywords. I see how it is implementable with 402 status as well.

That seems reasonable to me. Browsers ended up dealing with 401 and "Basic Auth", in a semi-standard way, for example. "Payment required" isn't substantially different. More complex for sure, but the context shared seems sufficient.

I am totally going to throw a 406 Not Acceptable in production now. Thank you for the idea.

Not sure how I’ll find an excuse, but I have years to plan.

406 Unacceptable would have made it more useful for today’s tumultuous world.

Error 406: Not Acceptable

User address "123 Main st, Scunthorpe" includes unacceptable language.

IME HTTP 406 is not as uncommon as you imply. Ask a Java/JAX-RS service for accept: foo/bar and 406 Not Acceptable is what you'll probably get.

422 Unprocessable Entity is also relatively commonly used in the enterprise Java world for validation errors, despite technically only being for WebDAV

But there's relatively little value in that being an extra code: How often do you encounter systems that do something with the fact that it is 406 and not a generic 400 and e.g. switch their content type? I think that's what the parent is referring to.

> But there's relatively little value in that being an extra code

For the developer of the client, there is value in that it tells you to start your troubleshooting by looking at the "Accept" header. By contrast, a generic 400 doesn't give you that hint; maybe there will be something in the response headers or body to give you that hint, maybe there won't be.

I agree you probably won't find a lot of systems that will automatically "fix" a 406 or 422, or that have an ordered list of more than one media type they are willing to accept in followup requests. OTOH I do think it's useful to be able to convey and log this type of error in a standard way. 406 can be more or less interesting than 400 retrospectively.

Also, isn't HTTP 503 just about the same as 500 as far as your garden variety client is concerned--even if it's technically plausible some client, somewhere, might want to more aggressively seek out a stale cached result for a 503?

503 could still be useful if combined with the Retry-After header but I wonder if any generic client code/library uses it in that way.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re...

503 is useful for me as a human. If I see a 500 on an internal site I assume they're throwing exceptions; if I see 503 I know the service is gone and I should redeploy.

Yes; the idea behind 406 and the whole content negotiation thing seemed to be that clients and servers would somehow notice they didn't negotiate a common representation and then do "something" sensible in response to that, not that it would be essentially treated no differently from any other 4xx error.

Quite a lot of the web would just ignore the mismatch and serve the resource up anyhow with whatever Content-Type the server wanted, and, honestly, that seems to work OK.

It's not that it may not be an error in some sense, it's that the standard has this rich set of specific errors that seems as if it's coming from some parallel universe where the web evolved much differently. And it's not that there is no overlap with our real universe, it's just that if you read the standard and kind of do a postmodernism-style analysis of what the authors must have thought was important, what you get doesn't much resemble the real-world web of today. Or in the case of HTTP 1.1, even the real world web of the time.

One hypothetical use case I can think of going forward is sites/content that only work with JavaScript enabled, and that you must request with "Accept: application/html" to signal you're willing to execute their code, as opposed to "text/html" to signal you're not specifically going to switch on JavaScript for their trackers, and expect at least basic static content to show up. Of course, "application/html" needs to be registered at IANA first, and then should be used on most of today's aggregator sites.

You'd have to also change all clients to not execute JavaScript in text/html, if you want application/html to have any bite. Another option, perhaps more likely to have an effect, is to have clients with no JavaScript enabled to include a parameter with the accept type, e.g. `text/html;profile=no-js` or some such. Servers that don't recognize the parameter can just ignore it, and servers that do recognize it can serve up a no-js response. Any client that doesn't know this parameter will just functions as normal. No need to invent a new content type.

Also mostly automatically returned by Rails

I think the problem is that "payment required" is an issue that's better dealt with in a layer higher up the protocol stack.

One can simply respond with the appropriate redirection code to a login/signup page (or whatever) when the client attempts to access a page where "payment is require" (just like any other authentication/authorization issue).

That is not what the user would want though.

Getting nothing is adequate when nothing exist. Pretending it was something else is just confusing at best.

Well, you're supposed to read the docs, not just guess from the label. But I agree that they could have done better; maybe "No acceptable response".

>406 "Not Acceptable"

That is fairly amusing considering the sort of nonsensical popularity of that phrase (at least in the US).