> So how would we go about switching the entire software industry to use LISP more?
There are two answers to this, depending on how broadly you interpret the "Lisp" part of the question. The fundamentalist, and worse, answer is simply: you don't. The objections to Lisp are not mere surface-level syntactic quibbles, it's deeper than that. But also, the syntax is a barrier to adoption. Some people think in S-expressions quite naturally, I personally find them pleasant if a bit strange. But the majority simply do not. Let's not downplay the actual success of Common Lisp, Scheme, and particularly Clojure, all of them are actively used. But you've got your work cut out for you trying to get broader adoption for any of those, teams which want to be using them, are.
The better, more liberal answer, is: use Julia! The language is consciously placed in the Lisp family in the broader sense, it resembles Dylan more than any other language. Critically, it has first-class macros, which one might argue are less eloquent than macros in Common Lisp, but which are equally expressive.
It also has multiple dispatch, and is a JIT-compiled LLVM language, meaning that type-stable Julia code is as fast as a dynamic language can be, genuinely competitive with C++ for CPU-bound numeric computation.
Perhaps most important, it's simply a pleasure to work with. The language is typecast into a role in scientific computing, and the library ecosystem does reflect that, but the language itself is well suited to general server-side programming, and library support is a chicken-and-egg thing: if more people start choosing Julia over, say, Go, then it will naturally grow more libraries for those applications over time.
Elixir feels like that as well (in the same broader sense).
Their trick of 'use' being require+import (ala perl) becomes brilliant when you realise import can be a macro, and that's how libraries inject things into your current namespace.
Julia looks beautiful but I keep forgetting to play with it. I'd use the startup time as an excuse, but it really is just 'I keep forgetting' if I'm honest.
* Scheme is just too fragmented and lacking in arguments to seduce the current industry; its macros are unintuitive, continuations are hard to reason with (https://okmij.org/ftp/continuations/against-callcc.html), small stdlib, no concept of static typing, etc...
* CL is old, unfashionable and full of scary warts (function names, eq/eql/equal/equalp, etc...), and its typing story is also pretty janky even if better than the others (no recursive deftype meaning you can't statically type lists/trees, no parametric deftype, the only one you'll get is array). Few people value having such a solid ANSI standard when it doesn't include modern stuff like iterators/extensible sequences, regexps or threads/atomics.
* Clojure is the most likely to succeed, but the word Java scares a lot of people (for both good and bad reasons) in my experience. As does the "FP means immutable data structs" ML cult. And its current state for static typing is pretty dire, from what I understand.
All of these also don't work that well with VSCode and other popular IDEs (unless stuff like Alive and Calva has gotten way better and less dead than I remember, but even then, SLIME isn't the same as LSP). So, basically, that and static typing having a huge mindshare in the programming world.
I agree with your comment for the most part (particularly the IDE situation)
I do want to callout that while Racket started off from Scheme (and still maintains compatibility with a bunch of it), it should be considered a different platform at this point and solves a bunch of the problems you called out with Scheme - macros are apparently much better, continuations are usually delimited, has user-mode threads and generators, so you very rarely need to reach for raw continuations, a very nice concurrency system, large stdlib, and Typed Racket also adds static typing.
The DrRacket/SLIME experience is great for smaller projects. I do agree that the language server needs more love.
However I still think it gets a lot of stuff right, and is much faster than Python/Ruby due to being powered by Chez Scheme.
The link is an argument against call/cc, not continuations themselves.
That entire website is an exploration into the shift/reset paradigm of delimited continuations, and explores in detail why continuations are incredibly useful and powerful idioms to build all sorts of control structures.
Thank you so much for the info! I am mostly interested in CL.
>Few people value having such a solid ANSI standard when it doesn't include modern stuff like iterators/extensible sequences, regexps or threads/atomics.
For everyone: are there plans to include such topics in the Standard, or are there canonical extensions that people are using?
There is little chance the Standard is gonna get updated again, especially since CDR (the Common Lisp Directory Repository) kinda stopped. That said there are a few libraries that are kinda considered to be the default go-tos for various needs:
CL-PPCRE for regular expressions.
Bordeaux-threads for threads.
Alexandria for various miscellaneous stuff.
Trivia for pattern matching.
CFFI for, well, FFI.
And Series for functional iterative sequence-like data structures.
There's also ASDF and UIOP, but I'm not sure whether or not they're part of the Standard.
Too bad cl-ppcre is extremely slow in my experience, but that's what we have.
ASDF and UIOP are the same as the rest, de facto standards, but not ANSI nor CLtL.
Truly, someone could tie himself to SBCL and its extensions and get a much more modern environment, but I think it's worth targeting ECL and CCL in addition.
ASDF is one of the symptoms of what is wrong about CL. You have this thing that is not really sure whether it is image based or source based and has a bunch of global state while reading the source. Solution to that is ASDF, which is a ridiculously hairy and complex thing.
Lisp is too powerful and flexible. That's an advantage and a disadvantage.
On the one hand, you can use that power to write elegant programs with great abstractions.
One the other hand it's really really easy to create unmaintainable messes full of ad-hoc macros that no-one, not even you next month, will be able to understand.
Even if the program is well-written, it can be a really steep learning curve for junior hires unless it's very well documented and there's support from the original writers available.
Compare it to other languages that limit your ability to express yourself while writing code.
The truth is that most programs are not really interesting and having a code base that can be hacked-on easily by any random programmer that you can hire is a really valuable proposition from the business point of view, and using less powerful languages is a good way to prevent non-experts from making a mess.
There are two answers to this, depending on how broadly you interpret the "Lisp" part of the question. The fundamentalist, and worse, answer is simply: you don't. The objections to Lisp are not mere surface-level syntactic quibbles, it's deeper than that. But also, the syntax is a barrier to adoption. Some people think in S-expressions quite naturally, I personally find them pleasant if a bit strange. But the majority simply do not. Let's not downplay the actual success of Common Lisp, Scheme, and particularly Clojure, all of them are actively used. But you've got your work cut out for you trying to get broader adoption for any of those, teams which want to be using them, are.
The better, more liberal answer, is: use Julia! The language is consciously placed in the Lisp family in the broader sense, it resembles Dylan more than any other language. Critically, it has first-class macros, which one might argue are less eloquent than macros in Common Lisp, but which are equally expressive.
It also has multiple dispatch, and is a JIT-compiled LLVM language, meaning that type-stable Julia code is as fast as a dynamic language can be, genuinely competitive with C++ for CPU-bound numeric computation.
Perhaps most important, it's simply a pleasure to work with. The language is typecast into a role in scientific computing, and the library ecosystem does reflect that, but the language itself is well suited to general server-side programming, and library support is a chicken-and-egg thing: if more people start choosing Julia over, say, Go, then it will naturally grow more libraries for those applications over time.