At the risk of sounding inflammatory... I would be willing to believe that purely functional languages are the "one true way" if there were even the slightest bit of empirical evidence to support it. But where are all the successful large scale projects built with Haskell (or Clojure or whatever)?
The biggest FP project I was aware of was Twitter using Scala and they ended up completely backing away from it. Yikes.
The only other popular site that springs to mind is Hacker News, and frankly, it sucks. It's practically the Hello World of discussion forums.
The truth is, at the end of the day almost everything is being built with Java, Python, Ruby, or JavaScript. Can you really blame the product owners for being wary of writing their whole product in a functional language?
> But where are all the successful large scale projects built with Haskell (or Clojure or whatever)?
How about Erlang. Let's try it, just from the top of my head:
* AXD 301 Switch -- Classic. Powers and control phone switch networks of whole countries. Claims 99.9999999% reliability.
* Basho Riak -- popular NoSQL database, highly scalable nodes are peers
* RabbitMQ -- popular message queue used by enterprises
* WhatsApp -- messaging application, processes billions of messages a day. Built and maintained by only a few engineers. Sold for billions of dollars.
* Klarna -- European payment processing
* Bet365 -- Sports bets
* Stock trading -- Allegedly Goldman Sachs uses it.
* Erlyvideo.org -- Video streaming service
* eNodeB -- Ericsson cell to internet gateway. There is a 50% if you want to look at cat pictures from your smartphone and Erlang breaks, you won't see your cat pictures.
* Wings3D -- A subdivision 3D modeler. I guess you can build your 3D cat and then send it over WhatsApp via a eNodeB to your friend.
Those are just I know of. So there you go. Maybe you should learn Erlang ;-)
Well OP asked about Clojure so Erlang fits the profile.
But you make an interesting point in a way. One can ask, ok here are two non-strictly functional languages that seem to be fairly practical. Used for large distributed mission critical systems.
Would they be better off written in Haskell?
Would they have been written at all if creators only knew about Haskell?
How large is the relationship between fault tolerant and safe systems as a product vs strong compiler type safety in the language it was written.
>But you make an interesting point in a way. One can ask, ok here are two non-strictly functional languages that seem to be fairly practical. Used for large distributed mission critical systems.
I think there're at least these problems with Haskell:
* The language is badly designed. See the number of extensions which aren't standardized but widely used. See also obvious problems such as requirement to use renames when importing code, and problem with record names conflicts which isn't solved for years.
* The focus of the language isn't correct. Lazy evaluations creates a lot of problems both in program performance and complexity of the compiler. The most important parts are pureness and side effect control.
* The language is already obsolete. Many its constructions and related complexities are redundant if we unify type and expression level, and get as a result dependently typed system.
I hope some language in the future will implement these ideas, and we will have a widely used pure functional dependently typed language with explicit control of effects.
I see two candidates for such a language:
* Pure script - it's Haskell done right, compiledd to JS. Unfortunately, it doesn't support dependent types.
* Idris - it's the language which I describe, but I fear that it's impossible to create a widely used commercial language without major industry players' support.
> the number of extensions which aren't standardized but widely used.
Actually, I think this is something that Haskell did right. It allows experimentation with new language ideas in a way that allows them to be tested safely with existing codebases. Edward Kmett recently had this to say about it:
> I appreciate Haskell's willingness to take a very very long view, and explore the design options, before committing to a path. In many ways, it is this feature of our community and our language that keeps other programming languages looking to us for features and designs to steal.
> The language is already obsolete.
By that argument so are C, C++, Java, etc. But it's about the existence of a mature ecosystem, not about what is theoretically known.
> Pure script
PureScript is great, but its compile to JS basis makes it inappropriate as a general purpose language.
> Idris
Idris looks promising, but it doesn't have a large enough ecosystem yet. I would certainly be willing to switch from Haskell to Idris in the future, but Haskell already is at least a little bit dependently typed [1] and as hinted at above I wouldn't be surprised if Haskell is able to continue improving its dependent typing capability as research advances.
> PureScript is great, but its compile to JS basis makes it inappropriate as a general purpose language.
Except that Javascript is rapidly making inroads on everything except systems programming. If Java is a general purpose language (can't be used for systems without translation due to the JVM), then so is anything-which-compiles-to-JS.
I think the really important thing is that, while I've been enthusiastic and interested in Haskell for a long time now (~8 years?), the vast majority of people will never understand even a comparatively simple concept like monads, let alone anything higher level than that.
Until you can 'get work done' without the equivalent of graduate-level math classes, there's simply no way to develop that 'mature ecosystem' that everyone is hoping for. People have been using 'better if you understand the complication' languages for a long time now, and there's a good reason they're still niche, unfortunately.
Erlang is a great language, but it's very definitely not of the same scale of difficulty to understand as Haskell. You can noodle around with Erlang and end up with something that works reasonably well - in my experience with Haskell, if you can't mathematically formalize what you're talking about, you're going to have a bad time, unless you do go the 'embedded scheme interpreter' route, which is a fun project, but why not just use Racket to begin with?
> If Java is a general purpose language (can't be used for systems without translation due to the JVM), then so is anything-which-compiles-to-JS.
Fair point. I guess my thinking here is that PureScript is unlikely to replace Haskell. It is possible, but I think there are a few reasons that probably won't happen. The first is that it's mission statement "PureScript is a small strongly typed programming language that compiles to JavaScript" points it in the wrong direction. Maybe Javascript is the counter to this, but that feels like an anomaly. Second is that it is way behind Haskell in terms of infrastructure, ecosystem, and a lot of core language/compiler features that enable general purpose programming. Could swarms of people come in and change this? Sure. But if they did, it seems to me that PureScript is similar enough to Haskell that people could just switch to Haskell and avoid that work.
> Until you can 'get work done' without the equivalent of graduate-level math classes
This is a big misconception that is demonstrably untrue. I know many people who have developed a solid working grasp of monads without doing graduate-level math.
> Second is that it is way behind Haskell in terms of infrastructure, ecosystem, and a lot of core language/compiler features that enable general purpose programming.
No, it's way ahead because it's based on Javascript, and javascript is having umpty-billion libraries developed for it right now. Of course that's not actually a good thing at the moment but they'll settle down at some peoint.
> This is a big misconception that is demonstrably untrue. I know many people who have developed a solid working grasp of monads without doing graduate-level math.
Ref my reply to Mr. Church, but it's not just monads - you never stop running into category theory if you're using Haskell, it's baked in.
If someone without any knowledge of category theory rewrote the language, that statement might become true (though I'd be skeptical they'd get it right), but as of right now, you need to know category theory to understand the language and it's base libraries.
I know, I've struggled with it for a long time - every time I have a problem, I have to go back and reteach myself category theory to understand it.
Until you can 'get work done' without the equivalent of graduate-level math classes
The problem is the name "monad", not the concept itself. You don't actually need to know anything about category theory to do I/O. You just need enough exposure to the concepts to know why the Monad type class is so repeatedly relevant as to be given a named abstraction.
People have a hard time getting past the name. You don't need to be a category theorist to use monads in Haskell.
> You don't need to be a category theorist to use monads in Haskell.
I find that demonstrably false. Every time I run into problems and ask for help, the answer is "there is this other mathematical concept that solves that problem".
It's not just monads, it's comonads, monad transformers, GADTs, Functors, phantom types, arrows, arrow transformers, the list keeps going. In addition, they're only described in terms of other mathematical terms - the same problem that plagues monads: nomenclature.
And these aren't just 'things you can use', if you want to understand the libraries you're using, you need to be able to keep up with all of those and more.
It's not like lisp, where you can mostly rearrange cons cells to make nearly any data type you like - there's no way to bootstrap your brain into understanding arrows from a standing start, at least in my experience as an autodidact trying to learn.
GADTs, phantom types, and arrows (not to mention monad transformers) aren't "mathematical terms" in that the names aren't borrowed from mathematical terminology.
They're just _terms_. But to you they are new terms. So you call them "mathematical terms" because obviously if you haven't heard of it, then it must be complicated math.
It isn't.
If you stop thinking that every new concept you meet in the world (or at least in Haskell) is a "mathematical concept" and start thinking it is "just a thing that is new and I can learn" then maybe you will stop feeling this way.
For that matter, none of the things you list except for Functors and Monad Transformers (which are just "ways to build monads") are idiomatic for "basic" haskell programming, and the next step up, GADTs is a straightforward feature to learn when you need.
If you read something like RWH or LYAH, you'll notice they don't cover arrows or GADTs that much. That's because they're not considered core concepts. If you set out to learn "all the things" and then discover that there are a lot of things, perhaps the problem is just that you set out to learn "all the things" under the false impression that you need to in some immediate way.
I should amend this. I have never noticed something to be an arrow transformer in the wild, neither because it was called out in documentation or naming, nor independently. I have much less confidence nothing I have encountered has ever been an arrow transformer...
Sometimes you have to find the tutorials and blog posts first. The lens type signatures aren't the easiest to understand, and sometimes its easier to just use things "empirically" without fully understanding them, and then get a sense of the type signatures when you really want to know what the compiler's actually doing. Being able to use lenses is not as hard as understanding why its intimidating rank-2 type signature is the right one.
Finally, sometimes it helps to de-generalize. To replace (Monad m) with IO, for example, and think about what that special case might mean.
For example, the type signature for Control.Arrow.first is:
first :: Arrow a => a b c -> a (b, d) (c, d)
If you're a beginning Haskeller, your reaction is going to be "WTF"? Well "Arrow" is a type class more general than function. (It could be the "effectful function" type or Kleisli arrow, a -> IO b.) If you specialize with a = (->), you get the more concrete type signature:
first :: (b -> c) -> (b, d) -> (c, d)
... and it's much more clear what it does.
As for nomenclature, I don't disagree. I had this discussion with Brian Hurt (and owe him for the insights): we'd be more marketable if we called Monoid "Appendable" and Functor "Mappable". Monad is one where it's hard to come up with a more accessible name for it: it abstracts over computational context (i.e. what do we actually mean when we talk about computational effects) but that would be a mouthful of words and not any more clear.
I know all these things now - my point is that it took me years to get them to a level where I understand what they mean well enough to implement them anew. You can't say that about other languages.
Nobody is going to 'build a community' around a language that is so effectively obtuse, and when called on it, says 'here are tutorials' - tutorials are not the point. RTFM isn't a decent response to 'your language is difficult to learn and use'.
Given that not even basic functional programming has become popular yet, I can't see dependent types being widely adopted for decades. The learning curve for using dependent types is incredibly steep and capturing interesting properties in types inevitably requires you to understand and write tricky maths proofs.
I wouldn't say that was the case until there is wide support for algebraic data types, type inference, pattern matching and immutable data. I see more languages allowing functions as parameters, basic type inference and encouraging immutable data, but they're still missing out on a lot of benefits offered by functional languages like Haskell and OCaml by ignoring these features.
I do see functional programming features making their way in to imperative languages but most people are still coding in an imperative way. Mutable data is still the default rather than immutable data for example.
This is a terrible and also hilarious argument. Instead of saying Haskell has too many newfangled ideas, you're instead arguing that it has an insufficient amount of them.
So Haskell has newfangled innovations compared to the big players, and it _also_ has a mature ecosystem in widespread use.
This puts it ahead of Purescript and Idris for adoption in large scale projects (although I would readily agree that both those languages are pretty great too).
I maintain that lazy evaluation works great, it just requires optimizing along different paths than devs are used to -- but it doesn't incur more "overhead" to optimize code than is the case in typical strict languages, which _also_ have to pay that cost, just in different sorts of optimization and reasoning.
There is what I like to call The IRC Problem, which is that it takes a bit of work to really make IRC into an awesome, persistent chat environment.
First, you need to be persistently connected so that you don't miss messages when you're not actually at your computer.
Second, you need a decent interface to use when you are actively reading or chatting.
Third, it would be great to have the same across all devices (phone, tablet, laptop/desktop), with everything syncing correctly. If you read some scrollback on your phone, you want your laptop to also know that you already read those messages, and vice versa.
It's certainly possible to achieve all this with irssi and a combination of other techniques. However, you can also use something like IRCCloud and get every problem solved at a very high level of competence immediately, plus a bunch of other stuff (like push notifications and embedded social media, images, videos, tweets, pastebins, etc.).
It's up to you. For me, I really just want everything to work and I don't care at all about doing it myself. It's a means to an end. The $5/month fee is not important to me given the product quality. I sort of liken it to Dropbox -- it's not about doing something that couldn't be done before, it's about just making it streamlined for those who don't care to do it themselves.
> First, you need to be persistently connected so that you don't miss messages when you're not actually at your computer.
I consider that a feature of IRC myself. I don't need to be wired in at all times and read eveerything everyone says. If I'm not there, I'm not there, and if there's something important I really need to hear, I'll hear it when I'm there. If I don't hear it, it never was important.
There is memoserv in many IRC networks that does relay messages when you come back online. I have seldom had important messages relayed that way either. Most conversations are not that important.
I don't see IRC as something that should follow me everywhere across all devices. It's like radio or TV. Turn it on, turn it off. Not on all the time.
Lisp, Haskell and other FP are currently killing it in the quiet world of DSLs, especially Mining, Oil & Gas industries which need to parse petaflops of seismic data and reservoir simulations, need rapid prototyping, need formally verified drilling platform components that field engineers can interact with easily, and HPC algorithms for financial trading.
As for Hacker News UX I wish every site was this simple instead of popups everywhere to get me to download their phone app, piles of slow loading fonts, trackers, generic stock images of business suits pointing at a screen, information buried underneath annoying popup FAQ screens, ect.
> The truth is, at the end of the day almost everything is being built with Java, Python, Ruby, or JavaScript.
You need to normalize against the number of programmers writing in Java/Python/... compared to the number of programmers writing Haskell for this assertion to be meaningful. And when you do that, the picture starts to look a lot better.
In the past few years I have seen more and more people start writing real projects in Haskell and be successful. There was a presentation a couple years ago at CUFP [1] discussing one company's experience porting a 43k line Groovy app to 8200 lines of Haskell. You don't need someone to build a highly visible multi-million line piece of software in Haskell to be confident that it can benefit your project. I think there's enough evidence out there today to make a very compelling case.
That may be survivor's bias, though. How many software products end up not making it, at all? If you look at all software projects, everywhere, combined, and split them up: functional vs non-functional. The latter group would several orders of magnitude larger than the former.
Now what percentage of the non-functional ones ends up becoming successful enough for you to hear about them and know what they were built with? Apply that percentage to the functional projects, you might just be left with naught.
I have a feeling that if there's one thing more rare than functional programs, it's highly successful ones.
I would use strongly typed functional programming languages for everything if I could.
However, to be productive, you're usually constrained by what the dominant language is for your domain. For example, Android apps are usually written in Java and web apps are usually written in JavaScript. Each has a huge ecosystem of tools and libraries based around these languages.
I'm sure there's some way I could use Haskell or OCaml to write Android and web apps. However, by going against the grain, it's likely to be very time consuming and frustrating to integrate with existing libraries and tools. I'm therefore going to lose any advantage I was hoping to gain from using a functional language.
> I'm sure there's some way I could use Haskell or OCaml to write Android and web apps.
I don't know about Android, but for Javascript apps the balance tipped for me just last month. I'm probably enough of an insider that it's not quite true yet for the average Haskell developer, but I believe that will change soon.
What framework are you using? I've investigated Dart, Flow and TypeScript to improve static checking but I'm still having pain points: they work fine when dealing with your own self-contained code but integrating with third party libraries isn't always straightforward.
I'm using GHCJS and Reflex. Reflex is a recently released FRP library that makes a huge difference in one's ability to abstract GUI code. I personally feel that this combination of GHCJS + Reflex is the first thing I've used that makes web front end programming tolerable. A week ago there was a presentation about it at the NY Haskell Meetup. Look for videos of it to be posted early next week.
It's new, so not right now. But docs are coming. The NYHUG presentation will probably be the first thing along those lines. My company just rewrote a significant front-end app in it, so we clearly are invested in its future. In the meantime there's a #reflex-frp IRC channel on Freenode that you could use to talk to some of the people currently working with it.
It depends on if your server side code is heavily reliant on third party libraries written in non-functional languages and which languages your team are experienced with.
I'd like to think that the Erlang has proven its value at building reliable, distributed applications. It's a go to language for distributed key value stores (Riak, Couch, et al), and it's in wide use in Motorola and Ericsson to build critical infrastructure. I call that successful.
That said, you do have a valid point: The presentation provides justifications for those that want to switch in the first place, but to be compelling, it needs evidence.
I work in Erlang and love it, but I would make the friendly argument that Erlang is really distinct (and special) even among the FP languages for a number of reasons. I always had the impression that Erlang was functional to support the guarantees made by the runtime rather than for ideological reasons. Some of the Functionistas I've run into seem to reject it out of hand for not being Haskell, but as a C guy I find it freaking amazing for writing stuff that has to stay running all the time. I guess I use in spite of it being functional rather than because of it, but it feels like I'm getting good value even when I run into things that are a pain in the ass to do functionally (which is not super often).
> The biggest FP project I was aware of was Twitter using Scala and they ended up completely backing away from it. Yikes.
You must be joking, Twitter is far and away the largest Scala shop in the world.
Presumably you're mixing up LinkedIn's recent move from Scala to Java. An important caveat is that they're still using the Play framework (Typesafe product), just now in Java instead of Scala.
It's unfortunate that Play serves two masters (i.e. Scala and Java), but industry demands it -- the argument that FP is the one true way goes out the window in face of current reality: enterprise by and large will go with mature tooling and cheap developers in a large-team-friendly language like Java over a paradigm shifting FP language like Scala, Clojure, or Haskell.
Be careful what you mean by 'mostly'. Though there may be many small outfits that use it for this particular features, there are others that use it very differently (and at completely different scales). Examples such as Facebook, Jane Street and Esper come to mind. There are likely many other companies who have less of a web-presence too (cf defence industries, as mentioned in the slide deck).
Well, Twitter didn't backed away from Scala, but it doesn't qualify as purely functional.
Turing completeness ensures there's no "true way", but some languages offer better abstractions than others. In my experience, once you learn the abstractions, the functional paradigm has an edge about the ability to reason about the code.
Of course, the adoption of new ideas and technologies takes time, they need to be explored and fully developed. But even right now, you see many mainstream languages adopting functional flavors.
I believe that eventually, there will be enough successful large scale projects build with pure functional languages to convince skeptics, there's nothing wrong with "playing it safe" and wait.
If Scala is good enough as an example, I see an increasing adoption.
As an anecdote, one of the biggest and Very Serious(tm) investment bank, is using Scala for most of their new initiatives, with a big team spread across the world.
And the language itself was selected by a committee. You can't get more serious than that...
Sure maybe it's not good for FP if you're Edward Kmett.
> The reason, I'm hesitant to give it a fifth star is ultimately that I don't believe that Scala is a good language in which to actually do functional programming.
Literally the only other mainstream language that he would be able to say is capable of "fp" better than Scala would be Haskell.
At least people use it, which is more than what could ever be said about Haskell, so it had a larger impact overall ... while getting things done at the same time.
You only need to have one look at it's module system/Cabal/Hackage to see that it was never intended for larger code bases than people's PhD thesis.
Yes, it can scale: In the same way PHP and JavaScript does.
But where are all the successful large scale projects built with Haskell (or Clojure or whatever)?
Standard Chartered has a huge amount of infrastructure in Haskell. Clojure's too young to have the types of stories you're looking for, but it's had plenty of moderate-sized successes (as has Haskell).
The biggest FP project I was aware of was Twitter using Scala and they ended up completely backing away from it. Yikes.
Scala is different from Haskell. Scala has a lot of features that don't play well together, it has stability issues, and it has inherited some of the Java culture which means that there's a lot of terrible Scala code out there.
I've been working with Spark over the past month and I'm shocked by how bad the JVM/Hadoop legacy is. Spark itself is a neat idea. Getting it to work on an actual production cluster is a huge PITA. I'm tempted to just use Cloud Haskell.
I know people who've moved away from Scala (either to Java or Clojure or Haskell). No one who's moved to Haskell has ever regretted it. I know of about 15 companies that use it (not all on million-line projects) and they're all happy with it. I'm only aware of one case, in the past 7 years, where a company moved away from using Haskell: they moved to F# because they picked Microsoft up as a client and MSFT wanted to be able to say that $COMPANY_X was using their products.
The only other popular site that springs to mind is Hacker News, and frankly, it sucks. It's practically the Hello World of discussion forums.
It's written in Arc, which lost to Clojure. Also, HN is as good as it needs to be. It's successful on its own terms. It doesn't need to be a wonder of software engineering.
It will be once it gets a decent mobile view akin to viewing reddit on /.compact.
Anyway, good point--I've heard people complain that Haskell is hard to learn/understand, but I've never heard of anyone actually using it in production and regretting that decision.
The biggest FP project I was aware of was Twitter using Scala and they ended up completely backing away from it. Yikes.
The only other popular site that springs to mind is Hacker News, and frankly, it sucks. It's practically the Hello World of discussion forums.
The truth is, at the end of the day almost everything is being built with Java, Python, Ruby, or JavaScript. Can you really blame the product owners for being wary of writing their whole product in a functional language?