The title suggests that there's something unique about Go, either the language or its standard library, that enables bandwidth savings. In fact, Cloudflare have written some software which they claim enables them to reduce their bandwidth, and this software happens to be written in Go. This might be an excellent choice (and I suspect it probably is), but it's not Go per se that is reducing the bandwidth usage.
I agree. The benefit of using Go is that it's fast to write and has good concurrency features. To give you an idea of the size, there are 7,329 lines of Go code in Railgun (including comments) and a 6,602 line test suite.
In the process we've committed various things back to Go itself and at some point I'll write a blog on the whole experience, but one thing that made a big difference was to write a memory recycler so that for commonly created things (in our case []byte buffers) we don't force the garbage collector to keep reaping memory that we then go back and ask for.
The concurrency through communication is trivial to work with once you get the hang of it and being able to write completely sequential code means that it's easy to grok what your own program is doing.
We've hit some deficiencies in the standard library (around HTTP handling) but it's been fairly smooth. And, as the article says, we swapped native crypto for OpenSSL for speed.
The Go tool chain is very nice. Stuff like go fmt, go tool pprof, go vet, go build make working with it smooth.
sigh This is by far go's biggest wart IMO, and one that frequently sends me back to a pauseless (hah! at least less pausy:) systems language. I sure do like it in almost every other meaningful regard. But I wish latency wasn't something the designers punted on.
I occasionally hear this kind of complaint, but I've yet to see any silver-bullet memory management system. AFAICT, the best we've been able to accomplish is to provide a easier path to correctness with decent overall performance. Also, GC latency isn't the only concern. As soon as the magic incantation "high performance" is uttered, all bets are off.
There's been decades of work on real-time garbage collection yet all of those approaches still have tradeoffs. Consider that object recycling is a ubiquitous iOS memory management pattern. This reduces both memory allocation latencies and object recreation overhead. Ever flick-scroll a long list view on an iPhone? Those list elements that fly off the top are virtually immediately recycled back to the bottom -- it's like a carousel with only about as many items as you can see on screen. The view objects are continually reused, just with new backing data. This approach to performance is more holistic than simply pushing responsibility onto the memory allocator.
Memory recycling here also reminds me of frame-based memory allocator techniques written up in the old Graphics Gems books, a technique likewise covered in real-time systems resources. Allocating memory from the operating system can be relatively expensive and inefficient, even using good ol' malloc. A frame-based allocator grabs a baseline number of pages and provides allocation for one or more common memory object sizes (aka "frames"). Pools for a given frame size are kept separate, which prevents memory fragmentation. Allocation performance is much faster than straight malloc, while increasing memory efficiency for small object allocation and eliminating fragmentation. Again, this is a problem-specific approach that considers needs beyond latency.
"AFAICT, the best we've been able to accomplish is to provide a easier path to correctness with decent overall performance."
Precisely. Which is why for performance-critical systems code it's important to give the programmer the choice of memory allocation techniques, but to add features to the language to make memory use safer.
Garbage collection is great, but occasionally it falls down and programmers have to resort to manual memory pooling. Then it becomes error-prone (use after free, leaks) without type system help such as regions and RAII.
I can't speak for the grandparent, but for my part I agree with your point that allocation patterns matter and that there is no silver bullet to memory management, which is exactly the reason that GC'd languages like Go are uninteresting as systems languages. Why use a language where you have to work around one of its main features when you care about performance?
I find Rust's approach much more interesting, because GC is entirely optional, but it provides abstractions that make it easier to write clear and correct manual memory management schemes.
Do they have a standard ABI or FFI for interaction with C? If so, they probably designed the assumption of a conservative GC into it. You can always make an incompatible change, but it's a pain.
Go does look awesome. I've spent some time with Erlang, Clojure and Scala (roughly in the order that I liked them most), but Go passed the "get started writing useful code quickly" test better than any of them. Haven't gone beyond the basics yet, but I think it might occupy a sweet spot of ease of use combined with "power", loosely defined.
I live in Lancashire at the moment, the only issue for me would really be spending £150 on a round trip to London, do you guys do preliminary Skype interviews?
(I'm aware that the post might not be prestigious as say, engineering - however, I feel that having someone with strong web development experience (who is a user of Cloudflare already) would more than offset the slight inconvenience on your part.)
We typically conduct the first interviews on the phone or Skype for interesting candidates. If it makes sense to do in-person interviews, we're happy to cover the cost of transportation for candidates we're excited about. In other words, if you're excited about working with CloudFlare, don't let the £150 stand in the way of applying.
We definitely do the initial interview via phone/Skype. There absolutely is room to grow and move into other areas of the company with experience. I highly recommend becoming familiar with the platform through the "front lines" of support. It gives engineers a different perspective on our service.
Ok, that sounds great! (Same to the comment above, replying to this one for continuity) - let me mull it over this week.
And I couldn't agree more, being placed in the firing lines of customers is often more telling than building the software yourself - "normal" people tend to notice things which we as developers are prone to miss or gloss over unintentionally.
-----
The awkward moment when I notice I blanked the CEO
Large changes in lifestyle, such as a complete relocation, new job, etc, should not be something undertaken lightly - if I interviewed and got offered the job I would be under a lot of pressure, which I can mitigate now by thinking more carefully before undertaking anything, I wouldn't want to waste both my time and the time of the people at Cloudflare by making an important decision without carefully weighing the pros and cons.
How about just emailing them? I feel like I've seen these kinds of 'wow I where do I apply for a job' posts on cloudflare news articles before. Smells like astroturfing.
The title only suggests something unique about Go to those who didn't read the article.
There's a good chunk of that article dedicated to discussing the language choice and how other languages could have been used instead but -in this specific instance- wasn't chosen. The language choice is as much a part of the topic as the compression routines themselves. So it makes a lot of sense to include the term 'Go' in the title given that's a large focus of the article.
It's really no different to all these articles that spring up about fancy demos being built in Javascript or CSS tricks. Yet in those instances nobody says "the title is misleading. You could write that demo in C++ as well."
>The title only suggests something unique about Go to those who didn't read the article.
The thing is, many people use the title to determine whether the article is worth reading. As is, the title suggests that there is something unique about Go that reduces the bandwidth needed by the program, implying that this is something that other common languages fail to achieve. This is obviously impossible (any widely used language is capable of serializing an output byte stream in any way the programmer desires). As a result, the title sets off the alarm for "Language fanboyism", and "mathematically impossible claims", and goes swiftly into the "don't bother" pile together with "universal lossless compression algorithm invented!"[1], "perpetual motion machine" and "My favourite X language is faster that C/C++/Assembler!1!1"
> The thing is, many people use the title to determine whether the article is worth reading.
That same argument could be used for having the language in the title as people who are not interested in programming are going to be less interested in a thread about programming.
And language fanboyism is going to happen with or without this title (given the content of the article). What's happening here is more a case of lazy members wanting to commentate on articles they've not even read. It's basically the lowest form of blogging.
I've never used go professionally and most of my spare time is split between C++ and Scheme at the moment, but when I did go spelunking with Go, I found it a breeze to write complicated functionality in it - it felt like C++, but easier and more initially powerful.
I still feel that C++ is generally a better choice, but if I only had a short time to write something in, I would definitely go for Go.
For me, I just don't see enough benefits of Go over C++. I already know how to use C++ in a way that avoids or mitigates the problems Go solves. With C++11 support starting to take off Go's advantages are even smaller.
On the other hand, if I didn't know C++ and I was looking for a native compiled language to learn, I'd probably choose Go over C++.
Depending on what you're doing, the libraries make a giant difference. Look over Go's standard library packages and then imagine what a pain in the ass it would be to find and manage all the separate C/C++ libs it would take to replicate all that functionality (or to write it yourself).
Performance, I also feel more in control, it's hard to describe the feeling, it's just as though Go is providing an abstracted interface to the hardware, where as C++ provides raw, unfiltered but potentially dangerous access.
That kind of fits Rob Pike's explanation for why Go isn't more popular with C++ programmers (although he gave it a negative spin).
In a neutral way, it's like: if you spent all that effort mastering this language to get such fine-grained control, why would you give that up again? And really, I understand: why would you give that up? Especially if you know how to use C++ in a fairly painless way.
> Performance, I also feel more in control, it's hard to describe the feeling, it's just as though Go is providing an abstracted interface to the hardware, where as C++ provides raw, unfiltered but potentially dangerous access.
Funny I though C++ did exactly the same thing. Where are the L1, L2 and L3 caches references, multiple opcode execution pipelines, processor instructions ?
