So many people seem enamored of the idea of building software to protect users from NSA, and respond by designing new cryptosystems. These efforts invariably fail, because crypto is very hard, and professional crypto requires 10x verification (particularly at the design stage) as it does implementation.
It baffles me that nobody has instead picked up the challenge of taking the PGP/GPG cryptosystem and making it more usable. That seems like an incredibly valuable project that also makes great use of the skills of the sorts of people that tend to want to work on privacy software.
Paradoxically, the more you know about crypto, the less inclined you usually are to build new crypto.
I've been working on using GPG behind-the-scenes, so I can speak to a few reasons why it might not be used more frequently.
1) GPG is -terrible- to use as a library.
There are a lot of wrappers for various languages.
Almost most all of them call the gpg executable, and scrape it's input. This is crazy-town loony-toons.
In theory, gpgme is supposed to be the solution.. But gpgme ends up calling the binary itself in many cases!
2) GPG assumes you want to use the whole ecosystem.
GPG is sort of a mega-framework. It assumes you want to do everything the GPG way - Use gpg subkeys, keyservers, WOT, Import keys to a central keychain, etc.
I've been able to go through with options flags and disable things I don't want, but it's all opt-out, not opt-in.
3) Because of 1 and 2, you can't run GPG entirely in memory.
It writes a lot of files to the filesystem.. I've been bypassing this by running in ramdisks with ephemeral directories, but it's a PITA.
4) 2.1 has been in the works.. FOREVER.
The first betas of gpg 2.1 were in 2010.. I understand they want to get this right, but that's a really really really log dev-cycle. I donated to the crowdfunding, and I hope it helps get gpg out sooner.
GPG 2.1 is the first stable gpg release that includes ECC support (specifically, DJB's 25519). This is a really really big feature, that a lot of people have been waiting for.
In any event, I like GPG, and I'm using GPG because it's been hardened, and I'm not smart enough to do my own crypto.
Unfortunately, there are a couple of misunderstandings here.
It is true that libraries call the gpg binary. This is a feature, not a bug. This architecture makes bugs in the applications less likely to affect the gpg code / data. For instance, it's a lot harder to dump the variable containing the secret key if it is in another process.
The wrappers don't scrape the output. There is a well-defined protocol for interacting with the binary.
Cool. Maybe our definitions of scrape are what's causing the disagreement.
Sorry about that!
When I look at the code that calls the GPG binary, in the python lb at https://code.google.com/p/python-gnupg/ it seems to be reading/writing stdout/stdin.
It doesn't appear to be using a socket connection, or any other non-typical way of accessing it.
return Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)
def _read_response(self, stream, result):
Internal method: reads all the stderr output from GPG, taking notice
# only of lines that begin with the magic [GNUPG:] prefix.
etc
In trying to gpg in my apps, having a library which lets me generate a pgp key in memory, read it in to a string, write it out to a string, and sign/encrypt strings or binaries would be optimal.
I don't want to have to worry about where the .gnupg home directory is being created.. I don't want any files stored. I don't want to access any keyservers, I'm doing all the key verification through other channels.
Essentially, I'd like to be able to use GPG more like NaCL/Libsodium.
Being force to run the GPG program as a subsidiary process is not a feature; it is a bug in the design. It's bad enough that replacing the GPG binary itself with a new implementation would also be a worthy project; just don't muck with the cryptosystem itself, so we can constrain the audit target.
If anything, it's a bug in the language design: If the client is written in C or C++, a wild pointer could potentially point to any data in the process's memory space, including sensitive key information, and from there it's a small step to revealing that information in a core dump or similar. (Or, you know, printing it to stdout because something hinky was passed in to printf.)
The solution is to make everyone only write code in languages which use a runtime that controls memory access better than the OS can. Python or Java, for example. Failing that, relying on the OS is the safest way, and the simplest, most secure way to do that is to keep GPG in its own process.
so as a digression, which language do you think has a runtime that controls memory access better than OS? I wonder can a language like Rust and Golang meet that criteria?
> so as a digression, which language do you think has a runtime that controls memory access better than OS?
I mentioned two: Python and Java.
In general, I mean any language where the runtime only hands programs valid pointers and the language doesn't allow pointer arithmetic. This means it's possible to guarantee that pointers will only point into objects that can be validly accessed by the code which holds those pointers.
This is generally done to make garbage collection easier, but it can be done to implement security policy: Basically, if your security policy trusts a given runtime, you can use that runtime to enforce access policies on objects of all kinds, including memory.
I misread you sentences. I thought you meant Python and Java fails memory management than OS. That caused me to ask the question. Thank you for getting back to me.
> There are a lot of wrappers for various languages. Almost most all of them call the gpg executable, and scrape it's input. This is crazy-town loony-toons.
> In theory, gpgme is supposed to be the solution.. But gpgme ends up calling the binary itself in many cases!
I've heard of this many times, but is there a good reason behind this? Are there extra security considerations behind making this a shared library? Is it just "not the focus of development" much in the same way that changing the GIMP UI was always "off the table" for the longest time because the devs didn't want to do it?
One of the things that's insane to me is how poorly funded so many of these crucial internet projects are. To my understanding, the majority of gpg is implemented by Werner Koch.. He's amazing, but is only human.
Tech Giants spend more money choosing the color of blue than they ever donate, and they seldom hire people to contribute code back upstream. :/
Why does this guy not get more attention? I've used GPG for what, ten years, I have an issue I use the mailing list and Werner is there answering the question. I look at the commit log on a release and he's all over the place. GPG is a massively important piece of software. Would it exist without him? He is a borderline hero in my book.
Completely agree on making existing key-based crypto more usable. A few weeks/months ago when someone started a thread on OpenPGP best practices, I asked why lots of key based tooling wasn't already baked into operating systems like linux. More details in the original thread [0]. It seems like this is really low hanging fruit; much lower than creating a new crypto system.
I am trying to dumb it down. But that means the system may lose its flexibility as the existing gnupg command line tools. Also what I want to do is kind of building some systems on top of pgp's cryptography framework. So end users will not touch gnupg directly.
For example, it can be a file sharing system that uses pgp cryptography framework and a centralized private hkps key server. A user only knows to lock/unlock a data file with a key. They may generate keys all the time or used the same keys. Or share the keys to other people in the system.
I don't will it work or not. It is just some reflections and attempts after using gnupg in current day job and I am trying to express the idea in a prototype.
Except that the tool itself is so painful to use, and has such a shallow integration with things like email clients, that using it requires a considerable degree of self-discipline.
I think I installed it once, and then when my machine required rebuilding, I just couldn't bring myself to install it again.
I wish it weren't so, but a new website is not something that is going to speed the adoption of GnuPG.
If you use Mail.app on OS X, GPGmail comes as close as anything to providing transparently usable email encryption. Admittedly, "as close as anything" does not mean "within a mile of it", but if you're deliberate about keeping your mail secure, GPGmail is usable.
I started using GPG seriously once the "easypg" integration landed in Emacs 23. When you open a .gpg file it prompts for your passphrase and decrypts transparently, and when saving a new .gpg file it prompts you with a list from which you can select recipients to encrypt to.
The integration with Emacs mail clients is fantastic too; just M-x mml-secure-sign or M-x mml-secure-encrypt when composing, and incoming messages with signatures are typically checked for you.
This totally changed the way I used GPG. (from "not at all" to "every day")
I'm working through the irony of your finding GPG easier to use once it landed in emacs. I mean... Emacs for God's sake.
In all honesty, however, part of the "difficulty" of GPG is really by design. You need to be deliberate about signing and encrypting because security should be a careful and deliberate act. If you're hoping for the program to take care of it transparently, you're probably not being safe enough. At least Emacs requires you to go through an explicit process. As you have to do with everything in Emacs.
True, but we're putting a lot of trust in third parties in that situation, no? HTTPS requires us to say "sure Verisign [or insert authority here], I believe you." If I'm doing that, why not just trust Google? Why use email encryption at all?
SSH is more in our control, but it's still trusting a third party (a server) and it still requires a deliberate act to set up and use (unlike HTTPS).
Agreed. Its simply not easy enough to use so I could recommend it to my friends with lesser it knowledge. However I have hopes for projects like MailPile (https://www.mailpile.is/)
According to the descriptions, the new web site is not only about a new theme, but also about creating and collecting content that explains how to use GnuPG and encryption, in general.
So do you actually use it? What is the rate of adoption among your friends and colleagues? How pleasant do you find introducing and explaining the whole process to a nontechnical person?
Same experience here. It's just barely good enough to claim the space of 'technically correct OS crypto implementation', thereby discouraging alternative efforts.
Client side encryption\decryption(can be done in javascript) with Gmail storing and passing nothing but encrypted data they themselves have no means of reading.
Of course, that would mean Google has no means of directing ads based on content so they would have no incentive to run Gmail in the first place...
It baffles me that nobody has instead picked up the challenge of taking the PGP/GPG cryptosystem and making it more usable. That seems like an incredibly valuable project that also makes great use of the skills of the sorts of people that tend to want to work on privacy software.
Paradoxically, the more you know about crypto, the less inclined you usually are to build new crypto.