I've used Octane, which is a GPU-only renderer. Unfortunately, it is VRAM-per-GPU bound. In other words, you cannot render any scene larger than the usable VRAM per GPU. High end video cards with lots of VRAM also tend to have more than one GPU, so the VRAM-per-GPU figure is actually (total VRAM)/(total GPUs), with slightly less than that actually available for the scene.
I've also experimented a bit with Lux which has a hybrid CPU/GPU mode. However I've found it isn't necessarily any faster than CPU only on my system (which has a lot of CPU cores) and it isn't as stable.
AFAIK, there are no video cards currently available with more than 6GB per GPU, since something like a nVidia Titan Z with 12GB has to share that between 2 GPUs.
It's conceivable that as GPU rendering becomes more commonplace we'll start to see manufacturers loading more and more RAM only high end cards, possibly at the expense of compute units if power consumption is a problem. After all, a render farm with many separate cards is just as fast as one card with more compute units, but VRAM per GPU is currently a hard limit that will affect anyone rendering very large, complex scenes.
I've also experimented a bit with Lux which has a hybrid CPU/GPU mode. However I've found it isn't necessarily any faster than CPU only on my system (which has a lot of CPU cores) and it isn't as stable.
AFAIK, there are no video cards currently available with more than 6GB per GPU, since something like a nVidia Titan Z with 12GB has to share that between 2 GPUs.
It's conceivable that as GPU rendering becomes more commonplace we'll start to see manufacturers loading more and more RAM only high end cards, possibly at the expense of compute units if power consumption is a problem. After all, a render farm with many separate cards is just as fast as one card with more compute units, but VRAM per GPU is currently a hard limit that will affect anyone rendering very large, complex scenes.