Of course, people have done this before [1]. There is quite of bit of research looking into Multi-task learning. Just look through some of the references in that Luong et all paper. Deepmind has been putting out some amazing research lately, but this paper definitely does not fall in that category.

1. http://nlp.stanford.edu/pubs/luong2016iclr_multi.pdf