Computer Graphics: Principles and Practice is a fairly reasonable substitute for an introductory graphics course, and having a good understanding of vectors and matrices. Other than that all you really need is the willingness to go and find resources when you don’t feel you fully grasp something.

You might also want to try this sort of thing in a study group. You definitely get more out of exercises if you have people to discuss them with.