Originally from SF area, I have been living in Tokyo for over a year now, and since moving I rarely speak with native English speakers anymore. I've been told by friends and family on occasion my accent and grammar is slowly deteriorating.
The Americans I knew when I lived overseas for three years (on each of two stays that long) definitely had trouble with vocabulary retrieval, the more trouble the longer they had been away from native speakers of English. Correspondingly, Chinese people I know who have been away from places with lots of Chinese printed text often have a lot of trouble writing Chinese characters that everyone learns in school. Even "overlearned" skills that become second nature can degrade through lack of continued practice.
Similarly, I've spent a lot of time in the US lately and people can no longer really place my accent. It sounds neither native to here, nor what people from back home have.
Yes, this is common, due to lack of use. Though you should notice that you have much less trouble reading and listening than writing and speaking.
I immigrated right around the "easy"-language-acquisition barrier age and frustratingly can't speak either of my languages fully right - one because of leftover accent, one because of out-of-use vocab and grammar atrophy.
