Now see, that's why everybody should use Tor now and then! :) That way this unique characteristic will become common enough for a generic filter on Tor users to become ineffective (by their measures). If you don't want to be seen connecting to Tor directory authorities and to relays directly, use a Tor bridge. Better yet, run one yourself! If you are using obfuscated bridges, you can (as of now) fool even pretty sophisticated DPI boxes which won't be able to fingerprint your traffic. (cf. continued Tor developers' battles with China's intense DPI infrastructure)
Can I suggest next time you have a perfectly normal and mundane need to access any government website - you fire up TOR Browser first? Go look up your local representative's name, or the garbage collection days for your suburb, but do it over TOR, and leave the trails in their logfiles.