You might need to use another user if you want to set its shell to `gitolite-shell username` (no command= for password authentication) but then you'd need to chain sudo or something to have Gitolite run under its own user again... Seems very tricky.
Or maybe you can write a shell that runs a gitolite-shell command is its arguments are not already gitolite-shell?
But gitolite is so easy to setup & maintain, it's not a big difference and for r/w-access management within teams, it's priceless.
I guess one could even hack anonymous access with "PermitEmptyPasswords yes" and "AuthenticationMethods none"