Over at perl.org we generally don't give anyone shell access for any reason, but instead make people upload files and update sites with Subversion or WebDAV. It might be a bit more hassle for the contributors, but as the not so recent "incident" at the FSF shows, it's a mighty good idea. They got hacked by a local user in march (!) and just found out a few weeks ago.
We might still expose a vulnerability somehow some day (knock on wood), but with fewer people having local access, it's much less likely.
It's also a reminder why signing releases would be a good idea.