Yesterday, I had the opportunity to ask a long-time software developer & manager about what he's seen in the recruiting process for software development talent that works (yes, many years, and yes, at many very recognizable companies, and he asked that he not be named). Specifically, what types of measurable criteria has he seen used to good effect in hiring the right people, or building a great team.
Our conversation went all over the map. We compared the hiring practices of some notable large companies in the area (cough, Microsoft, cough, Amazon, cough, Adobe, cough, cough). We also spoke about the methods he's used at the companies he's worked at, and which skill areas he thinks he can measure accurately in any given candidate.
The end result: He doesn't think there are any statistically valid criteria being used in hiring Software Developers by any firm. Further, he thinks that many companies are deluding themselves with their statistics. "We hire 90% of the top candidates for our team" - but how do they know? Do they conduct A/B tests on the candidates, which would force them to NOT hire developers they think are great?
Does anyone out there have any sound statistical methods they use in hiring their developers? Have you personally gone through a hiring process that you thought measured your skills accurately, and also gave you a good sense of the work you were about to perform?