When a prospective employer asks for references, they aren’t looking for more information, they’re seeking validation. They have a feeling they’ve made the right decision, but (usually out of formality) they want someone else to tell them that as well.
Every day, we’re forced to make decisions with very little information. And certainly after 3 conversations, you’re not going to know everything about your candidate. Checking references, therefore, acts as a smokescreen. It gives you more time and more information. However, the new information you receive is random (some people exaggerate, some people lie), and the time it takes you to check those references is wasted (when you could be spending that time with your candidate).
More information isn’t the answer—certainly not from people who skew the data (aka, references).