You do not need to know the most people, but you do need to know the right people. I just took my bosses advise and got rid on over 50 contacts who never respond or participate. This is not in arrogance or being mean, but rather picking the people whom I’d like to better assist and who can and want to assist me.
This is business, and I choose to develop more personal business relationships. I have limited time and want to focus on the people and businesses that want to work with us.
Find and work the with people and businesses that are relevant to you, if it is better to have 10 people willing to work with you than it is to have 500 who just know your name (maybe) and spend your energies maintaining that list, then do so. This is also a good time manage exercise.