Why does Socrates argue that it's not good to follow the opinion of the many? What should we follow?

Socrates's principal engagement with the question of the opinion of the many occurs in a Platonic dialogue known as the Crito. This dialogue takes place on the day before Socrates, who is imprisoned, is to be executed for corruption of the youth. Crito, a wealthy friend of Socrates, comes to him and implores him to escape with his assistance. Crito values Socrates's friendship and does not want him to die, but his motives for offering to assist Socrates's escape are also partially selfish. He is a fixture of the community, and is worried that if he cannot persuade Socrates to escape, "the many will not be persuaded that I wanted you to escape, and that you refused." In other words, Crito believes that if Socrates dies, even by his own decision, it will make Crito look bad. He is concerned about and swayed by the opinion of his peers, and believes that "the opinion of the many must be regarded...because they can do the very greatest evil to anyone who has lost their good opinion."   

Socrates, boldly, takes a different view of the situation, even at the potential expense of his own life. He advises Crito that the only opinions worthy of regard are those that come from "good men," because unlike the masses, who can often be swayed, like Crito himself, by peer pressure or self-serving motives, a good man will only "think of these things truly as they happened." They will be honest about circumstance and situations, and not merely stump for the outcome that's best for them. He dismisses Crito's suggestion that the many can do evil to those that oppose them, because in his opinion, "they can do neither good nor evil...whatever they do is the result of chance." He also suggests that the opinion of one individual can matter far more than that of the many, provided that one individual is more informed about the circumstances and situation than the many may be. He uses the example of a gymnast who puts far more stock in his trainer's opinion of his abilities than of the public, because his trainer is wise in the ways of gymnastics and physical competition, and is thus better informed to pass judgment on the situation than the public, who know only what they see the gymnast doing, not how and why he does it.  

Socrates argues that instead of obeying merely the dictates of the masses, we must instead first operate from the principle of never answering injustice or evil with further injustice for evil, for like begets like.  It's a similar notion to the famous Gandhi quote which reminds us that "an eye for an eye makes the whole world blind."  And surprisingly, and with uncompromising adherence to notions of law and justice, Socrates argues that the only way he can see justice done, in this circumstance, is to abide by the will of the court and see his sentence carried out. Speaking for the courts and lawgivers of Athens, whom he argues are the "good men" who know more about the successful operation of a state than he or the many would, he says that he was nurtured, brought up by, and supported by this same state throughout all of his life and educational career. His agreement with and living within those laws does not them give him the right or license to cherry-pick which of those laws he will abide by and which he will discard. By teaching in a manner which the lawgivers regard as corrupting to the youth, Socrates violated the dictates of a system which he had previously agreed to live within lawfully, and even though in a larger, more universal moral sense he has done no wrong, he has committed wrongs against the laws of the state. (In an unspoken way, Socrates does acknowledge the overarching moral rightness of his crimes by not simply disavowing them, but instead admitting freely to what he has done; he has acted against the law of the land, but he is not ashamed of his "crimes.") He imagines the "good men" of the courts advising Socrates to "think of justice first, that you may be justified before the world of the princes below." In other words, if he allows himself to be executed at the order of the "good men" and against the opinion of the many, he dies "a victim," but if he defies the system and answers the perceived injustice of his incarceration with the concrete injustice of his escape, then he is acting in violation of a system that has been set up for the good of all, by those most qualified to determine its goodness. Socrates is a hard judge of circumstances, but he is eminently fair and always in service to reason, so he values the informed opinions of the good over the opinion of the man...even at the potential expense of his own life.   

All quotes from the text are taken from the Benjamin Jowett translation of the Crito, available on the website of the classics department at Massachusetts Institute of Technology.