It's not merely optimism, but rather that a total lack of expertise that renders the ignorant unable to recognize their deficiencies.

A growing body of psychology research shows that incompetence deprives people of the ability to recognize their own incompetence. To put it bluntly, dumb people are too dumb to know it. Similarly, unfunny people don't have a good enough sense of humor to tell.
 
This disconnect may be responsible for many of society's problems.
 
With more than a decade's worth of research, David Dunning, a psychologist at Cornell University, has demonstrated that humans find it "intrinsically difficult to get a sense of what we don't know." Whether an individual lacks competence in logical reasoning, emotional intelligence, humor or even chess abilities, the person still tends to rate his or her skills in that area as being above average.
 
Dunning and his colleague, Justin Kruger, formerly of Cornell and now at New York University, "have done a number of studies where we will give people a test of some area of knowledge like logical reasoning, knowledge about STDs and how to avoid them, emotional intelligence, etcetera. Then we determine their scores, and basically just ask them how well they think they've done," Dunning said. "We ask, 'what percentile will your performance fall in?'"
 
The results are uniform across all the knowledge domains: People who actually did well on the test tend to feel more confident about their performance than people who didn't do well, but only slightly. Almost everyone thinks they did better than average. "For people at the bottom who are really doing badly — those in the bottom 10th or 15th percentile — they think their work falls in the 60th or 55th percentile, so, above average," Dunning told Life's Little Mysteries. The same pattern emerges in tests of people's ability to rate the funniness of jokes, the correctness of grammar, or even their own performance in a game of chess. "People at the bottom still think they're outperforming other people."

It's not merely optimism, but rather that their total lack of expertise renders them unable to recognize their deficiency. Even when Dunning and his colleagues offer study participants a $100 reward if they can rate themselves accurately, they cannot. "They're really trying to be honest and impartial," he said.
 
If only we knew ourselves better. Dunning believes people's inability to assess their own knowledge is the cause of many of society's ills, including climate change denialism. "Many people don't have training in science, and so they may very well misunderstand the science. But because they don't have the knowledge to evaluate it, they don't realize how off their evaluations might be," he said.
 
Moreover, even if a person has come to a very logical conclusion about whether climate change is real or not based on their evaluation of the science, "they're really not in a position to evaluate the science."
 
Along the same lines, people who aren't talented in a given area tend not to be able to recognize the talents or good ideas of others, from co-workers to politicians. This may impede the democratic process, which relies on citizens having the capacity to identify and support the best candidate or policy.
 
The ultimate takeaway of the research is the reminder that you really may not be as great as you think you are. And you might not be right about the things you believe you're right about. And if you try to joke about all this, you might not come off as funny as you think.