Other studies in medicine and elsewhere have shown the same thing: confidence and competence are not the same thing at all. Knowing this fact, I never believe a junior doctor who tells me they know how to do a practical procedure, they are always subject to further questioning from me, and I have experienced several what educators call 'Oh my God!' moments when observing confident but incompetent trainees. Experience does not count - without study, deliberate practice, reflection and expert feedback you cannot become an expert. This is evidence-based education, something we need to build in to our work-based training programmes.
Potchen EJ, Sierra AE. Value judgements in diagnostic radiology: how do we decide? Radiology 1981; 138 (2): 501-4.