Research suggests that self-efficacy is one of the central factors predicting students' engagement, participation and retention in STEM fields. Physics is one of the STEM fields in which women are severely underrepresented. Prior research has found that there is a gender gap in conceptual assessments and sometimes even in the final exam favoring men. Women also report lower self-efficacy than men in physics. The origins of these gender disparities are complex, not well understood, and include systemic societal biases and stereotypes that disadvantage women from a very young age both in and out of classroom. Since self-efficacy can impact performance and vice versa, lower physics self-efficacy than men can disadvantage women in physics classes. We studied female and male students' self-efficacy and its relation to learning outcomes in calculus-based introductory physics courses in which women are severely underrepresented. In particular, we discuss an investigation examining students' self-efficacy scores across gender and investigate the extent to which self-efficacy mediates learning outcomes for male and female students, controlling for students' relevant prior academic preparation such as AP Physics or SAT scores. We found that gender differences in course grade were partially mediated by students' prior knowledge and gender effect became non-significant factor after we include students' pre self-efficacy scores. This study can be helpful in catalyzing design and structuring of the physics classroom environment and curriculum to improve the self-efficacy and learning of all students, particularly those from traditionally disadvantaged groups such as women.