Self-efficacy is an aspect of students' motivation that has been shown to play a critical role in students' engagement, participation, and retention in academic careers in science, technology, engineering, and mathematics (STEM). Since women are underrepresented in STEM domains such as physics, we studied female and male students' self-efficacy and its relation to learning outcomes in physics that can be useful for creating equitable and inclusive learning environments. In a longitudinal study, we surveyed approximately 1400 students in calculus-based physics 2 courses to investigate students' motivational beliefs in physics using a validated survey. We examined female and male students' self-efficacy scores and the extent to which self-efficacy related to learning outcomes (students' grades and conceptual post-test scores), especially the significant gender difference in conceptual post-test scores. To reveal the unique contribution of self-efficacy on outcomes, we controlled for several other variables including Physics 1 grades, SAT math scores, and conceptual pretest scores in physics. We found that the gender differences in conceptual post-test performance were mediated by the model variables. In particular, initial self-efficacy differences showed a direct effect on outcomes even when controlling for students' prior physics knowledge and skill differences, and self-efficacy also had the strongest total gender effect on conceptual learning. Given these findings, future work should focus on better understanding the drivers of these self-efficacy differences including the role that societal stereotypes and biases play in these in order to mitigate these differences.