Ideally, instructors and their institutions would have a shared set of metrics by which they determine teaching effectiveness. And, ideally, these metrics would overlap with research findings on measuring teaching effectiveness. Unfortunately, the current situation at most institutions is far from this ideal. As part of a larger interview study, 72 physics instructors were asked to describe how they and their institutions assess teaching effectiveness. Results suggest that institutions typically base most or all of their assessment of teaching effectiveness on student evaluations of teaching. Instructors, on the other hand, base most or all of their assessment of teaching effectiveness on student exam performance and nonsystematic formative assessments. Few institutions and instructors use assessment practices suggested by the research literature. In general, instructors are much more positive about the methods they use to evaluate their teaching than the methods their institutions use to evaluate their teaching. Both instructors and institutions could benefit from broadening the assessment sources they use to evaluate teaching effectiveness through increased use of standardized measures based on student learning and greater reliance on systematic formative assessment.