Among other qualifications, effective teachers must have a robust knowledge of the subject matter that they teach. Many states require that prospective teachers of physics pass one or more standardized tests designed to measure content knowledge. Our survey of sample questions from these tests and from commercial examination preparation booklets suggests that there are significant deficiencies in these materials. Some questions contain physics errors. More often, the questions probe only the ability of test-takers to memorize facts or plug numbers into a formula, with little emphasis on conceptual understanding or pedagogical content knowledge. These deficiencies undermine the very purpose of such content tests.