Web-based homework is becoming widespread practice in physics and other technical courses, primarily for logistical and economic reasons. Although people argue whether this is a positive or negative development for student learning, there is limited research assessing the pedagogical effect of changing the medium from written, hand-graded homework to web submitted, computer-graded work. Towards this end, we assessed student performance with web homework and with traditional paper homework in a realistic classroom setting for large enrollment service courses. One of two nearly identical sections of introductory physics received paper-based, hand-graded homework while the other received the standard web-based homework, and we compared the two groups on conceptual and problem-solving performance measures. We conducted this experiment two times, once in a calculus-based course and once in an algebra based course. No significant differences in student performance were found that could be attributed to the homework method used. This work was supported by the Spencer Foundation and the National Science Foundation under grant 9714546.