Pre/post testing is an important tool for improving science education. Standard in-class administration has drawbacks such as `lost' class time and converting data into electronic format. These are not issues for unproctored web-based administration, but there are concerns about assessment validity, compliance rates, and instrument security. A preliminary investigation compared astronomy students taking pre/post tests on paper to those taking the same tests over the web. The assessments included the Epistemological Beliefs Assessment for Physical Science and a conceptual assessment developed for this study. Preliminary results on validity show no significant difference on scores or on most individual questions. Compliance rates were similar between web and paper on the pretest and much better for web on the posttest. Remote monitoring of student activity during the assessments recorded no clear indication of any copying, printing or saving of questions, and no widespread use of the web to search for answers.