This paper presents the first results from draw-a-scientist tests (DASTs) over five years that were used to measure the effect of 8–10 week long astronomy clubs and week long summer camps on 3rd–5th grade elementary school students' perceptions of scientists. We facilitated these DASTs prior to these clubs or camps, which provide a baseline for a student's initial conception of scientists, and once at the end, to determine whether their conception changed, possibly as a result of their involvement. In total we analyze 89 pairs of DASTs using a numerical grading scheme designed to quantify the presence of various features in the drawn scientist and their activities. We find that there is a gender imbalance in both the pre- and postclub drawings, with only 32% and 35%, respectively, of students drawing female scientists. We also find that a third to a half of the scientists have a stereotypical appearance and/or are performing stereotypical activities. Although we find insignificant changes (<5%) in most categories, we do find an 8% increase in the number of scientists that have a stereotypical appearance, which is worth following up, but a significant 12% decrease in the number of scientists who are performing stereotypical activities. In addition, we present some possible improvements to implementing DASTs and discuss other possible assessments that could provide a more direct method of gauging the effect of these astronomy clubs or camps.