Studies have shown that embedding scientific argumentation in problem solving can enhance problem solving skills. However, research has also indicated that students have difficulties constructing arguments without appropriate scaffolds. We investigated the use of argumentation scaffolds on students' argumentation quality, conceptual quality, and solution strategies on conceptual problems in an introductory physics class. In this mixed method study we compared students' performance in two guided conditions – constructing an argument and evaluating two arguments – as well as one control condition. Our results indicate that the use of guiding prompts improves the argumentation and conceptual quality of students' solutions. Further, students in the guided conditions tended to use a wider variety of problem solving strategies than in the control condition. We discuss the implications of these results on the use of argumentation prompts on conceptual problems in introductory physics.