Young children, students, and adults may have alternative ideas about the motion of the Sun and stars as we observe them in the sky. However, a good understanding of this apparent motion is essential as a starting point to study more advanced astronomical concepts, especially when these include astronomical observations. In this paper, we describe the development and validation of the apparent motion of Sun and stars (AMoSS) test, which can measure to what extent students have insight into the apparent motion of the Sun and stars. We propose a framework that allows one to compare students' understanding of the specific aspects of these apparent motions in relation to the time of the day, time of the year, and the observer's latitude. For each of these aspects, we designed test items for both the Sun and the symmetric apparent motion aspect of the stars. The reliability and validity of the test are established by analyzing answers of both secondary school and university students and by presenting the questions to a panel of experts. We report on the design and validation process and present the final version of the test.