As physics departments increasingly emphasize computational training within the physics curriculum, there is a need for educators to have guiding principles for deciding how and when to use computational approaches over analytical math and vice versa. We investigated the use of analytical and computational mathematics in professional practice by conducting ten semi-structured interviews with PhD students in the physical sciences. The interviews revealed context-rich situations where computational and analytical math were valued and used. Through an emergent and thematic coding process, key contextual features were distilled. Although analytical math was valued as a calculational tool (e.g., manipulating equations), the most prevalent use of analytical math was to develop a preliminary understanding of a problem, which included modeling systems through equations, developing simplified "toy models", understanding background concepts, and understanding how varying parameters affected system behavior. Computational tools had a complementary role of data analysis, complex numerical simulations, and visualization.