Understanding the nature of causality is a key component of conceptual understanding in science. The hypothesis of this study is that certain types of causal inference are more challenging than others. If correct, particularly challenging causal inferences may provide a unified explanation for different conceptual difficulties across physics content areas. This paper investigates two aspects of a simple, qualitative force-and-motion problem that may impact the difficulty of the causal inferences required. The first aspect is the type of causal inference required: cause-to-effect (CE), effect-to-cause (EC), or cause-to-cause (CC). The second aspect is information about an alternative cause, which can be explicitly constant, explicitly unknown, or ambiguous. To test the impact of these two problem aspects on participants inference accuracy we conducted an on-line experiment in which participants were randomly assigned to one of thirty-six conditions that systematically varied these two aspects across conditions. The results show that (i) for explicitly constant alternative causes, CC inferences are more difficult than CE or EC inferences, (ii) inferences given explicitly unknown alternative cause information are more difficult than inferences given explicitly constant alternative causes, and (iii) ambiguous alternative cause information is treated as implying the alternative causes are explicitly constant, which is in line with conversational assumptions rather than a formal, logical perspective. These results hint at the potential fruitfulness of understanding the causal inferences underlying conceptual difficulties in physics.