Many reports, research, and initiatives have presented evidence-based strategies to create strong departments. The Effective Practices for Physics Programs (EP3) Initiative is a collaborative effort between the American Physical Society (APS) and the American Association of Physics Teachers (AAPT) to compile and curate such resources in an online "Guide" for departments to empower high-quality physics education. To ensure the Guide helps improve physics education, this work aims to understand whether the Guide (a written document) can effectively support departmental change efforts (which can be complex). We draw from findings from a 2020 survey (N=310), 2022 survey (N=239), and 2022 interviews (N=8) of physics department chairs at colleges and universities in the US. In the survey, 22% of respondents had used the Guide and 18% had plans to use it, e.g., for strategic planning.  Our interviewees spoke about their limited ability to engage in anything that is not immediately urgent and/or requires a significant investment of time. However, many also talked about getting good ideas from the EP3 Guide and using the EP3 Guide in a strategic way, such as during faculty meetings to spark discussion. We find that among EP3 Guide users, the Guide is a potentially effective tool to support first-order change, i.e., change that works within existing systems and worldviews, since interviewees reported using content from the EP3 Guide to understand, frame, and promote their departmental change efforts. These successes can also be leveraged to potentially engage the EP3 audience in second-order change, i.e., change that requires reframing goals and/or values, developing new structures, or other transformational processes. However, additional active supports such as webinars, short courses, or leadership institutes may be necessary for effective and sustained second-order change.