The past few years have seen a rapid growth in research on early warning signals (EWSs) in psychological systems. Whereas early studies showed that EWSs are associated with sudden changes in clinical change trajectories, later findings showed that EWSs may not be general and have low predictive power. In this study, we demonstrate that two common practices in psychological EWS studies are not warranted by theories and may lead to false-negative or false-positive results, explaining the mixed findings in the literature. These two practices are (1) using loosely-defined time windows for early warning indicators and (2) using different variables for detecting transitions and calculating early warning indicators. We first review the theoretical background of EWSs and current research practices for EWS studies. Two simulation studies with different types of system changes are used to demonstrate the possible consequences of the two practices. In Study 1, we show that when the time window for early warning indicators is not strictly before the transition, the transition process itself and the system dynamics after the transition may confound the result. In Study 2, we show that when the transition and early warning indicators are measured from different variables in the same system, the predictive relationship may not exist. Based on our findings, we provide suggestions for future EWS studies in terms of theory construction, study design, and data analysis.