Idiographic modeling is rapidly gaining popularity, promising to tap into the within-person dynamics underlying psychological phenomena. To gain theoretical understanding of these dynamics, we need to make inferences from time series models about the underlying system. Such inferences are subject to two challenges: first, time series models will arguably always be misspecified, meaning it is unclear how to make inferences to the underlying system; and second, the sampling frequency must be sufficient to capture the dynamics of interest. We discuss both problems with the following approach: we specify a toy model for emotion dynamics as the true system, generate time series data from it, and then try to recover that system with the most popular time series analysis tools. We show that making straightforward inferences from time series models about an underlying system is difficult. We also show that if the sampling frequency is insufficient, the dynamics of interest cannot be recovered. However, we also show that global characteristics of the system can be recovered reliably. We conclude by discussing the consequences of our findings for idiographic modeling and suggest a modeling methodology that goes beyond fitting time series models alone and puts formal theories at the center of theory development.