Depression has been found to be associated with cognitive decline. This study evaluated the association of general depressive symptoms and motivational-related symptoms with cognitive impairment 6 years later and to explore the role of potential underlying mechanisms. In 2690 cognitively healthy persons aged ≥60 from the Swedish National study on Aging and Care in Kungsholmen (SNAC-K) depressive symptoms were derived from the Montgomery Åsberg Depression Rating Scale (MADRS). Cognitive performance was assessed at baseline and 6 years later in 1810 persons with the Mini Mental State Examination (global cognition), Digit Span Forward (short-term memory), Digit Span Backward (working memory), Clock-test (visuospatial construction), and the 5-item test (immediate and delayed recall). Bi-factor analysis on the MADRS yielded a General Depression factor and an unrelated Motivational factor. After adjusting for demographics, the General Depression factor was only associated with 6-year impairment in delayed recall (OR (95% CI): 1.18 (1.04–1.34)). This association was no longer significant after adjusting for demographics, cardiovascular risk, lifestyle factors and medication use. The Motivational factor was not significantly associated with future cognitive impairments after adjusting for demographics. Concluding, almost all associations of general depressive symptoms and motivational-related symptoms with future cognitive impairments appeared to be confounded by demographics. Only the association of general depressive symptoms with future memory impairments appeared to be explained by a combination of demographics, cardiovascular risk, lifestyle and medication use.