Cross-sectional studies have shown that the polygenic risk score for schizophrenia (PRSSCZ) may influence heterogeneity in cognitive performance although evidence from family-based longitudinal study is limited. This study aimed to identify trajectories of cognitive function and assess whether the PRSSCZ is associated with baseline cognitive performance and predicted six-year trajectories. We included 1119 patients with a schizophrenia spectrum disorder, and 1059 unaffected siblings and 586 unrelated controls who are eligible at baseline. Genotype data were collected at baseline, whereas clinical and sociodemographic data were collected at baseline, three and six years. Group-based trajectory modeling was applied on a weighted standardized composite score of general cognition to unravel cognitive subtypes and explore trajectories over time. We followed a standard procedure to calculate the polygenic risk score. A random-effects ordinal regression model was used to investigate the association between PRSSCZ and cognitive subtypes. Five cognitive subtypes with variable trajectories were found in patients, four in siblings and controls, and six in all combined samples. PRSSCZ significantly predicted poor cognitive trajectories in patients, siblings and all samples. After Bonferroni correction and adjustment for non-genetic factors, only the results in all combined sample remained significant. Cognitive impairment in schizophrenia is heterogeneous and may be linked with high PRSSCZ. Our finding confirmed at least in all combined samples the presence of genetic overlap between schizophrenia and cognitive function and can give insight into the mechanisms of cognitive deficits.