Background: Previous research has shown an association between adverse childhood experiences (ACEs) and secondary mental health problems in youth with prenatal substance exposure (PSE), but the association between ACEs and neurodevelopmental disorders is less clear. Methods: This longitudinal register-based cohort study investigated relationships between health at birth, ACEs (out-of-home care (OHC) and maternal adversities), and neurodevelopmental disorders among youth with PSE (alcohol/drugs, n = 615) and matched unexposed controls (n = 1787). Hospital medical records and register data were merged and analysed using Cox regression models. Results: Conduct and emotional disorders (International Statistical Classification of Diseases and Related Health Problems ICD-10, F90–F94) were more common among the exposed than the controls but only when the exposed and controls with no OHC were compared. The difference appeared in hyperkinetic disorders (ADHD, F90), mixed disorders of conduct and emotions (F92) and emotional disorders with onset specific to childhood (F93). Among the exposed and controls with OHC, over 40% had received an F90–F94 diagnosis. Regarding specific developmental disorders (F80–F83, e.g., impairments in speech and language and scholastic skills) the moderate difference between the exposed and controls attenuated after adjustment for OHC. Again, the rates were highest among the exposed with OHC and the controls with OHC. OHC and maternal risks were interrelated and, together with male sex and low birth weight, were associated with neurodevelopmental disorders both among the exposed and controls and decreased the difference between them. Conclusions: A strong association between ACEs and neurodevelopmental disorders was found. Brain research is needed to examine whether ACEs worsen neurodevelopmental outcomes caused by PSE.