Aim: This study aimed to examine the prognostic value of extramural venous invasion observed in preoperative MRI on survival and recurrences. Method: In total, 778 rectal cancer patients were evaluated in multidisciplinary meetings in Helsinki University Hospital during the years 2016–2018. 635 patients met the inclusion criteria of stage I–III disease and were intended for curative treatment at the time of diagnosis. 128 had extramural venous invasion in preoperative MRI. Results: The median follow-up time was 2.5 years. In a univariate analysis extramural venous invasion was associated with poorer disease-specific survival (hazard ratio [HR] 2.174, 95% CI 1.118–4.224, P = 0.022), whereas circumferential margin ≤1 mm, tumour stage ≥T3c or nodal positivity were not. Disease recurrence occurred in 17.3% of the patients: 13.4% had metastatic recurrence only, 1.7% mere local recurrence and 2.2% both metastatic and local recurrence. In multivariate analysis, extramural venous invasion (HR 1.734, 95% CI 1.127–2.667, P = 0.012) and nodal positivity (HR 1.627, 95% CI 1.071–2.472, P = 0.023) were risk factors for poorer disease-free survival (DFS). Circumferential margin ≤1 mm was a risk factor for local recurrence in multivariate analysis (HR 5.675, 95% CI 1.274–25.286, P = 0.023). Conclusion: In MRI, circumferential margin ≤1 mm is a risk factor for local recurrence, but the risk is quite well controlled with chemoradiotherapy and extended surgery. Extramural venous invasion instead is a significant risk factor for poorer DFS and new tools to reduce the systemic recurrence risk are needed.