Indoleamine 2,3-dioxygenase (IDO) and Arginase-1 (ARG1) are amino acid metabolizing enzymes frequently highly expressed in cancer. Their expression may deplete essential amino acids, lead to immunosuppression, and promote cancer growth. Still, their expression patterns, prognostic significance, and spatial localization in the colorectal cancer microenvironment are incompletely understood. Using a custom 10-plex immunohistochemistry assay and supervised machine learningbased digital image analysis, we characterized IDO and ARG1 expression in monocytic cells, granulocytes, mast cells, and tumor cells in 833 colorectal cancer patients. We evaluated the prognostic value and spatial arrangement of IDO and ARG1 expressing myeloid cells and tumor cells. IDO was mainly expressed by monocytic cells but also by some tumor cells, whereas ARG1 was predominantly expressed by granulocytes. Higher density of IDO+ monocytic cells was an independent prognostic factor for improved cancer-specific survival both in the tumor center (Ptrend=0.0002; HR for the highest ordinal category Q4 (vs Q1) 0.51, 95% CI 0.33–0.79) and the invasive margin (Ptrend=0.0015). Higher density of granulocytes was associated with prolonged cancer-specific survival in univariable models and higher FCGR3+ARG1+ neutrophil density in the tumor center also in multivariable analysis (Ptrend=0.0020). Granulocytes were, on average, located closer to tumor cells than monocytic cells. Furthermore, IDO+ monocytic cells and ARG1– granulocytes were closer than IDO– monocytic cells and ARG1+ granulocytes, respectively. The mRNA expression of the IDO1 gene was assessed in myeloid and tumor cells using publicly available single-cell RNA sequencing data for 62 colorectal cancers. IDO1 was mainly expressed in monocytes and dendritic cells, and high IDO1 activity in monocytes was associated with enriched immunostimulatory pathways. Our findings provided in-depth information about the infiltration patterns and prognostic value of cells expressing IDO and/or ARG1 in the colorectal cancer microenvironment highlighting the significance of host Journal Pre-proof immune response in tumor progression.