Robotic vision is a subfield of computer vision intended to provide robots with the capability to visually perceive the surrounding environment. For example, a robotic manipulator leverages its visual perception system to gather visual data through cameras and other sensors, then uses that input to recognize different objects in order to safely perform an autonomous operation.

However, in many robotics applications, robots have to face a cluttered and dynamic scene, where classic computer vision algorithms show the limitation of tackling the environmental uncertainty. Such scene understanding requires a fusion of traditional and modern approaches involving classic computer vision, machine learning and deep learning methods.

This thesis examines visual perception challenges in remote handling and the mining industry. It begins with two research questions: Can the robustness of targetobject pose estimation be improved in challenging real-world, heavy-duty robotic scenarios? Can fast detection and localization for objects be obtained without prior known geometry in a scenario with piles of overlapping objects? Six publications cover the methods from algorithm design to system-level integration used to solve real-world problems.

In the ITER fusion reactor, the operator teleoperates a robotic manipulator to perform maintenance tasks amidst a high level of noise and erosion. The operator cannot fully rely on the virtual reality (VR) system, which may not reflect the current scene accurately, as physical conditions may have changed in the harsh environment. Meanwhile, every operation inside the reactor requires robust, millimeterlevel accuracy. This thesis analyzes research questions and presents a novel edgepoint iterative closest point (ICP) method as a solution for target-object detection, tracking and pose estimation. Using the knuckle of a divertor cassette as an example, the overall accuracy of the developed visual system meets ITER requirements, and the conducted experiments with the manipulator demonstrated the efficiency of the method.

Smartbooms2 is a project in the mining industry that requires a heavy manipulator with a hydraulic hammer to autonomously break rocks in a cluttered outdoor environment. Based on the output data of the three-dimensional (3D) sensors, several solutions are proposed. Examining a popular time-of-flight (TOF) sensor, this thesis explores state-of-the-art unsupervised machine learning methods and proposes a novel clustering method. Using an industrial stereo camera, this thesis proposes a novel 3D rock detection and localization pipeline. The results and system accuracy are detailed in published research papers.