Perception
To gather the different resources, first it is necessary to segment the image based on the color of the resources (red, green, blue and yellow) and second, find the location of the center of each resource to plan the appropriate grasping strategy.
To segment the image based on color, using OpenCV a masks for each color is applied over the image to indicate which pixels are of a particular color. To make the masking process more robust to lighting changes and shadows, the image is processed in the HSV (Hue, Saturation, Value) representation, rather than in the RGB color model. The HSV values for the different masks are summarized in Table 1.
To segment the image based on color, using OpenCV a masks for each color is applied over the image to indicate which pixels are of a particular color. To make the masking process more robust to lighting changes and shadows, the image is processed in the HSV (Hue, Saturation, Value) representation, rather than in the RGB color model. The HSV values for the different masks are summarized in Table 1.
Table 1. Hue, Saturation, Value (HSV) Ranges for Image Segmentation
To reduce the amount of discontinuities on the mask, the image is then dilated with a unit kernel of size six by six. Since there may be multiple blocks of the same color on a given image, it becomes necessary to identify the center location of each block in the image. To do this, a canny edge detector is applied to detect the boundaries of each block of a given color, followed by a contour detection (the dilated mask of each block gives a closed contour around each block).
To filter out undesired contours that may result after applying the Canny Edge Detector, only the largest contours (using a threshold of 500 in perimeter) are considered. Each contour can then be approximated by a convex hull, who's center can be easily computed by the image moment.
To filter out undesired contours that may result after applying the Canny Edge Detector, only the largest contours (using a threshold of 500 in perimeter) are considered. Each contour can then be approximated by a convex hull, who's center can be easily computed by the image moment.
Figure 1. Image Segmentation for Resource Location. (a) Mask for yellow block, (b) Mask dilation, (c), Canny Edge Detection and (d) Center of each block (Image moment) represented as a white dot.