Object tracking in real time video is used to control the direction and to change the speed of a small two wheeled robot. The direction and speed of the mobile robot is controlled by tracking the position of the colored objects associated with the hand of the input user.

An FPGA is used to track the position of the objects in the images generated by the digital camera. Images received by the camera are in the form of Bayer pattern which are then transformed into RGB color space. To get a binary image out of the RGB image, color space is transformed into HSV so as to improve the result of color segmentation in different light intensities. After the binary image is generated, projection histograms are used to track the positions of the objects. Different positions of the objects are associated with different instructions which are sent to the robot. The robot receives these instructions through wireless transceiver and the controller inside the robot decodes these instructions and acts accordingly.

By changing the position of the objects, the robot can be moved forward with slow or high speed, stopped, turned left or right with different angles, moved in reverse direction and can be rotated clock-wise or in counter-clock wise direction.