Reebotics Rover Opens Doors using AI


Reebotics Rover Opens Doors using AI

We have an exciting update here about our friends at Reebotic.  The team there is using the 4WD Open Rover platform and our upcoming 5 axis arm to open doors autonomously!

For years, the idea of empowering a robot with the ability to open doors seemed very hard. Now, with deep learning technology, Reebotic’s AI robot makes this a reality. The core technology behind this is Reebotic’s deep learning models which are embedded in the AI robot.

To accomplish tasks such as door opening, Reebotic’s engineers first collect thousands of photos and mark out the door knobs. Then, these photos are loaded into a Nvidia GPU server to train the deep learning model. After the training, when fed with a new image, the model is able to detect the door knob and mark out its pixel location in the image. Finally, this deep learning model is downloaded into Reebotic’s AI robot for the door opening task.

Check out the video here:

https://youtu.be/4dtH7gbVXEs

 

Detailed processes for this door knob opening task:

(1) 500+ photos with different door knobs are first loaded into a GPU server to train a deep learning model. After the training phase, this deep learning model can detect a door knob and mark out its pixel location in an image frame with near 100% accuracy.

(2) After the training, the deep learning model is downloaded from the server and loaded into the Reebotic AI robot for real time door knob detection. The Reebotic AI robot is embedded with an Nvidia TX2 GPU board.

(3) A camera is mounted to Reebotic AI robot’s arm gripper. The live camera video is then fed into the deep learning framework and with Nvidia TX2 GPU, the door knob is detected in real time. The AI robot adopts this information to guide its arm to target the door knob, get close to it, grab it, and finally open the door.

*Huge credit to Nvidia TX2 embedded GPU board and its JetPack SDK.