As I wrote earlier this month, I bought a LeapMotion device (during the pre-selling phase), and as it arrived I tried to find some way of playing with it, not only with the apps from its store, but with its API, in particular integrated with Processing.
After some experiments with movement detection, coloured boxes and whatnot, I decided to try something physical. I had a couple of servos lying around, and they too were still looking for some test application (I had never used them), so I decided to put everything together. I built a crane using some Legos (from my youth) that were in a box in my closet here in my parents’ house, adding the two servos even if they do not fit perfectly (but the connection of the servos was almost perfect with the Lego gears, so in the end it was not that bad). The crane has two degrees of freedom: the red platform rotates with a continuous servo, and the black arm goes up and down with a standard servo.
The Arduino has the Adafruit motor shield (the old version actually), and I uploaded a firmata sketch to send commands from Java (Processing) without having to come up with a specific protocol; on the computer side, a Processing sketch receives frames from the LeapMotion and detects two movements: the inclination of the palm of the hand (detected only if there are at least 4 fingers in view, this means that there should be some distance between the single fingers, otherwise it doesn’t work) and the circle gesture, that happens when a finger performs a circle clockwise or counterclockwise; the first movement raises the arm, the second movement moves the platform.
The video shows it working: the palm movement is sometimes detected with some delay, I think because of the angle values of the palm inclination that I put in the sketch (also, sometimes the angles assume some strange values, completely different from the previous ones, I have no idea why…*). Anyhow, it was nice to write some really easy code to detect stuff with the LeapMotion and apply it to physical stuff (instead of virtual stuff, like the apps), even if it was a toy application.
* I have to add that Linux users, at least as of today, are still second-class citizens for the LeapMotion world: not only there are no apps (but I don’t really care about this), but the SDK has a quite low version number with respect to Windows and Mac OS X, even if the latest news update from LeapMotion devs says that the latest version of the SDK is coming to us as well. This may be the cause of some misdetection happening from time to time.