I stopped into Best Buy yesterday and picked up something I had
been considering toying with: a LEAP Motion. This is
an interesting little $79 device which turns the airspace at your
computer into a live space for interaction. It recognizes all ten
fingers on your hands, differentiates between left and right hand,
open and closed, and can read the pitch, roll, and yaw of your
hand, pinch and swipe gestures and more, all in meatspace.
I haven't yet torn it down to see what type of imagine sensors
it is using to map the space around it. Of course, the folks at
SparkFun already have, however, so feel free to look at their
article if you want to see what's in this tiny box.
It appears to be sensitive to IR light, similar to how the
Kinect is sensitive to too much sunlight in a room. I turned on a
compact fluorescent to take a photo and this popped up:
For some reason, the LEAP Motion uses a proprietary USB
connector. It's normal USB on one end and then what looks like a
mini and micro next to each other on the other end. I have no idea
why they did this, as there's no electrical reason given the
standard single USB connector on the USB side. They did think to
provide two lengths of USB cable, so that's a help. However, if the
LEAP Motion is something you intend to travel with, you'll need to
make doubly sure you have the right cable, as you can't just borrow
one from a friend or pick up a spare locally.
CORRECTION: This is not a proprietary connector. It's a
USB3.0 Micro-B connector -- the first I've seen in the wild. Still,
good luck finding a cable in a store. At least it's a standard
cable, however. Also, this suggests that plugging into a USB 3.0
port would be a good idea.
As a matter of taste, I prefer my USB cables on my desk to be
black so I don't see them. White cables are distracting. Also, this
cable is a bit stiff and tends to pull the LEAP Motion
Anyway, once plugged in, the firmware updated. Unfortunately,
that bricked it and afterwards, it wasn't recognized. It showed up
as "Westbridge" under "other devices" in the device manager. I
searched on this, and ended up on the LEAP Motion site where they
very helpful troubleshooting page and firmware reset
utility to fix this exact problem.
Once I saw "Leap Dev Kit" in the device manager, I knew it was
working. It is interesting that it shows up as a "Dev Kit" and not
a consumer device name. Perhaps the firmware fix download resets it
to some sort of dev status, or was intended for developers.
As an aside, the LEAP Motion appears to be referred to both as
"Leap Motion" and "LEAP Motion" on their website and by their
employees. Not sure which is correct, but "Leap Motion" appears to
be what shows up all over the PC after the installation. I'm
surprised their branding folks didn't go crazy on them for mixing
Testing with the Visualizer
Now that the device was plugged in and recognized, and showing
the green power LED, I wanted to test it out. I right-clicked the
LEAP Motion icon in the tray and selected "Visualizer…" so that I
could test the basic operation.
This launched the "Leap Motion Visualizer" program. Here you can
see a screen shot of the visualization of a single hand. It works
with both hands (but I needed one to hit print-screen), and also
identifies the position of each finger. This was with zero training
or setup other than completing the basic install.
Cool, so it's working. For grins, let's see if I can control
Windows 8.1 with it.
LEAP Motion includes an app store. I hate that they have a
separate app store with their own payment model, DRM, and more when
each supported platform has its own story anyway. I would have
preferred they listed their app in the Windows Store as a desktop
One of the apps in the app store is "Touchless". This is a free
app, but you need to create a LEAP Motion store account to download
it. (Really. I thought I bought a device, not an ecosystem. It
makes me nervous to have yet another company with my account info
and, as you'll see later, my payment info.)
Once I created an account (with a fake birthday - why is
birthday required?) and clicked the button to get the app, I was
then able to see it in the Airspace client. Note that the apps
don't download from the store, you need to use the Airspace client
app. However, the airspace client includes no provision for
interacting with the store. Have I mentioned my annoyance at having
yet another store?
I launched the Airpspace client:
Which opened a full-screen window with the "Airspace Home",
complete with all of the installed apps. It immediately started
downloading the Touchless app I purchased on the web.
The touchless app is, by default, a single-point mouse
replacement app. This is interesting, but not super useful. If
you've ever tried to use your finger as a mouse using Kinect, you
know there are more efficient ways to interact with your PC. I was
more interested in the multi-touch side. To access multi-touch,
right-click the Touchless tray app and select
Now the app sends each finger as discrete points. I test this
out, I launched one of my favorite multi-touch Windows 8 apps:
It takes some practice, but I was able to get the app to
recognize from one to five touch points. I should be able to get it
to recognize all ten, but I didn't have time to noodle with that
much more. Note that MorphWiz was written without any knowledge of
the LEAP Motion - it just saw it as touch input.
Not all apps worked like this, however. For example, I was
unable to get the "Vintage Synth" app to recognize any of the
finger inputs. In the future, I'll try the LEAP Motion with some of
my own Windows 8.1 code to see if it's a question of how you use
the pointer/mouse APIs.
LEAP Motion also have an interesting free app named "Kyoto"
which lets you interact with a seed, tree, water, moon, meteors and
other stuff while playing music reminiscent of Minecraft. It's free
and you'll find it amusing for a few minutes.
Controlling a Synthesizer
When I bought this, I had no intention of using it to control
Windows. I'm really happy with my
Logitech Touchpad in that capacity. What I'm really interested
in is how this could work for musicians and performers as another
way to control dynamics.
This seems to be a scenario of interest to the Leap folks as
well, as one of the bundled Airspace store apps, Lotus, is all
about music performance control. Once you poke the eye you get
offered a set of musical experiments that you select by pointing
with one, two, three, or four fingers at once. All are worth a try,
but I find myself coming back to the one with a head in it. Using
two hands to control, you can get some really interesting
performance capabilities. For example, by closing your fist, you
close the filter. By spreading your hands out, you increase the
stereo width. Lots of other effects as well. Very neat.
I searched their store for "MIDI" to find some MIDI controller
The prices are high. That's one problem with a niche market and
a small app store. $30 is almost unheard of in app stores these
days, especially for controller software that requires $80 in
investment in specialized hardware. I didn't both with the AeroMIDI
app, but did decide to try out the Geco MIDI app. It's relatively
expensive as well ($10), and had no trial, but I was able to find
decent information on its operation on their web site. Besides, $10
is like a few coffees. Purchased.
This is installed like the other store apps. As this one
requires payment, you have to have a credit card set up ahead of
time. I was hoping to use PayPal, but credit card is the only
method of payment accepted.
One installed, go ahead and run it from the Airspace shell. Be
sure to turn off any other apps (such as Touchless) first, as they
will interfere with each other.
I ran the app, and went into the "Document Settings" dialog to
configure the MIDI connection. The configuration for which messages
are assigned to which gestures, and which device to use, are all
stored as a document.
(note the branding in the title bar. Another confused company:
GECO in the app, Geco in the listing).
I wanted to control the filter cutoff of my Moog Slim Phatty.
So, I pulled up the manual and looked up the MIDI CC (Continuous
Controller) messages. The Sub Phatty supports 14 bit (high
resolution) MIDI, but GECO does not. So, I had to go with the 0-127
value 7-bit classic MIDI.
The Filter Cutoff Frequency is CC #19. The value range shown in
the table is for 14 bit. For 7 bit, it's 0-127. I then set this up
in the GECO app, assigned to the vertical position of my open left
One that is set, all I needed to do was play a note on my Sub
Phatty with my right hand and then use my left hand to control the
filter cutoff. Despite the lack of 14 bit MIDI support, it sounded
great. The Sub Phatty does a decent job of interpolating the 7 bit
MIDI messages to avoid obvious stepping with filter cutoff (I could
hear stepping with resonance, however).
I also mapped resonance to roll inclination. That was a bit
awkward as your hand needs to be turned almost upside down to get
the full range, but not too bad once you got used to the behavior
at the extreme values.
I pulled up an older sequence in Cubase with the intent of using
both hands to control the dynamics while the sequence plays.
However, as GECO is not yet a VST, and you can't share a MIDI
output between two programs at once on Windows, I was unable to use
both GECO and Cubase to control the Sub Phatty. A workaround here
is to use one over USB and the other over regular DIN MIDI, but I
did not get a chance to try that out. Instead, I did the gesture
control with my left hand and playing with my right.
This is a pretty neat unit, especially for the price. Apps are
(for a modern app store) overpriced, but there's a developer
program you can sign up for to write your own apps. Given that it
requires a custom driver, this won't work on Windows RT, but I'll
check it out in Windows 8.1.
The LEAP Motion was incredibly smooth and well-tuned right out
of the box. Like any airspace-gesture technology, it required a
steady hand for things like touching points on the screen. In that
way, it's a bit of a novelty. As an additional vector for
performance control, however, it really shines. It's like the
Roland D-Beam from their V-Synth and others, with many more types
of recognition other than just distance.
Costing not much more than a decent mouse, the LEAP Motion is a
great device to add to your desk, especially if you like to
experiment with new ways of interacting with your computer and/or
instruments. My advice: get one if you have other creative
uses in addition to normal UI control (musicians, folks playing
with NUI, etc.). If you're looking to buy one just to
replace a touch screen on Windows or OSX, I don't think
you'll be happy in daily use.
I didn't have a chance to record a video today, but once I learn
more about how to incorporate the LEAP Motion into my work, I'll
post some examples. For now, here's some very short audio: right
hand is on the Sub Phatty, left hand is using the LEAP Motion to
control filter cutoff with open hand Up & Down Position
(1/19/1%/100%) and resonance with open hand Roll Inclination
(1/21/-1%/-). On the keyboard, all I did was hold down a single key
- all the dynamics are coming from MIDI control via the LEAP
The above screen shot shows status of hand height (left bar) and
roll (second bar).