This wire covered glove is capable of turning your hand gestures to speech, and it does so wirelessly. The wide range of sensors include nine flex sensors, four contact sensors, and an accelerometer. The flex sensors do most of the work, monitoring the alignment of the wearer’s finger joints. The contact sensors augment the flex sensor data, helping to differentiate between letters that have similar finger positions. The accelerometer is responsible for decoding movements that go along with the hand positions. They combine to detect all of the letters in the American Sign Language alphabet.
An ATmega644 monitors all of the sensors, and pushes data out through a wireless transmitter. MATLAB is responsible for collecting the data which is coming in over the wireless link. It saves it for later analysis using a Java program. Once the motions have been decoded into letters, they are assembled into sentences and fed into a text-to-speech program.
You’ve probably already guess that there’s a demo video after the break.
http://www.youtube.com/watch?v=Mt8RCyzaQ_4
Now you just need a gorilla that knows sign language, and you can go looking for diamonds in the Congo.
This is great tech making life better for those less fortunate. I also love the idea above. Gorillas have been taught sign language and even to use touch screens to talk for quite some time. Animals are way more intelligent then people give them credit for. I would probably pay to see a gorilla talking to its trainer using this. This is much more exciting then it seems.
In desperate need of miniaturization but a good idea, this could come in very handy for deaf people who have a difficult time speaking (deaf from birth) communicate with people who don’t sign.
“…and it does so wirelessly.”
>sees arm covered in wire rats nest.
Made me chuckle 😀
Yah.. Gorilla.. thats the first thing in my mind also.
Amy, good gorilla.
Amy!
I know more than a few def people. This would just get in the way, and writing is much quicker for non signing people.
Epic none the less. Now get it to recognize signed words and not just the alphabet. Keep up the good work.
Correct me if I am wrong, but to recognize sign “language” (as opposed to basic ASL (the difference between German 101 and a native speaker)), you would need to wire both arms up to at least the elbow? And isn’t there as much dialectic variation, slang, and nuanace in ASL as in any spoken language?
I imagine that word-for-word translation (from ASL to english) would not be alot better than word-for-word translation between english and german, or spanish and italian.
I think this is a terrific thing, but it is a LONG way from a usable translator. I agree that writing is probably faster (now), but this could become much more compact. Remember the first mobile phones were the size of a medium-sized purse. The translation of spoken languages has gotten pretty good, but people have been working on speech recognition and text-text translators for DECADES. I would be interested in seeing how far this tech can go.
Yep,
I learned ASL to tutor someone while in college (as much as someone can learn in a semester). Very few things are spelled out manually. Most are acted out as gestures with both hands. The issue of nuance and slang could be overcome, as many speech texts have done, but you’d have to come up with a much larger system to sense actions that mean entire words.
this is still awesome, and a step in the right direction.
you might wanna see this project we did last year –
similar but different 🙂
http://www.youtube.com/watch?v=syXVs9DmOCg