Facial recognition software can tell when you're frustrated with Xbox Live

Most of us have been faced with the anguish of being shot in the head repeatedly by 12-year-olds. There are also the times when we’re overjoyed by defeating the Mother Brain and making it out of the caverns of Zebes. If we wanted to scientifically quantify how happy, sad, or angry we are while playing video games, we wouldn’t know what to do. [Dale] came up with a very interesting way to gauge someone’s state of mind while either playing Xbox, or watching TV.

To get a measure of how happy or sad he is, [Dale] put a webcam underneath his TV and pointed it towards his couch. Every 15 seconds or so, the webcam snaps a picture and sends if off to the face.com API. After face.com sends a blob of JSON containing information about all the faces detected in the photo, a short Python script plots it on a graph.

[Dale] admits he’s not entirely scientific with this project; the low resolution of the webcam, coupled with images being captured every 15 seconds means he runs into the limitations of his hardware very quickly. Also, there’s the confound of [Dale] paying attention to something else in the room – like his kids – rather than the TV. Still, it’s an interesting use of hardware and software that would be loved by a market researcher or QA designer.

Comments

  1. healthy says:

    is this graphing showing him getting to the end of mass effect 3 and it’s oh so sad mishandling of the ending?

    • davidcdean says:

      I’d say it’s an accurate graph of my gaming experience, most of the time.

      One day I’ll wake up and realize my gaming time is more stressful than my average work day. 😉

  2. Colecago says:

    I have finger recognition to tell my mood.

  3. Addidis says:

    So game makers should look at that graph with horror. Basically games make you angry 94% of the time and make you happy 4% of the time. I dont think surprise is really worth while to have on there since surprise is an instantaneous reaction that quickly fades more then likely before the 15 second interval between pics.

    Great idea.

  4. bandwagon says:

    It’d be neat to see a video capture of what’s playing on TV with the graph overlaid on top.

    • PI says:

      I am reminded of a time I was walking down the Vegas strip, and somebody stopped and offered me dinner and free show tickets to participate in a “survey.” The “survey” was actually the review of the pilot of a proposed television sitcom.

      I was led into a cubicle farm, each cube equipped with a computer, a set of headphones, and a hand-held switch. My instructions were to watch the show. When I felt happy, amused, or entertained, I was to push the green button. If I was offended, bored, or otherwise disgruntled, I was to push the red button.

      At the end of the exercise, I they tallied the button presses of many hundreds of people, and created a moving strip chart graph that was superimposed on the program. This would allow developers to review the pilot and simultaneously see what “humor” worked with the audience and what did not.

      It seems that the gizmo in this HAD post could be used to automate or auto-collect the same kind of information.

      The sitcom, by the way, was unimaginative and unfunny crap. That is to say, you should expect to see it on network TV any day now 😉

  5. xiuix says:

    I could see this technology being used in a Game like deadspace. If the game thought you were not scared SH**less from scary monsters it could throw more your way.

  6. Retroplayer says:

    This would be useful for a lot of games. If a particular point in a level has gotten too difficult and you just can’t get past it, it would be nice if the games were smart enough to know to make it just a little easier. Of, course then go back to normal.

    Then again, the game could simply track that by the number of repeated tries.

    Nothing is worse than being stuck at a point in a game and just finally giving up and not playing that game anymore.

    • Tim says:

      This is what happened to me with the original Alice 🙁 There was probably tons of beautiful game I never got to play cause I didn’t have the skills.

  7. Nate says:

    Haha I would say that’s pretty accurate.

  8. John says:

    Be interesting for play-testing using the kinect camera.

  9. echodelta says:

    Marketeers and Big Brother-The Grid.

  10. torcue says:

    Yeah, now ms will be tracking for market research, and all we’ll ever get are happy Smiley game 7… not realizing that the journey , story and drama are more to a good game than happy

  11. Mental2k says:

    Would be a fantastic use of the kinect! I always assumed games were supposed to frustrate a certain amount, otherwise there’s no satisfaction from actually completing it. Look at vvvvvv!

  12. ExtraRedeal says:

    Very good thing, if the player gets VERY frustrated AND a vibrational sensor inside the console (The player beats it), some automatic difficulty level decrease could be triggered, or an extra life (action) or deal (card) could be given. Better spoiling a cheap game than the expensive hardware, (or even life as the console contains 120 Volts!) But how to know if the player is truely frustrated and wants to break the console?

  13. Roger Wilco says:

    I bet it depends on what game you are playing. I react different to a first person shooter than to a driving game.

  14. Web Design says:

    I can’t make it work. I keep getting errors. any1 successful?

Speak Your Mind

*

Related Hacks in digital cameras hacks

  • Hackaday Links May 9th 2012
  • Converting a manual camera lens to use motorized zoom and focus
  • 6 camera face scanning rig
  • Quick and easy wildlife camera
  • Adding a remote shutter to a cheap digital camera