Emotiv Brainwave Technology for Gaming

http://www.pcauthority.com.au/promos/interstitial/interstitial.html?l=15;icJbOR6Fv5AvztDu33FTcyrJUzm67AGlDkXuim5ZDb96gfqkgd;20091201_PCA_Dell_102381_Dec

A demonstration of Emotiv in 2008



Future tech: Interview with Emotiv co-founder Nam Do, and the science behind mind control
by Daniel Long on Dec 3, 2009

Controlling computers with our minds may sound like science fiction, but one Australian company is doing just that. Daniel Long spoke to co-founder Nan Do about the inner workings of the Emotiv technology

Emotiv Systems co-founder and CEO Nam Do is an excitable type of guy. Easy-going and bubbling with energy, he's been doing the talk circuit for a while now, spruiking the wonders of the company's EPOC device - a device that's gained attention for its potential to let gamers control their PCs with their thoughts.

The Emotiv device has been garnering attention at trade shows and conferences for several years, and now the company says it is set to launch the Emotiv EPOC headset on December 21. The Emotiv site lists the price at US$299, and says the product will be available to US customers only.

So what exactly is Emotiv's vision for the groundbreaking device, and does it live up to the hype? With Nam Do in Sydney to present the first public Australian demonstration of Emotiv at the X-Media-Lab's Sydney "Global Media Cultures" event at the Sydney Opera House, we sat down with him to talk about the technology.

While the technology certainly looks exciting, mind control is not all smooth sailing, as we discovered. Last year Emotiv's technology has suffered a technical mishap before in public demonstrations, as this video shows.

This time round, there were also technical problems when Emotiv tried to demonstrate using the device with a Harry Potter game.


Challenges such as latency are also sure to be thorny issues for gamers seeking maximum speed during their gameplay. And Emotiv isn't the only company keen to push mind control technology. OCZ and NeuroSky are also present in the market, albeit with limited neuro sensors.

Still, there's no doubt that Nan believes in Emotiv and it's hard not to get caught up in the excitement surrounding this futuristic-looking device and its many potential applications. Nan was gracious enough not to hold back with his answers. He even hinted at a relationship with Hollywood, including James Cameron's upcoming Avatar.

PC Authority: There are many technical challenges that have been holding back the technology. Are you waiting for computing power or software development to catch up?

Nam Do: I think the biggest challenge now is the signal processing part of it. EEG has been around for 80 years, but the EEG community hit a road block a long time ago and is not progressing anywhere.

We try to use a totally different approach when using signal processing. We take electrical impulses from the sensor and try to put together an image in a roughly 90,000 dimensional space and we try to classify it.

So every time you think about particular actions or you feel particular feelings, it has the same pattern - so that's why the computer knows how to do it. One of the biggest challenges is that the cerebral cortex is folded individually. It's like a fingerprint. Your cortex is different with mine.

Even if we're lucky enough to find a signal that is exactly the same that comes from deep inside the brain, by the time it projects from the skull it looks random. So the first very big breakthrough that scientists at Emotiv be able to do, is to remap that signal back to its source. That's the first breakthrough that allows us to unlock the brain.

PC Authority: Let's look at the development path. Six years ago you began and today you've shown your first Australian public demonstration. You're going to retail in two or three weeks...?

Nam Do: Yes, in three weeks

PC Authority: Has the product been released in the US yet?

Nam Do: The SDK (software development) has been out for a couple of months now. But the consumer [product] - not yet.

PC Authority: You're launching with some games, one of the ones we saw today. Are there any big titles that Emotiv will be compatible with? Or will that be something for the future?

Nam Do: Alright, what I wasn't able to before, was show an existing game.

PC Authority: Unfortunately, that's probably what a lot of people were interested in.

Nam Do: Yeah, because of the AV system

PC Authority: The graphics resolution?

Nam Do: Yeah [laughs]. But what we will be able to do with the software is called emo-key, which is basically a keyboard emulator. So you'll be able to use the headset with any compatible games.

PC Authority: That's a really big step I think. From my understanding, people who want to buy this, they'll want to play this with a lot of games, not just one or two games.

Nam Do: Absolutely. Even with the existing games they have at home, like Harry Potter, it's a completely different experience.

PC Authority: We've heard about "mind reading" technologies before. Why haven't they taken off yet do you think?

Nam Do: First of all commercially, there's not many people who commit resources like ourselves. So in the market there's only two or maximum three other commercial companies that are trying to do this.

But they are more interested in creating a gadget that takes the existing research, like the alpha and beta waves to try and push an object - so it's not really thought reading per se'. What we do is classify different thoughts and expressions.

PC Authority: So, mind reading isn't technically correct then? Is it reading the actual mind or the relationship between those thoughts and actions?

Nam Do: The brain has billions of neurons. Every time you think or you feel, neurons interact with each other. But when neurons interact with each other, they naturally emit electrical impulses - very small electrical impulses. So, if you're able to measure those impulses and be able to put it into a map for a computer to understand, then you see the relationship between those electrical pictures and your thoughts in reality. If the guy is trying to pull an object, then the computer knows the guy is trying to do that, just by reading the impulses.

PC Authority: I'm sure this isn't indicative of the software, but when I watched the demonstration today, I noticed the volunteer had a great deal of trouble trying to move the rock.

Nam Do: Yes, that's actually a mistake that new users make. When they move, then tense up. You have to understand that this is totally a new way of interaction, as you're not used to thinking about it moving and seeing it move - so you normally associate that with a physical action. So your body tenses its muscles up. But that doesn't actually help - it's a bad thing for it. If you're more relaxed and just think about it - it will do it.

For example, it's very easy for me to do it. I just think about it moving and it moves. The more relaxed you are, the better.

PC Authority: So, I assume that the training for the software is very important then? How long would it take for a first time user like we saw today, get to the point where they can play an existing game and be as good at that game as they are with their regular keyboard?

Nam Do: One training session is about 8 seconds and he (the volunteer) did one training session today. So normally within a couple of training sessions, around two to three - you'd be able to control one action pretty well.

And if you want to add actions on top of that, it takes more practice. Normally, in the context of a game, it's perfect. Because normally you don't straight away know 10 different actions and you learn one action at a time and when you master one, you move onto another. Within 10 minutes, anybody can do two or three actions easy [with the Emotiv].

PC Authority: The game we saw demonstrated today was an RPG. Would Emotiv technology work with FPS games as well?

Nam Do: It can, but the thing is I don't think it would make that much sense. Right now, the technology is not at that stage. Because before people see it, they think it's crap, but when they actually see it, they think "Wow! Okay I'm going to think left and its going to move left straight away". You're basically looking at the computer in the 70s. It's going to take years.

PC Authority: Is that where we are right now? Looking at a computer in the 70s?

Nam Do: I think so, I think so. We're looking at the tip of the iceberg. We're looking at the computer of the 70s. Everybody knows this is going to be awesome in the future and do a lot of things, but it's not there yet, just as you couldn't expect photo realistic graphics on the computer in the 70's. So the answer is no. It's mainly because of two things. One is the latency.

PC Authority: That's what I noticed, there was quite a bit of lag on screen.

Nam Do: The actual technological latency is actually 150 milliseconds - which is already very large for a first person shooter. But then there is the time for you to separate between different thoughts.

PC Authority: In the context of a first person shooter game, you have emotions such as fear or stress that would also have to be dealt with by the brain....

Nam Do: ...Yes, you can put that layer into a game. Lots of people are doing that now. A game like Fear for example can sense when people are calm and you could know exactly when the moment to scare somebody.

PC Authority: That could be very cool. There's a huge gaming potential there.

Nan Do: Yes, because right now, games like Fear are only good the first time you play it. The second time you know that little girl will appear after the door. Now with this, the AI of the computer knows that you're expecting it and the girl won't show up in the same way.

PC Authority: It reads the stress levels...

Nam Do: Yep.

PCA: Would you then need something connected to your pulse to read those stress levels?

Nam Do: No, not at all. Everything is coming from the brain - it's actually a lot faster. The pulse and skin conductance has about a 4 second delay.

PCA: So, will this be available for all platforms or just the PC?

Nam Do: The first stage is the PC. We're going to move to other platforms later. We're not Microsoft (laughs). So we don't have the kind of money and resources to make it all at once.

PC Authority: Many people must talk to you about making apps for people disabilities, particularly those lacking mobility. How do you see your company working in those areas?

Nam Do: We're already working with a lot of people, to make applications for disabled people. There are quite a few applications we're [already] seeing from independent developers just trying to create these things.

For example, some of these people can't even move. So things like the keyboard are very important. Just by thinking about it, they can put words together and start to communicate.

PC Authority: I think that's amazing. It's great to have the gaming part, but that could really transform people's lives.

Nam Do: Absolutely. Even though gaming has a lot of following, you don't realise that when you're talking about the community at large - a lot of the applications are non-gaming. Like medical or healthcare applications.

For example, university researchers and doctors are currently working on applications to treat depression and addiction - without drugs. It's a state of brain. You fall into it and stay in it. So now if you could predict that, you could have different brain exercises to keep you out of that mindset.


PC Authority: Stage one is the PC. Could Emotiv work with other devices? Televisions, mobile phones, those types of things.

Nam Do: Yes. People like Samsung are doing it as well. Controlling small things like light switches. They have applications they're working on in hospital for example, for patients in a coma. So they know exactly how the body feels. The idea is to know how they feel. It will start with applications like that and slowly go into everyday life.

PC Authority: And finally, there have been rumours about you're been working on James Cameron's Avatar? What's happening with that at the moment?

Nam Do: I'm not sure if I can tell you actually. But in general, directors are looking at it as a new method of doing focus testing. Right now everybody knows the problem with focus testing is that when somebody says the movie is shit, even though everybody likes it - they won't say it.

PC Authority: When we talk about focus testing, we're talking about general members of the public who review the film first right? The ones who write a score at the end of films before they're released? So if they really hate it, the directors will want to know why or how they hate it?

Nam Do: And how they feel when they hate something in the film.

PC Authority: And at the exact moment in the movie...

Nam Do: Right. What I didn't show today is a graph of how people feel as they engage in the game; with their own level of excitement during the game.

So if you were to overlay that graph with a movie, you could see if the director wanted people to feel excited in those scenes, or if it's not happening, then they could go back and reshoot them. And if it's too boring for the audience, they know too. It's the same with video games. But with video games, you could do it on the fly. It's called Dynamic Difficulty Adjustment.

PC Authority: So in the movie focus testing example, each person in the audience would be wearing an Emotiv headset then? And that would be connected to a central computer?

Nam Do: Yes, every person would have a small computer that they record it with. We collect the data, then overlay it with the movie so the director can look at the movie and plot it as one average, as a graph for everyone or plot it into different age groups or any other things they may want to do with it. But the director doesn't want any interpretation. They just want the raw data and they can decipher it for themselves.

PC Authority: Based on everything we've spoken about today, do you think Emotiv will change the world?

Nam Do: We hope so. That's exactly the reason why we started doing this. I mean, as you know, it's been six years of our life to get to this point. And it's going to be a long way to go.

PC Authority: So from sci-fi to reality, it looks like it's going to be quite a ride. Thanks for your time.

Nam Do: Thank you.