This entry is about using luvcview under Linux for some very simplistic video recording for your eye tracker, or any other purpose. I discovered these things while I was trying to set up the visible spectrum Starburst algorithm, but after trying out several algorithms I decided to use a different one…  so I don’t know if these will be of any use to anyone! Still, here goes.

Once you’ve bought a webcam (which will be the subject of a different post) you can test it out using luvcview, a very simple video app. (Remember, you want to test with something simple, and if it works, you can move on to something more complicated.) This little Linux app is even suited to recording two concurrent video streams for your eyetracker, try this:

luvcview -d /dev/video1 -i 30 -o videoA.avi
luvcview -d /dev/video0 -i 30 -o videoB.avi

luvcview saves config with F1, loads with F2! To say this is underdocumented is an understatement – it’s not in the help or any of the obvious places, it took me half an hour to find it.

Here is a sample config file. I confess I have no idea where I found it – Google only brings me up the source code for luvcview.

id value # luvcview control settings configuration file
9963776 128 # name:Brightness type:1 min:0 max:255 step:1 def:128
9963777 32 # name:Contrast type:1 min:0 max:255 step:1 def:32
9963778 32 # name:Saturation type:1 min:0 max:255 step:1 def:32
9963788 1 # name:White Balance Temperature, Auto type:2 min:0 max:1 step:1 def:1
9963795 255 # name:Gain type:1 min:0 max:255 step:1 def:0

Unfortunately there is no way (that I could find) to save exposure settings in luvcview. This is really annoying, because one of the webcams I used required the exposure to be set to a nonstandard setting to capture at 30 fps.

I tried and tried and tried to get two streams to start recording at the very same time (either with luvcview or something more complicated) and I couldn’t find a good solution – I wasted a whole day with this, but then I realized Joel Clemens who made the Windows version of Starburst couldn’t find one either:

- While recording your eye tracking videos, be sure to flash the user with a camera flash or equivalent strobe to allow a spot where you can see where to sync the videos. (this is because it is not reasonable to assume that we can start recording from the 2 cameras at exactly the same time)

So I was a bit relieved His solution worked very well for me in practice. I simply used my cell phone camera to produce a flash, and then found the first frame and cut both video streams there. (I haven’t tried his Windows port of Starburst because I only have Matlab for Linux – but I have to say his documentation was very useful even for the Linux version!)

Simply using the camera flash method and luvcview to record two streams produces two nice streams which can be input to Starburst… but Starburst will be the subject of a separate post, IY”H.

What to do if you experience framedropping, glitches, etc. while recording from two USB webcams at the same time which are not present when you’re only recording from one of them? 1. Don’t panic 2. read this quick howto!
Basically you have two webcams taking images at a resolution of at least 640 x 480 at 30 frames per second. If you’re unlucky like me, then you end up in a situation where both cams work fine when used individually, but when both are plugged in, the video seems to stutter and drop frames, even though you have USB 2.0 slots. Unfortunately this situation was not covered on any of the low-cost eyetracking websites I came across. I was really stumped, at first I thought it was a driver issue with my Logitech webcams, but it arose both under Linux and under Windows, so eventually had to I rule that out.

The real reason is that on some laptops (maybe even desktop computers…), all of the USB slots are assigned to the same USB bus. An USB bus is inside your computer, so normally you can’t tell by inspecting your laptop visually. Even slots on different sides of your laptop can be assigned to the same bus!

This is how to tell:

1. Unplug all of your USB devices.
2. Run the command “lsusb” under Linux. This shows you all of your USB buses.
3. Plug one of your devices into one of the slots.
4. Run lsusb again. The device should appear on the list, assigned to a specific bus.
5. Unplug the device.
6. Repeat 3-5 until you’ve found out which USB slots on your laptop connect to which buses. I have 3 slots, all of which belong to the same bus! (There might even be internal buses you can’t access without taking the laptop apart – in my laptop there is a built-in fingerprint reader which has its own bus all to itself, as you’ll see in the sample image below.)
7. Connect both cams to USB slots which belong to different buses.
8. Record!

This is an example of what you should see when you’re finished – both cameras plugged in, each on a different bus (click to enlarge):

If you’re unlucky, all of the available slots belong to the same bus. Or maybe you have slots where your device does not fit (some laptops have slots in really unwieldy locations) or you just don’t want to unplug something. What to do now?

The solution is to buy a PCMCIA to USB converter. These are reasonably cheap and most laptops have a PCMCIA slot where you can put them (or even two). Put the PCMCIA card into your laptop, boot up, plug one of your cams into a free USB slot and the other into the card. This way the data flow from both cameras won’t be directed through the same USB bus and you will hopefully experience smooth video on both.

Disclaimer:
This worked for me and is a cheap solution to a persistent problem I had when building my eye tracker. Still, no guarantees! It might not work for you. Try it at your own risk.

Unfortunately there is a nasty followup to this, namely that the bandwidth required is quite a lot – take a look at device bandwidths on Wikipedia and you can see that USB ranks quite high on the list. So if you want to make your head-mounted rig wireless, then Bluetooth is probably not an option, having much less bandwidth than USB (though I haven’t tried yet, but it looks futile). The ideal solution would be WUSB or the new third-generation Bluetooth protocol, but these are really “emerging technology” in the worst sense of the word – expensive, easy to break, hard to find, etc. Still, in a few years you will probably be able to do that in a relatively cheap and painless way, so keep this in mind!

For the time being, if you really need to make your recording rig portable, you can mount your laptop on your back… I have an ultraportable, so this works for me! One might think Eee PCs or similar netbooks would also be suitable, but unfortunately video recording is quite processor-intensive, so you want a powerful computer (or some custom-built setup). And powerful ultraportables are really expensive (I didn’t buy mine for eye tracking either…). Anyway, most eye tracking these days is aimed at people sitting in front of a screen, not wandering around and interacting with random objects, so it’s probably not a problem if your rig is not wireless (it’s quite portable anyway). I just think it would be cool.

Also, lots of eye trackers only use one camera, but I don’t like those setups – compensating for head movements is much easier if you have a scene camera in addition to the eye camera. Of course, I’m doing offline measurements… a lot depends on exactly what you want to do, and there is a tradeoff between accuracy and online processing.

I’m a doctoral student in Clinical Neuroscience at the Medical University of Vienna and my thesis is about the computational modelling of visual attention in autism spectrum disorders (and hopefully eventual clinical applications, e.g., for diagnosis, etc.). The focus is on the modelling aspect – my advisor, Georg Dorffner, is a mathematician – but I wanted to gather some empirical data, and hence I have recently started looking into low-cost eye tracking. The reasoning was that since most of the thesis would be about the model ling work anyway, it might not make sense to buy a top-of-the-line eye tracker just to gather a bit of data. Also, I like building things, so why not?

Unfortunately many of the claims are quite exaggerated when it comes to algorithm performance. Building an eye tracker (even a two-camera head-mounted setup with IR) is quite easy – much easier than most of the online tutorials would make you think -, but there are serious problems with software stability, reliability, or just accuracy in general. (Stability would be easier to fix…)

I still think it is possible to gather data on the cheap, but you have to be tricky about it. One of the main reasons I set up this new website, and do so using a blog format, is that I want to rant about various aspects of building a cheap eye tracker ;]  I’m on the 5th PhD conference of the MUV today, I have a poster about building an eye tracker and choosing algorithms etc., but there are so many things that don’t fit on a poster, and generally the internet is a good place for, well duh, online tutorials. And I’m also interested in the exchange of ideas, etc., the comments section is there for a reason! (Yes, I’m looking at you!)

I’m also a masters student in Theoretical Linguistics at Eötvös Loránd University in Budapest. I started this degree when I realized I was running out of available courses my psychology masters and I wanted to study something else while I was at it. (It was a scary snap decision. I wanted to do something else related to cognitive science because psychology just didn’t seem substantial enough by itself. At the time I was working part-time in a lab and I saw this from up close.) I’m now ABD and have been so for a year – I haven’t even been in Budapest since January! -, so I should really really finish my thesis and defend it ASAP, and then publish. I’ve already put a lot of effort into it so I don’t want to abandon it altogether, even though I’m now a doctoral student somewhere else.

It is about a Construction Grammar model of Hungarian (not all of Hungarian, obviously…. but I aimed for a reasonably broad coverage) which has already been tested on psycholinguistic data for my psychology thesis and it worked fine. Now for my other thesis I’m making software which will allow everyone to use my model and build on it (add constructions, etc.). It will have a GUI (I’ve already made that part) and the only prerequisite for using it will be the ability to use XML. XML is easy! I’ve also written a lot of documentation… quite an amount of the thesis is essentially documentation so far.

Basically you have a set of constructions in an XML file, and the software parses Hungarian sentences using these constructions and provides a Construction Grammar (-ish) representation. I’m saying “-ish” because no one really agrees on what Construction Grammar really is and how it should work in practice. The parser and construction matching algorithm is written in Python and I’m not a great programmer so this has taken up most of the time so far. But Python is useful and versatile, so any experience I gain with it is worthwhile I’d say. Besides, the video game I’m developing in my spare time also uses Python (and pygame and renpy) so the experience I gain in one should transfer to the other… in an ideal situation… well I don’t need to parse XML with Python for a video game!

Anyway, the “meat” of it is in the construction declaration framework and the Python code that makes use of it, and the actual set of constructions. So far a lot of work went into defining the constructions the way I want them to be.

I really want to finish my thesis this summer and defend it this winter. If I post here, that will hopefully help me with that, because then I will have this urge of “must-post-something” and if I haven’t made any advances I could post about, well, too bad. And there are very few linguists reading my personal blog which, when it is not about my thesis projects, is all about my little emo life. (Dear readers, you will be spared all that, this website is for my public projects. If you want to read about my daily aggravations, you will have to find a way to read my friends-only blog somewhere else, it’s not public and for good reason! My personal life just isn’t that interesting.) So anyway this seems a good place for myself to urge myself on, and hopefully get other people interested in the process.