Monday, July 25, 2011

So are you an application man or an algorithmic man

Have you tried the Microsoft's Kinect?
Regardless your opinion on Microsoft's products, it is hard to argue that the Kinect isn't a good piece of technology. The sensors it boasts are quite good and the algorithms it uses are state of the art human recognition algorithms. (Andrew Blake was working with the Kinect team)

So, if you were part of the Kinect team, would you like to play with the cameras and sensors, or would you like to see your algorithms in action? 

Most of the times, there are 2 sides of a research story, the application and the algorithms you used. Mathematicians and Statisticians work is focused on theory and algorithms. Grad students on Maths or Statistics are sure to develop an extension of an existing algorithms or, in a good case scenario, a new algorithm or theorem. Engineers, however, focus in the application part. Few times you'll see an engineering thesis and find a new algorithm, or a deep mathematical analysis of whats happening in a system (Control Theory people are an exception). You'll find, though, a really good explanation of the hardware and the best way to do an implementation on it.

But what about computer science, and of course Machine Learning?

While some researchers view Machine Learning as a tool, others view it as an end. This primal statement will shape your research in ML. If you view Machine Learning as a tool, you'll probably will want your research to focus in a specific application. Let say computer vision, robotics or bioinformatics. In these applications the algorithms you'll use are already developed and tested by the theory people. You'll find yourself that while your papers may not easily get accepted in conferences like NIPS, ICML or COLT (though they do have application tracks), they might be accepted in things like IROS (for the robotics people) or SIGGRAPH (For Computer Vision). And while your insight of the algorithms might be less than perfect, you'll know a lot of your specific application.

If, however, you see machine learning as an end, and want your graduate thesis to be an extension over an existing work, or an entirely new algorithm - then you'll have to study hard math. You'll have to read dense books, such as convex optimization, learning theory, computational complexity , game theory, etc. Reading this books will give you an insight on how to create a new algorithm and will also allow you to understand how the algorithms are really working. By the end, you'll have an algorithm that might be applied to all kinds of different problems, yet you'll probably will focus in a very shallow problem to confirm your expectations. Remember Nash wasn't even aware his equilibrium could be used in so many applications before they told him.

All of this is advice to new grad students, some tenured professors like Mike Jordan and Alex Smola are behemoths in application and theory, and have accepted papers in both kinds of conferences and journals. And serious Machine Learning Professors have a really good grasp on the applications and theory. But this is something you'll be able to do only after long years as an academic.

So going back to our kinect example, the algorithm people probably created the human detection algorithms - which can be used for a ton of different applications. And the application people were busy implementing those algorithms for the case of the Kinect, its sensors and the architecture of the processors.

Both approaches have their merits and advocates, you just have to be sure it is what you want to do.

In the next post we will discuss how to approach an application path, and after, we will discuss on how to pursue an algorithm path.

Take Care

Remember to visit my website
And my twitter feed @leonpalafox

Wednesday, July 20, 2011

[Special Edition Post] Tablets and research

Disclaimer: This post is mostly about my personal opinions on current technologies for studying, not an actual pragmatic advice on how to do it.

Some time ago, a friend and I got in a heated discussion. It was about the need of a laptop computer in an MD course. She insisted a laptop was necessary for an MD course. I , of course, disagreed and pointed out that if that were truth, physicians until now were either wizards or time travelers.

Today I asked myself the same question. Are tablets necessary for a Machine Learning researcher? In the ACML 2010 I saw a couple of researchers with iPads, my previous professor bought one himself. And in the MLSS in Singapore, more than 30% of the people had a tablet.  I indeed saw the practicality, my netbook seemed too bulky and bothersome to use while just reading papers and following slides.

After pondering a lot, I bought a Motorola Xoom, I did this because I needed a way to read journal papers and ebooks on the train without carrying 5000 pages in my bag.  I did not choose a Kindle because as far as I saw, the small version was less than useless to read journal papers and math ebooks, and the DX was almost the same price as a normal tablet. (I got my Xoom for less than 200 USD)

I can say that it has helped me a lot, I can read my papers wherever I go, and I always have them with me, I do not have to worry about printing them anymore, or underlining the reference to look it up later, since I have a 3G->Wi-Fi converter (another advantage against the Kindle)

The reason I did not choose an iPad was that, as an iPhone user, I find the iOS too restrictive to do real work. I have yet to find a way to import PDF's to an iPad without using iTunes. Call me old-fashioned but nothing beats a good old plug and play and just copying and pasting your files.  The fact that I have access to the filesystem of the device is another plus.

And here comes the question? Is then a tablet necessary to do good Machine Learning research?

It is a great help, and someone with a tablet does have a clear advantage against someone who doesn't, but then again, I stand with my earlier point, it is not necessary. Most of the greatest work on ML has been done so far without the help of a tablet, and I'm pretty sure it'll keep being that way for many years to come.

Tablet are still a long shot from being the perfect form of paper reading. Their lack of support for precise stylus-like devices is a bother (I love to make notes on my papers). And the slow response of most of them is still something that dampens your productivity.

I'll probably keep buying my math books, but for a quick commuting refreshing, or only if I wish to stay sharp on a particular topic by surveying some papers I think a tablet is an unbeatable companion.

Thank You and see you later

Remember to visit my website
And my twitter feed @leonpalafox

Friday, July 1, 2011

Where should I start, what should I do?

So you are all set in a Machine Learning Grad Course (I'll leave the admission niceties to you, since they change exponentially from country to country)

If you're lucky and have a good adviser, you'll probably have a project right away, but if not?

A lot of students have the feeling that they are alone, stranded and unwanted. And Machine Learning is no exception. Sometimes you won't really know where to start looking. Even if you have a project, to actually start doing things may take you some time.

In case you do not have a project, try looking around for what people are doing in you laboratory. It's always a good idea to try to work with someone, since you'll have feedback and a sense of commitment to other person. These simple things will help you progress in your research.

You can always go with your professor and see what he's working in (remember I told you it was important to have an active researcher as a professor) and offer your help. Even coding simple things are a great help for him, and give you a pretty good insight on advanced work and which problems need solution.

You should also pick up basic books on the topics you have interest in. A very good introductory book to the different areas of ML is Bishop's Book, (Be aware that you'll need a good background of Linear Algebra, Probability and Calculus to grasp most of the contents.). In a future post we will put a detailed list of which books may help you in your research.

Try also to look for the most recent conferences in a topic you like, see what the world is working on, and what unsolved problems are there. If you're lucky, your professor may pay for you to go to some of these conferences, even if you have no accepted papers.

But do you want to solve fundamental problems, or do you want to solve technical problems? Different problems have different sources.

There is another thing to post next time for choosing your research. Do you want to apply Machine Learning, or do you want to develop ML algorithms?

See you next time.

Don't forget to pay a visit to my webpage and leave some comments here.