I was going through Murphy's Machine Learning book to remember some ML concepts that I needed. While I had seen it really fast, and the sections I am most familiarized with are well written. I found out that the book feels very rushed. The online erratas are huge, and each reprinting just seems to have more.
These mistakes might be passable if you are familiarized with the material, but if you are learning it, and taking the equations stated there at face value you might have some issues.
Also, terminology is not clearly explained, and for the sake of saving space, he refers you back to the very first time he introduced it, which most of the time is one of the very first chapters. I kid you not, I had to go back three times to find out he never really explained one symbol in the equation.
I used mostly Bishop's book to start doing Machine Learning, and in contrast to Murphy's, it is a pretty self contained book, he goes extensively through doing a restatement of most of the nomenclature he uses through the book, which is useful if you don't feel like going back to the very beginning to figure out what is he talking about.
Some sections, like the sampling, is very well explained, and way better than in Bishop's, or any other book. However, the sampling examples assume that the reader is familiarized with his particular terminology, which just takes more time than it should.
But for an entry level Grad student, or even an undergraduate, the book is just not friendly enough, the code is not really well documented, so aside from reproducing the figures in the book, there is really not much aggregated value in having it available, most of the time you'll spend most of your time just figuring out what the code does, and since it is not cross referenced with the equations in the book, is not really tractable as a learning experience.
It also has its issues as a reference book, since as I told you, it does too much back tracking when it comes to equations.
Some good things, are its explanation of Boltzmann Machines, which Bishop just really lacks, mostly because it was written after the deep net boom.
Wednesday, July 30, 2014
Tuesday, October 22, 2013
Why should textbooks be free.
Since I've started studying, I noticed something really strange, textbooks and science books are surprisingly expensive.
Economically speaking, it just does not make any sense, the US government most likely is paying many of the grad students, postdocs and professors that are writing the book via grant money. The professor may or may not get an advance on the book, and the royalties he will get on the book are around 10% of the cost of the book. Which means that both Oppenheim and Willsky should get around 15 bucks for every book that is sold.
Some books that I would consider essential for an engineering undergrad, like Oppenheim and Willsky's book on Signals and Systems go well above 150 USD. I've even seen biological sciences books go for way more than that.
Furthermore, is unlikely you will ever use Oppenheim's book for the whole length of an undergraduate course, and unless you start doing research in that area, it is very likely you will never touch that book again in your entire life. There are always the possibility to rent the book, which goes for about 99 USD in Amazon.
Scientific books are thought to be expensive because there is a whole deal of research behind them, there are tons of money invested so the book can be written and a professor (who is already being paid) has to devote some time to write it. There is a proofreading process (which sometimes is done by undergrad students also being paid already)
Economically speaking, it just does not make any sense, the US government most likely is paying many of the grad students, postdocs and professors that are writing the book via grant money. The professor may or may not get an advance on the book, and the royalties he will get on the book are around 10% of the cost of the book. Which means that both Oppenheim and Willsky should get around 15 bucks for every book that is sold.
Also, is not like scientific books can earn you big bucks, mandatory books like S&S may get you good money, but most likely you won't see a lot of royalty money, especially for highly topical books in advance graduate courses.
Then the next question is: Why charge for it, originally it made sense, since printing was the only way to communicate new ideas and teach scientific ideas, and printing is overall an expensive process. However, the internet brought that down, if you are living in the internet age, and you are writing a book because you want to educate people, there is no good reason you cannot give your book for free. With 10% of royalties you are clearly not getting rich, and we can distribute a thousand copies with the click of a mouse.
You can always publish it, and expect someone will buy it in print (I know I still do sometimes), but I do believe is a researcher's duty to allow people to access freely to the contents of the book.
You can always publish it, and expect someone will buy it in print (I know I still do sometimes), but I do believe is a researcher's duty to allow people to access freely to the contents of the book.
Why? For one, the money to develop the knowledge that you use to write the book is most likely taxpayers' money. The money given to you so you have a hefty team of undergrads, grads and postdocs probably is also taxpayers money. And the fact that you have students going out of their way to write a book, might actually hurt them in their pursue of a graduate degree.
And finally, I do believe that as educator, the ultimate goal of the professor should be to pursue the education of as much people as they can reach. If their objective is to make money, they are probably in the wrong business anyway. The main question I like to make is: Do you care that people pay for your book, or do you care that people read your book? If the answer is the former, you probably do not care the IEEE and other printing houses charge 20 bucks for an 8 page article.
And finally, I do believe that as educator, the ultimate goal of the professor should be to pursue the education of as much people as they can reach. If their objective is to make money, they are probably in the wrong business anyway. The main question I like to make is: Do you care that people pay for your book, or do you care that people read your book? If the answer is the former, you probably do not care the IEEE and other printing houses charge 20 bucks for an 8 page article.
Luckily I'm not alone on this, and many great Machine Learning professors have made their books freely available in the internet. I do believe there is a possibility to get a great ML education based only on free books, although some of the best books are still not available for free download.
Thursday, May 23, 2013
Stationarity in ML Data (An EEG application)
A stationary process in statistic is one where the distribution does not change in time or space. Which in layman terms just means that if we measure the data today and we test it tomorrow, the underlying distribution must remain more or less the same.
This is a fundamental concept to Machine Learning. Stationarity in data allow us to train models today and expect them to work in data that we gather in the future. Sure there is a constant retraining of the models, but most of the models will assume that all the data comes from a stationary process. There is no point in modeling your data with some parameters today (mu and sigma if it is Gaussian) if you expect that tomorrow's parameters are going to be wildly different. However much of the data out there is non stationary.
Think about training a robot following a path based on images taken in spring, and then try to have the robot follow the same path in winter. We ran in this issues a lot when working in the Tsukuba Challenge Project, they allowed you to take the data in the summer, but the competition was in fall, when the trees have no leaves to show for.
Interesting problems like these arise in many other areas, like computer vision, where we would like to think that the objects we use to train are not rotated or transformed, when in reality they are. For a more extensive review of CV approaches to this, you can check LeCun's Convolutional Neural Networks.
Some of the most interesting problems in Neuroscience are also non stationary (we like to think they are, but they aren't). EEG readings that we do today are often affected by many environmental and subject conditions. Not to mentions that reading EEG from human scalp is not an exact science. The whole process tends to be messy, time consuming and difficult to replicate.
One cool approach to deal with this is to transform the data in such a way that you can obtain non invariant features of the data. For example, if you train a CV vision system, you could extract these features from objects, instead of measuring size and color of a circle you could measure it's radius and circumference and if the parameters follow the circle's circumference equation, you could assure it was a circle. Convolutional NN do something of sorts with input images. You could also do an extensive training, which means to train with every possible transformation of the data.
In EEG we try to use frequency analysis, since it tends to be more reliable than the simple time series (in theory). A recent paper in the MIT Journal of Neurocomputation has a great introduction on this topic, and how non stationarity is attacked using things like stationary feature extraction and adaptive model algorithms.
Stationary feature extraction is the jargon for what I described before with CV, many people use things like Common Spatial Patterns that tend to remove all of the EEG non stationarity and leave us with nice stationary features to use with our favorite ML algorithm.
Adaptive model algorithms are those that change the algorithms' parameters in subsequent recording sessions. As users get accustomed to have their EEG reading plotted in front of them, they also tend to learn how to control them better. And as such, adaptive algorithms are used to address this non stationarity. Think of it as a videogame that learns your behavior as you get better playing it, and can react better to your inputs.
The approach in the aforementioned paper is interesting in the sense that they used a really simple statistical concept, the Kullback-Leibler (KL) divergence, which is a fancy term for a measure of how the difference between different probability distributions.
They assume that if you have a training session done in day 1, and a small sample of data from day 2, you can use the KL divergence to measure how different the probabilities from day 1 and day 2 are, and create a linear transformation such that the information from day 1 is relevant to that you will obtain in day 2.
The rest of the paper goes on how to obtain these transformation matrices using different flavors of this approach, one where the labels of the day 2 are available, and one where they aren't.
The method looks eerily similar to what you would do in a State Model, where you try to approximate (predict) the values of the next state given the previous state's parameters and then do an update as you can read the data for the next state (Update).
The paper goes the safe route and assume both probabilities are gaussian, I could think in a nice extension where you approximate them to be gaussian but in reality you can have different probability distributions modeling the shape of the data. Using simple gaussian approximations that should not be so hard.
The paper is nice, but still in preprint, so you will need a university account to access it, and maybe a couple of tricks from your university's library.
This is a fundamental concept to Machine Learning. Stationarity in data allow us to train models today and expect them to work in data that we gather in the future. Sure there is a constant retraining of the models, but most of the models will assume that all the data comes from a stationary process. There is no point in modeling your data with some parameters today (mu and sigma if it is Gaussian) if you expect that tomorrow's parameters are going to be wildly different. However much of the data out there is non stationary.
Think about training a robot following a path based on images taken in spring, and then try to have the robot follow the same path in winter. We ran in this issues a lot when working in the Tsukuba Challenge Project, they allowed you to take the data in the summer, but the competition was in fall, when the trees have no leaves to show for.
Interesting problems like these arise in many other areas, like computer vision, where we would like to think that the objects we use to train are not rotated or transformed, when in reality they are. For a more extensive review of CV approaches to this, you can check LeCun's Convolutional Neural Networks.
Some of the most interesting problems in Neuroscience are also non stationary (we like to think they are, but they aren't). EEG readings that we do today are often affected by many environmental and subject conditions. Not to mentions that reading EEG from human scalp is not an exact science. The whole process tends to be messy, time consuming and difficult to replicate.
One cool approach to deal with this is to transform the data in such a way that you can obtain non invariant features of the data. For example, if you train a CV vision system, you could extract these features from objects, instead of measuring size and color of a circle you could measure it's radius and circumference and if the parameters follow the circle's circumference equation, you could assure it was a circle. Convolutional NN do something of sorts with input images. You could also do an extensive training, which means to train with every possible transformation of the data.
In EEG we try to use frequency analysis, since it tends to be more reliable than the simple time series (in theory). A recent paper in the MIT Journal of Neurocomputation has a great introduction on this topic, and how non stationarity is attacked using things like stationary feature extraction and adaptive model algorithms.
Stationary feature extraction is the jargon for what I described before with CV, many people use things like Common Spatial Patterns that tend to remove all of the EEG non stationarity and leave us with nice stationary features to use with our favorite ML algorithm.
Adaptive model algorithms are those that change the algorithms' parameters in subsequent recording sessions. As users get accustomed to have their EEG reading plotted in front of them, they also tend to learn how to control them better. And as such, adaptive algorithms are used to address this non stationarity. Think of it as a videogame that learns your behavior as you get better playing it, and can react better to your inputs.
The approach in the aforementioned paper is interesting in the sense that they used a really simple statistical concept, the Kullback-Leibler (KL) divergence, which is a fancy term for a measure of how the difference between different probability distributions.
They assume that if you have a training session done in day 1, and a small sample of data from day 2, you can use the KL divergence to measure how different the probabilities from day 1 and day 2 are, and create a linear transformation such that the information from day 1 is relevant to that you will obtain in day 2.
The rest of the paper goes on how to obtain these transformation matrices using different flavors of this approach, one where the labels of the day 2 are available, and one where they aren't.
The method looks eerily similar to what you would do in a State Model, where you try to approximate (predict) the values of the next state given the previous state's parameters and then do an update as you can read the data for the next state (Update).
The paper goes the safe route and assume both probabilities are gaussian, I could think in a nice extension where you approximate them to be gaussian but in reality you can have different probability distributions modeling the shape of the data. Using simple gaussian approximations that should not be so hard.
The paper is nice, but still in preprint, so you will need a university account to access it, and maybe a couple of tricks from your university's library.
Thursday, September 6, 2012
I don't know about you, but to me it sounds like a ripoff
Have you ever stopped and pondered about why ink cartridges are so expensive?
The market explanation is that is with the cartridges that the companies get their profit, they are probably losing money with each printer they sell.
So now begs the question, why are Journals so expensive?
Have you tried downloading a paper from things like Elsevier and the IEEE without being connected to a network that has a license for any of them?
The standard price of your paper will be about $30(USD). Which again begs the question? Is it really that expensive to publish your paper?
Recently the IEEE solved my question, by telling me that I (and only I) have the possibility of making my paper free for the world by paying a unique fee of $3000 USD!!
Of course, your contract with the IEEE says that the exact camera-ready paper that you submitted is the one that they can make open access. If you change a single paragraph, you can actually upload it to your favorite server for null or almost null fees.
The IEEE says that price, the $3000, is to cover the publishing costs. So they have answered my question, the publishing costs are 3000 USD, because they wouldn't think on using my money to get a profit right?
So, what if I want to publish my own journal, the IEEE just told me that the price for each paper is $3000 USD, so publishing a single Volume with 15 papers should have a cost of 45,000 USD right?????
A single volume, would be equivalent to about a book of 120 pages, if each paper is 15 pages (because they publish on both sides)
So, there we have it, it costs $45,000 USD to publish a full Edition of a volume. Now, I do not know how many copies do they release per Volume, but lets do an estimate.
The market price of publishing your own book is about $5 USD (150 color pages) for a paperback,so 45,000 USD get you 9000 printed volumes.
And then we ask, do they publish 9000 volumes? Perhaps they do, I don't really know, and it's a number they don't seem to publish or make public. If each University receives printed copies of the Journals, they might well be over the 9000 printed volumes and their numbers would make sense. But on the other hand, if only the authors receive this copies, it's just a huge ripoff. Since I hardly think that the hosting costs of the papers are over (45,000 x 6 volumes per year) $270,000 USD.
Friday, August 10, 2012
Bad Reviews and Trolling
"You weren't smart enough two capitalize Christian
The cosmos exist, and they have an obvious design.
God exists."
-Random Comment Found in YouTube
I recently tweeted that "Bad Reviews are like Trolling", and I kept wondering that perhaps such a thought deserved a bit of explaining, and we might as well learn a bit on how to do a good review.When someone asks you to review a paper for a journal or a conference, they expect you'll devote an appropriate amount of time to read, analyze and review the paper. The thing is, people might get 5 or 6 of these every month, and it may increase in months when there is a conference looming. I have to say that I've receive most bad reviews than good ones, I'm not saying I got rejected, I'm saying reviews were lazy, ill written and obviously rushed. Like a Youtube comment, there are so many videos that demand your attention that you cannot bother on writing good comprehensive reviews for each of them. So a lot of people do what is commonly known as trolling, that is, they just give a negative comment without any suggestion or space to discuss.
- Bad grammar, trolling is obviously done without care, so there is no care in writing well either.
- Usually a negative comment without any suggestion or room for discussion.
- If they disagree it is based on a deep personal belief rather than a well informed and researched decision.
If you see these three points, a bad review of a paper usually has these same characteristics. Most reviewers won't even tell you what is wrong if they rejected it.
What is worse, most times, a reviewer is the pipeline worker and final judge on whether a paper is accepted or rejected for publication or a conference.It seems almost unfair that months of work get to be evaluated in a short burst by someone who might be unprepared or not willing to do the job.
But not everything is lost, I've seen great reviews, with constructive criticism, and always a chance to reply the comments. Not only that, they also are written in pristine and clear English, so the review itself is not confusing to the authors.Some suggestions you might like to follow when doing a review:
- If you attack the author's spelling, try to give concrete examples of what you think is a mistake, perhaps the authors do not consider it that way.
- If you attack the author's methodology, try suggesting a better one, and point out the errors or points you would change in what they are doing so far.
- If you attack the idea in general: Please do not do it, if at the end it was a good idea, you'll look foolish, ideas are too personal and a paper should never be rejected on the basis "I did not like your idea"
- If you attack the organization of the paper: Perhaps you could suggest a better way to organize it, and give a suggestion or two on why the paper should be organized in the way you are suggesting.
- If you have issues with the theory behind the paper, be clear to point out why aren't you convinced and point out references or proofs that the theory is wrong.
Remember that you are not in a review committee as an almighty god but rather as a humble quality control manager, your job is to see that the work is not "plagiarized", that the work makes scientific sense and that the work is readable to most of the audience. You are not an editor, so you do not get to impose your style of writing and you are not a scientific adviser, so you are not to impose your scientific ideas.
Friday, June 22, 2012
My last seminar and Non Parametrics for the Lay Man
In The University of Tokyo, we have to present 3 seminars through the course of our PhD's, the first one is a survey, the second one is your midterm evaluation and the last one is another survey.
They are also called Rinko (輪講), It roughly translates as reading (or discussion) circle.
The format of the seminar is the following:
Given this format, it is tricky to introduce to them a new topic like Non Parametric Bayesian methods . I had to decide either to spend my time trying to teach them the inner workings of things like the Dirichlet Process or the Gamma Process, and then try to explain how things like Naive Bayes or LDA benefit from this; or spend my time showing them some cool applications and areas where they could use it, prepare an easy reading paper, not to deep and with lots of citations for them to go look if they were interested.
Needless to say, I went with the last option, and here it is, my version of what I'd call Non Parametric Bayesian Methods for the layman.
This document is NOT TO LEARN Non Parametric Methods, but rather to see how can you use it and have a friendly introduction to the topic, I introduced basic things about DP and IBP, but I did not mention things like inference or Gibbs Sampling.
If you wish to learn DP or IBP, you can always go to the papers I cite. But I commend you not to use this as your main source of information, I know I wouldn't do it.
I would like to make this a living document, so if you have suggestions or ideas, I can always add them to the final paper, I left a ton of things out of the paper due to space constrains (8 pages). So send me an email or a tweet if you wish to add something, or point out a typo, I'm sure it's full of those. @leonpalafox
Paper
Slides
Note: These slides and document are free for you to use, distribute and modify as you wish, if you want to give me a little credit, just point out to here or my webpage, and it would be more than enough
Cheers
They are also called Rinko (輪講), It roughly translates as reading (or discussion) circle.
The format of the seminar is the following:
- Presenters: 3 Presenters (PhD and Master students)
- Audience: A room packed with about 70 students from different research groups, you have people from every background in Electric Engineering and CS, information, semiconductors, power systems, computer science, robotics. You also have your Prof, and usually, a couple other Professors who might or might not be related to your topic. They are the Prof. of the other 2 guys presenting with you. The administration tries to have Prof. on related fields.
- Time: 25 minutes to present your slides
- Materials: You have to present a Paper-Like document of at most 8 pages with your topic. Slides for your presentation
- Questions: 5-10 minutes at the end, either from the Prof or the Students.
Given this format, it is tricky to introduce to them a new topic like Non Parametric Bayesian methods . I had to decide either to spend my time trying to teach them the inner workings of things like the Dirichlet Process or the Gamma Process, and then try to explain how things like Naive Bayes or LDA benefit from this; or spend my time showing them some cool applications and areas where they could use it, prepare an easy reading paper, not to deep and with lots of citations for them to go look if they were interested.
Needless to say, I went with the last option, and here it is, my version of what I'd call Non Parametric Bayesian Methods for the layman.
This document is NOT TO LEARN Non Parametric Methods, but rather to see how can you use it and have a friendly introduction to the topic, I introduced basic things about DP and IBP, but I did not mention things like inference or Gibbs Sampling.
If you wish to learn DP or IBP, you can always go to the papers I cite. But I commend you not to use this as your main source of information, I know I wouldn't do it.
I would like to make this a living document, so if you have suggestions or ideas, I can always add them to the final paper, I left a ton of things out of the paper due to space constrains (8 pages). So send me an email or a tweet if you wish to add something, or point out a typo, I'm sure it's full of those. @leonpalafox
Paper
Slides
Note: These slides and document are free for you to use, distribute and modify as you wish, if you want to give me a little credit, just point out to here or my webpage, and it would be more than enough
Cheers
Monday, May 28, 2012
Reading List for the ICML 2012
The list of accepted papers for the ICML 2012 is out, and following some of my colleagues, I'll post the papers that at first hand cached my eye:
(Disclaimer: Since my research tends to be on nonparametric statistics, I tend to gravitate towards paper on those topics)
Gaussian Process Regression Networks
Andrew Wilson, David Knowles, Zoubin Ghahramani
(Disclaimer: Since my research tends to be on nonparametric statistics, I tend to gravitate towards paper on those topics)
Gaussian Process Regression Networks
Andrew Wilson, David Knowles, Zoubin Ghahramani
Abstract: We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the nonparametric flexibility of Gaussian processes. GPRN accommodates input (predictor) dependent signal and noise correlations between multiple output (response) variables, input dependent length-scales and amplitudes, and heavy-tailed predictive distributions. We derive both elliptical slice sampling and variational Bayes inference procedures for GPRN. We apply GPRN as a multiple output regression and multivariate volatility model, demonstrating substantially improved performance over eight popular multiple output (multi-task) Gaussian process models and three multivariate volatility models on real datasets, including a 1000 dimensional gene expression dataset.
Quick Opinion: Based on the abstract and a quick reading of the arxiv version, this sure looks like a nice variation for Gaussian Processes. And merging both Bayesian Neural Networks with GP seems both a good idea for specific problems like Gene Regulatory Networks Inference, given that some people have been using Recursive Neural Networks for such tasks.
Modeling Images using Transformed Indian Buffet Processes
KE ZHAI, Yuening Hu, Jordan Boyd-Graber, Sinead Williamson
Abstract: Latent feature models are attractive for image modeling; images generally contain multiple objects. However, many latent feature models ignore that objects can appear at different locations, or require pre-segmentation of images. While the transformed Indian buffet process (tIBP) provides a method for modeling transformation-invariant features in simple, unsegmented binary images, in its current form it is inappropriate for real images because of computational constraints and modeling assumptions. We combine the tIBP with likelihoods appropriate for real images. We also develop an efficient inference scheme using the cross-correlation between images and features that is both theoretically and empirically faster than existing inference techniques. We demonstrate that, using our method, we are able to discover reasonable components and achieve effective image reconstruction in natural images.
Quick Opinion: I could not find the pdf, so based on the Abstract, the paper seems pretty interesting, although I'm curious in which way did they extend tIBP using likelihoods.
Abstract: Bayesian models offer great flexibility for clustering applications—Bayesian nonparametrics can be used for modeling infinite mixtures, and hierarchical Bayesian models can be utilized for shared clusters across multiple data sets. For the most part, such flexibility is lacking in classical clustering methods such as k-means. In this paper, we revisit the k-means clustering algorithm from a Bayesian nonparametric viewpoint. Inspired by the asymptotic connection between k-means and mixtures of Gaussians, we show that a Gibbs sampling algorithm for the Dirichlet process mixture approaches a hard clustering algorithm in the limit, and further that the resulting algorithm monotonically minimizes an elegant underlying k-means-like clustering objective that includes a penalty for the number of clusters. We generalize this analysis to the case of clustering multiple data sets through a similar asymptotic argument with the hierarchical Dirichlet process. We also discuss further extensions that highlight the benefits of our analysis: i) a spectral relaxation involving thresholded eigenvectors, and ii) a normalized cut graph clustering algorithm that does not fix the number of clusters in the graph.
Quick Opinion: Based on the abstract and a quick reading of the arxiv version, this sure looks like a nice variation for Gaussian Processes. And merging both Bayesian Neural Networks with GP seems both a good idea for specific problems like Gene Regulatory Networks Inference, given that some people have been using Recursive Neural Networks for such tasks.
Modeling Images using Transformed Indian Buffet Processes
KE ZHAI, Yuening Hu, Jordan Boyd-Graber, Sinead Williamson
Abstract: Latent feature models are attractive for image modeling; images generally contain multiple objects. However, many latent feature models ignore that objects can appear at different locations, or require pre-segmentation of images. While the transformed Indian buffet process (tIBP) provides a method for modeling transformation-invariant features in simple, unsegmented binary images, in its current form it is inappropriate for real images because of computational constraints and modeling assumptions. We combine the tIBP with likelihoods appropriate for real images. We also develop an efficient inference scheme using the cross-correlation between images and features that is both theoretically and empirically faster than existing inference techniques. We demonstrate that, using our method, we are able to discover reasonable components and achieve effective image reconstruction in natural images.
Quick Opinion: I could not find the pdf, so based on the Abstract, the paper seems pretty interesting, although I'm curious in which way did they extend tIBP using likelihoods.
Abstract: Bayesian models offer great flexibility for clustering applications—Bayesian nonparametrics can be used for modeling infinite mixtures, and hierarchical Bayesian models can be utilized for shared clusters across multiple data sets. For the most part, such flexibility is lacking in classical clustering methods such as k-means. In this paper, we revisit the k-means clustering algorithm from a Bayesian nonparametric viewpoint. Inspired by the asymptotic connection between k-means and mixtures of Gaussians, we show that a Gibbs sampling algorithm for the Dirichlet process mixture approaches a hard clustering algorithm in the limit, and further that the resulting algorithm monotonically minimizes an elegant underlying k-means-like clustering objective that includes a penalty for the number of clusters. We generalize this analysis to the case of clustering multiple data sets through a similar asymptotic argument with the hierarchical Dirichlet process. We also discuss further extensions that highlight the benefits of our analysis: i) a spectral relaxation involving thresholded eigenvectors, and ii) a normalized cut graph clustering algorithm that does not fix the number of clusters in the graph.
Quick Opinion: I think this extension was something that was missing in ML, I'm very intrigued on this paper in particular, I remember reading on how K-means was a relaxation for Mixture of distributions with circular Gaussians.
Subscribe to:
Posts (Atom)