Algorithms and methods for attribute/feature selection?

views:

139

answers:

+3 Q:

Algorithms and methods for attribute/feature selection?

I have data with continuous class and I'm searching for good methods to reduce number of attributes. Now I'm using correlation based filters, random forests and Gram–Schmidt algorithm.

What I want to achieve is answer which attributes are more important/relevant to class attribute than others.

By using methods that I mentioned before I can reach this goal, but is there any other good algorithms worth noticing?

Principal Component Analysis is a fairly common technique used to reduce the dimension of sampled data. You can find a very good implementation in R.

Il-Bhima 2010-07-10 18:28:30

Of course PCA reduces number of dimensions but also looses information about original attributes - it doesn't provide answer which attributes are 'better' than others.

pixel 2010-07-10 19:44:06

In PCA you can look at the composition of components, so you can see what attributes were mainly used for creating first PCs. Nevertheless it is useless for data with complex or subtle dependencies.

mbq 2010-07-10 21:50:09

The Task view on Machine Learning and Statistical Learning is a good starting point for question like this.

Dirk Eddelbuettel 2010-07-10 18:56:57

My heart will be always with RF, but still you may take a look at Rough Sets. Especially LERS works quite good in case of massively disturbed data.
You may also try with importance obtained from other classifiers, like SVMs or Random Naive Bayes.

mbq 2010-07-10 20:00:16

Actually my data is rather small (~100 examples) and classifier is a result of using PCA (that's why it's continuous). Again can I use LERS, SVM or RNB with continuous classifier?

pixel 2010-07-10 20:37:53

@pixel For SVM certainly (it is called SVM regression), for RNB there might not be an implementation, but it can be done as in RF. If not, and with LERS you may just partition the decision into a discrete groups -- it is a common trick and will be sufficient for feature selection.

mbq 2010-07-10 21:43:49

ansaurus

tags:

views:

answers:

Algorithms and methods for attribute/feature selection?

related questions