.arff files with scikit-learn?

Question

I would like to use an Attribute-Relation File Format with scikit-learn to do some NLP task, is this possible? How can use an .arff file with scikit-learn?

Uppercase

Answer 1

I really recommend liac-arff. It doesn't load directly to numpy, but the conversion is simple:

import arff, numpy as np
dataset = arff.load(open('mydataset.arff', 'rb'))
data = np.array(dataset['data'])

Chromonema

Thanks for the feedback. Any idea of how can i use this conversation to classify?. – Uppercase

I all ready know that for instance with SVM the basic idea for classification is:from sklearn import svm s = svm.SVC() lables = [label1, label2] s.fit(training_data, labels) How could i present an arff file to a classification algorithm? – Uppercase

Were you able parse TF-IDF scores from .arff file and use it in sklearn? – Procora

I get the following exception when loading an arff file using liac-arff: BadAttributeType: Bad @ATTRIBUTE type, at line 21. I think the reason is the presence of "relational" attributes in my arff file. Has anyone a solution? Thank you. – Bellinzona

Answer 2

I found that scipy has a loader for arff files loadarff() to load them as numpy record arrays. I am not 100% sure that those arrays are suitable for direct consumption by scikit-learn but that should get your started.

Benthamism

Do you think i'll need to parse those numpy arrays?... What kind of preprocess would i need to do in order to feed some classification algorithm in scikit-learn? – Uppercase

Those looking for code to use scipy import scipy.io.arff as arff data = arff.loadarff(open(file_path,'rt')) – Scrumptious

Answer 3

Solution with scipy.arff

Code:

from scipy.io import arff
import pandas as pd

data = arff.loadarff('file.arff')
df = pd.DataFrame(data[0])
df.head()

Hoe

This is the preferred solution if you don't want additional dependencies. +1 – Goeger

Answer 4

Follow renatopp's answer: assume your data is the iris dataset, there should be 5 dimensional with last one is the class label column.

s = svm.SVC()
data_input = data[:,0:4]
labels = data[:,4] # this is the class column
s.fit(data_input, labels)

I think this is something you want.

Humor

Answer 5

If your "arff" file is a text file, try the following code instead:

import arff, numpy as np
dataset = arff.loads(open('mydataset.arff', 'rt'))
data = np.array(dataset['data'])

Sheatfish