FastText - Cannot load model.bin due to C++ extension failed to allocate the memory

Question

I'm trying to use the FastText Python API https://pypi.python.org/pypi/fasttext Although, from what I've read, this API can't load the newer .bin model files at https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md as suggested in https://github.com/salestock/fastText.py/issues/115

I've tried everything that is suggested at that issue, and furthermore https://github.com/Kyubyong/wordvectors doesn't have the .bin for English, otherwise the problem would be solved. Does anyone know of a work-around for this?

Kenosis

I'm having the same problem. Have you managed to solve it? – Maturity

Answer 1

Try the following steps to solve the issue. Its worked with me for Python 3.5 while loading a language identification model.

git clone https://github.com/facebookresearch/fastText.git
cd fastText
pip install .

Narcosis

Answer 2

You can't use the package fasttext installed by pip. You use install fastText (capital T) from fasttext repo.

Atropos

Answer 3

This error generally comes because of the version mismatch. Make sure you are using the same version of fasttext during the training and model loading time.

Scope

Answer 4

The package you are looking for is not on PyPI yet. You have to install it yourself by cloning the original repo and running the install script.

There is, however, an up-to-date version in PyPi Test which you can test and use.

You can also use this mirror, which has a copy of the official package you are looking for.

Polonium

Answer 5

you may try pyfasttext instead:

!pip3 install pyfasttext
from pyfasttext import FastText
model = FastText("./model_position")

works, but some compatibility problems, e.g. you may use model.args["dim"] to replace get_dimension()

Interstratify