Want to know how to load word embeding vectors from fastText as a matrix in R with rownames as the words

[TangWenYing]

Hello! I am a student from China. Recently, I am trying to do some research about Japan. I want to use the CMDist, but there is a problem that the matrix of word embeding vectors i used has a different structure from yours, so CMDist did not work. The next step, I will try to use the word embeding vectors of Japanese from fastText, but I don't know how to deal with it so that it could be loaded as a matrix in R with rownames as the words.
Could you teach me how to do it? Thanks a lot if you can help me!

[dustinstoltz]

Hello! Happy to help.

First, for the embeddings you are having a problem with, can you provide the output to str(your_embeddings)?

Second, as for preparing the fastText embeddings, I suggest starting with this package: https://github.com/statsmaths/fasttextM/

remote::install_github("statsmaths/fasttextM")
library(fasttextM)

This will download and prepare the model for japanese:

ft_download_model("ja", mb = 500)

"ja" is the language code for Japanese and if you want a more words, you will increase the "mb". The model will be saved locally as an Rds in "fasttextM/extdata"

Then, this will load the full model into memory:

ft_load_model("ja")

Then to get just the word vectors you need into your R session:

vocab <- colnames(dtm)
wv <- ft_embed(vocab, lang = "ja")

If you run into further problems, please provide some of the output to help diagnose the issue.

-Dustin

[TangWenYing]

thanks!!! You are so kind. I will try it.

[gaborcsardi]

Hi, this is a read only mirror of CRAN, please see the package authors in the DESCRIPTION file. Look for Maintainer, BugReports and URL. Thanks!