Same " Political Donors " data example, but using the euclidean_inner_join() Fx...

[sfd99]

Hi Beniamino,

Thanks for the zoomerjoin PKG.

Tried your very clear
" Political Donors " data EXAMPLE,
using the
jaccard_inner_join( ) Fx.

Really, really fast!. :-)

Could you please show
an R code Example
with the =same= " Political Donors " data,
but using the
euclidean_inner_join( ) Fx ?.

Thanks - very highly appreciated...

SFd99
San Francisco
latest zoomerjoin PKG v0.1.0
latest R, Rstudio, Ubuntu Linux 20.04

[beniaminogreen]

Thanks for your kind words about the package. I am not sure that I could create an example in the package documentation. The chief difficulty is that the Euclidean distance is defined for two or more points in space, while the Jaccard similarity is defined for sets.

To use the euclidean join functions with text data, you would have to encode the text fields into vectors. One way to do this would be with a neural network which can create vector embeddings that group similar text fields together this has given me some good results in the past, but you have to deploy your own network, or use one that is made available through an API.

OpenAI has an API that you can use to generate embeddings for text fields. I have used this with some success to join observations between datasets based on semantic similarity, which has the advantage that it can link strings that are dissimilar but semantically related (such as "tea party" and "the republican party").

I can't find a way to make a reproducible example of this at the moment, but I might write an article to advertise this functionality if I can find a compelling and simple example in the future.

Best,
Ben

[sfd99]

Thanks Ben
for the very clear explanation.

Yes, i now see.
We are comparing distances
in Sets,
not Points...

So,
the jaccard_inner_join( )
is really the appropriate Fx
for the " Political Donors " data EXAMPLE.

Have a great week!.
SFd99

[beniaminogreen]

Happy to help! I'll make sure to add a warning in the next few days to flag if a user is trying to use the Euclidean / Jaccard functions with the wrong datatype, just to avoid confusion in the future.