Big multi label classifier on db

[francescoagati]

Hi,
Can multi label classifiers with big data writed to a SQL DB and streamed in classify phase? Or the classifier must work always in memory? Form big classifier is best split the data in many classificators, serialize it and calo sequentually or in parallel?

[erelsgl]

This is an interesting question.
I believe it can be done but requires some additions and adaptations.

[francescoagati]

and using many serializers and every serializer is trained with a split of data?

[Berkmann18]

Since the classifier can be serialized using serialization, I believe it's possible (assuming that the SQL DB in question does support JSON data).

Would you mind expanding on the big classifier bit?
If it's one classifier then you're probably better off using one serializer (unless the performance costs of splitting it into several JSON data is better).

[francescoagati]

yes but the problem is the ram. you must always load all the serialized json in ram

[Berkmann18]

Good point.