-
Notifications
You must be signed in to change notification settings - Fork 34
Open
Labels
Description
Currently, once you construct a Parseset object from a training file, you can't reuse it with a different file. This means that when parsing thousands of files, you'll create thousands of new Parseset objects with their own TensorFlow graphs, each only being used once. This could slow down parsing, and if for some reason they don't get garbage collected it could result in memory overflow.
Solution: only construct one Parseset when parsing files, and let the model reuse it by deleting the bucket data without overwriting the bucket TensorFlow objects.