Intermittent exceptions when disconnecting a component and leaking memory

[dannyvanhoof]

We started using ViewComponents in combination with Motion to render orders in our application. We now somehow have the feeling (by diving into it with get_process_mem and rack_mini_profiler) that the memory allocated by these components is not released when the page is closed, so that our Rails application keeps on growing for days, to eventually be thrown out by Heroku.

Question: can this indeed be the case? And if yes: what can we do to somehow trigger cleanup?

[alecdotninja]

That is very strange. Motion does keep a cached copy of the component around in the channel, but that should be released by ActionCable when the subscription is removed by the client (This is done so that the client doesn't have to send the state back up to the server past the initial mounting).

Also, I wouldn't expect this object to be particularly large. A marshalled representation of it is in the client's markup, after all. It is usually on the order of hundreds of bytes.

@dannyvanhoof Why do you think that Motion is leaking memory?

[alecdotninja]

The only other longer-lived piece of state created by Motion (besides the configuration and a few service objects) that I can think of is a renderer for the websocket connection. This should only live as long as the client is connected though -- it is actually stored on the connection object itself.

[dannyvanhoof]

Alec - we're just in the early stage of our investigation. Since we started using ViewComponents + Motion, we saw our memory usage on Heroku rising, and were thrown out couple of times. Early this week I added the gem_process_mem gem to our application, and started looking at the memory usage (production) at the end of loading a page (after starting garbage collection manually), and again after closing the tab on the browser. That's when I noticed that loading a page containing VC + Motion increased the memory used, and it didn't go down after closing the tab. But clearly, this will need more investigation.

[dannyvanhoof]

Update: we checked the size of the marshalled objects on the client side, and for some of our pages we found like 240 VC's, each with 30Kb data. Instead of storing the object as instance variable in the VC, we now store the object's id. This reduced the data on the client side to 240 x 30 bytes. The page itself also loads much faster now, and we see a much smaller increase in memory. I guess I'll stop the investigation as for now. What I learned from this: don't store the object itself as instance variable - unless it's really very small...

[alecdotninja]

I think an intermittent failure that occurred on CI this morning might be related (unfortunately, I retried and that seems to have erased the logs). It was an exception from within ActionCable saying that it could not find a subscription to remove.

When I get some more time, I'll dive into that.

@dannyvanhoof State size is a known pain-point in Motion. We have plans to support server-side state with Redis or Memcached which would significantly cut down on the amount of data over the wire, but that has yet to be implemented.

[alecdotninja]

Ah, it happened again. I won't retry the job this time: https://travis-ci.com/github/unabridged/motion/jobs/383344606

[dannyvanhoof]

@dannyvanhoof State size is a known pain-point in Motion. We have plans to support server-side state with Redis or Memcached which would significantly cut down on the amount of data over the wire, but that has yet to be implemented.

Alec - wouldn't it be an idea to allow for instance variables that could be excluded from being marshalled? this way, we could use the original object to render the first time, but not marshall it, and marshalling the object's id instead. So a first rendering could use the original object (preventing an n+1), while an update would first have to retrieve that object.