Explicit 'using_shard' in Sharded Associations

[adomokos]

Something we enjoyed in Octopus was the seamless AR associations once you made it into the shard.

For example, this worked for us:

new_account = Account.using_shard(:mysql_group, :shard1).first
expect(new_account.users.first).to be == new_user

However, with rails-sharding, the shard has to be specified like this:

new_account = Account.using_shard(:mysql_group, :shard1).first
# See the using_shard(:mysql_group, :shard1) call on the users collection
expect(new_account.users.using_shard(:mysql_group, :shard1).first).to be == new_user

Would it be hard to improve this logic and "tag" the users AR relation with the currently used shard, so the explicit using_shard(:mysql_group, :shard1) would not be needed?

[hsgubert]

I don't think it would be too hard, but it is also not trivial. I didn't put much effort into that because I basically use the block syntax on my app.

Just as a general notice, extending AR scopes can be tricky, and in fact, if you take a look at the specs, I was not able to make using_shard(:group, :shard) work for associations, as Rails tries to access the database to get the model column names before the using_shard method is even called.

In this case, the solution would be to create two volatile instance attributes on the model, when Rails::Sharding::ActiveRecordExtensions is included, and then, before returning the final result of a query here, we would have to mark the object(s) with the shard group and shard they came from. This solves the first half of the problem.

The other half is to intercept all updates and destroys. They all go through _update_row and _delete_row. You would probably want to intercept reload calls as well, so models can be reloaded from the correct shard.

I think the bad side of a tighter integration with AR is that the gem tends to break more often. These internal interfaces are not meant to be used from outside, so it is normal for them to change even on minor version updates of AR.

Have you seen how this is implemented on Octopus? Perhaps they found another way of doing this.