Skip to content

Updated to latest gems and tested with cassandra 0.7 #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cirode wants to merge 21 commits into
base: master
Choose a base branch
from
Open

Updated to latest gems and tested with cassandra 0.7 #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
ae4946f
Replaced returning() with tap() to avoid deprecation warning message.
Jul 1, 2011
963a750
got the tests running
Jul 7, 2011
b114dc9
fixed two bugs to do with old style assertion mapping
cirode Jul 7, 2011
c523a53
Fix: get_range() was expected to return just keys, while it is return…
Jul 7, 2011
f9b0493
removing the data from the vendored cassandra
cirode Jul 7, 2011
6436b59
removed the vendored cassandra within vendor cassandra
cirode Jul 7, 2011
a528103
ignoring the cassandra data
cirode Jul 7, 2011
b2c7380
modified the cassndra log to not create many subdirectories
cirode Jul 7, 2011
842a24b
removed the cassandra logs as well
cirode Jul 7, 2011
aac0c35
added some development notes on testing
cirode Jul 7, 2011
56191e6
Added clear! method to clear non-reversed associations.
Jul 12, 2011
da1c9b7
updated the gems to the latest and reflected in gemspec
cirode Jul 12, 2011
2dc6b72
Added count() method for base class, to return the number of items st…
Jul 20, 2011
ca1ff7d
Fix: the :boolean attribute type was not supported.
Jul 21, 2011
dbb9125
The :natural key now supports methods in the list of attributes.
Aug 11, 2011
d6ecda8
Updated cassandra gem to latest version (0.12.1).
Sep 9, 2011
afe2794
Made sure we always call get_range with a :key_count parameter. Other…
Sep 9, 2011
c9b0490
Updated cassandra gem dependency to version 0.12.0.
Oct 18, 2011
ca3717d
doesn't require :limit to be passed in order to iterate through all o…
Nov 28, 2011
03ed5e1
Updated activesupport and activemodel dependencies to 3.1.x.
Feb 29, 2012
a2f8af8
Added .rvmrc to gitignore.
Feb 29, 2012
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,7 @@ doc/
bin/*
vendor/gems/*
!vendor/gems/cache/
vendor/cassandra-0.7/data/*
vendor/cassandra-0.7/log/*
.bundle
.rvmrc
8 changes: 2 additions & 6 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
source :gemcutter

gem "activesupport", "3.0.0.beta3"
gem "activemodel", "3.0.0.beta3"

gem "cassandra", '0.8.2'
gem "shoulda"
gem "nokogiri"
gem 'ruby-debug19', :require => 'ruby-debug'
gemspec
59 changes: 59 additions & 0 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
PATH
remote: .
specs:
cassandra_object (0.6.0.pre)
activemodel (~> 3.1.0)
activesupport (~> 3.1.0)
cassandra (~> 0.12.0)
nokogiri
shoulda
thrift_client (~> 0.7.1)
tzinfo (~> 0.3.29)

GEM
remote: http://rubygems.org/
specs:
activemodel (3.1.3)
activesupport (= 3.1.3)
builder (~> 3.0.0)
i18n (~> 0.6)
activesupport (3.1.3)
multi_json (~> 1.0)
archive-tar-minitar (0.5.2)
builder (3.0.0)
cassandra (0.12.1)
json
rake
simple_uuid (>= 0.2.0)
thrift_client (>= 0.7.0)
columnize (0.3.4)
i18n (0.6.0)
json (1.6.1)
linecache19 (0.5.12)
ruby_core_source (>= 0.1.4)
multi_json (1.1.0)
nokogiri (1.5.0)
rake (0.9.2)
ruby-debug-base19 (0.11.25)
columnize (>= 0.3.1)
linecache19 (>= 0.5.11)
ruby_core_source (>= 0.1.4)
ruby-debug19 (0.11.6)
columnize (>= 0.3.1)
linecache19 (>= 0.5.11)
ruby-debug-base19 (>= 0.11.19)
ruby_core_source (0.1.5)
archive-tar-minitar (>= 0.5.2)
shoulda (2.11.3)
simple_uuid (0.2.0)
thrift (0.7.0)
thrift_client (0.7.1)
thrift (~> 0.7.0)
tzinfo (0.3.29)

PLATFORMS
ruby

DEPENDENCIES
cassandra_object!
ruby-debug19
7 changes: 7 additions & 0 deletions README.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,10 @@ Only if you're looking to help out with the development, there are a bunch of ro
## Why do you use a superclass and not a module.

Because.

##Development Notes
In order to facilitate testing, we have included a vendorised version of cassandra. This must be running with a valid test schema before running the tests

cd vendor/cassandra-0.7/
bin/cassandra
bin/cassandra-cli -host localhost --file conf/schema-sample.txt
10 changes: 7 additions & 3 deletions cassandra_object.gemspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,13 @@ Gem::Specification.new do |s|
s.summary = %q{Maps your objects into cassandra.}
s.homepage = %q{http://github.com/NZKoz/cassandra_object}

s.add_dependency('activesupport', '>= 3.0.0.beta3')
s.add_dependency('activemodel', '>= 3.0.0.beta3')
s.add_dependency('cassandra', '>= 0.8.2')
s.add_dependency('activesupport', '~>3.1.0')
s.add_dependency('activemodel', '~>3.1.0')
s.add_dependency('tzinfo', '~>0.3.29')
s.add_dependency('cassandra', '~> 0.12.0')
s.add_dependency('thrift_client', '~> 0.7.1')
s.add_dependency('shoulda')
s.add_dependency('nokogiri')

s.files = Dir['lib/**/*'] + Dir["vendor/**/*"]
s.require_path = 'lib'
Expand Down
1 change: 1 addition & 0 deletions lib/cassandra_object.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
require 'i18n'
require 'active_support'
require 'active_support/version'
require 'tzinfo'

module CassandraObject
class << self
Expand Down
19 changes: 15 additions & 4 deletions lib/cassandra_object/associations/one_to_many.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
module CassandraObject
class AssociationClearError < StandardError; end

class OneToManyAssociation
def initialize(association_name, owner_class, options)
@association_name = association_name.to_s
Expand All @@ -12,7 +14,7 @@ def initialize(association_name, owner_class, options)
def find(owner, options = {})
reversed = options.has_key?(:reversed) ? options[:reversed] : reversed?
cursor = CassandraObject::Cursor.new(target_class, column_family, owner.key.to_s, @association_name, :start_after => options[:start_after], :reversed => reversed)
cursor.find(options[:limit] || 100)
cursor.find(options[:limit])
end

def add(owner, record, set_inverse = true)
Expand All @@ -22,6 +24,11 @@ def add(owner, record, set_inverse = true)
end
end

def clear!(owner)
raise AssociationClearError, "Cannot clear reversed association #{@association_name}" if reversed?
connection.remove(column_family, owner.key.to_s, @association_name)
end

def new_key
SimpleUUID::UUID.new
end
Expand Down Expand Up @@ -83,7 +90,7 @@ def each
def <<(record)
@association.add(@owner, record)
if loaded?
@target << record
target << record
end
end

Expand All @@ -107,15 +114,15 @@ def all(options = {})
# @return [CassandraObject::Base] the newly created object
#
def create(attributes)
returning @association.target_class.create(attributes) do |record|
@association.target_class.create(attributes).tap do |record|
if record.valid?
self << record
end
end
end

def create!(attributes)
returning @association.target_class.create!(attributes) do |record|
@association.target_class.create!(attributes).tap do |record|
self << record
end
end
Expand All @@ -132,5 +139,9 @@ def target
def loaded?
defined?(@loaded) && @loaded
end

def clear!
@association.clear!(@owner)
end
end
end
2 changes: 1 addition & 1 deletion lib/cassandra_object/attributes.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ module Attributes

module ClassMethods
def attribute(name, options)

unless type_mapping = attribute_types[options[:type]]
type_mapping = { :expected_type => options[:type],
:converter => options[:converter] }.with_indifferent_access
Expand Down
17 changes: 0 additions & 17 deletions lib/cassandra_object/base.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,23 +56,6 @@ def self.lookup_ancestors

extend ActiveModel::Naming

module ConfigurationDumper
def storage_config_xml
subclasses.map(&:constantize).map(&:column_family_configuration).flatten.map do |config|
config_to_xml(config)
end.join("\n")
end

def config_to_xml(config)
xml = "<ColumnFamily "
config.each do |(attr_name, attr_value)|
xml << " #{attr_name}=\"#{attr_value}\""
end
xml << " />"
xml
end
end
extend ConfigurationDumper

include Callbacks
include Identity
Expand Down
34 changes: 20 additions & 14 deletions lib/cassandra_object/cursor.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ def initialize(target_class, column_family, key, super_column, options={})
@validators = []
end

def find(number_to_find)
limit = number_to_find
def find(number_to_find=nil)
limit = number_to_find || 100
objects = CassandraObject::Collection.new
out_of_keys = false

Expand All @@ -20,32 +20,39 @@ def find(number_to_find)
start_with = nil
end

while objects.size < number_to_find && !out_of_keys
while !out_of_keys && (number_to_find.nil? || objects.size < number_to_find)
index_results = connection.get(@column_family, @key, @super_column, :count=>limit,
:start=>start_with,
:reversed=>@options[:reversed])




out_of_keys = index_results.size < limit

if !start_with.blank?
index_results.delete(start_with)
end

keys = index_results.keys
values = index_results.values

missing_keys = []

results = values.empty? ? {} : @target_class.multi_get(values)

results.each do |(key, result)|
if result.nil?
missing_keys << key
end
end

unless missing_keys.empty?
values_hash = {}
index_results.each do |key, value|
values_hash[value] = key
end
@target_class.multi_get(missing_keys, :quorum=>true).each do |(key, result)|
index_key = index_results.index(key)
index_key = values_hash[key]
if result.nil?
remove(index_key)
results.delete(key)
Expand All @@ -54,20 +61,19 @@ def find(number_to_find)
end
end
end

results.values.each do |o|
if @validators.all? {|v| v.call(o) }
if @validators.all? {|v| v.call(o)}
objects << o
else
remove(index_results.index(o.key))
remove(index_results[o.key])
end
end

start_with = objects.last_column_name = keys.last
limit = (number_to_find - results.size) + 1
limit = number_to_find.nil? ? 100 : (number_to_find - results.size) + 1

end

return objects
end

Expand All @@ -83,4 +89,4 @@ def validator(&validator)
@validators << validator
end
end
end
end
2 changes: 1 addition & 1 deletion lib/cassandra_object/identity.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def key(name_or_factory = :uuid, *options)
end

def next_key(object = nil)
returning(@key_factory.next_key(object)) do |key|
@key_factory.next_key(object).tap do |key|
raise "Keys may not be nil" if key.nil?
end
end
Expand Down
2 changes: 1 addition & 1 deletion lib/cassandra_object/identity/natural_key_factory.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ def initialize(options)
end

def next_key(object)
NaturalKey.new(attributes.map { |a| object.attributes[a.to_s] }.join(separator))
NaturalKey.new(attributes.map { |a| object.send(a) }.join(separator))
end

def parse(paramized_key)
Expand Down
2 changes: 1 addition & 1 deletion lib/cassandra_object/migrations.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def instantiate(key, attributes)
attrs
end

returning super(key, attributes) do |record|
super(key, attributes).tap do |record|
record.attributes_changed!(original_attributes.diff(attributes).keys)
end
end
Expand Down
18 changes: 10 additions & 8 deletions lib/cassandra_object/persistence.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ module Persistence
end

VALID_READ_CONSISTENCY_LEVELS = [:one, :quorum, :all]
VALID_WRITE_CONSISTENCY_LEVELS = VALID_READ_CONSISTENCY_LEVELS + [:zero]
VALID_WRITE_CONSISTENCY_LEVELS = VALID_READ_CONSISTENCY_LEVELS

module ClassMethods
def consistency_levels(levels)
Expand Down Expand Up @@ -59,10 +59,13 @@ def multi_get(keys, options = {})
def remove(key)
connection.remove(column_family, key.to_s, :consistency => write_consistency_for_thrift)
end

def count
connection.count_range(column_family)
end

def all(keyrange = ''..'', options = {})
results = connection.get_range(column_family, :start => keyrange.first, :finish => keyrange.last, :count=>(options[:limit] || 100))
keys = results.map(&:key)
keys = connection.get_range(column_family, :start_key => keyrange.first, :finish_key => keyrange.last, :batch_size => 100, :key_count => (options[:limit] || options[:key_count])).keys
keys.map {|key| get(key) }
end

Expand All @@ -71,13 +74,13 @@ def first(keyrange = ''..'', options = {})
end

def create(attributes)
returning new(attributes) do |object|
new(attributes).tap do |object|
object.save
end
end

def write(key, attributes, schema_version)
returning(key) do |key|
key.tap do |key|
connection.insert(column_family, key.to_s, encode_columns_hash(attributes, schema_version), :consistency => write_consistency_for_thrift)
end
end
Expand All @@ -86,7 +89,7 @@ def instantiate(key, attributes)
# remove any attributes we don't know about. we would do this earlier, but we want to make such
# attributes available to migrations
attributes.delete_if{|k,_| !model_attributes.keys.include?(k)}
returning allocate do |object|
allocate.tap do |object|
object.instance_variable_set("@schema_version", attributes.delete('schema_version'))
object.instance_variable_set("@key", parse_key(key))
object.instance_variable_set("@attributes", decode_columns_hash(attributes).with_indifferent_access)
Expand Down Expand Up @@ -130,7 +133,6 @@ def read_consistency_for_thrift

def consistency_for_thrift(consistency)
{
:zero => Cassandra::Consistency::ZERO,
:one => Cassandra::Consistency::ONE,
:quorum => Cassandra::Consistency::QUORUM,
:all => Cassandra::Consistency::ALL
Expand Down Expand Up @@ -190,4 +192,4 @@ def reload

end
end
end
end
Loading