Skip to content

Add feed merging and diffing #47

@cardmagic

Description

@cardmagic

Summary

Add methods to merge multiple feeds and diff feeds to find new/removed items.

Proposed Features

merge(*feeds)

Combine multiple feeds, deduplicating by guid or link:

def merge(*feeds)
  all_items = [items, *feeds.map(&:items)].flatten
  
  seen = Set.new
  merged_items = all_items.select do |item|
    key = item[:guid] || item[:id] || item[:link]
    next false if key.nil? || seen.include?(key)
    seen.add(key)
    true
  end
  
  # Sort by date, newest first
  merged_items.sort_by! { |i| i[:pubDate] || i[:updated] || Time.at(0) }.reverse!
  
  # Return new SimpleRSS-like object or just the array
  merged_items
end

Usage:

feed1 = SimpleRSS.parse(source1)
feed2 = SimpleRSS.parse(source2)

combined = feed1.merge(feed2)
# or
combined = SimpleRSS.merge(feed1, feed2, feed3)

diff(other)

Find differences between two versions of a feed:

def diff(other)
  my_keys = items.map { |i| i[:guid] || i[:id] || i[:link] }.compact
  other_keys = other.items.map { |i| i[:guid] || i[:id] || i[:link] }.compact
  
  {
    added: other.items.select { |i| !my_keys.include?(i[:guid] || i[:id] || i[:link]) },
    removed: items.select { |i| !other_keys.include?(i[:guid] || i[:id] || i[:link]) }
  }
end

Usage:

old_feed = SimpleRSS.parse(cached_source)
new_feed = SimpleRSS.parse(fresh_source)

changes = old_feed.diff(new_feed)
changes[:added]    # New items
changes[:removed]  # Items no longer in feed

Deduplication helper

def dedupe
  seen = Set.new
  @items = items.select do |item|
    key = item[:guid] || item[:id] || item[:link]
    next true if key.nil?  # Keep items without identifiers
    next false if seen.include?(key)
    seen.add(key)
    true
  end
  self
end

Benefits

  • Aggregate multiple feeds into one stream
  • Track new items for notifications/alerts
  • Handle feeds with duplicate entries
  • Useful for feed reader applications

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions