EM-Synchrony doesn't work with Enumerator

[prepor]

Hello! Simple case:

require 'em-synchrony'
EM.synchrony do
  start = Time.now
  enum = Enumerator.new do |y|
    3.times do |i|
      EM::Synchrony.sleep 100
      y.yield i
    end
  end
  puts enum.next
  puts "Time: #{Time.now - start}"
  EM.stop
end

And it will return value (nil) instantly, because Enumerator uses Fibers internally and we have yielded from its fiber in EM::Synchrony.sleep.

[igrigorik]

Hmmm, wow.. I am really surprised this hasn't show up before! Nice one :-)

This seems to do the trick for me:

require 'em-synchrony'

EM.synchrony do
  start = Time.now

  enum = Enumerator.new do |y|
    3.times do |i|
        y << i
      end
  end

  enum.each do |v|
    puts v
    EM::Synchrony.sleep 1
  end

  puts "Time: #{Time.now - start}"
  EM.stop
end

# $> ruby test.rb 
# 0
# 1
# 2
# Time: 3.001568

[fl00r]

The actual problem was with slow IO in Enumerator. The usecase was something like

enumerator = Enumerator.new do |enum|
  ActiveRecordModel.some.query.each do |item|
    enum.yield item.title
  end
end

EM::Synchrony::FiberIterator.new(enumerator, 10).each do |title|
  # processing
end

And in EM::Iterator next is called on enumerators, so it fails

[prepor]

In fact, there are more simple trick:

require 'em-synchrony'
EM.synchrony do
  start = Time.now
  enum = Enumerator.new do |y|
    3.times do |i|
      EM::Synchrony.sleep 1
      y.yield i
    end
  end
  enum.each { |i| puts i }
  puts "Time: #{Time.now - start}"
  EM.stop
end

So, rule is sound like this: "do not use em-sync in body of enumerator with #next"

[funny-falcon]

https://gist.github.com/2023354

[prepor]

Another way:

ROOT = Fiber.current

EM.run do
  Fiber.new do
    en = Enumerator.new do |y|
      3.times do |i|
        f = Fiber.current
        EM.add_timer(1) { f.transfer }
        ROOT.transfer
        y << i
      end
    end
    puts en.next
    EM.stop
  end.resume
end

Fiber.yield in EM.Synchrony is always "ok, we can't do anything more in this fiber and must wait callback", so always transfer root fiber (with event loop) is ok, i think.

[igrigorik]

@funny-falcon's patch seems to do the trick.. but I'm having a hard time trying to figure out the actual patch in there :-)

[igrigorik]

So, do we want to merge any of this into master? If not, I'll close the bug.

[prepor]

But bug are exist, but it is impossible to fix it in current EM-Syncrhony correctly.