[RFC] Implement Ruby's Enumerator Class

I really like Ruby's Enumerator class and I think it would be great to have something similar in Crystal.
The Enumerator class can be used to implement generators.
Here's an example from the Ruby docs:
fib = Enumerator.new do |y|
  a = b = 1
  loop do
    y << a
    a, b = b, a + b
  end
end

p fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

The y block parameter in the example is called the yielder and the iterator returned from new blocks until the yielder receives a value or the block ends.
Since I was porting some Ruby code that depended on Enumerator to Crystal, I wen't ahead and implemented my own generic Enumerator class using the Iterator module and Channel to implement the concurrent behavior:
class Enumerator(T)
  include Iterator(T)

  def initialize(&block : Yielder(T) -> Nil)
    @channel = Channel(T).new
    @block = block
    @block_called = false
    spawn_block
  end

  def next
    @channel.receive
  rescue Channel::ClosedError
    stop
  end

  def rewind
    if @block_called
      @channel = Channel(T).new
      @block_called = false
      spawn_block
    end
    self
  end

  private def spawn_block : Nil
    spawn do
      begin
        @block.call(Yielder(T).new(@channel))
      ensure
        @channel.close
        @block_called = true
      end
    end
  end

  private struct Yielder(T)
    def initialize(@channel : Channel(T))
    end

    def <<(data : T)
      @channel.send(data)
      self
    end
  end
end

The Fibonacci generator could now be implemented similar to the Ruby example:
fib = Enumerator(UInt32).new do |y|
  a = b = 1_u32
  loop do
    y << a
    a, b = b, a + b
  end
end

p fib.first(10).to_a # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]

Please let me know if you think this is a useful addition to the Crystal Standard Library.
If you do, I would be happy to polish the code up into a PR along with documentation and specs.
I've also implemented Enumerator#peek locally, which allows to look at the next iterator value without moving it forward by buffering the value on peek, but left it out of the example for simplicity.

Yup, it's been discussed in the past:
https://groups.google.com/forum/#!searchin/crystal-lang/Enumerator|sort:date/crystal-lang/L08Hz8oXuIU/WqIDLxzKDgAJ
https://play.crystal-lang.org/#/r/aob
The main issue is that it's super slow. Like, 1000x times slower than Iterator. And we don't want to encourage slow code, much less in the standard library...
Well, at least that was the last time I benchmarked it, maybe it should be benchmarked again.
By the way, fibonacci can be currently implemented with Iterator, and it's super fast, no need for fibers nor channels:
a = b = 1
fib = Iterator.of do
  a.tap { a, b = b, a + b }
end

p fib.first(10).to_a

https://play.crystal-lang.org/#/r/4gz8
But, of course, more complex enumerators which yield in the middle of a method and continue from there can't be implemented with Iterator.of.

maybe similar to #4438 ?

@asterite Yeah, my actual use case is much more complicated. I have a nested while loop with XML::Reader that builds records from expanded sub nodes and yields them into the iterator.
It can certainly implemented with a simple yield without an iterator, but it makes it harder to do stuff like each_slice which you get for free with iterators.
In my case the performance of the enumerator shouldn't really matter, since most of the processing is in libxml2 anyways.
I'm not partial to the ruby specific implementation, but some form of a generator abstraction that doesn't have Iterators limitations would be great.
@bew Thanks for the link, I searched the existing issues for "enumerator" but not for "generator".

@felixbuenemann You can implement Iterator and use a stack. That's a way to remove recursion (for example to implement Iterator for a binary tree).
I just tried to implement Enumerator with just fibers, no channels, like in Ruby. It's just a bad implementation because I'm confused about where a fiber should resume, but this basic example works:
class Enumerator(T)
  include Iterator(T)

  struct Yielder(T)
    getter! enumerator

    def initialize(@enumerator : Enumerator(T))
    end

    def <<(value : T)
      @enumerator.value = value
    end
  end

  getter! fiber : Fiber
  getter! yielder : Yielder(T)

  def initialize(&@block : Yielder(T) ->)
    @current_fiber = Fiber.current
    @yielder = Yielder(T).new(self)
    @fiber = Fiber.new do
      @block.call(@yielder.not_nil!)
    end
  end

  @value : T?

  def value=(@value : T)
    @current_fiber.resume
  end

  def next
    # This is probably not correct
    @current_fiber = Fiber.current

    stop unless fiber.alive?

    fiber.resume

    fiber.alive? ? @value.as(T) : stop
  end
end

fib = Enumerator(Int32).new do |y|
  3.times do |i|
    y << i
  end
end

p fib.next
p fib.next
p fib.next

That is, using some sort of cooperative fibers, instead of the non-cooperative fibers we have now. But that @current_fiber.resume should probably be Ruby's Fiber.yield.
But still, doing a simple benchmark of that implementation using Fibers vs Iterator gives 0.37s for Enumerator vs. 0.005s for Iterator. It's a dramatic difference. The code is something like this:
e = Enumerator.new do |y|
  loop do
    y << 1
  end
end

time = Time.now
10_000_000.times do
  e.next
end
puts Time.now - time

But then in Ruby that same example takes 12 seconds... so maybe it's not that bad? :-)
If we can figure out how to implement it with fibers and context switching, without channels, it might be a good fit for the standard library, even if it's slower than a manually written Iterator (a hand-written Iterator can always be written if Enumerator becomes the bottleneck of your program).

Oh, and in the example above I defined Fiber#@alive and Fiber#alive? for it to work (just a boolean that's true when the fiber starts, and false when it ends). But for some reason once the fiber finishes, everything crashes with a segfault... probably because the fiber is not registered in the Scheduler.

Hey, I was also looking for a way to find out if a fiber was still running, because at first I tried to save the fiber from spawn to an instance variable and then restart the fiber if needed during rewind.
So I think adding such a flag to Fiber would be generally useful.
I'll see if I can get your improved version running with my code.
Can you post the patch you did to the Fiber class?

It's just setting @alive = true in Fiber, then in run ensure it's set to false.

@asterite I tried you implementation, but it just hangs.
Here's the test suite I used (quick conversion from minitest.cr to crystal spec):
require "spec"
require "../src/enumerator"

describe Enumerator do

  describe "#initialize" do
    it "returns an instance of Enumerator" do
      instance = Enumerator(String).new {}
      instance.should be_a(Enumerator(String))
    end
  end

  describe "#next" do
    it "returns the values in the order they were yielded" do
      enumerator = Enumerator(String?).new do |yielder|
        yielder << "hello"
        yielder << nil
        yielder << "world"
      end

      enumerator.next.should eq("hello")
      enumerator.next.should be_nil
      enumerator.next.should eq("world")
      enumerator.next.should be_a(Iterator::Stop)
    end
  end

  describe "#peek" do
    it "peeks at the next value without affecting next" do
      next # Not implemented
      enumerator = Enumerator(String?).new do |yielder|
        yielder << "hello"
        yielder << "world"
      end

      enumerator.peek.should eq("hello")
      enumerator.peek.should eq("hello")
      enumerator.next.should eq("hello")
      enumerator.peek.should eq("world")
      enumerator.peek.should eq("world")
      enumerator.next.should eq("world")
      enumerator.peek.should be_a(Iterator::Stop)
      enumerator.next.should be_a(Iterator::Stop)
      enumerator.rewind
      enumerator.peek.should eq("hello")
    end
  end

  describe "#each" do
    it "iterates the yielded values" do
      enumerator = Enumerator(String).new do |yielder|
        yielder << "hello"
        yielder << "world"
      end

      values = [] of String
      enumerator.each { |value| values << value }
      values.should eq(["hello", "world"])
    end
  end

  describe "Enumerable" do
    it "responds to Enumerable methods" do
      enumerator = Enumerator(String).new do |yielder|
        yielder << "hello"
        yielder << "world"
      end

      enumerator.to_a.should eq(["hello", "world"])
    end
  end

  describe "#rewind" do
    it "rewinds the iterator" do
      next # Not implemented
      enumerator = Enumerator(String).new do |yielder|
        yielder << "hello"
        yielder << "world"
      end
      # We rewind first to check that block isn't called multiple
      # times if it was not enumerated before calling #rewind.
      enumerator.rewind.should be_a(Enumerator(String))

      enumerator.to_a.should eq(["hello", "world"])

      # Must be rewound, should be empty.
      enumerator.to_a.should eq([] of String)
      enumerator.rewind.should be_a(Enumerator)
      enumerator.to_a.should eq(["hello", "world"])

      # And make sure rewind properly resets state.
      enumerator.to_a.should eq([] of String)
      enumerator.rewind.should be_a(Enumerator)
      enumerator.rewind.should be_a(Enumerator)
      enumerator.to_a.should eq(["hello", "world"])
    end
  end

  describe "Nil type support" do
    it "works when the type is Nil" do
      enumerator = Enumerator(Nil).new do |yielder|
        yielder << nil
        yielder << nil
      end

      enumerator.to_a.should eq([nil, nil])
    end
  end

  describe "yielder chaining" do
    it "allows to chain yielder calls" do
      enumerator = Enumerator(String).new do |yielder|
        yielder << "hello" << "world"
      end

      enumerator.to_a.should eq(["hello", "world"])
    end
  end

  describe "Fibonacci enumerator" do
    it "generates the first 10 numbers in the Fibonacci sequence" do
      # Example adapted from Ruby Enumerator docs.
      fib = Enumerator(UInt32).new do |y|
        a = b = 1_u32
        loop do
          y << a
          a, b = b, a + b
        end
      end

      fib.first(10).to_a.should eq([1, 1, 2, 3, 5, 8, 13, 21, 34, 55])
    end
  end

end

I know it hangs. I'm not sure cooperative and non-cooperative fibers can coexist. I believe maybe @waj will know how to implement this, but I don't think he'll have time.

@asterite I got it working!
The problem with your implementation was that it didin't resume the @current_fiber after the fiber had finished.
I came up with the following implementation based on you code, but using a Deque for storage, so that it works with nil values. I also added #rewind and #peek and replaced the Fiber#alive? checks with a simple @done flag.
class Enumerator(T)
  include Iterator(T)

  def initialize(&@block : Yielder(T) ->)
    @current_fiber = Fiber.current
    @stack = Deque(T).new(1)
    @yielder = Yielder(T).new(self)
    run
  end

  def next
    fetch_next
    @stack.shift { stop }
  end

  def rewind
    run if @done
    self
  end

  def peek
    fetch_next
    if @stack.empty?
      stop
    else
      @stack[0]
    end
  end

  protected def <<(value : T)
    @stack.push(value)
    @current_fiber.resume
  end

  private def fetch_next : Nil
    @done || @fiber.not_nil!.resume if @stack.empty?
  end

  private def run
    @stack.clear
    @done = false
    @fiber = Fiber.new do
      begin
        @block.call(@yielder.not_nil!)
      ensure
        @done = true
        @current_fiber.resume
      end
    end
  end

  private struct Yielder(T)
    def initialize(@enumerator : Enumerator(T))
    end

    def <<(value : T)
      @enumerator << value
      self
    end
  end
end

Results:
Iterator 333.58M (   3.0ns) (± 3.15%)  0 B/op        fastest
   Fiber  31.65M (  31.6ns) (± 4.48%)  0 B/op  10.54× slower
 Channel  19.21M ( 52.07ns) (± 5.16%)  0 B/op  17.37× slower

Let me know what you think!

Awesome! If it's just 10 times slower than an Iterator I think it's a really good candidate for the std.
By the way, this doesn't work as expected:
e = Enumerator(Int32).new do |y|
  y << 1
end

spawn do
  p e.next
end

Fiber.yield

The problem is that the current fiber is saved in initialized, but when calling next we are in another fiber. The solution is simple: set @current_fiber = Fiber.current upon entering next. I think that will always work, but I'm not sure.
I'd say let's go for this, if it's solid. Sometimes writing an Iterator is very boring, and the 10x slowness of some parts of the code will probably not be the bottleneck of a program, and this boosts productivity a lot.

Thanks for the explanation. I was wondering why you were setting @current_fiber inside #next.
I think I'll have some time this weekend to package it all up into a PR and add some polish to the specs.

I've updated the code above to set @current_fiber = Fiber.current inside #fetch_next so your example now works with both #peek and #next.

By the way, maybe a Deque is a bit too overkill for this. We can use a union type for @value, or maybe even uninitialized if we really need too. I'm also not sure what's the use of peek, or why we'd like to add it, given that Iterator doesn't have a peek method. We could always have a PeekIterator(T) type that peeks a value and stores it.
Then, rewind doesn't seem to work completely well. It works only if you already consumed the entire iterator. For example this is how it works in Ruby:
e = Enumerator.new do |y|
  y << 1
  y << 2
  y << 3
end

p e.next # => 1
p e.next # => 2

e.rewind

p e.next # => 1
p e.next # => 2

But I'm not sure how we would implement it. So maybe for now rewind can raise until we figure it out.

There's another problem. This code doesn't free the memory:
loop do
  e1 = Enumerator(Int32).new do |y|
    3.times do |i|
      y << i
    end
  end
  e1.next
end

Fibers are created and maintained in a stack. I think these particular fibers should not be placed in that stack, so when they are not referenced anymore they are GCed. The source code in fiber.cr should probably be modified for this...

Peek is useful, when you don't yet know what's the next value is and you want to do different stuff based on that. I agree that it would make sense to have this for all Iterators.
Regarding the rewind edge case: I think we can handle it by adding another flag that signals to the yielder that it should abort execution.
I didn't know that you can check if an instance variable is initialized. Can you post an example?

The fiber linked list is required for the GC, you can't just remove a fiber from it: https://github.com/crystal-lang/crystal/blob/master/src/fiber.cr#L325
You need to do this with def finally.

@RX14 But the GC will find the fiber reference in that linked list, thus never try to collect it, and will never invoke finalize. A fiber that may never finish will never be collected.
We can't just keep a list of active stack roots, because the GC would then collect all fibers.
Maybe we need 2 lists: one for "attached" fibers (default), and one to keep the stack roots of "detached" fibers? Once the fiber is no longer referenced, the GC collects it and invokes its finalizer that should return the stack root back to the pool.

@ysbaddaden I think he meant a finalize on Enumerator: once the enumerator isn't referenced anymore, it will delete the Fiber and remove it from that linked list.

You just need to make sure the fiber doesn't have any references to the enumerator, which is... difficult. You'll probably need to employ WeakRef(T)

Why would the fiber have a reference to the Enumerator?

Probably because the @yielder is passed to the @block that the @fiber wraps and the yielder has a reference to the @enumerator.

I'm also not sure what happens if you call an IO method inside Enumerator. That would make the fiber be rescheduled, not sure when and how will that resume. I did a few tests and it seems to work, but mixing cooperative and non-cooperative fibers will be tough.

@asterite but all fibers are by definition cooperative because there's no method to preempt them

@RX14 In Ruby, Fiber#resume stars/resumes a fiber, and Fiber.yield yields control back to the fiber that resumed that fiber. That's known as cooperative fibers. In Crystal it's different because there's no Fiber#resume. Well, there is, but it's for implementation purposes, you are not supposed to use it like that. And Fiber.yield just tries to find a fiber that can run, it's not resuming the one that called resume.
Anyway, Enumerator has indeed a reference to the fiber, which in turn has a reference to the enumerator through a closure, so finalize is never invoked. I'm not sure how it can be implemented, because the reference from fiber to enumerator happens through a closure, not something we can change to be WeakRef. Or maybe it needs to be a special Fiber, I don't know.

the fiber has the reference to the yielder through the closure, the weakref should be from the yielder to the enumerator, surely?

@RX14 The fiber is in the linked list, so it can't be GCed. Then the fiber has a reference to the Enumerator through the closure, which can't be turned into a WeakRef. So I can't see how to implement this.
I guess it would be a nice to have, but it seems it's kind of impossible to implement this right now...

@asterite why does the fiber have a reference to the enumerator and not just the yielder?

@RX14 This is part of Enumerator's code:
  private def run
    @stack.clear
    @done = false
    @fiber = Fiber.new do
      begin
        @block.call(@yielder.not_nil!)
      ensure
        @done = true
        @current_fiber.resume
      end
    end
  end

Here the fiber is accessing instance variable from Enumerator, which means self is being captured. self is Enumerator.
We could move all the instance vars to local variables (or maybe not, @done needs to be set to true, the current fiber must be resumed (and this is another leak, the fiber created by Enumerator never ends and never gets removed from the linked list!)), but still, @yielder has a reference to Enumerator, so the reference will still be there.

we can proxy all these calls through @yielder, and use yielder = @yielder to make the only thing in the closure the yielder.

I experimented with this a bit and I almost got it, but there's still something not working well.
The Enumertor code is this one:
require "weak_ref"

class Enumerator(T)
  include Iterator(T)

  @fiber : Fiber?

  def initialize(&@block : Yielder(T) ->)
    @current_fiber = Fiber.current
    @stack = Deque(T).new(1)
    @yielder = Yielder(T).new(self)
    run
  end

  def next
    fetch_next
    @stack.shift { stop }
  end

  def rewind
    run if @done
    self
  end

  def peek
    fetch_next
    if @stack.empty?
      stop
    else
      @stack[0]
    end
  end

  protected def <<(value : T)
    @stack.push(value)
    @current_fiber.resume
  end

  private def fetch_next : Nil
    @current_fiber = Fiber.current
    @done || @fiber.not_nil!.resume if @stack.empty?
  end

  private def run
    @stack.clear
    @done = false

    block = @block
    yielder = @yielder.not_nil!

    fiber = Fiber.new do
      block.call(yielder)
    end
    fiber.on_finish do
      yielder.done
    end
    @fiber = fiber
  end

  protected def done
    @done = true
    @current_fiber.resume
  end

  def finalize
    LibC.printf "finalize\n"
  end

  private struct Yielder(T)
    @enumerator : WeakRef(Enumerator(T))

    def initialize(enumerator : Enumerator(T))
      @enumerator = WeakRef.new(enumerator)
    end

    def <<(value : T)
      @enumerator.value.try &.<< value
      self
    end

    # :nodoc:
    def done
      @enumerator.value.try &.done
    end
  end
end

You'll also need this diff applied:
diff --git a/src/fiber.cr b/src/fiber.cr
index bc90d8790..7b2b16185 100644
--- a/src/fiber.cr
+++ b/src/fiber.cr
@@ -116,6 +116,9 @@ class Fiber
     end
   end

+  def on_finish(&@on_finish)
+  end
+
   def run
     @proc.call
   rescue ex
@@ -145,7 +148,11 @@ class Fiber
     # Delete the resume event if it was used by `yield` or `sleep`
     @resume_event.try &.free

-    Scheduler.reschedule
+    if on_finish = @on_finish
+      on_finish.call
+    else
+      Scheduler.reschedule
+    end
   end

   @[NoInline]

Now, if you run this:
puts "Running..."
loop do
  e1 = Enumerator(Int32).new do |y|
    y << 1
  end
end

you'll see a list of "finalize" being printed to the console. That means the Enumerator is being collected. The memory still grows because at that point we must somehow tell the Fiber to remove itself from the linked list, but that's another thing to do.

loop do
  e1 = Enumerator(Int32).new do |y|
    y << 1
  end
  e1.next
  e1.next
end

it also works: a list of "finalize" get printed, and this time memory doesn't grow because the fiber ends.
But if you run this:
puts "Running..."
loop do
  e1 = Enumerator(Int32).new do |y|
    y << 1
  end
  e1.next
end

then "finalize" never gets printed... so for some reason someone is still holding a reference to Enumerator. I can't understand why starting the fiber and interrupting it in the middle causes this.
So someone will have to figure those two things, but we are closer :-)

I found the reason!
It's because Yielder#<< is doing:
@enumerator.try &.value.<< value

and that will eventually switch to the caller fiber. But try will put enumerator in a local variable, so it will be in the fiber's stack, and this is a strong reference (past the weak ref). The solution is not to keep the enumerator in the stack:
require "weak_ref"

class Enumerator(T)
  include Iterator(T)

  @fiber : Fiber?

  def initialize(&@block : Yielder(T) ->)
    @current_fiber = Fiber.current
    @stack = Deque(T).new(1)
    @yielder = Yielder(T).new(self)
    run
  end

  def current_fiber
    @current_fiber.not_nil!
  end

  def next
    fetch_next
    @stack.shift { stop }
  end

  def rewind
    run if @done
    self
  end

  def peek
    fetch_next
    if @stack.empty?
      stop
    else
      @stack[0]
    end
  end

  protected def <<(value : T)
    @stack.push(value)
  end

  private def fetch_next : Nil
    @current_fiber = Fiber.current
    @done || @fiber.not_nil!.resume if @stack.empty?
  end

  private def run
    @stack.clear
    @done = false

    block = @block
    yielder = @yielder.not_nil!

    fiber = Fiber.new do
      block.call(yielder)
    end
    fiber.on_finish do
      yielder.done
    end
    @fiber = fiber
  end

  protected def done
    @done = true
    @current_fiber.resume
  end

  def finalize
    LibC.printf "finalize\n"
  end

  private struct Yielder(T)
    @enumerator : WeakRef(Enumerator(T))

    def initialize(enumerator : Enumerator(T))
      @enumerator = WeakRef.new(enumerator)
    end

    def <<(value : T)
      # Try to do it without keep a reference to enumerator here
      pass_to_enumerator(value)
      current_fiber = get_current_fiber
      current_fiber.try &.resume
      self
    end

    def pass_to_enumerator(value : T)
      @enumerator.value.try &.<< value
    end

    def get_current_fiber
      @enumerator.value.try &.current_fiber
    end

    # :nodoc:
    def done
      @enumerator.value.try &.done
    end
  end
end

Now I just need to kill a fiber on finalize. Of course this can't be done right there, because it might allocate memory (the fiber is put back in a stack pool), but I bet it can be done in that event we have every 5 seconds to clean the stack pool.

Okay, got it :-)
The code is a bit ugly, but I did it like that to try it out. The infinite loop above never releases memory because it will never give a change for the scheduler to run the clean-up fiber, but that's also what happens when you do loop { spawn { } } so it's not a problem, real programs don't look like this and will probably give control back to the scheduler.  So my example now is this:
loop do
  e = Enumerator(Int32).new do |y|
    y << 1
  end
  e.next
  sleep 0.001
end

memory will get high at first but then it stabilizes around 20~40 MB.
Here's the diff. It's a bit ugly that it requires modifying Fiber... maybe there's another way to implement it.
diff --git a/src/enumerator.cr b/src/enumerator.cr
new file mode 100644
index 000000000..2884f0f10
--- /dev/null
+++ b/src/enumerator.cr
@@ -0,0 +1,100 @@
+require "weak_ref"
+
+class Enumerator(T)
+  include Iterator(T)
+
+  @fiber : Fiber?
+
+  def initialize(&@block : Yielder(T) ->)
+    @current_fiber = Fiber.current
+    @stack = Deque(T).new(1)
+    @yielder = Yielder(T).new(self)
+    run
+  end
+
+  def current_fiber
+    @current_fiber.not_nil!
+  end
+
+  def next
+    fetch_next
+    @stack.shift { stop }
+  end
+
+  def rewind
+    @fiber.not_nil!.kill
+    run
+    self
+  end
+
+  def peek
+    fetch_next
+    if @stack.empty?
+      stop
+    else
+      @stack[0]
+    end
+  end
+
+  protected def <<(value : T)
+    @stack.push(value)
+  end
+
+  private def fetch_next : Nil
+    @current_fiber = Fiber.current
+    @done || @fiber.not_nil!.resume if @stack.empty?
+  end
+
+  private def run
+    @stack.clear
+    @done = false
+
+    block = @block
+    yielder = @yielder.not_nil!
+
+    fiber = Fiber.new do
+      block.call(yielder)
+    end
+    fiber.on_finish do
+      yielder.done
+    end
+    @fiber = fiber
+  end
+
+  protected def done
+    @done = true
+    @current_fiber.resume
+  end
+
+  def finalize
+    @fiber.not_nil!.kill
+  end
+
+  private struct Yielder(T)
+    @enumerator : WeakRef(Enumerator(T))
+
+    def initialize(enumerator : Enumerator(T))
+      @enumerator = WeakRef.new(enumerator)
+    end
+
+    def <<(value : T)
+      pass_to_enumerator(value)
+      current_fiber = get_current_fiber
+      current_fiber.try &.resume
+      self
+    end
+
+    def pass_to_enumerator(value : T)
+      @enumerator.value.try &.<< value
+    end
+
+    def get_current_fiber
+      @enumerator.value.try &.current_fiber
+    end
+
+    # :nodoc:
+    def done
+      @enumerator.value.try &.done
+    end
+  end
+end
diff --git a/src/fiber.cr b/src/fiber.cr
index bc90d8790..6ecfb0f82 100644
--- a/src/fiber.cr
+++ b/src/fiber.cr
@@ -22,6 +22,8 @@ class Fiber
   protected property next_fiber : Fiber?
   protected property prev_fiber : Fiber?
   property name : String?
+  @must_die = false
+  @@fibers_that_must_die = 0

   def initialize(@name : String? = nil, &@proc : ->)
     @stack = Fiber.allocate_stack
@@ -116,6 +118,27 @@ class Fiber
     end
   end

+  def self.remove_dead
+    return if @@fibers_that_must_die == 0
+
+    fiber = @@first_fiber
+    while fiber
+      if fiber.@must_die
+        fiber.die
+        @@fibers_that_must_die -= 1
+      end
+      fiber = fiber.next_fiber
+    end
+  end
+
+  def on_finish(&@on_finish)
+  end
+
+  def kill
+    @must_die = true
+    @@fibers_that_must_die += 1
+  end
+
   def run
     @proc.call
   rescue ex
@@ -127,6 +150,16 @@ class Fiber
     ex.inspect_with_backtrace STDERR
     STDERR.flush
   ensure
+    die
+
+    if on_finish = @on_finish
+      on_finish.call
+    else
+      Scheduler.reschedule
+    end
+  end
+
+  def die
     @@stack_pool << @stack

     # Remove the current fiber from the linked list
@@ -144,8 +177,6 @@ class Fiber

     # Delete the resume event if it was used by `yield` or `sleep`
     @resume_event.try &.free
-
-    Scheduler.reschedule
   end

   @[NoInline]
diff --git a/src/kernel.cr b/src/kernel.cr
index 8ce3509ca..17aaab9ff 100644
--- a/src/kernel.cr
+++ b/src/kernel.cr
@@ -473,6 +473,7 @@ end
   spawn do
     loop do
       sleep 5
+      Fiber.remove_dead
       Fiber.stack_pool_collect
     end
   end

Oh, and I forgot to say that the above diff also makes rewind work well now :-)

Cool that you figured it out! 🎉
My only other idea was to raise an exception from the << method in the yielder causing the code in the block to blow up and rescuing from that exception in the wrapping fiber block.
I'm not sure I fully understand your changes.

Killing the fiber will make it so that later (there's a fiber that runs every 5 seconds) it will get deleted... so yeah, it is stopped at the middle of execution.

I benchmarked the updated version and performance regressed a bit due to the added complexity:
  Iterator 332.06M (  3.01ns) (± 3.15%)  0 B/op        fastest
Enumerator  26.09M ( 38.34ns) (± 4.39%)  0 B/op  12.73× slower

I don't think it really matters, since these were microbenchmarks with absolutely minimal load in the enumerator block and in most use cases where an enumerator is needed instead of an iterator that overhead is very minimal compared to the computation in the block.
On my machine (Broadwell i7-5557U 3.1GHz) 10,000,000 iterations with minimal load take only 400ms.

@asterite looks awesome!
But simply unscheduling a fiber won't unwind it's stack, so it can't release its resources (file descriptors etc.).

If it ends up that close to the performance of a channel, why not just use a channel?

Good point, I didn't even think about that.
I'll add a spec to make sure that doesn't happen.

To make discussion of the implementation easier, I pushed a first version in #6385.
Let me know what you think!
I would also be grateful for a short example that can only be implemented with Enumerator.
I tried to describe the use cases for both Iterator and Enumerator as best as I can, but the current Fibonacci generator example is not so great, since it can also be implemented with Iterator.

@felixbuenemann There's no such example. Anything can be implemented with Iterator, and without needing channels at all. That's what C# does with IEnumerable and automatic rewriting of yield. So Enumerator is just a convenience when you don't want to write that Iterator. So... I'm not sure about the documentation you used for Enumerator. Channels, blocking, etc., shouldn't be mentioned at all. It should just mentioned that it uses a fiber under the hood, and context switching, which is the reason of the slowness.

@asterite With your patch, this ensure block will never be called:
loop do
  e = Enumerator(Int32).new do |y|
    y << 1
  ensure
    puts "ensure"
  end
  e.next
  sleep 0.001
end

Such ensures are often used for cleaning up open resource handled, such as closing files, network connections etc.
The enumerating fiber shouldn't just be terminated without unwinding it's stack. It should work similarly like an exception was raised in Yielder#<< if the fiber is cancelled.

Well, someone could create an issue in Ruby and see what they answer :-)

There's at least already an issue for Crystal: #3561 =)

Not gonna happen. There's no way to do it in Go, so si doubt we'll manage to do it.

After all the discussion, I was unsure how useful an addition to the Crystal language this would be.
Since most simple examples are not useful for that I wen't ahead and converted the Enumerator code in the ETL app I'm currently working on to use Iterator instead.
I must say looking at the code side-by-side I much prefer the Enumerator version – it is much easier to comprehend.
It also took me about 4.5 hours to convert the code until the tests were green and another 2.5 hours to reimplement the early abort logic in the underlying XML pull parser to get the Iterator implementation to the same performance level as the Enumerator one (these took maybe 15 minutes to implement initially in the Enumerator version).
Here's an example of one of the Enumerators I replaced. The other one not shown here is powering the commands method in the example, but would be much more verbose to show, since I had to split it up from one into several methods.
Enumerator Version:
def documents(languages = ["de"])
  Enumerator(Document).new do |documents|
    attributes = {} of String => Attribute
    classification_groups = {} of String => ClassificationGroup
    commands(["Attribute", "ClassificationGroup", "Product"]).each do |reader|
      case reader.name
      when "Attribute"
        attribute = AttributeParser.new(reader, languages)
        attributes[attribute.id] = attribute.parse
      when "ClassificationGroup"
        classification_group = ClassificationGroupParser.new(reader, languages)
        classification_groups[classification_group.id] = classification_group.parse
      when "Product"
        product = ProductParser.new(reader, languages).parse
        languages.each do |language|
          document = DocumentBuilder.new(
            attributes,
            classification_groups,
            language,
            product
          ).build
          documents << document
        end
      end
    end
  end
end

Iterator Version:
def documents(languages = ["de"])
  return Iterator.of(Iterator.stop) if languages.empty?
  attributes = {} of String => Attribute
  classification_groups = {} of String => ClassificationGroup
  _commands = commands(["Attribute", "ClassificationGroup", "Product"])
  _commands.each do |reader|
    case reader.name
    when "Attribute"
      attribute = AttributeParser.new(reader, languages)
      attributes[attribute.id] = attribute.parse
    when "ClassificationGroup"
      classification_group = ClassificationGroupParser.new(reader, languages)
      classification_groups[classification_group.id] = classification_group.parse
    when "Product"
      product = ProductParser.new(reader, languages).parse
      products = Iterator.of do
        reader = _commands.next
        if reader.is_a?(Iterator::Stop)
          Iterator.stop
        else
          ProductParser.new(reader.as(XML::Reader), languages).parse
        end
      end
      _languages = languages.each
      return Iterator.of do
        language = _languages.next
        if language.is_a?(Iterator::Stop)
          language = _languages.rewind.next
          product = products.next
        end
        if product.is_a?(Iterator::Stop)
          Iterator.stop
        else
          DocumentBuilder.new(
            attributes,
            classification_groups,
            language.as(String),
            product.as(Product)
          ).build
        end
      end
    end
  end
  return Iterator.of(Iterator.stop)
end

As can be seen the iterator version involves several internal Iterator::SingletonProc instances and hard to follow logic to switch the different parsing phases (Attributes / ClassificationGroups / Products). It also might be interesting that the underlying XML::Reader parses through several gigabytes of XML that is uncompressed on the fly, so it cannot rewind the IO and start over.
So overall, I think it is very worthwhile to have this in the standard library, if only for the productivity gains (and me keeping my sanity ;-).

You could make your own Enumerator class perhaps?  Make it a library?

A small note that unless one needs next/peek, Enumerator doesn't use or need a Fiber (and it still has all the Enumerable methods of course).

Yes I'm sure:
enumerator = Enumerator.new do |y|
  i = 0
  loop {
    y << i += 1
  }
end
p enumerator.take(10) # => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
p enumerator.take(10) # => [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

So the "suspend" and the need for a Fiber is only for next/peek (and next_values/peek_values) which are stateful, all other Enumerator/Enumerable methods are not stateful and always "restart from the start" of the Enumerator.
As a note #zip uses Enumerator#next internally when given non-array arguments.

I think the difference is that Enumerator in Ruby is not lazy. If you write take(10) it consumes the entire thing.
In Crystal we'd like take(10) to return another "Enumerator" (it would be an Iterator) that will only consume the data once you call to_a, etc.

Enumerator::Lazy also doesn't create Fibers:
p Fiber.current # => #<Fiber:0x00007fb8fc423328 (resumed)>
enumerator = Enumerator.new do |y|
  i = 0
  loop {
    p Fiber.current # => #<Fiber:0x00007fb8fc423328 (resumed)>
    y << i += 1
  }
end.lazy
p enumerator.take(3) # => #<Enumerator::Lazy: #<Enumerator::Lazy: #<Enumerator: #<Enumerator::Generator:0x00007fb8fc3ecee0>:each>>:take(3)>

p enumerator.take(3).to_a # => [1, 2, 3]

@eregon So how is this implemented in Ruby if it's not using fibers under the hood, or having the VM suspend the execution at some places?

Basically it just remembers what it has to do in the Enumerator::Lazy object(s) and then finally eager evaluate and possibly using (non-local) break/return.
You can see it at https://github.com/oracle/truffleruby/blob/ddaa0759a460fd2a23655c3f5fa5d03bdcd05207/src/main/ruby/truffleruby/core/enumerator.rb#L282

@eregon Would you mind trying to implement this for Crystal? I'm almost sure it's impossible.
This is how far I got:
class Enumerator(T)
  class Yielder(T)
    def <<(value : T)
    end
  end

  def initialize(&@block : Yielder(T) ->)
  end

  def each
    yielder = Yielder(T).new
    # @block.call(yielder)
  end
end

e = Enumerator(Int32).new do |y|
  a = b = 1
  loop do
    y << a
    a, b = b, a + b
  end
end

e.each do |x|
  puts x
  break if x > 10
end

How do you implement Enumerator#each? It somehow needs to call the block given to the initialize method, and when a value is appended to the yielder it somehow needs to know to yield something back to the caller.

This works, I wrote it in Ruby because it's easier for me.
It's basically what you had, except the Yielder saves the block given to each and calls it on <<.
class MyEnumerator
  class Yielder
    def initialize(&block)
      @block = block
    end

    def << value
      @block.call(value)
    end
  end

  def initialize(&block)
    @block = block
  end

  def each(&block)
    yielder = Yielder.new(&block)
    @block.call(yielder)
  end
end

e = MyEnumerator.new do |y|
  a = b = 1
  loop do
    y << a
    a, b = b, a + b
  end
end

e.each do |x|
  p x
  break if x > 10
end

Nice, thanks!
This works:
class Enumerator(T)
  include Enumerable(T)

  class Yielder(T)
    def initialize(@block : T ->)
    end

    def <<(value : T)
      @block.call(value)
    end
  end

  def initialize(&@block : Yielder(T) ->)
  end

  def each(&block : T ->)
    yielder = Yielder(T).new(block)
    @block.call(yielder)
  end
end

e = Enumerator(Int32).new do |y|
  a = b = 1
  10.times do
    y << a
    a, b = b, a + b
  end
end

e.each do |x|
  p x
end

I changed the loop to 10.times because... break doesn't exist in Crystal inside captured blocks. So that functionality will be missing.
But then we can't make Enumerator be Enumerable because that relies on yield, not on captured blocks.
So in the end Enumerator done this way isn't very useful (I think)... we really need to be able to call next on Enumerator and have each do yield.

Just to clarify: In Crystal there is a clear separation between captured blocks (which becomes procs and can be stored) and yielding blocks (which are inlined into the yielding method and don't exist as a separate item at runtime).

That said, when we were originally thinking about Crystal we really wanted to be able to break from a captured block. The way that would work is that procs always return two values: the actual value and a second value that determines whether you want to break, next or return. Then the method calling the proc could check that second value, etc.
But we never got to do it. It's probably a lot of work.

I changed the loop to 10.times because... break doesn't exist in Crystal inside captured blocks. So that functionality will be missing.

Does non-local return exist in Crystal? That would be a workaround, just wrapping the e.each do |x| in a method and use return to escape it.
Regarding non-local break/return they are typically implemented with an exception, unwind or a similar mechanism (and some ID to know how far to unwind), and that doesn't need to check the return value on every return.

But then we can't make Enumerator be Enumerable because that relies on yield, not on captured blocks.

I don't know Crystal, but Ruby Enumerable only depends on each, so the code above should work without problems with Ruby Enumerable's semantics. There are some additional complications for a full Enumerator implementation, that's captured in the TruffleRuby code (notably for Enumerator#size and there are some additional indirections).
Maybe Crystal could have a way to capture a "yielded" block into a Proc object?

have each do yield.

Would this work maybe?
class Enumerator(T)
  def each(&block : T ->)
    yielder = Yielder(T).new(Proc.new { |x| yield x })
    @block.call(yielder)
  end
end

Does non-local return exist in Crystal?

No. Or at least not yet.

Regarding non-local break/return they are typically implemented with an exception, unwind or a similar mechanism (and some ID to know how far to unwind), and that doesn't need to check the return value on every return.

We could eventually do that in Crystal! See https://github.com/crystal-lang/crystal/issues/6357#issuecomment-1324149835 . But I think exceptions are slow so I'm not sure I'd like to introduce slow features.

Would this work maybe?

No because you can't yield from a captured block.
To be honest, I'm not exactly happy about the yield vs. "captured block" semantic we have in Crystal, but then Crystal isn't interpreted and doing those things in a compiled-to-native-code language is very hard, and maybe also slow.

Rust is adding gen fn which seems to serve a similar purpose. https://github.com/oli-obk/rfcs/blob/iter_fn/text/0000-gen-fn.md
https://github.com/rust-lang/rfcs/pull/3513


Unable to serve files from symlink (packages) dir with http_server

All dart apps use symlinks. There is a symlink dir inside of web called packages. The http_server's VirtualDirectory can't seem to serve files out of packages.
This line: if (split(path).first == "..") return new Future.value(null);  completes the Future with a null because path == '../packages'
I have "followLinks" set to true.
Bug or did I misconfigure something?

This issue has been moved to dart-lang/http_server#8.


added examples for built-in safe filter

It is an improvement in documentation. I think some examples must be added to explain the use of 'safe' built-in filter. So I have added two examples to show the difference in using this filter and not using this filter.

Thanks for this patch, however I don't think it's an improvement. It's too detailed and some sentences are misleading e.g.
{{ var|safe }}

If ``var`` is ``
something
``, the output would be ``something``.

that's not true, it suggest that safe truncates HTML tags, the only difference is that
 would be rendered as
 tag. IMO the current description is sufficient.


Pushing a monitor with certain imports results in error

While pushing a monitor with certain imports, the bundlers is not able to successfully bundle the journeys.
For example consider the following journey:
import { journey, step, monitor, expect } from '@elastic/synthetics';

import Jsbarcode from 'jsbarcode';
import { createCanvas, Canvas } from 'canvas';

const barcodeString = 'Test Barcode XYZ-100';

journey('Journey Lib Test', ({ page, params }) => {
  monitor.use({
    id: 'lib-test-001',
    enabled: true,
    schedule: 20,
  });

  // let canvas: Canvas;
  let canvas;

  step('Generate Barcode', async () => {
    canvas = createCanvas(200, 200);
    Jsbarcode(canvas, barcodeString);
    await page.goto(canvas.toDataURL());
  });

  step('Parse barcode', async () => {
    await page.goto('https://zxing.org/');
    await page.fill('#u', canvas.toDataURL());
    await page.click('[type=submit]');
    await page.click(`text=${barcodeString}`); // Asserts text in document
  });
});

Pushing the project to Kibana results in the following error:

Notes

The commented syntax let canvas: Canvas; also causes the error if uncommented:

I was able to push a journey with a simple import like uuid library.
Running the above journey directly with Synthetics works, e.g.

We dont support native bindings and its called out on the release specifically - https://github.com/elastic/synthetics/releases/tag/v1.0.0-beta.34.


Logit config: Handle as DEBUG (i.e., drop) spammy type=SYSCALL logs

What
Handle as DEBUG (i.e., drop) spammy type=SYSCALL logs. We send way too many GBs of logs to Logit and these look like they can be safely dropped.
How to review

Code review
Try it out on PaaS Dev.

🚨⚠ Please do not merge this pull request via the GitHub UI ⚠🚨

hold on - not sure this works

Nope I apparently just hadn't applied it to dev at all. Guess I just hit Validate.


How to run authenticated query or mutation in GraphQL Playground?

Hi @LawJolla! Thanks for the awesome example. Very cool!
I cannot understand how to run authenticated query or mutation in GraphQL Playground?
I tried to set Authorization1 header in the playground in the following way:
{
  "Authorization" : {
    "Bearer": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IlJERTBNamN3UkRVNFFrTTNOakkyTkRRNE9EbERPRFZGTmp..."
  }
}

So far it did not work. What else needs to be done to make it work?

Hi @sakhmedbayev ,
Thanks for checking it out!
The format is:
{
  "Authorization": "Bearer <JWT>"
}


Tried to queue_build using python client but got too many 502 error responses

Got the following issue when trying to use the python client to trigger a build.
Traceback (most recent call last):
  File "/Users/zhenbang.liu/Desktop/python_scripts/azure_devops.py", line 28, in <module>
    build_client.queue_build(build, project.name)
  File "/usr/local/lib/python2.7/site-packages/azure/devops/released/build/build_client.py", line 314, in queue_build
    content=content)
  File "/usr/local/lib/python2.7/site-packages/azure/devops/client.py", line 104, in _send
    response = self._send_request(request=request, headers=headers, content=content, media_type=media_type)
  File "/usr/local/lib/python2.7/site-packages/azure/devops/client.py", line 63, in _send_request
    content=content, **operation_config)
  File "/usr/local/lib/python2.7/site-packages/msrest/service_client.py", line 336, in send
    pipeline_response = self.config.pipeline.run(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/__init__.py", line 197, in run
    return first_node.send(pipeline_request, **kwargs)  # type: ignore
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/__init__.py", line 150, in send
    response = self.next.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/requests.py", line 72, in send
    return self.next.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/requests.py", line 137, in send
    return self.next.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/__init__.py", line 150, in send
    response = self.next.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/pipeline/requests.py", line 193, in send
    self.driver.send(request.http_request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/universal_http/requests.py", line 333, in send
    return super(RequestsHTTPSender, self).send(request, **requests_kwargs)
  File "/usr/local/lib/python2.7/site-packages/msrest/universal_http/requests.py", line 145, in send
    raise_with_traceback(ClientRequestError, msg, err)
  File "/usr/local/lib/python2.7/site-packages/msrest/exceptions.py", line 54, in raise_with_traceback
    raise error
msrest.exceptions.ClientRequestError: Error occurred in request., RetryError: HTTPSConnectionPool(host='dev.azure.com', port=443): Max retries exceeded with url: /zhenbangliu/test/_apis/build/Builds (Caused by ResponseError('too many 502 error responses',))

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d9e313c5-9120-9172-bc27-c21d10fcd5ae
Version Independent ID: 1a837182-591f-3cbc-1632-7b50d857b13c
Content: Release triggers for stages, branches, and pipelines - Azure Pipelines
Content Source: docs/pipelines/release/triggers.md
Product: devops
Technology: devops-cicd-release
GitHub Login: @RoopeshNair
Microsoft Alias: ronai

@zhenbang-databricks Thank you for the feedback, we're sorry for the difficulties.
Does this issue apply to an example in the article?  If so, please provide any details below.
With user-specific scenarios, provided support in the documentation forum is limited unfortunately. For additional support options, please refer to:

Azure DevOps community
Azure DevOps Support
Azure DevOps on StackOverflow

Closing this issue now.  For related posts on 502 errors, see here.


Node v4 fails with ECONNRESET errors on Travis

While I haven't tested on a local Node v4 instance, Travis fails on anything < 5. It's a pretty common issue but I haven't been able to solve it through the common solutions. For example explicitly setting NPM v3* in the Travis config did not solve the problem for Trilogy as it has elsewhere.
npm ERR! Linux 3.13.0-40-generic
npm ERR! argv "/home/travis/.nvm/versions/node/v4.0.0/bin/node" "/home/travis/.nvm/versions/node/v4.0.0/bin/npm" "install"
npm ERR! node v4.0.0
npm ERR! npm  v3.10.7
npm ERR! code ECONNRESET
npm ERR! errno ECONNRESET
npm ERR! syscall read
npm ERR! network read ECONNRESET
npm ERR! network This is most likely not a problem with npm itself
npm ERR! network and is related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly.  See: 'npm help config'

npm ERR! Linux 3.13.0-40-generic
npm ERR! argv "/home/travis/.nvm/versions/node/v4.0.0/bin/node" "/home/travis/.nvm/versions/node/v4.0.0/bin/npm" "install"
npm ERR! node v4.0.0
npm ERR! npm  v3.10.7
npm ERR! Callback called more than once.
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR!     /home/travis/build/citycide/trilogy/npm-debug.log

This could potentially be an issue with Travis since related issues say it's intermittent.

Case in point: the most recent build passed without any of these issues. There might not be any need for action but I'll keep an eye on it.

Several builds have passed now, so that along with configuring Travis to use npm@3 on older node versions is enough for me to say this is an intermittent, potentially Travis-exclusive issue. Closing.


title in strings doesn't work

I am trying to set a value to title in strings object of options to change the default UI but it's not working I would like to find out if I am missing out anything
here is my options
const locateOptions = {
position: "bottomright",
icon: "map-location-icon",
strings: {
title: "my current location",
}
};
and here is the component I wrote for locate control
class LocateControl extends Component {
componentDidMount() {
const { options, startDirectly } = this.props;
const { map } = this.props.leaflet;
const lc = new Locate(options);

lc.addTo(map);

if (startDirectly) {
  // request location update and set location
  lc.start();
}

}
render() {
return null;
}
}

What title does the online demo in this repo show doe you?

If I get you correctly you mean this part of the code
lc = L.control.locate({
    strings: {
        title: "Show me where I am, yo!"
    }
}).addTo(map);

But I'm doing the same thing. In my options I'm passing strings exactly as the code said and still not wining

The title shows up when you hover over the control. You can see it in action at https://www.domoritz.de/leaflet-locatecontrol/demo/. Since it works in the demo, the issue must be in your code.

@domoritz You are right. I thought it must add a string next to the icon. so I wanted to ask if there is a way to do so


Publish charts on Artifacthub

Publish our helm chart on artifacthub properly.
Would also be good to become a verified publisher.
Original thread: https://gigantic.slack.com/archives/CLPMFRVU6/p1681457734303279

@JosephSalisbury I've created an issue from the note. Do you have plans already?

Yeah. Depends on the amount of work. Would be good to stay top of mind.


SNI routing by egressgateway stopped working with Istio Auth

Describe the bug
SNI routing by egressgateway stopped working in https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-1.0-20180723-09-15/ with Istio Auth
Expected behavior
Directing HTTPS traffic thru the egressgateway, as described in https://preliminary.istio.io/docs/tasks/traffic-management/egress-gateway/#direct-https-traffic-through-an-egress-gateway, must work.
Steps to reproduce the bug
Follow the steps in https://preliminary.istio.io/docs/tasks/traffic-management/egress-gateway/#direct-https-traffic-through-an-egress-gateway. In step 5, error code 35 will be returned.
Run istioctl proxy-config listeners $(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -n istio-system. You will get Error: no listeners found
Version
https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-1.0-20180723-09-15/
Is Istio Auth enabled or not?
Yes
Environment
Kubernetes

The bug does not exist in https://github.com/istio/istio/releases/tag/1.0.0-snapshot.1

Your gateway spec doesn't have SNI in the tls section

Gateway TLS options do not have SNI.

Split the virtual service into two parts. Tls route for mesh to egress. And then tcp route for egress gateway. All under same virtual service

To specify the requirements:

Allow egress access to edition.cnn.com and to www.google.com, forbid access to anything else:

cat <<EOF | istioctl create -f -
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
      name: cnn
    spec:
      hosts:
      - edition.cnn.com
      ports:
      - number: 443
        name: tls
        protocol: TLS
      resolution: DNS
EOF

@rshriram

And then tcp route for egress gateway. All under same virtual service

but how the tcp route will route by the hostname? I need to distinguish edition.cnn.com and www.google.com.

The configuration 3a in https://github.com/istio/istio/issues/7361#issuecomment-407720960 works, closing the issue.

Trying with two virtual services combined:
cat <<EOF | istioctl create -f -
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: cnn
spec:
  hosts:
  - edition.cnn.com
  ports:
  - number: 443
    name: tls
    protocol: TLS
  resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: google
spec:
  hosts:
  - www.google.com
  ports:
  - number: 443
    name: tls
    protocol: TLS
  resolution: DNS
---
  apiVersion: networking.istio.io/v1alpha3
  kind: Gateway
  metadata:
    name: istio-egressgateway
  spec:
    selector:
      istio: egressgateway
    servers:
    - port:
        number: 443
        name: tls-cnn
        protocol: TLS
      hosts:
      - edition.cnn.com
      tls:
        mode: MUTUAL
        serverCertificate: /etc/certs/cert-chain.pem
        privateKey: /etc/certs/key.pem
        caCertificates: /etc/certs/root-cert.pem
    - port:
        number: 443
        name: tls-google
        protocol: TLS
      hosts:
      - www.google.com
      tls:
        mode: MUTUAL
        serverCertificate: /etc/certs/cert-chain.pem
        privateKey: /etc/certs/key.pem
        caCertificates: /etc/certs/root-cert.pem
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: cnn-to-egressgateway
spec:
  hosts:
  - edition.cnn.com
  gateways:
  - mesh
  tls:
  - match:
    - gateways:
      - mesh
      port: 443
      sni_hosts:
      - edition.cnn.com
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        subset: cnn
        port:
          number: 443
      weight: 100
    tcp:
  - match:
    - gateways:
      - istio-egressgateway
      port: 443
    route:
    - destination:
        host: edition.cnn.com
        port:
          number: 443
      weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: google-to-egressgateway
spec:
  hosts:
  - www.google.com
  gateways:
  - mesh
  tls:
  - match:
    - gateways:
      - mesh
      port: 443
      sni_hosts:
      - www.google.com
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        subset: google
        port:
          number: 443
      weight: 100
  tcp:
  - match:
    - gateways:
      - istio-egressgateway
      port: 443
    route:
    - destination:
        host: www.google.com
        port:
          number: 443
      weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
   name: set-sni-for-egressgateway
spec:
  host: istio-egressgateway.istio-system.svc.cluster.local
  subsets:
      - name: cnn
        trafficPolicy:
          loadBalancer:
            simple: ROUND_ROBIN
          portLevelSettings:
          - port:
              number: 443
            tls:
              mode: ISTIO_MUTUAL
              sni: edition.cnn.com
      - name: google
        trafficPolicy:
          loadBalancer:
            simple: ROUND_ROBIN
          portLevelSettings:
          - port:
              number: 443
            tls:
              mode: ISTIO_MUTUAL
              sni: www.google.com
EOF

test:
kubectl exec -it $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name}) -c sleep -- curl -sL -o /dev/null -w "%{http_code}\n" https://edition.cnn.com; kubectl exec -it $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name}) -c sleep -- curl -sL -o /dev/null -w "%{http_code}\n" https://www.google.com ; kubectl exec -it $(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name}) -c sleep -- curl -sL -o /dev/null -w "%{http_code}\n" https://www.wikipedia.org

the expected output:
200
200
000

verify that the traffic went thru the egress gateway:
kubectl exec -it $(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -c egressgateway -n istio-system -- curl -s localhost:15000/stats | grep edition.cnn.com.upstream_cx_total ; kubectl exec -it $(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}') -c egressgateway -n istio-system -- curl -s localhost:15000/stats | grep www.google.com.upstream_cx_total

expected output:
cluster.outbound|443||edition.cnn.com.upstream_cx_total: 1
cluster.outbound|443||www.google.com.upstream_cx_total: 1

cleanup:
istioctl delete serviceentry cnn google
istioctl delete virtualservice cnn-to-egressgateway google-to-egressgateway
istioctl delete gateway istio-egressgateway
istioctl delete destinationrule set-sni-for-egressgateway

@vadimeisenbergibm trying to set up egress rules for external services similar to https://github.com/istio/istio/issues/7361#issuecomment-407720960 (Direct the egress traffic thru the egress gateway with disabled mtls) but that generates pilot outbound listener conflict
"pilot_conflict_outbound_listener_tcp_over_current_tcp": {
            "<IP_ADDRESS>:443": {
                "proxy": "reviews-v2-65fbdc9f88-c8dqj.digital-services",
                "message": "Listener=<IP_ADDRESS>:443 AcceptedTCP=www.google.com RejectedTCP=edition.cnn.com TCPServices=1"
            }
        },

Any suggestion to resolve that conflict?
Istio version 1.4.4


Selecting correct htlc from probe attempt

Description:
Probes can return multiple htlcs but our code assumes there can only be one. This means that we can end up selecting a non-viable route sometimes.
Motivation and Context:
Improve probe success rate.
How Has This Been Tested?
Manually
Types of changes:
Bug fix

Coverage remained the same at 22.021% when pulling 3edd322a2a396865dc463c97228b82aa3386ec61 on fix/probe-htlc-selection into bb27f8d22a7f91db68f5d49dad5bab12857465db on master.


Remove git merge lines

These lines were erroneously introduced in d638a594d270dbbd123062ee5ab8c69a632548a0 due to a merge I presume

@KilnOfTheSecondFlame lib/domains/stoplist.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@KilnOfTheSecondFlame lib/domains/stoplist.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@KilnOfTheSecondFlame lib/domains/stoplist.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@KilnOfTheSecondFlame lib/domains/stoplist.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@KilnOfTheSecondFlame Your request is rejected. I have to close the request without merging. Please ensure you are familiar with this repository rules located at the bottom of this page and submit a new request with correct information. Thank you for your understanding!


JFXTimePicker.setIs24HourView to JFXTimePicker.set24HourView

I believe this is the right format. It would also allow for Kotlin interoperability (e.g.: timePicker.is24HourView = true).

yes sure, thanks for pointing out the issue 👍

No worries, thanks for such a great library!


第八章direct_sparse.cpp有一处笔误

https://github.com/gaoxiang12/slambook/blob/master/ch8/directMethod/direct_sparse.cpp#L161
Line 161       cout<<"usage: useLK path_to_dataset"<<endl;
此处是不是应该为direct_sparse path_to_dataset么，虽然不影响程序运行，但也算是个"bug"吧

谢谢指出。


Camera RTP Stream Management

Camera RTP Stream Management seems missing from Accessory Information service list.

Sorry, HomeSpan does not presently support Services that require TLV data, which include video streaming, audio streaming and advanced lock management (though simple locks are supported).

Sorry, HomeSpan does not presently support Services that require TLV data, which include video streaming, audio streaming and advanced lock management (though simple locks are supported).


store.dispatch or context.router

What is better
   this.context.store.dispatch(push('/'));
or
this.context.router.push('/');
?

They are roughly the same; they end up running the exact same code. We only provide the action creators as a convenience for those who like doing everything through Redux (it's debatable if it's a good thing to so heavily rely on Redux code patterns for everything).


Solution for Inkpainter with repeated texture.

Dear InkPainter programmer,
Painting texture only works when the uvs are not repeating on the texture. Otherwise, we get the following behaviour as shown in the .gif (your sample cube vs unity cube). Is there a solution for this problem ? Can we modify the uv through script so that we do not have a repetition ?
Thank you again for your response,

As you said, InkPainter depends on UV.
So it can be solved by setting UV of the model.
This is usually done using modeling software.
http://esprog.hatenablog.com/entry/2017/04/22/000204
You can do it by script.
This is simply to rewrite Mesh.uv.


Dan dev3

in this pull request im adding again the carousel video in edit tutorial so is easy to handle deletion  and also a better user experience

PLEASE REVIEW THIS PULL REQUEST I TESTED AND WORKS BUT JUST TO MAKE SURE REVIEW IT CAREFULLY


Removed walkRules

Derp, been a while since I've played with PostCSS. Also good catch on the stray var.
👍 LGTM, thanks!


botkit - Botframework as app service in azure , not working

I have developed a bot using bot kit-bot framework. It is running locally with ngrok / local tunnel with the webhook endpoint.As endpoint changes, every time bot is restarted  So I want to host this bot on Azure as app service. I have deployed bot through GitHub as continuous deployment. Bot is not running and error in sending messages on ms-bot test, could you help me how to make it run as azure app service
Can you please help me how to publish bot kit - Botframework as app service on Azure and what could be endpoint URL. As I wanted to run bot on skype.. It is easy to deploy botbuilder node js bot on azure

app.js
var Botkit = require('botkit');
var controller = Botkit.botframeworkbot({
});
var bot = controller.spawn({
appId: '5e1a87ee-9253-4c11-8a78-62513f5bdd9d',
appPassword: 'W0bS9NAJHQo9aXbbqD4sW4w'
});
// if you are already using Express, you can use your own server instance...
// see "Use BotKit with an Express web server"
controller.setupWebserver(process.env.port||3000,function(err,webserver) {
controller.createWebhookEndpoints(controller.webserver, bot, function() {
console.log('This bot is online!!!');
});
});
// user said hello
controller.hears(['hello'], 'message_received', function(bot, message) {
bot.reply(message, 'Hey there.');
});
controller.hears(['cookies'], 'message_received', function(bot, message) {
bot.startConversation(message, function(err, convo) {
convo.say('Did someone say cookies!?!!');
convo.ask('What is your favorite type of cookie?', function(response, convo) {
    convo.say('Golly, I love ' + response.text + ' too!!!');
    convo.next();
});

});
});
package.json
{
"name": "botkitresetbot",
"version": "1.0.0",
"description": "",
"main": "app.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"botkit": "^0.5.8",
"restify":"^5.2.0",
"botbuilder":"^3.9.1",
"os":"0.1.1",
"command-line-args":"4.0.7",
}
}

What is the error ? can you share some logs ?

Can you please help me how to publish botkit - Botframework as app service on Azure and what could be endpoint url

app.js
var Botkit = require('botkit');
var controller = Botkit.botframeworkbot({
});
var bot = controller.spawn({
appId: '5e1a87ee-9253-4c11-8a78-62513f5bdd9d',
appPassword: 'W0bS9NAJHQo9aXbbqD4sW4w'
});
// if you are already using Express, you can use your own server instance...
// see "Use BotKit with an Express web server"
controller.setupWebserver(process.env.port,function(err,webserver) {
controller.createWebhookEndpoints(controller.webserver, bot, function() {
console.log('This bot is online!!!');
});
});
// user said hello
controller.hears(['hello'], 'message_received', function(bot, message) {
bot.reply(message, 'Hey there.');

});
controller.hears(['cookies'], 'message_received', function(bot, message) {
bot.startConversation(message, function(err, convo) {

    convo.say('Did someone say cookies!?!!');
    convo.ask('What is your favorite type of cookie?', function(response, convo) {
        convo.say('Golly, I love ' + response.text + ' too!!!');
        convo.next();
    });
});

});
package.json
{
"name": "botkitresetbot",
"version": "1.0.0",
"description": "",
"main": "app.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"botkit": "^0.5.8",
"restify":"^5.2.0",
"botbuilder":"^3.9.1",
"os":"0.1.1",
"command-line-args":"4.0.7",
}
}

Messages are not sent

yes

And is your azure based endpoint reachable? It will likely be something like <app_name>.azurewebsites.net

nope even i tried https://<app_name>.azurewebsites.net/botframework/receive

Ok sounds like you are using the correct example app:
https://github.com/howdyai/botkit/blob/master/examples/botframework_bot.js
What happens when you run that with:
debug:true
Please post any console errors.

Ya I am using same example. On my local machine it is running with no problems but as i hosted on azure messages are not sent an i find some http errors

Please share what errors you encounter.

Please suggest me steps to deploy as it's not the problem in azure I think its problem in bundling and packaging when I deploy  since it is working fine locally

Application has thrown an uncaught exception and is terminated:
SyntaxError: Unexpected token ILLEGAL
at Module._compile (module.js:434:25)

at Object..js (module.js:464:10)

at Module.load (module.js:353:31)

at Function._load (module.js:311:12)

at Module.require (module.js:359:17)

at require (module.js:375:17)

at Object.<anonymous> (D:\home\site\wwwroot\lib\Botkit.js:2:16)

at Module._compile (module.js:446:26)

at Object..js (module.js:464:10)

at Module.load (module.js:353:31)

Application has thrown an uncaught exception and is terminated:
SyntaxError: Unexpected token ILLEGAL
at Module._compile (module.js:434:25)

at Object..js (module.js:464:10)

at Module.load (module.js:353:31)

at Function._load (module.js:311:12)

at Module.require (module.js:359:17)

at require (module.js:375:17)

at Object.<anonymous> (D:\home\site\wwwroot\lib\Botkit.js:2:16)

at Module._compile (module.js:446:26)

at Object..js (module.js:464:10)

at Module.load (module.js:353:31)

my bot is running on app service now but it is not responding

Sorry I do not have a lot of experience deploying on Azure, I would check that it can indeed reach the outside web, and that the endpoint url is setup in bot framework correctly.

It started working now ....with pipe thank you

Good news!

i am unable to publish my app from visual studio for bot

@sahootanmay sorry this repo isn't the best place for visual studio support.

Hi @saikiran1993
How can you fix this :D

I am having the exact same problem, how did you fix this? @saikiran1993 @thaoth58


error: ‘MMAL_PARAMETER_CLOCK_REQUEST_THRESHOLD_T’ has no member named ‘threshold_enable’

#Got this on the latest build of Rasbian and OF 9.2 and 9.3 am i missing something? Thanks!
More specifically this:
../../../addons/ofxCvPiCam/libs/mmal/src/interface/mmal/core/mmal_clock.c:836:4: note: in expansion of macro ‘LOG_TRACE’
LOG_TRACE("new clock request values: threshold %"PRIi64,
^
../../../addons/ofxCvPiCam/libs/mmal/src/interface/mmal/core/mmal_clock.c:840:36: error: ‘MMAL_PARAMETER_CLOCK_REQUEST_THRESHOLD_T’ has no member named ‘threshold’
private->request_threshold = req->threshold;
^
../../../addons/ofxCvPiCam/libs/mmal/src/interface/mmal/core/mmal_clock.c:841:43: error: ‘MMAL_PARAMETER_CLOCK_REQUEST_THRESHOLD_T’ has no member named ‘threshold_enable’

This seemed to fix mv libs libs-old


Infer only with box, but without points?

Hello, nice wok!
My question is, if I just want to promt with box, but without points, what should I do?
Another question is, I have browsed previous issues, I found that you said you have merged wrong branch to master. Should I pull master branch now?
Appreciated!

Hello, I have done it! just commit the point related input for decoder!


Revert "Issue-931: MavenArtifactControllerTest proper cleanup"

Reverts strongbox/strongbox#932, as the build is still failing, as seen here.

@ptirador ,
Could you please have another look? Thanks! :)


fix wrong kernelspec

Hey, @choldgraf,  the fix in this PR resolved https://github.com/executablebooks/jupyter-book/issues/584 for my octave jupyter book.

@joergbrech amazing! thanks for the fix :-)  do you think you could add a test for this? I think if we added a notebook w/ a kernel that wasn't python3 here: https://github.com/executablebooks/sphinx-book-theme/tree/master/tests/sites/base then we could test to make sure that the proper button link was created in test_build. WDYT?

Sure! I will check it out next week.

Ok, I found some free time today :smile:

amazing! Thanks very much for the fix :-)


GameOfLife activity resets on changing the size of the canvas.

Hey @llaske , I came across a functionality break(maybe) as when I changed the slider to change the size of the canvas for the GameOfLife activity, it resets the game and even the pattern with which we start the game, I've attached some screenshots for the same. Please try and replicate this and please give your thoughts.

Yes you're right but it seems logical since this slider redraw the board. Patterns buttons have the same behavior.


CMake cleanup pass (remove unused deps, set some defaults...)

Resolves #3, resolves #6, resolves #7

We should keep in mind that when we will rewrite the ROS nodes in C++, the cmake will need to find pinocchio, aligator and ndcurves among others.

These should be transitively found by linking to simple-mpc which links to all of them I think

This PR is already big enough. We can split off into a second one after reviewing the changes in this one.


fix(tagComp): updated hex value to color token

Summary
Updated Hex color to color token
ADO Story or GitHub Issue Link
https://dev.azure.com/Kyndryl/Shidoka - Bridge Design System/_workitems/edit/1931328
To Do

Update shidoka foundation version

Checklist

[x] Used Conventional Commit messages as outlined in the contributing guide.

[ ] Noted breaking changes (if any).

[] Documented/updated all props, events, slots, parts, etc with JSDoc.

[x] Ran the analyze command to update Storybook docs.

[ ] Added/updated Stories with controls to cover all variants.
[ ] Ran test locally to address any failures.
[ ] Added/updated the Figma link for the Story's Design tab.
[ ] Added any new component exports to the src/index.ts file

Screenshots
(if any)

This can't be tested/reviewed until after the foundation changes are published.

:tada: This PR is included in version 1.38.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Naively comparing function signatures messes with clojures

https://github.com/Olian04/brynja/blob/f0318e72649e9b94f108ce01a1cd70155c2f4edc/src/updateNode.ts#L86-L90
Example
Bugged: https://jsfiddle.net/olian04/guwmy0c7/3/
Not bugged: https://jsfiddle.net/olian04/b9t1j8cx/4/

The simple solution would be to remove the naive compare and simply always update the handlers. However that would severely increase the number of dom operations during the happy path.

Likely fixed when fixing #35
Will leave this issue open for the time being.


Changing Bot Message

Is it possible to change the text from the chat bot:

Yes, you would have to make the corresponding changes in the Resources.resx file to change the Icebreaker bot text. You can change the text of the labels and buttons of the Icebreaker bot in the file located here: https://github.com/OfficeDev/microsoft-teams-apps-icebreaker/blob/main/Source/Icebreaker/Properties/Resources.resx.
You will have to make these changes in a forked copy of the Icebreaker app, Once you've pushed these changes, then start the manual deployment process of the Icebreaker app, and in step 2, when deploying to Azure, ensure that the field pointing to the Icebreaker Github repo is not pointing to https://github.com/OfficeDev/microsoft-teams-apps-icebreaker and is instead pointing to the forked repo where you have made the required changes.

Thx for the response.
I will try this out.


Log the driver exit status explicitly

Currently it's not so obvious to tell when the spark job has succeeded or failed in the output. Compare these two:
2017-05-17 02:53:48 INFO  LoggingPodStatusWatcher:54 - State changed, new state:
         pod name: spark-pi-1494989589236
         namespace: default
         labels: spark-app-id -> spark-pi-1494989589236, spark-app-name -> spark-pi, spark-driver -> spark-pi-1494989589236
         pod uid: f6d399dc-3aab-11e7-99db-08002774bad8
         creation time: 2017-05-17T02:53:11Z
         service account name: default
         volumes: spark-submission-secret-volume, default-token-q8ktn
         node name: <IP_ADDRESS>
         start time: 2017-05-17T02:53:11Z
         container images: example.com/lins05/spark-k8s-driver:latest
         phase: Succeeded
         status: [ContainerStatus(containerID=docker://4109f34b2e13bd01e69e799d7d90bd3349b497ed357f0e148fefdb0d0f21210a, image=example.com/lins05/spark-k8s-driver:late
st, imageID=docker-pullable://example.com/lins05/spark-k8s-driver@sha256:01ef2073c94954f845b7abe01da2e3bd6f92d18c734a484d09741f334e3a1a95, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=false, restartCount=0, state=ContainerState(running=null, terminated=ContainerStateTerminated(containerID=docker://4109f34b2e13bd01e69e799d7d90bd3349b497ed357f0e148fefdb0d0f21210a, exitCode=0, finishedAt=2017-05-17T02:53:48Z, message=null, reason=Completed, signal=null, startedAt=2017-05-17T02:53:12Z, additionalProperties={}), waiting=null, additionalProperties={}), additionalProperties={})]
2017-05-17 02:53:48 INFO  Client:54 - Application spark-pi-1494989589236 finished.
2017-05-17 02:53:48 INFO  ShutdownHookManager:54 - Shutdown hook called

v.s.
2017-05-17 02:57:11 INFO  LoggingPodStatusWatcher:54 - Application status for spark-pi-1494989799975 (phase: Running)
2017-05-17 02:57:15 INFO  LoggingPodStatusWatcher:54 - State changed, new state:
         pod name: spark-pi-1494989799975
         namespace: default
         labels: spark-app-id -> spark-pi-1494989799975, spark-app-name -> spark-pi, spark-driver -> spark-pi-1494989799975
         pod uid: 747368bd-3aac-11e7-99db-08002774bad8
         creation time: 2017-05-17T02:56:42Z
         service account name: default
         volumes: spark-submission-secret-volume, default-token-q8ktn
         node name: <IP_ADDRESS>
         start time: 2017-05-17T02:56:42Z
         container images: example.com/lins05/spark-k8s-driver:latest
         phase: Failed
         status: [ContainerStatus(containerID=docker://0efd66beb19bd258625567fb34ad5b71f95a7fcf0bb82e0f123cb429b17a0f56, image=example.com/lins05/spark-k8s-driver:latest, imageID=docker-pullable://example.com/lins05/spark-k8s-driver@sha256:01ef2073c94954f845b7abe01da2e3bd6f92d18c734a484d09741f334e3a1a95, lastState=ContainerState(running=null, terminated=null, waiting=null, additionalProperties={}), name=spark-kubernetes-driver, ready=false, restartCount=0, state=ContainerState(running=null, terminated=ContainerStateTerminated(containerID=docker://0efd66beb19bd258625567fb34ad5b71f95a7fcf0bb82e0f123cb429b17a0f56, exitCode=1, finishedAt=2017-05-17T02:57:14Z, message=null, reason=Error, signal=null, startedAt=2017-05-17T02:56:43Z, additionalProperties={}), waiting=null, additionalProperties={}), additionalProperties={})]
2017-05-17 02:57:15 INFO  Client:54 - Application spark-pi-1494989799975 finished.
2017-05-17 02:57:15 INFO  ShutdownHookManager:54 - Shutdown hook called

The former one finished successfully while the latter failed. But you need to look very hard to locate the value of phase:  to recognize that.
I suggest add the exit status into this line, e.g.:
2017-05-17 02:57:15 INFO  Client:54 - Application spark-pi-1494989799975 finished successfully

Or
2017-05-17 02:57:15 INFO  Client:54 - Application spark-pi-1494989799975 failed with exit code 1. You may want to check the driver pod logs.

Looks like this would be a great change -- @lins05 can you send a PR to do this?

Sure, i'll do that.
On Thu, May 18, 2017 at 3:01 AM Andrew Ash<EMAIL_ADDRESS>wrote:

Looks like this would be a great change -- @lins05
https://github.com/lins05 can you send a PR to do this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache-spark-on-k8s/spark/issues/276#issuecomment-302196410,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAryM5m5XSG4UgQj9CXFF3E_SNHQ6Kcsks5r60QRgaJpZM4NdUwl
.

Included in https://github.com/apache-spark-on-k8s/spark/pull/283


Removing rubocop-rspec

It isn't compatible with the last version of rspec
nevir/rubocop-rspec#78

I released a new version of rubocop-rspec, so you can use rubocop 0.39.0 now.

Thank you, @nijikon

Appreciated, @nijikon!


Kusto: Suppress R2016 warning

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-js

Can one of the admins verify this patch?

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-node

Automation for azure-sdk-for-python

Automation for azure-sdk-for-go

Automation for azure-sdk-for-java

Automation for azure-sdk-for-net
A PR has been created for you:
https://github.com/azure/azure-sdk-for-net/pull/5405
.NET SDK Commits:
https://github.com/adxsdknet/azure-sdk-for-net/pull/5405/commits/f53936054446b6337a266ea96720e1021f3bd59d


Add vertexCacheSize to constructors

When the vertex-cache-limit is hit the first time in my game there is a really noticable lag/long frame.
To avoid this it would be great to be able to set the size of the vertex cache from the beginning.

Are you sure the lag is that? I think the only way you'll actually trigger this is if you're specifically using a PolygonBatch and are drawing largish filled shapes (eg a polygon with 400+ sides). The only reason it's even necessary is because if you draw a large number of vertices and are specifying triangle indices, it would be difficult to split the indices (you'd probably have to duplicate some vertices).
I ran some quick tests and I can't can't get much lag jumping from a triangle to a a polygon with 4000 sides using a PolygonSpriteBatch - what is a noticeable lag for you?
On a related note I had originally put in the idea of startCaching/stopCaching, which keeps filling up the float[] verts in BatchManager until it's full instead of pushing it to the Batch after every ShapeDrawer method call, but I don't really think there's even a performance difference, and I've been considering just removing it for simplicity for a while.

@earlygrey Yes, I can 100% reproduce the lag. A lag in this case is a frame with about 58-65ms (logged while running, not estimated) instead of the 16-17ms. This is very noticable if there are (faster) moving objects on the screen.
I basically draw a filled circle which expands quickly over time until it fills the whole screen and then fades out.
My solution at the moment is to extend ShapDrawer and to overwrite estimateSidesRequired(float radiusX, float radiusY) to cap at 200 sides (which "fills up" exactly the 2000 verts of the BatchManager) and not the 1722 (logged while running) it normally would return.
Unfortunately there is no way I could use startCaching/stopCaching for shapeDrawer.filledPolygon I guess.
Nevertheless, imho it would be a great additon to make it possible to adjust the vert cache size somehow, be it to make it bigger or smaller than the 2000. If that is not possible, it would be cool to be able to change and configure the BatchManager somehow.
But, of course it's you're decision.

I'm a bit confused - this is only triggered if you're drawing a filled polygon (or filled circle) using a PolygonBatch - but it's triggered at 399 sides, not 200. The way it's drawn in PolygonBatchFilledPolygonDrawer#polygon, an n-sided polygon needs n+2 vertices in verts, each takes 5 floats so (n+2)*5 floats total. You can test this with the test app - just put a break point or print line in the increaseCacheSize() method in BatchManager, and see when it's triggered. As far as I can see, here and here are the only places that can trigger it.
Are you sure it's not something like the cache size of your Batch? You should have something like new SpriteBatch(Short.MAX_VALUE / 4) or  new PolygonSpriteBatch(Short.MAX_VALUE) to get the largest cache size (confusingly they use different units). I can't possibly see how new float[4000]; (the next size increase) would take 60+ ms.
Actually re-reading your comment, 1722 is a huge number of sides, I don't think anyone would have a screen with enough pixels to warrant that. Are you calling drawer.update(); after setting your batch projection matrix? Or do I have to change the estimateSidesRequired method? To be fair, if Shape Drawer is trying to draw a 1722 side polygon with a PolygonBatch that would require a new float[16000] call, which on a low end system might be causing your lag. But then the problem is with the unnecessarily large number of sides.

If you really need the constructor argument then it can be added, though it would probably make more sense to use number of vertices as the units instead of how many floats it takes up (so the default float[2000] would correspond to 400).

Actually, I can just fix everything by overwriting estimateSidesRequired and capping it at 200. So I can basically just close this PR and https://github.com/earlygrey/shapedrawer/pull/41.
I'm just thinking that other people might run into the same problem I had. @earlygrey Do you have any idea how to prevent that?

Yeah I think you're right that it's probably a problem others have. Something I need to make clearer in the docs is that estimateSidesRequired() uses information about how big a world unit is in pixels, and this is calculated when you call drawer.update(); (or you can set it manually). I actually intended for users to override estimateSidesRequired when needed, but maybe it would be best to either cap the number of sides, or write a better method that asymptotes at 400 or something.

Closed because of another solution.


Section 'Customize-non-PHP_INI_SYSTEM directives' outdated for nginx

Using Linux App Service with PHP 8.2, Apache web server has been switched-out for nginx. This page: https://learn.microsoft.com/en-us/azure/app-service/configure-language-php?pivots=platform-linux#Customize-non-PHP_INI_SYSTEM directives recommends using .htaccess to customize PHP directives. However this does not work with nginx and thus Linux App Service w/ PHP 8.2. You must use the Customize PHP_INI_SYSTEM directives method. Please make this clear.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: a3209bb1-99cd-9e32-ae48-2605831aee02
Version Independent ID: 743bc03e-d0d1-1e9e-acb1-52ae01daa0cc
Content: Configure PHP apps - Azure App Service
Content Source: articles/app-service/configure-language-php.md
Service: app-service
GitHub Login: @cephalin
Microsoft Alias: cephalin

@samrueby Thanks for the feedback! We are currently investigating and will update you shortly.

@cephalin This doc appears to have guidelines from before we switched to nginx. Can you please review and update as necessary? Thanks


Houdini opens in Indie only.

Hello, when I add egLib to Houdini packages, Houdini will only open in Limited Commercial (Indie) mode. Even if I click on open Houdini FX commercial. Is there a reason for this?

Hey,
as the assest provided are .hdalc Houdini will only open as Indie. I have no Full Houdini License, so I cannot provide full commercial assets. I added this to the readme file.


fix: cloud-provider-region add cloudaccount_domain_id

这个 PR 实现什么功能/修复什么问题:
修正：cloud-provider-region增加cloudaccount_domain_id
是否需要 backport 到之前的 release 分支:

release/2.10.0
release/2.11
release/2.12

Codecov Report

Merging #2387 into master will increase coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2387      +/-   ##
=========================================
+ Coverage    7.07%   7.07%   +<.01%
=========================================
  Files         456     456
  Lines       71688   71682       -6
=========================================
+ Hits         5073    5074       +1
+ Misses      66197   66190       -7
  Partials      418     418

Flag
Coverage Δ

#aFlag
7.07% <0%> (ø)
:arrow_up:

Impacted Files
Coverage Δ

pkg/compute/models/snapshotpolicy.go
3.68% <ø> (ø)
:arrow_up:

pkg/compute/models/snapshots.go
1.88% <ø> (+0.02%)
:arrow_up:

pkg/cloudcommon/db/fetch.go
0% <0%> (ø)
:arrow_up:

pkg/compute/models/cloudproviderregions.go
0.38% <0%> (-0.01%)
:arrow_down:

pkg/util/seclib2/seclib.go
91.48% <0%> (-4.26%)
:arrow_down:

pkg/util/rbacutils/rbac.go
70.89% <0%> (+0.92%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1043958...91e52f9. Read the comment docs.

/area region

/approve
/lgtm


chore(components): prepare for component renames

Refs #2208.
Changelog
Changed

Renamed v2 to non-suffix in our data table code where we can rope them off (CSS class names and attributes)
Adjusted pagination handlebars code (config/template) so the default HTML points to v2

Testing / Reviewing
Testing should make sure pagination and data table are not broken.

Roped off version of joshblack#1 also renaming pagination - Wanted to see with @shinytoyrobots and @alisonjoseph if it's worth taking the risk. Thanks!


Retrieving the page's redirections

It's not an issue but rather a question -
Is it possible to view the page's navigation history? I've tried using Page.getNavigationHistory but it gave me incorrect results.
Same goes for Network.requestIntercepted which returns incorrect information..
Thanks in advance.

I've tried using Page.getNavigationHistory but it gave me incorrect results.
Same goes for Network.requestIntercepted which returns incorrect information..

Hi, care to elaborate and provide an example?
Also, what's your use case? Redirections are not part of the navigation history.
When a request is originated by a redirection the redirectResponse field of Network.requestWillBeSent is defined.

Well, what I mean is the following -
Assuming you navigate to page A (Page.navigate).
Page A then redirects you to page B which redirects you page C.
How can one get this navigation list?
I've tried both ways listed and none worked well.

Try this:
const CDP = require('chrome-remote-interface');

CDP(async (client) => {
    const {Network, Page} = client;
    Network.requestWillBeSent(({request, redirectResponse}) => {
        console.log(`${(redirectResponse || {}).url} -> ${request.url}`);
    });
    try {
        await Network.enable();
        await Page.enable();
        await Page.navigate({url: 'https://httpbin.org/redirect/10'});
        await Page.loadEventFired();
    } catch (err) {
        console.error(err);
    } finally {
        client.close();
    }
}).on('error', (err) => {
    console.error(err);
});

Well,
I've made a simple redirect chain using js.do and bit.ly to cnn - http://js.do/code/177866
I've tried using your code but it doesn't seem to work for some reason.

You're using:
window.top.location = 'http://cnn.it/1cTNZ3V';

which is not a proper redirect, that is, it doesn't trigger an HTTP 3xx response. You can inspect the initiator field of the Network.requestWillBeSent event to decide if a request is part of what you consider to be a redirect.
Conversely http://bit.ly/1cY78RZ results in:
HTTP/1.1 301 Moved Permanently
Location: http://example.com/


Rainbow Brush - Fixing a flawed design (thanks to Discord's PureDarkRain #9449 for the tip!)

So, as Fairy mentioned... the relic as it is is not something the player can trust. In a nutshell, even with only the base game cards, there's always the possibility you aren't playing as Defect and your cards come.
As a Boss Relic, that essentially gives a benefit? It's a bad relic.
Possible options:

Change tier to Rare. - _Bad option. Rare relics break rules, this does not solve the problem of it.
Make it Event only, don't add it in an event and make a Custom Modifier to it. - Bandage solution.
"Choose three Colors when you equip it. The randomized cards are from one of those colors". - Good but way too much work.
The Xth card is from another class. - ...no.

My main problem is that the design itself is flawed. No matter how many buffs I add to it, the main problem is that the relic is unreliable.
In other words, the solution is to make an Event for it. Since I don't have the energy for that, Special Tier, custom Relic it is then.

Of course, a fix will be needed in the future. Just not now.

The fixing will be done at #632.


Quotas for Identities Application Refactoring

Readiness checklist

[X] I added/updated unit tests.
[ ] I added/updated integration tests.
[X] I ensured that the PR title is good enough for the changelog.
[X] I labeled the PR.

Did you notice that the transport and integration tests are failing? It seems like the requests to Messages and Files are causing a 500 status code, which lead to consequential errors.


Don't crash when not loaded

Checking if Amplitude SDK is loaded before calling runQueuedFunctions()

Related #189


Media3

:sparkles: What kind of change does this PR introduce? (Bug fix, feature, docs update...)
Introduces Media3 (Fixes: https://github.com/Baseflow/ExoPlayerXamarin/issues/157)
@martijn00 This fully introduces Media3, I've updated the PackageId's as I think it would break to have media3 published on the current ExoPlayer Id's..
As well as that, if you merge this.. Could you set media3 as the main branch for this repository? This will allow us to keep the current develop and exoplayer branches should we need to do anything with them.
:arrow_heading_down: What is the current behavior?
:new: What is the new behavior (if this is a feature change)?
:boom: Does this PR introduce a breaking change?
:bug: Recommendations for testing
:memo: Links to relevant issues/docs
:thinking: Checklist before submitting

[ ] All projects build
[ ] Follows style guide lines
[ ] Relevant documentation was updated
[ ] Rebased onto current develop

@martijn00 When you have time to look this over can you approve and merge this?

Hi! Is there a reason why this has not been integrated and released to the nuget package ?

@Mackly Because it was decided not to be adopted, also Xamarin will have a Media3 package available soon (Went into review this week: https://github.com/xamarin/AndroidX/issues/710)

Thanks for the reply, I am following closely


Create a {caption} template that is alias for {descr}

On iOS Photos refers to the description as the Caption. Adding a new template alias may improve discovery and clarity.

Consider add PhotoInfo.caption as well

Also add --caption as an alias for wherever --description is used.


Font issue on Android 4.x WebView

Is this a bug report?
Yes. It appears when this commit is merged. On RN 0.50.0 above, Android 4.x WebView cannot display text/html format.

Have you read the Contributing Guidelines?
Yes
Environment
Environment:
OS: macOS High Sierra 10.13.2
Node: 8.9.1
Yarn: Not Found
npm: 5.5.1
Watchman: Not Found
Xcode: Xcode 9.2 Build version 9C40b
Android Studio: 3.0 AI-171.4443003
Packages: (wanted => installed)
react: 16.0.0 => 16.0.0
react-native: 0.50.0 => 0.50.0
Target Platform: Android 4.1.2 (API 16)

Steps to Reproduce

Render this component:
`export default class TestWebView extends Component<{}> {
render() {
    const html = "\n " +
    "※Facebook，並びにGoogle＋へ許可なく投稿がされることはありません。" +
    "\n";

    return (
        <WebView
			style={{flex:1}}
			source={{html}}/>
    );
}

}`
Expected Behavior

Actual Behavior

Reproducible Demo

(Paste the link to an example project and exact instructions to reproduce the issue.)

I have the same issue :(

Hi @GrigoryPtashko , how to resolve in case I load a string html in source property? btw, it's still a problem in v0.53.0 and newer.

@huynhtansi see my comment above. baseUrl must be ''.


feat: add Llama 3.1 8B to cloudflare provider options

Description
Added Llama 3.1 8B as an option for the Cloudflare provider
Checklist

[x] The base branch of this PR is dev, rather than main
[x] The relevant docs, if any, have been updated or created

Screenshots
[ "For visual changes, include screenshots." ]
Testing
[ For new or modified features, provide testing instructions. ]

a big day for open-source : )


"Collapse consecutive whitespace": BUG?

Hi. I used feature called 'Collapse consecutive whitespace' available under 'Edit cells / Common transforms'. I see that the GREL code under this operation is (Undo/Redo Extract):
value.replace(/[\p{Zs}\s]+/,' ')
In particular I performed this code:
"asssa".replace(/[\p{Zs}\s]+/,' ')
and I obtained this result:
"a a"
It sounds very strange, isn't it?
OpenRefine 3.5.2 used a different GREL expression:
value.replace(/\s+/,' ')
which works well for me

Hi @fabiolinus ! You likely accidentally swapped the \ before the p to after the p ?
My tests show the following:
"asssa".replace(/[p\{Zs}\s]+/, ' ')
--> a a
"asssa".replace(/[\p{Zs}\s]+/, ' ')
--> asssa
It's a simple typo mistake on your part, no worries.
Our code is correct however, and we have extensive tests for this now:
https://github.com/OpenRefine/OpenRefine/blob/5e6581837d146b68f499baac6d7cec8f88b02ade/main/webapp/modules/core/scripts/views/data-table/menu-edit-cells.js#L409
Closing as not a bug.

stupid gh weirdness... NOW it updates! lol.

Thanks to all and sorry for my own mistake