Building a training set of tags for ruby

Hello lovely maintainers :wave:
We've recently added "tags" to student's solutions. These express the constructs, paradigms and techniques that a solution uses. We are going to be using these tags for lots of things including filtering, pointing a student to alternative approaches, and much more.
In order to do this, we've built out a full AST-based tagger in C#, which has allowed us to do things like detect recursion or bit shifting. We've set things up so other tracks can do the same for their languages, but its a lot of work, and we've determined that actually it may be unnecessary. Instead we think that we can use machine learning to achieve tagging with good enough results. We've fine-tuned a model that can determine the correct tags for C# from the examples with a high success rate. It's also doing reasonably well in an untrained state for other languages. We think that with only a few examples per language, we can potentially get some quite good results, and that we can then refine things further as we go.
I released a new video on the Insiders page that talks through this in more detail.
We're going to be adding a fully-fledged UI in the coming weeks that allow maintainers and mentors to tag solutions and create training sets for the neural networks, but to start with, we're hoping you would be willing to manually tag 20 solutions for this track. In this post we'll add 20 comments, each with a student's solution, and the tags our model has generated. Your mission (should you choose to accept it) is to edit the tags on each issue, removing any incorrect ones, and add any that are missing. In order to build one model that performs well across languages, it's best if you stick as closely as possible to the C# tags as you can. Those are listed here. If you want to add extra tags, that's totally fine, but please don't arbitrarily reword existing tags, even if you don't like what Erik's chosen, as it'll just make it less likely that your language gets the correct tags assigned by the neural network.

To summarise - there are two paths forward for this issue:

You're up for helping: Add a comment saying you're up for helping. Update the tags some time in the next few days. Add a comment when you're done. We'll then add them to our training set and move forward.
You not up for helping: No problem! Just please add a comment letting us know :)

If you tell us you're not able/wanting to help or there's no comment added, we'll automatically crowd-source this in a week or so.
Finally, if you have questions or want to discuss things, it would be best done on the forum, so the knowledge can be shared across all maintainers in all tracks.
Thanks for your help! :blue_heart:

Note: Meta discussion on the forum

Exercise: gigasecond
Code
class Gigasecond
  attr_reader :date

  def initialize(start_date)
    @date = (start_date.to_time + 1e9).to_date
  end
end

Tags:
construct:add
construct:assignment
construct:class
construct:date
construct:double
construct:floating-point-number
construct:initialize
construct:instance-variable
construct:invocation
construct:method
construct:number
construct:parameter
construct:readme
construct:variable
construct:visibility-modifiers
paradigm:imperative
paradigm:object-oriented

Exercise: nth-prime
Code
class Prime
Inf = 1.0/0
def self.is_prime?(a)
	return a == 2 if a % 2 == 0
	(3..Math.sqrt(a)).step(2) do |i|
		return false if a % i == 0
	end
	true
end
def self.nth(n)
 raise ArgumentError if n < 1
	(2..Inf).lazy.select { |i| i if Prime.is_prime?(i)}.first(n).last
end

end

Tags:
construct:class
construct:divide
construct:do
construct:double
construct:floating-point-number
construct:if
construct:invocation
construct:lambda
construct:method
construct:number
construct:parameter
construct:return
construct:step
construct:throw
construct:variable
construct:visibility
paradigm:functional
paradigm:object-oriented
technique:exceptions
technique:higher-order-functions
uses:Enumerable.lazy
uses:Math

Exercise: bob
Code
class Bob
  class << self
    def hey(remark, language_map = LanguageMap)
      speak = language_map.new(remark)
      speak.public_methods(false)
           .select { |a| a.match(/bob.*/) }
           .sort
           .each do |rule|
             response = speak.send(rule)
             return response unless response.empty?
           end
    end
  end
end

class LanguageMap
  attr_reader :remark

  def initialize(remark)
    @remark = remark.strip
  end

  def bob_1_I_know_what_I_am_doing
      return '' unless phrase_contains_uppercase? && last_character_question_mark? && !includes_OK?
      "Calm down, I know what I'm doing!"
  end

  def bob_2_sure
      return '' unless last_character_question_mark?
      'Sure.'
  end

  def bob_3_chill_out
      return '' unless phrase_contains_uppercase? && !includes_OK?
      'Whoa, chill out!'
  end

  def bob_4_fine_be_that_way
      return '' unless remark.length.zero?
      'Fine. Be that way!'
  end

  def bob_9_whatever
      'Whatever.'
  end

  private

  def includes_OK?
    !remark.match(/OK/).nil?
  end

  def phrase_contains_uppercase?
    !remark.match(/[A-Z]{2}/).nil?
  end

  def last_character_question_mark?
    remark[-1] == '?'
  end
end

Tags:
construct:assignment
construct:boolean
construct:class
construct:constructor
construct:do
construct:endless-loop
construct:invocation
construct:lambda
construct:method
construct:method-overloading
construct:number
construct:optional-parameter
construct:parameter
construct:pattern
construct:readme
construct:return
construct:ruby
construct:string
construct:throw
construct:unless
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:exceptions
technique:higher-order-functions
technique:looping
technique:regular-expression
uses:Bob
uses:LanguageMap

Exercise: strain
Code
class Array
  def keep
    each_with_object([]) do |item, result|
      result << item if yield item
    end
  end

  def discard
    keep do |item|
      not yield item
    end
  end
end

Tags:
construct:class
construct:method
construct:parameter
construct:visibility-modifiers
paradigm:object-oriented
technique:inheritance

Exercise: trinary
Code
class Trinary
  def initialize(trinary_string)
    @digits = trinary_string.reverse.chars.map { |c| c.to_i }
  end

  def to_decimal
    result = 0
    @digits.each_with_index do |d, i|
      result += d * 3**i
    end
    result
  end
end

Tags:
construct:assignment
construct:char
construct:class
construct:do
construct:initializer
construct:int
construct:integral-number
construct:invocation
construct:lambda
construct:map
construct:method
construct:multiply
construct:number
construct:parameter
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:higher-order-functions
technique:looping

Exercise: acronym
Code
class Acronym
  def self.abbreviate(string)
    string.scan(/\b[a-z]/i).join.upcase
  end
end

module BookKeeping
  VERSION = 4
end

Tags:
construct:class
construct:method
construct:module
construct:parameter
construct:regexp
construct:visibility-modifiers
paradigm:object-oriented
technique:regular-expression

Exercise: scrabble-score
Code
class Scrabble
  VALUES = {
    "a" => 1,
    "b" => 3,
    "c" => 3,
    "d" => 2,
    "e" => 1,
    "f" => 4,
    "g" => 2,
    "h" => 4,
    "i" => 1,
    "j" => 8,
    "k" => 5,
    "l" => 1,
    "m" => 3,
    "n" => 1,
    "o" => 1,
    "p" => 3,
    "q" => 10,
    "r" => 1,
    "s" => 1,
    "t" => 1,
    "u" => 1,
    "v" => 4,
    "w" => 4,
    "x" => 8,
    "y" => 4,
    "z" => 10,
  }

  def initialize(word)
    unless word == nil
      @word = word.gsub(/\W/, '').downcase.split('')
    else
      @word = []
    end
  end

  def score
    score = []
    @word.each do |letter|
      score.push(VALUES[letter]).sum
    end
  end

  def self.score(word)
    new(word).score
  end

end

Tags:
construct:assignment
construct:class
construct:do
construct:method
construct:parameter
construct:regular-expression
construct:string
construct:variable
method:score
paradigm:imperative
paradigm:object-oriented
technique:regular-expression
uses:Scrabble

Exercise: hexadecimal
Code
require 'byebug'

class Hexadecimal
  DIGITS = %w{0 1 2 3 4 5 6 7 8 9 a b c d e f}

  def initialize(hex)
    @hex = hex
  end

  def to_decimal
    @hex.chars.reverse.each_with_index.reduce(0) do |sum, (hex_char, index)|
      return 0 unless valid_hex?(hex_char)
      sum + value_of(hex_char) * 16 ** index
    end
  end

  private

  def value_of(hex_char)
    DIGITS.index(hex_char).to_i
  end

  def valid_hex?(digit)
    DIGITS.include?(digit)
  end
end

Tags:
construct:add
construct:assignment
construct:char
construct:class
construct:each
construct:implicit-conversion
construct:index
construct:initializer
construct:int
construct:integral-number
construct:invocation
construct:lambda
construct:method
construct:multiply
construct:number
construct:parameter
construct:private
construct:return
construct:string
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:higher-order-functions

Exercise: triangle
Code
class Triangle
  # @param [Array] sides
  def initialize(sides)
    @sides = sides
  end

  # @return [Boolean]
  def equilateral?
    @sides.uniq.length == 1
  end

  # @return [Boolean]
  def isosceles?
    @sides.uniq.length == 2
  end

  # @return [Boolean]
  def scalene?
    @sides.uniq.length == 3
  end
end

Tags:
construct:assignment
construct:class
construct:method
construct:number
construct:parameter
construct:variable
construct:visibility-modifiers
paradigm:object-oriented

Exercise: secret-handshake
Code
class SecretHandshake
attr_reader :n
	def initialize(n)
		@n = n
	end
	def commands
		n = @n
	  return [] unless n.instance_of? Fixnum
		actions = ['wink',
						'double blink',
						'close your eyes',
						'jump']
		result = []
		(0..3).each do |i|
			result << actions[i] if n % 2 == 1
			n /=2
		end
		result= result.reverse if n % 2 == 1
		result
	end
end

Tags:
construct:assignment
construct:attribute
construct:class
construct:do
construct:each
construct:if
construct:indexing
construct:initialize
construct:integral_number
construct:invocation
construct:method
construct:number
construct:parameter
construct:range
construct:readme
construct:return
construct:variable
construct:visibility-modifiers
paradigm:imperative
paradigm:object-oriented
technique:looping

Exercise: luhn
Code
class Luhn

  def self.valid?(numbers)
    numbers = numbers.gsub(' ', '').chars.map(&:to_i)
    return false if check_invalid_size(numbers)
    sum_numbers(numbers) % 10 == 0
  end

  private

  def self.sum_numbers(numbers)
    numbers.each_with_index.inject(0) { |sum, (x ,i)| sum +=  check_count(x, i)  }
  end

  def self.check_count(number, i )
    if i.odd?
      number  *= 2
      number = number > 9 ?  number - 9 : number
    end
    number
  end

  def self.check_invalid_size(numbers)
    numbers.length < 2
  end
end

Tags:
construct:assignment
construct:boolean
construct:class
construct:if
construct:implicit-conversion
construct:invocation
construct:lambda
construct:method
construct:number
construct:parameter
construct:private-method
construct:return
construct:ternary
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:higher-order-functions

Exercise: linked-list
Code
class Element

  attr_accessor :datum,  :prev , :next

  def initialize(datum,next_element=nil, prev_element=nil)
    @datum = datum
    @next = next_element
    @prev = prev_element
  end

end

class Deque

  attr_reader :first, :last

  def push(datum)
    if @last.nil?
      @last = Element.new(datum)
      @first = @last
    elsif @last == @first
      @last = Element.new(datum,@first)
      @first.prev = @last
    else
      prev = @last
      @last = Element.new(datum,prev)
      prev.prev = @last
    end
  end

  def pop
    datum = @last.datum
    @last = @last.next if @last
    datum
  end

  def shift
    datum = @first.datum
    @first = @first.prev
    @first.next = nil  if @first
    datum
  end

  def unshift(datum)
    if @last.nil?
      @last = Element.new(datum)
      @first = @last
    elsif @last == @first
      @first = Element.new(datum,nil,@last)
      @last.next = @first
    else
      prev = @first
      @first = Element.new(datum,nil, prev)
    end
  end

end

Tags:
construct:assignment
construct:attribute
construct:boolean
construct:class
construct:constructor
construct:if
construct:invocation
construct:method
construct:nil
construct:optional-parameter
construct:parameter
construct:readme
construct:variable
construct:visibility-modifiers
paradigm:imperative
paradigm:object-oriented

Exercise: circular-buffer
Code
class CircularBuffer

  class BufferEmptyException < StandardError;  end;
  class BufferFullException < StandardError;   end;

  def initialize(size)
    @size = size
    @buffer = []
  end

  def write(data)
    raise CircularBuffer::BufferFullException if @buffer.size >= @size
    @buffer << data if data
  end

  def write!(data)
    if data
      @buffer << data
      @buffer.shift
    end
  end

  def clear
    @buffer = []
  end

  def read
    data = @buffer.shift
    raise CircularBuffer::BufferEmptyException unless data
    data
  end

end

Tags:
construct:assignment
construct:class
construct:custom-exception
construct:method
construct:parameter
construct:raise
construct:variable
construct:visibility-modifiers
paradigm:object-oriented
technique:exceptions

Exercise: binary-search
Code
class BinarySearch
  attr_reader :list

  def initialize list
  	raise ArgumentError if list != list.sort
  	@list = list
  end

  def search_for value
    search value, 0, @list.length-1
  end

  def middle index_left = 0, index_right = @list.length-1
    (index_left + index_right) / 2
  end

  private

  def search value, index_left , index_right
    _index_middle = middle index_left, index_right
    value_middle = @list[_index_middle]
    return _index_middle if value == value_middle
    raise RuntimeError if index_left == index_right

    if value > value_middle
      search value, _index_middle+=1, index_right
    else
      search value, index_left, _index_middle-=1
    end
  end
end

Tags:
construct:add
construct:assignment
construct:attribute
construct:class
construct:divide
construct:if
construct:indexing
construct:initialize
construct:int
construct:integral-number
construct:invocation
construct:lambda
construct:method
construct:number
construct:optional-parameter
construct:parameter
construct:private
construct:return
construct:subtract
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:higher-order-functions
technique:recursion

Exercise: binary-search
Code
class BinarySearch
  attr_reader :list

  def initialize(list)
    raise ArgumentError if list.sort != list
    @list = list
  end

  def middle
   (list.size - 1) / 2
  end

  def search_for(int)
    left, right = 0, list.size - 1
    while left < right
      mid = (left + right) / 2
      raise RuntimeError if left == mid ||right == mid
      if list[mid] == int
        return mid
      elsif list[mid] < int
        left = mid
      else
        right = mid
      end
    end
    mid
  end
end

Tags:
construct:add
construct:assignment
construct:attribute
construct:boolean
construct:class
construct:divide
construct:if
construct:indexing
construct:initializer
construct:int
construct:integral-number
construct:logical-or
construct:method
construct:number
construct:parameter
construct:return
construct:subtract
construct:variable
construct:visibility-modifiers
construct:while-loop
paradigm:imperative
paradigm:object-oriented
technique:boolean-logic
technique:looping

Exercise: pascals-triangle
Code
class Triangle
  def initialize(row_count)
    @row_count = row_count
  end

  def rows
    1.upto(@row_count).map { |n| row(n) }
  end

  private

  def row(row_number)
    1.upto(row_number).map { |column_number| value(row_number, column_number) }
  end

  def value(row_number, column_number)
    return 0 if column_number < 1
    return 0 if column_number > row_number

    return 1 if row_number == 1

    left  = value(row_number.pred, column_number.pred)
    right = value(row_number.pred, column_number)
    left + right
  end
end

Tags:
construct:add
construct:assignment
construct:class
construct:float
construct:if
construct:initializer
construct:int
construct:integral-number
construct:invocation
construct:lambda
construct:local-variable
construct:map
construct:method
construct:number
construct:parameter
construct:private
construct:return
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:higher-order-functions
technique:recursion

Exercise: connect
Code
module BookKeeping
  VERSION = 2
end

class Board
  def initialize(board)
    @board = board
  end

  def winner
    @winner ||= find_winner
  end

  private

  def find_winner
    return 'O' if o_wins? && !x_wins?
    return 'X' if x_wins? && !o_wins?
    ''
  end

  def o_wins?
    board = @board
    fill_first_line('O', 'o')
    while !last_line_has?(board, 'o')
      new_board = floodfill(board, 'O', 'o')
      break if new_board == board
      board = new_board
    end

    last_line_has?(board, 'o')
  end

  def fill_first_line(from, to)
    @board[0].gsub!(from, to)
  end

  def last_line_has?(board, symbol)
    board.last.include?(symbol)
  end

  def x_wins?
    board = @board
    fill_first_column('X', 'x')
    while !last_column_has?(board, 'x')
      new_board = floodfill(board, 'X', 'x')
      break if new_board == board
      board = new_board
    end

    last_column_has?(board, 'x')
  end

  def fill_first_column(from, to)
    @board.each {|line| line.sub!(/^(\s*)#{from}/, "\\1#{to}") }
  end

  def last_column_has?(board, symbol)
    board.any? {|line| line.chars.last == symbol }
  end

  def floodfill(board, from, to)
    board.map.with_index do |line, y|
      line.chars.map.with_index do |char, x|
        if char == from && touches(board, x, y, to)
          to
        else
          char
        end
      end
        .join
    end
  end

  def touches(board, x, y, char)
    has(board, x - 1, y - 1, char) ||
        has(board, x + 1, y - 1, char) ||
        has(board, x - 2, y,     char) ||
        has(board, x + 2, y,     char) ||
        has(board, x - 1, y + 1, char) ||
        has(board, x + 1, y + 1, char)
  end

  def has(board, x, y, char)
    return false if y < 0 || y >= board.length || x < 0 || x >= board[y].length

    board[y][x] == char
  end
end

Tags:
construct:add
construct:assignment
construct:boolean
construct:break
construct:class
construct:if
construct:index
construct:instance_variable
construct:int
construct:integral_number
construct:invocation
construct:lambda
construct:logical_and
construct:logical_or
construct:method
construct:method_overloading
construct:module
construct:number
construct:parameter
construct:private
construct:return
construct:string
construct:subtract
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:imperative
paradigm:object-oriented
technique:boolean-logic
technique:higher-order-functions
technique:looping

Exercise: tournament
Code
class Tournament

end

Tags:
construct:class
construct:method
construct:ruby
paradigm:object-oriented

Exercise: octal
Code
class Octal
  attr_reader :octal

  def initialize(octal)
    @octal = octal
  end

  def to_decimal
   valid? || octal.to_i(8)
  end

  def valid?
    0 if octal.chars.any? { |number| number > "8" }
  end
end

Tags:
construct:assignment
construct:boolean
construct:class
construct:def
construct:method
construct:parameter
construct:readme
construct:string
construct:variable
paradigm:object-oriented
technique:boolean
uses:Octal.to_decimal

Exercise: grep
Code
=begin
Write your code for the 'Grep' exercise in this file. Make the tests in
`grep_test.rb` pass.

To get started with TDD, see the `README.md` file in your
`ruby/grep` directory.
=end
class Grep
  def self.grep(pattern, flags, files)
    output = ''
    files.each do |file_name|
      File.readlines(file_name).each_with_index do |line, n|
        match = false
        # if flags affect matching
        if line.include?(pattern) then match = true end

        if flags.include?("-i")
          match = false
          if line.downcase.include?(pattern.downcase) then match=true end
        end

        if flags.include?("-x") and flags.include?("-i")
          match = false
          if line.downcase == pattern.downcase then match=true end
        end

        if flags.include?("-v") then match = !match end

        if flags.include?("-l") and match
          output += "#{file_name}\n"
          break
        end

        line_number = ""
        if flags.include?("-n") then line_number = "#{n.to_s}:" end

        file_included = ""
        if files.length > 1 then file_included = "#{file_name}:" end

        if match
          output += "#{file_included}#{line_number}#{line}".strip
        end
      end
    end
    output
  end
end

Tags:
construct:class
construct:comment
construct:method
construct:string-interpolation
construct:assignment
construct:boolean
construct:break
construct:do
construct:double-quoted-string
construct:if
construct:invocation
construct:lambda
construct:logical-and
construct:method-overloading
construct:nested-block
construct:number
construct:parameter
construct:variable
construct:visibility-modifiers
paradigm:functional
paradigm:object-oriented
technique:boolean-logic
technique:higher-order-functions
technique:looping

This is an automated comment
Hello :wave: Next week we're going to start using the tagging work people are doing on these. If you've already completed the work, thank you! If you've not, but intend to this week, that's great! If you're not going to get round to doing it, and you've not yet posted a comment letting us know, could you please do so, so that we can find other people to do it. Thanks!

You not up for helping: No problem! Just please add a comment letting us know :)

That would describe me.


CARBON-15591 upgrading carbon feature plugin to alpha

Do not merge this PR until alpha is released!

This PR is no longer valid right?

This PR is no longer valid


Support for RelaxNG schemas

Add support for parsing and validating RelaxNG schemas from the libxml2 relaxng module

I have this working in a branch, still needs a few tweaks, if this is functionality you'd want added to your library.

DTDs would be great too. They might seem obsolete, but they are straightforward to read and understand.


Add ZK configurations

@Huanli-Meng could u pls help review? thanks

@merlimat thanks for your review


Missing traces and metrics when using OtelJava.global

The exporters aren't flushed since the OtelJava.global has no finalization logic. Therefore, some traces and metrics may be missing, because the exporters aren't finalized properly.
def run: IO[Unit] = // some traces may be missing
  OtelJava.global.flatMap { (otel4s: Otel4s[IO]) =>
    otel4s.tracerProvider.tracer("example").get.flatMap { implicit tracer: Tracer[IO] =>
      // code
    }
  }

Also, OpenTelemetry discourages using global instance:

Due to the inherent complications around initialization order involving this class and its single global instance, we strongly recommend not using GlobalOpenTelemetry unless you have a use-case that absolutely requires it. Please favor using instances of OpenTelemetry wherever possible.

I have two considerations:

Add a similar warning to the OtelJava.global
Make OtelJava.global return a Resource[F, Otel4s[F]]

Well, the JVM shutdown is registered automatically:
https://github.com/open-telemetry/opentelemetry-java/blob/main/sdk-extensions/autoconfigure/src/main/java/io/opentelemetry/sdk/autoconfigure/AutoConfiguredOpenTelemetrySdkBuilder.java#L395-L397
So the traces may be missing due to different reasons.


build: update package.json export order as per Angular CLI warning

A warning appeared in ng build about types not taking effect due to coming after default. Solving that here

#858  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Page index unresponsive

Describe the bug
Page Index unresponsive after 767x760px.
How to correct it
Use media query and verify responsiveness using inspection.
Expected behavior
"Previous" and "Next" buttons should maintain its width.
Screenshots
If applicable, add screenshots to help explain your problem.

Simple solution: Remove the space between the arrow and the text. This causes the browser to render the text as one 'word' so it stays on the same line

I'll work on this as part of GCI task

@myfrom you can claim the task given on GCI dashboard, there's no need to comment here unless you wanna work on an issue that's not present in the task list.

Oh, ok. I wasn't sure if the tasks disappear once you claim them and didn't want to get a few people working on the same thing 😅

Don't worry, we can still modify the number of instances. Just ping us on the Slack channel in case you're unable to claim an unfinished task.

Is putting the pagination in a grid a valid solution? (would fix some styles, like align that better etc)

I think only other options would be to:

make the pagination horizontally scrollable or
decrease the number of pages shown on all screen sizes

Pagination is usually grid like in nature. But one can use other designs as well. Go with the first option. You can explore other ways to create an aesthetic pagination. I'll mark this issue under design.


Add WMS layers from an ArcGIS REST services endpoint

Add the ability to populate the layer manager with layers obtained from an ArcGIS REST services endpoint.

Examine commit from https://github.com/WorldWindEarth/explorer/commit/780603f7d437161b5874359e4d5818bff363e808 for an example of querying an ArcGIS REST endpoint.


Let's make a generic "jobs" UI in the table page

There have been several additions to the "jobs" features in Pinot. forceCommitStatus and rebalanceStatus have both been added recently, but #10088 chose to not include force commits because this is "reload status" only. I'm proposing we change the Reload Status button to a Jobs button.
This probably require a bit of design to decide what the end state should look like and what are reasonable steps to get there. But first, I want to gauge interest in this work.

cc @Jackie-Jiang @jayeshchoudhary

cc @swaminathanmanish

Yes @jadami10. We need to revisit the UI for this. @Jackie-Jiang and I were talking about this as as well. We may need a drop-down for the list of jobTypes that the user can choose from.
@jayeshchoudhary - What are you thoughts?

I don't have the context about recent changes so not very clear about the requirements
I can start once the requirements are clear.

I think we want to change the behavior of Reload Status button.
We recently changed the UI for this button, clicking reload status button now shows two tabs i.e jobs and status
Current Behaviour of Reload Status button
https://user-images.githubusercontent.com/41536903/228197216-4026c5bf-f4af-4755-8e8a-4d37dc064bd1.mp4

Thanks @jayeshchoudhary .Yes you are right about changing reload status to jobs (make it generic). Recently we introduced tableRebalance status as well and now we have 3 types of jobs (reload, force_commit, rebalance). Does it make sense to provide a list/drop-down from the jobs button that lets the user choose the jobType to see status ?
It looks like reload status already has the capability to show different jobTypes (one page to show all jobTypes) ?

this is a good enhancement to the job status UI. @swaminathanmanish let's figure out exactly what we'd like to show and then work with @jayeshchoudhary by providing rough wireframes and relevant APIs. Will DM you


address slow heatmaps

define too large: e.g., >10,000 samples/cells?
ideas

for large data (define too large?) do not do heatmaps showing features and data, but instead determine distance matrices and show those via the given metric eg correlation.
do not plot it if either observations or dimensions exceed e.g., 50k
use Heatgraphy, a new visualization package for multi-dimensional data.

GitHub repo: https://github.com/Heatgraphy/heatgraphy
Documentation: https://heatgraphy.readthedocs.io/en/latest/index.html
Web version: https://heatgraphy.streamlit.app/

fast distance matrix computation (which metric?)

https://www.kaggle.com/code/cpmpml/ultra-fast-distance-matrix-computation
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.euclidean_distances.html
numba?
knn to distance matrix from dynamo

(favorite) downsample to equal size of the smallest group provided in the metadata with a minimum of 100 or 10?
min(100,table(metadata$column))

Two options

Downsample if observations/features > 50k

downsample observations to equal size of the smallest group provided in the metadata with a minimum of 100 or 10? min(100,table(metadata$column))
downsample features to "most variable"

Precompute

distance matrix

in python with scipy.spatial.distance.pdist or even faster fastdist
in R with Rfast

hierarchical cluster with fastcluster (available for R and python and probably equally fast as both implementations are essentially in C++) https://danifold.net/fastcluster.html
plot using ggplot geom_tiles (without dendrogram) or provide ComplexHeatmap with precomputed dendrogramslibrary(fastcluster)
library(ComplexHeatmap)

data <- matrix(rnorm(10000), nrow = 1000)

# Compute the distance matrix
dist_matrix <- dist(data)

# Perform hierarchical clustering
row_hc <- fastcluster::hclust(dist_matrix, method = "complete")
col_hc <- fastcluster::hclust(dist(t(data)), method = "complete")
Heatmap(data,
        cluster_rows = as.dendrogram(row_hc),
        cluster_columns = as.dendrogram(col_hc))

tried with fastdist but abandoned, due to Error when using metric correlation. Anyway probably not as stable as scipy (although faster).
Traceback (most recent call last):
  File "/research/home/sreichl/projects/unsupervised_analysis/.snakemake/scripts/tmpj83a3tj1.distance_matrix.py", line 43, in <module>
    dist_mtx = fastdist.matrix_pairwise_distance(data_np, metric_function, metric, return_matrix=True)
ZeroDivisionError: division by zero

Observation downsampling: random with random seed to 1000?
Feature cut off by variability to 10k
Or both as configurable parameters?
Sample_proportion
highly_variable_feature_proportion

Downsampling done in distance matrix step
Heatmap script: filter for downsampled observations/features

Both configs accept float 0-1 as proportion or int as the absolute number of observations/features to be downsampled to.


Compile with ui library

I want the method that capacitate a developer for compiling harbor with "Harbor UI Library".
I add the  COMPILEHARBORUIFLAG .
If we execute make install with this flag, we can compile with LOCAL Harbor UI Library.

Coverage increased (+0.04%) to 53.506% when pulling 3a52ed63629e087ffa96a5c33de01a86fd4f6928 on m-masataka:compile-with-UI-Library into 7040e5ad5ae08eb703d2ef920a7c0830015016b7 on vmware:master.

@m-masataka
Thanks for submitting PR to Harbor. From your comments, we still did not get what's your main purpose for creating this PR? Could you please explain more?

@steven-zou
Thank you for replying me.
In Development Environment, I want to develop and build harbor with harbor-ui.
Unfortunately, current Makefile doesn't use LOCAL harbor ui library (src/ui_ng/lib/ ) but public harbor-ui's npm package when target compile_clarity: . (Is this correct?)
This specification is not developer friendly.
So I added the option what to use for make harbor with local harbor ui library(src/ui_ng/lib/ ).

@m-masataka
Sounds great! Actually your PR will help us a lot! Thanks!
@wy65701436 ,
Please help to review the makefile code in this PR, thanks.

@m-masataka, it's definitely a good idea for the developer and helpful for us.
Is it possible to add an new target instead of modifying the existing one(compile_clarity)? This is because that our CI/CD is using Make to generate build, it may break our exising process.

Coverage increased (+0.6%) to 54.079% when pulling 4ab4003f38af0e03197848397c043b79e7d5ef18 on m-masataka:compile-with-UI-Library into 7040e5ad5ae08eb703d2ef920a7c0830015016b7 on vmware:master.

Coverage increased (+0.6%) to 54.079% when pulling 8b4bea5244bcdfd473e4aeacb03cd6f600262eaa on m-masataka:compile-with-UI-Library into 7040e5ad5ae08eb703d2ef920a7c0830015016b7 on vmware:master.

@m-masataka
Any updates for this PR?

@steven-zou
No , that all.

@m-masataka
What I mean in my last comment is @wy65701436  has raised some comments to this PR and you might need to consider those comments.

@m-masataka
Thanks again for providing this excited functionality to Harbor community. As we're reaching the deadline of next new release, to avoid the possible influences on the releasing progress, we may hold this PR for a while and then merged back to master.

@steven-zou
Thank you for your comments, and Sorry my late reply.
I've responded to @wy65701436 's comment and fixed Makefile.
I'll fetch and merge master to this branch. Is it OK?

@m-masataka
Sure, no problem as long as this PR keeps clear(no conflicts) when merging back to master.

Coverage decreased (-0.005%) to 54.814% when pulling 2abcfac1f194571e958c2e6ddcadda3fabef4861 on m-masataka:compile-with-UI-Library into 9b9b5e414fde78d49960946ef2f766fac7ba7402 on vmware:master.

Coverage increased (+0.008%) to 54.694% when pulling 4848c1778b2cf81e308dcb164832b5f188efc037 on m-masataka:compile-with-UI-Library into 8a1b10ad7e81e8ff738e9857a7e72c2db22038f4 on vmware:master.

Coverage decreased (-0.04%) to 54.708% when pulling 15a18965e80d30f4b9966fea5efdd6bb69159812 on m-masataka:compile-with-UI-Library into 21c4e45cd3cd47e08eaf00c062abcb308cebe475 on vmware:master.

Coverage remained the same at 55.08% when pulling b704dc30061e49d3599d1bfc68671e69ea1b5470 on m-masataka:compile-with-UI-Library into b2420c035f2ecfff96699e58a18b6b29907c2467 on vmware:master.

Coverage increased (+0.5%) to 55.555% when pulling bd643dc1fd7219eaf1cc690509c1f801295a694e on m-masataka:compile-with-UI-Library into b2420c035f2ecfff96699e58a18b6b29907c2467 on vmware:master.


add publishToMavenLocal task to fix zip plugin not published to maven local issue when running this script

Description
add publishToMavenLocal task to fix zip plugin not published to maven local issue when running this script
Issues Resolved
add publishToMavenLocal task to fix zip plugin not published to maven local issue when running this script
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

LGTM but let's confirm a few things. @zane-neo & @opensearch-project/engineering-effectiveness can you add details on the reason for this change and if this affects the build scripts that call on this plugin's build script?

@ohltyler  We have a pipeline to create e2e test env which will leverage opensearch-build and all dependent plugins, fetching opensearch-core and plugins to build from main branch with non-snapshot parameter. Some repo is not publishing plugins to local maven repo, so the non-snapshot plugin can't be found and this impacts the following plugins. This PR is to make sure the plugin is published to maven local. @opensearch-project/engineering-effectiveness please confirm from your side if this change is good.


It Is Hard To Read In The Ui

When i go read a story the ui looks scuffed and hard to read.
Maybe change the text color to pure black?
Also this ui following the material you design/color scheme or dark mode would be nice in a future release.

Oh yes I forgot to force the text color to pure black and to not follow the Material Theme, I'm going to fix this for today's release.

Also this ui following the material you design/color scheme or dark mode would be nice in a future release.

It's actually a choice to have made the background white to improve readability as the default Material You color scheme
isn't really good for the reading surface but I'll think about it :+1:


sqlitecloud-js / database emit event

Implement method that emits events that are compatible with both node and web clients

https://github.com/primus/eventemitter3


Vips hangs when drawing an image after a fork call

ruby-vips hangs when drawing a image after a fork call.
Confirmed on both MacOS 14.2.1 and 13.6.6. Likely also affects linux as our CI system has the same issue.
I've attached the following test file which will hang; once you terminate the script you will also want to terminate the rouge process left behind from the fork.
require 'tempfile'
require 'vips'

class Image
  class << self
    def png(**kwargs)
      f = Tempfile.new(['image', '.png'], nil, encoding: 'ascii-8bit')
      draw_image(**kwargs).pngsave(f.path)
    end

    def draw_image(width: nil, height: nil, depth: 8, alpha: false)
      width ||= rand(256)  + 1
      height ||= rand(256) + 1
      bands = alpha ? 4 : 3
      image = Vips::Image.black(width, height, bands: bands)
      (0...width).each do |x|
        color = Array.new(bands) { rand(2^depth) }
        y1, y2 = Array.new(2) { rand(height) }.sort
        # After fork gets stuck here and process lingers even after terminating
        # parent process
        image = image.draw_line(color, x, y1, x, y2)
      end
      image
    end
  end
end

Image.png # Works

fork {  Image.png }

# Won't terminate because inside the fork hangs
Process.wait

This might be related to #155, but I don't believe so.

Hello @malomalo,
libvips is very threaded, so like all threaded libraries, you must be very careful when mixing it with fork.
When a process with several running threads forks, none of the threads copy over, instead you will have a new process with a single main thread. mutexes are in an indeterminate state, confusion is everywhere, nothing will work.
It's safest to only require 'ruby-vips' in the child and after the fork. On some platforms you can do the require, then fork, and then process an image, but not all. You can never process some images, fork, and then process some more.

Interesting
I'm running into this in my Rails test env when trying to parallelize test. I will look into what is happening before the fork to cause this as error #155 mentions it must be required before the fork
Is there anyway to reset vips after a fork perhaps?

You're right, I'd forgotten all the detail around appkit.
No, libvips doesn't support unload and reload, unfortunately.
This is an aside, but have you seen draw_line!? It's usually many times faster than plain draw_line since it can avoid a large copy on each operation.
https://www.libvips.org/2021/03/08/ruby-vips-mutate.html

Ah...
We have a Rails gem with an initializer that uses Vips to know image sizes on boot. Then Rails forks for testing and this issue presents itself.
Thanks re the mutate, new to me.


Satellite Improvements

Various cleanup and improvements to all of the satellite materials as well as a wildly expanded declarative notebook. I've also removed the declarative directory thinking that it makes more sense for those to be with their topical areas as that's what the users are interested in, not as much the interface distinction.
Closes #32

This should be ready for review/merge if all looks well. It ended up being longer than I thought!


Add type_name field to P4Info Metadata message

Add a description of this field to the text of the section describing
the contents of ControllerPacketMetadata messages.
Also add a description of the type_name field in the ValueSet section.
It was already re-using the MatchField messages used by table search
key fields, and already had this type_name field in it, but it seems
best to explicitly mention that it can be included in MatchField
messages for ValueSet objects, too.

This time I remembered to create a fork in this repository, so you should be able to see preview of generated output if you want, I hope.  I checked PDF output on my machine anyway, and it looked reasonable.
These changes might not be what we want, of course.  They seemed reasonably small and quick to write, in case it is true that we want to add this type_name field based on issue: https://github.com/p4lang/p4runtime/issues/187

Understood.  No rush on merging it in on my part.  I only noticed this because of doing some nit-pick level checking of details in some notes I am writing up on how I believe P4Info files should be generated for arbitrary P4_16 programs that have user-defined type and/or typedef definitions for these types, with or without p4runtime_translation annotations on them.  Will send you a link to that when I think it is ready to review.

I think this has been opened for long enough :)


[ContextMenu] Select on right click

Bug report
Current Behavior
Select menu element on fast rightclick
https://github.com/radix-ui/primitives/assets/46008279/c73f6d7f-97f0-4be2-be60-dce02bf6fcea
Reproducible example
https://codesandbox.io/p/devbox/radix-context-menu-bug-zlqgfp
Your environment
Check sandbox

Software
Name(s)
Version

Browser
google chrome

Operating System
windows
11

The context menu item is selected by right click, which should not happen.
In this example, the problem does not appear to be critical. But in my real application, when I select a context menu item, a radix modal window opens. The modal window also uses the portal. A bug occurs, after which the context menu cannot be opened.
I tried to make a minimal example, because looks like the problem is in the right-click selection.
If necessary, I can make an example with a modal window.

I still don't see what you mean in the video or when I try with the sandbox.

The context menu item is selected by right click, which should not happen.

What do you mean here?

Function "onSelect" should be called only on left mouse click, but in the video it's called on right mouse click.

Yeah I see, I can't seem to reproduce it using your sandbox though…

You need to quickly right-click and move the mouse to the right and down.

I see, the speed doesn't matter though.
What you are doing is what I do here in the video but just super fast, so it feels like a bug but isn't really one:

right-click but keep holding (pointerdown)
move cursor over an item (pointermove)
release cursor (pointerup)

https://github.com/radix-ui/primitives/assets/1539897/defd3cbe-20d5-4894-816d-2423d46ce25f
I can do exactly the same thing with the native menu going fast too.

Yes it is.
Then it's probably worth closing the issue. I’ll create a new issue for a context menu with modal window bug later.

It would be the same thing though right?

Need to check.

I won't create issue "context menu + modal dialog bug" because it was fixed by updating @radix-ui/react-dialog from 1.0.4 to 1.0.5 in my project)


first login

How do you login for the first time as an admin once you have built the server? the only place in the settings.yml that could possibly constitute "first time login credentials" don't seem to work. I must be totally missing something... any help/direction would be super awesome! thank you!
The settings.yml:
admin_email: @gmail.com
smtp_mail_username:  # Enter your SMTP email username
smtp_mail_password:  # Enter your SMTP email password

This is poorly documented, using the same credentials as the demo worked for me.
See: https://github.com/helpyio/helpy#live-demo for details.

@kwanda
thanks for the tip! i just tried, no luck. the only thing I really customized was the settings.yml. I changing settings.yml to the original values, but still no luck. "Invalid email address or password."

You might need to seed the database.
rake db:seed
Sent from my iPhone
On 11 Oct 2016, at 07:21, jakobii <notifications<EMAIL_ADDRESS>wrote:
@kwandahttps://github.com/kwanda
thanks for the tip! i just tried, no luck. the only thing I really customized was the settings.yml. I changing settings.yml to the original values, but still no luck. "Invalid email address or password."

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/helpyio/helpy/issues/378#issuecomment-252824005, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHWbFY8AnlkcOplLKwr88H2zZ9Nok4Pwks5qyyrdgaJpZM4KTRZs.

@CGA1123 is the first admin login  Admin User<EMAIL_ADDRESS>and password: "12345678"?  I've followed both of these tutorials closely:
http://support.helpy.io/en/knowledgebase/11-installing-helpy-yourself/docs/1-installing-helpy-on-using-the-digital-ocean-rails-image
http://support.helpy.io/en/knowledgebase/11-installing-helpy-yourself/docs/27-installing-helpy-on-ubuntu-14-04-using-passenger-and-nginx

yes, that is what you should use the first time you login.  you will go through a short onboarding that prompts you to change your password on your first time logging in.

oh ok great, thanks for the confirmation! at least now I can start narrowing down what I'm doing wrong

I have the same problem..."Invalid email address or password." Using User<EMAIL_ADDRESS>and password: "12345678" but continúe the problem

I ran "RAILS_ENV=production rake db:seed" as the user "Rails" and then the
demo login worked.
On Wed, Oct 12, 2016 at 6:37 AM domosorio<EMAIL_ADDRESS>wrote:

I have the same problem..."Invalid email address or password." Using User:
<EMAIL_ADDRESS>and password: "12345678" but continúe the problem
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/helpyio/helpy/issues/378#issuecomment-253215483, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AS6DRlov_8FhrPFYuIR2KrnD_s4ausrGks5qzOKtgaJpZM4KTRZs
.


Missing scrollbar in v10

Foundry build 291
5e version 2.0.3
All modules turned off except Storyteller? ✅
Despite the recent update that I've seen rolled out, the scrollbar simply will not appear in my storyteller sheets at all.

You can enable the scrollbar in the settings. This feature is not activated by default.


Fix some bugs of both code and document description.

code in shape component has a bug.
the document descriptions of  textfield and icon-button components have bugs

@ctuu Thanks for catching these issues :)

@ctuu Can you split the commit in three?

fix(shape): Property typo
docs(textfield): Fix examples
docs(icon-button): Fix slot description

@matsp  ok i will try to do it.

@matsp It seems that the commit now be split into three commits.

Thanks!:blush:


Docs?

I want to try Nimgram, but seems like there's no documentation :(

The project is still in early development so ATM no docs are being written.
But keep in mind that i'm not the developer of this project so please wait for an official answer, if any.

But keep in mind that i'm not the developer of this project so please wait for an official answer, if any.

Well, thanks anyway! I'll wait

I want to try Nimgram, but seems like there's no documentation :(

As of now Nimgram does not have so many high-level methods, and since it's a priority adding them, documentation is the last thing we need to work on.
Anyway, the test file does have some examples on using Nimgram:
https://github.com/nimgram/nimgram/blob/master/tests/test.nim
If you want instead to use raw TL methods, you can see an old version of test.nim:
https://github.com/nimgram/nimgram/blob/029a3d7010ac6da45dc6bfe40b17a4d65098887e/tests/test.nim
Mostly everything is the same as of now, but you also need to import nimgram/private/rpc/raw

@dadadani Hmm... So as @alsoGAMER said, Nimgram is still in early development, well that makes sense. Thanks anyway for the answer! :)


DOM Mutation Events

https://developer.mozilla.org/en-US/docs/Web/Events/DOMSubtreeModified
Would love to have some data on this.
DOMSubtreeModified	(none)	Fires when the subtree is modified	Yes	No
DOMNodeInserted	(none)	Fires when a node has been added as a child of another node	Yes	No
DOMNodeRemoved	(none)	Fires when a node has been removed from a DOM-tree	Yes	No
DOMNodeRemovedFromDocument	(none)	Fires when a node is being removed from a document	No	No
DOMNodeInsertedIntoDocument	(none)	Fires when a node is being inserted into a document	No	No
DOMAttrModified	(none)	Fires when an attribute has been modified	Yes	No
DOMCharacterDataModified	(none)	Fires when the character data has been modified	Yes	No

Duplicate of #1074


Update Infinium tutorial

Thank you for contributing to the GTN! :yellow_heart:
Please check that:

[x] Your pull request has a good title, e.g.

"Fix typo in ansible-galaxy tutorial"
"Add new transcriptomics tutorial covering a new sequencing technology"

[x] Please replace this entire pull request message with a description of your changes, it will help us review your pull request faster
[x] Check that your images are allowed to be re-hosted by the GTN

Once these are done, you're ready to go!

Consider sustainable computing with Draft Mode :green_heart:
Is your pull request finished? 100% ready to be merged? Or do you want some time to work on it first (which we encourage! It's great to have visibility)
If not, then please consider creating this pull request as a draft.
Regular pull requests will trigger automated test runs when they are created
and every time you update them.
As a small contribution to sustainable computing, we skip the most energy intensive of these tests for pull requests marked as drafts. Once your pull request is ready to go, then you can click a button saying "Ready for Review" and the tests will be run!

Ready for Review

Hi @hexylena
After updating the YAML file with the output tests I keep getting this error that talks about API permission.
Can you please advise on that?
https://github.com/galaxyproject/training-material/actions/runs/6611382915/job/17955148162?pr=4439

@khaled196 sure, of course!
The API permission issue is just that reviewdog does not have permissions to publish the review as comments directly on your PR, due to github token permissions. That said, the error messages are still valid, you can find them in the 'files' tab (ctrl-f for reviewdog). One is just a simple file name issue, and the other was the workflow license which you added in a subsequent commit. But once you rename the file this is ready to go and I'll merge it! Thanks for this contribution ! :)

Looks great @khaled196 and great workflow tests too, great to have them so detailed and actually testing outputs!


Treatment of whitespace is inconsistent/inconvenient

Consider a python snippet like the following with the cursor placed at the caret:
print( "a" , "b" , "c" )
            ^

If you try to select parameter.inner or parameter.outer, nothing will happen.
If the cursor is in a different location:
print( "a" , "b" , "c" )
                ^

Now parameter.outer will select something (the "b" parameter up to its comma, but not its leading space),
but parameter.inner still does nothing.
This is probably technically correct according to the way the syntax tree maps to tokens, but it's not very convenient for editing.  You'll get similar behavior if you happen to be in whitespace for other text objects, e.g.:
def foo(bar):
    print(bar)
  ^

Here function.outer will select the entire function and body as expected, but function.inner selects nothing.
When the cursor is on whitespace, it should probably "snap" to the next non-whitespace token before looking for the surrounding text object.

The behavior is even weirder when selecting function.inner in the beginning whitespace of a nested function: function.outer selects the inner function, whereas function.inner selects the outer function.

At the moment we have no treatment of whitespaces at at all in this plugin. I always wanted to do this finally but I've never come to point to actually handle it. But I hope that at least some weirdness regard parameter.{inner/outer} got fixed by https://github.com/nvim-treesitter/nvim-treesitter-textobjects/pull/87. For all other white space issues I would really recommend to submit a whitespace handling strategy to this plugin because at the moment we don't have any.

I just tried to fix some whitespace stuff here for parameter.inner and outer, (#87). The problem basically is that treesitter "smartly" ignores whitespace. The parameter node in the syntax tree doesn't include the white space. The only way to capture whitespace (that i know) is to make a range from two nodes, which will capture the whitespace between those. This is why outer sometimes works in the cases you mentioned but inner generally won't
If you know a way to capture pre or post whitespace (without capturing the prev/next token) then we could fix a lot of these problems. I'd be happy to do the work if someone finds a technique

This is already fixed, you need to set the lookahead option to true https://github.com/nvim-treesitter/nvim-treesitter-textobjects/blob/f51c3feb5abe1dea556c9076b8a8928d3bf56790/doc/nvim-treesitter-textobjects.txt#L21.

Look ahead doesn't work.  In my second example, it would actually have to look backwards to find the parameter.

@stsewd can you reopen this, please?


local() file not uploaded to storage

@johanneskoester I was able to get my workflow to run by designating the pi_MPI.c as local, e.g., the input here:
rule compile:
    input:
        local("pi_MPI.c"),
    output:
        "pi_MPI",
    log:
        "logs/compile.log",
    resources:
        mem_mb=0,
    shell:
        "mpicc -o {output} {input} &> {log}"

but then it doesn't seem to upload to storage and down to the worker (and is not present in my working directory):
Retrieving input from storage.
Using shell: /usr/bin/bash
Provided cores: 4
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=0, mem_mib=0
Select jobs to execute...
Execute 1 jobs...
[Sat Dec  9 05:05:23 2023]
localrule compile:
    input: pi_MPI.c
    output: s3://snakemake-testing-llnl/pi_MPI (send to storage)
    log: s3://snakemake-testing-llnl/logs/compile.log (send to storage)
    jobid: 0
    reason: Forced execution
    resources: mem_mb=0, mem_mib=0, disk_mb=<TBD>, tmpdir=/tmp
...
  File "/opt/conda/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/snakemake/jobs.py", line 739, in prepare
    await wait_for_files(
  File "/opt/conda/lib/python3.11/site-packages/snakemake/io.py", line 922, in wait_for_files
    raise IOError(
OSError: Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
pi_MPI.c

I tried adding a copy step, but I don't think it worked:
rule copy:
    input:
        local("pi_MPI.c"),
    output:
        "pi_MPI.c",
    log:
        "logs/copy.log",
    resources:
        mem_mb=0,
    shell:
        "cp {input} {output} &> {log}"

I thought that might get me a little further, but it still couldn't find it:
    |     (await self.storage_object.managed_mtime()) if self.is_storage else None
    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/conda/lib/python3.11/site-packages/snakemake_interface_storage_plugins/storage_object.py", line 158, in managed_mtime
    |     raise WorkflowError(f"Failed to get mtime of {self.query}", e)
    | snakemake_interface_common.exceptions.WorkflowError: Failed to get mtime of s3://snakemake-testing-llnl/pi_MPI.c
    | ClientError: An error occurred (404) when calling the HeadObject operation: Not Found
    +------------------------------------

To step back - how do I tell snakemake to take my local file, put it in storage, and then use it for this step?

in principle you had the right intuition. I will have to test why this did not work.

Actually, jobs with local input files should automatically become local jobs as well, and not run in the cloud.

So, two things to change here. I will try to do that ASAP.

Fix should be here: https://github.com/snakemake/snakemake/pull/2541

okay I tried the run last night (with the copy step) and the file was 404 in storage - I'll try it again today (I think my computer shut down last night and maybe that was related).

Update (ran again)! Here is the current Snakefile:
# https://github.com/snakemake/snakemake/blob/main/tests/test_slurm_mpi/Snakefile
# Note that in reality, the mpi, account, and partition resources should be specified
# via --default-resources, in order to keep such infrastructure specific details out of the
# workflow definition.

localrules:
    all,
    clean,
    copy,

rule all:
    input:
        "pi.calc",

rule clean:
    shell:
        "rm -f pi.calc"

rule copy:
    input:
        local("pi_MPI.c"),
    output:
        "pi_MPI.c",
    log:
        "logs/copy.log",
    resources:
        mem_mb=0,
    shell:
        "cp {input} {output} &> {log}"

# TODO need to flag this with a wrapper only
rule compile:
    input:
        "pi_MPI.c",
    output:
        "pi_MPI",
    log:
        "logs/compile.log",
    resources:
        mem_mb=0,
    shell:
        "mpicc -o {output} {input} &> {log}"

rule calc_pi:
    input:
        "pi_MPI",
    output:
        "pi.calc",
    log:
        "logs/calc_pi.log",
    resources:
        mem_mb=0,
        tasks=1,
        mpi="mpiexec",
    shell:
        # todo where does ppn go?
        "{resources.mpi} -hostfile $BATCH_HOSTS_FILE -n {resources.tasks} {input} 10 > {output} 2> {log}"

I can see in my terminal that it knows to send to storage:
[Mon Dec 11 10:52:52 2023]
localrule copy:
    input: pi_MPI.c
    output: s3://snakemake-testing-llnl/pi_MPI.c (send to storage)
    log: s3://snakemake-testing-llnl/logs/copy.log (send to storage)
    jobid: 3
    reason: Missing output files: s3://snakemake-testing-llnl/pi_MPI.c (send to storage)
    resources: tmpdir=/tmp, mem_mb=0, mem_mib=0

The log is empty, and I don't see the file in storage.
To step back - is there any reason this small file (in my local PWD) would not be uploaded into the working directory context? Why do I need the explicit copy step? And given I have the copy step, why does it seem to report "green" that it's working but the file isn't there? :thinking: Let me know what I might be doing wrong / what we should try next.

Mhm, this is definitely some kind of bug. It will not "automatically" just upload any local files. It would rather automatically turn jobs with local files into local jobs (not running in the remote executor but on the host).
Can you please post the full log? I think it is related to the upload logic for local jobs. There might be a bug.

Can you please post the full log? I think it is related to the upload logic for local jobs. There might be a bug.

The full log for copy? It appears to be an empty file:
cat example/hello-world-intel-mpi/.snakemake/storage/s3/snakemake-testing-llnl/logs/copy.log
# no output

Here is what I see in the terminal:
$ snakemake --jobs 1 --executor googlebatch --googlebatch-region us-central1 --googlebatch-project llnl-flux --default-storage-provider s3 --default-storage-prefix s3://snakemake-testing-llnl --googlebatch-snippets intel-mpi
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided remote nodes: 1
Job stats:
job        count
-------  -------
all            1
calc_pi        1
compile        1
copy           1
total          4

Select jobs to execute...
Execute 1 jobs...

[Mon Dec 11 10:52:52 2023]
localrule copy:
    input: pi_MPI.c
    output: s3://snakemake-testing-llnl/pi_MPI.c (send to storage)
    log: s3://snakemake-testing-llnl/logs/copy.log (send to storage)
    jobid: 3
    reason: Missing output files: s3://snakemake-testing-llnl/pi_MPI.c (send to storage)
    resources: tmpdir=/tmp, mem_mb=0, mem_mib=0

[Mon Dec 11 10:52:53 2023]
Finished job 3.
1 of 4 steps (25%) done
Select jobs to execute...
Execute 1 jobs...

[Mon Dec 11 10:52:53 2023]
rule compile:
    input: s3://snakemake-testing-llnl/pi_MPI.c (retrieve from storage)
    output: s3://snakemake-testing-llnl/pi_MPI (send to storage)
    log: s3://snakemake-testing-llnl/logs/compile.log (send to storage)
    jobid: 2
    reason: Missing output files: s3://snakemake-testing-llnl/pi_MPI (send to storage); Input files updated by another job: s3://snakemake-testing-llnl/pi_MPI.c (retrieve from storage)
    resources: tmpdir=<TBD>, mem_mb=0, mem_mib=0

🌟 Setup Command:
export HOME=/root
export PATH=/opt/conda/bin:${PATH}
export LANG=C.UTF-8
export SHELL=/bin/bash

sudo yum update -y
sudo yum install -y wget bzip2 ca-certificates gnupg2 squashfs-tools git
cat <<EOF > ./Snakefile
# https://github.com/snakemake/snakemake/blob/main/tests/test_slurm_mpi/Snakefile
# Note that in reality, the mpi, account, and partition resources should be specified
# via --default-resources, in order to keep such infrastructure specific details out of the
# workflow definition.

localrules:
    all,
    clean,
    copy,

rule all:
    input:
        "pi.calc",

rule clean:
    shell:
        "rm -f pi.calc"

rule copy:
    input:
        local("pi_MPI.c"),
    output:
        "pi_MPI.c",
    log:
        "logs/copy.log",
    resources:
        mem_mb=0,
    shell:
        "cp {input} {output} &> {log}"

# TODO need to flag this with a wrapper only
rule compile:
    input:
        "pi_MPI.c",
    output:
        "pi_MPI",
    log:
        "logs/compile.log",
    resources:
        mem_mb=0,
    shell:
        "mpicc -o {output} {input} &> {log}"

rule calc_pi:
    input:
        "pi_MPI",
    output:
        "pi.calc",
    log:
        "logs/calc_pi.log",
    resources:
        mem_mb=0,
        tasks=1,
        mpi="mpiexec",
    shell:
        # todo where does ppn go?
        "{resources.mpi} -hostfile $BATCH_HOSTS_FILE -n {resources.tasks} {input} 10 > {output} 2> {log}"
EOF
cat ./Snakefile
sleep $BATCH_TASK_INDEX

# Note that for this family / image, we are root (do not need sudo)
yum update -y && yum install -y cmake gcc tuned ethtool

# This ONLY works on the hpc-* image family images
google_mpi_tuning --nosmt
# google_install_mpi --intel_mpi
google_install_intelmpi --impi_2021
source /opt/intel/mpi/latest/env/vars.sh

# This is where they are installed to
# ls /opt/intel/mpi/latest/
export PATH=/opt/intel/mpi/latest/bin:$PATH
MPI_LD_PATH=/opt/intel/mpi/latest/lib:/opt/intel/mpi/latest/lib/release
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${MPI_LD_PATH}
# Only the main job should install conda (rest can use it)
echo "I am batch index ${BATCH_TASK_INDEX}"
export PATH=/opt/conda/bin:${PATH}
if [ $BATCH_TASK_INDEX = 0 ] && [ ! -d "/opt/conda" ] ; then
    workdir=$(pwd)
    url=https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
    wget ${url} -O ./miniconda.sh
    chmod +x ./miniconda.sh
    bash ./miniconda.sh -b -u -p /opt/conda
    rm -rf ./miniconda.sh
    conda config --system --set channel_priority strict
    which python
    /opt/conda/bin/python --version
    url=https://github.com/snakemake/snakemake-interface-common
    git clone --depth 1 ${url} /tmp/snakemake-common
    cd /tmp/snakemake-common
    /opt/conda/bin/python -m pip install .
    url=https://github.com/snakemake/snakemake-interface-executor-plugins
    git clone --depth 1 ${url} /tmp/snakemake-plugin
    cd /tmp/snakemake-plugin
    /opt/conda/bin/python -m pip install .
    git clone --depth 1 https://github.com/snakemake/snakemake /tmp/snakemake
    cd /tmp/snakemake
    /opt/conda/bin/python -m pip install .
    cd ${workdir}
fi

🐍 Snakemake Command:

export HOME=/root
export PATH=/opt/conda/bin:${PATH}
export LANG=C.UTF-8
export SHELL=/bin/bash

$(pwd)
ls
which snakemake || whereis snakemake
export PATH=/opt/intel/mpi/latest/bin:$PATH
MPI_LD_PATH=/opt/intel/mpi/latest/lib:/opt/intel/mpi/latest/lib/release
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${MPI_LD_PATH}

find /opt/intel -name mpicc

# This is important - it won't work without sourcing
source /opt/intel/mpi/latest/env/vars.sh

if [ $BATCH_TASK_INDEX = 0 ]; then
  ls
  which mpirun
  echo "pip install --target '.snakemake/pip-deployments' snakemake-storage-plugin-s3 && python -m snakemake --deploy-sources s3://snakemake-testing-llnl/snakemake-workflow-sources.0031744e6280b8613a91533c9ebe9530ee78e44f17f78854c9e678a1971d4203.tar.xz 0031744e6280b8613a91533c9ebe9530ee78e44f17f78854c9e678a1971d4203 --default-storage-prefix 's3://snakemake-testing-llnl' --default-storage-provider 's3' --storage-s3-retries 5 && python -m snakemake --snakefile 'Snakefile' --target-jobs 'compile:' --allowed-rules 'compile' --cores 'all' --attempt 1 --force-use-threads  --resources 'mem_mb=0' 'mem_mib=0'  --force --target-files-omit-workdir-adjustment --keep-storage-local-copies --max-inventory-time 0 --nocolor --notemp --no-hooks --nolock --ignore-incomplete --rerun-triggers mtime params input code software-env --conda-frontend 'mamba' --shared-fs-usage 'none' --wrapper-prefix 'https://github.com/snakemake/snakemake-wrappers/raw/' --latency-wait 5 --scheduler 'greedy' --storage-s3-retries 5 --default-storage-prefix 's3://snakemake-testing-llnl' --default-storage-provider 's3' --default-resources 'tmpdir=system_tmpdir' --mode 'remote'"
  pip install --target '.snakemake/pip-deployments' snakemake-storage-plugin-s3 && python -m snakemake --deploy-sources s3://snakemake-testing-llnl/snakemake-workflow-sources.0031744e6280b8613a91533c9ebe9530ee78e44f17f78854c9e678a1971d4203.tar.xz 0031744e6280b8613a91533c9ebe9530ee78e44f17f78854c9e678a1971d4203 --default-storage-prefix 's3://snakemake-testing-llnl' --default-storage-provider 's3' --storage-s3-retries 5 && python -m snakemake --snakefile 'Snakefile' --target-jobs 'compile:' --allowed-rules 'compile' --cores 'all' --attempt 1 --force-use-threads  --resources 'mem_mb=0' 'mem_mib=0'  --force --target-files-omit-workdir-adjustment --keep-storage-local-copies --max-inventory-time 0 --nocolor --notemp --no-hooks --nolock --ignore-incomplete --rerun-triggers mtime params input code software-env --conda-frontend 'mamba' --shared-fs-usage 'none' --wrapper-prefix 'https://github.com/snakemake/snakemake-wrappers/raw/' --latency-wait 5 --scheduler 'greedy' --storage-s3-retries 5 --default-storage-prefix 's3://snakemake-testing-llnl' --default-storage-provider 's3' --default-resources 'tmpdir=system_tmpdir' --mode 'remote'
fi
Job projects/llnl-flux/locations/us-central1/jobs/compile-4c2d49 has state SCHEDULED
STATUS_CHANGED: Job state is set from QUEUED to SCHEDULED for job projects/1040347784593/locations/us-central1/jobs/compile-4c2d49.
Job projects/llnl-flux/locations/us-central1/jobs/compile-4c2d49 has state SCHEDULED

You can safely ignore everything after copy - that is a lot of extra printing for my own FYI as I develop! Also note there are a bunch of these errors all along the way (I wonder if that is related to the file not uploading)?
/home/vanessa/Desktop/Code/snek/snakemake-executor-plugin-googlebatch/env/lib/python3.11/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'snakemake-testing-llnl.s3.amazonaws.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

I can confirm that I don't see "copy" as a job in batch anymore, so it is running locally. Also I don't know if this is relevant, but I see that file in the logs directory? (I didn't put it there!) :laughing:
$ cat example/hello-world-intel-mpi/.snakemake/storage/s3/snakemake-testing-llnl/
logs/     pi_MPI.c

Ok, I think I have fixed it in the main branch. The storage upload was deactivated for local jobs. Now it should properly upload the files.
Regarding this:

/home/vanessa/Desktop/Code/snek/snakemake-executor-plugin-googlebatch/env/lib/python3.11/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'snakemake-testing-llnl.s3.amazonaws.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

It is not related, but I would definitely love to fix this.

It is a boto issue: https://github.com/boto/botocore/issues/2630

That worked! We made it through the compile step, and I see pi_MPI in storage. Now it looks like the mpiexec (same workflow above) is failing but I don't see any error why:

The error I see after that is a failure to delete it from storage (but it's an output file so it wasn't generated yet). This is what I see locally, and I can't see any output files generated:
Job projects/llnl-flux/locations/us-central1/jobs/compile-23bd66 has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/compile-23bd66 has state SUCCEEDED
[Tue Dec 12 05:52:34 2023]
Finished job 2.
2 of 4 steps (50%) done
Select jobs to execute...
Execute 1 jobs...

[Tue Dec 12 05:52:34 2023]
rule calc_pi:
    input: s3://snakemake-testing-llnl/pi_MPI (retrieve from storage)
    output: s3://snakemake-testing-llnl/pi.calc (send to storage)
    log: s3://snakemake-testing-llnl/logs/calc_pi.log (send to storage)
    jobid: 1
    reason: Missing output files: s3://snakemake-testing-llnl/pi.calc (send to storage); Input files updated by another job: s3://snakemake-testing-llnl/pi_MPI (retrieve from storage)
    resources: tmpdir=<TBD>, mem_mb=0, mem_mib=0, tasks=1, mpi=mpiexec

🌟 Setup Command:

🐍 Snakemake Command:
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
STATUS_CHANGED: Job state is set from QUEUED to SCHEDULED for job projects/1040347784593/locations/us-central1/jobs/calc-pi-0689fe.
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state SCHEDULED
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state RUNNING
Job projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe has state FAILED
[Tue Dec 12 06:01:57 2023]
Error in rule calc_pi:
    message: Google Batch job 'projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe' failed. For further error details see the cluster/cloud log and the log files of the involved rule(s).
    jobid: 1
    input: s3://snakemake-testing-llnl/pi_MPI (retrieve from storage)
    output: s3://snakemake-testing-llnl/pi.calc (send to storage)
    log: s3://snakemake-testing-llnl/logs/calc_pi.log (send to storage), .snakemake/googlebatch_logs/calc_pi.log (check log file(s) for error details)
    shell:
        mpiexec -hostfile $BATCH_HOSTS_FILE -n 1 .snakemake/storage/s3/snakemake-testing-llnl/pi_MPI 10 > .snakemake/storage/s3/snakemake-testing-llnl/pi.calc 2> .snakemake/storage/s3/snakemake-testing-llnl/logs/calc_pi.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    external_jobid: projects/llnl-flux/locations/us-central1/jobs/calc-pi-0689fe

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-12T054309.090988.snakemake.log
WorkflowError:
At least one job did not complete successfully.

I'm guessing mpiexec returned exit code non zero, but it's not clear why (it doesn't give me a log or clear error but just tells me it can't find the output for storage). Is there a good way to debug this?

Indeed mpiexec seems to fail. I would try it locally first, that should be easier to debug. I am by no means an mpi expert, so that I am of little help here. Have you checked that log file you pipe into? (Could be that it is not uploaded right? This is an error I can fix)

I think I have a fix here that should make the log file available:
https://github.com/snakemake/snakemake/pull/2545

okay ran that - where should it be?  I don't see anything on the local machine.
 ls .snakemake/storage/s3/snakemake-testing-llnl/logs/
copy.log

And the log referenced:
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-12-15T084605.597382.snakemake.log
WorkflowError:
At least one job did not complete successfully.

.snakemake/log/2023-12-15T084605.597382.snakemake.log is available but just the same content as what I see in the terminal. Is there something in s3 that isn't local?

Looking at the code - I think there might be! I can't connect to my storage atm (VPN not working) but will check a bit later and update here.

okay I was able to log into the storage, but I still only see the copy.log (so I don't think it's saving anything). I think I'll ask shamel how to debug this.

okay I'm going to try first removing all piping into a log (to see if I can see in the console). And then I'm going to live dangerously and set everything up, try adding a sleep, and see if I can shell interactively into an instance.

I think I see a possible error - the executable was not made executable it seems:

[proxy:0:0@calc-pi-784bed-31d00cde-6651-402c-8320-group0-0-cx12] HYD_spawn (../../../../../src/pm/i_hydra/libhydra/spawn/intel/hydra_spawn.c:151): execvp error on file .snakemake/storage/s3/snakemake-testing-llnl/pi_MPI (Permission denied)

That's strange, I'll look at the file and make sure it's actually a binary, and then ensure it is made executable.

okay this particular problem seems fixed - the workflow finishes successfully but I'm debugging an empty output file. Closing -thanks for all the help here!

Can I get a rule to run remotely when one of the inputs/outputs are flagged as local?


Add game link to repository settings and README

Currently there is no link to the game inside this repository. I would suggest adding the link to the repository settings and inside the README. This will allow players searching for TAO on Github to be directed to the site.
I've added this to the README in this pull request.

Accepted


Add CommComm Section to Node.js Website

I recently noticed that on the Node.js website there's a larger section on Governance that discusses the TSC at length, and a section about Working Groups. Neither of these sections mentions the Community Committee (or the Board, for that matter).
I'd like to propose we come up with some copy for the CommComm and work on PRing it into the website WG. Additionally, with the way that WGs work under the CommComm, it may be smart to PR a section for CommComm WGs into the Working Groups page.
Would also like to see if @nodejs/website has any feedback 🙏

Going to remove cc-agenda label for now - we've discussed, and it's a known task. If anyone has time/cycles to work on this, we'd be happy to help with questions/comments if you need feedback or guidance. ❤

I'd actually like giving it a try, but I'll definitely need guidance, advices, and inspiration! I just don't want to overstep on anyone...

@bnb, just got back from a war room in NYC with I'm finally getting free cycles (what is sleep again?). I have the site up and running with a new page added for CommComm – just need content now. How would you like to move forward with drafting it, should I just take a stab at it and put it in PR?

Heyo, heard that there might be the need for some help with this. I'm just an observer for now, but willing to PR / assist with whatever.

@bnb, @rachelnicole Prelim PR is up! Feel free to start wordsmithing, I left a few comments to kick off the process: https://github.com/nodejs/nodejs.org/pull/1465#pullrequestreview-75930192
Largely mimicked the TSC's page – figured we can save work on a fancier layout for when the site gets a bigger IA redesign.

thanks @amiller-gh ! Checking it out now. :D

Wanted to give an update:
Looks like https://github.com/nodejs/nodejs.org/pull/1465 was merged! 💥
From here, I'd like to ask if there are any additional steps we'd like to take, or if what's up is sufficient.
Thoughts?

Asked in today's meeting and no further feedback was provided. Going to close this 👍


Fix 'Copy Attribution' buttons on safari

As mentioned in #277 by @janeatcc that Copy Attribution buttons aren't working properly on Safari.
The problem was using the document.execCommand("copy") API and it isn't supported in Safari.
I tried to find a working solution for it, and I've attached a reference to it.

Hi @brenoferreira ,
It's my first time to hear about clipboard.js, but I will read its docs and try to implement it. Can you give me a chance to do so? Thanks!

Marked as in progress since @elharony is working on it.

Hi @brenoferreira ,
Here are my the latest updates about clipboard.js:

We can't determine the type of copy we want. (i.e. text/html or text/plain). So, everything will be copied as Rich Text. Can't make it Plain or HTML as clipboard.js get triggered once we click on the Copy button, pointing to its target Image Attribution Paragraph, which means - We can't even hard-coded this part while using clipboard.js.
Other developers already mentioned this problem here
The clipboard.js author opened an issue about upgrading it to V3.0 since 23 Jan 2019 and it's not released yet.

It seems to be a trade-off here. What do you think? Should we stick with the current one that might have some problems with a few Safari Versions? Or use - almost - fully supported one, and lose a major feature of the page (HTML Code, Plain Text)?
Until I hear back from you, I will try to find a better alternative! 🔍

This is fixed in #323
Thank you for your contribution

Thanks for your support & guidance, @brenoferreira ! 🙏


Self signed certificate in certificate chain error - Variable Group - Azure Key Vault

I have a new release I'm working on, it uses a Variable group that is linked to an Azure key vault. The key vault is accessed and the variables are downloaded without issue in one of the deployment phases(once previously existed in VSTS), but on the phase which uses a new deployment group definition for servers that haven't been used before. I'm getting the error below when it tries to access the key-vault variable group. I do have another Release setup that is working for a different deployment group but I'm not sure what to look at to see what the difference is between the deployment group / servers that work and the new one that doesn't.
Is there another step I'm missing on the deployment group definition or on the agent ?

2018-08-29T21:51:20.6612146Z ##[section]Starting: Download secrets: mykeyvault-name
2018-08-29T21:51:20.6612146Z ==============================================================================
2018-08-29T21:51:20.6612146Z Task         : Azure Key Vault
2018-08-29T21:51:20.6612146Z Description  : Download Azure Key Vault Secrets
2018-08-29T21:51:20.6612146Z Version      : 1.0.21
2018-08-29T21:51:20.6612146Z Author       : Microsoft Corporation
2018-08-29T21:51:20.6612146Z Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=848891)
2018-08-29T21:51:20.6612146Z ==============================================================================
2018-08-29T21:51:21.0012350Z Key vault name: mykeyvault-name.
2018-08-29T21:51:21.0062353Z Downloading secret value for: MYDATABASE-LOGIN.
2018-08-29T21:51:21.0312368Z Downloading secret value for: MYDATABASE-PASSWORD.
2018-08-29T21:51:21.1612446Z ##[error]
MYDATABASE-LOGIN: "self signed certificate in certificate chain"
MYDATABASE-PASSWORD: "self signed certificate in certificate chain"
2018-08-29T21:51:21.1612446Z ##[section]Finishing: Download secrets: mykeyvault-name

Please create an issue in the azure-pipelines-tasks (was vsts-tasks) repo and we'll get over to the right team.

Having the same issue, was this cas ever solved?


Adds getDependencyByName method

Coverage remained the same at 100.0% when pulling 462f5d17d7417015d0a72b77b90743fdc58da473 on addGetMethod into 26c1755ba21a570fe96bc900d8af50838b509650 on master.


Avoid reliance on kernel version

Fixes #212

If you're ok with this new way of doing things I'll continue, just give me the go ahead.

Great work! Please do continue.

I think I finally got it right, please review!

What about versions earlier than 2.7, where choices_as_values didn't
exist? Unless I'm missing some, flipping the array will change the behavior
in that case.
On Mon, Feb 13, 2017, 17:32 Grégoire Paris<EMAIL_ADDRESS>wrote:

@greg0ire commented on this pull request.
In Form/ChoosePaymentMethodType.php
https://github.com/schmittjoh/JMSPaymentCoreBundle/pull/213:

             $options['choices'][$label] = $method;

           if (Legacy::needsChoicesAsValuesOption()) {

               $options['choices_as_values'] = true;

this changes the behaviour
The option is change, but the contents of the array is changed too (it is
flipped). So the result is the same, w/o the deprecation error
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/schmittjoh/JMSPaymentCoreBundle/pull/213, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAhmAaype4TO-AK4y_4MmG5dTrW12LcZks5rcIWZgaJpZM4L7PF3
.

I reviewed this and it looks good but I wanted to test it in an example
app, using sf 2.3, 2.7 and 3.0, to really make sure behavior hasn't
changed. I haven't had a chance to do that yet, my sincere apologies.
Any chance you'll have the availability to perform these tests yourself?
On Tue, Mar 7, 2017, 09:17 Grégoire Paris<EMAIL_ADDRESS>wrote:

@greg0ire commented on this pull request.
In Form/ChoosePaymentMethodType.php
https://github.com/schmittjoh/JMSPaymentCoreBundle/pull/213#discussion_r104620630
:

             $options['choices'][$label] = $method;

           if (Legacy::needsChoicesAsValuesOption()) {

               $options['choices_as_values'] = true;

ping @regularjack https://github.com/regularjack … how can I get this
merged?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/schmittjoh/JMSPaymentCoreBundle/pull/213#discussion_r104620630,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAhmAaiOTQuS5gZzSRGVUuRrSu350J-fks5rjSDCgaJpZM4L7PF3
.

Any chance you'll have the availability to perform these tests yourself?

I don't think so, no… sorry.

Thank you for the contribution.

Thanks for merging!


PHAsset for a newly taken media

Feature
Provide PHAsset for a newly taken media.
Idea: There's a method for saving the image in YPPhotoSaver. we could get the asset after saving is completed.
related to #196

Too old issue. Feel free to open new.


What semantics do we want for the MeTTa import op?

From the implementation of ImportOp
"Import" can mean several (3) different things.  In Python parlance, it can mean

"import module" opt. ("as bar")
"from module import foo" opt. ("as bar")
"from module import *"

Do we want one MeTTa operation with multiple ways of invoking it?  Or do we want different
implementations for different versions of the import operation?  (since we don't have key-words)
like "from" and "as" (unless we want to add them)
The old version of this operation supported 1. or 3., depending on whether a "space" argument
mapped to an atom that already existed or not.  If the space atom existed and was a Space, then
the operation would perform behavior 3.  Otherwise it would perform behavior 1.
For now, I have only implemented behavior 1 in this operation.
Underlying functionality for behavior 2 exists in MettaMod::import_item_from_dependency_as, but it isn't called yet because I wanted to discuss the way to expose it as a MeTTa op.
For behavior 3, there are deeper questions about desired behavior around transitive imports, Tokenizer entries, etc.  I have summarized those concerns in the discussion comments above MettaMod::import_all_from_dependency

I've come to feel somewhat strongly that import should be renamed to use to avoid the various (and often contradicting) expectations voiced by many people about import is supposed to work.

I've come to feel somewhat strongly that import should be renamed to use to avoid the various (and often contradicting) expectations voiced by many people about import is supposed to work.

I don't have strong opinion here except we need accurately migrate the code. I also feel that main issue is semantics of the import is not explained well and fixed. For instance nobody expects from import in Python that it behaves as include. So we need to fix and clearly explain these semantics on the first place. Renaming import to use is a second priority and we may do this to if @Necr0x0Der doesn't have objections.

I agree it makes no sense to do it now.  But when we redo the arguments, I believe the name should change as well.

If we change the signature of import in such a way that it will be incompatible with the existing code, then renaming will be indeed a good idea to reduce confusion.


TabNine is still unavaliable in VS2022

Hi!
Currently, Tabnine exists for Visual Studio 2019.
However, we do plan to expand to VS 2022. If we'd see enough interest from the community, we will make sure such a plugin exists.
Originally posted by @maya-tabnine in https://github.com/codota/TabNine/issues/476#issuecomment-968785085
gz#15917

I see the issue last year, but till now Tabnine is not in VS2022's markplace, I really want to try it in VS2022

Hi! You are correct. I get many requests for VS 2022, which is considered to be supported.
I am sorry I don't have any concrete news about that since a year ago. When we do add it, I will make sure to publish it here!


Name and icon for restoring graph button changed

@pradeeban
Can be tested here: https://rahuljagwani.github.io/concore-editor/

Perfect. Thanks.

@Rahuljagwani Nice Icon, but I think its thickness is not consistent with other icons

@parteekcoder will make an excellent QA tester. :) I did not notice. But he is right.
This is not urgent. But when you have time, please replace it with an icon that is the same size and thickness as the others.

@parteekcoder will make an excellent QA tester. :)

I didn't understand that @pradeeban

@parteekcoder, I was just complimenting your QA (Quality Assurance) capabilities. Thanks. :)

@parteekcoder, I was just complimenting your QA (Quality Assurance) capabilities. Thanks. :)

Thanks @pradeeban


⚠ Bofrost.es has degraded performance

In 57df300, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1566 ms

Resolved: Bofrost.es performance has improved in edd3334 after 13 minutes.


fix: make the trait Task Send/Sync

Due to storage of Task in a Box in an Arc, the task can be declared Send/Sync. https://rust-lang.github.io/rust-clippy/master/index.html#/arc_with_non_send_sync

@genedna I don't use cryptographic signatures in my commits. Is this really a requirement for contribution to this repository?


Update GitHub workflows and add PowerShell script to use Ingest API

Support to use "Azure Monitor Ingest API" for pushing AzADSPI data to a custom table of Microsoft Sentinel/Log Analytics workspace.

@cjtous1 oh my, that was the xmas grinch.. will check.. thanks for call out


Update specs2-core, specs2-scalacheck to 4.12.8

Updates

org.specs2:specs2-core
org.specs2:specs2-scalacheck

from 3.+ to 4.12.8.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.specs2" } ]

labels: test-library-update

Superseded by #38.