workflow RDF

implements spec part of #477
with a few assumptions on open questions...

A preview of the workflow RDF documentation can be found here:
https://github.com/bioimage-io/spec-bioimage-io/blob/gh-pages-workflow_rdf/workflow_spec_0_2.md

adapted after furrther discussion with @oeway and @constantinpape

hey @oeway !
I was just working on the workflow RDF and found some more aspects we didn't think about before and started discussing quite a bit with @constantinpape .
We really oppose inputs with defaults as it makes it very unclear when to use inputs and when kwargs.
We iterated over several solutions and arrived at this:

no kwargs field, only inputs
an input may have a default only if all following inputs have one too
outputs of one step are always positonal args to the next (like before)
outputs of one step may be named (like before)
inputs to a step may be given as a list (like before)
inputs to a step may be given as a dict (like we gave kwargs)
if an input dict is missing inputs, these are taken from the args (outputs of the previous step) if present

no interleaving: no input following an input given in an input dict can be taken from the present args

Essentially we don't try to come up with our own logic, but mirror how python callables behave.

inputs to a step may be given as a list (like before)
inputs to a step may be given as a dict (like we gave kwargs)
...

Essentially we don't try to come up with our own logic, but mirror how python callables behave.

been thinking about this part a bit more...
we can also mix positional inputs and named inputs with list entries containing a dict with a single key (that is the name of an input):
steps:
- op: ...
  inputs:
  - arg0
  - arg1
  - key0: value0
  - key1: value1

key word order is ignored:
equivalent to above:
steps:
- op: ...
  inputs:
  - arg0
  - arg1
  - key1: value1
  - key0: value0   # key1 and key2 are given as key word arguments, their order is ignored 👍

Assuming the previous step returns [arg0, arg1], this is equivalent to the above:
steps:
- op: ...
  inputs:   # positional inputs (arg0, arg1) are taken from previous step, optional inputs may be overwritten 👍
    key0: value0
    key1: value1

but I would not allow:
steps:
- op: ...
  inputs:  # arg0 is filled in from previous step 👎
  - arg1
  - key0: value0
  - key1: value1

and maybe not even specifying optional inputs as a list like:
steps:
- op: ...
  inputs:  # arg0 and arg1 are filled in from previous step 👍? 👎?
  - key0: value0
  - key1: value1

This pattern implies that input values that are dicts with one key need to be given like this:
steps:
- op: ...
  inputs:
    arg0: {key: value}

or
steps:
- op: ...
  inputs:
  - arg0: {key: value}

because
steps:
- op: ...
  inputs:
  - key: value

would be interpreted as the optional input 'key' with value 'value' and not as the first positional argument (here arg0) with value {key: value}.
Of course we could compare the expected arg name and the given key, but I would not rely on that and mandate to specify single key dicts only as a named input.

and @oeway one more think @constantinpape and I discussed:
We should add axes to input if input type: tensor.

Hi, In the first glance, I like the idea of merging inputs and kwargs. However, when think it a bit further I find it actually making the usage much more complicated and confusing.

We really oppose inputs with defaults as it makes it very unclear when to use inputs and when kwargs.

I don't really get what this means, the rule is pretty clear to me. Same for defining a python function, if it's required, go for inputs, and if it's optionally with default, go for kwargs. For inputs, you don't need to name them, but kwargs you need to name them. inputs is a list while kwargs is a dictionary. You can of course merge the two by exploiting list of dict, or dict as you are proposing, but feels very hacky and needs a lot of additional rules when using it.

Essentially we don't try to come up with our own logic, but mirror how python callables behave.

But the actually python callables actually separates the args (a list) and kwargs (a dict). Using a list with 1-key dict or dict to encapsulate the two into one object make it feel hacky and hard to use.
In the following example:
  - op: zero_mean_unit_variance
  - op: model_inference
     kwargs:
         model_id: fearless-crab
         preprocessing: false # disable the preprocessing
         postprocessing: false # disable the postprocessing
  - op: stardist_postprocessing
     kwargs:
        diameter: 2.3

If I understand correctly, to adapt what your proposed it will become:
  - id: step1
     op: zero_mean_unit_variance
     outputs: [out1]
  - id: step2
     op: model_inference
     inputs:
         - step1.outputs.out1
         - model_id: fearless-crab
         - preprocessing: false # disable the preprocessing
         - postprocessing: false # disable the postprocessing
     outputs: [out2]
  - op: stardist_postprocessing
     inputs:
        - step2.outputs.out2
        - diameter: 2.3

Note that, because in step2 and step2 we need to set the kwargs, we are forced to name the inputs, and this propagate to all the other steps to force us set id for the first two steps and name the outputs. This introduces a lot of hassle for the naming, and making sure they are identical.
We can potentially allowing ignoring the positional inputs when we only want to change the kwargs, however, this make the inputs fields looks so confusing, since we are not writing out all the inputs.
Overall, I felt the merging of inputs and kwargs will require more rules, the list of 1-key dict feels hacky, separating them actually make more sense to me.

We should add axes to input if input type: tensor.

Sure, maybe we should bring in more tensor spec (e.g. shape) too, however, for simplicity, maybe adding axes, shape, dtype etc. as an optional field would help.

Note that, because in step2 and step2 we need to set the kwargs, we are forced to name the inputs, and this propagate to all the other steps to force us set id for the first two steps and name the outputs. This introduces a lot of hassle for the naming.

no. I would not force naming/specifying outputs if inputs is given.
Also named inputs could be overwritten without touching the positional inputs by providing them as dict:
  - id: step0
     op: zero_mean_unit_variance
  - id: step2
     op: model_inference
     inputs:
         model_id: fearless-crab
         preprocessing: false # disable the preprocessing
         postprocessing: false # disable the postprocessing
  - op: stardist_postprocessing
     inputs:
        diameter: 2.3

If positional inputs need to be specified this would become
  - id: step0
     op: zero_mean_unit_variance
     outputs: [out1]
  - id: somehting-in-between  # added this step to the example, because otherwise out1 would not need naming
     op: ...
  - id: step2
     op: model_inference
     inputs:
         - step1.outputs.out1
         - model_id: fearless-crab
         - preprocessing: false # disable the preprocessing
         - postprocessing: false # disable the postprocessing
  - op: stardist_postprocessing
     inputs:
        diameter: 2.3

to refine my idea of merging inputs and kwargs I wanted to propose a slight update:

if inputs are given as a list and the last entry is a dictionary, those are named inputs (kwargs).
This avoids a list of len 1 kwargs which indeed feels very hacky.

The above example then becomes:
  - id: step0
     op: zero_mean_unit_variance
     outputs: [out1]
  - id: somehting-in-between  # added this step to the example, because otherwise out1 would not need naming
     op: ...
  - id: step2
     op: model_inference
     inputs:
         - step1.outputs.out1
         - model_id: fearless-crab
           preprocessing: false # disable the preprocessing
           postprocessing: false # disable the postprocessing
  - op: stardist_postprocessing
     inputs:
        diameter: 2.3

However, I have to agree with @oeway . this is still hacky...
the issue we tried to address, and the confusion we mentioned before was that inputs may have defaults. So another solution to a clearer specification would be mandatory inputs:

inputs are have no defaults!
if inputs  are not specified previous outputs are given as inputs
we rename kwargs  (too python specific) to options (other suggestions welcome!), which always have a default value and a name
options are specified as a dict and always referred to by their name (key), because an arbitrary subset of them may be overwritten. This further distinguishes them from inputs and avoids confusing the two sets of parameters.
[I would like this one, but it is not a must] inputs may also be given as a dict via their names, but stay separate from the options.

Sure, maybe we should bring in more tensor spec (e.g. shape) too, however, for simplicity, maybe adding axes, shape, dtype etc. as an optional field would help.

I agree that shape and dtype may be added as optional fields, but I think we cannot handle tensors reasonably without knowing their axes. We don't want a bunch of transpose ops littered throughout the workflows which would be the consequence...
many of the ops (pre-/ and postprocessing) defined in spec and implemented in core have an axes kwarg. (mean per channel etc... )...

@oeway @constantinpape I know this is a lot to read through and think about, but it would be really convenient for me, if you could take a brief moment and just comment if we can proceed with the current state or need more discussion (if so that does not need to happen right away ;-) )
I tried to comment all discussed/proposed decisions in the following example:
inputs:
- name: ipt0
  type: tensor
  axes: bxy      # if type=='tensor' then 'axes' is mandatory

options:
- name: param0  # although options are given as a dict we specify them as a list to match inputs/outputs pattern
  type: int
  default: null   # options always need a default value (may be null or of type 'type')

outputs:
- name: ...
  type: tensor
  axes: bxy
- name: ...
  type: bool

steps:
  - id: step0
     op: zero_mean_unit_variance
     outputs: [out1]
  - id: somehting-in-between  # added this step to the example, because otherwise out1 would not need naming
     op: ...
  - id: step2
     op: model_inference
     inputs: [step1.outputs.out1]
# inputs as list or alternatively by name as dict, but either way always all inputs need to be specified:
#     inputs:
#       raw: step1.outputs.out1
     options:
         model_id: fearless-crab
         preprocessing: false # disable the preprocessing
         postprocessing: false # disable the postprocessing
  - op: stardist_postprocessing
     options:
        diameter: 2.3

test_steps:  # analog to 'steps', but do not take any inputs/options (it may call the 'steps' multiple times with different fixed inputs/options though
  - ...

@oeway:

I don't really get what this means, the rule is pretty clear to me. Same for defining a python function, if it's required, go for inputs, and if it's optionally with default, go for kwargs.

I agree with this. But when I discussed this with @FynnBe, he said that you wanted to allow default values for inputs. And this would not make sense to me. Judging from your comment here that might have been a misunderstanding.

inputs have no defaults!
if inputs are not specified previous outputs are given as inputs
we rename kwargs (too python specific) to options (other suggestions welcome!), which always have a default value and a name
options are specified as a dict and always referred to by their name (key), because an arbitrary subset of them may be > - overwritten. This further distinguishes them from inputs and avoids confusing the two sets of parameters.
[I would like this one, but it is not a must] inputs may also be given as a dict via their names, but stay separate from the options.

That sounds good to me. About the last point: I would rather not do it since I think we need to introduce more rules etc. and it makes reading everything more complex. But I don't have a strong opinion on this, so would be fine with this either way.
I also agree that we should have axes mandatory for tensors.

I know this is a lot to read through and think about, but it would be really convenient for me, if you could take a brief moment and just comment if we can proceed with the current state or need more discussion (if so that does not need to happen right away ;-) )

Looks good from my end. My time to look into this will be very limited in the next few weeks, so please don't wait on my feedback to go ahead here.

Hi @FynnBe I like this! No strong opinion about changing kwargs to options. If we do that, would it also make sense to change list/dict to array/object then? To align with the json types.

I also don't have a strong opinion here, we can also stick to kwargs to be consistent with the rest of our spec where we use this.

A minor suggestion, would it make sense to make name field under inputs optional? just to align with the ops spec, so we can use the default name self.inputs.0 (same as unnamed inputs in ops).

I think it would be better to be more explicit here and require the name if we have references to it.

First of all: Thanks for the fast feedback!

For axes, would it make sense to bring https://github.com/bioimage-io/spec-bioimage-io/issues/70#issuecomment-825737433 into the tensor spec?

yes, I think it would! I'll get on that...

I think it would be better to be more explicit here and require the name if we have references to it.

I agree, let's keep it explicit with mandatory input names. Furthermore, I would like to require the input names to be valid python identifiers for simplicity. '0'--not a valid identifier--would therefore require an exception to that rule...

self.inputs

this is a minor detail, but if you have no objections I will stay with our previous formulations and reference inputs by ${{ inputs.<input name> }} (without the self scope...

kwargs/options ... list/dict to array/object

is options a thing in JSON??
actually kwargs does not imply that it is optional, therefore I would prefer options here. (many of the kwargs for pre-/postprocessing in the model spec are mandatory)
I would stick to list/dict as I think it is much more expressive compared to array and object. array is easily confused with tensor and object is very generic.

from the BioImage.IO meeting:
named or not named that is the question.
  steps:
  - op: my_op
     inputs: [5, 1]  # positional
     inputs:  #  or named
       mean: 5
       std: 1
     inputs:  # the benefit of named inputs -- order does not matter!
       std: 1
       mean: 5

# named, mandatory inputs (options would be any kwargs with a default)
def my_op(*, mean, std):
    ...

# positional or named
def my_op(mean, std):
    ...

# if needed: in other langauges we can remember the name order to map named inputs to positional ones
y_op_arg_names = ["mean", "std"]
def my_op(arg1, arg2):
     ...

As we discussed, I think it's better to make positional inputs actually positional and only support list, not dict.
In python it directly map to args, and it will make it much easier to implement in other language, e.g. javascript too.  In other languages, introducing additional naming (as you proposed) for positional arguments is just another layer of complexity, without that, we will be able to map all the functions in a whole library and make them available as ops.
Even in Python, there are positional-only arguments (see https://peps.python.org/pep-0570/ and you can find the good motivation on why this exists).
def equation(x , y , / , z):
    return x * y + z
#The arguments to the left of '/' (in this case x and y) can only be passed postionally whereas the argument to the right of '/' can be passed positionally or with a keyword

print(equation(2 , 3 , 5)) #all three arguments are positional
print(equation(2 , 3 , z = 5)) #the first two arguments are positional while the third is a keyword argument

If you do do this, it will raise an error:
>>> equation(x=1,y=2, z=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: equation() got some positional-only arguments passed as keyword arguments: 'x, y'

There are functions (e.g. the print function) defined as:
def process(*args, **kwargs):
   ...

And the named positional arguments won't work in this case. I would see passing positional arguments as keyword arguments is really a very special feature of python, it is not very commonly used too.
Plus when the inputs to the ops are not written, we will go for the positional case anyway, right? If we allow specifying it as a dict, does that mean that a previous op can also return a dictionary and we feed it into the next op? This makes it look very confusing.
Practically, the mean and std example is not common too, they will likely be options, not inputs. In most cases, the inputs will be a single tensor, and we won't have tens of inputs in most cases. I am not sure how useful it is to have this additional feature too.
For the sake of simplicity, I would suggest making positional arguments really positional, names doesn't matter when calling the ops.

One more thing, I think it's might be better to change the inputs and outputs under root as inputs_spec and outputs_spec, so we are clear this is the type annotation, not the actual inputs as we define for the ops in the steps.

As we discussed, I think it's better to make positional inputs actually positional and only support list, not dict.

I agree with the only positional arguments for inputs, since this will also keep things simpler.
(It also means these should never have default values, but I think we all agree on this now.)

For the sake of simplicity...

Yes agreed. Below some points I just couldn't keep to myself, feel free to ignore

Even in Python, there are positional-only arguments (see https://peps.python.org/pep-0570/ and you can find motivations on why this exists).

just to clarify the situation for Python:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
      -----------    ----------     ----------
        |             |                  |
        |        Positional or keyword   |
        |                                - Keyword only
         -- Positional only

(from https://peps.python.org/pep-0570/)
so the question is if we choose inputs to be 'positional only', 'positional or keyword', or 'keyword only'
'positional only' arguments (the pep @oeway linked) were introduced in Python 3.8.
We officially support Python 3.7, so we cannot use them in code (or we require at least py3.8, which might be ok?). Otherwise, from the Python side we only have the option between 'positional or keyword' and 'only keyword' arguments.
('keyword only' were already introduced in Python 3.0; see https://peps.python.org/pep-3102/)

def process(*args, **kwargs):
   ...

And the named positional arguments won't work in this case. I

Even if we only accept key word arguments, one can simply add args as a kwarg (maybe name it 'tensors' or sometheing else more expressive though):
def process(*, args: list, **kwargs):
   ...

in the workflow RDF this would look like:
steps:
- op: ...
  inputs:
     args: [${{ ... }}, ${{ ... }}]

(if we allow both positional and key word arguments this would also not be a problem; we would simply forbid the named inputs for ops using *args, or require them being named 'args' (which would forbid using args as a kwarg name)).

(It also means these should never have default values,

I agree that we should enforce inputs never to have default values, but from the Python side this is not true:

Therefore, the following would be valid function definitions:
def name(p1, p2, /, p_or_kw, *, kw):
def name(p1, p2=None, /, p_or_kw=None, *, kw):  # 'only positional' with default
def name(p1, p2=None, /, *, kw):  # 'only positional' with default
def name(p1, p2=None, /):  # 'only positional' with default
def name(p1, p2, /, p_or_kw):
def name(p1, p2, /):

(adapted from https://peps.python.org/pep-0570/)
I still think the cleanest, most explicit would be to use 'keyword only' arguments and list all 'keyword only' args with a default as options, and all 'keyword only' without a default as inputs. (pretty much what @k-dominik suggested).
The biggest argument for this is that the order of positional arguments is generally not obvious, thus reading the worklow RDF without directly comparing it to the operator docs is less informative than if the workflow RDF always includes the input names.
In consequence any op should return a dictionary (and not a tuple as we currently assume). Thus output names need to be mapped explicitly as well.
Naming all op inputs and outputs would certainly make any workflow RDF very readable.
And if we are really that worried about boilerplate code, we can treat missing inputs/outputs as the trivial mapping to shorten cases like this:.
steps:
- op: A
  outputs:  # may be left out
     tensor: tensor
- op: B
  inputs:  # may be left out
    tensor: tensor

or
steps:
- op: A
  outputs:
     mean_tensor: tensor
- op: B
  inputs:  # may be left out
    tensor: tensor

One more thing, I think it's might be better to change the inputs and outputs under root as inputs_spec and outputs_spec, so we are clear this is the type annotation, not the actual inputs as we define for the ops in the steps.

also one more thing, op outputs are also not real outputs, but a mapping of outputs to a name. We might want to consider calling the field output_names or map_outputs... For symmetry I'd be fine with keeping it outputs, though.

Yes agreed. Below some points I just couldn't keep to myself, feel free to ignore

Good! Let's go for that!
We actually already discussed the option of doing keyword-only for the workflow during the hackathon, and we decided to go for positional for simplicity if I remember correctly.
For what it worth, I'd like to add some response here:

'positional only' arguments (the pep @oeway linked) were introduced in Python 3.8.
We officially support Python 3.7, so we cannot use them in code (or we require at least py3.8, which might be ok?).

This is exactly the reason why we should stick with 'positional only' inputs, so we will be all fine with 3.7 and 3.8+ versions. If we do keyword-only argument, it won't work with the positional only functions in python, e.g. range() in python 3.8 unless we wrap every such function and make them keyword-only. But, the question is why bother doing that if we have a way to just map them as is, and we might want to expose an entire python module as ops.
This seems to be a more logical choice to me, to reduce development/maintenance effort, and also simplify the usage for writing workflows as we already discussed.

The biggest argument for this is that the order of positional arguments is generally not obvious, thus reading the worklow RDF without directly comparing it to the operator docs is less informative than if the workflow RDF always includes the input names.

We need the docs anyway, how would one write the outputs/inputs without reading the docs if we go for keyword-only inputs? I'd argue the contrary, if we go for keyword-only, it make it harder to stack the ops, because we need to carefully read each ops to see what are the keys they accepts and what they returns. One op might return output as a key, the other one might return tensor or data, while it accept input_tensor or x as inputs. Mapping these is very much error prone, simply because we need to check and write them exactly.

And if we are really that worried about boilerplate code, we can treat missing inputs/outputs as the trivial mapping to shorten cases like this:.

steps:
- op: A
  outputs:  # may be left out
     tensor: tensor
- op: B
  inputs:  # may be left out
    tensor: ${{ A.outputs.tensor }}

The issue is that if we do keyword-only, we won't be able to left them out anymore. In the case above, we can only left out the inputs if we know exactly the key is called tensor, same as the key produced from the last step. Since we won't likely to have the exact key matching between steps, we will have to write it out all the time, and checking docs to retrieve the exact input key for every op, which is a lot more complicated and error prone.
Most importantly, with positional-only inputs and outputs, we simply don't care their name but the order. And the vast majority of the ops only accept 1 tensor in inputs and returns 1 tensor as outputs,  we won't bother writing them out at all -- this greatly simplify the writing of most workflows!
For most cases, not sure how much it will adds by writing them out as tensor: tensor, or input_tensor: A.outputs.output_tensor.
We are designing a new language here, and since most of the other programming languages uses positional arguments for function calls, and they don't see this as the source of error, I don't see why we should force it to keyword-only.
After all, we still have the options, if the developers would like to force keyword only, they can still use the options with default to null, and throw error if the value is null. As a workflow, I don't think we should force everyone to use keyword only.

One more thing, I think it's might be better to change the inputs and outputs under root as inputs_spec and outputs_spec, so we are clear this is the type annotation, not the actual inputs as we define for the ops in the steps.

also one more thing, op outputs are also not real outputs, but a mapping of outputs to a name. We might want to consider calling the field output_names or map_outputs... For symmetry I'd be fine with keeping it outputs, though.

Good points! I am fine with changing them too, but it looks nicer to use outputs as you said.

it won't work with the positional only functions in python, e.g. range() in python 3.8 unless we wrap every such function and make them keyword-only

range returns a range object--a generator. I don't think we want support for generators in the workflow definitions, so we'd have to wrap this anyway.

After all, we still have the options, if the developers would like to force keyword only, they can still use the options with default to null, and throw error if the value is null.

These implicitly mandatory inputs would lead to runtime errors which could be avoided by statically checking if invalid defaults were not present.

skimage.segmentation.watershed

if you want to support 3rd party functions out of the box, the workflow ops have no chance to be language agnostic. Additionally, if we wanted to use these we would be well advised to wrap them to be more independent of api changes in that 3rd party library, otherwise we will have to pin contrib dependencies very hard.

We are designing a new language here,

Maybe this is were we go all wrong. Maybe we just shouldn't. If implementations of ops are limited to Python, then maybe the workflows themselves should be treated just as other ops--also defined in Python. A workflow is just another op. Once we agree on how to describe inputs and outputs in terms of function signature and output structure in Python, we could just generate a high level yaml description for non-coders to browse and then focus on the bioimageio.contrib library.
In other words: what if we drop steps? Yet another way of thinking about it: Isn't Python the most suitable approach to implement steps?
I also tried to imagine writing a workflow RDF... The moment it get's complicated I would probably first write a Python script equivalent, which is much easier to debug and then translate to 'workflow RDF', but if all we do is translate the workflow RDF back to Python.... ??

Sorry, I don't have the time to read through all the discussion now, but from what I understand we actually have a viable solution that we can go with and that everyone is in principle fine with:
inputs are positional only and are mandatory (i.e. no default values!) and kwargs/options (I don't have a strong opinion on what to choose for the name) have default values and are specified by keyword. Leaving the question of how exactly this maps to python, I think this is a good first minimal solution that supports all our initial needs.

if you want to support 3rd party functions out of the box, the workflow ops have no chance to be language agnostic.
Additionally, if we wanted to use these we would be well advised to wrap them to be more independent of api changes in that 3rd party library, otherwise we will have to pin contrib dependencies very hard.

I agree that direct support of 3rd party functions is too ambitious in the beginning and will make it not possible to be language agnostic. I think we have to live with the fact that we will need to have a wrapper in these cases.

I also tried to imagine writing a workflow RDF... The moment it get's complicated I would probably first write a Python script equivalent, which is much easier to debug and then translate to 'workflow RDF', but if all we do is translate the workflow RDF back to Python.... ??

As far as I understand one of the primary goals is to enable running the same workfows in java, and @carlosuc3m has already said he wants to explore implementing these workflows in the bioimageio java libraries.
Overall my suggestion would be:

implement the minimal viable spec now (see above)
make a prototype implementation of some contrib ops and workflwos in python and usage in the bioengine
try porting these to java (@carlosuc3m)
treat all of this as a prototype without officially declaring it as part of the spec yet, after these first implementations in python and java reevaluate the design and see if we need any changes.

Hello everyone,
answering to @constantinpape :

As far as I understand one of the primary goals is to enable running the same workfows in java, and @carlosuc3m has already said he wants to explore implementing these workflows in the bioimageio java libraries.

I plan to start implementing this prototype of worflows in the JAva core next week yes. With that I guess I will be able to give some feedback on the robustness of the design.
Regarding the last point of @constantinpape's list

try porting these to java (@carlosuc3m)

In principle this should not be necessary to translate the new ops into Java .The idea is to support Python scripting and execution in Java using JEP to avoid code repetition and to maximize reproducibility. A core library that bridges Java and Python through JEP already exists and it will be soon integrated in the JAva model runner core.
The library allows automatic installation of miniconda and Python, facilitates installation of Python libraries and enables easy BioImage.io python execution. With this, Java softwares will be able to execute both JAva and Python ops (pre- and post-processing) indistinctly, just parsing what is specified on the rdf.yaml.
What do you think? Would you still rather translate some of the easy ops into JAva? I think that for some cases, such as cell-pose or stadist it does not make sense to have them in Java, but some simple ones might be good to increase the execution efficiency (using Python in JAva implies loading some native libraries).

Thanks for the clarification @carlosuc3m! I wasn't aware that the JEP integration was this advanced already.

I think that for some cases, such as cell-pose or stadist it does not make sense to have them in Java,

I agree. (For stardist there actually is a java implementation already that we can use, but indeed porting more complex algorithms is not scalable, so the JEP option is really good!)

What do you think? Would you still rather translate some of the easy ops into JAva?

I think having examples for one or two simple ops in Java and more complex ones in JEP would be really cool, but it depends on how much extra work that is for you!

As one more high-level comment: I think I mostly stay by my comments here. The one thing to consider is that we are re-inventing workflow description languages. BUT the problem with them is that there are many of them, and probably none of them will really fit our use-case. So I think it will take us longer to find a suitable one and adapt it to our purposes compared to implementing the prototype layed out here (if we keep it simple ;)).
But we should acknwoledge that what we do now is not set in stone and treat this as experimental.

most content---workflow implementations and workflow exectuion logic--was moved to https://github.com/bioimage-io/workflows-bioimage-io-python
the resolved nodes remain.


Docs.MS Release Updates for azure-data-schemaregistry

Update docs metadata and targeting for release of azure-data-schemaregistry

Docs Build status updates of commit ab5d6ee:
:clock10: Full build: the average full build time is 91 min(s) 49 sec(s), this is based on the last 782 build(s) for this repository.

Docs Build status updates of commit ab5d6ee:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-services/data-schemaregistry-readme-pre.md
:white_check_mark:Succeeded
View (azure-java-stable) View (azure-java-preview) View (azure-java-legacy)

package.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Option to show all calendar names for a duplicate event

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
When an event is duplicated on multiple calendars, it only shows the name of one calendar it is duplicated on.

Expected behaviour
The event should have the option to show the name of all calendars it is on.
An example of what it might look like:

Possible Solutions
No response
Context / Reason
This is helpful, so I know I am also invited/going to a particular event along with my wife.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

+1

+1 :)


Define specific queries on specific CDEs modelled by Care-SM

Define SPARQL queries that can be used to answer the information needs described in the use case flash card and the mindmap linked from #57 by relying on the information models defined in the Virtual Platform Specification (VIPS)[^1] with extensions only where necessary.
List of models from VIPS used (add as necessary):

EJP RD meta data model – findability of rare disease resources
Clinical And Registry Entries (CARE) Semantic Model – core data standard describing common data elements essential for RD research

List of models not in VIPS used (add as necessary):

FAIR Genomes metadata schema – semantic metadata schema to power reuse of NGS data

Queries to implement:

[ ] Query 1: …
[ ] Query 2: …

[^1]: See VIPS 2.0, page 17

This issue is to define a specific case that we can perform, using a minimal number of data elements including a minimal number of CDEs such to demonstrate the use of Care-SM in queries.
Special request

Can we do a rare disease and oncology case in parallel: this will help adoption at the local institutes. Marco will bring this up with Karolis for the LCCO project.

During biohackathon on Mai 3rd @wna-se and @andrawaag made this a place to collect query example to be transformed into SPARQL

@markwilkinson Added you here as discussed during today’s meeting. Please link to / add any queries that you can share here and / or to the ejp-rd-vp/DistributedAnalysis repository.

@NuriaQueralt For those working on Phenopackets and/or phenotypic data that could be present in case reports, the following datasets consisting of published case report translated to Phenopackets may be useful?

Peter N Robinson. (2021). Phenopackets for case reports of structural variants (Version 1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5071267
Peter N Robinson. (2020). 384 Phenopackets (1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3905420

Also, the JSON Schema validator from the phenopackets / phenopacket-tools could perhaps be a useful resource to inspire the RDF/Shacl-mapped version. Notably, the folder with the JSON Schema gives an example of how they have mirrored the structure of the authoritative protobuf definitions, the validation rules could probably be translated into Shacl, and the the choice of uris used to reference the definitions could also be useful.
Is there an issue specifically for the Phenopacket work?

@andrawaag : @mroos said that you would be a great person to take the lead on this task. As we are working on preparing the synthetic data we have for the VP it would be great to have some examples of genomic-related that we could prioritise mapping to, ideally a few queries based on the @mroos’ mindmap (see reference in the description of #57) and using the CARE-SM and FAIR Genomes semantic schema.

Update 10/6
@hbcesar, Annika, @andrawaag  working on example queries.
@pabloalarconm asked to provide example data for the queries (CSV + conversion method).

[ ] @ericprud asks to share resulting RDF into github for others in this group to use (or repo of choosing) @pabloalarconm

@NuriaQueralt For those working on Phenopackets and/or phenotypic data that could be present in case reports, the following datasets consisting of published case report translated to Phenopackets may be useful?

Peter N Robinson. (2021). Phenopackets for case reports of structural variants (Version 1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5071267
Peter N Robinson. (2020). 384 Phenopackets (1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.3905420

Also, the JSON Schema validator from the phenopackets / phenopacket-tools could perhaps be a useful resource to inspire the RDF/Shacl-mapped version. Notably, the folder with the JSON Schema gives an example of how they have mirrored the structure of the authoritative protobuf definitions, the validation rules could probably be translated into Shacl, and the the choice of uris used to reference the definitions could also be useful.
Is there an issue specifically for the Phenopacket work? Perhaps also relevant to @rosazwart ?

@andrawaag Above are two references to collections of Phenopackets that represent published case reports and could be useful source materials to produce a realistic graph to query using @NuriaQueralt and @rosazwart mapping. The synthetic data that we have been working on in Sweden is a subset of files derived from the Rare Disease Synthetic Dataset available in full from the European Genome-Phenome Archive (EGA) through accession number EGAD00001008392, see example phenopacket, PDF describing the data and the full subset as well as derived files in NBISweden/ejprd-data/ .

The CARE-SM/beaconAPI4CARESM also contains some SPARQL templates that can be used to serve a Beacon endpoint.

Hi @wna-se @mroos
Some of these tasks are tagging me in this conversation but its not clear what you need. As the main maintener of CARE-SM nowadays, what is exactly what you need from my contribution of your use case? (Probably you discussed in a meeting Im not involved to)

ShEx files for schema validation are already included at here

SPARQL queries have been always here There's two examples, but let me know if you need more cases to add here. SPARQL queries fragments from beaconAPI4CARESM are just fragments, hard to reuse in a first attempt but let me know if you need help with that (I can connect to a meeting to discuss its implementation out of this API)

I will add examplar RDF data to the CARE-SM implementation repo. DO you need to for every specific data element? Or a single example representation?

Bests,
Pablo

Pasted from e-mail by @NuriaQueralt on 20 June:

Dear all,
I have finished the phenopackets RDF model, in ShEx. You can have a look in github, in branch “”v2”. I modelled ONLY the elements required for the GDI use case. Rosa, I also modelled the Variant related elements for the LUMC data, so you can start adapting your RDFization pipeline.
Good news! We have a bunch of phenopackets that follow the current scheme version here: https://monarch-initiative.github.io/phenopacket-store/ I suggest to use these set for our POC. I may refine the model adding some RDF examples using these data, so I may do some changes to the model.
My apologies, I cannot make it to today-s meeting due to a clash in my agenda.
With kind regards,
Núria

I will add examplar RDF data to the CARE-SM implementation repo. DO you need to for every specific data element? Or a single example representation?

Ideally, we'd have a couple nice examples that demonstrated the breadth of the expressions. This will serve as documentation and inspiration for schema and queries. Such examples could be cobbled together from multiple instances of the current JSON data.
Having all the data would also be handy as it would help us verify schema and queries and provide a corpus for tests. Would also be nice for demos.


Observe Notify example not working

The client OnNotify method is never called

I'm getting "port not available" on OS X. I'm not sure if the incoming UDP request is blocked, or the application isn't listening on the port

Hey Mark, which port are you running on?


Can you write one test and just change the environment variable?

Is it possible that you can have one test in your *.http file and just simply just the variable on one iteration, and then another iteration have it change?
I have a use case for this as I want to run tests against a CDN and the Load Balancer behind the CDN. An example would be like:
// Iteration 1&2
GET {{cdnDns}}/healthcheck
GET {{lbDns}}/healthcheck

// Assertions should be the same, since response should be the same.

// Assert Result on both iterations

This code should do the trick
### health_cdn
GET {{cdnDns}}/healthcheck
### health_lb
# @ref health_cdn
GET {{lbDns}}/healthcheck

{{
  const { equal } = require('assert');
  test('should be same response', () => {
    equal(JSON.stringify(health_lb),JSON.stringify(health_cdn));
  });
}}

@AnWeber How would I extract the status codes out of health_lb and health_cdn  so I can do something like
    equal(health_lb.status, 200);
    equal(health_cdn.status, 200);

@FrankRua  not directly. My main use case is that I want to share the response body between requests. status is no longer important to me. For this reason you have to define a variable with the response of first request yourself
### health_cdn
GET https://httpbin.org/json

{{
  exports.health_cdn_response = response
}}

### health_lb
# @ref health_cdn
GET https://httpbin.org/json

{{
  const { equal } = require('assert');
  test('should be same response', () => {
    equal(JSON.stringify(health_lb),JSON.stringify(health_cdn));
  });
  test('should be same status', () => {
    equal(response.statusCode, health_cdn_response.statusCode);
  });
}}

Here is the interface for a httpResponse

I would meanwhile already set the response as a variable, but initially overlooked it and now I don't want to change the api.

I have just released the new version. The feature is included.

:-) sorry I closed the wrong issue.

@AnWeber Is there a way to see if the body of a response contains a string? I see the assert library has a "match" but I don't think I can use it with our library as it gives me match is not defined.

Jest is another test runner. httpyac is also used here as a test runner. I have also already tested the assertion with chai, here is a example. Maybe you can also use jest for this, But does not feel right.
 You would have to install httpyac as CLI tool and start the call in VSCode in the javascript debug terminal (httpyac test.http). Add a debugger; to your Code and view response. Here are the steps documented in more detail

Is it possible to do a "contains" on on the response with HttpYac?

Yeah, the script Context is a real NodeJS Context. You can use all functionality of JavaScript, I prefer using indexOf but NodeJS should also support includes (not sure). You could JSON.stringify the complete response and use indexOf

If there are any questions just contact me again:-)


remove section setup-msui_settings,json for special tests from docs

This posibility was removed from the tests and needs to become removed from the developer documentation
https://github.com/Open-MSS/MSS/blob/develop/docs/development.rst#setup-msui_settingsjson-for-special-tests

Can you assign it to me?


similar to colongd

idk if its just me but it looks simillar to this portafolio.
feel free to close this

I took some inspiration from it, the basic design and the title at the top left. I made mine more cleaner and uniform.


KAFKA-6260: Ensure selection keys are removed from all collections on socket close

When a socket is closed, we must remove corresponding selection keys from internal collections. This fixes an NPE which is caused by attempting to access the selection key's attached channel after it had been cleared after disconnecting.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

cc @rajinisivaram
Note I'm still working on test cases, but I thought I may as well get this fix out there so that the reporter can validate.

Note the test case I added does reproduce the NPE when the fix is removed.

The test failures are unrelated: https://issues.apache.org/jira/browse/KAFKA-6193 and https://issues.apache.org/jira/browse/KAFKA-6094.

@rajinisivaram Thanks for the reviews. I have updated according to your suggestion. I also relocated the detachment of the channel from the selection key as discussed above.

@hachikuji The JDK7 PR build failed. Do you know if that is a transient test failure?

@rajinisivaram I was able to reproduce the failure on trunk, so it seems not related. I will file a jira and see if I can figure out the problem.

Merging to trunk and 1.0.


how to disable github commit in pom.xml

I am new to java, and I am using mvn clean install or mvn package to build my trino plugin. however, the error told me to be Failed to execute goal io.github.git-commit-id:git-commit-id-maven-plugin:5.0.0:revision (default) on project.
I don't need that feature, how could I disable this in pom.xml. I could find nothing related to git-commit-id in the pom.xml.
thanks.

Can you include a full stack trace? This is used to populate some fields in the manifest.

mvn archetype:generate -DarchetypeGroupId=pl.net.was -DarchetypeArtifactId=trino-plugin-archetype -DarchetypeVersion=1.15 -DgroupId=com.aitopia -DartifactId=trino-plugin-xdb
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] >>> archetype:3.2.1:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< archetype:3.2.1:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO]
[INFO]
[INFO] --- archetype:3.2.1:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[WARNING] No archetype found in remote catalog. Defaulting to internal catalog
[WARNING] Archetype not found in any catalog. Falling back to central repository.
[WARNING] Add a repository with id 'archetype' in your settings.xml if archetype's repository is elsewhere.
Define value for property 'classPrefix': XDB
Define value for property 'connectorName': xdb
Define value for property 'githubOwner': daizhang
[INFO] Using property: githubOwner = daizhang
Define value for property 'githubRepo': none
[INFO] Using property: groupId = com.aitopia
[INFO] Using property: artifactId = trino-plugin-xdb
Define value for property 'version' 1.0-SNAPSHOT: : 0.1-SNAPSHOT
[INFO] Using property: package = com.aitopia
Confirm properties configuration:
classPrefix: XDB
connectorName: xdb
githubOwner: daizhang
githubOwner: daizhang
githubRepo: none
groupId: com.aitopia
artifactId: trino-plugin-xdb
version: 0.1-SNAPSHOT
package: com.aitopia
Y: : Y
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: trino-plugin-archetype:1.15
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: com.aitopia
[INFO] Parameter: artifactId, Value: trino-plugin-xdb
[INFO] Parameter: version, Value: 0.1-SNAPSHOT
[INFO] Parameter: package, Value: com.aitopia
[INFO] Parameter: packageInPathFormat, Value: com/aitopia
[INFO] Parameter: classPrefix, Value: XDB
[INFO] Parameter: package, Value: com.aitopia
[INFO] Parameter: githubOwner, Value: daizhang
[INFO] Parameter: githubRepo, Value: none
[INFO] Parameter: groupId, Value: com.aitopia
[INFO] Parameter: connectorName, Value: xdb
[INFO] Parameter: artifactId, Value: trino-plugin-xdb
[INFO] Parameter: version, Value: 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  40.317 s
[INFO] Finished at: 2023-08-10T11:08:11+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-archetype-plugin:3.2.1:generate (default-cli) on project standalone-pom: A Maven project already exists in the directory /Users/daizhang/Arcana/service/trino/trino-plugin-xdb -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I am sorry for not knowing some terms in java world. I wonder if I correctly understand the word of the full stack trace. If the logs above is not the one you need. I will paste something more.

Looks like you're getting a different error now. It says the directory for the new plugin already exists. Please rename it or choose a different name for the new plugin.

I am sorry that I upload the wrong log for the first time. I update it and now it points to the error of git issues. it needs me to have git configured, however I don't have a ci/cd system thus I don't want to push my code every time I build my package.

You can use git locally, without pushing it. Even if you don't want to use it at all, you could just initialize the repository, by running git init, and forget about it.

I run the git init but without any push. however, it gives me the error of Error: Could not get HEAD Ref, are you sure you have some commits in the dotGitDirectory. should I commit something to the project.

Sounds like you need to make an initial commit, you can commit anything, or even have an empty commit.


Increase SelectionCountField margins

Changes
We had no padding on the right side of the SelectionCountField and it looked weird when there was a separator after it (the separator removes padding between different status bar fields). Changes:

I added margins (--iui-size-xs) from both left and right sides now. I chose margins instead of paddings because padding is considered a part of the width so it basically decreased the min-width that we set (more specifically it would decrease the space reserved for more digits).
Changed the min-width from --iui-size-2xl (equivalent to 4rem) to 3.75rem. This slightly decreases the min width of the component and is perfect to fit 5 digits without shifting other status bar content. I made this change because we previously had padding-left which is now switched to margin-left (again, the padding eats into the width and margin does not) so the freed up space was not needed anymore.

Adding some screenshots for comparison. Note that the first two images have separators.
Before:

After:

Fixes #978.
Testing
Tested in test-app.

@Mergifyio backport release/4.16.x


The current Intellij plugin should support newer version 13

We support up to 11.  We have a workaround for 12 but 13 is the latest available version.

Is there a future plan to support 15?


[No QA] Cleanup Authentication error handling

Details
More cleanup for the Network code. The code should work the same as before and no new behavior is being added.
Summary of Changes

Move the error handling in Authenticate() to a util so we don't have to throw an Error inside API.Authenticate()
When we run out of retries in the main queue we are resolving the promise with a special client side jsonCode which can be handled in the .then() of an API command
We account for this change in reauthenticate()
Then fix up some API method usages where a .catch() was implemented as they were previously handling a CONST.ERROR.API_OFFLINE and now need to handle a CONST.JSON_CODE.UNABLE_TO_RETRY.
Some automated tests were also fixed to reflect the new changes

Fixed Issues
$ https://github.com/Expensify/App/issues/8864
Tests
No specific QA other than to make sure reauthentication flows are working.

[ ] Verify that no errors appear in the JS console

PR Review Checklist

PR Reviewer Checklist

[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] iOS / native
[ ] Android / native
[ ] iOS / Safari
[ ] Android / Chrome
[ ] MacOS / Chrome
[ ] MacOS / Desktop

[ ] I verified there are no console errors (if there’s a console error not related to the PR, report it or open an issue for it to be fixed)
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained “why” the code was doing something instead of only explaining “what” the code was doing.
[ ] I verified any copy / text shown in the product was added in all src/languages/* files
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named “index.js”. All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] Any functional components have the displayName property
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose and it is broken down into smaller components in order to separate concerns and functions

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn’t already exist
[ ] The style can’t be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.

QA Steps
No specific QA other than to make sure reauthentication flows are working.

[ ] Verify that no errors appear in the JS console

Screenshots
❌

Ah shucks I got conflicts. Fixed now 🙇

cc @Luke9389 able to give this a quick look?

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@aldo-expensify Nice catch. I think returning the response could work and would just mean we would continue and set the same error to account again? But why is the call to createTemporaryLogin() failing? That sounds like a different bug... Do you want to fix this or should I create a follow up PR here?

That sounds like a different bug...

Yeah, I think this wasn't working properly before your changes... so we can treat it separately.

But why is the call to createTemporaryLogin() failing?

I was trying to reproduce the infinite spinner + error this user is getting here: https://github.com/Expensify/Expensify/issues/202627 and noticed that we get similar symptoms with an expired short lived auth token
I got the expired token by reusing the URL that takes you to New Dot. While this is a weird case, I think we should fix it and probably kick the user to the login screen. Do you agree with taking the user to the login screen? or are we handling expired sessions some other ways?

Do you want to fix this or should I create a follow up PR here?

I'm somewhat concern about the user not being able to use NewDot for a long time now, but I cannot tell yet if this would fix the problem for the user or not. I'm ok with trying to find a solution if your plate is full, and if I find out that this is not related to the problem the user is having, I can even leave it for external contributors.

While this is a weird case, I think we should fix it and probably kick the user to the login screen. Do you agree with taking the user to the login screen? or are we handling expired sessions some other ways?

Hmm I'm not sure if I agree because I'm not sure if I understand what is happening. What are the reproduction steps?
If the short lived token is expired I think one of two things would be true...

They are logged in already as the correct user

Short lived token isn't needed because we already have an active session and stored credentials. So why are we logging them in at all.

They are not logged in at all

In this case... they wouldn't need to be "logged out" or brought to the login screen. If we can log them in, but can't create a temporary login that sounds like a bug.

They are logged in as a different user

If this is true then we should sign them out then sign them back in with the short lived token and then we are back to the previous scenario ☝

cc @neil-marcellini who has looked into these flows a ton and might have better context.

I'm ok with trying to find a solution if your plate is full, and if I find out that this is not related to the problem the user is having, I can even leave it for external contributors.

For now, I will just patch this so the behavior is the same at least and not throw any errors.
I do not think this is a good issue for contributors since it's difficult to test the OldDot > NewDot transition.

🚀 Cherry-picked to staging by @sketchydroide in version: 1.1.57-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by @chiragsalian in version: 1.1.57-17 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[Code Cleanup]: Remove use of unsupported FindBugs tool

FindBugs is no longer maintained. We should re-factor the code to no longer depend upon it. At a glance it appears the primary use case throughout the code is with the @CheckNull annotation. This can be replaced with use of java.util.Optional throughout.

Resolved with release 0.8. The supported spotbugs tool from the parent pom takes its place so there is no need to change the use of @CheckNull.


docs: update readme to point client library documentation

Codecov Report

Merging #7 into master will decrease coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master       #7      +/-   ##
============================================
- Coverage     62.63%   62.50%   -0.14%
+ Complexity      553      552       -1
============================================
  Files            31       31
  Lines          5023     5008      -15
  Branches        480      449      -31
============================================
- Hits           3146     3130      -16
+ Misses         1711     1709       -2
- Partials        166      169       +3

Impacted Files
Coverage Δ
Complexity Δ

...in/java/com/google/cloud/storage/StorageBatch.java
88.00% <0.00%> (-4.00%)
13.00% <0.00%> (ø%)

...gle/cloud/storage/testing/RemoteStorageHelper.java
63.93% <0.00%> (-0.59%)
9.00% <0.00%> (ø%)

...main/java/com/google/cloud/storage/BucketInfo.java
80.85% <0.00%> (-0.20%)
84.00% <0.00%> (ø%)

...ain/java/com/google/cloud/storage/StorageImpl.java
80.15% <0.00%> (-0.13%)
131.00% <0.00%> (ø%)

...e/src/main/java/com/google/cloud/storage/Blob.java
82.08% <0.00%> (-0.11%)
29.00% <0.00%> (-1.00%)

...om/google/cloud/storage/spi/v1/HttpStorageRpc.java
1.52% <0.00%> (+<0.01%)
1.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2bab58...cc6cdf1. Read the comment docs.


Fix action to correctly use http/https proxy.

Updated to NodeJS 21.
Updated to use new Octokit API with .rest.

@Geekdude big thanks for your outstanding effort on this pull request! Your contributions are greatly appreciated. 🙌


v7.3.1 -  amplify pull failed

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
14.x
Amplify CLI Version
7.3.1
What operating system are you using?
macos monterey
Amplify Categories
Not applicable
Amplify Commands
pull
Describe the bug
amplify crash when pulling a project.
Failed to pull the backend.
Found '{' where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace) at line 67,5 >>>    {
      "rootSta ...
Error: Found '{' where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace) at line 67,5 >>>    {
      "rootSta ...

Expected behavior
the project was working fine with previous cli v6.3
Reproduction steps
just pull a project:
amplify pull --appId xxx --envName dev
GraphQL schema(s)

# Put schemas below this line

Log output

amplify pull --appId xxxx --envName dev
✔ Successfully received Amplify Admin tokens.
Amplify AppID found: xxxx. Amplify App name is: xxxx
Backend environment dev found in Amplify Console app: xxxx
? Choose your default editor: None
? Choose the type of app that you're building javascript
Please tell us about your project
? What javascript framework are you using none
? Source Directory Path:  src
? Distribution Directory Path: dist
? Build Command:  npm run-script build
? Start Command: npm run-script start
? Do you plan on modifying this backend? Yes
✔ Successfully pulled backend environment dev from the cloud.
✖ There was an error initializing your environment.
Failed to pull the backend.
Found '{' where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace) at line 67,5 >>>    {
      "rootSta ...
Error: Found '{' where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace) at line 67,5 >>>    {
      "rootSta ...

Additional information
No response

Our entire build is failing due to this very strange 6.4.0 -> 7.3.1 upgrade. It can't find certain parameters anymore.
Side note: this seems like an... unusual approach to semver?

@kouz75 @TheDutchCoder Could it be possible for you'll to send your amplify folders to<EMAIL_ADDRESS>for us to investigate? We're closely monitoring it and will look into it right away.

@kouz75 @TheDutchCoder Could it be possible for you'll to send your amplify folders to<EMAIL_ADDRESS>for us to investigate? We're closely monitoring it and will look into it right away.

I'd really rather not, but here's the full log of the failed build. Reverting to 6.4.0 makes it work again:
                                 # Starting phase: preBuild
                                 # Executing command: curl -X POST -H 'Content-type: application/json' --data '{"text":"['develop'] Amplify Backend deployment starting"}' [redacted]
2021-11-15T16:24:11.245Z [WARNING]: % Total    % Re
2021-11-15T16:24:11.245Z [WARNING]: ceived % Xferd  Average Speed   Time    Time     Time  Current
                                    Dload  Upload   Total   Spent    Left  Speed
                                    0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-11-15T16:24:11.296Z [WARNING]: 100    58    0     2  100    56
2021-11-15T16:24:11.297Z [WARNING]: 39   1098 --:--:-- --:--:-- --:--:--  1137
2021-11-15T16:24:11.297Z [INFO]: ok
2021-11-15T16:24:11.298Z [INFO]: # Completed phase: preBuild
                                 # Starting phase: build
2021-11-15T16:24:23.177Z [INFO]: Amplify AppID found: [redacted]. Amplify App name is: [redacted]
2021-11-15T16:24:23.345Z [INFO]: Backend environment staging found in Amplify Console app: [redacted]
2021-11-15T16:24:26.845Z [WARNING]: - Fetching updates to backend environment: staging from the cloud.
2021-11-15T16:24:29.798Z [WARNING]: - Building resource auth/userPoolGroups
2021-11-15T16:24:29.799Z [WARNING]: - Building resource auth/[redacted]
2021-11-15T16:24:29.802Z [WARNING]: - Building resource storage/s30625986b
2021-11-15T16:24:29.934Z [WARNING]: ✔ Successfully pulled backend environment staging from the cloud.
2021-11-15T16:24:30.608Z [INFO]:
2021-11-15T16:24:35.654Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2021-11-15T16:24:41.663Z [WARNING]: - Initializing your environment: staging
2021-11-15T16:24:54.030Z [WARNING]: - Building resource auth/userPoolGroups
2021-11-15T16:24:54.034Z [WARNING]: - Building resource auth/[redacted]
                                    - Building resource storage/s30625986b
2021-11-15T16:24:54.035Z [WARNING]: ✔ Initialized provider successfully.
2021-11-15T16:25:53.021Z [WARNING]: - Uploading files...
2021-11-15T16:25:53.032Z [WARNING]: - Uploading files...
2021-11-15T16:25:53.314Z [WARNING]: - Uploading files...
2021-11-15T16:25:54.609Z [WARNING]: - Uploading files...
2021-11-15T16:25:54.614Z [WARNING]: - Uploading files...
2021-11-15T16:25:59.481Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.835Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.849Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.857Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.874Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.902Z [WARNING]: - Uploading files...
2021-11-15T16:26:00.930Z [WARNING]: - Uploading files...
2021-11-15T16:26:01.012Z [WARNING]: - Uploading files...
2021-11-15T16:26:04.402Z [WARNING]: - Uploading files...
2021-11-15T16:26:05.652Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.524Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.562Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.578Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.584Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.589Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.596Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.613Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.622Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.630Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.656Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.668Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.689Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.699Z [WARNING]: - Uploading files...
2021-11-15T16:26:07.893Z [WARNING]: - Uploading files...
2021-11-15T16:26:08.227Z [WARNING]: - Uploading files...
2021-11-15T16:26:08.568Z [WARNING]: - Uploading files...
2021-11-15T16:26:08.597Z [WARNING]: - Uploading files...
2021-11-15T16:26:12.303Z [INFO]: GraphQL schema compiled successfully.
                                 Edit your schema at /codebuild/output/src709363363/src/[redacted]/amplify/backend/api/[redacted]/schema.graphql or place .graphql files in a directory at /codebuild/output/src709363363/src/[redacted]/amplify/backend/api/[redacted]/schema
2021-11-15T16:26:13.116Z [WARNING]: - Uploading files...
2021-11-15T16:26:14.627Z [WARNING]: - Updating resources in the cloud. This may take a few minutes...
2021-11-15T16:26:20.997Z [INFO]:
2021-11-15T16:26:20.999Z [INFO]: UPDATE_IN_PROGRESS amplify-[redacted]-staging-05753 AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:15 GMT+0000 (Coordinated Universal Time) User Initiated
2021-11-15T16:26:25.920Z [INFO]:
2021-11-15T16:26:25.926Z [INFO]: UPDATE_IN_PROGRESS functionDefineAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPreSignup                   AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPostConfirmation            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPreTokenGeneration          AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionCreateAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionCreateAndInitBusiness       AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionSyncShopifyProducts         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS hostingamplifyhosting               AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionVerifyAuthChallengeResponse AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:23 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionDefineAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPostConfirmation            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPreTokenGeneration          AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionCreateAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPreSignup                   AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionVerifyAuthChallengeResponse AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionCreateAndInitBusiness       AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionSyncShopifyProducts         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    hostingamplifyhosting               AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:24 GMT+0000 (Coordinated Universal Time)
2021-11-15T16:26:30.954Z [INFO]:
2021-11-15T16:26:30.956Z [INFO]: UPDATE_IN_PROGRESS [redacted] AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:28 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_FAILED      [redacted] AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:29 GMT+0000 (Coordinated Universal Time) Parameters: [functionDefineAuthChallengeName, functionPreTokenGenerationName, functionVerifyAuthChallengeResponseArn, functionPreSignupArn, functionPreSignupName, functionDefineAuthChallengeArn, functionPostConfirmationArn, functionVerifyAuthChallengeResponseName, functionCreateAuthChallengeArn, functionPreTokenGenerationArn, functionPostConfirmationName, functionCreateAuthChallengeName] do not exist in the template
2021-11-15T16:26:35.926Z [INFO]:
2021-11-15T16:26:35.928Z [INFO]: UPDATE_ROLLBACK_IN_PROGRESS amplify-[redacted]-staging-05753 AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:31 GMT+0000 (Coordinated Universal Time) The following resource(s) failed to update: [[redacted]].
2021-11-15T16:26:50.953Z [INFO]:
2021-11-15T16:26:50.955Z [INFO]: UPDATE_IN_PROGRESS functionCreateAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS hostingamplifyhosting               AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPreTokenGeneration          AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPostConfirmation            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionSyncShopifyProducts         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionVerifyAuthChallengeResponse AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionDefineAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionCreateAndInitBusiness       AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionCreateAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionPreSignup                   AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPreTokenGeneration          AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionSyncShopifyProducts         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionVerifyAuthChallengeResponse AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:48 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionDefineAuthChallenge         AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:49 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPostConfirmation            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:49 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionPreSignup                   AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:49 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    hostingamplifyhosting               AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:49 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    functionCreateAndInitBusiness       AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:49 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    [redacted]                   AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:50 GMT+0000 (Coordinated Universal Time)
2021-11-15T16:26:55.958Z [INFO]:
2021-11-15T16:26:55.959Z [INFO]: UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS amplify-[redacted]-staging-05753 AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:54 GMT+0000 (Coordinated Universal Time)
2021-11-15T16:27:00.959Z [INFO]:
2021-11-15T16:27:00.960Z [INFO]: UPDATE_COMPLETE          hostingamplifyhosting                  AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:56 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          [redacted]                      AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:56 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionCreateAndInitBusiness          AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:56 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionSyncShopifyProducts            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:56 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionCreateAuthChallenge            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionPreSignup                      AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionPostConfirmation               AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionPreTokenGeneration             AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionDefineAuthChallenge            AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          functionVerifyAuthChallengeResponse    AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:57 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_ROLLBACK_COMPLETE amplify-[redacted]-staging-05753 AWS::CloudFormation::Stack Mon Nov 15 2021 16:26:58 GMT+0000 (Coordinated Universal Time)
2021-11-15T16:27:21.129Z [INFO]: Following resources failed
2021-11-15T16:27:21.132Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
2021-11-15T16:27:21.151Z [WARNING]: ✖ There was an error initializing your environment.
2021-11-15T16:27:21.157Z [INFO]: ResourceNotReady: Resource is not in the state stackUpdateComplete
                                     at constructor.setError (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:182:47)
                                     at Request.CHECK_ACCEPTORS (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:44:12)
                                     at Request.callListeners (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
                                     at Request.emit (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                     at Request.emit (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:686:14)
                                     at Request.transition (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                     at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                     at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                     at Request.<anonymous> (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                     at Request.<anonymous> (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:688:12)
                                     at Request.callListeners (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
                                     at Request.emit (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                     at Request.emit (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:686:14)
                                     at Request.transition (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                     at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                     at /root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                     at Request.<anonymous> (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                     at Request.<anonymous> (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:688:12)
                                     at Request.callListeners (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
                                     at callNextListener (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
                                     at IncomingMessage.onEnd (/root/.nvm/versions/node/v12.21.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/event_listeners.js:335:13)
                                     at IncomingMessage.emit (events.js:326:22)
                                     at IncomingMessage.EventEmitter.emit (domain.js:483:12)
                                     at endReadableNT (_stream_readable.js:1241:12)
                                     at processTicksAndRejections (internal/process/task_queues.js:84:21) {
                                   code: 'ResourceNotReady',
                                   retryable: false,
                                   time: 2021-11-15T16:27:16.124Z,
                                   statusCode: 200,
                                   retryDelay: 30000
                                 }
2021-11-15T16:27:21.211Z [ERROR]: !!! Build failed
2021-11-15T16:27:21.212Z [ERROR]: !!! Non-Zero Exit Code detected
2021-11-15T16:27:21.212Z [INFO]: # Starting environment caching...
2021-11-15T16:27:21.212Z [INFO]: # Uploading environment cache artifact...
2021-11-15T16:27:21.299Z [INFO]: # Environment caching completed
Terminating logging...

hi @TheDutchCoder
Can you share the cli-inputs.json file present in auth category folder after migration?
Since this file contains your resource state , just redact any confidential info from it before sharing to<EMAIL_ADDRESS>

@TheDutchCoder Totally understand the confidentiality.
If we could get the keys of your parameters.json object in amplify/backend/auth folder, that'll really help us reproduce the issue on our end and provide a fix.

@TheDutchCoder Totally understand the confidentiality. If we could get the keys of your parameters.json object in amplify/backend/auth folder, that'll really help us reproduce the issue on our end and provide a fix.

Sure thing, here you go:
{
    "identityPoolName": "[redacted]_identitypool",
    "allowUnauthenticatedIdentities": true,
    "resourceNameTruncated": "creati0626f80b",
    "userPoolName": "[redacted]_userpool",
    "autoVerifiedAttributes": [
        "email"
    ],
    "mfaConfiguration": "OFF",
    "mfaTypes": [
        "SMS Text Message"
    ],
    "smsAuthenticationMessage": "Your authentication code is {####}",
    "smsVerificationMessage": "Your verification code is {####}",
    "emailVerificationSubject": "Your verification code",
    "emailVerificationMessage": "Your verification code is {####}",
    "defaultPasswordPolicy": false,
    "passwordPolicyMinLength": 8,
    "passwordPolicyCharacters": [],
    "requiredAttributes": [
        "email",
        "family_name",
        "given_name"
    ],
    "userpoolClientGenerateSecret": true,
    "userpoolClientRefreshTokenValidity": 30,
    "userpoolClientWriteAttributes": [
        "email",
        "family_name",
        "given_name"
    ],
    "userpoolClientReadAttributes": [
        "email",
        "family_name",
        "given_name"
    ],
    "userpoolClientLambdaRole": "creati0626f80b_userpoolclient_lambda_role",
    "userpoolClientSetAttributes": true,
    "sharedId": "0626f80b",
    "resourceName": "[redacted]",
    "authSelections": "identityPoolAndUserPool",
    "authRoleArn": {
        "Fn::GetAtt": [
            "AuthRole",
            "Arn"
        ]
    },
    "unauthRoleArn": {
        "Fn::GetAtt": [
            "UnauthRole",
            "Arn"
        ]
    },
    "useDefault": "manual",
    "thirdPartyAuth": false,
    "usernameAttributes": [
        "email"
    ],
    "userPoolGroups": true,
    "adminQueries": true,
    "triggers": "{\n  \"CreateAuthChallenge\": [\n    \"custom\"\n  ],\n  \"DefineAuthChallenge\": [\n    \"custom\"\n  ],\n  \"PostConfirmation\": [\n    \"add-to-group\"\n  ],\n  \"PreSignup\": [\n    \"custom\"\n  ],\n  \"VerifyAuthChallengeResponse\": [\n    \"custom\"\n  ],\n  \"PreTokenGeneration\": [\n    \"alter-claims\"\n  ]\n}",
    "hostedUI": false,
    "userPoolGroupList": [
        "Users",
        "Admins"
    ],
    "adminQueryGroup": "Admins",
    "serviceName": "Cognito",
    "parentStack": {
        "Ref": "AWS::StackId"
    },
    "permissions": [
        "{\n  \"policyName\": \"AddToGroupCognito\",\n  \"trigger\": \"PostConfirmation\",\n  \"effect\": \"Allow\",\n  \"actions\": [\n    \"cognito-idp:AdminAddUserToGroup\",\n    \"cognito-idp:GetGroup\",\n    \"cognito-idp:CreateGroup\"\n  ],\n  \"resource\": {\n    \"paramType\": \"!GetAtt\",\n    \"keys\": [\n      \"UserPool\",\n      \"Arn\"\n    ]\n  }\n}"
    ],
    "dependsOn": [
        {
            "category": "function",
            "resourceName": "CreateAuthChallenge",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        },
        {
            "category": "function",
            "resourceName": "DefineAuthChallenge",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        },
        {
            "category": "function",
            "resourceName": "VerifyAuthChallengeResponse",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        },
        {
            "category": "function",
            "resourceName": "PreTokenGeneration",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        },
        {
            "category": "function",
            "resourceName": "PreSignup",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        },
        {
            "category": "function",
            "resourceName": "PostConfirmation",
            "triggerProvider": "Cognito",
            "attributes": [
                "Arn",
                "Name"
            ]
        }
    ]
}

Hi @TheDutchCoder
Can you also check  amplify/cli.json file if  it has breakcirculardependency flag  set to true?

Hi @TheDutchCoder
Can you also check amplify/cli.json file if it has breakcirculardependency flag set to true?

It doesn't. Here's the config:
{
  "features": {
    "graphQLTransformer": {
      "addMissingOwnerFields": true,
      "validateTypeNameReservedWords": true,
      "enableIterativeGSIUpdates": true,
      "secondaryKeyAsGSI": true
    },
    "auth": {
      "enableCaseInsensitivity": true,
      "useInclusiveTerminology": true
    },
    "appSync": {
      "generateGraphQLPermissions": true
    },
    "codegen": {
      "useAppSyncModelgenPlugin": true,
      "useDocsGeneratorPlugin": true,
      "useTypesGeneratorPlugin": true,
      "cleanGeneratedModelsDirectory": true,
      "retainCaseStyle": true
    }
  }
}

@TheDutchCoder Could you give it a try in Amplify Console with latest CLI version and confirm if the issue has been mitigated?

@TheDutchCoder Could you give it a try in Amplify Console with latest CLI version and confirm if the issue has been mitigated?

We'll give it a shot tomorrow in the AM and I'll report back!

@TheDutchCoder Could you give it a try in Amplify Console with latest CLI version and confirm if the issue has been mitigated?

Yes, the issue seems to have been fixed by 7.4.0. Thanks guys!
What's the exact reason for this, somewhat unorthodox, semver approach btw?

@TheDutchCoder Glad that you are  unblocked!
We did a major version bump since we did an internal change to support overrides of resources and custom resources for the Amplify CLI (https://aws.amazon.com/amplify/extensibility/) which would directly impact customer files (within the amplify folder) in some cases. Whenever we do customer impacting changes which might potentially require file-changes (and we try to auto-migrate all these changes), we do a major version bump for the CLI.


registration is not recognized

My registration is not recognized. The demo mode works.  What am I doing wrong.
Many thanks for your help.
Regards Uwe
Log.pdf

Sorry, misser this one is the error stil present ?

Sorry, misser this one is the error stil present ?


Support default_entry_point on components

This feature has been added. To add a default entry point to a component. Add a class member called DEFAULT_ENTRY_POINT
https://github.com/agentos-project/agentos/blob/0f3d437f9ca11b5f1d01308b8f81de578bb18b4c/agentos/component.py#L285-L290


Using other models?

Is it possible to run the inference script using other models? I'm specifically thinking of VR models, such as 5_HP-Karaoke-UVR.pth or UVR-BVE-4B_SN-44100-1.pth.
Similarly, could we consider integrating models like Medley Vox?

Currently it's not possible

Not even by using a rough or less conventional method? 😥

Is it possible to run the inference script using other models? I'm specifically thinking of VR models, such as 5_HP-Karaoke-UVR.pth or UVR-BVE-4B_SN-44100-1.pth.
Similarly, could we consider integrating models like Medley Vox?

Maybe you can try our project, You can use this webUI to infer the MSST model and VR Models.

Is it possible to run the inference script using other models? I'm specifically thinking of VR models, such as 5_HP-Karaoke-UVR.pth or UVR-BVE-4B_SN-44100-1.pth.
Similarly, could we consider integrating models like Medley Vox?

Maybe you can try our project, You can use this webUI to infer the MSST model and VR Models.

Wow, this looks promising and very intuitive. I really like the features you've added—SOME seems interesting and worth testing.
Would it be possible to integrate Medley Vox into this Web UI via a manual download?


Make encounter_deletion general for low and increased risk encounters

This PR updates the text of the FAQ entry encounter_deletion. It makes the text general, so that the FAQ is not specific to low risk encounters anymore.

Preview of the change

German
English

@Ein-Tim
What about the fact that the date of the encounter is shown in the Journal?  Probably this article was written before that enhancement was introduced in 1.12 (https://www.coronawarn.app/en/blog/2021-02-10-corona-warn-app-version-1-12/).
This point could be added to the FAQ so users are aware that they can look in the Journal before the exposure is deleted.

@MikeMcC399
This sounds like a good idea and I will add a commit implementing this soon.

@MikeMcC399
I added a reference to the blog post with 2baf401.
I'd be happy to hear your feedback!

@dsarkar
Will you have time to look at this soon?

@Ein-Tim
You currently have conflicts. It would be helpful to resolve these before further reviewing.

@MikeMcC399
Thanks! Resolved merge conflicts in fc3a513.

@Ein-Tim
Unfortunately I now think that my suggestion to mention the contact journal was a  bad one!
In the case of #encounter_deletion the situation is that a risk encounter is already deleted, so there will be nothing from the deleted risk encounter to see in the contact journal. The risk encounter will also have disappeared in the contact journal. If there have been multiple encounters, then there may be other ones listed in the contact journal, but not the one which has disappeared (by definition!).
I'm very sorry, but I think this paragraph should be removed. What do you think?

Morning, @Ein-Tim @MikeMcC399! We will have a look tomorrow! Enjoy the weekend!

@MikeMcC399
Yes, you're right! No need to apologize, I also didn't think about this! Will remove the paragraph tomorrow.
Thank you @dsarkar!

c9a84fe removed the reference to the blog post. I rebased to the current master, this PR is ready to be merged now.

I rebased on the current master, squashed all commits and removed the reference to the blog post in 1aed72c.
Ready to be merged!

Oh no, I just see that there are many unwanted commits in here, stand by please...

Continue in #2421.


wait-on never resolves when providing multiple urls

Hello team,
I observed a strange behavior, where if I provide more than one url in wait-on the job never completes and never actually runs the cypress tests. There are no errors.
 uses: cypress-io/github-action@v4
      with:
        working-directory: ./web
        start: |
          npm start -w @cubby/storefront-components-test-app
          node ./apps/storefront-components/server/main.js
        wait-on: 'http://localhost:3033, http://localhost:8090' <<<<<< If I remove the second url, it works normally
        wait-on-timeout: 10000
        command: npm run test:e2e -- --spec cypress/e2e/storefrontComponents/**/*.cy.ts --config baseUrl=http://localhost:3033

@venelinpetrov
I suggest you update to the current version of the action @v5 and check out

https://github.com/cypress-io/github-action#wait-on-with-nodejs-18
https://github.com/cypress-io/github-action#debugging
https://github.com/cypress-io/github-action#debugging-waiting-for-url-to-respond

@venelinpetrov

command: npm run test:e2e -- --spec cypress/e2e/storefrontComponents/**/*.cy.ts --config baseUrl=http://localhost:3033

Note also

https://github.com/cypress-io/github-action#custom-test-command

which is not related to your wait-on issue, however if you use the command parameter to run Cypress test, then you bypass a lot of other functionality in github-action.
Use instead

the spec parameter https://github.com/cypress-io/github-action#specs
the config parameter https://github.com/cypress-io/github-action#config

@venelinpetrov
Have you been able to review the suggestions and comments I provided?

This issue should now be closed due to inactivity.

Closing due to inactivity.