TypeScript 2: type guard is forgotten inside function

interface X { type: 'x', bar: string }
interface Y { type: 'y', baz: string }
type union = X | Y;
const x = { type: 'x', bar: 'bar' } as union;

((x) => {
    if (x.type === 'x') {
        x.bar; // fine
        () => x.bar // error: main.ts(30,17): error TS2339: Property 'bar' does not exist on type 'X | Y'.
    }
})(x)

This is the intended behavior. There's no way to know when the function expression is going to be invoked. For example, forEach invokes its callback immediately (so type guards should apply), whereas setTimeout invokes it later (so type guards should not apply, because the value may be modified in the interim).

That makes sense, however when upgrading the TS2 I noticed this also breaks my user defined type guard:
interface X { type: 'x', bar: string }
interface Y { type: 'y', baz: string }
type union = X | Y;
const x = { type: 'x', bar: 'bar' } as union;
const isX = (union: union): union is X => x.type === 'x';

((x) => {
    if (isX(x)) {
        x.bar;
        () => x.bar
    }
})(x)

This worked in TS1.

In general the flow-based analysis is going to find slightly more possible-bugs. I'd recommend writing something like this (not sure what your actual scenario looks like):
interface X { type: 'x', bar: string }
interface Y { type: 'y', baz: string }
type union = X | Y;
const x = { type: 'x', bar: 'bar' } as union;
const isX = (union: union): union is X => x.type === 'x';

((x: union) => {
    if (isX(x)) {
        const b = x.bar;
        () => b;
    }
})(x)

Got it, thanks!


RegisterWithoutUsername issue

Describe the bug
If I use the RegisterWithoutUsername functionality there is a problem in the code if there is an error during the registration process. Since RegisterWithoutUsername calls the regular Register process with a new model, if the error returns the model in Register I get an error that the ViewModel should be RegisterWithoutUsernameViewModel not RegisterViewModel.
To Reproduce
Steps to reproduce the behavior:
Just produce and error during Register with the RegisterWithoutUsername functionality like a duplicate email address
Relevant parts of the log file
<log goes here>

Hi, please, can you send a PR which fixed this use case? Thanks

I just duplicated the code from Register instead of calling Register with a new ViewModel from RegisterWithoutUsername.

Ok, I will test. Thanks for reporting @johndcollins 👍

Solved in master branch. 😉👍


import issues

pip install pyECG went successful but import gave me an error,
code:
from pyecg import ECGRecord
error:
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Anaconda3\lib\site-packages\pyecg_init_.py", line 6, in
from pyecg.dataset import RecordTicket, ECGDataset
File "C:\ProgramData\Anaconda3\lib\site-packages\pyecg\dataset.py", line 12, in
from ishneholterlib import Holter
File "C:\ProgramData\Anaconda3\lib\site-packages\ishneholterlib_init_.py", line 9, in
from PyCRC.CRCCCITT import CRCCCITT
ModuleNotFoundError: No module named 'PyCRC'
PS:  I  also installed PyCRC but this problem remains the same. can this be a version issue?

Same issue of @krutarth0

pip install pyECG went successful but import gave me an error,
code:
from pyecg import ECGRecord
error:
Traceback (most recent call last):
File "", line 1, in
File "C:\ProgramData\Anaconda3\lib\site-packages\pyecg__init__.py", line 6, in
from pyecg.dataset import RecordTicket, ECGDataset
File "C:\ProgramData\Anaconda3\lib\site-packages\pyecg\dataset.py", line 12, in
from ishneholterlib import Holter
File "C:\ProgramData\Anaconda3\lib\site-packages\ishneholterlib__init__.py", line 9, in
from PyCRC.CRCCCITT import CRCCCITT
ModuleNotFoundError: No module named 'PyCRC'
PS: I also installed PyCRC but this problem remains the same. can this be a version issue?

The install package for 'pycrc' named pythoncrc
Try this:

pip install pythoncrc

The issue persists for me even after "pip install pythoncrc" . Any other suggestions?

Commit 95f854c90713ffb370917f20b11e4d9c1ed6d5c6 introduced a fix, but it was not pushed to Pypi. What worked for me was :
first
pip uninstall PyCRC
and then
pip install pythoncrc

Commit 95f854c introduced a fix, but it was not pushed to Pypi. What worked for me was :
first
pip uninstall PyCRC
and then
pip install pythoncrc

I uninstalled pycrc and installed pythoncrc. but the problem exists


Multi-value fields do not mix well with lookups

For the simplest example, consider datasource that has a record that contains a multi-value field:
{ "key": ["key1", "key2"], /* ... */ }

A query is made with the following dimension spec:
"dimensions" : [
  {
    "type" : "default",
    "dimension" : "key",
    "outputName" : "key"
  }, {
    "type" : "extraction",
    "dimension" : "key",
    "outputName" : "value",
    "extractionFn" : {
      "type": "lookup",
      "lookup": {
        "type": "map",
        "map" : { "key1": "value1", "key2": "value2" }
      }
    }
  },
  /* ... */
]

Intuitively, I would expect to have the following two rows returned:
{ "key": "key1", "value": "value1", /*...*/}
{ "key": "key2", "value": "value2", /*...*/}

However, instead I get this:
{ "key": "key1", "value": "value1", /*...*/}
{ "key": "key1", "value": "value2", /*...*/}
{ "key": "key2", "value": "value1", /*...*/}
{ "key": "key2", "value": "value2", /*...*/}

I do understand why it may currently work this way, but I do believe a much better user experience can be achieved in such cases. In fact, as far as I understand, there currently is no good solution for this use case and also for filtering by lookup result on a multi-value field. A possible workaround is to change the lookup to be like {"key1": "key1:value1"}, but that does not work well for non-injective lookups and is generally very ugly.
I was unable to find any discussion of this topic, so it would be great if someone could elaborate on this. I am ok with taking time to implement a nice solution, but would like to have an agreement on the expected behavior. A possible solution could be to optionally (if a parameter is set to true) return the original pre-lookup value along with the value that was looked up, although I'm pretty sure there can be better ways.

Can you elaborate on what query you are doing that gets these results? (topN, groupBy?)

nevermind :) groupby given the multiple dimension spec

I'm pretty sure Dimension tuples should roll up together in a groupBy.

Uh, I'm not sure I understand what you mean by Dimension tuples being rolled up. Could you elaborate, please?

@gvsmirnov, your query is grouping on two multi-value dimensions ('key' and another one 'value' created from key). Since Druid explodes multi-value dimensions when grouping, that means you get the cross product out.
I think what you're looking for is for Druid to treat 'value' as something derived from 'key' and not as a separate independent dimension. You might be able to do that with a custom extraction function that does basically f(x) = x + ":" + lookup(x), and then only including the extracted dimension in your grouping (not the original one). Or by patching the existing lookup extraction function to have an option to prepend the key.

I would expect each combination of dimension values to appear only once, as per your expected behavior.

@gianm You made a lot of improvements in the groupBy engine for an upcoming release, does this strike you as an unintended side-effect somewhere? and do you know if the new grouper does the same thing?

@gianm that was my idea for the workaround, too. Brushing aside the kludginess, this would not work for non-injective lookups where one wants to group by the value that was looked up.

I miss-read the report.
Does { "key": "key2", "value": "value1", /*...*/} have anything in it?
it looks like it is rolling up dimension tuples, but you just have more tuples than you expect

@drcrallen this seems to me like intended (but possibly unintuitive) behavior. But both engines do the same thing where any multi-value dimension gets independently exploded, yielding a cross product. I just wrote a PR with a new test for that: #3415
@gvsmirnov if you're wanting to group by the looked up value on a non-injective lookup, you probably don't want to include the key at all, right? Since it would potentially vary and interfere with the grouping. So you could use the version/option of the extraction function that doesn't prepend the key in that case.

Ok, I get it now, thanks @gianm for that good test.
I was wrong previously. Due to the fact that the multi-value dimension is both value1 and value2, the behavior presented by druid and shown by @gianm in tests is the correct behavior as groupBy stands today.
This definitely deserves to be called out in docs.

Given that the multi-value dimension takes on both values, @gvsmirnov, can you elaborate on what kind of behavior would be helpful?

It does no good to return a bunch of data that isn't actually useful or intended.

@drcrallen @gianm yep, I do understand that both key and value end up being multi-value dimensions currently. However, IMO it would make sense to, if a lookup is using a multi-value dimension (key) as a source, to apply it after the key is unrolled into key1 and key2, to each individually.
Do you think that the current cross-product behavior is a more viable use case than returning only rows where the lookup result actually corresponds to the lookup key?
@gianm fair point about not wanting to keep the key at all if grouping by a non-injective lookup. Still, the workaround is quite a sad one.

To clarify: I believe the helpful behavior would be to only return rows where the looked-up value corresponds to the look-up key.

fwiw, I think the current groupBy behavior when you have two multi-value dims is "correct" and internally consistent with how multi-value dims are generally dealt with, but is probably not actually that useful.
@gvsmirnov I think the question there is, how does Druid know that you want to treat key/value as a single unit, rather than as two completely independent dimensions? The way you provide them in the query (two separate dimensions) tells Druid they should be treated independently. IMO if they should be treated as a unit, they should be provided as one dimension, which points towards the extraction function "workaround" as actually being a good approach, since that is one dimension rather than two.
I wonder what's sad about it? Would it be less sad if we could return "kv" : {"k": "k1", "v": "v1"} rather than "kv": "k1:v1"?

@gvsmirnov can you see if https://github.com/druid-io/druid/issues/3367 captures your needs adequately? it would be worth commenting there that you have a use-case for multi-value dimensions. If that does capture the needs correctly I propose closing this down as working as intended (confusing though it is) and having only one issue for "id and value" being returned for lookups.

@gianm  I would expect it to know it by seeing that in the dimension spec:
"dimensions" : [
  {
    "type" : "default",
    "dimension" : "key", // <-
    "outputName" : "key"
  }, {
    "type" : "extraction",
    "dimension" : "key", // <-
    "outputName" : "value",
    "extractionFn" : {...}
  }
]

There are two entries that have the same "dimension" field. That means that they share the same source and should be treated as a single unit.
As for the sadness, consider a case where you are looking up an object by id, and the object has multiple fields, and you would like to be able to filter by those fields, too. Now you could have the whole object as a string and then do some perversions for filtering and then splitting etc. Another part of the sadness is that I don't see a way to prepend the original Id besides putting it in the lookup source data (e.g. a kafka topic would contain mappings like key1->key1:value1, which is sad in itself. Or am I missing a way to do the prepending without it?
@drcrallen partially it would, yes. But consider the case where the object looked up has multiple properties that the user wants to filter by. A nice approach would be to have them all as separately looked-up dimensions.
Since from the query it's obvious that multiple output dimensions are based on the same dimension in the source data, then Druid can be nice enough to avoid cross-producting the dimension essentially with itself multiple times.

Also a part of the sadness of the workaround: if you want the looked-up value to match exactly yo xyz, you cannot use a simple selector filter.

@gvsmirnov putting the key in the source lookup is probably not the best way to go, for the all reasons you mentioned (extra data in kafka, interferes with filtering, can't be disabled, etc…). I was thinking more like an option in the extraction function for prepending the key, which you could enable or disable at query time.

Specifically I was thinking of something like, "includeKey": true, "keyDelimiter": "|" as things you could ask the extraction function to do. That would probably solve #3367 as well. It wouldn't be as nice as returning a real JSON object but it would be easy to fit into the existing system.

One important thing here is that the dimension you group by and the dimension you filter by do not have to be defined the same way. You could filter by value and then group by "key:value", and that would work fine.

How about having something like this:
{
  "dimension": "key",
  "type": "multiOutput",
  "output": [
     {"type": "default" ...},
     {"type": "extraction" ...}
  ]
}

@gvsmirnov what would the query result look like in that case?

That may vary. For example, given
{
  "dimension": "key",
  "type": "multiOutput",
  "output": [
     {"type": "default", "outputName": "key"},
     {
      "type": "extraction",
      "outputName": "someLookupValue",
      "extractionFn" : {
        "type": "lookup",
        "lookup": {
          "type": "map",
          "map" : { "key1": "someValue1", "key2": "someValue2" }
        }
      }
     },
     {
      "type": "extraction",
      "outputName": "someOtherLookupValue",
      "extractionFn" : {
        "type": "lookup",
        "lookup": {
          "type": "map",
          "map" : { "key1": "someOtherValue1", "key2": "someOtherValue2" }
        }
      }
     }
  ]
}

And a record like {"key": ["key1", "key2"] /*...*/}
I'd be ok to get
{ "key": "key1", "someLookupValue": "someValue1", "someOtherLookupValue": "someOtherValue1", /*...*/}
{ "key": "key2", "someLookupValue": "someValue2", "someOtherLookupValue": "someOtherValue2", /*...*/}

But basically, this kind of spec would clearly express the intent that for each value of "key", each output should computed once, and each output value should be in the record corresponding to the key.

If you give me a few pointers on where to start, I'd be ok to implement it while avoiding clubbering everything around :)

If you wanted to do the extractionFn options, they could be added to RegisteredLookupExtractionFn, LookupExtractionFn, and LookupDimensionSpec (the three internal ways lookups can get used). It might be nice to generalize them a bit, i.e. rather than having keyDelimiter, instead have a "format string" that could be something like "%k:%v" or "%v (%k)" or whatever. Perhaps also an option to escape keys/values to handle the case where it's not possible to select an un-ambiguous format string.
I think your multiOutput suggestion is not implementable right now, since dimension values currently need to be strings. We are working on lifting that restriction but there's a bunch of internal refactoring needed.

@gianm aha, after looking through the codebase I see that making this multiOutput would require touching a lot of code. I currently (grudgingly :) ) implemented the workaround with just concatenating stuff, so my need is not so urgent any more.
IMO it would be much better to implement only the long-term solution, as the additional extractionFn option is quite a half-measure. Where could I track the progress of lifting the restriction on string-only values, by the way? I found an assortment of various issues related to that, but no umbrella discussion.

@gvsmirnov there have been some discussions about opening up extractionFn return types to numerics as well as strings, but not yet to more complex types like list, map, etc.

An update: Druid extractionFns still can only return strings, but expressions can return numerics. They can't return complex types, but may be generalized at some point (list is particularly likely).
Closing this, since it seems like the original issue has been solved or worked around.


Install doesn't function

All installs fail to complete, however they do report as installed.
PS C:\Users\John> cinst nodejs
=====================================================
Chocolatey (<IP_ADDRESS>) is installing nodejs (from https://go.microsoft.com/fwlink/?LinkID=206669) to "C:\NuGet\lib"
=====================================================
Package License Acceptance Terms
-------------------------
Please run chocolatey /? for full license acceptance verbage. By installing you accept the license for the package you a
re installing...
-------------------------
-------------------------
NuGet
-------------------------
'nodejs 0.5.9' already installed.

-------------------------
-------------------------
Chocolatey Runner (NODEJS)
-------------------------
-------------------------
Chocolatey Installation (chocolateyinstall.ps1)
-------------------------
Looking for chocolateyinstall.ps1 in folder C:\NuGet\lib\nodejs.0.5.9
If chocolateyInstall.ps1 is found, it will be run.
-------------------------
Running powershell -NoProfile -ExecutionPolicy unrestricted -Command "& import-module -name  'C:\NuGet\chocolateyInstall
\helpers\chocolateyInstaller.psm1'; & 'C:\NuGet\lib\nodejs.0.5.9\tools\chocolateyInstall.ps1'". This may take awhile and
 permissions may need to be elevated, depending on the package.
Exception calling "Start" with "1" argument(s): "The system cannot find the file specified"
At C:\NuGet\chocolateyInstall\chocolatey.ps1:33 char:43
+   $s = [System.Diagnostics.Process]::Start <<<< ($psi);
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

You cannot call a method on a null-valued expression.
At C:\NuGet\chocolateyInstall\chocolatey.ps1:34 char:17
+   $s.WaitForExit <<<< ();
    + CategoryInfo          : InvalidOperation: (WaitForExit:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

[ERROR] Running powershell with -NoProfile -ExecutionPolicy unrestricted -Command "& import-module -name  'C:\NuGet\choc
olateyInstall\helpers\chocolateyInstaller.psm1'; & 'C:\NuGet\lib\nodejs.0.5.9\tools\chocolateyInstall.ps1'" was not succ
essful.
-------------------------
Executable Batch Links
-------------------------
Looking for executables in folder: C:\NuGet\lib\nodejs.0.5.9
Adding batch files for any executables found to a location on PATH. In other words the executable will be available from
 ANY command line/powershell prompt.
-------------------------
There are no executables in the package.
-------------------------
=====================================================
Chocolatey has finished installing nodejs
=====================================================
PS C:\Users\John> cver
You have the latest version of chocolatey (<IP_ADDRESS>) based on (https://go.microsoft.com/fwlink/?LinkID=206669).

Name                           Value
----                           -----
found                          <IP_ADDRESS>
latestCompare                  00000.00009.00008.00011
latest                         <IP_ADDRESS>
foundCompare                   00000.00009.00008.00011

PS C:\Users\John> cup all
You have the latest version of Console2 (2.0) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
You have the latest version of grepwin (1.5.3) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
You have the latest version of nodejs (0.5.9) based on (https://go.microsoft.com/fwlink/?LinkID=206669).

Are you seeing this across the board?

Yeah, on every package. Just tried again on a random one:
Windows PowerShell
Copyright (C) 2009 Microsoft Corporation. All rights reserved.

PS C:\Users\John> cinst pstools
=====================================================
Chocolatey (<IP_ADDRESS>) is installing pstools (from https://go.microsoft.com/fwlink/?LinkID=206669) to "C:\NuGet\lib"
=====================================================
Package License Acceptance Terms
-------------------------
Please run chocolatey /? for full license acceptance verbage. By installing you accept the license for the package you a
re installing...
-------------------------
-------------------------
NuGet
-------------------------
Successfully installed 'pstools <IP_ADDRESS>'.

-------------------------
-------------------------
Chocolatey Runner (PSTOOLS)
-------------------------
-------------------------
Chocolatey Installation (chocolateyinstall.ps1)
-------------------------
Looking for chocolateyinstall.ps1 in folder C:\NuGet\lib\pstools.<IP_ADDRESS>
If chocolateyInstall.ps1 is found, it will be run.
-------------------------
Running powershell -NoProfile -ExecutionPolicy unrestricted -Command "& import-module -name  'C:\NuGet\chocolateyInstall
\helpers\chocolateyInstaller.psm1'; & 'C:\NuGet\lib\pstools.<IP_ADDRESS>\tools\chocolateyInstall.ps1'". This may take awhile
and permissions may need to be elevated, depending on the package.
Exception calling "Start" with "1" argument(s): "The system cannot find the file specified"
At C:\NuGet\chocolateyInstall\chocolatey.ps1:33 char:43
+   $s = [System.Diagnostics.Process]::Start <<<< ($psi);
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

You cannot call a method on a null-valued expression.
At C:\NuGet\chocolateyInstall\chocolatey.ps1:34 char:17
+   $s.WaitForExit <<<< ();
    + CategoryInfo          : InvalidOperation: (WaitForExit:String) [], RuntimeException
    + FullyQualifiedErrorId : InvokeMethodOnNull

[ERROR] Running powershell with -NoProfile -ExecutionPolicy unrestricted -Command "& import-module -name  'C:\NuGet\choc
olateyInstall\helpers\chocolateyInstaller.psm1'; & 'C:\NuGet\lib\pstools.<IP_ADDRESS>\tools\chocolateyInstall.ps1'" was not s
uccessful.
-------------------------
Executable Batch Links
-------------------------
Looking for executables in folder: C:\NuGet\lib\pstools.<IP_ADDRESS>
Adding batch files for any executables found to a location on PATH. In other words the executable will be available from
 ANY command line/powershell prompt.
-------------------------
There are no executables in the package.
-------------------------
=====================================================
Chocolatey has finished installing pstools
=====================================================
PS C:\Users\John> cup
You have the latest version of chocolatey (<IP_ADDRESS>) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
PS C:\Users\John> cup all
You have the latest version of Console2 (2.0) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
You have the latest version of grepwin (1.5.3) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
You have the latest version of nodejs (0.5.9) based on (https://go.microsoft.com/fwlink/?LinkID=206669).
You have the latest version of pstools (<IP_ADDRESS>) based on (https://go.microsoft.com/fwlink/?LinkID=206669).

Does the second window even pop up for you?  Do me a favor and look in C:\Nuget\chocolateyInstall for a chocolateyPSInstall.log file. This may help lead us to what is really going on.
From what it looks like, it is not finding powershell on your machine to execute. What version of powershell do you have installed?

I found the problem, For some reason powershell wasn't in my path. (I didn't check because I figured it would be there by default)
Stock windows 7 PS2.0 btw.

sounds like we need to specify the full path to powershell and possibly make it configurable?

https://github.com/chocolatey/chocolatey/issues/23


fix: enable support for edit_url in TechDocs

Hey, I just made a Pull Request!
enable support for edit_url in TechDocs
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

enabled edit_url in this pr @awanlin


[Feature] A function or event to obtain a value in the mouse drag range

What problem does this feature solve?
Drag onto the Series area of the chart to select a range to request the ability to obtain a value that corresponds to the range. ECharts does not offer these features, and although you can make them similar with dataZoom, there is always a problem of zooming.
Toolbox allows you to use similar functions, but you must always click the button to activate the drag.

What does the proposed API look like?
chartInstance.on('selection', (event) => {
  // Example of dragging Series to get the value of the X-axis
  if (event.xAxis[0]) {
    const xAxisInfo = event.xAxis[0]
    const startValue = xAxisInfo.start
    const endValue = xAxisInfo.end
    ...
  }

  // If the return value is false, only the drag event is called and the operation is terminated
  return false
})

toolbox brush provides this functionality and more, see live example.
If you don't want to use toolbox, then try building a custom drag with events mousedown and mouseup.

toolbox brush provides this functionality and more, see live example. If you don't want to use toolbox, then try building a custom drag with events mousedown and mouseup.

Thank you. I'll apply it.


Add IndoNLI dataset

This PR adds IndoNLI dataset, from https://aclanthology.org/2021.emnlp-main.821/

@lhoestq thanks for the review! I've modified the labels to follow other NLI datasets.
Please review my change and let me know if I miss anything.


Changed field propagator interface to account for error and modified examples to use this.

This addresses a bug in the field propagator, currently hard-limited to 10 iterations. An interface change was needed to:

allow the user define the cutoff in number of iterations, if not used set to 100
return the propagation success flag, set to false if the cutoff is being hit, giving the responsibility to the caller to kill the track (likely a slow electron)
require the user to call with a safety value computed at the starting point. However, if the value is not computed, the method will accept 0.0, with the meaning (not using it). This feature will be preserved in the further optimizations

The safety value is not used in this PR, but allows further optimizations based on safety without changing the interface.
After this change, the output for the examples doing propagation in field is numerically identical (checked for example9 to 12 and TestEM3). This addresses the discussion opened on #148

I'm fine with the change of the name of the method (no strong opinion about it). I can also accommodate returning the distance as before and pushing the safety parameter to the end to get a default value, but the examples need to be changed even if not using safety, to account for the possible error in propagation. Let's try to merge first #149 and adapt this after.

FWIW the conditions are met 77 times in TestEm3 with magnetic field when simulating 1000 electrons (the other examples are indeed not affected, possibly because they don't have materials yet and example12 and example13 don't enable magnetic fields).

Yes, 321 tracks are killed in propagation when simulating 100 electrons in example13 with a magnetic field of Bz=1T.

Indeed, if in a particular example this becomes significant, we have the means to increase the default number of propagation sub-steps tolerated (default is 100)


KO Components inside sortable

Hello, is something along the lines of this, possible with the plugin as it is at the moment?

Also, what is the binding context of the sortable binding?

Ok, it can be done, it wasn't rendering because of a different issue, thanks!


Add o2.pl to domains list

Hello,
The list is auto-generated. If you want to add the domain, please add it at script level:
https://github.com/Kikobeats/free-email-domains/blob/master/scripts/postinstall.js#L12


Protobuf and .NETFramework 4.0

After some latest commits about Protobuf I get warning:
The primary reference "Google.Protobuf" could not be resolved because it was built against the ".NETFramework,Version=v4.5" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.0"
Can we change Protobuf to work with .NETFramework 4.0?

Hello Kapeck,
Or-tools was using beta versions of Protobuf, which was hard to maintain. Now we're using Protobuf 3.0.0 which, as mentionned in the release notes, targets .NET 4.5.
We're sorry about this compatibility problem.
Regards


Add bug bash to the release process

Why
I find bug bash as a very pragmatic testing tool. While automated tests can find a lot of issues their test suite is "closed". Moreover, nothing replaces the creativity of a human being trying to break things. If colleagues from other teams could be involved they can stress also other problems that we are not aware of  (like inconsistencies between languages, not clear documentation). At last bug bash is usually end-to-end testing therefore it helps finding issues in other systems such as Collector or Backend.
Side note: I remember that we had periodic bug bashes organized by @flands. I found them useful also as an onboarding experience (and I was able to find issues as a newbie),
What
Add bug bash to the release process.

I do not like the idea of manual regression phase. Before release we should check the installation process for msi/other packages. Other things should be covered by our CI process.
If we speaking about cross GDI Bug Bash - I cannot imagine that we will organize for each release (not only .NET release).

I do not like the idea of manual regression phase. Before release we should check the installation process for msi/other packages. Other things should be covered by our CI process.

The idea is not to have a "regression test" but "exploration test".
The testing installation packages should be automated. I propose to create stories for instead of doing it manually.

I see no interest. Closing


User image expands when adding a comment

When adding a comment as a Git Hub user who has not set their avatar (Git profile pic), the default git image does not scale down properly.

PR #626 merged hence closing this issue.


Fix AIOOBE in Curve.java, line 119

When the for-loop in line 99-104 terminates without going into a break, indexPos will have a value of 4 and generate an AIOOBE in line 119 if furthermore index1 == index2 holds (which can happen, I recreated the complex planet example and was faced with this). Using lastIndex fixes this problem.

Coverage remained the same at 0.107% when pulling 236da2f350c808bc5e11c7abc5f726085b2b8c8b on NetzwergX:bugfix into 23db1232d80edd5228c23a4cc0b71485b935b5ba on flow:develop.

You are right. I took a look at the original libnoise, the corresponding code is fount at curve.cpp, line 102-104:
if (index1 == index2) {
return m_pControlPoints[index1].outputValue;
}
I will change the PR accordingly

Coverage increased (+0.3%) to 0.428% when pulling c5f925c12c12556143441bdf84cdb3214231fc85 on NetzwergX:bugfix into 23db1232d80edd5228c23a4cc0b71485b935b5ba on flow:develop.

Has this been fully tested? I'll take your word for it and merge it.

I am in the process of porting the complex planet example from libnoise to flow noise. With this change, the flow noise version comes significantly closer to the original libnoise version (its still not a 100% match, but I suspect there are other modules who have slight differences, which I am still tracking down).


Possibly wrong big data serialization

The .data Array{Uint8,1} returned has a bunch of 0x00 at the end causing the JSON string in it to be miss-formed.  It seems to work with similar code on a small piece of data.

WIP, more to follow on Wednesday

Okay, this we have this figured out. Its actually two bugs:

Our gateway was doing strange encoding on the binary data. Thats why we could only see it on the running cloud server. That explains why it was larger. Ive changed the SDK to work with strings and it base64 encodes binary data when you use the helper encodeBinaryData.
In addition, our server-side stream reader was truncating the data if it was large. Subequently changed out and im redeploying it tonight. That explains why we saw the ends of the files as zeros and only saw it with data > a couple kb

So, two changes made. @affie you may need to switch back to strings for data.  But otherwise should be ok.

#108

Can we confirm that all is ok?


fix(readme): add missing packages

This PR adds two missing packages needed by make serve command in step 7
$ make serve
poetry run mkdocs serve
ERROR    -  Config value 'theme': Unrecognised theme name: 'material'. The available installed themes are: mkdocs, readthedocs
ERROR    -  Config value 'plugins': The "mermaid2" plugin is not installed
Aborted with 2 Configuration Errors!
make: *** [Makefile:137: mkd-serve] Error 1

This shouldn't be necessary, as we have this in the pyproject.toml:
https://github.com/linkml/linkml-project-cookiecutter/blob/60667e6d9dba52e894a7f3356dbbc1ab5580fd72/{{cookiecutter.project_name}}/pyproject.toml#L22

Closing for now


fix(dropdown): fixed bottom border for split button w/ action expanded

fixes https://github.com/patternfly/patternfly-next/issues/2448

PatternFly-Next preview: https://patternfly-next-pr-2449.surge.sh

:tada: This PR is included in version 2.41.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


History Cleanup Doesn't cleanup Entries

The working theory is that the HistoryCleanup Daemon might be trying to cleanup the process engine that doesn't house records.
There are two applicationContexts:
cws-ui:
	<bean id="processEngineConfiguration"  class="org.camunda.bpm.engine.spring.SpringProcessEngineConfiguration">
        <property name="processEngineName" value="default" />

cws-engine:
	<bean id="processEngineConfiguration"
		  class="org.camunda.bpm.engine.spring.SpringProcessEngineConfiguration">
        <property name="processEngineName" value="default2" />

Invoking the REST API to cleanup doesn't seem to work either.
Example of records not cleaned up:
MariaDB [cws]> select TYPE_, PROC_DEF_KEY_, TIME_, REMOVAL_TIME_ from ACT_HI_DETAIL order by REMOVAL_TIME_ asc limit 5;
+----------------+---------------+-------------------------+-------------------------+
| TYPE_          | PROC_DEF_KEY_ | TIME_                   | REMOVAL_TIME_           |
+----------------+---------------+-------------------------+-------------------------+
| VariableUpdate | disk_usage    | 2021-01-14 18:57:15.215 | 2021-01-28 18:57:15.532 |
| VariableUpdate | disk_usage    | 2021-01-14 18:57:15.216 | 2021-01-28 18:57:15.532 |
| VariableUpdate | disk_usage    | 2021-01-14 18:57:15.216 | 2021-01-28 18:57:15.532 |
| VariableUpdate | disk_usage    | 2021-01-14 18:57:15.216 | 2021-01-28 18:57:15.532 |
| VariableUpdate | disk_usage    | 2021-01-14 18:57:15.216 | 2021-01-28 18:57:15.532 |
+----------------+---------------+-------------------------+-------------------------+
5 rows in set (0.06 sec)

Fix:
https://github.com/NASA-AMMOS/common-workflow-service/pull/106


MaxJobTime does not match documentation for a value of 0

According to the documentation:

MaxJobTime seconds
Specifies the maximum time a job may take to print before it is canceled. Set to "0" to disable cancellation of "stuck" jobs. The default is "10800" (3 hours).

But setting "MaxJobTime 0" as the document suggests, instead of disabling the cancellation of stuck jobs, causes jobs to be canceled as "stuck" basically immediately. In my testing, perhaps half of the jobs made it through and were printed and half were canceled immediately by CUPS before they could be printed.
Relevant log line:
I [13/Nov/2018:21:41:11 -0600] [Job 2513920] Canceling stuck job after 0 seconds.

Investigating...

Probably best to set the job-cancel-after-default value to "no-value" (IPP_TAG_NOVALUE) so that we don't default it...

[master 8c7143551] Fix handling of MaxJobTime 0 (Issue #5438)
[branch-2.2 3e4dd4145] Fix handling of MaxJobTime 0 (Issue #5438)


Add autoupdate

updater.js can be executed as a stand-alone script to apply a K8s manifest, which will update bored-agent
This can be executed as e.g. a CronJob in a Kubernetes cluster
This will add and update resources, not delete
Node was updated to v16 to support String.prototype.replaceAll

Seems there was no @types/node, wonder how does typescript know which node version types to use...

There were many deps that depended on @types/node, but good point, I added it explicitely.


Margin causes different colour background in dropdown menu?

I've setup a simple demo to test things out and I've noticed when I just use a
 tag, its margin causes a different coloured background to show up? - I'm not sure how to get rid of this or why it is even there? I just want the dropdown menu to be white.
I've reproduced the issue here: https://codepen.io/embluk/pen/NWgPqvL
Here is a screenshot of the problem:

You see how the margin of the
 causes a change in background colour compared to the white background?

After some more testing it seems like the different background colour is coming from the .vsm-background-alt class.
What is this class for, am I doing something incorrectly, it feels wrong to just set the background-color of .vsm-background-alt to white in order to fix this issue?

Sorry, a little busy, quick answer (if I understood the question correctly):
Initially, the component was created on the basis of Stripe navigation, and there were 2 backgrounds that had different animations (you can see it on the component's website). So, this second background will be displayed if you have 2 or more root elements.
Those. instead of:
<component-item>

</component-item>

Need:
<component-item>

</component-item>

So..


Questions re refactoring from rc2 to rc4

Apologies for batching some questions together. I am refactoring my code to update to rc4 and have some left over bits I can't quite find the equivalents for. In each case I think a function will have just moved and it would be great if someone could point out where the function is now:

NeoFS client.New() doesn't exist anymore. What is the equivalent of:

	cli, err := client.New(
		// provide private key associated with request owner
		client.WithDefaultPrivateKey(privateKey),
		// find endpoints in https://testcdn.fs.neo.org/doc/integrations/endpoints/
		client.WithURIAddress(string(network), nil),
		// check client errors in go compatible way
		client.WithNeoFSErrorParsing(),
	)

My guess is something like:
func NewClient2(privateKey ecdsa.PrivateKey, network FS_NETWORK) {
	var c client.Client
	var prm client.PrmInit
	prm.SetDefaultPrivateKey(privateKey)
	c.Init(prm)
	var dialPrm client.PrmDial
	dialPrm.SetServerURI(string(network))
	//do I then do things like // cnrResponse, err := cli.ContainerPut(ctx, prmContainerPut) here
	//and then do I have to dial after calling ContainerPut?? Or is calling e.g cli.ContainerPut enough?
	err := c.Dial(dialPrm)
}

however I am not sure if/when to call Dial as above, and whether I need to dial at all. Previously I just could call things like ContianerPut on the client....

Creating session tokens for object actions.
My understanding is now session tokens have been seperated into session tokens for objects and session tokens for containers (??).
On that basis I am trying to update my functions that create the session tokens for the containers/objects. Here is my function part way through a refactor for getting an object

func CreateSessionWithObjectGetContext(ctx context.Context, cli *client.Client, owner *user.ID, containerID cid.ID, expiry uint64, key ecdsa.PrivateKey) (session.Object, error) {
	var prmSessionCreate client.PrmSessionCreate
	prmSessionCreate.SetExp(expiry)
	stoken := session.Object{}
	res, err := cli.SessionCreate(ctx, prmSessionCreate)
	if err != nil {
		return stoken, err
	}
	stoken.ForVerb(session.VerbObjectGet)
	stoken.BindContainer(containerID)
	//key here
	stoken.SetAuthKey(key) //what should this be now?
	stoken.SetSessionKey(res.PublicKey())
	stoken.SetID(uuid.New())
	stoken.SetExp(expiry)
	if owner == nil {
		tmpOwner := wallet.OwnerIDFromPrivateKey(key)
		owner = &tmpOwner
	}
	//setting the owner here
	stoken.SetOwnerID(owner)

	err = stoken.Sign(key)
	if err != nil {
		return stoken, err
	}
	return stoken, nil
}

I am not sure how I am supposed to set the Session/Auth key now, for the session token? and also how to set the owner (which is now in the User pkg). Also, the code res, err := cli.SessionCreate(ctx, prmSessionCreate) still seems valid, however unused by my code, which makes me think I am missing something there too?
Am I right in thinking that now I need seperate/different session tokens if the action is on a container as opposed to an object? (but thats not true for bearer tokens?)

I have functions that generate unsigned session tokens so that they can be signed out of band. Similarly to above these seem to have changed, but the main part I am interested in understanding how to do now is marshalling the session token:

	rawSessionToken := st.ToV2()
	binaryData, err := rawSessionToken.GetBody().StableMarshal(nil)

where st is my sessionToken I have created (but not yet signed). V2 seems to have been deprecated, so how would I sign it now?

Just to confirm, passing a containerID from string to object, used to be

    cntID := cid.ID{}
    cntID.Parse(str)

Its now
    cntID := cid.ID{}
    cntID.Encode([]byte(str))

?
I think that should set me on my way.
Thank you all!
System:
github.com/nspcc-dev/neofs-sdk-go v1.0.0-rc.4
Darwin AW-RD-MBP-20 20.6.0 Darwin Kernel Version 20.6.0: Tue Apr 19 21:04:45 PDT 2022; root:xnu-7195.141.29~1/RELEASE_X86_64 x86_64
github.com/nspcc-dev/neofs-api-go/v2 v2.12.2
github.com/nspcc-dev/neo-go v0.98.2

Making some progress:
I believe I can create a new client like so:
//formalise the new client based on this:
//https://github.com/nspcc-dev/neofs-sdk-go/blob/3953c2166e538ec2ab5531434f019c0efec8ac33/pool/pool.go
//you can see how to wait for things (creation/deletion here too)
func NewClient(privateKey ecdsa.PrivateKey, network FS_NETWORK) (client.Client, error) {
	var c client.Client
	var prm client.PrmInit
	prm.SetDefaultPrivateKey(privateKey)
	c.Init(prm)
	var dialPrm client.PrmDial
	dialPrm.SetServerURI(string(network))
	//todo set a tieout on the dial object
	err := c.Dial(dialPrm)
	return c, err
}

and a session to get an object like so:
func CreateSessionWithObjectGetContext(ctx context.Context, cli *client.Client, owner *user.ID, containerID cid.ID, expiry uint64, key ecdsa.PrivateKey) (session.Object, error) {
	var prmSessionCreate client.PrmSessionCreate
	prmSessionCreate.SetExp(expiry)
	stoken := session.Object{}
	res, err := cli.SessionCreate(ctx, prmSessionCreate)
	if err != nil {
		return stoken, err
	}
	stoken.ForVerb(session.VerbObjectGet)
	stoken.BindContainer(containerID)
	//key here
	stoken.SetAuthKey((*neofsecdsa.PublicKey)(&key.PublicKey))
	stoken.SetID(uuid.New())
	//todo set these
	//tok.SetExp(11)
	//tok.SetNbf(22)
	//tok.SetIat(33)
	stoken.SetExp(expiry)
	err = stoken.Sign(key)
	if err != nil {
		return stoken, err
	}
	return stoken, nil
}

note - here still have the question of when/what to do with res, err := cli.SessionCreate(ctx, prmSessionCreate) now? when do I use that?

Hey @amlwwalker nice to see you as an SDK user! At the moment the code changes and breaks a lot, but this is only for the preparation of the first release. In the future we will ensure backward compatibility.
Client
We've changed client package to be the most generic.

I believe I can create a new client like so:

Yes, that's currently the right way to open client connection to the remote NeoFS storage node, and it won't be changed. Your function NewClient what we leave for the implementation of applications for now, however, perhaps in the future we will provide it natively as an auxiliary function. Separating control over the client into separate steps is necessary for applications that regulate the stages of initialization, connection, etc. Btw we discussed with @alexvanin that pool.Pool entity can be considered as a sugared client.Client in some sense - it is easier to initialize, sessions are opened inside calls, etc. But for now main goal of pool.Pool is switching connections, and working with it in single-client mode adds costs that we have not yet accurately estimated.
Sessions
The only thing for which the Client.SessionCreate now needed is the trusted object creation in remote node side: specify container, owner, user attributes, payload and here we go. To do this private session key should be allocated on server side since it's needed for object formation. In total, it makes sense to form a session through a handshake with a node only for PUT action.
For other actions, you do not need to contact the node directly: just fill in all the fields and sign the token.

Am I right in thinking that now I need seperate/different session tokens if the action is on a container as opposed to an object?

Yes, object and container sessions are used in different contexts, although they are similar ideologically and structurally.

so how would I sign it now?

Please clarify whether session tokens are signed in the runtime of your application, or is it done completely "offline"? We plan to provide the ability to override the session signature mechanism at runtime, but did not expect the need to do this outside the application. See https://github.com/nspcc-dev/neofs-sdk-go/blob/721df386c59999dd3566b9c3fa7429d944e16bff/crypto/signer.go#L46-L62
ID encoding
All identifiers provide at most three encodings:

string (EncodeToString / DecodeString)
fixed-length binary (Encode / Decode)
protobuf binary (Marshal / Unmarshal)

So old cntID.Parse(str) should be replaced with cntID.DecodeString(str).

Hi @cthulhu-rider
Thanks for getting back to me - really helpful.
So just to confirm with you - would be great to check if I am doing this correctly now for Object session generation. I am creating helper functions to create the sessions for GET, HEAD, PUT, DELETE objects - as you say, the only one that needs client.SessionCreate is PUT - do I need to attach it to the session in anyway? Anyway is this what you would consider correct construction of object sessions:
func generateSessionObject(verb session.ObjectVerb, lt acl.TokenLifetime) session.Object{
	var sToken session.Object
	sToken.ForVerb(verb)
	sToken.SetID(uuid.New())
	sToken.SetExp(lt.GetExp())
	sToken.SetNbf(lt.GetNbf())
	sToken.SetIat(lt.GetIat())
	return sToken
}
func CreateSessionWithObjectGetContext(ctx context.Context, cli *client.Client, containerID cid.ID, expiry int64, key ecdsa.PrivateKey) (session.Object, error) {
	var lt acl.TokenLifetime
	lt.SetExp(CalculateEpochsForTime(ctx, cli, expiry)) //set the token lifetime seconds
	sToken := generateSessionObject(session.VerbObjectGet, lt)
	sToken.SetAuthKey((*neofsecdsa.PublicKey)(&key.PublicKey))
	sToken.BindContainer(containerID)
	err := sToken.Sign(key)
	if err != nil {
		return sToken, err
	}
	return sToken, nil
}
func CreateSessionWithObjectHeadContext(ctx context.Context, cli *client.Client, containerID cid.ID, expiry int64, key ecdsa.PrivateKey) (session.Object, error) {
	var lt acl.TokenLifetime
	lt.SetExp(CalculateEpochsForTime(ctx, cli, expiry)) //set the token lifetime seconds
	sToken := generateSessionObject(session.VerbObjectHead, lt)
	sToken.SetAuthKey((*neofsecdsa.PublicKey)(&key.PublicKey))
	sToken.BindContainer(containerID)
	err := sToken.Sign(key)
	if err != nil {
		return sToken, err
	}
	return sToken, nil
}
func CreateSessionWithObjectPutContext(ctx context.Context, cli *client.Client, containerID cid.ID, expiry int64, key ecdsa.PrivateKey) (session.Object, error) {
	var lt acl.TokenLifetime
	lt.SetExp(CalculateEpochsForTime(ctx, cli, expiry)) //set the token lifetime seconds
	sToken := generateSessionObject(session.VerbObjectPut, lt)
	var prmSessionCreate client.PrmSessionCreate
	prmSessionCreate.SetExp(lt.GetExp())
	prmSessionCreate.UseKey(key)
	//no longer needed for anything except PUT <<---- @[cthulhu-rider](https://github.com/cthulhu-rider) here....
	//do I need to set it on sToken somehow, or just call it?
	_, err := cli.SessionCreate(ctx, prmSessionCreate)
	if err != nil {
		return sToken, err
	}
	sToken.SetAuthKey((*neofsecdsa.PublicKey)(&key.PublicKey))
	sToken.BindContainer(containerID)
	err = sToken.Sign(key)
	return sToken, err
}
func CreateSessionWithObjectDeleteContext(ctx context.Context, cli *client.Client, containerID cid.ID, expiry int64, key ecdsa.PrivateKey) (session.Object, error) {
	var lt acl.TokenLifetime
	lt.SetExp(CalculateEpochsForTime(ctx, cli, expiry)) //set the token lifetime seconds
	sToken := generateSessionObject(session.VerbObjectDelete, lt)
	sToken.SetAuthKey((*neofsecdsa.PublicKey)(&key.PublicKey))
	sToken.BindContainer(containerID)
	err := sToken.Sign(key)
	if err != nil {
		return sToken, err
	}
	return sToken, nil
}

OK those scenarios make sense @cthulhu-rider thanks.
When you say "pass the public key of the party you trust" you mean whoever is making the request as opposed to the key of a storage node I assume (as i don't know how I would know the storage nodes key....)
With regards to:
	create := client.PrmSessionCreate{}
	create.SetExp(lt.GetExp())
	_, err := cli.SessionCreate(ctx, create)
	if err != nil {
		return session.Container{}, err
	}

this is the one element I'm still not sure of - is it necessary? I don't really get it - its not attached to the session object as far as I can tell in anyway...

@amlwwalker sorry for the delay in reply.
If you use cli.SessionCreate storage node creates a new private key (which definitely differs with node's master key) and responds with client.ResSessionCreate which provides PublicKey method to receive public key of just created private session key. This scenario is only needed for object creation.
	create := client.PrmSessionCreate{}
	create.SetExp(lt.GetExp())
	res, err := cli.SessionCreate(ctx, create)
	if err != nil {
		return session.Container{}, err
	}

	var pubkey neofsecdsa.PublicKey
	err = pubkey.Decode(res.PublicKey())
	if err != nil { ... }

	var token session.Container
	token.SetAuthKey(pubkey)

	// put object via client

For all other needs you should just receive public key of some party in any convenient way and write it into session token.
Did I understand your question correctly?

@cthulhu-rider - this is great and a very important change in the rc4 - thank you. I have been using the request maker's public key here but the auth key needs to be the key from the client session!
and to make sure I understand, when doing anything except an object (or container???) creation, I can just use my public key in in the SetAuthKey field? - i.e it doesn't matter what its set to?
Thanks

@amlwwalker it makes no sense to pass your own key into SetAuthKey - you should pass some other key. Subsequently, when providing a token, the system will consider that the operation is performed on behalf of the person specified in SetAuthKey.
For example, you want me to create containers on your behalf. To do this, I send some public key KEY_LEO to you. You compose session token:
var token session.Container
token.SetAuthKey(KEY_LEO)
// set other fields
token.Sign(KEY_ALEX)

After that you send me the token, and I can create containers using the session:
var cnr container.Container
cnr.SetOwner(ALEX)
// set other fields

var prm client.PrmContainerPut
prm.SetContainer(cnr)
prm.WithinSession(TOKEN_FROM_ALEX)

var prmInit client.PrmInit
prmInit.SetDefaultPrivateKey(KEY_LEO)

var cli client.Client
cli.Init(prmInit)
// dial

_, err := cli.ContainerPut(ctx, prm)

So it will be your own container, but I created it using my KEY_LEO because you issued a power of attorney for this key.
If you do things from you person with your KEY_ALEX, you can just do it without a session - it's optional.

Seems to be done to me (and rc4 is outdated now anyway).


Issue with GET Request Params in Version 1.7

Loving this module so far, thank you for all your hard work on it!
There seems to be a bug in version 1.7 where if I build some query parameters and attach them to a GET request, they seem to not be formatted correctly and I am getting errors from the server I am making a request from. I can roll back the version to 1.6 and I don't get the same issue, but when I roll back forward again (with the same grasshopper code and inputs) and get the failure response from my server.
Let me know if there is a way to provide any more details to help resolve this issue.
Thanks!

Hi @ZB-Alex. Thanks for reporting! Due to some internal refactoring and a more stringent URL validation in version 1.7, passing query params as part of a URL is no longer allowed. Now you have to use the Create Query Param (CQP) component and then pass it to P input of the GET component. Let me know if this solves your issue


thoughts on reading from microphone data?

Any tips/tricks on how to tie in live microphone data into this library? Similar to the dictation system on macOS.

see my fork, https://github.com/lake-of-fire/SwiftWhisperStream with discussion in the other issue thread on streaming

Closing in favour of https://github.com/exPHAT/SwiftWhisper/issues/4


When some tag is missing in pom.xml, maven2sbt produce a empty string as value of tag in build.sbt

Bug
When some tag is missing in pom.xml, maven2sbt produce a empty string as value of tag in build.sbt
To reproduce
Given in the input pom.xml, <name> is missing in <repository>:
    <repository>
      <id>cloudera</id>
      <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
    </repository>

When run:
$ sbt package
..
[error] java.lang.IllegalArgumentException: requirement failed: Repository name cannot be empty.
[error] Use 'last' for the full log.
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?

Then the output build.sbt generated contains:
lazy val root = (project in file("."))
  .settings(
    resolvers ++= List(
      "" at "https://repository.cloudera.com/artifactory/cloudera-repos/" // exception because empty string here
    )
    )
  )

Solution possible
Use a default value instead of a empty string

Thanks @zhenleibb. Another bug found! 🙂

I'm working on it to fix this issue. I've decided not to use a default value since there can be more than one <repository> element with no name tags. I can use some default value with a random number suffixed or add a number in sequence but it seems like make a simple solution more difficult to solve.
My solution is

If <name> is missing or is an empty text (i.e. <name></name>), use <Id> value.
If <id> is missing as well, use the repo URL as the name.

<id> in the <repository> can be missing according to the schema for pom.xml.
http://maven.apache.org/xsd/maven-4.0.0.xsd
...
<xs:complexType name="Repository">
  <xs:annotation>
    <xs:documentation source="version">4.0.0+</xs:documentation>
    <xs:documentation source="description">A repository contains the information needed for establishing connections with remote repository.</xs:documentation>
  </xs:annotation>
  <xs:all>
    <xs:element minOccurs="0" name="releases" type="RepositoryPolicy">
      <xs:annotation>
        <xs:documentation source="version">4.0.0+</xs:documentation>
        <xs:documentation source="description">How to handle downloading of releases from this repository.</xs:documentation>
      </xs:annotation>
    </xs:element>
    <xs:element minOccurs="0" name="snapshots" type="RepositoryPolicy">
      <xs:annotation>
        <xs:documentation source="version">4.0.0+</xs:documentation>
        <xs:documentation source="description">How to handle downloading of snapshots from this repository.</xs:documentation>
      </xs:annotation>
    </xs:element>
    <xs:element minOccurs="0" name="id" type="xs:string">
   ...
    </xs:element>
  </xs:all>
</xs:complexType>
...

The minOccurs for id is 0.


Different results between version 1.3.1 and 1.4.0 for ordinary kriging with loop backend

First of all, thanks for providing this great Kriging tool! I have been using it for more than a year now, mostly on version 1.3.1.
After the update from version 1.3.1 to 1.4.0 (on conda-forge) suddenly one of my test which uses pykrige failed (https://github.com/pycomlink/pycomlink/issues/39). The differences in the interpolated fields were small but not so small that I would attribute them to numeric uncertainty.
I reproduced these differences between version 1.3.1 and version 1.4.0 in this notebook
https://gist.github.com/cchwala/fa498bbe64106b16f9f74b63fa4f8369
Which changes in the code might have caused that? Was the order changed, with which the loop goes through the points?
Which version to trust more?

Thanks for the report and the analysis!
Git bisect points to https://github.com/bsmurphy/PyKrige/pull/47 using the following test script (adapted from your gist),

import numpy as np
import pykrige
from numpy.testing import assert_allclose

def f(x, y):
    return (1 - x / 2 + x ** 5 + y ** 3) * np.exp(-x ** 2 -y ** 2)

n = 256
x = np.linspace(-3, 3, n)
y = np.linspace(-3, 3, n)
X, Y = np.meshgrid(x, y)

Z = f(X, Y)

np.random.seed(42)

N = 100
x_points = np.random.rand(N) * 6 - 3
y_points = np.random.rand(N) * 6 - 3

z_points = f(x_points, y_points)

ok = pykrige.OrdinaryKriging(x_points, y_points, z_points, variogram_model='spherical', weight=True)

Z_from_ok, _ = ok.execute(style='points', xpoints=X.flatten(), ypoints=Y.flatten(), backend='loop')
Z_from_ok = Z_from_ok.reshape(X.shape);

assert_allclose(Z_from_ok[0, :3],
                np.array([0.02247102, 0.02158971, 0.02070663]),
                rtol=1e-5)

will try to bisect that PR to see where this is coming from exactly and whether it was intentional.

More specifically the change in results appeared after https://github.com/bsmurphy/PyKrige/pull/47/commits/31ab7e33c3482a0e41007a854499b09fd227b7a2 from https://github.com/bsmurphy/PyKrige/pull/47
As you are using weight=True, my guess is that this is due to this change or otherwise to the fact that minimization bounds in the variogram fitting changed. @bsmurphy might be able to confirm.
Overall this looks intentional, but any case it might be good to add a few tests that constrain the output, and make sure it is not changed unless some bug fixes (or intentional improvements) are applied. We should also document this in the release notes.

Thanks for the info and digging through the changes.


Move search to top bar in mobile view

In the mobile the search bar should be moved to the main menu bar similar to the Android app.
Please show a search button and expand this search button into a search field on click.

@mariobehling I would like to work on this

@utkarshnath Please proceed. There is no one working on it presently. Thank you so much!!


feat: empty release

:tada: This PR is included in version 1.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Backend validation with $scope.$broadcast issue

$scope.schema = {
	"$schema" : "STANDARD SCHEMA",
	"title" : "Schema Title",
	"description" : "Schema Description",
	"id" : "Schema Id",
	"type" : "object",

	"properties" : {
		"name" : {
			"type" : "string",
			"title" : "Name",
			"description" : "Enter your full name"
		}
	}
};

$scope.form = [
	{
		"key" : "name"
	}
];

$scope.model = {};

$scope.handleForm = function(myForm){

//myForm is the ngFormController
		if(myForm.name.$viewValue === "Ajay"){
			$scope.$broadcast("schemaForm.error.name", "invalidName", "Invalid name");
		}

		else{
			$scope.$broadcast("schemaForm.error.name", "invalidName", true);
		}

		console.log("Model Errors :: ", myForm.name.$error);
		console.log("View Value :: ",myForm.name.$viewValue);
		console.log("Model value :: ",$scope.model);
		console.log("");
}

Hi,
I've found a bug in schema form.
User enters "Ajay" and clicks the submit button which calls the handleForm function on the scope. The handleForm function takes one argument, the ngFormController.
console output :
Model Errors ::  Object {invalidName: true}
View Value ::  Ajay
Model value ::  Object {name: "Ajay"}
The field is invalid and the model has one error {invalidName: true}. Now, the user enters "Ajay U" and clicks the submit button.
console output :
Model Errors ::  Object {schemaForm: true}
View Value ::  Ajay U
Model value ::  Object {name: undefined}
As you can see, the field is invalid and the model value is undefined and has an error {schemaForm: true}.
I'm confused :/ Is it a bug or am I missing something?

#642 is reported as fixing this issue


pause n resume upload

http://resumablejs.com/

+1

+1

+1

+1
Also, just found https://github.com/flowjs/flow.js. Looks like it has the pause/resume ability.

Feature added to version 7.3.2

Thanks!


Star the notes

Is this feature still up for grabs? If so, I would love to help!

This feature is not started yet. Please help!

@kazup01 After beginning development, I have a couple questions:

How should users be "starring" or "favoriting" these notes? I currently created the star icon as a toggle switch. To "favorite" the note, just tap the star-outline icon and it will mark the note as starred. This will also update the icon to be filled in.
What is the function of the toggle switch in the picture above? To only display "starred" notes?

Thanks!

Hi @robbawebba , thanks for your development!

I think toggle switch is OK.
Yes, its function is filtering notes.

Could you push your codes and create a PR?
Status of the PR will be work in progress.
It is OK.
I can check your codes earlier.

@kohei-takata Sorry for the delay! Work has been pretty busy lately so I've been behind on my personal development. I'll be sure to create an in-progress PR either today or tomorrow for the work I have so far.


Emmet problem in multi line edit and intellisense list

in multi line edit emmet not working at all.

when i press tab it set first item of intellisense list , we can ignore intellisense list when tab pressed because in speed write it very bad.

Closing this as duplicate of #9571

@pixieaka
Set emmet.useNewEmmet to true (in the latest Insiders dated 6/1 or above and eventually in the stable release of 1.13 and higher) to get emmet abbreviation expansions in the suggestion list.
This will work in multi cursor too
For details on this setting read https://github.com/Microsoft/vscode/issues/1952#issuecomment-305390464


feat(store): unapply SNARK work

Describe your changes
Adds the ability to "unapply" SNARK work upon reorg
Link issue(s) fixed

#1561

Checklist before requesting a review

[x] I have performed a self-review of my code.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] I have verified new and existing tests pass locally with my changes.
[ ] I verified whether it was necessary to increment the database version.

This is the PR before fixing a merge conflict and adding a test. One can review this and I'll push the remaining stuff in the morning.


Does it support VB.NET?

Does the library support WIndows Form Application using VB.NET on Framework 4.5.1?
See screen shot below. I can't get any mouse or keyboard events from the IKeyboardMouseEvents. From the intellisense dropdown list on m_GlobalHook (declared as IKeyboardMouseEvents) there is no events available to me to subscribe.

Realized that I have made a mistake in my VB coding. Should use AddHandler m_GlobalHook to wire up the event with a callback method. Closing this issue. Thanks.


Reopening bug in M3 - ChatResponse Metadata is overwritten by advisors

Reopening the issue reported in 1309
The issue is still not resolved.
I tried with M3 and ChatResponseMetadata still has EmptyUsage and EmptyRateLimit objects

@chetan-galgali can you share the model you are using with? Also you streaming or call?
Actually can you share a code that reproduce the issue please.

I cannot share the code, but below are the details:
model - gpt-35-turbo-16k
Dependency - spring-ai-azure-openai-spring-boot-starter
Code that calls the model (Its a call):
ChatResponse chatResponse = aiClient
                .prompt()
                .messages(systemMessage)
                .advisors(new QuestionAnswerAdvisor(vectorStore), new SimpleLoggerAdvisor())
                .user(userQ)
                .call()
                .chatResponse();
        ChatResponseMetadata metadata = chatResponse.getMetadata();
        Usage usage = metadata.getUsage();
        log.info("chatResponseMetadata: {}", metadata);
        log.info("Usage: {}", usage);
        log.info("usage.getPromptTokens: {}", usage.getPromptTokens());
        log.info("usage.getGenerationTokens: {}", usage.getGenerationTokens());
        log.info("chatResponse.tokenCount: {}", usage.getTotalTokens());

All the logging statements log as 0 tokens. But if I debug the code, AzureOpenAiUsage object has the right details, but somewhere, it gets overwritten with EmptyUsage objects.

Thank you @chetan-galgali ,
can you confirm that for the same request, if you comment out the advisors you receive non-empty usage?
Have you tried to turn only only the QuestionAnswerAdvisor or the SimpleLoggerAdvisor advisors?

Hey @tzolov
Below are the results:

No advisors - works fine
Only SimpleLoggerAdvisor - works fine
Only QuestionAnswerAdvisor - returns empty object
Both together - returns empty object

Seems like the issue might be in QuestionAnswerAdvisor
Let me know if you need any additional info

I've reproduced the issue.
This is a bug and the ChatResponse Builder. We will address it shortly.
@chetan-galgali thanks you for the report and the clarification!

Thank you @tzolov . Is there any workaround I can use? I really need this feature to be working. And I cannot wait until M4

@chetan-galgali , the workaround, require an modified QuestionAnswerAdvisor(2) that does not use the old buggy CharResponse.Builder#from.
This should work:
/*
 * Copyright 2024-2024 the original author or authors.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      https://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package org.springframework.ai.chat.client.advisor;

import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Predicate;
import java.util.stream.Collectors;

import org.springframework.ai.chat.client.advisor.api.AdvisedRequest;
import org.springframework.ai.chat.client.advisor.api.AdvisedResponse;
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisor;
import org.springframework.ai.chat.client.advisor.api.CallAroundAdvisorChain;
import org.springframework.ai.chat.client.advisor.api.StreamAroundAdvisor;
import org.springframework.ai.chat.client.advisor.api.StreamAroundAdvisorChain;
import org.springframework.ai.chat.metadata.ChatResponseMetadata;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.document.Document;
import org.springframework.ai.model.Content;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.ai.vectorstore.filter.Filter;
import org.springframework.ai.vectorstore.filter.FilterExpressionTextParser;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;

import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
import reactor.core.scheduler.Schedulers;

/**
 * Context for the question is retrieved from a Vector Store and added to the prompt's
 * user text.
 *
 * @author Christian Tzolov
 * @since 1.0.0
 */
public class QuestionAnswerAdvisor2 implements CallAroundAdvisor, StreamAroundAdvisor {

	private static final String DEFAULT_USER_TEXT_ADVISE = """

			Context information is below, surrounded by ---------------------

			---------------------
			{question_answer_context}
			---------------------

			Given the context and provided history information and not prior knowledge,
			reply to the user comment. If the answer is not in the context, inform
			the user that you can't answer the question.
			""";

	private static final int DEFAULT_ORDER = 0;

	private final VectorStore vectorStore;

	private final String userTextAdvise;

	private final SearchRequest searchRequest;

	public static final String RETRIEVED_DOCUMENTS = "qa_retrieved_documents";

	public static final String FILTER_EXPRESSION = "qa_filter_expression";

	private final boolean protectFromBlocking;

	private final int order;

	/**
	 * The QuestionAnswerAdvisor retrieves context information from a Vector Store and
	 * combines it with the user's text.
	 * @param vectorStore The vector store to use
	 */
	public QuestionAnswerAdvisor2(VectorStore vectorStore) {
		this(vectorStore, SearchRequest.defaults(), DEFAULT_USER_TEXT_ADVISE);
	}

	/**
	 * The QuestionAnswerAdvisor retrieves context information from a Vector Store and
	 * combines it with the user's text.
	 * @param vectorStore The vector store to use
	 * @param searchRequest The search request defined using the portable filter
	 * expression syntax
	 */
	public QuestionAnswerAdvisor2(VectorStore vectorStore, SearchRequest searchRequest) {
		this(vectorStore, searchRequest, DEFAULT_USER_TEXT_ADVISE);
	}

	/**
	 * The QuestionAnswerAdvisor retrieves context information from a Vector Store and
	 * combines it with the user's text.
	 * @param vectorStore The vector store to use
	 * @param searchRequest The search request defined using the portable filter
	 * expression syntax
	 * @param userTextAdvise the user text to append to the existing user prompt. The text
	 * should contain a placeholder named "question_answer_context".
	 */
	public QuestionAnswerAdvisor2(VectorStore vectorStore, SearchRequest searchRequest, String userTextAdvise) {
		this(vectorStore, searchRequest, userTextAdvise, true);
	}

	/**
	 * The QuestionAnswerAdvisor retrieves context information from a Vector Store and
	 * combines it with the user's text.
	 * @param vectorStore The vector store to use
	 * @param searchRequest The search request defined using the portable filter
	 * expression syntax
	 * @param userTextAdvise the user text to append to the existing user prompt. The text
	 * should contain a placeholder named "question_answer_context".
	 * @param protectFromBlocking if true the advisor will protect the execution from
	 * blocking threads. If false the advisor will not protect the execution from blocking
	 * threads. This is useful when the advisor is used in a non-blocking environment. It
	 * is true by default.
	 */
	public QuestionAnswerAdvisor2(VectorStore vectorStore, SearchRequest searchRequest, String userTextAdvise,
			boolean protectFromBlocking) {
		this(vectorStore, searchRequest, userTextAdvise, protectFromBlocking, DEFAULT_ORDER);
	}

	/**
	 * The QuestionAnswerAdvisor retrieves context information from a Vector Store and
	 * combines it with the user's text.
	 * @param vectorStore The vector store to use
	 * @param searchRequest The search request defined using the portable filter
	 * expression syntax
	 * @param userTextAdvise the user text to append to the existing user prompt. The text
	 * should contain a placeholder named "question_answer_context".
	 * @param protectFromBlocking if true the advisor will protect the execution from
	 * blocking threads. If false the advisor will not protect the execution from blocking
	 * threads. This is useful when the advisor is used in a non-blocking environment. It
	 * is true by default.
	 * @param order the order of the advisor.
	 */
	public QuestionAnswerAdvisor2(VectorStore vectorStore, SearchRequest searchRequest, String userTextAdvise,
			boolean protectFromBlocking, int order) {

		Assert.notNull(vectorStore, "The vectorStore must not be null!");
		Assert.notNull(searchRequest, "The searchRequest must not be null!");
		Assert.hasText(userTextAdvise, "The userTextAdvise must not be empty!");

		this.vectorStore = vectorStore;
		this.searchRequest = searchRequest;
		this.userTextAdvise = userTextAdvise;
		this.protectFromBlocking = protectFromBlocking;
		this.order = order;
	}

	@Override
	public String getName() {
		return this.getClass().getSimpleName();
	}

	@Override
	public int getOrder() {
		return this.order;
	}

	@Override
	public AdvisedResponse aroundCall(AdvisedRequest advisedRequest, CallAroundAdvisorChain chain) {

		AdvisedRequest advisedRequest2 = before(advisedRequest);

		AdvisedResponse advisedResponse = chain.nextAroundCall(advisedRequest2);

		return after(advisedResponse);
	}

	@Override
	public Flux<AdvisedResponse> aroundStream(AdvisedRequest advisedRequest, StreamAroundAdvisorChain chain) {

		// This can be executed by both blocking and non-blocking Threads
		// E.g. a command line or Tomcat blocking Thread implementation
		// or by a WebFlux dispatch in a non-blocking manner.
		Flux<AdvisedResponse> advisedResponses = (this.protectFromBlocking) ?
		// @formatter:off
			Mono.just(advisedRequest)
				.publishOn(Schedulers.boundedElastic())
				.map(this::before)
				.flatMapMany(request -> chain.nextAroundStream(request))
			: chain.nextAroundStream(before(advisedRequest));
		// @formatter:on

		return advisedResponses.map(ar -> {
			if (onFinishReason().test(ar)) {
				ar = after(ar);
			}
			return ar;
		});
	}

	private AdvisedRequest before(AdvisedRequest request) {

		var context = new HashMap<>(request.adviseContext());

		// 1. Advise the system text.
		String advisedUserText = request.userText() + System.lineSeparator() + this.userTextAdvise;

		var searchRequestToUse = SearchRequest.from(this.searchRequest)
			.withQuery(request.userText())
			.withFilterExpression(doGetFilterExpression(context));

		// 2. Search for similar documents in the vector store.
		List<Document> documents = this.vectorStore.similaritySearch(searchRequestToUse);

		context.put(RETRIEVED_DOCUMENTS, documents);

		// 3. Create the context from the documents.
		String documentContext = documents.stream()
			.map(Content::getContent)
			.collect(Collectors.joining(System.lineSeparator()));

		// 4. Advise the user parameters.
		Map<String, Object> advisedUserParams = new HashMap<>(request.userParams());
		advisedUserParams.put("question_answer_context", documentContext);

		AdvisedRequest advisedRequest = AdvisedRequest.from(request)
			.withUserText(advisedUserText)
			.withUserParams(advisedUserParams)
			.withAdviseContext(context)
			.build();

		return advisedRequest;
	}

	private AdvisedResponse after(AdvisedResponse advisedResponse) {

		var chatResponseMetadataBuilder = ChatResponseMetadata.builder();

		chatResponseMetadataBuilder.withKeyValue(RETRIEVED_DOCUMENTS,
				advisedResponse.adviseContext().get(RETRIEVED_DOCUMENTS));

		chatResponseMetadataBuilder.withId(advisedResponse.response().getMetadata().getId());
		chatResponseMetadataBuilder.withModel(advisedResponse.response().getMetadata().getModel());
		chatResponseMetadataBuilder.withRateLimit(advisedResponse.response().getMetadata().getRateLimit());
		chatResponseMetadataBuilder.withUsage(advisedResponse.response().getMetadata().getUsage());
		chatResponseMetadataBuilder.withPromptMetadata(advisedResponse.response().getMetadata().getPromptMetadata());

		Set<Map.Entry<String, Object>> entries = advisedResponse.response().getMetadata().entrySet();
		for (Map.Entry<String, Object> entry : entries) {
			chatResponseMetadataBuilder.withKeyValue(entry.getKey(), entry.getValue());
		}

		return new AdvisedResponse(
				new ChatResponse(advisedResponse.response().getResults(), chatResponseMetadataBuilder.build()),
				advisedResponse.adviseContext());
	}

	protected Filter.Expression doGetFilterExpression(Map<String, Object> context) {

		if (!context.containsKey(FILTER_EXPRESSION)
				|| !StringUtils.hasText(context.get(FILTER_EXPRESSION).toString())) {
			return this.searchRequest.getFilterExpression();
		}
		return new FilterExpressionTextParser().parse(context.get(FILTER_EXPRESSION).toString());

	}

	private Predicate<AdvisedResponse> onFinishReason() {
		return (advisedResponse) -> advisedResponse.response()
			.getResults()
			.stream()
			.filter(result -> result != null && result.getMetadata() != null
					&& StringUtils.hasText(result.getMetadata().getFinishReason()))
			.findFirst()
			.isPresent();
	}

	public static Builder builder(VectorStore vectorStore) {
		return new Builder(vectorStore);
	}

	public static class Builder {

		private final VectorStore vectorStore;

		private SearchRequest searchRequest = SearchRequest.defaults();

		private String userTextAdvise = DEFAULT_USER_TEXT_ADVISE;

		private boolean protectFromBlocking = true;

		private int order = DEFAULT_ORDER;

		private Builder(VectorStore vectorStore) {
			Assert.notNull(vectorStore, "The vectorStore must not be null!");
			this.vectorStore = vectorStore;
		}

		public Builder withSearchRequest(SearchRequest searchRequest) {
			Assert.notNull(searchRequest, "The searchRequest must not be null!");
			this.searchRequest = searchRequest;
			return this;
		}

		public Builder withUserTextAdvise(String userTextAdvise) {
			Assert.hasText(userTextAdvise, "The userTextAdvise must not be empty!");
			this.userTextAdvise = userTextAdvise;
			return this;
		}

		public Builder withProtectFromBlocking(boolean protectFromBlocking) {
			this.protectFromBlocking = protectFromBlocking;
			return this;
		}

		public Builder withOrder(int order) {
			this.order = order;
			return this;
		}

		public QuestionAnswerAdvisor2 build() {
			return new QuestionAnswerAdvisor2(this.vectorStore, this.searchRequest, this.userTextAdvise,
					this.protectFromBlocking, this.order);
		}

	}
}

Thank you! @tzolov . Works perfect!


Enhancement: improve splash screen logo resolution

The splash screen logo resolution is low.
My phone specs:
Redmi note 11 pro
OS: Android 13 Miui 14
Screen 1080 x 2400 pixels, 20:9 ratio (~395 ppi density)

This is something that's been bugging me also, I should hopefully figure it out soon!

Figured it out! https://github.com/SimonHalvdansson/Harmonic-HN/commit/824cc43d1e89d9832b924693a35d2f13d246db6d


Missing Aliases "With"

There are some keywords that are specified in the rockstar, one of which is "with", which is missing from this transpiller.
We need to:

[ ] Implement with keyword
[ ] Write test cases for with

Note, is context sensitive.

On line 39 in the transpile.py file I found a with, is this the one we thought was missing?


RunnerSet Cannot Deploy to Different Namespaces in the same Cluster

Unlike RunnerDeployments, RunnerSets cannot be deployed in the same Cluster even while being in a different Namespace.
I was previously able to successfully deploy RunnerDeployments to the same Clusters all with their own controllers and had no issues with them having shared values (different helm release names - see 782).. Now, I'm deploying RunnerSet and am seeing an issue where the API call is being shared from a separate namespace, even with watchNamespace set to the namespace of the controller.
Currently running helm chart version 0.13.2 and actions-runner-controller version 0.20.2
To reproduce, create a namespace, deploy a controller with a unique release name, deploy a RunnerSet to that controller.
Create another namespace, deploy a controller with a unique release name AND a unique GitHub App installation ID, OR (for my case) a PAT for GitHub Enterprise Cloud/GitHub Enterprise Server (whichever you want, in my case I have two RunnerSets for GHES for two separate organizations using the same PAT, when I try to deploy to our organization in GHEC and use a new PAT there for that instance, I am seeing the error:
create Pod xx-ghec-runnerset-0 in StatefulSet xx-ghec-runnerset failed error: admission webhook "mutate-runner-pod.webhook.actions.summerwind.dev" denied the request: failed to create registration token: POST https://github.xxx.com/api/v3/orgs/xxx/actions/runners/registration-token: 404 Not Found []

^ the API call above should be pointed to github.com, but instead is pointed to my GHES URL, when in my values.yml I did not specify any GitHub Enterprise Server URL:
env:
  GITHUB_ENTERPRISE_URL: ""

I expect to be able to deploy RunnerSet the same way we can deploy RunnerDeployments to different Namespaces within the same Cluster.
I'm moving to RunnerSets for now as it has the ability to accept dnsPolicy: Default which solves timeout errors we were seeing when deploying to our Kubernetes cluster. Would like to go back to RunnerDeployments if the dnsPolicy behavior becomes available there as we want to utilize autoscaling.
If this functionality will not be supported for RunnerSets and you guys recommend to only use RunnerDeployments please let me know.
Thanks!

@rxa313 Thanks for reporting! I have never thought about this but in my theory, this happens due to the admission webhook server from each unique installation of actions-runner-controller are interfering with each other.
We use the admission webhook server to inject github tokens to runner pods managed by StatefulSet managed by RunnerSet. RunnerDeployment doesn't need an admission webhook server to inject tokens to runner pods as we have full control over how we create pods in that case. So, that's the difference.
Probably we need to somehow tweak our admission webhook server code:
https://github.com/actions-runner-controller/actions-runner-controller/blob/8657a34f3275f72f67b4c21ce6482e444d087c55/main.go#L255-L259
Or tweak our webhook config so that each admission webhook is involved only when the target pod is in the desired namespace.
https://github.com/actions-runner-controller/actions-runner-controller/blob/master/charts/actions-runner-controller/templates/webhook_configs.yaml

@rxa313 I have some good news. There's namespaceSelector in an admission webhook configuration that allows you to restrict that webhook to be involved only in certain namespaces.
https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector
The below is the config for the webhook that intercepts pods managed by runnersets:
https://github.com/actions-runner-controller/actions-runner-controller/blob/5805e39e1fb9d5b433352bfe7c792782663b7500/charts/actions-runner-controller/templates/webhook_configs.yaml#L71-L92
Given all that, let's say you had two releases one and two, where one has watchNamespace=foo and two has watchNamespace=bar.
You label the namespace foo with a label actions-runner-controller/id: "one", and label bar with actions-runner-controller/id: "two"`.
In one's webhook config you set namespaceSelector like:
  name: mutate-runner-pod.webhook.actions.summerwind.dev
  namespaceSelector:
    matchExpressions:
    - key: actions-runner-controller/id
      operator: In
      values: ["one"]
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    resources:
    - pods

In two's webhook config you set it like:
  name: mutate-runner-pod.webhook.actions.summerwind.dev
  namespaceSelector:
    matchExpressions:
    - key: actions-runner-controller/id
      operator: In
      values: ["two"]
  rules:
  - apiGroups:
    - ""
    apiVersions:
    - v1
    operations:
    - CREATE
    resources:
    - pods

This way, ones webhook tries to mutate runnerset pods in foo only, where twos webhook tries to mutate pods in bar only.

@rxa313 Hey! Thanks a lot for the detailed report.

I've noticed that autoscaling does work on this version of GHES

Oh, awesome! Thanks for letting us know.
I've read the release note of that GHES version but didn't find any specific change regarding ephemeral runners and new workflow_job webhook events.
Perhaps I'd better ask GitHub folks for confirmation.

Scale target not found notification -- it actually does scale up and down accordingly as the INFO right after the above log is:

Ah, good catch. I've checked our implementation and this should be ok although it looks scary.
That important point here is that each workflow_job event has an action field with any of these values- queued, in_progress, or completed.
We currently handle only queued and completed, where the former triggers a scale-up and the latter triggers a scale-down.
in_progress is unhandled and results in Scale target not found.
https://github.com/actions-runner-controller/actions-runner-controller/blob/f7e14e06e87961a67f46e22cc33ba2c707b6e682/controllers/horizontal_runner_autoscaler_webhook.go#L201-L223

@rxa313 I've reread the special note part on your last comment and still unsure what's your goal is.
Are you basically asking me to verify if your below idea seems valid to me, or anything else?

users should only have one RunnerSet and one HRA per namespace in Rancher, i. e A namespace for org 1 with it's own RunnerSet and HRA and a namespace for org 2 with it's own config. Can you provide any insight to that?

@rxa313 I believe https://github.com/actions-runner-controller/actions-runner-controller/pull/927 removes the redundant Scale target not found error messages.

@mumoshu

Are you basically asking me to verify if your below idea seems valid to me, or anything else?

Yeah, can you verify if you can have one HRA/webhook managing two separate RunnerSets? I don't have my RunnerSet operating at an enterprise level, I have them at the org level.
What I did was create separate namespaces & controllers for each organization and installed a RunnerSet/HRA in their respective namespaces.
I wasn't sure if an HRA can support two RunnerSets in the same namespace.
As far as the fix in #927
I ran a few jobs and checked my logs and I can still see that in_progess workflows are throwing Scale target not found
Scale target not found. If this is unexpected, ensure that there is exactly one repository-wide or organizational runner deployment that matches this webhook event	{"event": "workflow_job", "hookID": "x", "delivery": "x", "workflowJob.status": "in_progress", "workflowJob.labels": [], "repository.name": "xxx", "repository.owner.login": "XX-CORP-IT", "repository.owner.type": "Organization", "action": "started"}

Basically, instead of GHEC's webhook reaching in to our internal Rancher environment to send information; is the reverse possible, where Rancher reaches out to GHEC and checks for workflow_job/other webhook events?

@rxa313 I believe a short-term solution would be our github webhook delivery forwarder I've built in https://github.com/actions-runner-controller/actions-runner-controller/pull/682.
We don't yet have a container image build and a Helm chart support for that component so you'd need to built your own container image for the forwarder and a K8s deployment to deploy it onto your cluster.

@rxa313 Also, you'd better file a feature request to GitHub so that they can add something like "List Workflow Jobs" API whose response includes all the jobs that are queued but missing runners and waiting for new runners to be available. The response should also include runner labels for missing runners of the queued jobs.
If we had such API, it's very easy to enhance HRA to provide an easy-to-configure autoscale functionality like you have today with workflow_job webhook events, without webhook.

one HRA/webhook managing two separate RunnerSets

No, that's not possible! You need to configure one HRA per RunnerSet, because the controller maps one "queued" webhook event to an HRA, and the HRA to a RunnerSet or a RunnerDeployment.

@mumoshu

We don't yet have a container image build and a Helm chart support for that component so you'd need to built your own container image for the forwarder and a K8s deployment to deploy it onto your cluster.

Is there plans to release this feature in near future? I'd love to test it out without having to create my own images in my cluster :)
Thanks for all the info.


Codes of Starters 54

Please assign this issue to me, Swoyam.

Please assign this issue to me, Swoyam.

Doing so many multiple contributions to the same repository will cause banning your contribution. It can also cause excluding this repo from the fest, this is why I can't assign this

Please assign this issue to me @swoyam2609

Cool Assigning it you @abhinaba-dash
Please make sure to resolve both issues #26 and this and open a pull request as creating two individual requests will can cause spam alerts

okay


Name "Workflows" ?

Hey,
I currently have two pages of agents, so I never use the '/agents' view anymore, but rather use the agent diagram. In that diagram, it seems to me like we could think of a nice name for a group of connected agents. Maybe a group could be called workflow? And maybe we could make it so that they can be named?
This'll help keeping an overview, and once we do a mobile app, it might help, too (as a different way to get to your agents).

I'm going to close this issue.  Please reopen if you want to discuss further or try building this!

I'm going to close this issue.  Please reopen if you want to discuss further or try building this!


Optimization: track actual file dependencies

In GitLab by @sstriker on Jul 31, 2017, 08:58
Pre-mature optimization; recording for later consideration.
If we track which files are actually being accessed from the staged build dependencies, then we can make a more detailed decision whether we need to rebuild when a dependency changed.  e.g. when project a.bst depends on project b.bst, but only on file /usr/include/b1.h, then when other files change in artifact(s) of b.bst should not trigger a rebuild.
This data would obviously only be available once a.bst has been built once.
Potentially implementable by a fuse layer that either keeps track of first use, or that implements staging on-demand.  The latter likely complicates the overall architecture.  Avoiding rebuilds at the expense of regular build performance will need to be kept in mind.

In GitLab by @sstriker on Nov 9, 2017, 13:11
Turns out the staging on-demand idea is more commonplace.  Leaving some links for future reference:

https://blog.bazel.build/2017/08/25/introducing-sandboxfs.html
https://github.com/bazelbuild/sandboxfs/

In GitLab by @tristanvb on Dec 25, 2017, 04:44

This data would obviously only be available once a.bst has been built once.

A tradeoff this may imply which is worth considering; if I need to compare files to find out if an alternative artifact can be used instead of building, then it means I need to have those files, which may equate to downloading artifacts in order to know if they can be used (bandwidth vs cpu). If this is the case, it's already probably not worth the complexity it adds.
This also opens up another notable can of worms: "A dependency can be assumed the same if it provides the same data for the files I need to build against it"; however, it can only be assumed the same "for this element" which accessed it the last time. This means we can have disagreements between elements on whether or not a given artifact can be used or needs to be rebuilt.

In GitLab by @sstriker on Mar 26, 2018, 11:10
I think that the impact of build avoidance should not be underestimated.  The bandwidth vs cpu tradeoff may remain well worth it.  Regardless, I think this can also be done with less of a bandwidth impact.

Under the non-strict cache key we store the paths contained in dependencies that were accessed during the build
Under a new cache key, based the element itself as well as the content of the actually depended upon files.
On a build: we first check the strict cache key as per usual.  If no artifact exists, we now check the non-strict cache key, retrieving the paths actually used, calculating the content cache key and checking for existence of an artifact there.  If so, it can be stored under the strict cache key and be used.

Note: above assumes that we can already track what files were actually used during the build.

In GitLab by @tristanvb on Mar 27, 2018, 05:54
In the abstract I think that this means:

BuildStream should ideally have it's own internal understanding of merkle trees for any of this to be efficient, which is aligned with other planned activities
As you say, we have to be able to track what files were accessed.

We have pretty much identified that fuse cannot be the vessel this type of activity, the overhead is too high with fuse.
But it is also aligned with our needs for better virtualization of filesystem access, in order to store and retrieve information such as arbitrary uid/gid ownership, and extended attributes.

In GitLab by @juergbi on Mar 27, 2018, 08:42

We have pretty much identified that fuse cannot be the vessel this type of activity, the overhead is too high with fuse.

Results of simple benchmark of FUSE overhead: https://gitlab.com/BuildStream/buildstream/issues/38#note_65260808

In GitLab by @sstriker on Apr 5, 2018, 22:13
Potential debunk of overhead of FUSE being too high: https://gitlab.com/BuildStream/buildstream/issues/38#note_66632621 courtesy of @juergbi

In GitLab by @richardmaw-codethink on Jun 19, 2018, 16:41
To be sure we're on the same page, I went through what this would solve with the rest of my team.
This is the worked example we came up with to explain how we thought it could work.
Does this make sense to you and would it solve your issue?

We have two elements, libc.bst and sh.bst. sh.bst depends on libc.bst.

libc.bst has no dependencies. When we build it we get an artifact with weak and strong keys being equivalent because they have no dependencies.

When we build sh.bst we track which files it accesses to build a manifest along with the artifact.

The sh.bst artifact is cached under its weak key without taking into account what it depends on.

The manifest is cached using the strong key of the artifact that was depended on (libc.bst) and the weak key of sh.bst.

After libc.bst is updated it requires a new artifact to be built. Since the new version has a new Source both cache keys are updated.

Since the only thing that's changed about sh.bst is that a dependency has changed, its weak cache key is identical.

We want to avoid rebuilding sh.bst if we don't need to, so we look in our cache for weakly cached version of sh.bst.

We can see from its metadata that it depended on the old version of libc.bst, and can using the weak cache key of sh.bst and the cache key of libc.bst, find the manifest of building sh.bst against the previous version of libc.bst.

Using the list of files used in the manifest we can compare the old version of the libc.bst artifact and the new version of the libc.bst artifact.

If all the files in the manifest are the same in both of libc.bst artifacts  then the new sh.bst artifact should be identical, and instead of rebuilding against the new libc.bst, we can add a reference for the new strong key of sh.bst to the old sh.bst artifact, saving the need to build sh.bst.

In GitLab by @sstriker on Jun 19, 2018, 18:35
@richardmaw-codethink I think that that matches the intent to avoid builds of reverse dependencies, if the relevant part of the dependency hasn't changed.
I think that these final two steps

Using the list of files used in the manifest we can compare the old version of the libc.bst artifact and the new version of the libc.bst artifact.

If all the files in the manifest are the same in both of libc.bst artifacts  then the new sh.bst artifact should be identical, and instead of rebuilding against the new libc.bst, we can add a reference for the new strong key of sh.bst to the old sh.bst artifact, saving the need to build sh.bst.

might be optimized by what was mentioned in a previous comment:

Under a new cache key, based the element itself as well as the content of the actually depended upon files.
On a build: we first check the strict cache key as per usual.  If no artifact exists, we now check the non-strict cache key, retrieving the paths actually used, calculating the content cache key and checking for existence of an artifact there.  If so, it can be stored under the strict cache key and be used.

We should look at what turns out more efficient in practice.  Fetching the [merkle tree of] the artifact of the previous version of libc.bst (which may already be local) and comparing against the current version of the libc.bst artifact vs calculating a new content based key and doing a lookup based on that.

In GitLab by @LaurenceUrhegyi on Oct 2, 2018, 14:18
some additional useful info:
https://docs.google.com/document/d/12c3oAPgedckLpue2yj0xGgJTEOyUm4mXWWBJ157J-8I/edit#
https://docs.google.com/document/d/1SQ44FtvO5AAzi-EWXpX_KyFpgYX8WgcNs1G6RNW0m5A/edit#heading=h.3wq1n0gftbco

In GitLab by @danielsilverstone-ct on Oct 23, 2018, 13:25
One thing which concerns me with this is determining the correctness of cacheability.  For example, if during the process of running a build step, a directory is opened and read, then in theory we cannot be certain whether or not the content or ordering of the directory itself formed part of the output artifact.  This doesn't sound too common until you appreciate that build systems which discover source code will perform this kind of scan quite often which could result in not being able to cache steps generically which might otherwise be cacheable.
If we ignore directories read, then such discovery won't work.  If we accept that a directory being read means that any change in the directory means we have to rerun the build, we run the risk of under-caching in a lot of cases.
We also need to key based on negative filesystem lookups.  E.g. consider a step which does: test -r somefile && touch someoutputfile in a case where somefile does not exist, we need to ensure we include that negative lookup in the decision for whether two trees are the same for input purposes.
I'm sure that others have considered this before, but I couldn't satisfy myself that it was in any of the above discussion.


allow ports to be specified in vhosts

right now, it doesn't seem like vhosts can have ports included at the end. on top of that, if i wanted to use port :443 or :80, browsers will remove that port number. That situation should work as well

The vhost property in a target map within targetArray?

@dovholuknf I assume you're referring to the vhost property in one of the targets defined in the ZITI_BROWZER_BOOTSTRAPPER_TARGETS env var.
Are you considering a scenario where a request HOST header contains a port different from the Bootstrapper's load balancer port?