[Feature] support map[key] operator

support map[key]
What type of PR is this：

[ ] BugFix
[x] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #10677
Problem Summary(Required) ：

map [] operator is used to access the value of the specified key, used just like map[key], the "map" is the column whose type is MapType and the "key" is the same type as MapType->keyType.
The design of map []operator is very similar to arraysubscriptExpr, so I rename arraysubscript to collectionsubscript which can be used by both array and map.
There are serveral main differences:

In the analyse phase, the arraysubscript will check the subscriptor if it's an int or can cast to int, the mapsubscript will check the subscriptor if it's an keyType or can cast to keyType. In the phase the return type is set as the MapType->valueType.
To transfer to be, the CollectionSubscriptExpr will check the child's type and than use ARRAY_ELEMENT_EXPR for array or MAP_SUBSCRIPT_EXPR for map.
In be, the ARRAY_ELEMENT_EXPR and MAP_SUBSCRIPT_EXPR are dealed separately.
In be, when the MAP_SUBSCRIPT_EXPR evaluates, check the keys one by one and get the selected index, use the key matched index generate the final result.

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

[FE PR Coverage Check]
:disappointed: fail : 21 / 34 (61.76%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/analysis/CollectionSubscriptExpr.java
3
7
42.86%
[59, 60, 62, 68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
7
16
43.75%
[263, 270, 271, 275, 276, 277, 279, 280, 281]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionSubscriptOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/AST2SQL.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

run starrocks_fe_unittest

run starrocks_be_unittest

run starrocks_fe_unittest

[FE PR Coverage Check]
:disappointed: fail : 27 / 34 (79.41%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/analysis/CollectionSubscriptExpr.java
3
7
42.86%
[59, 60, 62, 68]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
2
50.00%
[3935]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[270, 271]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionSubscriptOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/analyzer/AST2SQL.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:disappointed: fail : 29 / 37 (78.38%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/analysis/CollectionSubscriptExpr.java
3
7
42.86%
[59, 60, 62, 68]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[270, 271]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionSubscriptOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionSubscriptExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[270, 271]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionSubscriptOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionSubscriptExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionSubscriptOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionElementExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionElementOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionElementExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionElementOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionElementExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionElementOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionElementExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionElementOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

starrocks_be_unittest succeeded.

starrocks_be_unittest succeeded.

run starrocks_fe_unittest

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

[FE PR Coverage Check]
:heart_eyes: pass : 32 / 37 (86.49%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/sql/Explain.java
0
1
00.00%
[684]

:large_blue_circle:
com/starrocks/sql/optimizer/rewrite/BaseScalarOperatorShuttle.java
2
3
66.67%
[64]

:large_blue_circle:
com/starrocks/analysis/CollectionElementExpr.java
6
7
85.71%
[68]

:large_blue_circle:
com/starrocks/sql/analyzer/ExpressionAnalyzer.java
14
16
87.50%
[269, 270]

:large_blue_circle:
com/starrocks/sql/parser/AstBuilder.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/ScalarOperatorVisitor.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/scalar/CollectionElementOperator.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/plan/ScalarOperatorToExpr.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/OperatorType.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/SqlToScalarOperatorTranslator.java
2
2
100.00%
[]

run starrocks_be_unittest

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test


BGFX Conan package

Please make a Conan.io package for BGFX:
http://docs.conan.io/en/latest/

They don't support GENie: http://docs.conan.io/en/latest/integrations.html
Also you can make it, and maintain it. I'll add it to documentation.

Ok, I'll attempt to resolve their lack of GENie support. Thanks!

It's probably almost 100% identical to this: https://github.com/memsharded/conan-premake

I'm not sure, would it be appropriate to create new issue, but I just want to let others know, who were searching for conan package, that @firefalcom packaged it.
https://github.com/firefalcom/bgfx-conan

@markusgod :+1: I added it to docs too: https://bkaradzic.github.io/bgfx/build.html#alternative-build-systems


Layer B Encoder press # are incorrect

On layer B
encoder 1 press = ENC 10
encoder 2 press = ENC 11
etc
encoder 7 press = ENC 16
encoder 8 press does not have a number
Correction:
encoder 1 press should = 9
Love this program so far, works a treat (except above...)
Thanks for your efforts.

Sounds like a typical off by one error. Will look into this, this weekend

https://github.com/maartentamboer/X-Touch-Mini-FS2020/commit/14529f55323b1760b1393f39cd3bd751aa79c314

Fixed in https://github.com/maartentamboer/X-Touch-Mini-FS2020/releases/tag/v1.2.0

All good now. Thanks Maarten


How to set map zoom level ?

You can use third-party solution for change zoom level on your mkmapview such as https://github.com/johndpope/MKMapViewZoom. Or you can to try change mapRegionCoef parameter for influence to map zoom.

thank you

you are welcome!


BitmapData.threshold() issues

Short desc: it doesn't work consistently across targets (tested with openfl 2.2.6 + lime 2.1.1 on win7).
Now for the long one:
I've written some visual tests' code to see in which cases it wouldn't work (see https://gist.github.com/azrafe7/b1acee98175c98504ce1).
The results of the tests can be summed up by looking at these screenshots (green text -> test passed, red text -> test failed):

flash (i.e. what to be expected - all green!)
js
neko
cpp

The good news is that js seems to be easily fixed (passing all tests) by just commenting out these two lines calling __flipPixel(): https://github.com/openfl/openfl/blob/950f9e151e15d0e68f4e88fda15854e797137004/openfl/display/BitmapData.hx#L1280-L1281.
For cpp/neko I haven't been able to find a fix, but it's interesting to note that setPixel32() doesn't seem to work right on neko when involving alpha, while it does on cpp! (confront the bottommost test in the screenshots for neko and cpp).
Somehow related to this:

BitmapData.clone() has some issues in js, f.e. it won't work with the tests mentioned above (see https://gist.github.com/azrafe7/b1acee98175c98504ce1#file-thresholdtests-hx-L125)
setting the font to _typewriter fails on js, but works on other targets

I think this may be resolved now?

Don't know (yet), but I'll probably have time tomorrow to retest this against latest git. ;)

I've just tested with the gist I posted above (against both haxelib and git versions of the libs), and unfortunately the issues are still there. :c
Not sure it's only on my end but it could be (must admit I've also other issues, especially with native crashing on start unless using legacy - even hybrid fails me now, but worked some weeks ago).
I'm really open to suggestions on how to fix these ones, so if you're around (tomorrow night) on irc/gitter/something, we can try to go through this together and maybe find a solution.
By the way here's the steps I've gone through (in case you can spot something wrong):

I've haxelib git both openfl and lime (being sure they're set as git reviewing haxelib list output)
recompiled lime with openfl rebuild windows -Dlegacy (as the ndll wasn't there)
tested the gist on various targets

BTW I'm on Win7 64bit

I just checked, it looks the same now :smile:

Well... now for me js works correctly (and even the font issue seems resolved), but the output for cpp and neko is the same wrong one of the first post.
I can only run the gist in -Dv2 mode, as the app crashes on start if I omit it (and that's probably because I have an old Intel GPU - still wonder why noone has shown up with the same problem on the forum).
I'd be glad if you can run it (here's the little test project) and see if the output is the same as the flash one in both legacy and next.

The output is the same in native legacy because the fixes are for OpenFL 3 and Lime, not legacy. Does your project work if you use "--window-hardware=false"?

Yes, it works. :open_mouth: That's great! :smiley:
Never heard of that flag, what does it do? Falls back to software rendering I could guess?!
Anyway, can the fix be backported to legacy?

Yep, that flag forces software rendering. Are you running a system that has trouble with OpenGL 2 features? We need to add something to detect this case and fallback to software automatically, when GL is supported but not the version we're looking for

Well... GLView says the driver supports OpenGL 2.1 at 100%:

but I recall having the same crash-at-start problem with luxe (haven't retried in a while though).


refactor/move-kernelfunc

Extract kernelFunc to modules/poetry.nix.

Notice: https://github.com/tweag/jupyenv/pull/450#issuecomment-1455026847, which means I can not import such as https://github.com/tweag/jupyenv/blob/main/lib/overrides.nix from the evalModule.(I don't why).
my solution: https://github.com/tweag/jupyenv/pull/450/files#diff-8bf585666371b85c4eb513ad2163c69e20eaf72b2faa18abc1a0884c9827d70fR136
Also, I think it is necessary to provide an additional option for poetryEnv, not only the problem of overalys, the most important thing is that pythonEnv can be passed from the outside. (we can simply add a group deps of jupyenv for other poetyEnv ,  then add it)


BottomBar: Update menu button with new icon, replace role card icon with old menu icon.

Per aidan's suggestion, i changed the bottom bar icon. It should be in zep as a vector just like the other one. Note that the original menu icon (the at stake logo) is now the icon for the role card. It makes sense to use this instead of the blank head.

@jpailthorpe the 3-bar style menu bar is not downloadable in zeplin


Suggestion: date widgets can differ per date. Following logic change can help it.

  final Map<DateTime,Widget> markedDates;

  Widget _renderMarked(DateTime now) {
    if (widget.markedDates != null && widget.markedDates.length > 0) {
      for (DateTime key in widget.markedDates.keys) {
        if (key.year == now.year && key.month == now.month && key.day == now.day) {
          return widget.markedDates[key] == null
              ? widget.markedDateWidget
              : widget.markedDates[key];
        }
      }
    }
    return Container();
  }
}

See different date widgets for date 3 and 4.
Btw, awesome work guys!

I've updated to 1.1.8 to support this feature check this out.


team_clan_name

i am getting issues with team_clan_name for demos after the new update

Do you have a demo I could check out?

yeah but i cant post it here anywhere i can send it to you

https://drive.google.com/file/d/1tk0DIG8h1149J6PE1QZWUGP5dQOg9Qg0/view?usp=drive_link

problem is that i need the clan names and none of the teams seem to have any

With new demos it seems to work, closing for now.


[Feature Request] Allow for opening ranger in a floating window instead of default fullscreen

Hello there. Hope you're doing well. Thank you for creating ranger-nvim; it's the best ranger plugin for me.
As titled, I'd like the feature to open ranger in a floating window instead of default fullscreen, like below:

The above is achieved by using another plugin that supports ranger called fm-nvim. The following config is inspired by fm-nvim and maybe it can serve as a starting point?
require("ranger-nvim").setup({
  ui = {
    -- Default UI ("full" or "float")
    default = "float",

    float = {
      -- Floating window border (see ':h nvim_open_win')
      border = "none",

      -- Highlight group for floating window/border (see ':h winhl')
      float_hl = "Normal",
      border_hl = "FloatBorder",

      -- Floating Window Transparency (see ':h winblend')
      blend = 0,

      -- Num from 0 - 1 for measurements
      height = 0.8,
      width = 0.8,

      -- X and Y Axis of Window
      x = 0.5,
      y = 0.5,
    },
  },
})

Any input is much appreciated. Thank you!

Hey Kohane,
Thanks for the feature request and I am happy that the plug-in is serving your needs.
Sounds like a good idea, I am happy to add in the feature. I will make it an opt in feature with the current behaviour as the default behaviour.
I will add it on my todo list. I will close this issue when I have implemented this feature.
Thanks.

Hello @kelly-lin , hope you're doing well.
I just forked and did a quick proof-of-concept here: https://github.com/kohane27/ranger.nvim/commit/3a5d241ccf217fe9c166cba1139fcd95546e9281

It works flawlessly and is almost one-to-one copy from fm-nvim. Hope it serves as a starting point. Thank you again!

Thanks @kohane27! Nice work!

@kohane27 I have implemented the feature with a few changes from your proposal in the latest commit 03819b53.
Thank you for your suggestion and help!

I have just tested and it works perfectly! Thank you so much!


Custom world and/or tensor parallel

1. World

Streaming's model of the world/the run/the job is (num nodes, ranks per node) rank processes + (num nodes, ranks per node, workers per rank) worker processes.
For example, 8 ranks, 8 workers/rank -> 8 + 8 * 8 = 72 StreamingDataset replicas per node.
Currently they coordinate via filelocks and shared memory.
A World is just our wrapper around torch.dist and torch.utils.data.get_worker_info.
The World just tells you which one you are, out of how many (nodes, ranks/node, workers/rank).
There's always only one World, semantically speaking.
However, when in a rank, get_worker_info() will say we are worker 0 of 1. One has to keep that in mind.

2. StreamingDataset argument tensor_parallelism: Optional[int]

tensor_parallelism iterates the same samples for groups of adjacent GPUs (ranks).
It does this by scaling down (in number) and repeating ranks.
But the underlying data doesn't change, so we are correspondingly scaling up the ranks (in length).
It prefers to merge ranks within the same node first, then if it needs to merge further, across node boundaries.

3. StreamingDataset argument world: Optional[World]

If tensor_paralellism isn't flexible enough, we let you completely override World (instead of auto-detecting it).
This is probably going to be very useful for testing.

Now I get an error like:
  File "/streaming/streaming/base/dataset.py", line 1458, in __iter__
    sample_ids = self._get_work(world, epoch, sample_in_epoch)
  File "/streaming/streaming/base/dataset.py", line 1002, in _get_work
    shape_shm, data_shm = self._share_work(epoch_sample_ids)
  File "/streaming/streaming/base/dataset.py", line 949, in _share_work
    shape_shm = SharedMemory(name=name, create=True, size=size, auto_cleanup=False)
  File "/streaming/streaming/base/shared/memory.py", line 41, in __init__
    shm = BuiltinSharedMemory(name, create, size)
  File "lib/python3.9/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
FileExistsError: [Errno 17] File exists: '/000000_epoch_shape'

Now I get an error like:
  File "/streaming/streaming/base/dataset.py", line 1458, in __iter__
    sample_ids = self._get_work(world, epoch, sample_in_epoch)
  File "/streaming/streaming/base/dataset.py", line 1002, in _get_work
    shape_shm, data_shm = self._share_work(epoch_sample_ids)
  File "/streaming/streaming/base/dataset.py", line 949, in _share_work
    shape_shm = SharedMemory(name=name, create=True, size=size, auto_cleanup=False)
  File "/streaming/streaming/base/shared/memory.py", line 41, in __init__
    shm = BuiltinSharedMemory(name, create, size)
  File "lib/python3.9/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
FileExistsError: [Errno 17] File exists: '/000000_epoch_shape'

Investigating...

@andreamad8 how about now? You may need to call streaming.base.util.clean_stale_shared_memory()

Ok works now, If you give me 10 min I can test also with multiple nodes.

We will begrudge you one sidereal hour

was a bit longer than an hours :)
yes, it works I tested up to 16 nodes with different TensorParallel size 1, 4, 8.

Cleaned up the PR to have naming not TP-specific and fixed a bug that was preventing determinism (both elastic and non-elastic). Added tests as well. Pending regression tests, should be good to go!


Fixed crash when reading b/w tif images (fixes #5579)

Port of #5357 to master branch
FIxes issue #5579

:+1:


Question about type casting from c_char

Hi, Im trying to add set rich presence SteamAPI_ISteamFriends_SetRichPresence
C header from SDK virtual bool SetRichPresence( const char *pchKey, const char *pchValue ) = 0;
My implementation:
pub fn SteamAPI_ISteamFriends_SetRichPresence(instance: *mut ISteamFriends,  key: *const c_char, value: *const c_char) -> bool;
    /// Set rich presence of current game.
    pub fn set_rp(&self, key: &str, value: &str ) -> Result<(), ()> {
        let key_rp = CString::new(key).unwrap();
        let val_rp = CString::new(value).unwrap();

        let success = unsafe {
            sys::SteamAPI_ISteamFriends_SetRichPresence(self.friends, key_rp.as_ptr(), val_rp.as_ptr())

        };
        if success { Ok(()) } else { Err(()) }
    }

The question is, should key_rp.as_ptr() also cast using as *const _ ? I saw some of your implementation using as *const _. If you dont mind to explain what as *const _ do.
I tried using as *const _ and not using it, no compiler error.

If you dont mind to explain what as *const _ do.

as *const _ Just casts but lets the compiler figure out the type pointer.
As for why: its not needed in this case, just a bad habit on my part. Its going from os::raw::c_char to libc::c_char but they should always be the same type for the most part.


Enclose item documentation as Markdown code-block

I think this change may solve #260

Star (*) characters don't get interpreted as Markdown bold syntax and Python's built-in types get syntax color as if interpreted as code.

Sorry. It is not the correct solution.
I'm tackling it.

This should be fixed as part of #426


Making IsarLink final does not work; generator generates but code cannot compile due to illigal assignment

The docs say: "IsarLink and IsarLinks both have a zero-arg constructor which should be used to assign the link property when the object is created. It is good practice to make link properties final."
When I define a schema with anything like:
 final groups = IsarLinks<Group>();

the generator generates:
object.groups = IsarLinksImpl()
      ..attach(
        collection,
        collection.isar.groups as IsarCollectionImpl<Group>,
        object,
        0,
        false,
      );

This does not compile, as it attempts to set a final field.

This is fixed 👍


Avoid array index out of bounds on missing result size in tag

Select commands on GreenplumDB (possibly older versions of Postgresql as well) do not return the amount of result records in the tag.
Added a lenght check on the tokenized tag to prevent array index out of bounds.

@Monezz Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@Monezz Thank you for signing the Contributor License Agreement!

Thanks a lot. That's merged now.


[QUESTION] How to handle Download cancellations with Service Worker

Hi,
I use client-zip to download multiple large files over a service worker like in the worker.js example.
I just wondered if it is possible to propagate a cancel download event of the archive file to the fetch requests in the service worker so they don't continue running after the archive got cancelled.
I just found this issue: https://github.com/w3c/ServiceWorker/issues/1544 and i'm wondering if it is possible at all in todays browsers regarding your latest Output stream cancellation commit https://github.com/Touffy/client-zip/issues/58.

When the Service Worker's client cancels the download, that should cancel the Zip stream from the Worker's point of view.
As you noticed, the output stream cancellation should now throw an error up the chain of async generators that makes up downloadZip (I haven't actually tested it yet).
So at the source, if you passed an (async) iterable to downloadZip, you can catch that error in there and trigger the AbortSignal for any ongoing fetch. Here is a simplified example :
// assuming that `urls` is an array of URLs for the files
async function* source() {
  const downloadController = new AbortController()
  const { signal } = downloadController

  // of course you should not generally start all the requests at once like this
  // but it's easier to show why the signal is necessary
  const promises = urls.map(url => fetch(url, { signal }))
  try {
    for (const promise of promises) yield await promise
  } catch(error) {
    // this will immediately cancel ALL the requests, not just the request
    // whose body is currently consumed by downloadZip (if any)
    downloadController.abort(error)
  }
}
event.respondWith(downloadZip(source()))

As stated in the comment, you should probably not fetch every file immediately, but you can use a stream instead and still use the same signal for all the files, and trigger the signal when you catch an error just like in the example.


Update CONTRIBUTING.md

Added YT Video -> https://www.youtube.com/watch?v=_Nj9EYYuSqY
Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the maintainers.
What Type of PR is This?

[ ] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Preview environment: https://ci.pr.harness.io/pr-597


Use last entry when parsing DHCP status file

When a machine is stopped and restarted, it may get allocated a new IP.
The loop that parsing the status file returns on the first match,
which is the entry for the old IP so that machine fails to start up.

Other than that, I like this change, thanks!
This PR fixes a bug I have been seeing, as mentioned in https://github.com/kubernetes/minikube/issues/1107#issuecomment-278623525.

Fixed, PTAL

@r2d4 👍

Is this in the latest release then?


refactor(streaming): use bitmap builder for dispatcher visibility

Signed-off-by: Bugen Zhao<EMAIL_ADDRESS>I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
As title. 💦
Checklist

[x] I have written necessary docs and comments
[x] I have added necessary unit tests and integration tests
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

#3544

Excuse me, what are the functions of 'DispatchExecutor'?

Please resolve conflicts

Excuse me, what are the functions of 'DispatchExecutor'?

It's used for message passing between different streaming actors. BTW, you may join our Slack community at README.md if you are interested in RisingWave internals. 😄

Thanks for help, I have joined the Slack community.😊


Release new version of codespan-lsp

Since codespan-lsp 0.4.1 still requires lsp-types 0.60.0, language servers relying on this library are currently unusable on JetBrains IDEs such as IntelliJ or WebStorm because of an issue with the document version schema, currently fixed in lsp-types 0.61.0 (https://github.com/gluon-lang/lsp-types/pull/113). It appears that the lsp-types version requirement for codespan-lsp has already been bumped and merged into master in this commit, but it isn't yet available on Crates.io.

Ah! Thanks for letting me know. Wish we had a better versioning strategy for lsp-types that meant we didn't have to bump the versions as much 🤔

Adding a field is a breaking change in rust but it isn't in JSON so hard to avoid doing breaking releases :/

Looks like #[non_exhaustive] is in FCP https://github.com/rust-lang/rust/issues/44109 ! Perhaps best to wait for that, otherwise it could be faked reasonably well with a
#[doc(hidden)]
pub non_exhaustive: (),

field. There is an ergonomic hit when constructing the lsp-types however as .. Default::default() will be needed in structs.

Yeah I was wondering if something like that could be done. It is a bummer to have the ergonomic fit, but it kind of reflects the fact that the LSP is inherently designed to be forwards compatible to some extent. Could also mean that lsp-types could make it to 1.0?

I think it would let a 1.0 to be released, but maybe there are some edge case that isn't handled with this I'd expect that to be rare though, if it happens.
Looks like #[non_exhaustive] doesn't let functional record updates work though, so using the current workaround would be better in this case.
Another drawback is that matching on an enum would need a catch all case, even if no additional variants are ever added.
Should float the idea on the lsp-types repo at least to see what other users think.

The crate should be available online now!

@brendanzab @Marwes Many thanks to you both!


Improvement of Homepage Responsiveness

I want to work on this issue under JWOC

actually someone is already working in this issue so wait till tomorrow.... I will assign it to if they can do it.

okay

I want to work for on this issue under jwoc , please assign me

@TuhinBanerjee31 any update?

Will raise a PR by tonight.

ok


Rename 'docs' to 'doc'

NCO prefers doc to docs as the name of the documentation directory.

@gspetro-NOAA, we'll need to update the User's guide of the land-DA workflow before the new delivery. Could you change the docs directory to doc as you did for SRW when you update the document?

@chan-hoo Yes! That is one of the changes I've made in the PR I'm working on (not opened yet, but hopefully soon).


nixos/networking: Add libqmi to ModemManager runtime path if fcc unlock is enabled

Description of changes
Some of the modem FCC unlock scrips from ModemManager depend on CLI utilities from the libqmi package. This PR adds the libqmi package to the ModemManager systemd service descriptor PATH parameter so the modem unlock will work.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@ajvpot Did you intend to leave this marked as a draft? Are you still interested in seeing it merged?
If not, marking it ready for review will help signal that it's ripe. If so, can you comment on whether it's blocked by anything or if you need help? (And anything else you can think of that might help future triagers decide whether to leave this open or close it?)

Tentatively closing. Happy to reopen if you're still interested.


Bugfix for outline parameter in render_shapes

Codecov Report

Merging #119 (2f04d04) into main (bde6827) will increase coverage by 0.32%.
The diff coverage is 100.00%.

:exclamation: Current head 2f04d04 differs from pull request most recent head f130032. Consider uploading reports for the commit f130032 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #119      +/-   ##
==========================================
+ Coverage   75.84%   76.17%   +0.32%
==========================================
  Files           9        9
  Lines        1151     1150       -1
==========================================
+ Hits          873      876       +3
+ Misses        278      274       -4

Impacted Files
Coverage Δ

src/spatialdata_plot/pl/render.py
88.50% <100.00%> (+0.93%)
:arrow_up:

... and 1 file with indirect coverage changes

Nice! After addressing the comment on the argument type for me can be merged!


cannot start Apache with PHP lower than 7.0 after installed PHP 7.1.0

I was installed Apache 2, PHP 5.4, 5.5, 5.6, 7.0 with phpbrew.
I was be able to switch version of PHP on Apache using this comment and it works perfectly.
After I installed PHP 7.1.0, and switch to Apache -> PHP 7.1.0 everything works fine.
Switch to PHP 7.0.0 still works fine.
Switch to PHP 5.6.29 or lower and Apache cannot start and here is the log found in /var/log/apache2/error.log

[Sun Jan 15 16:03:26.422951 2017] [mpm_prefork:notice] [pid 7642] AH00171: Graceful restart requested, doing restart
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using <IP_ADDRESS>. Set the 'ServerName' directive globally to suppress this message
[Sun Jan 15 16:03:26.510214 2017] [core:notice] [pid 7642] AH00060: seg fault or similar nasty error detected in the parent process

Uninstall or remove or purge cannot make it back to work.

Did you look at the conf-enabled directory? Maybe theres 2 php configurations enabled.

Nope, only one.

could you upload your apache config file?

Sorry, I was removed the whole OS. (I use virtual box on Windows host and Ubuntu as guest pc.)

It's fine. I'm closing this issue and if your run into it again just reopen this one or create a new issue.
btw, php 5.x and php 7.x use different module name in apache config file, like
LoadModule php7_module xxxxxxxxx and
LoadModule php5_module xxxxxxxxx
it may be the reason why your apache fails to work.


DefaultPromise make listeners not volatile

Motivation:
DefaultPromise has a listeners member variable which is volatile to allow for an optimization which makes notification of listeners less expensive when there are no listeners to notify. However this change makes all other operations involving the listeners member variable more costly. This optimization which requires listeners to be volatile can be removed to avoid volatile writes/reads for every access on the listeners member variable.
Modifications:

DefaultPromise listeners is made non-volatile and the null check optimization is removed

Result:
DefaultPromise.listeners is no longer volatile.

This is the counterpart for https://github.com/netty/netty/pull/5503 for the 4.0 branch. The second commit is kept separate for now ... this commit isn't strictly related to this PR but is needed in order to backport the DefaultPromise fix.

@normanmaurer - PTAL

@Scottmitch LGTM

@normanmaurer - I'll pull in after the build completes just to be double sure all is well :)

@Scottmitch complete =P

4.1 (4baff69)


Need a Release Notes Section in the OpenFAST Documentation

The OpenFAST documentation has a great section describing the conversion from FAST v8.16 to OpenFAST v0.1 (http://openfast.readthedocs.io/en/latest/source/user/fast_to_openfast.html).  However, the changes between OpenFAST v0.1 and v1.0 are only briefly summarized at a high level in the release comments in the OpenFAST v1.0.0 tag (https://github.com/OpenFAST/openfast/releases).  A release notes section should be added to the OpenFAST documentation summarizing the key details of what is new in each release to aid the general user.  What's important to include in the release notes is a summary of the new features and changes to existing features that will effect the solution (e.g. bug fixes), including changes to the input files that the general user should understand when upgrading.  While anyone can review the commit logs to find all of the details of changes, there is generally far too much detail in the commit logs for the general user to comprehend.  The release notes should be the user-oriented summary of these commit logs.
I'm preparing to change the nwtc.nrel.gov website to link to OpenFAST and to publicly announce the release of OpenFAST, but a release notes section in the OpenFAST documentatoin for the upgrade from OpenFAST v0.1 to OpenFAST v1.0 should be added first.  This release notes section can then be updated as additional OpenFAST tags are added in the future.

@jjonkman , thanks for the comment.  However, at this time we are only planning to describe tagged releases at a high level on github (as you described).    However, if someone in the community would like to write up a document in the OpenFAST documentation framework describing releases in more detail (and submit a pull request), we would welcome such a contribution.

I talked to @rafaelmudafort and @ghaymanNREL and we all agree of the need for a release notes section.   I'd be happy to draft this up for the OpenFAST v1.0.0 tag, but I can't check it properly due to issue #63.  In the meantime, I could send the text to @rafaelmudafort for him to commit.


Improve transaction burn rate

With help from Niksa we discovered that currently there are several hardcoded valules around transaction handling in metric ingestion code.
Alongside with this, we received a number of customer reports of high transaction ID consumption rate. A review of codepaths involved hinted that our approach might be faulty and we might want to reconsider the way transactions are handled. At the very least it'd make sense to make batch timeout and batch size configurable, as now they're hardcoded to 20ms and 1000 respectively.

https://github.com/timescale/promscale/pull/1462/files May have already partially addressed this since it gets rid of a server-side try-catch block which burns a txn id. This should be measured.


Kanka Interface loadEntity Error

When I try to open the interface for kanka-foundry (clicking the kanka icon) the error "An error occurred while loading the list of entities from Kanka."
I can update entities that are already in my game, so I know the access token I have works. I did recently superboost my campaign though, so maybe that is an issue? I also noticed that kanka is using captchas more aggressively.
I looked for this in your code and it seems linked to 'loadEntity' which I think points to this being on kankas end. Let me know if you have any ideas about what the issue is, thanks!

Ok, with a hint from Jay from kanka, this issue is actually not related to the DDoS protection at all right now, but to the recent Kanka update. There is a bug in Kanka in regard to how paging works for long lists of entities (the links to the next page and others are broken). Jay is on it and it should be fixed soon I hope.


Update ChefDK to 0.12.0

Update ChefDK to the latest version.

Done in https://github.com/caskroom/homebrew-cask/pull/20025. But still thank you for your contribution.


Updating install worker script to accept a user defined directory whe…

…re the aws install can take place.
Issue #, if available:
#1164
Description of changes:
I've updated the install-worker script to enable users to specify where the aws install takes place. The default behavior has been left unaltered. This allows us to install aws v2 without using a tmp directory. Our base AMI is CIS hardened and doesn't allow executables with in tmpfs.

Testing Done
I've verified that the AMI still builds and I've tested it in our dev environment.

See this guide for recommended testing for PRs. Some tests may not apply. Completing tests and providing additional validation steps are not required, but it is recommended and may reduce review time and time to merge.

I don't think we should make this configurable via an environment variable, or at least not just for the AWS CLI.
The issue is that /tmp may not--in general--be the right working directory for AMI build, right? So, we could create a proper template variable for working_directory (that defaults to /tmp), for example, and then use that throughout the build process.
Otherwise, I'd prefer to just choose a working directory for the AWS CLI installation that's suitable for more use cases, something like the home directory of the packer user.

I can parse it as a variable if you want

I believe I've actioned all your comments. I've tested in my environment and running the unit tests looks to have worked.

I've fixed the linting error.

I don't think it makes sense to have a template variable specifically for the working directory of the CLI installation.
If you want to refactor this into a more general variable for the working directory of the wider build process, that's reasonable.

Hi, I've refactored it to use a more general template variable. I've not using it anywhere else as there isn't a needed, but it can be used in future if required.

I've continued this effort in #1231, using the working_dir in the template spec as well. Give it a look if you can!

I've continued this effort in #1231, using the working_dir in the template spec as well. Give it a look if you can!

Hi, had a look and left a commment. LGTM.


Clean build fails if nuget package restore isn't completed first.

MSBuild failed on clean build if nuget packages weren't restored first.

Hi @salmonz, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

When will the pull requests merge with the main repo?

@salmonz thaks


[Feature request] Custom fields

Add 'custom fields'

Hi @shodgson, Thank you for building and maintaining these n8n nodes. Great work! I was just about to ask about the custom fields. What’s the best way to go about adding custom fields to the Person Node?

Hey, always happy for contributions. I am keeping an eye on the Twenty API development - it's not stable enough right now for me to dedicate a lot of time, but I am trying to keep it active and will review any contributions.


feat(youtube)!: Use oauth-based api library for better stability

Checklist before requesting a review

[x] I have read the contributing guidelines.

Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Describe your changes
Migrate from youtube-music-ts-api, which only supports cookie-based auth -- to Youtube.js, which uses oauth auth.
Usage
Start multi-scrobbler with any existing YTM configuration. A log statement will prompt the user to visit a URL with a code (or it can be accessed from the dashboard)
[2024-10-09 15:24:17.358 -0400] INFO   : [App] [Sources] [Ytmusic - MyYTM] ERROR: Sign in with the code 'CLV-KFA-BVKY' using the authentication link on the dashboard or https://www.google.com/device

Visit the URL and enter the code, then finish the auth flow. Afterwards, refresh the dashboard and click Start on the Idle YTM source to start polling.
Issue number and link, if applicable

Fixes #195 missing (404) history endpoint which only appear when using cookies
Might fix #158 due to using a different authentication method

I think something in 671056d broke logging a lot :D  it seems that for everything it does, it creates a log file. My logs directory immediately fills up with thousands of mostly empty log files.

Ah, I didn't want to raise a separate issue since I had the PR image running and thought it came from that.
But yay, DST magic at it again :D

@fluffyclaws @xathon other than the DST issue how is this PR working for you? Have you experienced any of the issues reported in #195 (duplicate scrobbles) after using it? I need more feedback from users before I can merge this.

I think there was something. I'll check on it in the next couple of days and report back.

The latest pr-203 docker image has the DST fix incorporated.

Can confirm that there are no duplicate scrobbles the past few days. Only thing that I see is that the timestamps don't appear in the web log, the lines look like this:
Ad Infinitum - Aftermath --- Abyss @ N/A (Today)

Thanks for the feedback. The missing timestamps are intentional since MS is showing you what info it received from YTM (no timestamps).

Hey, sorry, I have to retract my confirmation. I checked the wrong place, since in the web log of multiscrobbler they show up as single, but in Maloja they're duplicated. Excerpt from the log when this happens:
[2024-11-02 17:15:25.296 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played

[2024-11-02 17:15:17.460 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Raw Payload:

[2024-11-02 17:15:17.459 +0100] INFO : [App] [Scrobblers] [Maloja - unnamed-mlj] Scrobbled (New) => (YTMusic) 妖精帝國 - IRON ROSE @ 2024-11-02T17:15:01+01:00 (S)

[2024-11-02 17:15:16.283 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-11-02T17:15:01+01:00 | Next check interval: 10.00s

[2024-11-02 17:15:16.282 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Discovered => 妖精帝國 - IRON ROSE @ 2024-11-02T17:15:01+01:00 (S)

[2024-11-02 17:15:15.417 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Raw Payload:

[2024-11-02 17:15:15.416 +0100] INFO : [App] [Scrobblers] [Maloja - unnamed-mlj] Scrobbled (New) => (YTMusic) 妖精帝國 - Paradiso≒Inferno @ 2024-11-02T17:15:01+01:00 (S)

[2024-11-02 17:15:15.392 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Found 40 recent scrobbles

[2024-11-02 17:15:15.380 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] Refreshing recent scrobbles

[2024-11-02 17:15:15.378 +0100] DEBUG : [App] [Scrobblers] [Maloja - unnamed-mlj] [Upstream Refresh] Should refresh => newest queued scrobble playDate is newer than last refresh

[2024-11-02 17:15:14.850 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Refreshing recently played

[2024-11-02 17:15:14.849 +0100] DEBUG : [App] [Sources] [Ytmusic - YT Music] Last activity was at 2024-11-02T17:15:01+01:00 | Next check interval: 10.00s

[2024-11-02 17:15:14.846 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Discovered => 妖精帝國 - Paradiso≒Inferno @ 2024-11-02T17:15:01+01:00 (S)

[2024-11-02 17:15:04.844 +0100] INFO : [App] [Sources] [Ytmusic - YT Music] Potential plays were discovered close to polling interval! Delaying scrobble clients refresh by 10 seconds so other clients have time to scrobble first

For anyone having auth and history fetching issues: please try out the new authentication methods detailed in #236 using the docker image foxxmd/multi-scrobbler:pr-236
If you try out the new docker image please post any issues you have with it in the issue for #236 instead of this one so I can better keep track of things, thanks.


Fix to support logging into a Windows Container

This PR contains a small addition to driver.pyto be able to login into a Windows Container.
Related to: https://github.com/ansible-collections/community.docker/pull/96

@ssbarnea as @chrisjsewell I'm wondering about the errors in the pipeline and ask, if the tests are just broken.

I am sure somethings are broken on CI and I hope someone will attempt to fix them. I am unable to do it myself but I will help with review and release. It is not uncommon to have to fix 2-3 other issues before you make you patch mergeable on open-source projects. :p ... not ideal, but the sad reality.

@ssbarnea Issue is that I even can't run tox locally in the master branch with success.
On issue that I spotted easily was the fact that the repo does not contain a docs folder so that the tox tests testenv:docs and testenv:docs-livereload fail. So I removed those test from tox.ini for testing.

@ssbarnea The tox tests pass on my machine (except the ones for python 3.7, 3.8, 3.9 because I don't have those versions installed) but the pipeline is failed with the following error:

So I suspect Ansible isn't correctly installed via pip in the CI pipeline.

I will close it as until https://github.com/ansible-collections/community.docker/pull/96 is merged and released we cannot do anything about it.


Rotating device - Walkthrough screen do not resize

When you rotate the device the Walkthrough screens do not resize. Is there a way to do this?
Thanks, cool control.

Actually this is not intended as feature for this version of BWWalkthrough. I'm working at version 2.0 that will be released this winter.


[Feature request] Ability to rotate an image on custom angle using fingers

It is awesome and powerful library, but looks like it doesn't have so important and I hope simple in implementation feature to be able to rotate image on custom angle using fingers. There are lots of use cases when user wants to crop image and slightly rotate it.
Is it possible to add such functionality in future releases?

It' hard to implement. The algorithm is not only for how to crop but UI,  i have no idea for now.

Thank you for the quick response. I will try to search for possible solutions and provide my insights on this. Do you mean it is hard to implement crop of rotated image on each platform? Or it is hard to reflect rotation in flutter UI?

I mean flutter UI,I didn't come up with a good idea to calculate boundary after rotate ,and how to convert ui to the arguments for native api.

Thank you for the quick response. I will try to search for possible solutions and provide my insights on this. Do you mean it is hard to implement crop of rotated image on each platform? Or it is hard to reflect rotation in flutter UI?

Did you find a solution to this?

https://github.com/xclud/flutter_crop
This lib can help you. It calculate boundary after rotate

I have switched to this library that can do all required things - https://pub.dev/packages/photo_view


Add EmptySearchContext. Use it for SingleRawAttribute.

@baldersheim : please review
@geirst : FYI

Query on tensor attribute has been blocked upstream. See https://github.com/vespa-engine/vespa/blob/9f4a7279ea632b9c77f3492d22c4cba7851d2b5e/container-search/src/main/java/com/yahoo/search/searchers/QueryValidator.java#L49 for details.


[SPARK-36244][BUILD] Upgrade zstd-jni to 1.5.0-3 to avoid a bug about buffer size calculation

What changes were proposed in this pull request?

This PR upgrades zstd-jni from 1.5.0-2 to 1.5.0-3.
1.5.0-3 was released few days ago.
This release resolves an issue about buffer size calculation, which can affect usage in Spark.
https://github.com/luben/zstd-jni/releases/tag/v1.5.0-3
Why are the changes needed?

It might be a corner case that skipping length is greater than 2^31 - 1 but it's possible to affect Spark.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

CI.

Test build #141417 has started for PR 33464 at commit f3aaed6.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45934/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45934/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45934/

Test build #141417 has finished for PR 33464 at commit f3aaed6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141417/

retest this please.

Test build #141441 has started for PR 33464 at commit f3aaed6.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45960/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45960/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45960/

Test build #141441 has finished for PR 33464 at commit f3aaed6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141441/

BTW, Jenkins doesn't run dependency test. Let's make sure testing it locally and verifiy it passes ..

Dependency test is covered here.

https://github.com/sarutak/spark/runs/3124666363?check_suite_focus=true

Merged to master/3.2


Use core user by default for SSH

core is the default SSH user for CoreOS.
We would expect CoreOS for OpenShift installations

@mulbc Thank you for the PR! The current implementation assumes that the Ansible host is already configured with the right SSH config to reach each node over SSH. I fear this change will break that default behaviour. I might be wrong. But would you explain why you cannot configure an SSH config on the Ansible host machine? Just trying to understand the requirement for this change.

You are right, this is mentioned in docs/inventory-notes.md and I missed it.


Error building when using Node 22.12

Hi I bumped to Node 22.12 and this addon broke with:
SB_CORE-SERVER_0007 (MainFileEvaluationError): Storybook couldn't evaluate your .storybook/main.ts file.

Original error:
ReferenceError: exports is not defined
    at file:///node_modules/storybook-addon-code-editor/dist/cjs/getStaticDirs.js:3:23

Now, I've tried a solution mentioned in this thread with a similar issue:
https://github.com/nodejs/node/issues/56127#issuecomment-2521706469
Removing the "type": "module" in the package json fixed the issue but again I am not sure if this is the correct way of doing things.

I can add {"type":"commonjs"} in the dist/cjs directory as part of building this


zeromq->nanomsg

welll... the failure was because travis couldn't pull from the repo for the python 2 build... but there are no python changes.


Optimize test cases for soul-metric-prometheus

// Optimize test cases for soul-metric-prometheus

Make sure that:

[ ] You have read the contribution guidelines.
[ ] You submit test cases (unit or integration tests) that back your changes.
[ ] Your local test passed mvn clean install -Dmaven.javadoc.skip=true.

Codecov Report

Merging #678 (999fc78) into master (e1b54fb) will increase coverage by 0.84%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #678      +/-   ##
============================================
+ Coverage     49.57%   50.42%   +0.84%
- Complexity      877      895      +18
============================================
  Files           237      237
  Lines          5456     5456
  Branches        604      604
============================================
+ Hits           2705     2751      +46
+ Misses         2491     2444      -47
- Partials        260      261       +1

Impacted Files
Coverage Δ
Complexity Δ

...etrics/prometheus/impl/collector/JmxCollector.java
13.06% <0.00%> (+1.80%)
6.00% <0.00%> (+2.00%)

...cs/prometheus/PrometheusMetricsTrackerFactory.java
100.00% <0.00%> (+12.50%)
5.00% <0.00%> (+1.00%)

...l/summary/RequestLatencySummaryMetricsTracker.java
100.00% <0.00%> (+16.66%)
4.00% <0.00%> (+1.00%)

...impl/counter/HttpRequestCounterMetricsTracker.java
100.00% <0.00%> (+22.22%)
4.00% <0.00%> (+1.00%)

...mpl/counter/RequestTotalCounterMetricsTracker.java
100.00% <0.00%> (+25.00%)
4.00% <0.00%> (+1.00%)

...stogram/RequestLatencyHistogramMetricsTracker.java
100.00% <0.00%> (+28.57%)
4.00% <0.00%> (+1.00%)

...cs/prometheus/PrometheusMetricsTrackerManager.java
86.95% <0.00%> (+86.95%)
7.00% <0.00%> (+7.00%)

.../prometheus/impl/collector/BuildInfoCollector.java
90.00% <0.00%> (+90.00%)
2.00% <0.00%> (+2.00%)

...mmary/PrometheusSummaryMetricsTrackerDelegate.java
100.00% <0.00%> (+100.00%)
1.00% <0.00%> (+1.00%)

...ram/PrometheusHistogramMetricsTrackerDelegate.java
100.00% <0.00%> (+100.00%)
1.00% <0.00%> (+1.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1b54fb...999fc78. Read the comment docs.


Source Editing

Done

route added
separated Modal... as a component
loading data
form schema for editing
put data in the initialValues
saving
updated dependencies

Todo (can be a separate PR)

check username
get a fix for https://github.com/patternfly/patternfly-react/issues/1711

There is a missing dependency for 'redux-mock-store for tests.


qtbase: Security fix for CVE-2024-25580

CVE fixed:
-CVE-2024-25580: qtbase: potential buffer overflow when reading KTX images Upstream-Status: Backport from https://download.qt.io/official_releases/qt/5.15/CVE-2024-25580-qtbase-5.15.diff

qtbase: Security fix for CVE-2024-25580


Day1

is it  better to do feature scaler before dataset split ?

那样就失去划分测试集训练集的意义了 测试集训练集是不能有联系的

thanks,I am a new learner too,so there may be  some wrong, lets study together.
I found some examples,it seemed that we use scaler.fit_transform(X_train) first,then scaler.transform(X_test),the meaning is to fit the scaler by X_train first and then use this scalar to transform X_test (though the code here might be wrong,here is scaler.fit_transform(X_test)),,,  so I am wondering if we can do  feature scaler before dataset split.
what you said might not be totally right. It seems that we do preditction under the hypotheses that the  distribution of train_data and test_data are similiar, actually,doing feature scaler before or after dataset split are equally.

I'm a new learner,too.
For example,LinearNormalization:f(x) = (x - min) / (max - min).There are maximum and minimum values involved.if we use do feature scaler before dataset split.Perhaps the maximum or minimum value exists in the test set.
But in fact,It's like we're predicting the stock price tomorrow,We don't know the stock price tomorrow,so If we want to train the model, we can only use the previous data, and feature scaling can only use the previous data.

your example is right,just you said,we can use the X_train-scalar to transform X_test.But we also need to split dataset first.

We can scale data before or after splitting.
Feature Scaling before splitting:
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)
Feature scaling after splitting:
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
X_test = sc_X.transform(X_test)
sc_y = StandardScaler()
y_train = sc_y.fit_transform(y_train)

cross_validation.py:41: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)


Login to contact Support

Please avoid submitting support questions here. You can contact our support team at https://support.sendgrid.com.
If you are able to login to https://support.sendgrid.com:

Click "Login & Contact Support."
Then, select "Contact Support" to see your contact options.

It's  crasy !
You ask to login to contact support, but i can't login and you ask to contact support !

Hi @remydev,
You don't need to log in in order to open a ticket. Go to https://support.sendgrid.com and click the "trouble logging in?" link to file a ticket.


Attempting a docker cli action from a logged in client locks user accounts when password has changed

When a user runs docker login and provides the proper credentials, their username:password combination get cached in ~/.docker/config.json. Thereafter, client actions will attempt to use these credentials to obtain tokens with the necessary scope (these tokens are not cached client-side, as far as I can tell).
I'm using docker version 1.12.6 (released 3 months ago), so I don't know if this is definitely still an issue, but the client appears to reattempt token fetching 10 times even if it has the wrong password. This means  that if a user has changed their password through the web console after logging in with the docker cli, any attempted action on the registry with the CLI will result in their account getting locked. Because of #1257, this will result in them being permanently locked out of their account.
Portus info

So the issue is:

You login with docker. It succeed.
You change the password in the web interface.
You push, but docker is using the old credentials because they have been cached.

Right ? If so, maybe a way to go is to add a warning whenever you change the password on the web interface stating that you should run docker login again. What do you think ?

After comparing portus and docker hub's token servers' response headers to token requests which contained bad credentials, I realized that Portus's token server would send a WWW-Authenticate: Basic realm="Application" header with a 401. Whereas the Portus registry uses its config for the header value here, the token figure gets this from the devise config.
Sure enough, after setting config.http_authentication_realm in config/initializers/devise.rb, attempting to use the docker client to push or pull against my portus registry prompts for credentials when the ones in its ~/.docker/config.json are incorrect (this is what happens with docker hub as well). Since I don't think we have enough information to construct the proper realm value here, our documentation should instruct Portus users to set this manually before deploying Portus.
In the long term, perhaps Portus should expect PORTUS_AUTH_HOSTNAME, PORTUS_AUTH_PORT, PORTUS_CONSOLE_HOSTNAME, and PORTUS_CONSOLE_PORT environment parameters. Right now the Portus web console and token server are the same, but in the future, perhaps they could be decoupled?

@HartS could you submit a pull request with those changes? The packaged deployment is no longer an issue and we recommend using containerized solutions. I'll take care of the documentation :+1:

@HartS I've submitted #1969, which simply shows a warning message about updating the password. I wanted to also include your commit, but after testing it, I honestly see the same result on the Docker CLI as if I left the realm untouched (tested with the default config.json and with secretservice support).
I'll keep this bug open as a reminder to take a further look, but I'll postpone this part for the next release :+1:

Hi, I just noticed the recent activity in my notifications. I haven't looked at Portus in a while and don't remember the details unfortunately

I wanted to also include your commit, but after testing it, I honestly see the same result on the Docker CLI as if I left the realm untouched

Do you mean that the user still gets locked out of their account when you try to perform an action after a password change? I'll try to take a look at this again if I have some time, though it might take me some time to get Portus back up and running.

Do you mean that the user still gets locked out of their account when you try to perform an action after a password change?

I basically saw the same results (hopefully I didn't do anything wrong :smile:).

I'll try to take a look at this again if I have some time, though it might take me some time to get Portus back up and running.

Thanks a bunch :clap:


RMIZ's future: React 16 & beyond RFC

Intro
With the latest React version and React's future as a whole, some things have come to light that need changing, and this has me thinking that it's time for an overhaul on this project.
Current State
While the project works fine as it is right now, here are some observations:

mixing the controlled vs uncontrolled component modes makes for some very messy code, and it results in some people requiring in code that has nothing to do with how they use the component
current lifecycle methods need to be changed & future-proofed (as well as one can)
there exists a currently seemingly insurmountable issue (https://github.com/rpearce/react-medium-image-zoom/issues/136) that is something I'm afraid I can't address with how things are built right now and feel I can't rip everything apart because...
there are no tests (https://github.com/rpearce/react-medium-image-zoom/issues/55)
keyboard accessibility is a bit hacky at the moment

The Future?
Split Controlled & Uncontrolled Components
For this, I'm thinking either 1 of 2 things:

offer up different components to import or
go the monorepo route and offer them up as different packages

I could use some feedback on this.
All the "business logic" around what makes these components tick would naturally be extracted, shared and tested. Unlike input components, which can be controlled or uncontrolled (given you provide a value or not), developers know from the get-go how they're going to use this component, so I'm of a mind to have them choose from the start.
Updates for future React

componentWillMount, componentWillRecieveProps, componentWillUpdate are all about to be deprecated, so those need to either be updated to something else or find a way around them. Here's a blog post from Dan Abramov about what, why and what's coming: https://reactjs.org/blog/2018/03/27/update-on-async-rendering.html. It's also listed as erroring as part of the new StrictMode component: https://reactjs.org/docs/strict-mode.html#identifying-unsafe-lifecycles.
should update all refs to use the new React.createRef() API: https://reactjs.org/docs/refs-and-the-dom.html. Here's the StrictMode section on it: https://reactjs.org/docs/strict-mode.html#warning-about-legacy-string-ref-api-usage.
...consider using hooks here? bad for supporting older versions? but then again this is supposed to be a rewrite for future react?
...what else?

Testing
I've gotten a lot better about testing React components since this was initially published and updates made. I might as well use this as an opportunity to test all the things instead of relying on Storybook.
Keyboard a11y
Not much to say here other than it seems a bit wonky across browsers. Can address more fully if it's baked in from the get-go.
General Cleanliness
Parts of this library are quite difficult to follow and are hacky because it's what I had to do to make the library work. That said, I think this could improve the quality of the code (and dependability with tests) and make it easier for others to contribute.
Portability Across Frameworks / Languages
I'd love to identify patterns and code blocks that can be used as inspiration for porting this to

vanilla JS
elm
purescript
vue (which I know nothing about)
...?
if I or someone else would like to.

Docs
Small point, but I'm going to move the documentation to separate markdown files and keep the README clean and to the point:

API.md
CONTRIBUTING.md
whatever else is needed

Thoughts? I'm asking all of you because you have contributed in this project in some way and should have a say:
cc @cbothner @jeremybini @ismay @rajit @rsaccon @wtfdaemon @aswinckr @alexshelkov @Snowsoul @kendagriff @HippoDippo @oyeanuj @ludwigfrank @serfgy @babyPrince

I’m in favor of exporting multiple components from the same package, rather than splitting into two packages. It’s likely a user will start with the stateful component and then grow into needing the lower level controlled component. My thinking is that it’s lower friction and more discoverable to change the import at the top of the file rather than package.json.

@cbothner thank you for the feedback.

Update: I've now broken ground on this. Here we go... Working from the rewrite branch

Update: This has been paused as I've been having hand+wrist+forearm pain and need to deal with that before doing this outside normal working hours.
Good news is that we'll be using latest & greatest React & hooks!

@spencerfdavis is going to be giving me a hand with this. Welcome to the project, @spencerfdavis, and I look forward to getting some things rolling together over the coming months!

Update: I've been putting a bit of work into the rewrite and am having a meeting with @spencerfdavis on Monday to figure out what path we want to go down with regard to some things.
We're going to

plan everything that needs doing out (and how ... this is interesting)
create a project board
break things into issues and place them on that board
use https://github.com/rpearce/react-medium-image-zoom/pull/159 as the main rewrite branch until we have something that solves all outstanding problems & we determine a good API. Note: what is there right now is a prototype for ideas around what we're trying to accomplish.

The ultimate goals are:

accessibility
simplify, be smarter, much less code
100% test coverage
easily work with the styles (that rewrite branch has me doing styleInject via rollup, but I've come to learn that content security policies do not jive with that sort of thing well...TBD)
ensure all outstanding issues are handled (that includes mobile)

There are also a few other folks who have expressed interest in contributing (thanks, Hacktoberfest!), so more to come on that, as well.

The prototype for the rewrite is out on the master branch, and it needs a lot of work. We'll be tracking its progress in https://github.com/rpearce/react-medium-image-zoom/projects/1, so I'm going to close this issue.


SkiaPerf movement related to submitting multiple command buffers

On https://github.com/flutter/flutter/commit/1f573e022e0f255bf43e0526713b69d0901b083a, there is some movement in a number of benchmarks. The commit in that roll that looks related to me is https://github.com/flutter/engine/pull/50180, which caused similar movement on a prior land/revert.
@jonahwilliams Could you take a look?
https://flutter-flutter-perf.skia.org/e/?begin=1705570686&end=1707234104&keys=X943ef1874c8228e3dda788af549925c0&requestType=0&xbaroffset=39074
https://flutter-flutter-perf.skia.org/e/?begin=1706648107&end=1706673690&keys=Xde34077be0c0ea195a5027b0f4d8c823&num_commits=50&request_type=1&xbaroffset=39083
https://flutter-flutter-perf.skia.org/e/?begin=1705608310&end=1707234104&keys=X091502ff91dd7b6844699fade37466dc&requestType=0&xbaroffset=39083

Thanks @zanderso
The frame_request_pending_latency was added to test https://github.com/flutter/engine/pull/43711 and its not clear to me that it is a meaningful number to target for Impeller. I think conceptually we want this number to be less. Because the animated advanced blend benchmark is submitting a lot of cmd buffers, it might be the cast that we should not batch as heavily and submit throughout the frame. That said, I don't really want to add heurstics here unless we have a better example than a micro benchmarks.
The second shows a slight increase in build time. That might be due to more work getting scheduled towards the end of the frame, and perhaps that is slightly overlapping with ui work? it seems small.
The last one shows a nice drop in an app that is submitting multiple command buffers.

Thanks for looking! Feel free to close if there's nothing actionable here.

ack ack!


Request: Add Set Primary-Sale-Happened as an option to the mint command

This will allow it to be done in the same transaction.
This covers use cases where the user has already paid for the NFT, and the mint transaction is the sale.

This makes sense and will be pretty easy to add. I'll look at doing it after I do a full release of v0.3.0.

This has been added to main and will be released in the v0.3.1 tag.


[QUESTION] Noob question: How do I upgrade?

Issue Template
Please use this template!
Initial Check

If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.

Make sure you've checked the following:

[x] Python version is 3.6;
[x] Updated Twint with pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint;
[x] I have searched the issues and there are no duplicates of this issue/question/request.

Command Ran
pip3 install twint
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint
Description of Issue
Sorry about this but I'm an amateur trying to figure this out. When I run the first command, I only get the 2019 version. When I run the second command I get:

WARNING: Discarding git+https://github.com/twintproject/twint.git@origin/master#egg=twint. Command errored out with exit status 1: git fetch -q --tags Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement twint (unavailable)
ERROR: No matching distribution found for twint (unavailable)

What am I doing wrong?
Environment Details
Windows 10

Same issue, although I was able to install it on windows earlier.

Reinstalled Python 3.6 and that fixed it.


Fix translation for zh_TW

Correct words for Chinese(Taiwan) , and add missing translation.

@sciuridae0603 i am preparing a new version 3.5, which has a number of additional rules that will need translation.
I could send you the current version of this language file to you for translation if you are willing

@mikeerickson sure , i can translation it :)

@sciuridae0603 I have sent it to you via email.  Thanks for your assitance.

@sciuridae0603 The email address I used from your github profile returned undeliverable.  Which email address would you like me to send the file?

@mikeerickson umm... try<EMAIL_ADDRESS>instead.

@sciuridae0603 OK, one more try!  I have placed the file on my Google Drive
https://drive.google.com/file/d/1WX47C_3u3EQnzkuf7PN95m20w8GxBl4_/view?usp=sharing

https://gist.github.com/sciuridae0603/d2808f5a9b1dca5bde5e4272c3d0d51c
Here you are!

So, it seems I had not updated the existing zh_TW file with all the additional rules added in v3.5
Please see this gist which contains the "updated" zh_TW file to contain everything new.  When you have a moment, if you could add the translation, I would appreciate it.
https://gist.github.com/mikeerickson/6780ff8c0b918086acec84e78766bad2

@mikeerickson
Hello,
I was not involved in the translation of "zh_TW".
I was only involved in "ja".
I don't even know what language "zh_TW" is.

@sciuridae0603 please see above
@sdr0x07b6 my apologies, I tagged the wrong person 🤓

@mikeerickson Umm... I think i need explanation of new rules , can you give me exmaple or explanation of new rules?


AB emoji in hex strings

The :ab: emoji is displayed in strings that contain colon separated hex bytes, e.g.
var s = "ff:00:ab:cd:ef:...";

Ahah. Yes :) That's a dup of #4 and #7, we're tracking it there.


Add commonly substituted characters to Trie data structure

Currently trie only builds based on exact words provided by Facebook. In reality, many users will substitute alphabet characters with symbols or numbers.
For example, a = @, s = $, o = 0, e/b = 3.
The trie should automatically populate with these substitutions in mind.

Common substitutions implemented, merge #29


Fix topic getting recreated immediately after deletion

Motivation
There is a bug in the delete topic procedure that can trigger the topic re-creation immediately after it gets deleted.
This can happen when the topic is not currently open. eg:

Request to delete the topic
Namespace bundle is not owned
Take ownership of bundle
Start background loading of topics for the bundle
Delete the topic

If (5) finishes before (4), then the the can recreate a new topic with the old name.
This bug is causing random failure on some of the tests like NamespaceOwnershipListenerTests where the namespace delete operation fails because "namespace is not empty" even though we just had deleted the only topic.
Modifications

Pass a flag to avoid background topic loading for operations that just require to prove the ownership
Instead of passing N boolean flags, consolidated the args into a LookupOptions data class.

Move to the pull request to 2.6.1


docs(@angular/cli): update proxy config story

Fixes #7772

Failing Travis in rebuild-error, but can't imagine it's because of this commit. Full error message:
Expected no error but an error was shown.
Error: Expected no error but an error was shown.
    at Promise.resolve.then.then.then.then (/home/travis/build/angular/angular-cli/tests/e2e/tests/build/rebuild-error.ts:27:19)
    at process._tickCallback (internal/process/next_tick.js:109:7)

Travis is now working thanks to #7989


NearInteractionGrabbable triggers exception when ManipulationHandler Component is disabled

Describe the bug
InvalidOperationException: NearInteractionGrabbable requires a BoxCollider, SphereCollider, CapsuleCollider or a convex MeshCollider on an object. Otherwise grab interaction will not work correctly.
On an active GameObject with disabled ManipulationHandler Component.
Collider[] colliders = gameObject.GetComponents<Collider>();
the GetComponent does not include disabled components, an alternative could be to use GetComponentsInChildren(includeInactive:true) which has an extra param (sadly the GetComponent does not have this extra param)
i don't know if searching in children would have any side effect here.
To reproduce
Steps to reproduce the behavior:

Create a GameObject with a ManipulationHandler
Disable the ManipulationHandler by hand or by code
When enabling the gameobject, the exception got triggered

Alternative workaround
by code, Enable gameobject first, then disable component
Expected behavior
no exception
Your setup (please complete the following information)

Unity Version [e.g. 2018.4.13f1]
MRTK Version 2.2/2.3

Hi @gilbdev, I cannot reproduce the issue using Unity 2018.4.24f1 and MRTK 2.4, could you try to update to the latest versions and see if the problem still persists? Also, if that still happens please give us a more detailed repro procedure, as the current one is slightly confusing (the supposedly buggy NearInteractionGrabbable script is not mentioned in the repro procedure)

Please reopen the issue if you are still encountering this problem.


Support multiple quotes in sensitive data

We had the case, that sensitive data wasn't masked correctly. We have a Docker config JSON which contains multiple quotes in a field.
Example:
{\"auths\":{\"https://some-registry.io\":{\"username\":\"docker-pull\",\"password\":\
would result in
=> "**"auths\":{\"https://some-registry.io\":{\"username\":\"docker-pull\",\"password\":\
now results in
=> "*******************************************************************************************"
This PR fixes this. Provided tests have been run.

@osterman Is this interesting for you?

@spr-mweber3 seems reasonable.


Parameter format byte breaks generation

The parameter format: byte for type: string breaks generation.
API definition:
---
swagger: '2.0'
x-api-first-package: controllers
info:
  version: 0.0.0
  title: Simple API Test
paths:
  /{bippy}:
    get:
      parameters:
        - in: path
          name: bippy
          type: string
          format: byte
          # path parameter required as a workaround for issue 5
          required: true
      responses:
        200:
          description: bippy response
# model definition required as a workaround for issue 18
definitions:
  bop:
    properties:
      name:
        type: string

Compilation:
[play-swagger-test] $ compile
[info] Updating {file:/Users/marco/Code/play-swagger-test/}root...
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[info] Compiling 17 Scala sources and 1 Java source to /Users/marco/Code/play-swagger-test/target/scala-2.11/classes...
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: No URL path binder found for type Byte. Try to implement an implicit PathBindable for this type.
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: not enough arguments for method fromPath: (implicit binder: play.api.mvc.PathBindable[Byte])play.core.routing.Param[Byte].
[error] Unspecified value parameter binder.
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: No URL path binder found for type Byte. Try to implement an implicit PathBindable for this type.
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: not enough arguments for method implicitly: (implicit e: play.api.mvc.PathBindable[Byte])play.api.mvc.PathBindable[Byte].
[error] Unspecified value parameter e.
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: No URL path binder found for type Byte. Try to implement an implicit PathBindable for this type.
[error] /Users/marco/Code/play-swagger-test/conf/bippy.yaml:0: not enough arguments for method implicitly: (implicit e: play.api.mvc.PathBindable[Byte])play.api.mvc.PathBindable[Byte].
[error] Unspecified value parameter e.
[error] 6 errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 4 s, completed Oct 13, 2015 7:19:49 PM

Following format are not supported:
format: binary, type: string	any sequence of octets
Following format is supported but incorrect:
format: byte, type:string
Should be: 	base64 encoded characters
Handled as: Numeric byte type

One relevant discussion can be seen here:
https://github.com/OAI/OpenAPI-Specification/issues/50

For now, it seems like type binary is only supposed too be supported as body parameter.
We could use mime type application/octet-stream as a marker that body must be encoded or decoded as a binary type.
For bytewe probably need to define appropriate mime type to recognise it as a body parameter. For path, query and header parameters there should be no problems with encoding and decoding.

Code generation is not broken anymore. Support for byteformat is added and tested. Support for binaryformat is added but not well tested. The question is: do we need a special mime type to distinguish it or do we use something else?

Released in 0.1.6


Pylance not showing suggestions for OpenCV module

Environment data

Python extension version: v2021.6.8646431
Pylance version: v2021.5.4-pre.1
Language Server version:
OS and version: ubuntu 21.04
Python version : 3.9.4 64-bit

Expected behaviour
showing suggestions like classes, functions, etc.
Actual behaviour
not showing anything
Logs
Search paths for /home/arian/Desktop/codes
/home/arian/.vscode/extensions/ms-python.vscode-pylance-2021.5.4-pre.1/dist/typeshed-fallback/stdlib
/home/arian/Desktop/codes
/home/arian/Desktop/codes/typings
/home/arian/.vscode/extensions/ms-python.vscode-pylance-2021.5.4-pre.1/dist/typeshed-fallback/stubs/...
/home/arian/.vscode/extensions/ms-python.vscode-pylance-2021.5.4-pre.1/dist/bundled/stubs
/usr/lib/python3.9
/usr/lib/python3.9/lib-dynload
/home/arian/.local/lib/python3.9/site-packages
/usr/local/lib/python3.9/dist-packages
/usr/lib/python3/dist-packages
/usr/lib/python3.9/dist-packages
[Error - 1:22:37 PM] Exception received when installing recursive file system watcher
[Info  - 1:22:37 PM] Searching for source files
[Info  - 1:22:37 PM] Auto-excluding /home/arian/Desktop/codes/Image_Processing
[Info  - 1:22:37 PM] Found 21 source files
[BG(1)] getSemanticTokens delta previousResultId:1621846280326 at /home/arian/Desktop/codes/Image_processing1.py (37ms)
Background analysis message: getSemanticTokens delta
[BG(1)] getSemanticTokens delta previousResultId:1621846281362 at /home/arian/Desktop/codes/numpy2.py (3ms)
Background analysis message: setConfigOptions
Background analysis message: ensurePartialStubPackages
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: analyze

Can you provide a code example of something that doesn't work but should? Or a screenshot showing where you're expecting info but are seeing none? As of a few releases ago, we should be shipping a basic stub for cv2.

after importing the library and using it, as the snippet below pylance does not show anything about the functions, classes or etc.
import cv2
cv2.

Same as numpy, when I want to input np.asarray, which is commonly used in my projects, there is not any suggestion; but when I use opencv functions, suggestions will be shown in my computer.

is this still happening with the latest pylance?
have you tried updating your numpy or opencv packages?

packages version were: opencv-contrib-python==<IP_ADDRESS>, opencv-python==<IP_ADDRESS>, numpy==1.20.3
I updated them to : opencv_python-<IP_ADDRESS>, opencv_contrib_python-<IP_ADDRESS>, numpy-1.21.0
the issue seems to be fixed.
the big issue is that on the opencv side the documentation is horrible, it is sad you know a great package with bad documentation!

packages version were: opencv-contrib-python==<IP_ADDRESS>, opencv-python==<IP_ADDRESS>, numpy==1.20.3
I updated them to : opencv_python-<IP_ADDRESS>, opencv_contrib_python-<IP_ADDRESS>, numpy-1.21.0
the issue seems to be fixed.
the big issue is that on the opencv side the documentation is horrible, it is sad you know a great package with bad documentation!

Aye, and so does the trackback information.🤣

The next release will have a partial stub fix for opencv. this should help with cases where opencv is installed in dist-packages


Update readme to match config

Tracked down why the example CSRF headers were not working correctly. I believe there is even an open issue regarding this.

Closing because I think the issue is solved and there has been no more feedback.


Adding support for arithmetic operations on typed columns

Here is an example of what this tries to achieve.
val e = TypedDataset.create[(Int, String, Long)]( (1,"a",2L) :: (2, "b", 4L) :: (2, "b", 1L) :: Nil )
e.select(e('_1) + 1).collect().run()
res1: Seq[Int] = WrappedArray(2, 3, 3)

Maybe there are other PR that try to achieve the same? Please advice! I will try to refine this based on comments. Thanks!
Given that I did the mistake of originating my previous PR from master, this PR also includes the changes from #47. Moving forward each PR will be from a separate branch that always branches of the current remote master of frameless. I will fix this as soon as we are done merging #47.

Regarding you comment on where to fork from for a PR, I think starting from master is the way to go (you did not do a mistake, it's all our fault if test are currently broken on master...). I think you would only want to fork from an existing PR where there is a dependency between them :)

Current coverage is 80.98% (diff: 58.33%)

Merging #50 into master will decrease coverage by 0.61%

@@             master        #50   diff @@
==========================================
  Files            20         20
  Lines           375        384     +9
  Methods         371        379     +8
  Messages          0          0
  Branches          4          5     +1
==========================================
+ Hits            306        311     +5
- Misses           69         73     +4
  Partials          0          0

Powered by Codecov. Last update 15888de...f82fded

Just added an update (not final still in progress. I want to see if I missed something and add comments as @OlivierBlanvillain suggested). One thing to notice is that I force the type after a devision to be double. I noticed this behavior in Spark, so I tried to replicate this with enforcing the type. Something else I have in mind is to support cross type operations. For example, what if you add an Int to a column that is Double? Have some idea on using a TypeCoercion type class:
trait TypeCoercion[LeftType, RightType] {
   type ResultType
}

implicit val intLong = new  TypeCoercion[Int, Long] {
   type ResultType = Long
}

implicit val intDouble = new  TypeCoercion[Int, Double] {
   type ResultType = Double
}

etc.

Any ideas? Is there anything we can reuse for implementing this?

@imarios this Coercion type class looks nice and indeed faithfully models Sparks behavior, which is also what you would find on most dynamically typed languages...
My personal opinion is that this (Spark's behavior) is a terrible idea, and I'm suggesting we do thing more explicitly in frameless by adding something like def widen[T: CatalystWiden]: TypeColumn[T] = noop. This is what it could look like for users:
val e = TypedDataset.create[(Int, String, Long)](???)

e.select(e('_1) + 4L)     // error: type mismatch; found   : Long required: Int
e.select(e('_1) + e('_3)) // error: type mismatch; found   : TypedColumn[Long] required: TypedColumn[Int]

e.select(e('_1) + e('_3).widen[Int]) // error: Implicit not found for CatalystWiden[Long, Int]: cannot down cast Long to Int, use `.map` instead or whatever,

e.select(e('_1).widen[Long] + 4L)
e.select(e('_1).widen[Long] + e('_3))

I think something similar to your TypeCoercion approach could be used to solved this thing: #51.
What do you guys thing?

@OlivierBlanvillain I agree that implicit widening is very dangerous.
I'm not sure about widen approach because it doesn't actually do any casts in Spark. It's more like hiding the problem, we might want to implement cast from Spark that is very close how Coercion is done in this PR, that would do actual casting so dataset schema would match to actual representation in Spark.

Yeah I just thought about it again and my widen does not work at all because it's out of the context of +...
Since all we need is something explicit (given that this is a corner use case), what about a second version of +:
def +[F, T, R](e: modelingSparkMadness[F, T, R])(TypedColumn[T]): TypedColumn[R]

Usage:
e.select(e('_1) +[Int, Long, Long] e('_3))

Or we could simply disable/disallow this feature and rely on Spark's cast :)

Thanks for the comments guys! let me look more into this spark cast approach. Just wanted to add here that the coercion (or widening) I tried to add here mimics exactly how Scala also evaluate expressions:
scala> val i: Int = 5
i: Int = 5
scala> 5L + i
res7: Long = 10

scala> 5L + 0.0
res8: Double = 5.0

scala> 5L * 1.0
res9: Double = 5.0

scala> 5 * 1.0
res10: Double = 5.0

I agree that coercion and widening has some really bad reputation (especially if you heart of coercion from JavaScript), but in this case, I mearly suggest we replicate the conversions that Scala allows.

Found an issue :( with the implementation. Every time we do an operation (+- udf etc.) on a column spark renames the column internally.
scala> case class Foo(i: Int, j: String)
defined class Foo
scala> e
frameless.TypedDataset[Foo] = [i: int, j: string]
scala> e.dataset.dtypes
Array[(String, String)] = Array((i,IntegerType), (j,StringType))
scala> e.select(e('i))
frameless.TypedDataset[Int] = [i: int]
scala> val y = e.select(e('i)+1)
y: frameless.TypedDataset[Int] = [(i + 1): int]
scala> y.dataset.dtypes
Array[(String, String)] = Array(((i + 1),IntegerType)) //  the name of the new column is the expression that created it :/

We somehow need to keep different names and conversions for these names. Otherwise we will not be able to typesafely reference the columns of TypedDatasets created from expressions.

I agree that coercion and widening has some really bad reputation, but in this case, I merely suggest we replicate the conversions that Scala (and Spark) allows.

I would assume anyone doing serious arithmetic compiles with -Ywarn-numeric-widen and -Xfatal-warnings, and I think we should do the same by default. That being said, I'm' be perfectly fine with enabling such madness thing using something like import fameless.silently.widen.numerics._ :)

Every time we do an operation (+- udf etc.) on a column spark renames the column internally.

Do you think we could systematic rename results of selects to override this behavior?

Do you think we could systematic rename results of selects to override this behavior?
@OlivierBlanvillain , I tried to partially address this for select. The edit it in my last commit. Let me know what you think!

@imarios 354c4a4 :+1:

@OlivierBlanvillain @imarios regarding column names in schemas, I've observed this behavior before, unfortunately, most of Spark functions do renaming. Each time you do select you create a tuple, that doesn't contain any information on column names, that's why there is no problem with statically referencing them :), so the problem is only in as operation we provide, it should be improved a lot to do proper renaming.

I'm not sure we can fix column names just by modifying select because there are other ways to create tuples, for instance, groupBy. And then each time we create a tuple we have to alias it, probably we can modify RecordEncoder to respect this, I will check this today.

Here we go, looks like it works. But we have to use TypedDataset.create instead of new TypedDataset https://github.com/adelbertc/frameless/pull/55

Hey guys! I am adding some documentation in TypedColumn and added few more operators and unit tests.

@OlivierBlanvillain @kanterov rebased on top of #57. Contains an expansion of select (up to 10 dimensions) with more unit tests. Also includes operations with constants e.g., d('i) * 0.2, includes unary - and devision. It also includes a revised a version of the dev REPL for us test the project in a colorful interactive REPL that starts and stops spark automatically.

LGTM!


Audit YAML parsing code path

The YAML parser we're using are not robust to untrust input, it should only be used to parse local file. We should add some basic limitation in the code base to prevent it happening.

👍 I'm a big fan of structurally preventing use of untrusted dependencies for trusted things.


⚠ Bofrost.lu has degraded performance

In 2246f3f, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 925 ms

Resolved: Bofrost.lu performance has improved in 2889637 after 6 minutes.


Fix sort order for greater than and less than range notation

Range notation support is currently valid for ">=" and "<=" but not for ">" or "<".
Example: "(max-width: 768px)" and "(width < 768px)" are treated as equivalent and will fall-through to a localCompare sort.
This edit assumes "(width < 100px)" is equivalent to "(max-width: 99px)" and "(width > 100px)" is equivalent to "(min-width: 101px)".

Hi @mattblackstone!
New version published on the NPM