Review: Scrapers

Overview
The goal of this review meeting is to get everyone up to speed with the installation of the system, and to do a review of all the scraper code, looking for ways to make it better structured and easier to maintain.
Review Branch
https://github.com/internaloha/internaloha/tree/review-100/scraper/scrapers
Files to review
Please review as many of the files as you can in https://github.com/internaloha/internaloha/tree/review-100/scraper/scrapers
Checklists

Design
JS and ESLint

Due date
Monday, January 18, 1:30pm
For more information
The review process is documented at: https://internaloha.github.io/documentation/docs/developers/review/review-process

Improve the CLI for scraping
We need a "production mode" interface to the scrapers.  That means a single top-level command that (under normal circumstances):

Runs any or all of the scripts without opening up a browser.
Does not produce any logging output.
Does not require a control-c to terminate.
Does all necessary post-processing of the data.
If there are scrapers that require user input, either fix them or we will deal with them separately.

Create a top-level interface
We need a top-level interface to all scrapers (for both development and production mode):

Each scraper definition file exports a single async function that promises to return a JSON object with the data.
A new top-level command (defined in its own file) loops through all scraper definition functions, invoking them, then outside the loop calls Promise.all to wait for all of the results. See https://eslint.org/docs/rules/no-await-in-loop for example code.  This enables all of the scraping to be done in parallel, which could speed things up 10x.
Once all of the JSON objects are available, postprocessing is done and final data is written in a UI-ready format.

Separate utility code from scraper definition code
The scraper/scrapers/ directory currently mixes scraper definition files and files containing utility code. To separate this out, we can have two directories: scraper/scrapers and scraper/util.  Put the utility files in scraper/util.
Switch from console.log to loglevel
We need a flexible way to provide output for development and debugging that can be disabled easily for production. The best way to do that is to start using a logging package such as https://www.npmjs.com/package/loglevel. Use "silent" for production, and use "trace", "warn", and "error" to categorize output.
Fix ESLint warnings and errors.
I had to make minor changes to ESLint to make it run, and now it finds many problems.
Generic issues that afflict most/all scraper definition code
Check for these in all files.  I only noted them the first time I saw them.

JS-04: Avoid IIAFE
DESIGN-06: UserAgent string should be imported?
JS-05: Collapse code onto one line.
DESIGN-06: Import fs.writeFile as utility function.
DESIGN-08: Modularize long code sections.
DESIGN-01: Avoid deeply nested code

Comments on specific scraper definition files
acm.js

L4?: JS-04: Avoid IIAFE
L13?: DESIGN-06: UserAgent string should be imported?
L29?: ESLINT-01: Add document as a global var to ESLint to eliminate need for warning.
L24: Why loop from 2 to 10? Need a comment.
L39: Why not calculate urls.length in for loop declaration?
L37: Why not calculate length in for loop declaration (L39)?
L43: Why is lastScraped calculated inside the for loop?
L48+: JS-03: No console log. Use a logger instead.
L96: What if there are more or less than 8 tabs?
L103: DESIGN-06: Import fs.writeFile as utility function.

aexpress.js:

L16: JS-05: Collapse code onto one line.
L29: DESIGN-03: Eliminate large comment block.

apple.js

L6: DESIGN-06: Import delay() as utility function.

Glassdoor.js

JS-01: File name should not be capitalized.
L22: JS-07: Use the ternary operator:
const daysBack = (posted.includes('h') || posted.includes('hours')) ? 0 : posted.match(/\d+/g);

idealist.js

createDate(): Why is this async? Needs documentation of parameters. Not sure what this is doing.
L186+: DESIGN-05: Refactor.

ihiretechnology.js

L31+: DESIGN-03: Eliminate large comment block.

indeed.js

L179, 188: JS-07: ternary operator.

linkedin.js

L19: DESIGN-03: Eliminate large comment block.
DESIGN-08: Modularize long code sections

monster.js

L26: DESIGN-03: Eliminate large comment block.
L63: JS-06: Don't use try-catch for control flow.

simplyHired.js:

L118: Skills not defined. ESLint does not pass.

stackoverflow.js, zipRecruiter.js

I looked at this briefly, didn't see anything outside of generic issues.

Improve the CLI for scraping
We need a "production mode" interface to the scrapers.  That means a single top-level command that (under normal circumstances):

Runs any or all of the scripts without opening up a browser.
Does not produce any logging output.
Does not require a control-c to terminate.
Does all necessary post-processing of the data.
If there are scrapers that require user input, either fix them or we will deal with them separately.

Create a top-level interface
We need a top-level interface to all scrapers (for both development and production mode):

Each scraper definition file exports a single async function that promises to return a JSON object with the data.
A new top-level command (defined in its own file) loops through all scraper definition functions, invoking them, then outside the loop calls Promise.all to wait for all of the results. See https://eslint.org/docs/rules/no-await-in-loop for example code.  This enables all of the scraping to be done in parallel, which could speed things up 10x.
Once all of the JSON objects are available, postprocessing is done and final data is written in a UI-ready format.

Separate utility code from scraper definition code
The scraper/scrapers/ directory currently mixes scraper definition files and files containing utility code. To separate this out, we can have two directories: scraper/scrapers and scraper/util.  Put the utility files in scraper/util.
Switch from console.log to loglevel
We need a flexible way to provide output for development and debugging that can be disabled easily for production. The best way to do that is to start using a logging package such as https://www.npmjs.com/package/loglevel. Use "silent" for production, and use "trace", "warn", and "error" to categorize output.
Fix ESLint warnings and errors.
I had to make minor changes to ESLint to make it run, and now it finds many problems.
Generic issues that afflict most/all scraper definition code
Check for these in all files.  I only noted them the first time I saw them.

JS-04: Avoid IIAFE
DESIGN-06: UserAgent string should be imported?
JS-05: Collapse code onto one line.
DESIGN-06: Import fs.writeFile as utility function.
DESIGN-08: Modularize long code sections.
DESIGN-01: Avoid deeply nested code

Comments on specific scraper definition files
acm.js

L4?: JS-04: Avoid IIAFE
L13?: DESIGN-06: UserAgent string should be imported?
L29?: ESLINT-01: Add document as a global var to ESLint to eliminate need for warning.
L24: Why loop from 2 to 10? Need a comment.
L39: Why not calculate urls.length in for loop declaration?
L37: Why not calculate length in for loop declaration (L39)?
L43: Why is lastScraped calculated inside the for loop?
L48+: JS-03: No console log. Use a logger instead.
L96: What if there are more or less than 8 tabs?
L103: DESIGN-06: Import fs.writeFile as utility function.

aexpress.js:

L16: JS-05: Collapse code onto one line.
L29: DESIGN-03: Eliminate large comment block.

apple.js

L6: DESIGN-06: Import delay() as utility function.

Glassdoor.js

JS-01: File name should not be capitalized.
L22: JS-07: Use the ternary operator:
const daysBack = (posted.includes('h') || posted.includes('hours')) ? 0 : posted.match(/\d+/g);

idealist.js

createDate(): Why is this async? Needs documentation of parameters. Not sure what this is doing.
L186+: DESIGN-05: Refactor.

ihiretechnology.js

L31+: DESIGN-03: Eliminate large comment block.

indeed.js

L179, 188: JS-07: ternary operator.

linkedin.js

L19: DESIGN-03: Eliminate large comment block.
DESIGN-08: Modularize long code sections

monster.js

L26: DESIGN-03: Eliminate large comment block.
L63: JS-06: Don't use try-catch for control flow.

simplyHired.js:

L118: Skills not defined. ESLint does not pass.

stackoverflow.js, zipRecruiter.js

I looked at this briefly, didn't see anything outside of generic issues.

Repeat code, could add as utility functions:

Write to file
Converting date to date time format
Evaluate --> Array.from(document... ) format
page set up (setBrowser up, angellist has good example)
Wait for selector then click...; make sure parameter has option to specify how long to wait before timeout

For all:

JS-04: Avoid Immediately Invoked Async Function Expressions
JS-06: Don't use try-catch for control flow

Create function to check if selector exists..
Ideas

DESIGN-08: Modularize long code sections.

Break up into separate functions or call from scraperFunctions util

DESIGN-01: Avoid deeply nested code
No await in loop

Note some awaits in the loop are dependent of each other (eg. click to show something, then scrape info

simplyHired:

Bunch of commented out code
await in loop
Undeclared variable for skills

ACM:

Lots of comments inline to disable eslint
Missing posted

Aexpress:

Unused autoscroll
commented out sections
Posted doesn't return as date time

AngelList:

playTest function name

Apple:

Posted doesn't return as date time

Glassdoor:

Scrapeinfo function confusing

idealist:

a lot of try/catch used for control flow
lot of if/else for data push

iHireTech:

lots of commented out / old code

Indeed:

hasNext = true uses try/catch for control flow

Linkedin:

Duplicated code for date conversion
Commented out code

Monster:

Commented out code block
nextPage uses try/catch for control flow

Statistics:

Uses lodash for concat

Ziprecruiter:

Uses try/catch for control flow

multi-parser:

Nested statements
no main

Single-parser:

Repeated code... could combine with multi-parser (if no args, parse everything. else arg matches the file name (use try/catch to see if name is inputted correctly)

Repeat code, could add as utility functions:

Write to file
Converting date to date time format
Evaluate --> Array.from(document... ) format
page set up (setBrowser up, angellist has good example)
Wait for selector then click...; make sure parameter has option to specify how long to wait before timeout

For all:

JS-04: Avoid Immediately Invoked Async Function Expressions
JS-06: Don't use try-catch for control flow

Create function to check if selector exists..
Ideas

DESIGN-08: Modularize long code sections.

Break up into separate functions or call from scraperFunctions util

DESIGN-01: Avoid deeply nested code
No await in loop

Note some awaits in the loop are dependent of each other (eg. click to show something, then scrape info

simplyHired:

Bunch of commented out code
await in loop
Undeclared variable for skills

ACM:

Lots of comments inline to disable eslint
Missing posted

Aexpress:

Unused autoscroll
commented out sections
Posted doesn't return as date time

AngelList:

playTest function name

Apple:

Posted doesn't return as date time

Glassdoor:

Scrapeinfo function confusing

idealist:

a lot of try/catch used for control flow
lot of if/else for data push

iHireTech:

lots of commented out / old code

Indeed:

hasNext = true uses try/catch for control flow

Linkedin:

Duplicated code for date conversion
Commented out code

Monster:

Commented out code block
nextPage uses try/catch for control flow

Statistics:

Uses lodash for concat

Ziprecruiter:

Uses try/catch for control flow

multi-parser:

Nested statements
no main

Single-parser:

Repeated code... could combine with multi-parser (if no args, parse everything. else arg matches the file name (use try/catch to see if name is inputted correctly)

For all files:

change console.log
change try-catch

For all files:

change console.log
change try-catch

Remaining issues in scraper code

[ ] idealist.js

Design-08: Modularize long code sections

[ ] indeed.js

Design-08: Modularize long code sections

[ ] linkedin.js

Design-08: Modularize long code sections

[ ] monster.js

Design-08: Modularize long code sections

[ ] simplyHired.js

Design-08: Modularize long code sections

[ ] stackoverflow.js

Design-08: Modularize long code sections

[ ] ziprecruiter.js:

Design-08: Modularize long code sections.
L112: why main.then()? why not just main() other scrapers?

[ ] ScraperFunctions.js

JS-00: Indentation and whitespace (remove whitespace, use // comment if you prefer)
JS-01: File violates naming conventions.

[ ] single_parser.js

JS-00: Indentation and whitespace (remove whitespace, use // comment if you prefer)
JS-01: File violates naming conventions.

[ ] Stackoverflow.js

Design-08: Modularize long code sections
Design-03: Eliminate large comment blocks
L112: why main.then()? why not just main() like other scrapers?

[ ] Statistics.js:

Consider loops? For example:

const counts = { company: 0, contact: 0, ... };
for (field in counts) {
  if (data[field]) {
    counts[field]++;
}

There are two other places where a loop would simplify inline code (_.concat, getStatistics).


test zuul 3.9.0

Trying to isolate the Zuul bug...

Yeah okay, you can see the failure in Travis: https://travis-ci.org/No9/localstorage-down/builds/104636351

zuul fixed this bug recently, updating to latest and re-testing...

Coverage remained the same at 86.322% when pulling d4465d4159a065e74c0c2c91dbb83a4353f0b84a on zuul-3.9.0 into fed5b8cc670a56aa29d03b5878468964ca5dc114 on master.

self-+1ing because it fixes the build!


i18n: add missing transaltions - Misc test fr

@quisquous or @Akurosia
For these 3 regex, I'm not sure. Is there a way to check them?
netRegexFr: NetRegexes.echo({ line: 'cactbot langue.*?', capture: false }),
netRegexFr: NetRegexes.echo({ line: 'cactbot test de réponse.*?', capture: false }),
netRegexFr: NetRegexes.echo({ line: 'cactbot montre d\'essai.*?', capture: false }),

Hmm.  I'm not sure what you're asking.  These are just commands in game, e.g. /echo cactbot langue.
That said, I'm not sure that these should have translations? I think these are just debugging commands to test things out, and so probably they should be the same in all languages.  What do you think, @Akurosia?

i would be fine in either scenario,
Best thing would be if both the english and the translated commnd would work^^'
But i think with the current implementation just english would be the best to avoid confusion (but tbh most users will never use any of these commands)

You can merged. I understood what the regex wanted to do. It's OK.

Thank you for your respons. That helped me.


Fix problem with "stackanetes command not working outside source code directory"

@pawelsocha
@PiotrProkop
Signed-off-by: DTadrzak<EMAIL_ADDRESS>

LGTM


Doesn't compile when editing from another dir

cd test
nvim fnl/t.fnl # no fnl/t.lua
cd fnl/ && nvim t.fnl # -> t.lua

test/.nfnl.fnl:
{
  :source-file-patterns ["*.fnl" "**/*.fnl"]
}

This is particularly bothersome given that most people are navigating to files using fzf or tree view  which don't change the cwd.
Am I missing something simple?

Oh interesting, I think I have some core assumptions where the .nfnl.fnl file is found by walking up from your CWD. I'll see how easy it'd be to base everything off of the current file path instead. The only danger here is that users configuring their own :fnl-path->lua-path function will need to potentially deal with times when the editor isn't in the expected CWD.
Just in case their logic depends on it, like in my example code from the readme.
(local config (require :nfnl.config))
(local default (config.default))

{:fnl-path->lua-path (fn [fnl-path]
                       (let [rel-fnl-path (vim.fn.fnamemodify fnl-path ":.")]
                         (default.fnl-path->lua-path (.. "some-other-dir/" rel-fnl-path))))}

Yea don't think there's a way around it if you want a custom output path.
Also just found this (seems kinda related) - there's a bug in the generated lua code comment depending on whether I do nvim util.fnl or nvim #then nav to util.fnl
diff:
--- [nfnl] Compiled from util.fnl by https://github.com/Olical/nfnl, do not edit.
+-- [nfnl] Compiled from  by https://github.com/Olical/nfnl, do not edit.
 local function dump(o)

What's odd is that the filename is missing when doing nvim util.fnl rather than when navigating from within vim which I thought might be the problem.

Could you try this again now? I may have accidentally found and fixed the bug which was to do with autocmd events sometimes including relative paths for the affected file and sometimes being absolute?
This seems to be something that depends on Neovim configuration because my test suite ran locally but failed in CI in the same way you described (blank file after "Compiled from"). This now has a e2e test in CI that checks this sort of thing automatically on push, so it shouldn't happen again. If this was the issue.
I just tried editing a fnl file from a directory above and beside another one and both compiled as expected wile the CWD was set wrong.
So I wrote nfnl to work like this all along, I just forgot and there was a bug that was breaking it in a weird way.

Yep, confirmed fix on both bugs - thanks!


Warp API

Unless I'm missing something (and I might be, TBH), we're missing the Warp API, (see https://gdal.org/tutorials/warp_tut.html).

Looks like we kinda have a start...


Envoy throws codec_error:The_user_callback_function_failed

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
What I'm doing?
I have a basic grpc js client and a grpc python server.
The grpc client it does 60 iterations, in each iteration it fires 300 grpc requests in parallel to the grpc server. The server processing is pretty simple, it sleeps 5 seconds and it respond success.
Both grpc client and grp server are in the same namespace in the same eks cluster version 1.25
Currently both grpc client and grpc server have both an istio side car version 1.18.5, and it is working perfectly.
But the problem is when I upgrade istio to 1.19.X (any version) or 1.20.X (any version)
What I expected to happen?
I expect when I upgrade istio from 1.18.5 to 1.19.X or to 1.20.X the application should behave the same way and finish all 60 iterations.
What is actually happening?
The script does not reach the 60 iterations. It starts to complete iterations with all responses success, but approx at the iteration 9 It fails with the following error:
D 2024-01-21T05:10:23.395Z | resolving_call | [5398] ended with status: code=13 details="Received RST_STREAM with code 2 triggered by internal client error: Session closed with error code 2"
D 2024-01-21T05:10:23.338Z | subchannel_call | [7199] Node error event: message=Session closed with error code 2 code=ERR_HTTP2_SESSION_ERROR errno=Unknown system error undefined syscall=undefined
This does not happen, because of a bad response from the server, I can see the server always respond the requests successfully, the client just throw this exception out of nowhere.
If I change the script to fire more requests in paralallel, lets say 600, the script finishes earlier, it throws this same error but in the iteration 4 approx.
If I change the script to fire less requests in parallel, lets say 100, the script finishes the 60 iterations successfully. So there is a number of requests in parallel, 250 approx (300 for sure) where the script start to fail.
The grpc client is using "@grpc/grpc-js": "1.8.21", and the grpc server python is using: grpcio = "==1.60.0". I was looking at this libraries to see if they have some issues, but these same versions work when i try with istio 1.18.5.
In fact, if I try the same script, without istio, i remove the side car injection, the script finishes without problems. For some reason, the new version of envoy that comes with the istio version 1.19.X and up, is making the grpc client fail.
With istio 1.18.5 it doesn't matter the amount of requests in parallel I fire from the client, 100, 200, 300, 600, even 1000, it always finishes success.
Exception I'm getting
The envoy side car in the client shows:
2024-01-27T07:18:14.817567Z	trace	envoy connection external/envoy/source/common/network/connection_impl.cc:615	[Tags: "ConnectionId":"7"] read ready. dispatch_buffered_data=0	thread=25
2024-01-27T07:18:14.817567Z	trace	envoy connection external/envoy/source/common/network/raw_buffer_socket.cc:24	[Tags: "ConnectionId":"7"] read returns: 26	thread=25
2024-01-27T07:18:14.817567Z	trace	envoy connection external/envoy/source/common/network/raw_buffer_socket.cc:38	[Tags: "ConnectionId":"7"] read error: Resource temporarily unavailable	thread=25

2024-01-27T07:18:14.817867Z	debug	envoy http2 external/envoy/source/common/http/http2/codec_impl.cc:1208	[Tags: "ConnectionId":"7"] sent goaway code=2	thread=25
2024-01-27T07:18:14.817867Z	debug	envoy http external/envoy/source/common/http/conn_manager_impl.cc:427	[Tags: "ConnectionId":"7"] dispatch error: The user callback function failed	thread=25
2024-01-27T07:18:14.817867Z	debug	envoy http external/envoy/source/common/http/conn_manager_impl.cc:1889	[Tags: "ConnectionId":"7","StreamId":"7112271742612094099"] stream reset: reset reason: connection termination, response details: -	thread=25

Which looks like is saying the client is sending the goway due to  a callback function failed and proceed to reset the stream
Then in the application I see these relevant lines:
2024-01-27T07:18:14.951034047Z stderr F HTTP2 18: Http2Session client: goaway 2 received [last stream id: 3999]
2024-01-27T07:18:14.951037006Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 connection closed by GOAWAY with code 2 and data undefined
2024-01-27T07:18:14.951041047Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | subchannel | (2) <IP_ADDRESS>:8081 READY -> IDLE
2024-01-27T07:18:14.951043381Z stderr F HTTP2 18: Http2Session client: marking session closed
2024-01-27T07:18:14.951045672Z stderr F HTTP2 18: Http2Session client: submitting goaway
2024-01-27T07:18:14.951047214Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | subchannel_refcount | (2) <IP_ADDRESS>:8081 refcount 2 -> 1
2024-01-27T07:18:14.951049006Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | pick_first | READY -> IDLE
2024-01-27T07:18:14.951050881Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | resolving_load_balancer | dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 READY -> IDLE
2024-01-27T07:18:14.951052756Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | connectivity_state | (1) dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 READY -> IDLE
2024-01-27T07:18:14.951054797Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | dns_resolver | resolution update delayed by "min time between resolutions" rate limit
2024-01-27T07:18:14.951056631Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | resolving_load_balancer | dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 IDLE -> CONNECTING
2024-01-27T07:18:14.951058131Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | connectivity_state | (1) dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 IDLE -> CONNECTING
2024-01-27T07:18:14.951059756Z stderr F HTTP2 18: Http2Session client: destroying
2024-01-27T07:18:14.951061297Z stderr F HTTP2 18: Http2Session client: start closing/destroying Error [ERR_HTTP2_SESSION_ERROR]: Session closed with error code 2
2024-01-27T07:18:14.951063089Z stderr F     at Http2Session.onGoawayData (node:internal/http2/core:690:21)
2024-01-27T07:18:14.951064881Z stderr F     at Http2Session.callbackTrampoline (node:internal/async_hooks:130:17) {
2024-01-27T07:18:14.951066714Z stderr F   code: 'ERR_HTTP2_SESSION_ERROR'
2024-01-27T07:18:14.951068631Z stderr F }
2024-01-27T07:18:14.951070172Z stderr F HTTP2 18: Http2Stream 2025 [Http2Session client]: destroying stream

2024-01-27T07:18:14.952236297Z stderr F HTTP2 18: Http2Session client: finishSessionClose
2024-01-27T07:18:14.952238089Z stderr F D 2024-01-27T07:18:14.875Z | v1.9.14 18 | subchannel_call | [7012] Node error event: message=Session closed with error code 2 code=ERR_HTTP2_SESSION_ERROR errno=Unknown system error undefined syscall=undefined

2024-01-27T07:18:14.964811172Z stderr F D 2024-01-27T07:18:14.901Z | v1.9.14 18 | resolving_call | [4999] ended with status: code=13 details="Received RST_STREAM with code 2 triggered by internal client error: Session closed with error code 2"
2024-01-27T07:18:14.964813089Z stderr F HTTP2 18: Http2Session client: finishSessionClose socket end null Error [ERR_HTTP2_SESSION_ERROR]: Session closed with error code 2
2024-01-27T07:18:14.964815214Z stderr F     at Http2Session.onGoawayData (node:internal/http2/core:690:21)
2024-01-27T07:18:14.964816714Z stderr F     at Http2Session.callbackTrampoline (node:internal/async_hooks:130:17) {
2024-01-27T07:18:14.964818381Z stderr F   code: 'ERR_HTTP2_SESSION_ERROR'
2024-01-27T07:18:14.964820172Z stderr F }
2024-01-27T07:18:14.964821631Z stderr F D 2024-01-27T07:18:14.934Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 connection closed with error Session closed with error code 2
2024-01-27T07:18:14.964823172Z stderr F D 2024-01-27T07:18:14.934Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 session closed
2024-01-27T07:18:14.980607506Z stdout F 2024-01-27T07:18:14Z scuttle: Received signal 'child exited', passing to child
2024-01-27T07:18:14.980790964Z stdout F 2024-01-27T07:18:14Z scuttle: Received signal 'urgent I/O condition', ignoring

Where I see also here the line: "Http2Session client: submitting goaway" which looks like show the client is sending the goaway confirming the envoy side car log. This log line comes from:
https://github.com/nodejs/node/blob/f38a9a59182027493852478b8a3abd5d7d5d1726/lib/internal/http2/core.js#L853
Which ends up calling this function: https://github.com/nodejs/node/blob/f38a9a59182027493852478b8a3abd5d7d5d1726/lib/internal/http2/core.js#L1508
which clarifies "Submits a GOAWAY frame to be sent to the remote peer."
And the relevant envoy logs from the client and server I see the following (there are a lot of requests like these):
ENVOY CLIENT LOG - Request: 7448123a-6637-4550-80d0-ea519c22cfa0
2024-01-27T07:18:14.861767964Z stdout F {"path":"/rpc.bookshop.v1.InventoryService/GetBookList","authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","downstream_local_address":"<IP_ADDRESS>:8081","upstream_host":"<IP_ADDRESS>:8081","start_time":"2024-01-27T07:18:09.524Z","downstream_peer_uri_san":null,"request_id":"7448123a-6637-4550-80d0-ea519c22cfa0","dd.trace_id":null,"upstream_transport_failure_reason":null,"upstream_service_time":null,"requested_server_name":null,"response_code_details":"codec_error:The_user_callback_function_failed","bytes_received":5,"route_name":"default","response_flags":"DPE","upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","duration":5333,"user_agent":"grpc-node-js/1.9.14","x_forwarded_for":null,"response_code":0,"x_envoy_external_address":null,"x_forwarded_proto":"http","connection_termination_details":null,"protocol":"HTTP/2","upstream_local_address":"<IP_ADDRESS>:56188","x_forwarded_client_cert":null,"downstream_remote_address":"<IP_ADDRESS>:34518","method":"POST","bytes_sent":0}
The client error seems to say there is a codec error.
ENVOY SERVER LOG - Request: 7448123a-6637-4550-80d0-ea519c22cfa0
2024-01-27T07:18:14.842076464Z stdout F {"upstream_service_time":"5175","x_envoy_external_address":null,"downstream_peer_uri_san":"spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","dd.trace_id":null,"protocol":"HTTP/2","route_name":"default","response_code_details":"via_upstream","x_forwarded_for":null,"response_flags":"-","duration":5227,"requested_server_name":"outbound_.8081_._.grpc-server.mcaruso-poc.svc.cluster.local","start_time":"2024-01-27T07:18:09.599Z","path":"/rpc.bookshop.v1.InventoryService/GetBookList","connection_termination_details":null,"response_code":200,"user_agent":"grpc-node-js/1.9.14","downstream_remote_address":"<IP_ADDRESS>:56188","bytes_received":5,"upstream_cluster":"inbound|8081||","downstream_local_address":"<IP_ADDRESS>:8081","method":"POST","upstream_transport_failure_reason":null,"upstream_local_address":"<IP_ADDRESS>:42447","x_forwarded_client_cert":"By=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-server;Hash=31094800fb20c490cceba2bd1ea0b3ed257f16934f4d2fc8858b32d79ad90e9e;Subject="";URI=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","bytes_sent":106,"upstream_host":"<IP_ADDRESS>:8081","request_id":"7448123a-6637-4550-80d0-ea519c22cfa0","authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","x_forwarded_proto":"http"}
ENVOY CLIENT LOG - Request 9664814f-e11d-43e4-ba61-df8293f63552
2024-01-27T07:18:14.825414881Z stdout F {"authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","protocol":"HTTP/2","path":"/rpc.bookshop.v1.InventoryService/GetBookList","dd.trace_id":null,"route_name":"default","response_code_details":"codec_error:The_user_callback_function_failed","bytes_received":5,"requested_server_name":null,"upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","start_time":"2024-01-27T07:18:09.576Z","downstream_peer_uri_san":null,"user_agent":"grpc-node-js/1.9.14","x_forwarded_proto":"http","bytes_sent":0,"x_envoy_external_address":null,"request_id":"9664814f-e11d-43e4-ba61-df8293f63552","x_forwarded_for":null,"upstream_transport_failure_reason":null,"method":"POST","response_flags":"DPE","duration":5243,"x_forwarded_client_cert":null,"response_code":0,"upstream_service_time":null,"upstream_host":"<IP_ADDRESS>:8081","downstream_local_address":"<IP_ADDRESS>:8081","downstream_remote_address":"<IP_ADDRESS>:34518","upstream_local_address":"<IP_ADDRESS>:56188","connection_termination_details":null}
ENVOY SERVER LOG - Request 9664814f-e11d-43e4-ba61-df8293f63552
2024-01-27T07:18:14.894895339Z stdout F {"connection_termination_details":null,"method":"POST","request_id":"9664814f-e11d-43e4-ba61-df8293f63552","upstream_service_time":null,"dd.trace_id":null,"upstream_host":"<IP_ADDRESS>:8081","bytes_received":5,"route_name":"default","requested_server_name":"outbound_.8081_._.grpc-server.mcaruso-poc.svc.cluster.local","x_forwarded_proto":"http","bytes_sent":0,"downstream_remote_address":"<IP_ADDRESS>:56188","upstream_transport_failure_reason":null,"start_time":"2024-01-27T07:18:09.641Z","response_code":0,"upstream_local_address":"<IP_ADDRESS>:42447","x_forwarded_for":null,"response_flags":"-","response_code_details":"http2.remote_reset","x_forwarded_client_cert":"By=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-server;Hash=31094800fb20c490cceba2bd1ea0b3ed257f16934f4d2fc8858b32d79ad90e9e;Subject="";URI=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","x_envoy_external_address":null,"downstream_peer_uri_san":"spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","downstream_local_address":"<IP_ADDRESS>:8081","upstream_cluster":"inbound|8081||","path":"/rpc.bookshop.v1.InventoryService/GetBookList","protocol":"HTTP/2","user_agent":"grpc-node-js/1.9.14","duration":5251,"authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081"}
Here we see in the server response that the remote client reseted the stream.
Version
istioctl version:

control plane version: 1.19.4
data plane version: 1.19.4

This also happens with any 1.19.X and 1.20.X version.

Additional Information
No response
Affected product area

[ ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Control Plane Revisions

Here I leave an example of code that you can use to replicate the issue, the client and server are pretty simple:
Here is an example of the grpc client javascript:
const VUS = 300
const ITERATIONS = 60
const inventoryServiceClient = new Client(URL)
const GetBookList = async (vu) => {
if (DEBUG) {
log.info(Sending request GetBookList: ${vu});
}
return new Promise((resolve, reject) => {
inventoryServiceClient.getBookList()
.then((response) => {
if (DEBUG) {
log.info(Success request GetBookList - Books: ${JSON.stringify(response)})
}
resolve()
})
.catch(err => {
log.withError(err).error("Failed to getBookList");
reject(err)
});
})
}
const TriggerIteration = async (iteration) => {
const vusArray = Array.from({ length: VUS }, (_, index) => index + 1);
log.info(`TriggerIteration: ${iteration}`)

return new Promise((resolve, reject) => {
    async.eachLimit(vusArray, VUS, GetBookList, (err) => {
        if (err) {
            log.withError(err).error('eachLimit ERROR');
            reject(err)
        } else {
            if (DEBUG) {
                log.info('eachLimit SUCCESS')
            }
            resolve()
        }
    });
})

}
const iterationsArray = Array.from({ length: ITERATIONS }, (_, index) => index + 1);
async.eachSeries(iterationsArray, TriggerIteration, (err) => {
if (err) {
log.withError(err).error('eachSeries ERROR');
return
}
log.info('eachSeries SUCCESS')
})
Here is an example of the grpc server method:
class InventoryService(bookshop_pb2_grpc.InventoryServiceServicer):
def GetBookList(self, request, context):
debug = os.getenv("DEBUG", 'False').lower() in ('true', '1', 't')
if debug:
print("GetBookList request before sleep")
time.sleep(5)
if debug:
print("GetBookList request after sleep")
return bookshop_pb2.GetBookListResponse(
books=[
bookshop_pb2.Book(title="The Hitchhiker's Guide to the Galaxy", author="Douglas Adams", page_count=42),
bookshop_pb2.Book(title="The Lord of the Rings", author="J.R.R. Tolkien", page_count=1234)
]
)

@murgatroid99 Here I have more logs:
The envoy side car in the client shows:
2024-01-27T07:18:14.818809214Z stderr F 2024-01-27T07:18:14.817867Z	debug	envoy http2 external/envoy/source/common/http/http2/codec_impl.cc:1208	[Tags: "ConnectionId":"7"] sent goaway code=2	thread=25
Which looks like is saying the client is sending the goway.
Then in the application I see these relevant lines:
2024-01-27T07:18:14.951034047Z stderr F HTTP2 18: Http2Session client: goaway 2 received [last stream id: 3999]
2024-01-27T07:18:14.951037006Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 connection closed by GOAWAY with code 2 and data undefined
2024-01-27T07:18:14.951041047Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | subchannel | (2) <IP_ADDRESS>:8081 READY -> IDLE
2024-01-27T07:18:14.951043381Z stderr F HTTP2 18: Http2Session client: marking session closed
2024-01-27T07:18:14.951045672Z stderr F HTTP2 18: Http2Session client: submitting goaway
2024-01-27T07:18:14.951047214Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | subchannel_refcount | (2) <IP_ADDRESS>:8081 refcount 2 -> 1
2024-01-27T07:18:14.951049006Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | pick_first | READY -> IDLE
2024-01-27T07:18:14.951050881Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | resolving_load_balancer | dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 READY -> IDLE
2024-01-27T07:18:14.951052756Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | connectivity_state | (1) dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 READY -> IDLE
2024-01-27T07:18:14.951054797Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | dns_resolver | resolution update delayed by "min time between resolutions" rate limit
2024-01-27T07:18:14.951056631Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | resolving_load_balancer | dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 IDLE -> CONNECTING
2024-01-27T07:18:14.951058131Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | connectivity_state | (1) dns:grpc-server.mcaruso-poc.svc.cluster.local:8081 IDLE -> CONNECTING
2024-01-27T07:18:14.951059756Z stderr F HTTP2 18: Http2Session client: destroying
2024-01-27T07:18:14.951061297Z stderr F HTTP2 18: Http2Session client: start closing/destroying Error [ERR_HTTP2_SESSION_ERROR]: Session closed with error code 2
2024-01-27T07:18:14.951063089Z stderr F     at Http2Session.onGoawayData (node:internal/http2/core:690:21)
2024-01-27T07:18:14.951064881Z stderr F     at Http2Session.callbackTrampoline (node:internal/async_hooks:130:17) {
2024-01-27T07:18:14.951066714Z stderr F   code: 'ERR_HTTP2_SESSION_ERROR'
2024-01-27T07:18:14.951068631Z stderr F }
2024-01-27T07:18:14.951070172Z stderr F HTTP2 18: Http2Stream 2025 [Http2Session client]: destroying stream

2024-01-27T07:18:14.952236297Z stderr F HTTP2 18: Http2Session client: finishSessionClose
2024-01-27T07:18:14.952238089Z stderr F D 2024-01-27T07:18:14.875Z | v1.9.14 18 | subchannel_call | [7012] Node error event: message=Session closed with error code 2 code=ERR_HTTP2_SESSION_ERROR errno=Unknown system error undefined syscall=undefined

2024-01-27T07:18:14.964811172Z stderr F D 2024-01-27T07:18:14.901Z | v1.9.14 18 | resolving_call | [4999] ended with status: code=13 details="Received RST_STREAM with code 2 triggered by internal client error: Session closed with error code 2"
2024-01-27T07:18:14.964813089Z stderr F HTTP2 18: Http2Session client: finishSessionClose socket end null Error [ERR_HTTP2_SESSION_ERROR]: Session closed with error code 2
2024-01-27T07:18:14.964815214Z stderr F     at Http2Session.onGoawayData (node:internal/http2/core:690:21)
2024-01-27T07:18:14.964816714Z stderr F     at Http2Session.callbackTrampoline (node:internal/async_hooks:130:17) {
2024-01-27T07:18:14.964818381Z stderr F   code: 'ERR_HTTP2_SESSION_ERROR'
2024-01-27T07:18:14.964820172Z stderr F }
2024-01-27T07:18:14.964821631Z stderr F D 2024-01-27T07:18:14.934Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 connection closed with error Session closed with error code 2
2024-01-27T07:18:14.964823172Z stderr F D 2024-01-27T07:18:14.934Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 session closed
2024-01-27T07:18:14.980607506Z stdout F 2024-01-27T07:18:14Z scuttle: Received signal 'child exited', passing to child
2024-01-27T07:18:14.980790964Z stdout F 2024-01-27T07:18:14Z scuttle: Received signal 'urgent I/O condition', ignoring

Where I see also here the line: "Http2Session client: submitting goaway" which looks like show the client is sending the goaway confirming the envoy side car log. This log line comes from:
https://github.com/nodejs/node/blob/f38a9a59182027493852478b8a3abd5d7d5d1726/lib/internal/http2/core.js#L853
Which ends up calling this function: https://github.com/nodejs/node/blob/f38a9a59182027493852478b8a3abd5d7d5d1726/lib/internal/http2/core.js#L1508
which clarifies "Submits a GOAWAY frame to be sent to the remote peer."
And the relevant envoy logs from the client and server I see the following:
ENVOY CLIENT REQUEST
2024-01-27T07:18:14.861767964Z stdout F {"path":"/rpc.bookshop.v1.InventoryService/GetBookList","authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","downstream_local_address":"<IP_ADDRESS>:8081","upstream_host":"<IP_ADDRESS>:8081","start_time":"2024-01-27T07:18:09.524Z","downstream_peer_uri_san":null,"request_id":"7448123a-6637-4550-80d0-ea519c22cfa0","dd.trace_id":null,"upstream_transport_failure_reason":null,"upstream_service_time":null,"requested_server_name":null,"response_code_details":"codec_error:The_user_callback_function_failed","bytes_received":5,"route_name":"default","response_flags":"DPE","upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","duration":5333,"user_agent":"grpc-node-js/1.9.14","x_forwarded_for":null,"response_code":0,"x_envoy_external_address":null,"x_forwarded_proto":"http","connection_termination_details":null,"protocol":"HTTP/2","upstream_local_address":"<IP_ADDRESS>:56188","x_forwarded_client_cert":null,"downstream_remote_address":"<IP_ADDRESS>:34518","method":"POST","bytes_sent":0}
The client error seems to say there is a codec error.
ENVOY SERVER RESPONSE
2024-01-27T07:18:14.842076464Z stdout F {"upstream_service_time":"5175","x_envoy_external_address":null,"downstream_peer_uri_san":"spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","dd.trace_id":null,"protocol":"HTTP/2","route_name":"default","response_code_details":"via_upstream","x_forwarded_for":null,"response_flags":"-","duration":5227,"requested_server_name":"outbound_.8081_._.grpc-server.mcaruso-poc.svc.cluster.local","start_time":"2024-01-27T07:18:09.599Z","path":"/rpc.bookshop.v1.InventoryService/GetBookList","connection_termination_details":null,"response_code":200,"user_agent":"grpc-node-js/1.9.14","downstream_remote_address":"<IP_ADDRESS>:56188","bytes_received":5,"upstream_cluster":"inbound|8081||","downstream_local_address":"<IP_ADDRESS>:8081","method":"POST","upstream_transport_failure_reason":null,"upstream_local_address":"<IP_ADDRESS>:42447","x_forwarded_client_cert":"By=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-server;Hash=31094800fb20c490cceba2bd1ea0b3ed257f16934f4d2fc8858b32d79ad90e9e;Subject="";URI=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","bytes_sent":106,"upstream_host":"<IP_ADDRESS>:8081","request_id":"7448123a-6637-4550-80d0-ea519c22cfa0","authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","x_forwarded_proto":"http"}
ENVOY CLIENT REQUEST
2024-01-27T07:18:14.825414881Z stdout F {"authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","protocol":"HTTP/2","path":"/rpc.bookshop.v1.InventoryService/GetBookList","dd.trace_id":null,"route_name":"default","response_code_details":"codec_error:The_user_callback_function_failed","bytes_received":5,"requested_server_name":null,"upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","start_time":"2024-01-27T07:18:09.576Z","downstream_peer_uri_san":null,"user_agent":"grpc-node-js/1.9.14","x_forwarded_proto":"http","bytes_sent":0,"x_envoy_external_address":null,"request_id":"9664814f-e11d-43e4-ba61-df8293f63552","x_forwarded_for":null,"upstream_transport_failure_reason":null,"method":"POST","response_flags":"DPE","duration":5243,"x_forwarded_client_cert":null,"response_code":0,"upstream_service_time":null,"upstream_host":"<IP_ADDRESS>:8081","downstream_local_address":"<IP_ADDRESS>:8081","downstream_remote_address":"<IP_ADDRESS>:34518","upstream_local_address":"<IP_ADDRESS>:56188","connection_termination_details":null}
ENVOY SERVER RESPONSE
2024-01-27T07:18:14.894895339Z stdout F {"connection_termination_details":null,"method":"POST","request_id":"9664814f-e11d-43e4-ba61-df8293f63552","upstream_service_time":null,"dd.trace_id":null,"upstream_host":"<IP_ADDRESS>:8081","bytes_received":5,"route_name":"default","requested_server_name":"outbound_.8081_._.grpc-server.mcaruso-poc.svc.cluster.local","x_forwarded_proto":"http","bytes_sent":0,"downstream_remote_address":"<IP_ADDRESS>:56188","upstream_transport_failure_reason":null,"start_time":"2024-01-27T07:18:09.641Z","response_code":0,"upstream_local_address":"<IP_ADDRESS>:42447","x_forwarded_for":null,"response_flags":"-","response_code_details":"http2.remote_reset","x_forwarded_client_cert":"By=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-server;Hash=31094800fb20c490cceba2bd1ea0b3ed257f16934f4d2fc8858b32d79ad90e9e;Subject="";URI=spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","x_envoy_external_address":null,"downstream_peer_uri_san":"spiffe://cluster.local/ns/mcaruso-poc/sa/grpc-client","downstream_local_address":"<IP_ADDRESS>:8081","upstream_cluster":"inbound|8081||","path":"/rpc.bookshop.v1.InventoryService/GetBookList","protocol":"HTTP/2","user_agent":"grpc-node-js/1.9.14","duration":5251,"authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081"}
Here we see in the server response that the remote client reseted the stream.

The very first line of the client log there is "Http2Session client: goaway 2 received". The client received a goaway before it sent one, so the primary cause of this connection closing is the goaway sent to the client by envoy.

@murgatroid99 That is not the issue, we can see there that the envoy in the client side is generating the goaway:
2024-01-27T07:18:14.817867Z	debug	envoy http2 external/envoy/source/common/http/http2/codec_impl.cc:1208	[Tags: "ConnectionId":"7"] sent goaway code=2	thread=25
Which makes me think the envoy side car in the client side is telling the client application to goaway.
 I had thousands of requests with the same error in the envoy client side: "response_code_details":"codec_error:The_user_callback_function_failed",,"response_flags":"DPE","
`
For me is pretty clear that there is a codec error. The envoy flag DPE means "The downstream request had an HTTP protocol error", and because this is the client envoy, the downstream is the nodejs application client container using the grpc-js having the protocol error.
The goaway is the end result of this, becasue there are several errors, then the client it proceeds to reset all the streams. And we now is the client resetting the streams, because the envoy server side car shows :"response_code_details":"http2.remote_reset",
We need to focus on the codec error, not in the goaway.

The timeline order is the following,
First I see a lot of requests like this in envoy client side where they are ok:
2024-01-27T07:18:14.817417547Z stdout F {"bytes_received":5,"connection_termination_details":null,"downstream_peer_uri_san":null,"upstream_local_address":"<IP_ADDRESS>:56188","method":"POST","x_envoy_external_address":null,"dd.trace_id":null,"downstream_local_address":"<IP_ADDRESS>:8081","upstream_host":"<IP_ADDRESS>:8081","requested_server_name":null,"start_time":"2024-01-27T07:18:09.537Z","path":"/rpc.bookshop.v1.InventoryService/GetBookList","upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","request_id":"4d2ba9ed-e3dd-4cd2-8ef4-87369be87a99","protocol":"HTTP/2","downstream_remote_address":"<IP_ADDRESS>:34518","x_forwarded_client_cert":null,"route_name":"default","user_agent":"grpc-node-js/1.9.14","x_forwarded_proto":"http","response_code":200,"upstream_service_time":"5215","bytes_sent":106,"duration":5269,"authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","response_flags":"-","x_forwarded_for":null,"upstream_transport_failure_reason":null,"response_code_details":"via_upstream"}
Then, at some point after a lot of successfull requests like the one above, I see the goaway is sent by the envoy client side:
2024-01-27T07:18:14.817867Z debug envoy http2 external/envoy/source/common/http/http2/codec_impl.cc:1208 [Tags: "ConnectionId":"7"] sent goaway code=2 thread=25
And after that I see a lot of requests in the envoy client side with the codec error like these:
2024-01-27T07:18:14.825341714Z stdout F {"bytes_sent":0,"response_flags":"DPE","user_agent":"grpc-node-js/1.9.14","upstream_service_time":null,"protocol":"HTTP/2","authority":"grpc-server.mcaruso-poc.svc.cluster.local:8081","duration":5240,"upstream_host":"<IP_ADDRESS>:8081","x_envoy_external_address":null,"method":"POST","downstream_peer_uri_san":null,"bytes_received":5,"x_forwarded_for":null,"dd.trace_id":null,"downstream_local_address":"<IP_ADDRESS>:8081","start_time":"2024-01-27T07:18:09.577Z","upstream_local_address":"<IP_ADDRESS>:56188","requested_server_name":null,"connection_termination_details":null,"response_code_details":"codec_error:The_user_callback_function_failed","upstream_cluster":"outbound|8081||grpc-server.mcaruso-poc.svc.cluster.local","downstream_remote_address":"<IP_ADDRESS>:34518","upstream_transport_failure_reason":null,"path":"/rpc.bookshop.v1.InventoryService/GetBookList","response_code":0,"request_id":"23cf4d24-9caa-4d24-b3ca-a136b81ae46d","route_name":"default","x_forwarded_proto":"http","x_forwarded_client_cert":null}
And after that, the client app receives the goaway, and closes the connection (here the app container logs):
2024-01-27T07:18:14.951034047Z stderr F HTTP2 18: Http2Session client: goaway 2 received [last stream id: 3999]
2024-01-27T07:18:14.951037006Z stderr F D 2024-01-27T07:18:14.866Z | v1.9.14 18 | transport | (3) <IP_ADDRESS>:8081 connection closed by GOAWAY with code 2 and data undefined
2024-01-27T07:18:14.951043381Z stderr F HTTP2 18: Http2Session client: marking session closed
2024-01-27T07:18:14.951045672Z stderr F HTTP2 18: Http2Session client: submitting goaway
The server logs doesnt show anything about a goaway, this all happens inside the pod of the client, between the envoy and the app container

We're seeing similar behavior with '@grpc-js': v1.9.13 clients calling golang services, originating with roughly the same Istio version upgrade (unfortunately, we went from v1.17.5 to v1.20.0, so we're not really narrowing scope).
However, I observe this:

Isto 1.19.3 release notes indicate that release resolves CVE-2023-44487.
The envoy fix for that CVE (envoyproxy/envoy#30055) includes conditions the proxy will send GOAWAY to clients (i.e. if it gets too many premature RST_STREAM in a time period).

So when I read this...

The server logs doesnt show anything about a goaway, this all happens inside the pod of the client, between the envoy and the app container.

... I'm theorizing that this may be the source. Is the grpc-js client library sending RST_STREAM in a pattern that Envoy considers improper?
From that Envoy PR:

Premature means the RST_STREAM arrived before response headers were sent and than the stream was alive for short period of time.

There appear to be several settings we can use in envoy to control this behavior, so I'm going to try to see if I can verify and resolve our issues by testing them. I'll report back if I can/can't find anything interesting.
In the meantime: I'd love input if this is a false trail, and also discussion about what should be done if this is the case (is envoy too sensitive? is grpc-js actually doing something incorrect? etc).

@abscondment I think if we find the problem causing this issue, it must work in the same way than istio 1.18.5, which means not throwing the codec error.

Hi folks: we've been able to resolve our issue in two different ways:

Upgrade to grpc-js v1.10.4 (which includes grpc/grpc-node#2695)
Even with grpc-js before that fix enable oghttp2 in the envoy proxy (see https://github.com/envoyproxy/envoy/issues/30882#issuecomment-1998582512).

My belief is that the grpc-js bug fixed in (1) is triggering nghttp2's internal RST_STREAM DDoS rate limiter, and oghttp2 handles the buggy client better.
I don't think (2) is recommended. envoy's oghttp2 use has some open bugs, especially in the 1.28 release branch.


grid records update hiding data

If I set the grid records to a new dataset with more data, the data past the last record is not visible while scrolling.
I've noticed that if I reset the total number of records it fixes the issue
eg.
w2ui.grid.records = newDataSet;
w2ui.grid.total = w2ui.grid.records.length;  // this line fixes it

I am glad you found a solution. The reason total is there is because when the data source is remote, it will only send first set of records, but I need to know how many more on the server.

Thanks Vitaly
I've been using your grid on a couple of small projects and I've got to say
it's an awesome piece of work.  It's very easy to configure and offers a
lot of features while staying really responsive.  As a programmer it
inspires me to write better code.  Thanks for everything you've done.
Chris
On Sun, Jul 19, 2015, 7:34 PM Vitali Malinouski<EMAIL_ADDRESS>wrote:

I am glad you found a solution. The reason total is there is because when
the data source is remote, it will only send first set of records, but I
need to know how many more on the server.
—
Reply to this email directly or view it on GitHub
https://github.com/vitmalina/w2ui/issues/991#issuecomment-122722541.

Thanks for the positive feedback.


BUG: Out of bounds memory access in github.com/open-policy-agent/opa

Fix the Vulnerability reported via github

To fix the vulnerability requires:
github.com/open-policy-agent/opa v0.40.0


error about number works

hi, i meet some errors when increase the num_works of dataloaders from 2 to 4:

then i meet some errors like:

could you please tell me that did you miss the same errors ? or provide some advices ~ thanks a lot .

same problem.

excuse me, can you run it successfully with number of worker 0?

Excuse me, have u met such problem

 Excuse me, have u met such problem

sorry, i didn't meet  such a problem


[RFC]: Add C implementation for @stdlib/math/base/special/fibonacci

Description
This RFC proposes adding C implementation for @stdlib/math/base/special/fibonacci.
Related Issues
Related issues #649.
Questions
No.
Other
No.
Checklist

[X] I have read and understood the Code of Conduct.
[X] Searched for existing issues and pull requests.
[X] The issue name begins with RFC:.

I would like to be assigned for this.

As the C implementation of @stdlib/math/base/special/fibonacci already exists, we can close this issue. I had confused this for some other package.


connection to rasa server failed

i build my custom web socket connector and i'm trying to connect with this interface .
when i run rasa everything works fine but when i run the interface server( it generate a url Local:   http://localhost:8081/ )  this show up in my console :

Vue-Socket.io: Received connection string
Vue-Socket.io: Vuex adapter enabled
Vue-Socket.io: Vuex socket mutations enabled
Vue-Socket.io: Vuex socket actions enabled
Vue-Socket.io: Vue-Socket.io plugin enabled
Vue-Socket.io: #connect subscribe, component: App
Vue-Socket.io: #disconnect subscribe, component: App
Vue-Socket.io: #session_confirm subscribe, component: App
Vue-Socket.io: #bot_uttered subscribe, component: App
GET http://localhost:5005/socket.io/?EIO=3&transport=polling&t=NeuIizl 400 (Bad Request)

i never worked with vuejs before so really i cant  find the bug

Thanks for raising this issue, @kalkbrennerei will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗

Thanks for raising this issue, @kalkbrennerei will get back to you about it soon
Please also check out the docs and the forum in case your issue was raised there too

i build my custom web socket connector and i'm trying to connect with this interface .
when i run rasa everything works fine but when i run the interface server( it generate a url Local: http://localhost:8081/ ) this show up in my console :

Vue-Socket.io: Received connection string
Vue-Socket.io: Vuex adapter enabled
Vue-Socket.io: Vuex socket mutations enabled
Vue-Socket.io: Vuex socket actions enabled
Vue-Socket.io: Vue-Socket.io plugin enabled
Vue-Socket.io: #connect subscribe, component: App
Vue-Socket.io: #disconnect subscribe, component: App
Vue-Socket.io: #session_confirm subscribe, component: App
Vue-Socket.io: #bot_uttered subscribe, component: App
GET http://localhost:5005/socket.io/?EIO=3&transport=polling&t=NeuIizl 400 (Bad Request)

i never worked with vuejs before so really i cant find the bug

i solved it , the problem is with the inti method in my input channel , i had to add jwt_method: Optional[Text] = "HS256" argument . now another shows :

http://localhost:8081 is not an accepted origin. (further occurrences of this error will be logged with level INFO)

i face the same problem

i have the same problem, if i try to get urls with EIO=4 it works, but i don't know how to change that parameter in the code


Add Universidad del Cauca

URL: http://www.unicauca.edu.co/versionP/
URL Programa Ingeniería de Sistemas: https://www.unicauca.edu.co/versionP/oferta-academica/programas-de-pregrado/ingenieria-de-sistemas
Dominio Universidad del Cauca: http://talos.unicauca.edu.co/derecho/Acerca de la Facultad/Téminos y Políticas de publicación en el Portal Web Institucional

@eduardoramirez7 lib/domains/unicauca.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@eduardoramirez7 lib/domains/unicauca.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@eduardoramirez7 lib/domains/unicauca.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@eduardoramirez7 lib/domains/unicauca.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.


[BUG Fix] route53 - list_tags_for_resources: Resolve when a single ID is provided

Fixes two bugs.

When list_tags_for_resource is passed only a single ID, the request is a string and not a list
The regex for deriving resource_type was always None

@bblommers Another bug fix for the route53 endpoint that slipped through :(


ci: fix exception in code snippet, add check to CI

closes #529

What type of PR is this? (check all applicable)

[ ] 💾 Refactor
[ ] ✨ Feature
[ ] 🐛 Bug Fix
[ ] 🔧 Optimization
[ ] 📝 Documentation
[ ] ✅ Test
[ ] 🐳 Other

Related issues

Related issue #
Closes #

Checklist

[ ] Code follows style guide (ruff)
[ ] Tests added
[ ] Documented the changes

If you have comments or can explain your changes, please do so below.

@FBruzzesi hey, this works! CI failed, then I fixed the snippet, and now it passes
merging then to fix the docs


Make sure submodule nonsense is in docs

Explain how to init submodule and make sure it gets built - document what has to be done for #6

don't need it, found python tag library


There is a way to use it from CLI?

I appreciate your work, there is a way to use it from command line?

There isn't but if someone wants to submit a PR, happy to include it.

okay i will try

I'd appreciate it if it didn't refactor the code too much (harder to review PR). There is a eval script that does run the generate code functionality in bulk from CLI: https://github.com/abi/screenshot-to-code/blob/main/Evaluation.md

I have add the STACK var but (i have defined at start an OpenAPi on .env fie):
python3 run_evals.py
Traceback (most recent call last):
File "/home/glande/Desktop/screenshot-to-code/backend/run_evals.py", line 43, in
asyncio.run(main())
File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home//Desktop/screenshot-to-code/backend/run_evals.py", line 31, in main
results = await asyncio.gather(*tasks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home//Desktop/screenshot-to-code/backend/evals/core.py", line 21, in generate_code_core
if not anthropic_api_key:
^^^^^^^^^^^^^^^^^
NameError: name 'anthropic_api_key' is not defined

You probably just need to set that env var

https://v0.dev/
It free from https://vercel.com/


How to dynamically change the width of the graph?

Hi,
I've been looking at examples of setting the graph width according to the parent or window, but I can't seem to get it to work using React hooks. My code currently looks like this:
const GraphComponent = ({ data }) => {
  const parentRef = useRef(null)
  const [graphWidth, setGraphWidth] = useState(2000) // high value intentional so that the first render looks bad
  useEffect(() => {
    setTimeout(() => {
      if (parentRef.current && parentRef.current.offsetWidth > 0) {
        setGraphWidth(parentRef.current.offsetWidth)
      }
    }, 2000) // 2 seconds to be able to see it
  }, [parentRef.current])

  const graphConfig = { width: graphWidth }

  return (

      <Graph id='graph-id' data={data} config={graphConfig} />

  )
}

The first render does in fact happen with 2000px width. And after 2 seconds, the svg element becomes the width of the parent div. However, all the nodes and links inside remain rendered in the original place. What do I need to do to make the svg content rerender after changing the config?
Thank you very much!

Something that ESLint yelled at me about when I tried what you have:

React Hook useEffect has an unnecessary dependency: 'parentRef.current'. Either exclude it or remove the dependency array. Mutable values like 'parentRef.current' aren't valid dependencies because mutating them doesn't re-render the component.

I also tried this out with useLayoutEffect but that didn't get me any further.
I ended up getting this working with a combo of  useCallback and ResizeObserver. Check it out:
import React, { useCallback, useState } from "react";

const GraphComponent = ({ data }) => {
  const [graphSize, setGraphSize] = useState({ width: 2000, height: 2000 });

  const parentRef = useCallback((containerNode) => {
    if (containerNode) {
      // set the graph size to the size of the parent when the component mounts
      setGraphSize({
        width: containerNode.offsetWidth,
        height: containerNode.offsetHeight,
      });

      const resizeObserver = new ResizeObserver((entries) => {
        // this is only ever a single element list of the one parent container we're observing
        for (let entry of entries) {
          if (entry.contentRect) {
            setGraphSize({
              width: entry.contentRect.width,
              height: entry.contentRect.height,
            });
          }
        }
      });

      resizeObserver.observe(containerNode);
    }
  }, []);

  const graphConfig = {
    // ... whatever config you have
    ...graphSize
  };

  return (

      <Graph
        id="graph-id"
        data={data}
        config={graphConfig}
      />

  );
};

ResizeObserver is supported by most browsers these days but if you need to support IE11 you'll need a polyfill.

Hey @tigermarques passing a new object reference of data or graphConfig should suffice to trigger a re-render of the whole thing. Check this example where I trigger a re-render when clicking on the Fullscreen button.


control plane installation removes gateway resources before they get reapplied

Bug Description
we have a custom ingressgateway that whenever we reapply the control plane operator spec has its resources deleted. here is the controlplane operator spec:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  revision: 1-11-5
  values:
    global:
      proxy:
        autoInject: disabled
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "while [ $(netstat -plnt | grep tcp | grep -v envoy | wc -l) -ne 0 ]; do sleep 1; done"]
        includeIPRanges: "<IP_ADDRESS>/16"
    telemetry:
      v2:
        enabled: true
  meshConfig:
    accessLogFile: /dev/stdout
    accessLogFormat: "[%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %RESPONSE_CODE_DETAILS% %CONNECTION_TERMINATION_DETAILS% \"%UPSTREAM_TRANSPORT_FAILURE_REASON%\" %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\" %UPSTREAM_CLUSTER% %UPSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_LOCAL_ADDRESS% %DOWNSTREAM_REMOTE_ADDRESS% %REQUESTED_SERVER_NAME% %ROUTE_NAME% %UPSTREAM_REQUEST_ATTEMPT_COUNT% %LOCAL_REPLY_BODY% %FILTER_CHAIN_NAME%"
    accessLogEncoding: TEXT

    protocolDetectionTimeout: 0s
    defaultConfig:
      proxyStatsMatcher:
        inclusionRegexps:
          - .*cluster_manager.*
          - .*listener_manager.*
          - .*cluster.*tcp_stats.*
          - .*cluster.*ssl.*
          - .*tls_inspector.*
          - .*http_inspector.*
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: false
      - name: private-ingressgateway
        namespace: foo
        enabled: false
    pilot:
      k8s:
        overlays:
        - kind: Service
          name: istiod-1-11-5
          patches:
          - path: metadata.annotations
            value:
              # annotations needed found here:
              # https://www.datadoghq.com/blog/how-to-monitor-istiod/#configure-the-datadog-agents-to-track-your-istio-deployment
              ad.datadoghq.com/endpoints.check_names: '["istio"]'
              ad.datadoghq.com/endpoints.init_configs: '[{}]'
              ad.datadoghq.com/endpoints.instances: |
                  [
                    {
                      "istiod_endpoint": "http://%%host%%:15014/metrics",
                      "send_histograms_buckets": true
                    }
                  ]
        env:
        - name: PILOT_SIDECAR_USE_REMOTE_ADDRESS
          value: 'true'
        replicaCount: 2
        hpaSpec:
          minReplicas: 2

and here is the gateways operator spec
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: istio-1-11-5-gateways # REQUIRED
spec:
  profile: empty # REQUIRED
  revision: 1-11-5
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
      - name: private-ingressgateway
        namespace: foo
        enabled: true
        k8s:
          podAnnotations:
            proxy.istio.io/config: |
              discoveryAddress: istiod-1-11-5.istio-system.svc:15012

when running istioctl install we see output like this
...
- Processing resources for Istio core.
✔ Istio core installed
- Processing resources for Istiod.
✔ Istiod installed
Removed HorizontalPodAutoscaler:foo:private-ingressgateway.
Removed HorizontalPodAutoscaler:istio-system:istio-ingressgateway.
Removed PodDisruptionBudget:foo:private-ingressgateway.
Removed PodDisruptionBudget:istio-system:istio-ingressgateway.
Removed Deployment:foo:private-ingressgateway.
Removed Deployment:istio-system:istio-ingressgateway.
Removed Service:foo:private-ingressgateway.
Removed Service:istio-system:istio-ingressgateway.
Removed ServiceAccount:foo:private-ingressgateway-service-account.
Removed ServiceAccount:istio-system:istio-ingressgateway-service-account.
Removed RoleBinding:foo:private-ingressgateway-sds.
Removed RoleBinding:istio-system:istio-ingressgateway-sds.
Removed Role:foo:private-ingressgateway-sds.
Removed Role:istio-system:istio-ingressgateway-sds.
- Pruning removed resources
✔ Installation complete
...

a new elb is created each time for the private-ingressgateway and I was wondering why the service is deleted when the control plane is applied? I've tried this with and without the explicit gateway enabled: false in the control plane spec but not really sure what's happening.
Version
$ istioctl version
client version: 1.10.6
istiod version: 1.10.6
istiod version: 1.10.6
istiod version: 1.11.5
istiod version: 1.11.5
data plane version: 1.10.6 (6 proxies), 1.11.5 (24 proxies)

$ kubectl version --short
Client Version: v1.20.7
Server Version: v1.20.7-eks-d88609

Additional Information
No response

You might try also adding metadata.name to the control plane IstioOperator? not certain that fixes it though

thanks @howardjohn, interestingly that seemed to fix it and now the custom ingressgateway resources are no longer being deleted

I have faced exactly the same issue (also affects upgrade). Solution kind of works. @howardjohn  if this is the way to go, could this be documented in https://istio.io/latest/docs/setup/additional-setup/gateway/ ?
@chriskim06 not sure, but maybe you could reopen the issue to track this documentation improvement thing or should I fill a new bug?


First Pass

I've been letting this sit for awhile as I've been pretty busy with other stuff, but noticed that even in its unpublished state there are a number of stars and followers, so I think I really need to do something with this.
In the interim, it appears that @sre has published https://github.com/sre/rust-gpiochip / https://crates.io/crates/gpiochip with similar or the same goals, so it may be best to combine forces or settle on a single implementation and look to move it towards being incorporated as part of the rust-embedded org more formally.  With this, we can also push people toward the new implementation(s) if they know they don't need to target old kernels.  A lot of this I punched out in a night to prepare to talk about it a bit at a meetup.
Trying to have a single API to target both is probably a waste of time as they have significant differences.
This is still rough form and this PR is meant mostly to get something (still changing) in development and I'm mostly only expecting myself to review (although I would appreciate feedback from others).  If there are specific goals that individuals would like to see from the future gpio library for linux, this wouldn't be a bad time to collect that feedback either.
CC @nastevens, @japaric, @sre

I've tried it out with a couple BeagleBone varieties. Looks pretty good so far.


More operations

Codecov Report

Merging #15 (3c3fc10) into main (b577622) will increase coverage by 0.12%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #15      +/-   ##
==========================================
+ Coverage   81.46%   81.59%   +0.12%
==========================================
  Files           2        2
  Lines         286      288       +2
==========================================
+ Hits          233      235       +2
  Misses         53       53

Impacted Files
Coverage Δ

..._and_things_with_membranes/_tests/test_function.py
100.00% <ø> (ø)

...egment_blobs_and_things_with_membranes/__init__.py
79.53% <100.00%> (+0.15%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Tests pass locally:
(bio_39) C:\structure\code\napari-cupy-image-processing>pytest
================================================= test session starts =================================================
platform win32 -- Python 3.9.13, pytest-7.1.2, pluggy-1.0.0
PyQt5 5.15.4 -- Qt runtime 5.15.2 -- Qt compiled 5.15.2
rootdir: C:\structure\code\napari-cupy-image-processing
plugins: anyio-3.5.0, napari-0.4.15, napari-plugin-engine-0.2.0, npe2-0.5.1, qt-4.1.0
collected 2 items

napari_cupy_image_processing\_tests\test_function.py ..                                                          [100%]

================================================== warnings summary ===================================================
..\..\..\Users\rober\miniconda3\envs\bio_39\lib\site-packages\_pytest\assertion\rewrite.py:168
  C:\Users\rober\miniconda3\envs\bio_39\lib\site-packages\_pytest\assertion\rewrite.py:168: DeprecationWarning: Please import PackageMetadata from 'npe2' or from 'npe2.manifest'
    exec(co, module.__dict__)

..\..\..\Users\rober\miniconda3\envs\bio_39\lib\site-packages\vispy\app\backends\_qt.py:93
  C:\Users\rober\miniconda3\envs\bio_39\lib\site-packages\vispy\app\backends\_qt.py:93: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    if LooseVersion(QT_VERSION_STR) >= '5.4.0':

..\..\..\Users\rober\AppData\Roaming\Python\Python39\site-packages\setuptools\_distutils\version.py:346
  C:\Users\rober\AppData\Roaming\Python\Python39\site-packages\setuptools\_distutils\version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
    other = LooseVersion(other)

napari_cupy_image_processing/_tests/test_function.py::test_measurements
  C:\structure\code\napari-cupy-image-processing\napari_cupy_image_processing\_cupy_image_processing.py:435: DeprecationWarning: napari-cupy-image-processing measurements() is deprecated. Consider switching to napari-simpleitk-image-processing label_statistics()
    warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================ 2 passed, 4 warnings in 3.68s ============================================


Review prerequisites for free tier feature - AAD SSPR

review prerequisites on licensing.  According to Azure AD Pricing page, SSPR for cloud-only users is not included in free tier.  Only self service password change is included:
https://azure.microsoft.com/en-us/pricing/details/active-directory/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cacc043b-aeec-5deb-e60f-b11e6381b0ba
Version Independent ID: aa2be87c-296f-b3ff-9549-4407c46746d4
Content: Enable Azure Active Directory self-service password reset
Content Source: articles/active-directory/authentication/tutorial-enable-sspr.md
Service: active-directory
Sub-service: authentication
GitHub Login: @Justinha
Microsoft Alias: justinha

@jbrysonMSFT
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@jbrysonMSFT
Thanks for your feedback! I have assigned this issue to the author who will investigate and update as appropriate.

@jbrysonMSFT I added that password change is supported in the Free tier, but password reset is not.

#please-close


Typo in spouse tutorial

I was reading the spouse tutorial. Below is the extract function that has been defined:
def extract(
        doc_id         = "text",
        sentence_index = "int",
        tokens         = "text[]",
        ner_tags       = "text[]",
    ):
    """
    Finds phrases that are continuous words tagged with PERSON.
    """
    num_tokens = len(ner_tags)
    # find all first indexes of series of tokens tagged as PERSON
    first_indexes = (i for i in xrange(num_tokens) if ner_tags[i] == "PERSON" and (i == 0 or ner_tags[i-1] != "PERSON"))
    for begin_index in first_indexes:
        # find the end of the PERSON phrase (consecutive tokens tagged as PERSON)
        end_index = begin_index + 1
        while end_index < num_tokens and ner_tags[end_index] == "PERSON":
            end_index += 1
        end_index -= 1
        # generate a mention identifier
        mention_id = "%s_%d_%d_%d" % (doc_id, sentence_index, begin_index, end_index)
        mention_text = " ".join(map(lambda i: tokens[i], xrange(begin_index, end_index + 1)))
        # Output a tuple for each PERSON phrase
        yield [
            mention_id,
            mention_text,
            doc_id,
            sentence_index,
            begin_index,
            end_index,
        ]

The comment above the yield statement is a little misleading - I think the comment should be amended to reflect that actual type of the output value, which is a list.

Thanks for the suggestion, but it'd be better if you could create a pull request.  I won't fix this as tuple vs. list isn't very important in this context.


Make this project a CDS Plugin Package

Unfortunately the documentation at cds-plugins is not yet released. But the GitHub repository for Capire is now online. So cds-plugins.md can be checked there. We also can follow how the cds-plugin-ui5 implement it.

This one was also in my mind, great idea. I just need to go through the documentation and see what is required to change the project to a real cds-plugin.

New major release is now a real CAP plugin thanks to @schiwekM! 🥇


layout does not looks good in arabic

Hey @magdy0000,
Can you provide screenshots of how the layout looks when text is in Arabic?
Can you also explain what the issue is, and explain how it should look in your opinion in Arabic?

 If so, please provide answers to the questions above.


Update atom-beta to 1.18.0-beta1

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Thank you for the contribution. It was merged directly as e89a3b05d56a19ea2144f75aeaa9a1c2c2a64150. Your contribution is still credited to you.


1.1.76 - OAuth 2 token generation no longer works

Trying to generate an OAuth 2 token stopped working in 1.1.76
My server no longer sees a request providing the Client Credentials
Note, I am trying to generate using the following fields/settings:

Client Credential
As Request Body

Note: I have downgraded to 1.1.75 and it works great. Upgraded to 1.1.76 again and it fails again. Downgraded again and it works great again. Seems to be 1.1.76

Works for me


plugin-webpack-filter; Documentation; Importing into ESM

The official documentation guide: Output webpack stats with webpack-stats-plugin
In the documentation the example given for importing the package is in CommonJs;
const filterWebpackStats = require('@bundle-stats/plugin-webpack-filter').default;
Importing the module using ESM syntax would then use the following;
import filterWebpackStats from '@bundle-stats/plugin-webpack-filter'
However, this then throws a run-time error from the following line in the documentation (error: 'not a function');
const filteredSource = filterWebpackStats(webpackStats);
The run-time error can be solved by using filterWebpackStats.default(webpackStats);;
However, this throws up a type-error:
Property 'default' does not exist on type '(source: StatsCompilation, options?: BundleStatsOptions | undefined) => WebpackStatsFiltered'
I was able to solve this by adding a declaration to my .d.ts file;

Basically, I'm looking to understand what is going on here and whether this is an error in the type definitions for the library, or whether it is intentional - from my research it seems like there is a history involving CommonJS, ESM, and how libraries have been constructed along the timeline of developments.

@SStranks thank you for opening an issue!
Can you please share your version of Node and whether the module you are trying to import is ESM or Typescript?
I was able to replicate your error by using an import inside an esm config file (.mjs). I believe that's a problem with the way we publish the ESM modules, I will investigate and get back to you with an update.

I'm using Node v.20.9.0. I was importing the package into my webpack config file (.js) - my package.json has "type" set to "module".
It's a TypeScript (v5.3.3) project; the webpack configs are using // @ts-check to allow TypeScript to check the configuration files.
I'm interested in understanding all this, so any insights or information on the whole thing would be appreciated :)

@SStranks i released a fix for it in v4.14.2-beta.2.
The current stable ships commonjs and esm in ".js" files and only sets "module" property in package.json. Bundlers are able to load the correct format, but node is not able to load the esm format:

if the package.json exports field is not set
when type is not set to "module" and if the file extension is ".js". when type is set to "module", the esm is loaded correctly, but commonjs format is not loaded correctly

More resources:

https://2ality.com/2019/10/hybrid-npm-packages.html
https://webreflection.medium.com/a-nodejs-dual-module-deep-dive-8f94ff56210e
--

We have an update that publishes correct mjs/cjs files, but because the change is breaking, it will wait until the next major. Until then, the workaround adds an exports field and adds type-module only for the esm folder.


Ansible Role for Podman

I am now maintenance an Ansible Role for Podman installation (https://github.com/alvistack/ansible-role-podman), by building the source code manually and install for following OS:

Ubuntu 16.04/18.04/19.10
CentOS/RHEL 7/8
openSUSE 15.1

Could we also introduce this as an alternative installation method, e.g. under https://github.com/containers/podman.io/blob/master/getting-started/installation.md#installing-development-versions-of-podman section?

I have no opposition. @TomSweeneyRedHat Thoughts?
PRs to podman.io are welcome - github.com/containers/podman.io

@hswong3i I'd very much welcome a PR for that.  Thanks!

@hswong3i Have you been able to work on this?

Sorry for my late update, PR now goes to https://github.com/containers/podman.io/pull/242, for installing nix-based static binary by using Ansible Role ;-)

Since this no longer applies to this repo, closing.


Add a created date to snapshots to avoid crash

The indicator list on the family detail page will crash if there is a snapshot without a date in it; this fixes that issue.

are you sure?
   if(p.getEstimatedDate()!=null) {                     String when = SimpleDateFormat.getDateInstance().format(p.getEstimatedDate());                     mWhen.setText(when);                 }


Provide a GUI for the support staff

Up to now we only provide a graphical interface for the user, that likes to share his desktop. Reworking the application with Java 10 (see #9) also allows us to create a graphical interface for the support staff. This makes it a lot easier for the support side to open a support session. No external requirements (separate VNC viewer, manual SSH tunneling, etc.) are needed.

fixed with the release of version 1.0.0


Merge MarketData and SingleCalculationMarketData

Rename DefaultSingleCalculationMarketData to SingleCalculationMarketData

That's everything. The RuntimeException comment is repeated quite a few times.


networking: libsnnet: Make API more explicit and get existing VNIC device by name

The API SetHardwareAddr() was implicitely creating a Vnic peer related
to the current Vnic, and it was setting the hardware address for this peer.
It was a very specific case for Ciao but we want more explicit things now
that libsnnet is used externally from other projects.
This PR simplifies SetHardwareAddr() function and adds the complexity
in the Ciao's calling code instead.
Also, it returns a default peer name in case the link name does not fit the
expected prefixes. It is needed because in case you set manually the link
name, you will get a creation error. Indeed, the peer name will be "", and
it will be impossible to create a link with that name.
libsnnet API lets us get an existing VNIC device only from its alias. Because this
API is used externally by other project such as virtcontainers, there are cases where
no alias is set for an existing interface, and we want to be able to retrieve a VNIC
object from this existing interface.
This PR duplicates the existing GetDevice() to provide a new GetDeviceByName, relying
on netlink LinkByName function. It builds a VNIC object according to the information found
for the related link name.

@sboeuf Please look at the travis builds - looks like you introduce a panic triggered by the unit tests.

@rbradford yes I have noticed those issues, I will investigate them today.

Coverage increased (+0.01%) to 66.626% when pulling ff86766668411cf6b8d17370d222ada64506b1bc on sboeuf:master into a4835f0572884c7d749e2edb9cae6025f94707aa on 01org:master.

Coverage increased (+0.01%) to 66.626% when pulling ff86766668411cf6b8d17370d222ada64506b1bc on sboeuf:master into a4835f0572884c7d749e2edb9cae6025f94707aa on 01org:master.

Coverage increased (+0.03%) to 66.646% when pulling fce606bf3537355683e3f8eedbaeda6824f6552e on sboeuf:master into a4835f0572884c7d749e2edb9cae6025f94707aa on 01org:master.

Coverage increased (+0.03%) to 66.646% when pulling fce606bf3537355683e3f8eedbaeda6824f6552e on sboeuf:master into a4835f0572884c7d749e2edb9cae6025f94707aa on 01org:master.


Solves issue #311

Why:
Closes #311
What's being changed:
Adds programme to subtract two numbers
Check off the following:

[x] I have reviewed my changes in staging (look for "Automatically generated comment" and click Modified to view your latest changes).
[x] For content changes, I have completed the self-review checklist.

Hey @Mrczarny
Thanks for your contributions to this open source project
This PR will be reviewed and merged shortly

https://codinasion.web.app/programme/subtract-two-numbers/


Color decorators do not react to settings changed

Testing #33295

after changing the color decorators settings, there is a need to resize an editor for the settings change to take effect. If it's only about the old deprecated settings, then please feel free to close.

Yeah, looks like it reacts correctly for the proper editor.colorDecorators setting. I think that's good enough.

FYI, for the deprecated css.colorDecorators.enable, we didn't listen to the change (which requires a listener for all content change in the workspace but not just editor).


Pep8 has been renamed to pycodestyle

Fixes kubernetes-client/python#666,
co-PR with kubernetes-client/python#667.

/lgtm

Codecov Report

Merging #95 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #95   +/-   ##
=======================================
  Coverage   91.72%   91.72%
=======================================
  Files          13       13
  Lines        1136     1136
=======================================
  Hits         1042     1042
  Misses         94       94

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d7b6f7...13d5711. Read the comment docs.

/approve


Possible memory leak in 4.2.9

Demo attached, tested on Android but have experienced crashes on iOS in a project using the same principles.
Continuously swiping through a carousel with an ItemSource containing 100 view models, each slide showing a custom view, a view is being created for each slide but never released from memory. Views are being recreated even when going back to slides previously seen. This is causing the amount of memory being used to increase indefinitely.
Running my demo on Android and observing the debug log, you can see that when the app is running low on memory, the garbage collector tries to clean up and the memory in use is shown. I was going to add images to the demo but since I could see the problem without them, I wanted to show you that images aren't causing the issue (you just need to swipe through all the slides and maybe back a few too to trigger the garbage collection). Although this very small app is unlikely to crash, on another app I'm building with each custom view having multiple images and running on a low-memory device, it quickly crashes with an out-of-memory exception after swiping through only a small number of slides.
I think this might be related to #125 but I'm no Xamarin nor C# expert so my coding or understanding might be wrong.
Let me know if you need any more information. Any advice would be greatly appreciated, thanks!

I applied some fixes in previous versions to fix this but they were causing crashes on fast scrolling.

Seeing this with a carousel and only four items. I added a map control to the carousel pages, and after swiping back and forth a few times, things get laggier, and the map seems to be recreated all the time. Still would love to see a switch to keep pages in memory once they were created to prevent that :)

I am trying a fix, lets see if works.

HA i think i fixed this thing :)

UIPageViewControllers keep a ghost list of ViewControllers used to facilitate smooth swiping and it controls internally how many of them there. When its not a custom View, like Maps, ChildViewControllers keep growing and not matter if i force Dispose them the custom views stay in memory.
What i am doing different? Well, instead of creating a new controller when swiping, i return it if exist in ChildViewControllers (the ghost UIViewControllers list that UIPageViewController keeps in code behind). I am also disposing the UIViews when Dispose is automatically called by UIPageViewController in my ViewContainer implementation.
It seems to work like a charm. Will release a new version as soon as i fix another important issue.

That's iOS only, right? (I noticed the laggyness and map recreation on Android.)

Thats completelly normal in Android. Its the way the ViewPager works, unless i create something like iOS to keep a ghost collection of pages which i dont recommend.

In Android i implemented SaveState before but code is commented because it's never called. Will try to something about it, which is the right way.

Btw, a related idea - let me know if you want me to create a feature request:
I was thinking that it might be helpful if the view and/or view model of the pages could implement an optional interface which would tell them when they are being activated or deactivated. That might allow developers to delay complex operations, or do cleanup accordingly. This would be easy to implement for you I guess, and similar to MVVM frameworks, where initialization code of your view/vm is often super important.

This is the only fix that works:
The problem:
To facilitate smooth swiping, UIPageViewController keeps a ghost copy of the pages in a collection named ChildViewControllers. This collection is handled internally by UIPageViewController to keep a maximun of 3 items, but when a custom view is used from Xamarin.Forms side, the views hang in memory and are not collected no matter if internally the UIViewController is disposed by UIPageViewController.
Fix explained:
Some code has been added to GetPreviousViewController and GetNextViewController to return a child controller of exists in ChildViewController. Also Dispose has been implemented in ViewContainer to release the custom views. Dispose is called in the finalizer thread (UI) so the code to release the views from memory has been wrapped in InvokeOnMainThread.
Releasing a new version.


not able to install asar package

Hi,
I want to convert my atom-shell app into asar archive. Used "npm install -g asar" command to install asar and it throws following error. Can you please give me some pointers as in what is going wrong?
	D:\>npm install -g asar
	-
	><EMAIL_ADDRESS>install C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle\node_modules\native-mate
	> echo

	ECHO is on.

	><EMAIL_ADDRESS>install C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle
	> node-gyp rebuild

	C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle>node "D:\Softwares\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" r
	ebuild
	Warning: Missing input files:
	C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle\node_modules\native-mate\tests\compilation.cc
	Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
	C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle\node_modules\native-mate\vendor\base-minimal\base.vcxproj(18,3): error MSB4019: The imported project "C:\M
	icrosoft.Cpp.Default.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
	C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle\node_modules\native-mate\native_mate.vcxproj(18,3): error MSB4019: The imported project "C:\Microsoft.Cpp.
	Default.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
	gyp ERR! build error
	gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\12.0\bin\msbuild.exe` failed with exit code: 1
	gyp ERR! stack     at ChildProcess.onExit (D:\Softwares\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:267:23)
	gyp ERR! stack     at ChildProcess.emit (events.js:98:17)
	gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:820:12)
	gyp ERR! System Windows_NT 6.1.7601
	gyp ERR! command "node" "D:\\Softwares\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
	gyp ERR! cwd C:\Users\mote_v\AppData\Roaming\npm\node_modules\asar\node_modules\chromium-pickle
	gyp ERR! node -v v0.10.35
	gyp ERR! node-gyp -v v1.0.1
	gyp ERR! not ok

	npm ERR<EMAIL_ADDRESS>install: `node-gyp rebuild`
	npm ERR! Exit status 1
	npm ERR!
	npm ERR! Failed at the<EMAIL_ADDRESS>install script.
	npm ERR! This is most likely a problem with the chromium-pickle package,
	npm ERR! not with npm itself.
	npm ERR! Tell the author that this fails on your system:
	npm ERR!     node-gyp rebuild
	npm ERR! You can get their info via:
	npm ERR!     npm owner ls chromium-pickle
	npm ERR! There is likely additional logging output above.
	npm ERR! System Windows_NT 6.1.7601
	npm ERR! command "D:\\Softwares\\nodejs\\\\node.exe" "D:\\Softwares\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "-g" "asar"
	npm ERR! cwd D:\
	npm ERR! node -v v0.10.35
	npm ERR! npm -v 1.4.28
	npm ERR! code ELIFECYCLE
	npm ERR! not ok code 0

Sorry for posting this issue here instead of posting it in asar project. Do let me know if I have to close this issue here and post it there.

Yeah please reopen this issue in the asar repo.
And have you tried to install other native modules like runas? It seems like a problem of your building environment.