dynamics-365-customer-engagement

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d0d40644-3c39-9e9a-c8d9-9f7f267bb3c5
Version Independent ID: 1b8e0bea-23a0-f3ca-be3e-ad82b65dc33e
Content: Overview of Community
Content Source: ce/customer-service/community-overview.md
Service: dynamics-365-customerservice
GitHub Login: @lalexms
Microsoft Alias: laalexan

### Tasks
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3187
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/pull/3126
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3176
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3186
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3185
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3188
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/639
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3184
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/2308
- [ ] https://github.com/MicrosoftDocs/dynamics-365-customer-engagement/issues/3068

### Tasks

### Tasks

Thanks for your comment, @Trinhan87. This issue has been assigned to the doc author for review.

@Trinhan87 could you please provide details about the updates you are looking for?

Closing due to no feedback.


[TW#23626] BLE Scanning: Reset/configure duplicate filter

Environment

Core (if using chip or module): |ESP32-WROOM-32U|ESP-WROOM-32D|ESP-WROOM32|ESP32]
IDF version (git rev-parse --short HEAD to get the commit id.):
e1a6e71
Development Env:		Make
Operating System:		MacOS
Power Supply:         USB

Not exactly a problem, but a question and a request
Question: For BLE scanning is it possible to reset the duplicate filter?  Is it a timer based solution now?
Request: Is it possible to change this to 500/1000 duplicate ?? May be if we use the WROVER32 for extra memory?
Sam

hi @buzzzsam  There are three ways to reset the duplicate filter.

stop scan
duplicate filter is full, the size is due to "Maximum number of devices in scan duplicate filter" in menuconfig
Time is up (about one minute)

you can change the "Maximum number of devices in scan duplicate filter" to 500/1000, you need modify some code in /components/bt/Kconfig.
config DUPLICATE_SCAN_CACHE_SIZE
int "Maximum number of devices in scan duplicate filter"
depends on BLE_SCAN_DUPLICATE
range 10 1000
default 20
help
Maximum number of devices which can be recorded in scan duplicate filter.
When the maximum amount of device in the filter is reached, the cache will be refreshed.


Use optimized logging pattern in routing

Routing is still using the 'simple' logging pattern. We should be using the more optimized pattern.

Fixed in #274


Create-AzureSnapshot/readme: add quick deployment option

Proposed Changes

Readme: add quick deployment option

(Todo: Prerequisites and fix per #3489)

@juju4 please complete the Readme doc (add in the prerequisites that are marked as TBD)

@juju4 Please make the required changes.

I have not identified all the requirements to make this logicapp run "as is" and plan to submit an alternate version.
it seems to me the information below is not made available anymore from sentinel block
@{body('Parse_JSON')?['ExtendedProperties']?['Query']}

at least, not present in my environment.

Hey @sarah-yo , please review and approve the changes, thanks.

Hey @juju4, can we close this for now, you can reopen or raise new PR once it is decided, thanks.

I have not identified all the requirements to make this logicapp run "as is" and plan to submit an alternate version.
it seems to me the information below is not made available anymore from sentinel block
@{body('Parse_JSON')?['ExtendedProperties']?['Query']}

at least, not present in my environment.

Closing.
It would be easier if initial author could confirm if current version is working and add requirements.


Zombie Process problem.

Hello,
Recently we talked about this problem in the issues #1823 and #1791.
Environment:

Puppeteer Version: 1.0.
Chrome Version: 64.0.3282.71 (https://github.com/adieuadieu/serverless-chrome/releases/tag/v1.0.0-34)
Platform / OS version: AWS Lambda
Node.js version: 6.10 (https://aws.amazon.com/about-aws/whats-new/2017/03/aws-lambda-supports-node-js-6-10/)

Use Case:
We are using puppeteer on AWS Lambda. We take a screenshot of given HTML template and upload it to S3 and use this image for future requests
It handles over 100 million requests each month. That's why every process should be atomic and immutable. (AWS Lambda has a disk and process limit.)
Example Code:
const browser = await puppeteer.launch({
  args: ['--disable-gpu', '--no-sandbox', '--single-process',
             '--disable-web-security', '--disable-dev-profile']
});
const page = await browser.newPage();
await page.goto('https://s3bucket.com/markup/a.html');
const response = await page.screenshot({{ type: 'jpeg', quality: 95 }});
browser.close();

Problem
When we are using example code, we got disk error from AWS Lambda.
Example /tmp folder:
2018-01-12T14:55:38.553Z    a6ef3454-f7a8-11e7-be0f-17f405d5a180    start stdout: total 226084
drwx------ 3 sbx_user1067 479 4096 Jan 12 14:55 .
drwxr-xr-x 21 root root 4096 Jan 12 10:53 ..
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:33 core.headless-chromi.129
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:15 core.headless-chromi.131
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:49 core.headless-chromi.135
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:52 core.headless-chromi.137
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:50 core.headless-chromi.138
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:51 core.headless-chromi.14
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:49 core.headless-chromi.15
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:36 core.headless-chromi.169
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:15 core.headless-chromi.174
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:52 core.headless-chromi.178
-rw------- 1 sbx_user1067<PHONE_NUMBER>8 Jan 12 14:50 core.headless-chromi.180
drwx------ 3 sbx_user1067 479 4096 Jan 12 14:14 .pki

When we investigated these files, we understood that it is a core dump. We removed these files after the process completed.
When we monitored process list, we saw zombie processes Zombie chrome processes have been growing increasingly.   AWS Lambda has a maximum process limit. (max 1024 process) That's why we reach the lambda limits.
483 1 3.3 1.6 1226196 65408 ? Ssl 22:07 0:05 /var/lang/bin/node --max-old-space-size=870 --max-semi-space-size=54 --max-executable-size=109 --expose-gc /var/runtime/node_modules/awslambda/index.js
483 22 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 73 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 119 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 166 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 214 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 262 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 307 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 353 0.0 0.0 0 0 ? Z 22:07 0:00 [headless-chromi] <defunct>
483 1915 0.0 0.0 0 0 ? Z 22:09 0:00 [sh] <defunct>

We couldn't use dump-init on lambda. Because lambda already has an init system.
How did we fix it? (very hacky method)
We used browser.disconnect() instead of browser.close(). We manualy managed chrome processes such as kill.
Example Code:
browser.on('disconnected', () => {
    console.log('sleeping 100ms'); //  sleep to eliminate race condition
    setTimeout(function(){
    console.log(`Browser Disconnected... Process Id: ${process}`);
    child_process.exec(`kill -9 ${process}`, (error, stdout, stderr) => {
        if (error) {
        console.log(`Process Kill Error: ${error}`)
        }
        console.log(`Process Kill Success. stdout: ${stdout} stderr:${stderr}`);
    });
}, 100);

Firstly we didn't use this method. We only killed the process after browser disconnect. We got the following error:
Error: read ECONNRESET at exports._errnoException (util.js:1018:11) at TCP.onread (net.js:568:26)

I think it looks like a puppeteer process management problem. When we used this method, we didn't receive any puppeteer related errors. How can we fix it?
Thanks.

I found that doing await page.goto('about:blank') helps in terms of reducing cpu and memory usage, even if reusing the tabs setting to about:blank between shots somehow seems to keep cpu and memory under control.

@bahattincinic - thanks, I've tried your method of disconnecting + killing the process, and while it does kill the "main" process returned by puppeteer.launch(), each run seems to leave another  zombie with a PID that is different than the killed one...
What's worse, when I run ps aux right after puppeteer.launch(), aside from the "main" process, there is already one that's , right away, before running code or trying to kill anything.
I've tried sending a kill -15, hoping that will allow the main process to clean up its children, but -15 or -9 doesn't make any difference, so I'm still stuck with an ever-growing list of zombies and rising memory...
Do you have any advice on how you managed to keep it clean of those as well (if you had a similar experience)? I'm also running on Lambda, same args used, puppeteer 1.1.1. Thanks!

@bahattincinic - thanks a lot for providing details - waitpid is an interesting approach and I'll definitely try with cleaning /tmp, hopefully that helps! If I don't manage to make it run reliably on Lambda, I'm going to have to try with docker - thanks for linking the example!

I also got some "cat" process defunct
root        22  0.0  0.0      0     0 ?        Z    16:06   0:00 [cat] <defunct>
root        23  0.0  0.0      0     0 ?        Z    16:06   0:00 [cat] <defunct>
root        25  0.0  0.0      0     0 ?        Z    16:06   0:00 [chrome] <defunct>
root        43  2.7  0.0      0     0 ?        Z    16:06   0:34 [chrome] <defunct>
root        51  0.0  0.0      0     0 ?        Z    16:06   0:00 [chrome] <defunct>

@bahattincinic How does stopping the parent process with waitpid help with reaping zombie processes?

Hi, I am also having problems with this, also serverless-chrome with AWS Lambda.
In my case, it looks like it does not have anything to do with the browser cleanup process. It looks like it is being caused by something that happens during Puppeteer launch.
Running ps alx immediately after browser launch gives me this:
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
4 487 1 0 20 0 1488300 271872 ep_pol Ssl ? 0:34 /var/lang/bin/node --expose-gc --max-semi-space-size=102 --max-old-space-size=1843 /var/runtime/node_modules/awslambda/index.js
1 487 13 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 59 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 107 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 153 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 203 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
0 487 246 1 20 0 1074400 67740 ep_pol Ssl ? 0:00 ./chrome/headless-chromium --disable-background-networking --disable-background-timer-throttling --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-translate --metrics-recording-only --no-first-run --safebrowsing-disable-auto-update --enable-automation --password-store=basic --use-mock-keychain --remote-debugging-port=0 --user-data-dir=/tmp/puppeteer_dev_profile-yQiw0t --headless --disable-gpu --hide-scrollbars --mute-audio --no-sandbox --disable-setuid-sandbox --disable-dev-shm-usage --single-process --disable-gpu --no-zygote --user-agent=REDACTED
1 487 248 246 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
0 487 280 1 20 0<PHONE_NUMBER> - R ? 0:00 ps -alx

See process 248 which is already defunct at this point.
And then after the browser closes:
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
4 487 1 0 20 0 1486912 282588 ep_pol Ssl ? 0:41 /var/lang/bin/node --expose-gc --max-semi-space-size=102 --max-old-space-size=1843 /var/runtime/node_modules/awslambda/index.js
1 487 13 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 59 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 107 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 153 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 203 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
1 487 248 1 20 0 0 0 - Z ? 0:00 [headless-chromi] <defunct>
0 487 292 1 20 0<PHONE_NUMBER> - R ? 0:00 ps -alx

Look at process with pid 248 which initially had a ppid 246 (and was already defunct at that point).
Is this even a Puppeteer bug?

I have puppeteer running in a Docker container both locally and in a Heroku dyno.
Over time, a growing number of chrome processes remain Sleeping which, once again, holds true both for local and Heroku setup as well apart from the fact that Heroku dynos, unlike my local containers are subject to Process/thread limits (in my case it's 256). After a while, the limit gets reached and the following error is raised:
2018-06-16T16:25:16.463094+00:00 app[web.1]: Error: spawn /app/node_modules/puppeteer/.local-chromium/linux-564778/chrome-linux/chrome EAGAIN
2018-06-16T16:25:16.463108+00:00 app[web.1]: at _errnoException (util.js:1022:11)
2018-06-16T16:25:16.463110+00:00 app[web.1]: at Process.ChildProcess._handle.onexit (internal/child_process.js:190:19)
2018-06-16T16:25:16.463111+00:00 app[web.1]: at onErrorNT (internal/child_process.js:372:16)
2018-06-16T16:25:16.463112+00:00 app[web.1]: at _combinedTickCallback (internal/process/next_tick.js:138:11)
2018-06-16T16:25:16.463114+00:00 app[web.1]: at process._tickDomainCallback (internal/process/next_tick.js:218:9)
2018-06-16T16:25:16.495669+00:00 app[web.1]: npm ERR! code ELIFECYCLE
2018-06-16T16:25:16.495980+00:00 app[web.1]: npm ERR! errno 1
2018-06-16T16:25:16.497099+00:00 app[web.1]: npm ERR<EMAIL_ADDRESS>start: `node app.js`
2018-06-16T16:25:16.497182+00:00 app[web.1]: npm ERR! Exit status 1
2018-06-16T16:25:16.497371+00:00 app[web.1]: npm ERR!
2018-06-16T16:25:16.497499+00:00 app[web.1]: npm ERR! Failed at the<EMAIL_ADDRESS>start script.
2018-06-16T16:25:16.497621+00:00 app[web.1]: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2018-06-16T16:25:16.502935+00:00 app[web.1]:
2018-06-16T16:25:16.503086+00:00 app[web.1]: npm ERR! A complete log of this run can be found in:
2018-06-16T16:25:16.503166+00:00 app[web.1]: npm ERR!     /app/.npm/_logs/2018-06-16T16_25_16_498Z-debug.log

How should I handle that?

@webyneter We're using an alpine-based image. I just installed tini, and added "/sbin/tini", "--" in front of our normal entrypoint/command.

@Multiply you mean, you did
ENTRYPOINT ["/sbin/tini", "--"]

?

@Multiply got that, will try Alpine image puppeteer guide with tini.

@webyneter i previously explained my solution. But you can use my example implementation
https://github.com/bahattincinic/puppeteer-docker-example
According to puppetter documentation, it suggests yelp/dump-init.
https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker
Also you can check https://github.com/ebidel/try-puppeteer this project. The repo creator is a core developer of puppetter.

@jdiamond did you try https://github.com/GoogleChrome/puppeteer/issues/1825#issuecomment-372241101 it ?

@bahattincinic thanks, I already implemented mine based on your suggestion to employ Alpine and tini. Although I followed the official guide, including the Tips section where it's said to --disable-dev-shm-usage Heroku gave me another surprise: "Less than 64MB of free space in temporary directory for shared memory files". Not sure yet how to deal with it, and if that's even possible from an end user's end...

Found the Increased size of /dev/shm on the Common Runtime defv article.

You don't have to use alpine to use tini, or dumbinit, it's just what we're using quite successfully so far.

I'm having a growing number of sleeping chrome process which is a real blocker for me since Heroku limits the number of processes on low tier plans.

Hi guys. I am having Lambda function with puppeteer. It runs perfectly returns what I expect but never exits and I got timedOut. The only thing I do not use on the puppeteer side is

await browser.close()
due to prevent of opening 100 browsers and pages.
Do you have any idea

this is the log I am receiving:

14:07:03
START RequestId: 6df967f7-787a-5cda-9e10-33b63c469bd2 Version: $LATEST

14:07:03
2018-10-17T14:07:03.877Z	6df967f7-787a-5cda-9e10-33b63c469bd2	getting browser

14:07:03
2018-10-17T14:07:03.885Z	6df967f7-787a-5cda-9e10-33b63c469bd2	setup local chrome

14:07:05
2018-10-17T14:07:05.376Z	6df967f7-787a-5cda-9e10-33b63c469bd2	setup done

14:07:06
2018-10-17T14:07:06.149Z	6df967f7-787a-5cda-9e10-33b63c469bd2	Launch chrome: HeadlessChrome/67.0.3361.0

14:07:06
2018-10-17T14:07:06.149Z	6df967f7-787a-5cda-9e10-33b63c469bd2	opening new page

14:07:06
2018-10-17T14:07:06.967Z	6df967f7-787a-5cda-9e10-33b63c469bd2	page opened

14:07:07
2018-10-17T14:07:07.069Z	6df967f7-787a-5cda-9e10-33b63c469bd2	viewport set

14:07:07
2018-10-17T14:07:07.170Z	6df967f7-787a-5cda-9e10-33b63c469bd2	userAgent set

14:07:07
2018-10-17T14:07:07.171Z	6df967f7-787a-5cda-9e10-33b63c469bd2	getting action from commandArray

14:07:07
2018-10-17T14:07:07.171Z	6df967f7-787a-5cda-9e10-33b63c469bd2	case connect

14:07:07
2018-10-17T14:07:07.171Z	6df967f7-787a-5cda-9e10-33b63c469bd2	connecting to: https://www.google.com/

14:07:07
2018-10-17T14:07:07.541Z	6df967f7-787a-5cda-9e10-33b63c469bd2	succesfully connected to https://www.google.com/

14:07:08
2018-10-17T14:07:08.542Z	6df967f7-787a-5cda-9e10-33b63c469bd2	case connect passed

14:07:08
2018-10-17T14:07:08.542Z	6df967f7-787a-5cda-9e10-33b63c469bd2

14:07:08
2018-10-17T14:07:08.542Z	6df967f7-787a-5cda-9e10-33b63c469bd2	{}

14:07:08
2018-10-17T14:07:08.543Z	6df967f7-787a-5cda-9e10-33b63c469bd2	puppeteer done

14:07:08
2018-10-17T14:07:08.543Z	6df967f7-787a-5cda-9e10-33b63c469bd2	finished

14:12:03
END RequestId: 6df967f7-787a-5cda-9e10-33b63c469bd2

14:12:03
REPORT RequestId: 6df967f7-787a-5cda-9e10-33b63c469bd2	Duration: 300100.24 ms	Billed Duration: 300000 ms Memory Size: 3008 MB	Max Memory Used: 354 MB

14:12:03
2018-10-17T14:12:03.975Z 6df967f7-787a-5cda-9e10-33b63c469bd2 Task timed out after 300.10 seconds

Hi @bryanlarsen, hi @bahattincinic,
thanks for your helpful comments. I am pretty new to the Docker&Puppeteer topic. So please, excuse me when I ask (again):

you either use an init or waitpid.

What do you mean @bryanlarsen? Can I just use the solution proposed on puppeteer/troubleshooting:
FROM node:8.11.1-alpine

# ... stuff

# It's a good idea to use dumb-init to help prevent zombie chrome processes.
ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 /usr/local/bin/dumb-init
RUN chmod +x /usr/local/bin/dumb-init

# ... stuff
ENTRYPOINT ["dumb-init", "--"]

# start the app
CMD [ "node", "app/index.js"]

Or how can I (should I) use waitpid?
@bahattincinic I can't find waitpid in your example-repo (see comment https://github.com/GoogleChrome/puppeteer/issues/1825#issuecomment-372241101). Am I missing something?
And you let

We delete /tmp folder after process completed (rm -r /tmp/core.* || true)

run from your node app after await browser.close();?
Thanks to both of you for your time and help!

@insanehong Can you try the following code?
const waitpid2 = require('waitpid2');
const child_process = require('child_process');

const cleanEnv = function() {
  console.log('cleanEnv:waitpid');
  while (waitpid2.waitpid(-1, 0 | waitpid2.WNOHANG) == -1);
  child_process.exec('rm -r /tmp/core.* || true');
}

const screenCapture = function() {
    // do something.
}

exports.handler = function(event, context, callback) {
    cleanEnv();
    screenCapture();
}

i tested , use dumb-init to start docker container is ok
https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#running-puppeteer-in-docker

@yale8848 yep it works. But you can't use dumb-init on AWS lambda.

@yale8848 yep it works. But you can't use dumb-init on AWS lambda.

but aws lambda supposes to be short-lived right?

Yes, you are right. But Lambda uses the same container for multiple requests. That's why each request can effect each other.
if you want to learn how lambda cycle process works,  you can read the following documents;

https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/
https://medium.com/@tjholowaychuk/aws-lambda-lifecycle-and-in-memory-caching-c9cd0844e072

@aslushnikov @bahattincinic  This might help other facing this problem.
I run Puppeteer session on AWS Lambda for my SaaS https://checklyhq.com. I noticed the same issue of defunct Chrome processes hanging around over multiple Lambda calls. Also, the /tmp directory was piling up with profiles.
Note, this did not happen in 99% of the cases, only when something unforeseen timed out of some other anomaly happened.
I think I've managed to fix this without injecting any code into Puppeteer scripts. Here is what I do in a nutshell.

I spawn a child node process for running the Puppeteer code
I store the PID
I wait for the child process to exit or time out
I rm -rf the /tmp dir
I explicitly kill the PID using the ps-tree package

In semi / pseudo Node.js code
const spawn = require('child_process').spawn
const psTree = require('ps-tree')

// cmd, args and opts are like "node script.js etc." where script is a .js file with some Puppeteer code
const child = spawn(cmd, args, opts)

// wait for the script to run, then kill

await kill()

// then cleanup

await cleanupTmp()

function kill () {
    return new Promise((resolve, reject) => {
      const signal = 'SIGTERM'
      const pid = child.pid
      psTree(pid, (err, children) => {
        if (err) { return reject(err) }
        [pid].concat(
          children.map(p => {
            return p.PID
          })
        ).forEach(tpid => {
          try {
            process.kill(tpid, signal)
          } catch (ex) {}
        })
        return resolve()
      })
    })
  }

function cleanupTmp () {
    return Promise.all([
      Runtime._rimrafPromise('/tmp/core.chromium.*'),
      Runtime._rimrafPromise('/tmp/puppeteer_dev_profile*')
    ])
  }

function rimrafPromise (pattern) {
    return new Promise((resolve, reject) => {
      rimraf(pattern, err => {
        if (err) return reject(err)
        resolve()
      })
    })
  }

I fixed it when I upgraded the headless chrome version.
I recommend that you refer to this repository.

I think, it's not way that puppeteer should work but using page.close() solve my zombie process problem. I think you should try it.

@bahattincinic I use await page.close() but I also have zombie processes =(

Using dumb-init resolved my problems.

I fixed this issue by adding
        '--disable-setuid-sandbox',
        '--no-zygote',

To my launch configs which then makes it so that only one chrome process is created on launch (as I discovered otherwise that when browser.close() was called it only closed one chrome process and it seems like 2 chrome processes are created on launch (of which the second becomes the zombie when the browser is closed))

Hello,
Thanks to all for the debug.
I run puppeteer inside inside docker.
To remove zombie process i added the argument --no-zygote and now, is work fine when i use browser.close(), all process stop.
My code sample:
router.get(`/share/image`, async function (req, res, next) {
    // ... some logic from my project ...

   // puppeteer code
        const browser = await puppeteer.launch({args: ['--no-sandbox', '--single-process', '--no-zygote']});

        try {
            const page = await browser.newPage();
            await page.setViewport({
                width: 1920,
                height: 1080
            });

            await page.setContent(html, {
                waitUntil: "networkidle0"
            });

            let image = await page.screenshot({
                type: 'jpeg',
                quality: 100
            });

            await browser.close();

            res.contentType('image/jpeg');
            res.send(image)
        } catch(e) {
            console.log(e);
            await browser.close();
            res.status(503).end();
        }
});

I not understand fully this flag, but here a little explication: https://codereview.chromium.org/2384163002
Thanks to all

I've overcome these issues by adding the flags for chrome headless:
const chromeFlags = [
    '--headless',
    '--no-sandbox',
    "--disable-gpu",
    "--single-process",
    "--no-zygote"
]

I think the child processes are orphaned when the parent is killed and that leads to the zombies. With this, I only get one process and it works pretty well

@marius080 Even with your settings zombies processes are left after browser.close().

what worked fine to me was to destroy the PID and subprocesses associated:
https://github.com/microlinkhq/browserless/blob/master/packages/browserless/src/driver.js#L66

This is is my code, tested pretty thoroughly, works great, no more dangling chromes :)
  const browser = await puppeteer.launch({args: ['--no-sandbox']});
  const process = browser.process()
  const killBrowser = (retries=5) => {
    if (process && process.pid && process.kill && !process.killed) {
      setTimeout(() => {
        console.log(`BROWSER Process Id: ${process.pid}, KILLING IT! retries:`, retries);
        if (!process.kill('SIGKILL')) {
          retries--
          killBrowser(retries)
        }
      }, 200);
    }
  }

This is is my code, tested pretty thoroughly, works great, no more dangling chromes :)
  const browser = await puppeteer.launch({args: ['--no-sandbox']});
  const process = browser.process()
  const killBrowser = (retries=5) => {
    if (process && process.pid && process.kill && !process.killed) {
      setTimeout(() => {
        console.log(`BROWSER Process Id: ${process.pid}, KILLING IT! retries:`, retries);
        if (!process.kill('SIGKILL')) {
          retries--
          killBrowser(retries)
        }
      }, 200);
    }
  }

If you run it under docker you need to use docker --init option.

If you run it under docker you need to use docker --init option.

Er
El lun., 11 de enero de 2021 6:28 p. m., g00dnatur3 <
<EMAIL_ADDRESS>escribió:

This is is the code i created to fix it, tested it, no more dangling
chromes :)
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const process = browser.process()
const killBrowser = (retries=5) => {
if (process && process.pid && process.kill && !process.killed) {
setTimeout(() => {
console.log(BROWSER Process Id: ${process.pid}, KILLING IT! retries:, retries);
if (!process.kill('SIGKILL')) {
retries--
killBrowser(retries)
}
}, 200);
}
}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/puppeteer/puppeteer/issues/1825#issuecomment-758311271,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARXQAM3WCJH4IIPK7JOOJ4LSZOJTXANCNFSM4EME5SAA
.

Er
El lun., 11 de enero de 2021 6:28 p. m., g00dnatur3 <
<EMAIL_ADDRESS>escribió:

This is is the code i created to fix it, tested it, no more dangling
chromes :)
const browser = await puppeteer.launch({args: ['--no-sandbox']});
const process = browser.process()
const killBrowser = (retries=5) => {
if (process && process.pid && process.kill && !process.killed) {
setTimeout(() => {
console.log(BROWSER Process Id: ${process.pid}, KILLING IT! retries:, retries);
if (!process.kill('SIGKILL')) {
retries--
killBrowser(retries)
}
}, 200);
}
}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/puppeteer/puppeteer/issues/1825#issuecomment-758311271,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARXQAM3WCJH4IIPK7JOOJ4LSZOJTXANCNFSM4EME5SAA
.

browser.on('disconnected', () => {
    console.log('sleeping 100ms'); //  sleep to eliminate race condition
    setTimeout(function(){
    console.log(`Browser Disconnected... Process Id: ${process}`);
    child_process.exec(`kill -9 ${process}`, (error, stdout, stderr) => {
        if (error) {
        console.log(`Process Kill Error: ${error}`)
        }
        console.log(`Process Kill Success. stdout: ${stdout} stderr:${stderr}`);
    });
}, 100);

Same problem on manjaro using headless. I will test your workaround

browser.on('disconnected', () => {
    console.log('sleeping 100ms'); //  sleep to eliminate race condition
    setTimeout(function(){
    console.log(`Browser Disconnected... Process Id: ${process}`);
    child_process.exec(`kill -9 ${process}`, (error, stdout, stderr) => {
        if (error) {
        console.log(`Process Kill Error: ${error}`)
        }
        console.log(`Process Kill Success. stdout: ${stdout} stderr:${stderr}`);
    });
}, 100);

Same problem on manjaro using headless. I will test your workaround

Hi everyone, just wanted to provide a quick warning about the --single-process flag. I have some integration tests that were broken after I started using this flag, because it broke the font rendering and kerning for the generated PDF. I found this Process Models page in the chromium docs:

Finally, for the purposes of comparison, Chromium supports a single process model that can be enabled using the --single-process command-line switch. In this model, both the browser and rendering engine are run within a single OS process.

The single process model provides a baseline for measuring any overhead that the multi-process architectures impose. It is not a safe or robust architecture, as any renderer crash will cause the loss of the entire browser process. It is designed for testing and development purposes, and it may contain bugs that are not present in the other architectures.

I can confirm that the single process model does cause a rendering bug (at least on Chromium 88.x), and that this rendering bug is not present when I remove the --single-process flag. So I would not recommend using this flag, since the docs say that it's only designed for testing and development purposes, and it shouldn't really be used in production.

I'm also using puppeteer in docker, and I had also tried the
puppeteer.launch({ args: ['--no-sandbox', '--no-zygote'] });
but that did not help.
Eventually, I figured out that the init:true flag solves the orphaned zombie process problem, according to docker documentation: https://docs.docker.com/compose/compose-file/compose-file-v3/#init
(@bryanlarsen and @zdm also mentioned the --init flag for docker, and I also gained inspiration from the great blog to
understand it more https://blog.phusion.nl/2015/01/20/docker-and-the-pid-1-zombie-reaping-problem/ )

This works for me on Debian 10, just kills the process group based on the browser pid:
const browser = await puppeteer.launch({ args: ['--no-sandbox'] })
const page = await browser.newPage()

try {
  await page.goto(url, { waitUntil: 'networkidle2', timeout: 10000 })
  await page.screenshot({ path })
  await page.close()
  await browser.close()
} catch (e) {
  console.error(e)
} finally {
  const pid = -browser.process().pid
  try {
    process.kill(pid, 'SIGKILL')
  } catch (e) {}
}

Here is what happens while the puppeteer is working:

Before sending request - all processes are empty:

STAT PID   PPID  COMMAND          COMMAND
S        1     0 dumb-init        /usr/local/bin/dumb-init -- node server.js
S        7     1 node             node server.js
S      135     0 sh               sh

After sending a request to render the web page - the puppeteer spawn 7 chromium processes

STAT PID   PPID  COMMAND          COMMAND
S        1     0 dumb-init        /usr/local/bin/dumb-init -- node server.js
S        7     1 node             node server.js
S      135     0 sh               sh
S      630     7 chrome           /usr/lib/chromium/chrome --extra-plugin-dir=/usr/lib/nsbrowser/plugins --allow-pre-commit-input --disable-background-networking --enable-features=NetworkServiceInProces
S      634   630 chrome           /usr/lib/chromium/chrome --type=zygote --no-zygote-sandbox --no-sandbox --headless --headless
S      635   630 chrome           /usr/lib/chromium/chrome --type=zygote --no-sandbox --headless --headless
S      655   630 chrome           /usr/lib/chromium/chrome --type=utility --utility-sub-type=network.mojom.NetworkService --lang=en-US --service-sandbox-type=none --no-sandbox --disable-dev-shm-usage --
S      659   635 chrome           /usr/lib/chromium/chrome --type=renderer --headless --lang=en-US --no-sandbox --disable-dev-shm-usage --disable-background-timer-throttling --disable-breakpad --enable-
S      698   634 chrome           /usr/lib/chromium/chrome --type=gpu-process --no-sandbox --disable-dev-shm-usage --disable-breakpad --headless --ozone-platform=headless --use-angle=swiftshader-webgl -
R      710   635 chrome           /usr/lib/chromium/chrome --type=renderer --headless --lang=en-US --no-sandbox --disable-dev-shm-usage --disable-background-timer-throttling --disable-breakpad --enable-

5 seconds after sending the request - you can see 3 zombie processes

S        1     0 dumb-init        /usr/local/bin/dumb-init -- node server.js
S        7     1 node             node server.js
S      135     0 sh               sh
S      630     7 chrome           /usr/lib/chromium/chrome --extra-plugin-dir=/usr/lib/nsbrowser/plugins --allow-pre-commit-input --disable-background-networking --enable-features=NetworkServiceInProces
S      634   630 chrome           /usr/lib/chromium/chrome --type=zygote --no-zygote-sandbox --no-sandbox --headless --headless
S      635   630 chrome           /usr/lib/chromium/chrome --type=zygote --no-sandbox --headless --headless
Z      655   630 chrome           [chrome]
Z      659   635 chrome           [chrome]
S      698   634 chrome           /usr/lib/chromium/chrome --type=gpu-process --no-sandbox --disable-dev-shm-usage --disable-breakpad --headless --ozone-platform=headless --use-angle=swiftshader-webgl -
Z      710   635 chrome           [chrome]

10 seconds after sending the request - everything back to normal thanks to dumb-init

STAT PID   PPID  COMMAND          COMMAND
S        1     0 dumb-init        /usr/local/bin/dumb-init -- node server.js
S        7     1 node             node server.js
S      135     0 sh               sh

My question is why processes 655,659,710 became zombie process ? what in the flow is not closing them right?
I prefer to avoid using 3rd party tool like dumb-init to overcome this issue
I'm using the Docker image node:16.14.2-alpine3.15 on AWS EKS 1.22
Command to list processes: ps -A -ostat,pid,ppid,comm,args
It seems that using the arg --single-process can solve this, because it will use only 1 process instead of 7,but due to some issues that mentioned above with the arg --single-process I prefer not to use that

Nothing of this helps when the browser crashes, which happens quite often after we upgraded to Node 18 and newer puppeteer.

I have thoroughly reviewed the documentation and exhausted all available solutions in an attempt to resolve the zombie process issue. Despite my efforts, the problem persisted. I attempted to terminate process IDs, but within the pods, the zombie processes remained resilient. Devoting several consecutive days to diligently updating every package eventually proved successful. The issue was ultimately resolved by making key adjustments: switching the operating system from Node Alpine to Node Slim Linux and transitioning from Chromium to Chrome as the browser. The specific changes implemented to rectify the problem are outlined below.
If you are working with Puppeteer and encountering zombie process issues, consider employing the following Docker commands. These commands have proven effective in preventing the creation of zombie processes.
FROM node:18-slim
RUN apt-get update
RUN apt-get upgrade
RUN apt-get update && apt-get install curl gnupg -y
&& curl --location --silent dl-ssl.google.com/linux/linux_sign... | apt-key add -
&& sh -c 'echo "deb [arch=amd64] dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
&& apt-get update
&& apt-get install google-chrome-stable -y --no-install-recommends
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update &&
apt-get upgrade && apt-get install -y vim
ADD ./puppetron.tar /usr/share/
WORKDIR /usr/share/puppetron
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
ENV SERVICE_PATH=/usr/share/puppetron
CMD node main.js;

Path of browser change to
executablePath: '/usr/bin/google-chrome',

I've overcome these issues by adding the flags for chrome headless:
const chromeFlags = [
    '--headless',
    '--no-sandbox',
    "--disable-gpu",
    "--single-process",
    "--no-zygote"
]

I think the child processes are orphaned when the parent is killed and that leads to the zombies. With this, I only get one process and it works pretty well

adding "--headless" worked for me, i had a lot of chrome_crashpad that were created per each request even after closing without errors the page and browser.


Base price fields in Checkout and Order should contain only catalogue promotion discount

Problem
Checkout and Order models have fields base_x_price. The fields store the values that we use to calculate the discount based on the catalog promotions. Some vouchers like specific_products are also included in base_x_price fields which causes misleading in the assumption that base price always contains only catalog promotion.
The idea here is to exclude the applying vouchers from base_x_price fields and move them to CheckoutLineDiscount and OrderLineDiscount. Thanks to it, the base_x field will always store the value of the product without any order discounts or voucher applied to a specific line.
The discount from OrderLineDiscount/CheckoutLineDiscount should be applied to price field.
The tax payload that we send to tax app should have included voucher/discount for specific-line
The avatax plugin should also include the voucher/discount for specific-line

@aniav This is not valid anymore as we decided to store all line-level discounts in base prices (we need them to properly apply order-level discunts)


BFD: Benes per Contract

Requested Feature
*** Migrated from Confluence ***
Beneficiaries per contract. Realistically there should be about a 1% change across months. Also, the changes by year should be about 10-20% across years.

This was fixed by commit https://github.com/synthetichealth/synthea/pull/770/commits/4bf06a7efdcc81140be01415b7cf5097e83d64ea


Quick Append/remove Item of virtual scroll cause critical error of Item Service Call

My Virtual Scroll template:
`

  SOME OTHER CONTENT

  <app-feedback-comment
      class="{{'feedback-' + feedback?.id}}"
      [data]="{
          room: 'feedback-' + feedback?.id,
          feedback: feedback
        }">
  </app-feedback-comment>

`
My app-feedback-comment component:
import {Component, OnChanges, OnDestroy, OnInit, SimpleChanges} from "@angular/core";
@Component({
selector: 'app-feedback-comment',
templateUrl: './feedback-comment.component.html',
styleUrls: ['./feedback-comment.component.scss']
})
export class FeedbackCommentComponent implements OnInit, OnChanges, OnDestroy {
public privacyUsers = [];
constructor(
public feedbackService: FeedbackService
) {
}
ngOnChanges(changes: SimpleChanges) {
// Others code

if (this.data.feedback != undefined) {

  this.feedbackService.getRespectiveManagers(forUser.id).subscribe(
    (respectiveManagersData: any) => {

      let _privacyUsers = [];
      // Other code
      let respectiveManagers = _.get(respectiveManagersData, 'respectiveManagers', []);

      respectiveManagers.forEach(
        (x: any) => {

          let idx = _privacyUsers.indexOf(x.id);

          if (idx > -1) {
            this.privacyUsers.push(x.name);
          }

        });

      this.privacyUsers.push(this.data.feedback.user.name);

    });

}

}
}
My app-feedback-comment template example:
`

{{privacyUser?.name}}
`
When we scroll fast, It append/remove Virtual Scroll Item while getRespectiveManagers service call terninated, and it push new data to privacyUsers array error. It cause show privacyUsers data to HTML template wrong completely.

Please update this issue description to include a reproduction in stackblitz (you can use our template to get starte https://stackblitz.com/edit/components-issue) as well as clear steps to reproduce and the actual result you are seeing vs what was expected

Closing since the issue didn't have a reproduction to begin with and it hasn't been updated in a long time.


fix links in download pulldown

looks like the feature of pulling old versions isn't working although the download button for the most recent version works.

not part of docs anymore


tests: spi: stm32: STM32 SPI driver unit tests

This is a proposal for adding unit tests for the STM32 SPI driver.
I have seen that there are no unit-tests for drivers. I don't know if it's a Zephyr policy to not have them.
Part of the source code needs to be adapted to be able to run unit tests. The 2 main points that summarize these changes are:

Since these are unit tests, it's necessary to create "bridge" functions that can be "faked" with FFF. Essentially, this means expanding the spi_ll_stm32.h header, which contains functions such as ll_func_tx_is_empty, and replace all the platform-specific calls in the driver by these. This cannot be static inline in a header file, they need to be defined in a *.c file so the interface can be mocked.
It's necessary to add conditional compilation statements to not include certain things in the unit tests (all the driver initialization with macros such as DT_INST_FOREACH_STATUS_OKAY for instance). This is something that has already being done in bluetooth.

What I'm expecting from this draft is a shallow review to determine if these changes "violate" any Zephyr policies. If not, I will move it from 'draft' to 'PR'. I have divided it in several little commits to make the review easier, so I strongly recommend to review it commit by commit.
How to test
The tests can be run with the following command:
west twister -O ./build/twister-custom-out --no-clean --inline-logs -T tests/drivers/spi/spi_stm32/

Hi @teburd and @erwango,
I created this draft to receive a shallow review before making it a real PR. This is to verify that the principle of this draft is admissible for Zephyr, rather than receiving detailed feedback (see description). My main question is if it's problem to create an interface to decouple the STM32 SPI driver from the STMCube low level functions.
In general, if the idea of this draft is approved, I will move it to an actual PR and then the actual review will start.
Thanks.

I forgot to mention that I have more test cases, but I didn't want to add them as they are not relevant for this draft's purpose.

Kind reminer to @teburd @erwango @tbursztyka
This draft has had no activity for quite a while. Could I receive some feedback about whether or not it's feasible to merge this change, after applying the (potential) requested changes? Maybe the change is considered too large and not worthy it as its purpose is to add unit tests to a driver; in this case I would appreciate to clarify it.

This draft has had no activity for quite a while.

If it's ready to review, please move it out of draft state (which is equivalent to "not ready for review")

@dgastonochoa If it's ready to review, please move it out of draft state (which is equivalent to "not ready for review")

@dgastonochoa If it's ready to review, please move it out of draft state (which is equivalent to "not ready for review")

It requires little changes to rebase, but in general it is ready for review.

@dgastonochoa please address the failed CI checks
@erwango please revisit

Unfortunately, I won't have time to keep working on this in the short term. I still would like to give it a try, so maybe I re-open this in a few months. I will close it for now, to avoid confusions. Thanks.


Unable To Conform E-Mail Verfication

Hi, After the new update I'm unable to verify my account on signup. It keeps saying,
Something wrong ((
Please try Sign up again.

I'm the only one having this issue or not?

Try to change https://github.com/3s3s/opentrade/blob/master/server/modules/registration/signup.js#L118
utils.render(res, 'pages/registration/signup_confirm', {error: true, message: 'Something wrong (( Please try again. ('+(err.message || JSON.stringify(err))+')'});
It will show the error

I made changes as you suggested still same message & No other error has been displayed except.
Something wrong ((
Please try Sign up again.

On the server log I got,
dIjWR0ybt/JqAPaKs=Q2GkrmiCdYLlKbq2JINwcZIE40%3D
{"dIjWR0ybt/JqAPaKsQ2GkrmiCdYLlKbq2JINwcZIE40%3D":{"body":{"username":"admin","email":"admin@admin.com","password1":"admin","password2":"admin"},"time":1527577857343}}

After clicking the link.

Before the merge #42 I can conform the e-mail verification without any problem.

Try to change
https://github.com/3s3s/opentrade/blob/master/server/views/pages/registration/signup_confirm.ejs#L17
https://github.com/3s3s/opentrade/blob/master/server/views/pages/registration/signup_confirm.ejs#L18
https://github.com/3s3s/opentrade/blob/master/server/views/pages/registration/signup_confirm.ejs#L19
      <div class="row" <% var fail = ''; var errMessage = message || "(unknown)"; if (!error) fail = 'style="display:none;"'%> <%- fail %>>

Something wrong (( <%- errMessage%>

And
https://github.com/3s3s/opentrade/blob/master/server/views/pages/registration/signup_confirm.ejs#L17
console.log('Something wrong (( Please try again. ('+(err.message || JSON.stringify(err))+')');
return;

Now I got a internal mail server error Error with your mail server: queryMx ETIMEOUT I think its my mail server problem. Right now I leave this issue open, If no one else have the same problem means we can close this issue after a couple of days.


Make DropDownMenuItem hoverable by default

Currently when we show DropdownMenu, items are not highlighted under the mouse cursor:
https://user-images.githubusercontent.com/5963351/123407833-1307b080-d5b5-11eb-960d-4987a7fc1d42.mp4
It looks very unintuitive on desktop platforms. Probably we need to highlight item when mouse is over the item.
Issue for Button's: https://github.com/JetBrains/compose-jb/issues/339

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Update typicalEmployeesExecutiveProDb.json to fix java CI

Update typicalEmployeesExecutiveProDb.json with the same fields as TypicalEmployees.java, fixes #258

LGTM


Need to upgrade the go-sdk-core and the secrets-manager-go-sdk dependencies to address CVE-2024-6104

The appconfiguration-go-sdk v0.5.3 release is currently depending on the go-sdk-core/v5 v5.15.0 and the secrets-manager-go-sdk/v2 v2.0.2 which contains CVE-2024-6104.
This CVE can be resolved by upgrading the deps:

go-sdk-core/v5 v5.15.0 to >= v5.17.4
secrets-manager-go-sdk/v2 v2.0.2 to >= v2.0.5

dependencies are updated and new version is released (v0.5.4) https://github.com/IBM/appconfiguration-go-sdk/releases
thanks for opening the issue


Typo in Visual Studio Runner web page

There's a repeated typo in Visual Studio Runner web page. At least a couple of times it says Exporer instead of Explorer.
Didn't know where to notify web site issues, so we opted for Github repo. HTH

Fixed, thank you.  For future reference, the content of the documentation site is in the following repo: https://github.com/fixie/fixie.github.io

Thanks for info. By any chance, is that link already mentioned on documentation site? I've seen other projects doing that and it seems useful for flagging things like this.

It's not yet, but that's a good suggestion.

This is one (very good) example I remembered:
http://omniscientjs.github.io/api/01-omniscient-api-reference/

@BrainCrumbz  A simple change to the documentation pages' layout has enabled this feature.  Documentation pages now have an "edit this page" link similar to the one you referenced.  Thanks a lot for the suggestion!

Cool!


Custom colormap normalization in power plot

I'd like to create a classic ERD/ERS time/frequency plot by computing a TFR (e.g. via tfr_multitaper). Power should be normalized to a baseline with mode='percent'. I want to set the limits to vmin=-1 and vmax=1.5 using cmap='RdBu' (because these are the values typically used in ERD/ERS maps). This works fine, but now I'd like to map the value 0 to white; negative values should linearly scale from -1 (red) to 0 (white), and positive values should linearly scale from 0 (white) to 1.5 (blue).
I found this example on how to normalize colormaps using the norm argument of pcolormesh (the relevant line in MNE that produces the plot seems to be here). Is it possible to do this after the plot has been created (if so, how)? Or is this only possible by specifying the norm argument for pcolormesh?

no idea. Never did this.

This SO post describes how this could be done with a custom colormap - which would mean that no internal changes are necessary. However, I wonder if MNE should provide this out of the box to support creating standard ERD/ERS maps.

provide a gist by passing a custom cmap

It's pretty straightforward, but I feel that generating a standard ERDS map (as per the settings used in the Pfurtscheller papers) could be useful for some people. That's why I initially created a small package, but later realized that this should really be done by MNE (because it already has all the stuff).
Here's basically what needs to be done:
vmin, vmax = -1, 1.5
vzero = abs(vmin) / (vmax - vmin)
cmap = plt.cm.RdBu
index_old = np.linspace(0, 1, cmap.N)
index_new = np.hstack([np.linspace(0, vzero, cmap.N / 2, endpoint=False),
                       np.linspace(vzero, 1, cmap.N / 2)])
cdict = {"red": [], "green": [], "blue": [], "alpha": []}
for old, new in zip(index_old, index_new):
    r, g, b, a = cmap(old)
    cdict["red"].append((new, r, r))
    cdict["green"].append((new, g, g))
    cdict["blue"].append((new, b, b))
    cdict["alpha"].append((new, a, a))

newcmap = LinearSegmentedColormap("erds", cdict)

power.plot([0], baseline=[-2.5, -0.5], mode="percent", vmin=vmin, vmax=vmax,
           cmap=(newcmap, False))

Of course this could be factored out into a function. Would it make sense to add this to the examples so that people actually find it?

Creating a custom colormap seems a bit of a hack when IIUC norm should do exactly what you want.
Maybe we should have a norm argument that:

If centered, will use vmin and vmax and make 0 at the center of the colormap
If anything else, pass to matplotlib (default None)

And then say in the docstring that centered can be useful for (wacky) asymmetrical plots where you still want zero at the center of the colormap.

Yes, this would be really nice, but we'd still have to hack together a class that does the rescaling. I wish the Matplotlib devs implemented colors.OffsetNorm like they say on their website, this would completely solve this issue. Otherwise, if you think this is worth adding a new argument I'd be happy to implement it. If not, I'd argue for adding an example script on how to plot such ERDS maps.

They give code for how to do it with MidpointNormalize. Let's put it in mne.fixes under the assumption they'll add it eventually, then we're good to go with my API suggestion right?

my first reaction is make it an example with entry in the FAQ

To me it depends on how standard / often used it is, which I don't know
But it looks like you can modify it after the fact with mesh.set_norm, so we can go the example route pretty simply

I know a lot of people who produce such ERDS maps, so I think MNE should make this task as straightforward as possible. I kind of tend toward adding an example too, since knowing a lot of people is no indicator of how standard or how often it is used :smile:. I'll submit a PR. Am I correct in assuming that the example should go in this section?

Yeah that's the one

yes


Figure out if we can stop weird CB wallet behavior when clicking tel link

https://coinbase.slack.com/archives/CB1DBBT29/p1716314152100049

Timebox to less than a day. I wonder if it's related to target="_self"

Blocked this issue because we can't fix it on our side. CB Wallet will have to investigate this issue and create a fix on their end.

@travisbloom-cb slack link has expired, can you explain the issue here?


In Search: a.toLowerCase is not a function

While searching from JSON data I am getting this error-
    Uncaught TypeError: a.toLowerCase is not a function
    at Object.l [as search] (jstree.min.js:5)
    at String.<anonymous> (jstree.min.js:5)
    at Function.each (jquery.min.js:2)
    at a.jstree.plugins.search.search (jstree.min.js:5)
    at a.jstree.plugins.search.<anonymous> (jstree.min.js:5)
    at a.jstree.plugins.search._load_nodes (jstree.min.js:2)
    at a.jstree.plugins.search._load_nodes (jstree.min.js:5)
    at a.jstree.plugins.search.<anonymous> (jstree.min.js:5)
    at Object.i (jquery.min.js:2)
    at u (jquery.min.js:2)

JS
   function recipeTreeStruc(){
	$.ajax({
        url : myURL,
        type:'POST',
        success: function(data) {
        	$('#question').keyup(function() {
        		  var ques = $(this).val();
        		  console.log(ques);
        		  console.log(".................");

        		  $('#recipeTree').jstree(true).search(ques);
          	});
        	$('#recipeTree').jstree({
        	    'core' : {
        	    	'check_callback' : true,
        	    	'multiple' : true,
        	        'themes' : {
        	            'dots' : false         	          	},
        	        'data' : data
        	    	},
		    	'massload' : {
		    		    'url' : myURL,
		    		    'data' : function (nodes) {
		    		      return { "ids" : nodes.join(",") };
		    		    }
		    		  },
        	    'plugins' : ['dnd','contextmenu','massload','search'],
        	    'search'  :{
        	    	'case_sensitive':false,
        	    	'show_only_matches' : true,
        	    	'fuzzy' : true,
        	    	'ajax' : {
        	    		'url' :myURL,
        	    		'data' : data
        	    	}
        	    }
        	 });
           }
      });
   }

My datas are being loaded as desired, but in while search I am having this issue.What am I doing wrong?

Before I can help you:

Provide a trace using non minified version of jstree
Provide the exact jstree version you are using
A demo would help a lot as I suspect there is something wrong with your data (a node not having a text property for example)

Please give me more information and I will help.

    jstree.js:7681 Uncaught TypeError: text.toLowerCase is not a function
    at Object.search (jstree.js:7681)
    at String.<anonymous> (jstree.js:7555)
    at Function.each (jquery.min.js:2)
    at $.jstree.plugins.wholerow.search (jstree.js:7553)
    at HTMLInputElement.<anonymous> (addMultipleFields.js:139)
    at HTMLInputElement.dispatch (jquery.min.js:2)
    at HTMLInputElement.y.handle (jquery.min.js:2)

   | search | @ | jstree.js:7681
   -- | -- | -- | --
   | (anonymous) | @ | jstree.js:7555
   | each | @ | jquery.min.js:2
   | search | @ | jstree.js:7553
   | (anonymous) | @ | addMultipleFields.js:139
   | dispatch | @ | jquery.min.js:2
   | y.handle | @ | jquery.min.js:2

JS TREE version 3.3.5
jQuery version 3.3.1
demoHere

Note: While fetching value from HTML its Working;
While fetching from json data its not working

Thank you for pointing it out. Couldn't have thought of this

Hi, I have found similar issue when from back-end data was sent like this:
"children": [
  { "id": 1, "text": "abc" },
  **{ "id": 2, "text":<PHONE_NUMBER>2 },**
  { "id": 3, "text": "test 3" },
  { "id": 4, "text": "test 3" },
]

I getting data from database as PHP array with legit PHP libraries, converting it with PHP function json_encode to JSON array and sending back to the front-end. This thing works with all modules, but of course text field, which is initialized like this on JS will be considered not as string type and probably won't have toLowerCase method after this... But property with name of text then will present.

Might be worth to re-convert it to the text at the initialization or during search with String(a) function? This of course possible to avoid simply processing specific array before giving it to jsTree. It might help for someone as well.


Fix no password wrong message displayed in case of incorrect password.

currently if incorrect password is entered , no warning is displayed we have to guess it by checking if document was correctly decrypted or not. We must add a mechanism to show password incorrect to improve usability.

Closing due to Lack of developer time .

Fixed in release version 1.1


Make guide more intuitive and update with more info

I can't think of any word to appreciate this work, maybe I should just add you to the credit ¯_(ツ)_/¯


Center mode + Variable width + Infinite false

When using center mode and infinite false with variable width you get extra white space at first slide left side and last slide right side.
Here is the link for JSFiddle.
What I want is that first slide should start from left edge whereas last slide should end at right edge without any white spaces.

In case anyone come across this, here is the code I modified starting at line 1105 in slick.js :
 if (_.options.centerMode === true && _.options.infinite === true) {             _.slideOffset += _.slideWidth * Math.floor(_.options.slidesToShow / 2) - _.slideWidth;         } else if (_.options.centerMode === true && _.options.infinite === false) {             if(slideIndex >1 && slideIndex < _.slideCount - 2 ) {                 _.slideOffset = 0;                 _.slideOffset += _.slideWidth * Math.floor(_.options.slidesToShow / 2);             } else if (slideIndex ===1) {                 _.slideOffset = 0;                 _.slideOffset += _.slideWidth;             }         } else if (_.options.centerMode === true) {             _.slideOffset = 0;             _.slideOffset += _.slideWidth * Math.floor(_.options.slidesToShow / 2);         }
My slider has 5 slides, so you may need to adjust depending on how many slides you use

@leggomuhgreggo duplicate of what issue exactly?

Hmm, when I closed it, I was pretty sure it was a duplicate of a number of either centerMode + infinite: false issues or variableWidth issues.
I still think chances are good it's related to those, but I will reopen and assign to that topic just to make sure we're factoring for it when those issues get addressed

Take a look at this merge request, his solution worked for me.
https://github.com/kenwheeler/slick/pull/2635


info message on login modal is cut off

The info message on the login modal is cut off.

Duplicate.


Rename curate to map

Deprecate it first. Proper transition

I think map would be a different function that takes a list of identifiers and returns a mapped dict. While curate is an inplace function that takes a df and returns a df indexed with curated ids.

This sounds great!


Post Moderation: Internal dashboard

What does this pull request aim to achieve? 💯
Adds internal moderation dashboard.
Please be concise and link any related issue(s):
closes #1882
closes #1943

[x] I have checked that no one else is working on similar changes.
[x] I have read the latest README and understand the development workflow.
[x] The name of this branch is: feature/<branch_name>, or hotfix/<branch_name> if this is a hotfix, where <branch_name> briefly describes the issue(s) resolved and is prefixed with the issue number(s) (e.g. 1127-1130-update-git-branching-model). The branch is created in a cloned version of this repo, not a forked repo.
[x] This branch is rebased/merged with the latest staging (or the latest production if it is a hotfix).
[x] There are no merge conflicts.
[x] There are no console warnings and errors.
[x] On the right hand side of this PR, I have linked the appropriate issue(s) under "Issues" and added a project progress under "Projects".
[x] The title describes the issue(s) being addressed using format: <issue_numbers> - <brief_issue_description> (e.g. #1127/#1130 - Update Git Branching Model)

@Zaydme @OzkanAbdullahoglu
Observed few more issues while testing for Post -moderation:
tested on mobile/iPhone, working fine no UI issue observed.

UI issue on profile page (same as it was on help board) Refer below screenshot - Not fixed

The button name in 'Delete' pop up is changed, it should be 'Delete' only, currently it is displayed as 'Delete your post' refer below screenshot.

The  names for Edit and Delete options needs to be updated as suggested in below screenshot.

Hi @rabbi08 ,
I fixed the texts as below,

I couldn't reproduce the first issue, can you elaborate?

Okay. Can you try again? @rabbi08

@OzkanAbdullahoglu
Verified the issues mentioned above. They are working fine. Please see below GIF and screenshot.

@Zaydme I have following observations for during initial testing of post moderation dashboard:

After we decide to 'Keep Post' -> the post is moved to 'Kept Post'
Issue: It should also be visible to the users like normal post on the Help Board.
Currently it is NOT visible to the user who reported it even after 'Keep Post' action.

Refer below GIF.

When user try to login using social media account -> the page direct to create profile page and then automatically direct the user to FP home page without login. I tried this scenario in STG, it is working fine but not here.
Please see GIF.

While using -> Google -> it navigate to create profile page -> clear the profile page -> do nothing
When I click on FP logo, and I land on FP page, I am logged in using google login

when there is NO search results to display, the message is displayed as "There are no "Pending" reports."
It should be something like "There are currently no matching posts available." as we have used word 'Post' with our labels in the dashboard.

Navigate to 'Audit Log' -> click on 'View Post' for any post -> Click on 'BAck' button
Issue: Users lands on Help Board page, it should be navigated back to the page from where he navigate i.e. 'Audit log' page.

On Dashboard -> user is able to view the post by click on the post description, even if they are read only.

As a 'Reader' ->

Following options are not displayed: PASS

Audit Logs: Not displayed
Manager Users: Not displayed

Can Only view cannot perform 'Keep' or 'Remove' actions: PASS

Note: Reader can have post view rights.
As a 'Moderator' ->

Following options are not displayed: PASS

Audit Logs: Not displayed
Manager Users: Not displayed

Can 'Keep post'
Can 'Remove' and 'Restore' post
After 'Restore' from 'Remove' the post is moved from "Removed -> Kept" : PASS

As a 'Inspector' ->

Following options are not displayed: PASS

Manager Users: Not displayed

Can NOT perform 'Keep post', 'Remove' and 'Restore' post: PASS
Can View 'Audit Logs'.

As a 'Administrator'

Can view all options and perform all the actions
Cannot edit other 'Admin' users: PASS

Thanks @rabbi08
1: We agreed that if a user reported a post, they should never see it again, even if a moderator kept it. If a user reported a post, they definitely don't want to see it again, they either think it is offensive or something.
2/3/4 I will look into them.
5: is expected behavior, anyone with access to the dashboard ("reader" or above) has the right to see reported and removed posts, even if they are "Read only". "Read only" means that you can't take action, but you can always "Read"

I apologize for not being active on slack/github lately. I am just a little bit busy with school 😅😅
I will be back as soon as I can!

Will be looking at this soon. Let's get this done!

Will be looking at this soon. Let's get this done!

Will be looking at this soon. Let's get this done!

rebase or merge the branch to the staging seems like a good idea 😁

Will be looking at this soon. Let's get this done!

rebase or merge the branch to the staging seems like a good idea 😁

tested with service/postmoderation-test review build, email notification works fine

tested with service/postmoderation-test review build, email notification works fine


Add paper sharing options

Added paper social sharing options for #465.

sharing url from current pdf links: Currently hardcoded for ACL anthology, MIT press journal pdf links and arxiv abstracts.
current message format is page_header+sharing_url. See below for an example tweet

@tlikhit Awesome! Thanks!
Maybe remove btn btn-primary from the "Share". The style is a little weird right now.

I guess something like this should be fine, but let me know if you have better ideas

@hao-fang Ok, updated styling to make it more consistent :)

@tlikhit Awesome! Thanks!


Lock CI

Summary

Ticket Link

Related Pull Requests

Has server changes (please link here)
Has redux changes (please link here)
Has mobile changes (please link here)

Screenshots

Trying to auto merge this PR.

Pull Request successfully merged
SHA: 02943efe04620b4cdf3be85c1e38be1b8681e105


_mm_maskload_ps non-native implementation can lead to illegal memory access

The culprit being here, where the whole memory is accessed first, before applying the mask:
https://github.com/simd-everywhere/simde/blob/c900d5e195c4692e0f6b1c3ad8e18e577cf55e11/simde/x86/avx.h#L4011
from the corresponding x86 docs:

...
The mask bit for each data element is the most significant bit of that element in the first source operand. If a mask is 1, the corresponding data element is copied from the second source operand to the destination operand. If the mask is 0, the corresponding data element is set to zero in the load form of these instructions, and unmodified in the store form.
...
Faults occur only due to mask-bit required memory accesses that caused the faults. Faults will not occur due to referencing any memory location if the corresponding mask bit for that memory location is 0. For example, no faults will be detected if the mask bits are all zero.
...

Code to reproduce
main.c:
#include <simde/x86/avx.h>
#include <stdio.h>
#include <sys/mman.h>

int main(void)
{

    float *ptr = mmap(NULL, 8 * sizeof(float), PROT_NONE , MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    const __m256i mask = _mm256_set_epi32(0, 0, 0, 0, 0, 0, 0, 0);
    __m256 test = _mm256_maskload_ps(ptr, mask);
    printf("test: %f\n", test[1]);
}

compile on arm64 silicon (this executable will fail with Bus Error):
clang -o repro -Isimde -DSIMDE_ENABLE_NATIVE_ALIASES main.c

compile on intel (this executable will run fine):
clang -o repro -Isimde -march=native main.c

big thanks to @svenpeter42 for helping a lot with debugging this and actually pinning down the issue :)

Thank you @k-dominik for finding this! Can you send a PR to fix this?


docs: Remove/Relax "Fedora Silverblue" in README description

This is a starting point Fedora Silverblue image designed to be customized to whatever you want

As the recipe is based on silverblue-main, this is correct.
Still, it might not be the  best wording for the description section of the startingpoint repository. For example, a user might wonder if it doesn't work for Kinoite (I know that this is clarified later in "Customization").
I'm still not 100% clear on what ublue OS (and thus the startingpoint) really is or shall be, so I find it hard to find a better wording myself. I currently see these options:

Fedora ostree + extensions
OCI defined OS which just happens to be Fedora but could be anything else
A framework/toolset to define/maintain an OCI defined OS

I see that the original topic for this issue led to very fundamental considerations which probably shouldn't even be in this repo, so sorry for that. Nevertheless, I hope my point became clear.
EDIT: Just saw https://github.com/orgs/ublue-os/discussions/125.

Hope this cleaned things up


Empty module -- requesting some love

It looks like the ZF2 implementation is empty. Is there any planned activity for this library?

ping @Gounlaf

Work still in progress here. It's working, but unit tests are required.

Just submitted a PR. Unit tests, CS checks, everything :)
Let me know your thoughts

Sweet, I'll check it out this week.

Fixed in #2

Delayed love is still love! Thanks :-)


SettingGateway Updates

All changes are untested and may not work, this PR is a backport of the current Komada 0.20.7's SettingGateway for Klasa, including its bugfixes, enhancements and multi-SG support.
Tested:

[x] CacheManager
[x] SQL
[x] SchemaManager
[x] SettingGateway
[x] SettingsCache

Todo:

[x] Default prefix (if it does not exist in the settings).
[x] Default language (if it does not exist in the settings).

All changes seem to be working correctly. Feel free to merge now.


let any layout able to be homepage

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Improvement (refactoring and improving code)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Description

separate posts from the homepage:

other suffixes of index is able to use
now any layout can be homepage by change page.layout of index.html
add posts layout same as the old home layout
retained the home layout for compatibility

Additional context
Known issues:

non index posts can not be paginated

neither of jekyll-paginate or jekyll-paginate-v2(production) can not work it out

Todo:

remove home layout

a breaking change

this is my very first pr.
sorry about commitlint and everything.
closing it and open a new one


Add gazetteer creation guide

Added a guide to gazetteer creation. Both the text and the code is a bit rough so feel free to suggest extensive changes if need be.
I'm also wondering if once you've reviewed whether we should make a docs version for this branch and get Sam Lee's feedback on this?
Fixes #125

Waiting on feedback from Sam.


Issue with single SAN provisioning

Hi,
I'm trying to use this module with the only one SAN, please see  below
data "aws_route53_zone" "this" {
  name         = "example.com"
  private_zone = false
}

module "acm" {
  source = "git::https://github.com/terraform-aws-modules/terraform-aws-acm?ref=v2.4.0"

  domain_name = "example.com"
  zone_id     = data.aws_route53_zone.this.zone_id

  subject_alternative_names = [
    "*.api.example.com",
  ]

  wait_for_validation = true
  validation_method   = "DNS"
}

and it fails with error:
module.acm.aws_acm_certificate_validation.this[0]: Creating...

Error: 1 error occurred:
	* missing *.api.example.com DNS validation record: _xxxxxxxxxxxxxxxxxxxxxxxxxxxx.api.example.com

  on .terraform/modules/acm/main.tf line 40, in resource "aws_acm_certificate_validation" "this":
  40: resource "aws_acm_certificate_validation" "this" {

terraform version
Terraform v0.12.6
+ provider.aws v2.35.0

At the same time, if there are more then one records in subject_alternative_names no issue exists

This has been fixed in #32, please use v2.5.0 of this module.


How to NOT install local bower & grunt?

A question: I have bower & grunt installed globally on my server. What would be the best approach to not install them locally, to minimize disk space usage?

I would have a local development server and then only deploy the theme files not the node and bower files, or even scss For example I have a local WP that I use as dev. I then would setup a separate grunt command to push to the test / production server.
Other than that I would use an ftp and only deploy the files, folders you need for the theme.

@pmccauley , thanks for your answer. Not quite what I was looking for. What I meant was how to run npm install without installing bower & grunt as local dependencies.

You could look at the changes that where made in https://github.com/olefredrik/FoundationPress/commit/85e631db7a7b9ded92a37caf48f38407ab0ed513 where the global dependencies where removed.

@Aetles, that's what I thought.. So it looks I have to install the 80 megabytes of node_modules for every theme development project. It was just mere 24 megabytes before the global dependencies were removed.

Yeah, that's a drawback but I still think it's better than using global installations. John Albin Wilkins wrote a convincing article about why installing packages globally is not recommended.

Ok, thanks, good read, I'm convinced, though I've never had any problems with global installations .


Can Discussions be enabled on this repo?

@rohityadavcloud Figured it might be good to have the Discussions on this repo for when there are topics or questions, that doesn't necessarily need to be an issue

@kiranchavala @vishesh92 @rohityadavcloud  I see that the v0.5.0 milestone is 100%.  That is awesome!  Just wondering when this is going to be released and published?

@CodeBleu we generally discuss on mailing lists that would be encouraged and we also have all general CloudStack discussions here https://github.com/apache/cloudstack/discussions
Kiran had started a discussion on the dev and users mailing list and then the alpha (pre RC) is currently out for testing for any blockers or regressions https://registry.terraform.io/providers/cloudstack/cloudstack/0.5.0-pre

@CodeBleu I am planning to cut 0.5 RC 1 and start the voting process on  April 1st

I see v0.5.0 on Terraform Registry,  but this repo  release section still shows a 2 year old v0.4.0

The v0.5.0 RC is up for vote @CodeBleu have you subscribed to the dev mailing list ? https://cloudstack.apache.org/mailing-lists
Once the vote passes, the release manager (@kiranchavala) will publish the git tag and create a Github release.

Closing this as https://github.com/apache/cloudstack/discussions can be used and Issue on this repo can be used for any other specific things to this repo/sub-project.


Suggestion: Support for GNU tar format

Hello,
As mentioned in https://github.com/kamranzafar/jtar/issues/6, jtar is not fully compatible with the GNU tar format. As gnu tar finds use on popular linux distributions such as ubuntu, you may want to consider supporting it in the future. It is possible to create ustar format archives with the gnu tar tool, though it is not the default.
cheers

Yeah, I met the problem of having my file name truncated to at most 100 characters.
This article explains some history background:
http://ftp.gnu.org/old-gnu/Manuals/tar/html_mono/tar.html#SEC119


Stabilize WS tests

The tests were failing time to time for me saying this:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x9f9f27]

goroutine 23 [running]:
github.com/surrealdb/surrealdb.go/pkg/gorilla.(*WebSocket).initialize.func1()
	/home/mumoshu/p/surrealdb.go/pkg/gorilla/gorilla.go:205 +0xe7
created by github.com/surrealdb/surrealdb.go/pkg/gorilla.(*WebSocket).initialize
	/home/mumoshu/p/surrealdb.go/pkg/gorilla/gorilla.go:196 +0x69

The paniic points to https://github.com/surrealdb/surrealdb.go/blob/main/pkg/gorilla/gorilla.go#L205, and I see ws.logger being nil when the panic happens.
We should make the logger non-nil, or make the ws.read goroutine wait until the logger is initialized. Otherwise, every time ws.read gets an error from the connection (e.g. the connection is closed?), AND the logger is nil (which is always the case for our no-opt gorilla ws impl in the tests), the panic happens.
That's said, let's always initialize the logger for now.
We may make the logger getting lazily initialized on demand, but I think that's not a priority for now :)

I have identified that there is a bug in the code causing std.out to be used instead of https://github.com/surrealdb/surrealdb.go/blob/main/pkg/logger/logger.go#L51C2-L51C2 when the path is empty. Rest assured, I will create a PR to fix this issue promptly. Thank you for bringing it to my attention.


Adjust the order of judgment

Hello, submit the public relations background is this:
In the ribbon sample-client.ribbon.OkToRetryOnAllOperations, this parameter indicates whether it is possible to retry all operations for this client(I think include the GET request).
When a retry occurs, we should really only judge if the parameter is true (because the meaning of this parameter is whether all operations should be retries), maybe I understand the mistake, I hope to get your answer.
So I submitted this PR, thank you, best wishes.

@ryanjbaxter Look forward to your reply, thank you very much.

Codecov Report

Merging #2514 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master    #2514   +/-   ##
=======================================
  Coverage   65.65%   65.65%
=======================================
  Files         207      207
  Lines        7646     7646
  Branches      943      943
=======================================
  Hits         5020     5020
  Misses       2268     2268
  Partials      358      358

Impacted Files
Coverage Δ

.../netflix/ribbon/RibbonLoadBalancedRetryPolicy.java
83.33% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05cd507...a49cedb. Read the comment docs.


Fix scim2/users list issue

Purpose

Revert the fix introduced with https://github.com/wso2/carbon-kernel/pull/3509. (Due to regression)
Improve getUserWithID method to use userUniqueIDManger.getUser(userID, this, userStore.getDomainName()).

Related Issues

https://github.com/wso2/product-is/issues/15656

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/4684747572

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/4684747572
Status: success


Elasticache's Redis with Chatwoot

I am having trouble in using Elasticache's Redis cluster (with cluster mode on or off both).
Can anyone please provide me a detailed step for it, it is not working as simply Redis does.

@shaheer-haider  could you add details about what error you are getting ?


[Suggestion] Improve tag display

Thanks again for adding the tags header, loving it =)
There's two small things I thought of while actually using it in regards to mailpit's frontend:

I feel it would look better if the tags were listed on a new line below the mail subject instead of in front of it. It's fine with one or two, but as soon as there are more it looks (imo) a bit odd and cluttered.
I think colored tags would improve visually 'scanability' of a longer message list on a central mailpit server. I have two levels of suggestion for it (depending on how far you want to go if you are willing to implement it):
-- Basic: color by hashed name.
-- Manual color from name: A tag called something akin to <#FF0000>I am red would be colored red (and the color part stripped for displaying the name)

Nice suggestions @tminich, thanks.
Tag placement
I get your point, and yes, this could be done with relative ease and makes sense to me. I assume you mean something like this?

For the most part that won't add must additional space usage as most emails have at least two lines of from/to addresses. It's only when dealing with longer email subjects is there the potential of additional space being taken up, and of course mobile view (though I'm not too sure how many actually view Mailpit from their phones).
Tag colors
This is a more difficult one. A manual color as part of the tag name isn't an option as tags are physical stings stored in the database with every message, and grouped & filtered as such.
Setting a generated color based on their hash is possible however. Each tag color here is calculated via color-hash using the tag name. In my first test it was too (CPU) expensive to generate the hash for each tag displayed on-screen when receiving a stream of new messages, however I changed that to cache color results in an array and use the array to look up the tags - this was much faster (in fact similar to without any colours). See below for my first color test:

My concern here not everyone may appreciate the vibrant colors. Whilst they are (in my test above) limited to a certain darkness to ensure readability, still many of them are too light to pass accessibility standards.
I played a bit more with the colours to get them even darker which proved more successful, although the color differences between them are less:

I personally find these colors a bit too dominating int he UI to be honest. Ideally speaking this would be something that could be toggled on/off, but in order to achieve that I have to build an entire personalisation section to the Mailpit UI which wasn't in the planning, and which becomes a much larger task for (currently) just one feature. It is also worth mentioning that these tag colours are limited only to the message overview, not the tags listed down the side, nor the manual editing of tags in the message display. What are your thoughts about the tests above, and was this what you had in mind?

Placement looks exactly as I imagined, perfect 👍
In regards to manual colors, to elaborate, the way I intended them was doing it the cheap way (from a programmers perspective). Internally the color tag would be, for all intents und purposes, be part of the name. Only the the displaying code would parse the tag and strip it from the (displayed) name.
Automated colors look fine by me. I personally don't mind more vibrant colors, but I agree that not everyone might agree. A user setting would of course be best to cover all bases. I don't think a whole system needs to be built around it. Mailpit is first and foremost (imo) a developer tool, so something simple like storing it in localStorage on browser level should (imo) be fine.
I fine with only the tags in the list having color, but in regards to the tag list something simple like this could work:

While I have never worked with go, Vue is the frontend framework I'm using for all our projects and thus am most familiar with. I'm happy to take a look once you settled on the basics if you want to =)

I've just released v1.6.18 which includes this optional color feature (plus of course the labels below the subject line). I'd say it's still "experimental" at this stage, mainly to get your feedback. Limiting colors wasn't as easy as I'd hoped, as they need to remain dark enough to read the tag text, but in the end this is more-or-less the type of colours it is generating:

The feature can be enabled/disabled here:

Please let me know what you think?

Personally I wouldn't mind them being more vibrant, but I can see how others would disagree. All in all seems perfectly fine to me, great job 👍


[Bug]: List.removeAt doesn't work

What happened?
when use list.remove data cannot be deleted
data  should be deleted.
Repro steps
FutureOr<void> removeCoordinatesImage(CoordPoint point, int index) async {
    var image = File(point.imagePaths[index]);
    if (image.existsSync()) image.delete();
    point.realm.write(() {
      point.imagePaths.removeAt(index);
    });
  }

imagePaths.length will not change
imagePaths type is List<String>
Version
0.4.0-beta
What Realm SDK flavor are you using?
Local Database only
What type of application is this?
Flutter Application
Client OS and version
Android 10
Code snippets
No response
Stacktrace of the exception/crash you're getting
No response
Relevant log output
No response

@Comanx The fix was just released with 0.5.0+beta


thread 'main' panicked at 'Unable to find libclang:

I've tried to flash my esp32 but i encountered issue
i've installed followed instruction in readme and installed espflash  and generated basic esp-idf-template
also i've installed libclang with sudo apt-get install -y libclang-dev
cargo run  output: bug.txt
Os:  Pop!_OS 22.04
Rust: rustc 1.70.0-nightly (f112def22 2023-05-31) (<IP_ADDRESS>)

Can you check if you have the LIBCLANG_PATH environment variable set, that variable must be set by sourcing the environment file, see https://esp-rs.github.io/book/installation/index.html#3-set-up-the-environment-variables

LIBCLANG_PATH is set to $HOME/.rustup/toolchains/esp/xtensa-esp32-elf-clang/esp-16.0.0-20230516/esp-clang/lib

LIBCLANG_PATH is set to $HOME/.rustup/toolchains/esp/xtensa-esp32-elf-clang/esp-16.0.0-20230516/esp-clang/lib

Are you building the project in the same terminal where you source the export file or print the LIBCLANG_PATH? Also, check the content of that folder just in case is empty (?)

found out $HOME/.rustup/toolchains/esp/xtensa-esp32-elf-clang/esp-16.0.0-20230516 is empty

Try reinstalling the environment with espup, do you remember having any issues during the installation?

I will be closing the issue, feel free to reopen the issue if you encounter any issue after reinstalling.

i have no longer issue with LIBCLANG_PATH but now i have enqunterd this build error:
error[E0063]: missing field `home_chan_dwell_time` in initializer of `esp_idf_sys::wifi_scan_config_t`
  --> /home/horryportier/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp-idf-svc-0.46.0/src/wifi.rs:93:13
   |
93 |             Self {
   |             ^^^^ missing `home_chan_dwell_time`

For more information about this error, try `rustc --explain E0063`.
error: could not compile `esp-idf-svc` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...

i have no longer issue with LIBCLANG_PATH but now i have enqunterd this build error:
error[E0063]: missing field `home_chan_dwell_time` in initializer of `esp_idf_sys::wifi_scan_config_t`
  --> /home/horryportier/.cargo/registry/src/index.crates.io-6f17d22bba15001f/esp-idf-svc-0.46.0/src/wifi.rs:93:13
   |
93 |             Self {
   |             ^^^^ missing `home_chan_dwell_time`

For more information about this error, try `rustc --explain E0063`.
error: could not compile `esp-idf-svc` (lib) due to previous error
warning: build failed, waiting for other jobs to finish...

See https://github.com/esp-rs/esp-idf-template/issues/139. It should be fixed by adding a patch for esp-idf-svc .


Streaming / Broadcasting

I am trying to use the media server to broadcast a video stream it receives from a client. I have a successful connection between the client the and the media server. My aim is to get the server to then broadcast the stream It is getting to any other device in the network but so far no luck. When I make a connection with broadcast (var ws = new WebSocket(url, "broadcast")) I get the message below:
[0x700007cb5000][1586391836.584][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.584][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.778][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.778][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.811][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.811][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.951][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.951][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:1665230601,rid:'']
[0x700007cb5000][1586391836.953][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [1665230601]
[0x700007cb5000][1586391836.953][DBG]-DTLSICETransport::onData() | Unknowing group for ssrc trying to retrieve by [ssrc:2355286230,rid:'']
[0x700007cb5000][1586391836.953][WRN]-DTLSICETransport::onData() | Unknowing group for ssrc [2355286230]

I tried looking into this  https://github.com/medooze/media-server-node/blob/master/examples/streamer.js but still no luck. Can someone guide me on a way to set up streaming to other devices from the media server. I know the /svc/ receives a stream and sends it back to the person who sends the stream. I am trying to send the stream to the server and then have the server broadcast the stream to other device.

can you post the code you are using?

Yes. On the client side, I have this
    initStream = ()  => {
        const url = "wss://"+window.location.hostname+":8000/";
        this.pc = new RTCPeerConnection();
        console.log(this.pc);
        var ws = new WebSocket(url, "broadcast");
        ws.onopen = () => {
            console.log("opened");
            console.log(url);
            navigator.mediaDevices.getUserMedia({
                audio: false,
                video: true
            }).then((stream) => {
                console.log(stream);
                this.gotStream(stream);
                //Add stream to peer connection
                this.pc.addStream(stream);
                return this.pc.createOffer();
            }).then((offer) => {
                console.log("create offer success");
                this.sdp = offer.sdp;
                this.pc.setLocalDescription(offer);
                console.log(this.sdp);
                ws.send(JSON.stringify({
                    cmd		: "OFFER",
				    offer		: this.sdp
                }));
                ws.send(JSON.stringify({
                    cmd		: "SELECT_LAYER",
                    spatialLayerId	: 1,
                    temporalLayerId	: 2
                }));
            }).catch((error) => {
                console.log(error);
            });
        };

        ws.onmessage = (event) => {
            const msg = JSON.parse(event.data);
            console.log("The message answer");

            console.log(msg.answer);

            this.pc.setRemoteDescription(new RTCSessionDescription({
                type:'answer',
				sdp: msg.answer
            }), function() {
                console.log("JOINED");
            }, function (error) {
                console.error("Error joining", error);
            });
        }

        ws.onerror = (error) => {
            console.log("error", error)
        }
    };

And on the server side, I just cloned the repo from here:
https://github.com/medooze/media-server-demo-node
I haven't changed anything so the index.js is still the same as well as all the js files in lib. The broadcast.js is this:
//Get the Medooze Media Server interface
const MediaServer = require("medooze-media-server");

//Get Semantic SDP objects
const SemanticSDP	= require("semantic-sdp");
const SDPInfo		= SemanticSDP.SDPInfo;
const MediaInfo		= SemanticSDP.MediaInfo;
const CandidateInfo	= SemanticSDP.CandidateInfo;
const DTLSInfo		= SemanticSDP.DTLSInfo;
const ICEInfo		= SemanticSDP.ICEInfo;
const StreamInfo	= SemanticSDP.StreamInfo;
const TrackInfo		= SemanticSDP.TrackInfo;
const Direction		= SemanticSDP.Direction;
const CodecInfo		= SemanticSDP.CodecInfo;

//Create new streamer
const streamer = MediaServer.createStreamer();

//Create new video session codecs
const video = new MediaInfo("video","video");

//Add h264 codec
video.addCodec(new CodecInfo("h264",96));

//Create session for video
const session = streamer.createSession(video,{
	local  : {
		port: 5004
	}
});

module.exports = function(request,protocol,endpoint)
{
	const connection = request.accept(protocol);

	connection.on('message', (frame) =>
	{
		//Get cmd
		var msg = JSON.parse(frame.utf8Data);

		//Get cmd
		if (msg.cmd==="OFFER")
		{

			//Process the sdp
			var offer = SDPInfo.process(msg.offer);

			//Create an DTLS ICE transport in that enpoint
			const transport = endpoint.createTransport({
				dtls : offer.getDTLS(),
				ice  : offer.getICE()
			});

			//Set RTP remote properties
			transport.setRemoteProperties({
				audio : offer.getMedia("audio"),
				video : offer.getMedia("video")
			});

			//Get local DTLS and ICE info
			const dtls = transport.getLocalDTLSInfo();
			const ice  = transport.getLocalICEInfo();

			//Get local candidates
			const candidates = endpoint.getLocalCandidates();

			//Create local SDP info
			let answer = new SDPInfo();

			//Add ice and dtls info
			answer.setDTLS(dtls);
			answer.setICE(ice);
			//For each local candidate
			for (let i=0;i<candidates.length;++i)
				//Add candidate to media info
				answer.addCandidate(candidates[i]);

			//Get remote video m-line info
			let audioOffer = offer.getMedia("audio");

			//If offer had video
			if (audioOffer)
			{
				//Create video media
				let  audio = new MediaInfo(audioOffer.getId(), "audio");
				//Set recv only
				audio.setDirection(Direction.INACTIVE);
				//Add it to answer
				answer.addMedia(audio);
			}

			//Get remote video m-line info
			let videoOffer = offer.getMedia("video");

			//If offer had video
			if (videoOffer)
			{
				//Create video media
				let  video = new MediaInfo(videoOffer.getId(), "video");

				//Get codec types
				let h264 = videoOffer.getCodec("h264");
				//Add video codecs
				video.addCodec(h264);
				//Set recv only
				video.setDirection(Direction.RECVONLY);
				//Add it to answer
				answer.addMedia(video);
			}

			//Set RTP local  properties
			transport.setLocalProperties({
				audio : answer.getMedia("audio"),
				video : answer.getMedia("video")
			});

			//Create new local stream with only video
			const outgoingStream  = transport.createOutgoingStream({
				audio: false,
				video: true
			});

			//Copy incoming data from the broadcast stream to the local one
			outgoingStream.getVideoTracks()[0].attachTo(session.getIncomingStreamTrack());

			//Get local stream info
			const info = outgoingStream.getStreamInfo();

			//Add local stream info it to the answer
			answer.addStream(info);

			//Send response
			connection.sendUTF(JSON.stringify({
				answer : answer.toString()
			}));

			//Close on disconnect
			connection.on("close",() => {
				//Stop
				transport.stop();
			});
		}
	});
};

It doesn't seem to be doing any broadcasting or creating a stream. I thought it would be similar to https://github.com/medooze/media-server-node/blob/master/examples/streamer.js

The streamer demo is doing the reverse way, from rtp broadcasting to webrtc.
For how to setup the incoming stream from webrtc correctly check:
https://github.com/medooze/media-server-node/blob/master/manual.md

The example in the document shows how to set the outgoing stream to the same client that sent the stream. There is no documentation on how to get the incoming stream and set it as outgoing stream/broadcast it to other clients that connect to the server

you are right, there is no documentation.


Fix Reorder2Default

Description
Fix issue #10809.
@zheng-da @pengzhao-intel @ashokei Please review.
@dwSun Feel free to try if you are instereted in building from source.
Checklist
Essentials

[ ] Passed code style checking (make lint)
[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] For user-facing API changes, API doc string has been updated. For new C++ functions in header files, their functionalities and arguments are well-documented.
[ ] To my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

looks good.

quick response.


If kubelet is configured with node-ip do not querry cloud provider

Do not ping cloud provider for node address if we are ignoring them.

@kokhang I agree, vSphere cloud provider will have to be re-visited and made more robust for not relying on vCenter every time a call is made to it.


Fabric Support?

Hey there, Im a fan of this mod for so long, but lately Ive been mostly using ViaFabric on mc 1.16.1, so I wanted to ask for some mc 1.16.1/Fabric support ;)
Thanks for this amazing mod!

lol #84 xD


Update Composer dependencies (2021-10-03-00-11)

Visual regression test passed!
View the visual regression test report


get a better versioning solution

Currently, to create a new release, I do a find-and-replace in my text editor to change all of the versions, which is tedious. We should figure out a better solution for bumping the version.

It feels like https://yarnpkg.com/api/modules/plugin_version.html should resolve this issue, but I haven't had much success with it just yet.

Note that we've never updated the version in the cli index.ts file

Git hooks could be a good solution, but I don't have a lot of experience with it.