npm server in GitHub Action stage responding with HTTP status 403

Description
Error
Failed to load resource: the server responded with a HTTP status of 403 (Forbidden)
Context
The issue is happening on a private repo when we updated from Ubuntu 18.04 to 20.04 it failed. We tried updating from Ubuntu 18.04 to 22.04 and latest. But we are getting HTTP status of 403 (Forbidden). This is causing our tests to fail.
We have already create a support ticket https://support.github.com/ticket/personal/0/2124288 for more information on the build links.
Note: It is a private repo and we are using GitHub team plan.
Platforms affected

[ ] Azure DevOps
[X] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[X] Ubuntu 20.04
[X] Ubuntu 22.04
[ ] macOS 11
[ ] macOS 12
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
2.303.0
Is it regression?
2.303.0
Expected behavior
The npm server is  responding with a HTTP status of 403 (Forbidden)
Actual behavior
The npm server should  respond with a HTTP status of 200
Repro steps

Update GitHub Actions workfile ubuntu image from 18.04 to 22.04 runs-on: ubuntu-22.04
step - uses: actions/checkout@v2
step - Use Node.js 16.13.1
step - actions/setup-node@v2
step - with:                node-version: '16.13.1'
step - npm run server
npm starts running server on localhost:8081
The npm server on port 8081 responds with HTTP status 403  (Forbidden)

@aseggemu-yoc please provide actual repro steps. There are no reason to share private project to do it as I see.

@aseggemu-yoc please provide actual repro steps. There are no reason to share private project to do it as I see.

I updated the reproduction steps. More details are in the build link of the github support.

@aseggemu-yoc , can you please try  adding sudo systemctl stop mono-xsp4 before test invocation ?

@ilia-shipitsin I tried it
  sudo systemctl stop mono-xsp4
  shell: /usr/bin/bash -e {0}

but it did not solve the issue. I added extra reproduction step if it helps as we use .unref(); to detach the server.

Hello @aseggemu-yoc. Sorry for the late response. I have some questions regarding to the issue:

Do you use any kind virtualisation ?
Does the issue reproduce only for port 8081 or others too ?
Does the issue reproduce for other Nodejs versions ?

I tried to reproduce the issue I created a simple server. Through the child_process I run server as a child process and unref() it. I could access it and get information from it in the parent process and the next step. Could you please correct me If I have a misunderstanding in my set up ? If this looks right, could you please provide a public repository?

Hello @aseggemu-yoc. Sorry for the late response. I have some questions regarding to the issue:

Do you use any kind virtualisation ?

We use docker compose for the virtualisation of the server comprising of php and nginx docker images as test server which listens to port 9000

Does the issue reproduce only for port 8081 or others too ?

The issue is also on port 3000 where we have a server running for ngrok

Does the issue reproduce for other Nodejs versions ?

No, we did not try other versions as we have projects on Node.JS version: 16.13.1. We would like to use only the version 16.13.1.

I tried to reproduce the issue I created a simple server. Through the child_process I run server as a child process and unref() it. I could access it and get information from it in the parent process and the next step. Could you please correct me If I have a misunderstanding in my set up ? If this looks right, could you please provide a public repository?

You are using it correct. We cannot provide a public repository. As mentioned our repo is a private repository.

@aseggemu-yoc could you please provide a minimal public repo (not mandatory consisting a code you use privately) so we could make sure we can reproduce your issue. Currently, if I understand correctly @dmitry-shibanov could not reproduce your issue.

@aseggemu-yoc could you please provide a minimal public repo (not mandatory consisting a code you use privately) so we could make sure we can reproduce your issue. Currently, if I understand correctly @dmitry-shibanov could not reproduce your issue.

I'm unable to create a minimal public repo as most of the code is private. It creates a lot burden on me to create new GitHub actions, new repo. I appreciate the GitHub internal team to coordinate and look into the private repo where the issue is happening.

The GitHub support staff was given access to the private repo. More details in the support ticket.
I'm closing this as the issue is not solved and I lost trust in GitHub actions.


Fix memory leak and crash

In http_send_error2 settings actually got never passed on to http_send_error, causing a crash.
Also fio_free was never called for fio_malloc of r.
All those other commits appear here. seems like i messed up branches, please cherry pick 1d1512f
1d1512f
and  e2e0a95
(I forgot a line, sigh)

Ill recreate all my PRs

ill recreate this one


Jest: Unexpected token export

First off, thank you so much for your help thus far. I'm doing my best to contribute back as much as possible as well as researching the issue before I create one here. I did a lot of digging around using Jest with CRA (create-react-app) however it looks like the issue is with your library according to Jest.
I wish I knew more about it but I'm new to testing and from what I can tell it doesn't like the way that you're exporting sortable from the file.
I'm opening this issue because:
When I run a test using Jest I get the following error
  ● Test suite failed to run

    /node_modules/html5sortable/dist/html5sortable.es.js:1078
    export default sortable;
    ^^^^^^

    SyntaxError: Unexpected token export

supporting information:

I use sortable version: 0.9.2
I tested in the following browser (incl. versions): N/A
Windows, OS X/macOS, or Linux?: macOS

How can the issue be reproduce the problem?

Setup a React app using create-react-app
npm install html5sortable
include the file import sortable from './node_modules/html5sortable/dist/html5sortable.es.js';
run npm test

Well I found out what the issue was here as well, funny what a lunch break will do for your mind. I'm using NodeJS and according to this stack overflow post NodeJS uses CommonJS not ES6 Modules.


Upgrade to pnpm

Description
Updated the pkgManager to pnpm for trial as discussed in #419
For deploying contract you need to use pnpm hardhat:deploy instead of just pnpm deploy because deploy seem reserved keyword
Will try to find if there exist any solution to above problem 🙌....But I think its ready to test and give a trial

I tried base scripts chain, hardhat:deploy, start, vercel and some others and everything works as expected 👍 . pnpm hardhat:deploy is a little bit longer than it was, but it becomes more clear what is deployed, at least for newbies.

and it shows me error when I use ctrl + c in pnpm chain terminal.

As I understand it doesn't affect anything. Fast googling didn't help me but we need to try to get rid of it.

Great job @technophile-04! Thank you for putting this together.
Tested it and everything works great.

For deploying contract you need to use pnpm hardhat:deploy instead of just pnpm deploy because deploy seem reserved keyword

Damn, that's going to be a hard pill to swallow for some people haha. The deploy command is the most used one + we are switching to pnpm: yarn deploy => pnpm hardhat:deploy is going to be hard to sell haha. As Rinat said, probably for newcomers it doesn't matter. We could also do contract:deploy.

It's working great on Windows, GJ @technophile-04 !!

As I understand it doesn't affect anything. Fast googling didn't help me but we need to try to get rid of it.

pnpm is less friendly than yarn when you write a non existing command:

I have been playing around with some configs mentioned here -> https://pnpm.io/npmrc#cli-settings but nothing works :(

I have also noticed a bug i.e patch is not applied, because if you deploy two contracts you get :

YourContarct is selected but not highlight
hydration warning in console

This signifies that patch is not working checkout https://github.com/scaffold-eth/scaffold-eth-2/issues/453#issuecomment-1655732858 for more description
Having a look at it 🙌

Update I got it working in pnpm-pathc branch, but to get it working I need to do some manual work :(, and also there if you run any pnpm command you always get this warning :

The field "pnpm" was found in /Users/..../scaffold-eth-2/packages/next js/package.json. This will not take effect. You should configure "pnpm" at the root of the workspace instead.

I have created an issue on pnpm -> https://github.com/pnpm/pnpm/issues/6885
Let's see what they have to say if there is some direct way to do this otherwise we merge pnpm-pathc into this branch
I will keep tinkering around it 🙌

Some updates :
Regarding deploy command :
To run scripts from package.json you need to use run similar to npm :
pnpm run script_name

Eg :
pnpm run deploy

Yeah :), we can use our old deploy command to deploy contracts was fixed at 714a6e8
And to run binaries inside node_modules eg : npx hardhat node instead of npx we directly use pnpm hardhat node
Rinna'ts comment :

and it shows me an error when I use ctrl + c in pnpm chain terminal.

As I understand it doesn't affect anything. Fast googling didn't help me but we need to try to get rid of it.

If you do cd packages/hardhat, and then run pnpm hardhat node --network hardhat --no-deploy   now if you hit ctrl +c it will close nicely
This is not a big deal as it doesn't affect anything but will try to find if there is some workaround 🙌

Regarding patch being not applied as discussed in https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1657750641 :
It seems there was some problem from pnpm side, and it was solved in recent version 🙌
I have also updated this at  073ce50 with new patch fix and it works nicely now

Blocking thing :
Checkout this issue I have created : https://github.com/pnpm/pnpm/issues/6890
I think this is the only issue which is blocking us now, once this is solve we can go through test once again and it will ready to merge 🙌

Hey guys this is ready for testing and review 🙌
Just one thing I noticed on subsequent installs on macOS
Pnpm takes 2.3s and Yarn takes 665ms

pnpm install might be slower on macOS because pnpm uses hard links on macOS and Yarn copies files. Copying is faster but uses more disk space. Copy-on-write copies would be the best (which we use on Linux filesystems that support it) but on macOS we can't use it because Node.js doesn't allow it (https://github.com/pnpm/pnpm/issues/5001)
~ It was mentioned here: https://github.com/pnpm/pnpm/issues/6447

But would love it if others could test it as well on Windows & Linux

@technophile-04 I successfully run pnpm install (a new version is online 8.7.1) and I was able to run all the command but pnpm hardhat:deploy
I got:
`
ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL Command failed with ENOENT: hardhat:deploy
spawn hardhat:deploy ENOENT
`
pnpm hardhat:test works fine and all the tests are passing.
I'm on Ubuntu 20.04.

Hey guys this is ready for testing and review 🙌
Just one thing I noticed on subsequent installs on macOS
Pnpm takes 2.3s and Yarn takes 665ms

pnpm install might be slower on macOS because pnpm uses hard links on macOS and Yarn copies files. Copying is faster but uses more disk space. Copy-on-write copies would be the best (which we use on Linux filesystems that support it) but on macOS we can't use it because Node.js doesn't allow it (pnpm/pnpm#5001)
~ It was mentioned here: pnpm/pnpm#6447

But would love it if others could test it as well on Windows & Linux

How are you measuring these times "Pnpm takes 2.3s and Yarn takes 665ms"?

@technophile-04 I successfully run pnpm install (a new version is online 8.7.1) and I was able to run all the command but pnpm hardhat:deploy ...

Actually to deploy, we should use pnpm run deploy actually I forgot to update the first PR comment my bad will update it but here is the detailed explanation  https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1667241333
Could you please try pnpm run deploy once and also make sure you append run while scripts from package.json ?
Also one thing regarding the error message you got, could you please do pnpm -v and see if your system pnpm version is 8.7.1 ? Because ideally the error message should have been a bit friendly :

How are you measuring these times "Pnpm takes 2.3s and Yarn takes 665ms"?

Actually my terminal outputs "Done with ..."

Pnpm
Yarn

Maybe other's could append time before running command eg: time pnpm install to get actual time taken by process

Also, this is on subsequent install with all the cache present

But would love it if others could test it as well on Windows & Linux

I'm playing with pnpm on Windows, everything seems to be working fine for now, except pnpm hardhat:test. It's throwing this exception:  ERR_PNPM_RECURSIVE_RUN_FIRST_FAIL<EMAIL_ADDRESS>test: REPORT_GAS=true hardhat test --network hardhat
If I delete REPORT_GAS=true from test script, it works fine.

On my windows desktop it takes 10s to pnpm install on subsequent installs, vs 2.5s from yarn install. So it seems the same 4:1 ratio.

Will play a bit more with my SE-2 build and report back if I find anything. GJ @technophile-04 🙌🙌

pnpm hardhat:deploy

It is working now! Thanks!!
I had 8.5.1 and now I updated to 8.7.4.
It seems like pnpm is a little slower than yarn but I wouldn't bother about it...

Amazing job @technophile-04 it works great!! And thank you all for testing.
So:
PROS

allows us to have individual .locks files per package.
Fancy new tool that people like more, now? :D Not a fan of this PRO tho haha

CONS

Remove OG yarn <script>. Not it's pnpm run <action>.
Slower. (for me too) Like Damu said, not a super big deal.

What do you all think?

Amazing job @technophile-04 it works great!! And thank you all for testing.
So:
PROS

allows us to have individual .locks files per package.
Fancy new tool that people like more, now? :D Not a fan of this PRO tho haha

CONS

Remove OG yarn <script>. Not it's pnpm run <action>.
Slower. (for me too) Like Damu said, not a super big deal.

What do you all think?

I think the really important issue is the advantage of having individual .locks files (the other PROS and CONS are not too relevant, I think).
So I vote to go with pnpm! ;-)

the other PROS and CONS are not too relevant, I think

This is very subjective, that's why I think it's important for us to vote :D
Austin is neutral:

I think I'm 51% pnpm / 49 % yarn :D
If we want to go with PNMP, we'll need to sync:

Merge this PR
Update the SE-docs site
Merge to CLI + publish. (We should also test it well)

We should probably have everything prepared and push the red button :red_circle: when ready.

In case we move to pnpm: PR in the SE-2 docs repo: https://github.com/scaffold-eth/se2-docs/pull/20
It's a bit weird to use pnpm <script> for all the scripts except for deploy. Should we recommend (in the docs) to use pnpm run for all the scripts, or do you think is it good as it is?

fine for now, except pnpm hardhat:test. It's throwing this exception:

On my windows desktop it takes 10s to pnpm install on subsequent installs, vs 2.5s from yarn install.

This is strange :(, @Pabl0cks could you please just check your pnpm version and making sure its above >= 8.7.1 because I had this similar 4:1 ratio before version 8.7.1 because of https://github.com/pnpm/pnpm/issues/6890 but its fixed now
Also regarding pnpm hardhat:test could you just try using pnpm run hardhat:test and also share an SS of full output 🙌

I agree with you guys !
I only vote for pnpm because it will make installation via CLI a lot faster and consistent, as we will already be having "pnpm-lock.yam" inside each extension / package right ? (LOL the main motive for me to create this PR was this reason) but if we are not able to include pnpm-lock.yaml inside each package then I don't think its worth it  (Please correct me If I missed something or lol I am wrong path or missed some other PRO's)
Regarding #419 this could be solved without using pnpm by uploading the whole Monorepo to vercel (which in turn will contain yarn.lock) but people need to mention "./packages/nextjs" when they run "yarn vercel" as mentioned in https://github.com/scaffold-eth/scaffold-eth-2/pull/231#issuecomment-1471632149 which might be ok then typing "pnpm run" every time

We should probably have everything prepared and push the red button 🔴 when ready.

Lol yup, I was wondering whether we should create another PR parallel / test locally at CLI branch utilizing pnpm to install dependency and see if everything works nicely there and then we can merge this PR once we are sure ?
Tysm guys!!

This is strange :(, @Pabl0cks could you please just check your pnpm version and making sure its above >= 8.7.1 because I had this similar 4:1 ratio before version 8.7.1 because of https://github.com/pnpm/pnpm/issues/6890 but its fixed now
Also regarding pnpm hardhat:test could you just try using pnpm run hardhat:test and also share an SS of full output 🙌

I had 8.6.10. Updated to 8.7.4 and installation went down to 8.6 seconds 🙌.
I still get the REPORT_GAS=true error, and I've noticed a error during the install, don't know if it can affect my pnpm hardhat:test results. Here is the SS with the 3 commands:

Here the SS using PowerShell instead:

@Pabl0cks I think Windows doesn't recognize env vars like that on the package.json.
Can you check if you get the same error when running yarn hardhat:test on main? We might need to use something like https://www.npmjs.com/package/cross-env

It's a bit weird to use pnpm <script> for all the scripts except for deploy. Should we recommend (in the docs) to use pnpm run for all the scripts, or do you think is it good as it is?

I would vote for using pnpm run <script> just to keep things consistent and also pnpm suggests it to run that way.
Also noticed pnpm run gives a slightly correct ERR_MESAAGE output if you misspell :

Lol but yeah I am kind of 51% and 49% so anything works 🙌

Can you check if you get the same error when running yarn hardhat:test on main?

Just tried a fresh scaffold-eth-2 and yarn hardhat:test works fine on main. Here is the sshot:

@technophile-04 since you fixed wagmi/viem types, original problem disappeared.
The only problem with yarn for me is this

But if we move that simple type to our project everything works again with yarn without locking wagmi/viem versions. Am I missing something?

The only problem with yarn for me is this
But if we move that simple type to our project everything works again with yarn without locking wagmi/viem versions.

Tysm Rinat, just created PR at #541

So it seems we don't need lockfiles anymore. Am I missing something?

Do you, mean to say separate lockfiles?
I think although the viem/wagmi issue is solved but it might with other libraries that use installs when using SE-2 for example checkout => https://github.com/scaffold-eth/se-2-challenges/pull/91 (this can solved as mentioned in https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1711647676 but it decreases DX while deploying to vercel )
also checkout Carlos comment here => https://github.com/scaffold-eth/scaffold-eth-2/issues/290#issuecomment-1714052355 🙌

Tysm Rinat, just created PR at https://github.com/scaffold-eth/scaffold-eth-2/pull/541

👍 ✅

Do you, mean to say separate lockfiles?

Yep

I think although the viem/wagmi issue is solved but it might with other libraries that use installs when using SE-2 for example checkout => https://github.com/scaffold-eth/se-2-challenges/pull/91

hm, agree, you're right. And I'm glad you're right )

(this can solved as mentioned in https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1711647676 but it decreases DX while deploying to vercel )

I tried it, doesn't work for me. It asks for additional packages like node-gyp. Didn't try to install it, I think it's not a good solution
❤

Can confirm that the error Pablo got https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1712319461 here is with all Windows users, tried it on my friend's Windows machine and got the same error.
After doing some digging I got it working on Windows by first running (needs to be run only once in life) :
pnpm config set shell-emulator=true

after doing this pnpm run hardhat:test worked fine (@Pabl0cks could you please try this once ? )
Reasoning :
checkout description for shell-emulator as pnpm says :

When true, pnpm will use a JavaScript implementation of a bash-like shell to execute scripts

Also, I think the reason why it works with yarn by default is because yarn handles it already internally check out this => yarn/features/scripting/portable-shell

Just some thoughts while I was playing on Windows:
Lol don't know why but I don't feel confident merging this PR (at least because of windows)

For some reason it took lot of time while doing install almost around 5m 6.6s (It might be due network condition but yarn install was comparatively a lot better)
On subsequest install was getting the same cpu-feature error as Pablo showed in his first image in https://github.com/scaffold-eth/scaffold-eth-2/pull/444#issuecomment-1712319461 (planning to debug this next 🙌)

After doing some digging I got it working on Windows by first running (needs to be run only once in life)

Since your pnpm config is inside .npmrc file you can just add it to npmrc example and it will work. Could you check it?
With this option, it also works on mac. Installed all deps in 15.3 sec (Mac pro 2021)

After doing some digging I got it working on Windows by first running (needs to be run only once in life) :
pnpm config set shell-emulator=true
after doing this pnpm run hardhat:test worked fine (@Pabl0cks could you please try this once ? )

Yeah, it worked !! Great job @technophile-04 🙌

Some updates that might affect this PR here: https://github.com/scaffold-eth/scaffold-eth-2/pull/544#issuecomment-1737451534

As we discussed, closing this for now. Let's keep the branch tho!


ANSI: nF-escape symbol "(" decoding issue

Type of changes

[x] Bug fix

Checklist

[x] I've run the latest black with default args on new code.
[x] I've updated CHANGELOG.md and CONTRIBUTORS.md where appropriate.
[x] I've added tests for new code.
[x] I accept that @willmcgugan may be pedantic in the code review.

Description
Resolves #2688, by skipping past the current token when a ( nF-escape control symbol is encountered.
I don't really know whether this is a sensible approach, but it seems worth sharing the attempt (I'm not aware of other work-in-progress to resolve the issue).
Todo:

[x] More test coverage

@jayaddison Any chance you can fix that error? Think it is just black formatting.

Thanks @willmcgugan!

Codecov Report
Patch coverage: 93.33% and project coverage change: -3.11 :warning:

Comparison is base (1876380) 98.38% compared to head (541686e) 95.27%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2799      +/-   ##
==========================================
- Coverage   98.38%   95.27%   -3.11%
==========================================
  Files          74       74
  Lines        7930     7922       -8
==========================================
- Hits         7802     7548     -254
- Misses        128      374     +246

Flag
Coverage Δ

unittests
95.27% <93.33%> (-3.11%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

rich/_export_format.py
100.00% <ø> (ø)

rich/cells.py
100.00% <ø> (ø)

rich/segment.py
98.72% <ø> (ø)

rich/pretty.py
96.41% <83.33%> (-0.12%)
:arrow_down:

rich/ansi.py
100.00% <100.00%> (ø)

rich/console.py
95.15% <100.00%> (-2.88%)
:arrow_down:

rich/traceback.py
96.25% <100.00%> (ø)

rich/_windows_renderer.py
0.00% <0.00%> (-100.00%)
:arrow_down:

rich/_win32_console.py
3.01% <0.00%> (-86.94%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Thanks


Update bgp-communities-in-expressroute.md

Added some Teams-specific information.

@nenonix @JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

@kelleyvice-msft - PR has been copy edited and is ready for final review.
Thank you.
cc: @kenwith


When set via browserslist query, can't output ESM with node 12.22 target

Bug report

What is the current behavior?
I'm outputting ESM (experiments.outputModule: true & output.library: "module").
When having target: "browserslist:node 12.22" in my webpack.config.js, I get the following error:
Error: For the selected environment is no default ESM chunk format available:
ESM exports can be chosen when 'import()' is available.
JSONP Array push can be chosen when 'document' is available.
Make sure that your 'browserslist' includes only platforms that support these features or select an appropriate 'target' to allow selecting a chunk format by default. Alternatively specify the 'output.chunkFormat' directly.

If I replace the above line with target: "node12.22", the error disappear.
If the current behavior is a bug, please provide the steps to reproduce.
See above (but can provide a minimal example if necessary).

What is the expected behavior?
Successful compilation.

Other relevant information:
webpack version: 5.68.0
Node.js version: 17.4.0
Operating System: MacOS Monterey
Additional tools: N/A

hm.. in browserlist we support from node 13.14 from node target from 12...

Why is that? 🤔

ESM support has been back ported to some minor node.js 12 version.
We need to update that. You can send a PR

Will do!
Le mar. 15 févr. 2022 à 09:23, Tobias Koppers @.***> a
écrit :

ESM support has been back ported to some minor node.js 12 version.
We need to update that. You can send a PR
—
Reply to this email directly, view it on GitHub
https://github.com/webpack/webpack/issues/15360#issuecomment-1039989138,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHC2WDOBTQWXWFN265EMLNDU3IERXANCNFSM5N6DQNOQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Théophile Avoyne


hardcoded path in FileUserServiceAdapter in auth.go

This form is for bug reports and feature requests. Major features will go through a spec process.
Feature:
Bug:
What happened:
Creating an auth secret using a volume path other than /var/run/asb-auth will cause the broker to be unable to use basic auth as an authentication mechanism.
What you expected to happen:
I should be able to configure a secret and a volume path of my choosing and have the broker use that for basic auth.
How to reproduce it:
In the deployment config create a mountPath using /var/run/sb-auth.
        - image: jmrodri/samplebroker:demo
          name: samplebroker
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: config-volume
              mountPath: /etc/ansible-service-broker
            - name: samplebroker-tls
              mountPath: /etc/tls/private
            - name: sb-auth-volume
              mountPath: /var/run/sb-auth

In the volumes section point the secret to use the above volume
        volumes:
        ...
          - name: sb-auth-volume
            secret:
              secretName: broker-auth-secret

You will see this in the broker logs:
[2018-04-27T07:53:33.354Z] [ERROR] - Error reading username. open /var/run/asb-auth/username: no such file or directory
[2018-04-27T07:53:33.354Z] [ERROR] - we had a problem building the DB for FileUserServiceAdapter. %!(EXTRA *os.PathError=open /var/run/asb-auth/username: no such file or directory)

Related Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1572521

Make the directory configurable for the FileUserServiceAdapter

new config addition to the auth section (default it to the current value /var/run/asb-auth
update the FileUserServiceAdapter struct to read the configuration

close won't fix


Short solution needed: "How to get unique values" (python-pandas)

Please help us write most modern and shortest code solution for this issue:
How to get unique values (technology: python-pandas)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Unique values in a specific data structure like a Data Frame; Or just in a column?

Better if you give both solutions

How to get unique values

unique() function works for either a Pandas Series or Pandas DataFrame. The difference being, for a DataFrame, you have to pass the index of the column. That's because unique() function accepts a 1D array, like a Pandas Series.
PS. it's faster than numpy.unique()
API reference: pandas.unique

Get unique values of a Serie
import pandas as pd

my_list = [10, 100, 100, 10, 1, 1000]
my_serie = pd.Series(my_list)

print("My serie")
print(my_serie)

print("Unique values:")
print(pd.unique(my_serie))

import pandas as pd - load lib:Pandas module
my_list - declare a python list and assign values
df = pd.Series(my_list) - constructing a Serie using pandas
print(pd.unique(my_serie) - unique() returns unique values of a one-dimensional array, like a Serie

Output:
my_serie
0      10
1     100
2     100
3      10
4       1
5    1000
dtype: int64
Unique values:
[  10  100    1 1000]

One line solution
print(pd.Series([2, 1, 3, 3]).unique())

Output
[2 1 3]
Get unique values of a DataFrame column
import pandas as pd

phoneReviewsSet = {
  'Phone': ['iPhone 5', 'iPhone 6', 'iPhone8', 'Galaxy S9', 'Galaxy Note 10'],
  'Stars': [1/5, 4/5, 3/5, 4/5, 5/5],
  'Phone Price': [204, 320, 501, 800, 990]
}

df = pd.DataFrame(phoneDataSet)
print(df)
print(df["Phone"].unique())

Advanced: Return unique values from all columns in a DataFrame
You would have to traverse all columns of the DataFrame and call the unique() function for each one. Or you could use drop_dublicates() function to return the distinct rows of a DataFrame

Thanks man!
Deployed here:
https://onelinerhub.com/python-pandas/how-to-get-unique-values-from-series
Dataframe solution already deployed:
https://onelinerhub.com/python-pandas/how-to-get-unique-values


how to get database call span in a trace

Hello everyone,
Question
I have a question regarding how to get spans for database calls?
Description

According to the documentation when we need to configure pods and service using the conventions given on this link.

the Second point is that applications need to pass these request params in egress requests but the issue is that I can handle these things on application code but how to do it on database calls side what I meant is that I cannot change the database client's code to pass those request headers.

Please move this question to our discuss forum, as that is a more appropriate venue for questions.  Thanks.

@douglas-reid I have created a topic on forum for it.


RFC-2 Error page simplification

RFC-2 Move ErrorPage in to optional module
Author: @jonom
Status: Pending review
Version: 1.2.2 (2016-03-20)
Purpose and outcome
This RFC aims to address a few issues with the current state of error page responses and error page content management in SilverStripe CMS:

Flaws Statically generated error pages are under-utilised and go stale quickly.
Inflexibility 'Error content' management is not a requirement for many websites, and a different approach may be preferred where it is.
Confusion The ErrorPage content management model leads to a confusing CMS experience.

Since the current ErrorPage implementation is suitable for some sites but not all, this proposal recommends removing the ErrorPage class from core and moving it in to a separate optional module. This would extend upon a simpler error response model in core, and pave the way for developers to create alternate solutions for managing, displaying, and caching error response content that suits their needs better.
Community is welcome to give feedback. The proposal will be submitted to the core-committers for approval.
Motivation
Static error pages easily become stale and broken
They are generated on dev/build but only if they don't already exist. After that they are only regenerated when re-published from the CMS. This means the static pages can easily become very out of date, and even include dev-environment content rendered when the website project was first initiated.
Static error pages are not widely utilised
On a regular SilverStripe installation directives in the .htaccess file suggest that error responses will be handled efficiently by Apache serving up a static error page. This is true for a 500 response, but all 404 responses in fact go through the full SilverStripe request cycle and a dynamic error response is generated, meaning the static 404 file is never used. In the case of a fatal error Apache serves the 500 error page because PHP has aborted the response. Hopefully this situation doesn't happen often, but when it does it is likely that a stale error page is being served.
Inflexible error response model

The use of ErrorPage content removes the ability to set a custom error message from code using ContentController::httpError($code, $message) ($message is ignored)
Error responses which don't have a corresponding ErrorPage published in the CMS will show the content for a 500 Server Error which is misleading.
Static error pages are only generated for the default language (can anyone confirm this?) and primary Subsite.

ErrorPages confuse regular (non-technical) CMS users

It doesn't make sense to show error pages in the Site Tree and allow drag and drop re-ordering, since they aren't part of a website's menu structure or information architecture.
Non-technical CMS users are generally not interested in editing error page content
Non-technical CMS users read the labels in the Site Tree and think something has gone wrong, not making the connection that there is content there they can edit

Proposal
Remove ErrorPage from core and split off in to an optional module
The ErrorPage class brings with it some UX problems and flaws as outlined above, and is not needed in every CMS installation, so let's separate it out in to an optional module. This will make the CMS experience simpler and cleaner for users who don't need this functionality.
Provide a simple error response mechanism that can be overridden or replaced
RequestHandler::httpError() already provides a plain-text error response with default descriptions for a range of status codes, with extension points that allow you to customise the response to your liking. This means we already have a good base in place that can be built upon with modules. It may make sense to modify this base to provide a branded error page by default, using a site's Page.ss templates.
Don't cache error page content by default
We're not effectively caching error page content at the moment and don't have built-in caching for any other content so let's leave that functionality to dedicated modules.
(Except for the 500 error page)
We can't serve up a dynamically generated response in the event of a fatal error, so let's provide a simple static html page for this purpose. This could be manually replaced or customised if desired to match the look of the website. This may not be ideal for all websites but it is just a starting point that can be expanded upon with modules.
Other considerations

This RFC does not propose any solutions for fixing existing problems with ErrorPage for developers that choose to go on using it as a module. Upgrades to that module would be a good idea but are out of scope for this RFC.
Who will be responsible for maintaining the ErrorPage module? (SilverStripe Ltd?)
Document upgrade path

install ErrorPage module or remove ErrorPage instances)
recommend some alternative modules and pros/cons if/when available

Update docs
Should Installer require ErrorPage as a dependancy or a simpler alternative?

(Partial) Alternatives
Fix generation of static error pages

How do we ensure error pages are regenerated regularly? dev/build is not done regularly and a basic SilverStripe installation doesn't currently require cron jobs to be set up. Generating and caching pages at request time may be an option.
How will we know we need to serve up a static 404 without going through the request process and hitting the database? If we can't solve this problem any performance gains to be made by saving a static error page will be similar to serving up a 404 page that has been cached using partial caching.
How to support multi-domains, translation, SSL vs non-SSL...?

These are difficult questions to answer so this doesn't seem like a good solution.
Separate out error pages from SiteTree
ErrorPages could be visually separated at the bottom of the SiteTree, or moved to the Settings section of the CMS, to reduce confusion and clutter. However if most websites don't actually need the ability to edit error page content in the CMS, and we're adhering to 'Convention over Configuration' then this functionality should be optional in the form of a separate module.
Impact

The ErrorPage module will need to be installed on websites where that functionality is required, including:

Websites that rely on a fully-themed 500 error response
Websites that are making use of the static 404 page (non-standard configurations)

If the ErrorPage module is not installed, developers will likely want to install an alternative module or implement their own solution to enhance error responses beyond plain-text.

If I'm wrong about anything here or left anything out please point it out. Thanks

awesome effort on this @jonom - I'll look at raising this with the core committers at the next hangout session. We can take a preliminary vote from them to see whether you'll want to do further community discussion to reach an acceptable solution to this :+1:

Sounds good!

By the way, I should note that while I think porting the ErrorPage functionality to a new module is important for backwards-compatibility, I'm not interested in maintaining this module myself as I don't want to use it in my own projects. So would be looking for SilverStripe Ltd or a developer who values this functionality to take ownership of it. I'm happy to otherwise take the development effort as far as I can but may need some help with the deeper elements (like the Controller error-handling chain).

@jonom there was some good points raised around your RFC in the core committers hangout if you want to check it out and perhaps address a few things. https://youtu.be/d6ADget_PfY?t=21m9s

Hey Cam, I tuned in for the hangout but I didn't catch anything in particular I haven't already addressed in the RFC. I think the main objections coming from Sam were that:

Being able to customise 404 content in the CMS is really important
Serving a statically cached 404 page is really important for performance
Serving a static 500 server error page is really important

In response:

I just don't think this is true for the majority of users - in my experience most CMS users ignore the ability to edit the content of the 404 page. I'm sure it's useful in some cases but if it's not important to the majority of users then I think it should be moved to an optional module.
I totally agree. But as far as I can tell SilverStripe simply doesn't do this. A static 404 file is generated but it doesn't ever get served up - even for requests in the assets directory. I'm dying to be proved wrong about this so please tell me if I'm mistaken :)
Agree - a static file is the only option here. But I don't think it's critical for all websites that this page is rendered the same way as any other page on the site. As Hamish pointed out, fixing this won't be trivial as there are issues around translation, compatibility with the subsites module, and performance overhead of generating the static files. I'm not convinced there's a one-size-fits-all solution here, which is why I think it would be better to start with something simple, and any more complex behaviour could be added with a module.

Part of the goal here is flexibility, if we simplify the approach and ensure it can be overloaded then it opens the door for people to develop and choose between different solutions - including restoring the current approach if they wish. I'm not against static caching, I'm very much for it - I just think it should be addressed outside of core, or if it's left in core then it needs some serious TLC and should be available to be used for more than just error pages.

I think that the error page should be editable via the CMS. The need for dynamically generated 404 pages with some information from the CMS content field is also something which we rely on heavily - "Did you mean ..." is something useful for a 404 page.
I believe there is a way to compromise all of this, even if it means retaining some of the functionality that is current but pulling them out of the Site tree. If the content isn't customisable then it should be easy to set up and instantiate / generate / whatever all the error page conditions which you wish to use. An example of this is that we heavily use 403 forbidden pages, and almost always have this content customisable for the client as they need to change the information as to why people cannot access certain parts of the site based on rules which they can set via "access levels" for each page type. Losing this sort of functionality and having to rely on programatically generated responses diminishes the functionality of these page types dramatically and would force us to choose another approach.

Hi @stevie-mayhew, this RFC is proposing that ErrorPage becomes an optional module, so you shouldn't have to lose any functionality if you don't want to - just install the module.
I figure if this RFC is accepted it could lead to multiple options for developers when rolling out a site:

Accept the default error page title+content for each error status code (no customisation)
Override some or all of the default error page titles and content in a yml config file (basic customisation)
Bypass the config system and administer error page content in the CMS (advanced customisation, would require some custom code or installation of a module).

Whichever approach is used, it should be possible to override any of this content in code by calling httpError($code, $message, $title). I added the $title argument.
I think option No. 1 would be a fine starting point and suitable for many basic websites. For those sites it will be nice to have the CMS a bit less cluttered and confusing.
The point is that not everyone needs or wants error page content to be administered in the CMS so it shouldn't be forced on us - but for the developers that do want the functionality it should be available as an add-on.

p.s. I would draw a comparison with the Lost Password page. You can't administer the content for that page in the CMS but no one seems to mind. If you wanted to you could create a LostPassword.ss template and come up with your own way of customising that content. As a baseline behaviour I think this is fine, something to be built upon if people want to.

Has anyone looked into the possibility/ability of having SilverStripe serve the default Apache error pages instead?  I wonder if there's a way to have PHP just tell Apache, "serve your 404 page," or a way to retrieve Apache's ErrorDocument settings which PHP can process from there....

Also need to keep multiple domains into consideration. As it stands, www.marketo.com is loaded through a cache proxy and the error pages use the domain name of our origin.

Yeah this is why I want SilverStripe to ship with a more basic approach to error handling that can be built upon - I think there is more than one solution and SilverStripe shouldn't push developers in to doing it one way but allow them to choose between different modules or implement their own approach.
I'm picturing perhaps a simple error response class in core that can be completely replaced through dependancy injection. I haven't delved in to the core code yet to see what's possible though.

@hafriedlander I think you were going to initiate some private discussion with other core devs who couldn't make it to the core dev hangout when this was raised - did anything come of that? If there's any specific things I need to address or revise in the RFC I look forward to hearing your thoughts - or if I'm on the right track here I look forward to receiving the go-ahead to start some work on it ;)

Jono, I think you've addressed Sam's concerns well, and I agree with your assessments.
You're right about 404 errors not using the static cached file. ModelAsController->getNestedController() calls ErrorPage::response_for(404). But that's defaulting to generating the page dynamically: https://github.com/silverstripe/silverstripe-cms/blob/3.1/code/model/ErrorPage.php#L53
I think @sminnee's performance concerns stem from the fact that the default .htaccess passes on ALL requests which aren't mapping to an existing file to SilverStripe (see https://github.com/silverstripe/silverstripe-installer/blob/3.1/.htaccess). So if you have a broken URL to an image on your server, these all run through the standard SilverStripe request cycle. I was under the impression that the ErrorDocument 404 /assets/error-404.html catches some of that, but that doesn't seem to be the case.
Overall, we're doing expensive SilverStripe 404 error page generation already in pretty much all cases, which means there's no performance regressions. Sam, am I missing anything here? That being said, I think we should address this issue. I would suggest caching the generated 404 template through SS_Cache for an hour or so, which should have roughly the same effect: It smooths out traffic spikes from bots. dev/build would clear the cache. Tradeoff between showing possibly outdated template data if it changed in the last hour, and allowing specific control via dev/build on deployments etc.

Thanks for the feedback and confirming my conclusions @chillu! Caching the generated 404 for an hour sounds like a good idea to me but I just think it's important that if a solution like this is added to core, it doesn't block other caching solutions being implemented. Some devs might want a dynamic response at all times or need a more complex caching solution.
I think caching in general is in it's infancy with SilverStripe and I'm really excited to see how modules like dynamiccache, cacheinclude and staticpublishqueue continue to develop. I want to be able to install a module like this on every site I develop and have it cache my 404 responses along with all other content so I can get that instant responsiveness we all love, on all websites great and small.
I know caching of 404 responses is important in terms of server load but I think if we can nail the best way to do that, it should be able to applied to other content as well - so overall I feel like it should be handled by a module rather than core. Sam suggested that 'ErrorPage the module' is included with the Installer as a dependancy - perhaps we could also bundle a caching module and use it for 404s as a starting point? Then people could remove the module if they don't want it or switch it out with something else depending on their needs.

I'd keep any "pluggable error page caching" API small, since its such a special case. In the end, error pages are just another page which requires caching - they just happen to be requested more often on an average SilverStripe installation. In order to get this RFC across the line, I think any discussion about bundling caching modules are distracting (although I agree we need to do better in this regard).
@silverstripe/core-team Are you OK with Jono to start coding on this?
Jono, can you please update the RFC with the outcomes of the discussion above? Makes it easier for the core team to review the proposal without going through long comment threads (which for me was the whole point of RFCs heh).

@chillu I thought that everything I've written in my comments on this thread was already in the original RFC, so I obviously didn't explain myself very clearly in the beginning :) I've revised the RFC to hopefully be a lot clearer now.

Nice work on the RFC @jonom - especially for taking the time to consider the user experience.
@clarkepaul and I just had a chat about the UX aspect of your RFC. We agree that removing the error pages from the SiteTree is a good idea. But we're not sold on moving the ability to edit them into a module.
We have work planned for our current sprint, based on the UX research from November, which involves re-factoring the 'Filters' panel. An alternative solution to removing error pages entirely, might be to incorporate them in the re-factored menu.
Simply moving the current bad UX into a module doesn't solve the problem. It will resurface for anyone who installs the module.
We'll put together some wireframes of what the error pages might look, if relocated to the new menu, and post them here by the end of the week.

Thanks @flashbackzoo - yeah as I outlined above I really don't want to be responsible for an ErrorPage module as personally won't use it and I would rather just get rid of it all together. I thought we would need to provide this as an optional module for people that want to restore the functionality but if you can provide a better replacement that will be brilliant!!

If you can provide a better interface for managing ErrorPage content that will be brilliant! I did hope that would be addressed as part of the separation in to a separate module. I still would prefer to see this be an optional module as for all of my clients the ability to edit error page content is just clutter in the CMS, and I'm sure I'm not the only one. Is this something we could do a poll on maybe to see how many website owners have actually taken advantage of the ability to edit their error messages? I can tell you that for all of my clients, the error pages have only ever been published by me, so I would have rather set the content in the config system.

Hey, we are starting to work on splitting out the options for how you view the site tree from the search as this was the worst performing task from our tested audience. We asked them to find the deleted pages and not one user could perform the task from memory.
This could be a possible option to remove the error pages from the site tree but still allowing users to access them if needed, it is still a wireframe so likely to change. I still need to think how the error pages overlap with the pagetype select dropdown in the search area.
Anyways, let us know what you think.

@jonom The reason I'm against ErrorPage content editing being a module, is it goes against a CMS's core competency, enabling non-technical users to update their site's content. Although ErrorPages are not always customised, their content is publicly available by default, so that content should be able manageable by default too.
But yeah a poll definitely something we can look at doing.

In theory, everything should be content editable. In practice, its a tradeoff between flexibility and usability. There's lots of similar content which you can't edit in the CMS, e.g. "lost password" labels. email content and feedback. So while we might make all of that "secondary content" editable at some point, I don't think it should block moving the error page type feature into a module. We could achieve the "content editing abilities" by having a HTML field in /admin/settings as well.
@flashbackzoo That being said, I like new filtering UX. It could also make it easier to hide pages which CMS authors usually shouldn't touch, like "search results". When are you expecting to get some user testing done on that? Interested to see if CMS users would actually figure out how to edit error page content. Its such a rarely used feature anyway, I think pushing it into a filter will take it from "rarely used" to "never used".
@jonom Do you think its feasible to implement your proposal without removing ErrorPage from core? We could still remove the static error page generation, make the "server error" a generic static template, and make the error handling in the controller layer more pluggable, right?

@chillu Yeah being able to edit labels, email content etc is a step too far. But I think having the ability to edit all page content blocks is important. Having an HTML field in Settings is a reasonable solution.
We have user testing planned into our current sprint. We're developing the above wireframes a bit more and will test those.

@chillu yes, if we do decide that ErrorPage should stay in core we can still complete most of the proposal and make a lot of improvements. But...

Yeah being able to edit labels, email content etc is a step too far. But I think having the ability to edit all page content blocks is important.

@flashbackzoo for me, the login and lost password 'pages' are public-facing content too and are no less important than error messages – but they're not represented in the CMS. If we don't make this content editable I don't see why we should make a special exception for error messages. These pieces of content are developer centric and if most users aren't interested in editing them, we don't need to clutter the CMS with it out of the box. This is what 'convention over configuration' is all about. I definitely acknowledge that some people want to do this and that's why I think it would be a good fit for a module.
The wireframes are looking good but the error pages are still represented in a SiteTree which is one of the issues raised in this RFC. Really ErrorPage shouldn't extend SiteTree as it shouldn't need to implement the Hierarchy class, nor should it need URLSegment,ShowInMenus,ShowInSearch etc. It's misleading to represent them as Pages so like @chillu I would prefer to see this type of content administered in the settings section of the CMS. If the utility 'pages' were visually separated somehow and not allowed to be rearranged with the other pages this might help though.
This is getting a bit off the RFC topic now :) but @clarkepaul I don't think it is intuitive to search out a filter to restore an archived page, usually filters show everything by default so it's not where I would go to look to restore a page. Perhaps a fourth icon in the row that is a kind if 'rewind/undo' icon dedicated to letting you restore a page? Or what if when you click 'Add a Page' that interface gives you two options 'Add a new page' and 'Restore an archived page'? I think that could work pretty nice actually, and people would be reminded it's a possible action every time they create a page. Also (they're probably planned but) some tooltips for those icons would be needed I think.

@jonom Good point - I hadn't considered the login and reset password pages. I think you've just about convinced me.
One other question. Would sites that currently have custom error pages break if they upgrade and don't install the module?

We'd have to make sure they don't break, not sure exactly how that will look yet. May need to be an upgrade script to archive any existing ErrorPages on build if the module isn't installed, otherwise I think they might be changed to regular Pages during a build if the ErrorPage class is missing? But yeah, would certainly need to make sure that upgrading without the module is smooth. I'll add a note about that to the RFC.

Cool - I'm on board with it.

:clap: Nice!

@jonom You made a good case for them not showing in the tree - I was thinking the error pages would show separately from the site tree (divided by a line or similar).
The area where you can select what you see in the tree (currently eye icon), I don't see it as a filter but more like "view options".

@clarkepaul Hmmm if I unchecked the Modified box though... would you say the Modified pages would be filtered out? :)

Yup they are essentially the same thing, I'm making the assumption users don't think of it as "filtering" as that is quite a technical term. It's like the way we use the settings area for a pagetype to change the view of that page.

In any case I think it does make Archived content easier to access, but I feel like a lot of users probably won't use that view options box, so there's a visibility issue there, plus there's still a bit of a jump to work out that you can filter archived content in to the site tree, then find the page you want, then restore it - it's a bit abstract. Adding it to the Add New Page interface and explicitly spelling out the option might help educate users that the action is available and hopefully they would then be able to recall where to find it.

Yeah obviously that is a wireframe so we have some tough decisions to make whether text is better to use, or what iconography to use. What you cant see in those wireframes is that when you select "Pages" you are presented with both icons and the supporting text but as with any UI some exploration is needed for tasks you don't do regularly.
FYI We are also trying to find a solution that will match other areas of the CMS that will also have an actions bar eg. the files area

@silverstripe/core-team I don't expect to hear anything more from you guys until after 3.2 ships but just FYI this is on pause for me until I get the go ahead or some more direction about anything that needs to change. I haven't started any development work on this yet but I assume 4.0 is a long way off (a year?) so I'm not in a rush.

Sorry, rfc label was renamed to "rfc/accepted" after it was added to this issue. I've applied the correct label now.

Guys, what about an ErrorPage controller, which would render a static file from the theme on default. The controller would still take the dynamic content as triggered by httpError(). It would have the standard routes of /404, /400 etc. The default would serve just a default Page.ss template file. If an ErrorPage_404.ss is added to the theme, THAT is used when rendering.
As far as editing, here's an approach: The default routes would be setup with the module and populate defaults / passed parameters (e.g. Title / Content), but the controller could also check for an ErrorPage having been inserted onto the Stage, the ErrorPage would need a dropdown to select the error type. e.g. ErrorPage with ErrorType being 404. In this case, /404 would serve the custom template, but populated from the stage.

@KINKCreative that could be a good option but my opinion is that we should start with some really simple functionality (plain text response), and allow people to extend it to make it more complex - so an approach like you've outlined could be achieved with a module that extends httpError(). That way people can apply whatever caching system and error content management approach makes the most sense for their site. My main gripe with the current model is that a) it's broken, and b) it's inflexible. I'd like to strip this functionality back to basics and then developers will be free to upgrade it however they like.

Agreed @jonom. As long as no Error Pages are added to the stage on /dev/build - that would be rad :) I'll try playing around with my approach (the top part is what I see could be part of the core / extension as you set it up now) and if it's worthy of a PR, I'll hit you up.

@KINKCreative I've started looking at separating out ErrorPage so it can be removed and looks pretty straightforward - see https://github.com/silverstripe/silverstripe-cms/pull/1352. If you use that as a starting point you just need to delete the ErrorPage class and remove the ErrorPageControllerExtension extensions in cms/_config/config.yml to be free of ErrorPage, including having any added on build.
I need to reduce the scope of this RFC a bit accordingly.

Have updated this RFC to reduce the scope a bit - now is centred squarely on making ErrorPage optional, as there is a reasonable base available already to allow implementing alternative solutions. This reduced scope would mean changes are limited to the CMS project.

RequestHandler::httpError() already provides a plain-text error response with default descriptions for a range of status codes, with extension points that allow you to customise the response to your liking. This means we already have a good base in place that can be built upon with modules.

I don't think a plain text response for "page not found" is a good default in terms of user experience - it'll erode trust in visitors if you don't show them a branded page with further navigation options. In my opinion, we should generate this dynamically from the default theme - but with non-editable content by default. Given the number of bots hitting 404s, this increases the importance of partial caching and optimised code in the average SS installation (more work than passing through assets/page-not-found.html via SilverStripe).
Another factor here: SS4 comes with a new filesystem implementation, which maps unpublished file paths. For example, a request to assets/<hash>/secret.pdf will actually route through assets/.protected/<hash>/secret.pdf via SilverStripe if you have permissions. Which means we can no longer assume that because a file doesn't exist in assets/, we can return a (fast) Apache 404. We don't do that by default, but this optimisation path is now closed for higher performance use cases.

I don't think a plain text response for "page not found" is a good default in terms of user experience

@chillu That's true, and I was imagining that we would add a new dependency to Installer to account for that. I was thinking rather than making ErrorPage-the-module a dependancy (as has been suggested previously) we could have a very simple module that renders the default message using a standard page template - just like what you described. (I would make a module that does this as part of carrying out this work and happy to transfer ownership to SilverStripe Ltd.)
We could instead make that a core feature, I just wondered if some performance zealots would have an issue with that - since the default response would cover static resources and ajax requests as well. As long as that behaviour is replaceable and doesn't leave behind any overhead I agree it would be a good default.

I was thinking rather than making ErrorPage-the-module a dependancy (as has been suggested previously) we could have a very simple module that renders the default message using a standard page template

Given the relatively small amount of code, I'd just make this core behaviour - but ensure you can customise it. If we treat every customiseable feature as a module which you can opt-out of, we can go on holidays while composer install runs ;)
So as long as the default user experience is considered, it's a +1 from me.

I agree that we should remove ErrorPage - I'm not ever sure a module makes sense - does anyone want to maintain it? The current way error pages are managed and generated is broken.
The times I've wanted a custom error page, it's been important to know where I am in the site heirachy (eg: is this a 404 on what should be a blog page? if so, I could show other blogs) but because you are in this floating.
I think as long as there's a way to hook into the error response (when someone does Controller::httpError())  and they can render their own template, then that's all a dev needs to get started. I'm not sure a SiteTree object adds any value to rendering error pages.

I think we need to find a way to make it content manageable. We use it for most clients to provide custom content (i.e they put links to major sections or recent changes). Whether this content is stored in a unique admin tab for Error Module or Settings tab for Error Page management.

I think we need to find a way to make it content manageable.

That would be part of the errorpage module, surely? Otherwise we're not splitting this functionality into a module, we're just rewriting the functionality.

Yeah exactly - a single solution will not suit everyone. Some people will want to go on using the ErrorPage module but splitting it out means devs are not locked in to it and are free to implement any alternative solution they like.

That would be part of the errorpage module, surely? Otherwise we're not splitting this functionality into a module, we're just rewriting the functionality.

Exactly.

@sminnee @chillu I'm just watching the last cc hangout and noticed you discussing performance degradation as a result of this RFC being implemented. Please note that there would be no performance degradation. The static 404 file is generated but never utilised - that is a key piece of information in this RFC.
Take a look at the code in ErrorPage::response_for() and tell me in what scenario the static file would be served? The static file is only served as a fallback, and the only way I can see you could get to that point is if you publish a 404 page in the CMS then delete that page from the CMS. But it will be re-generated the next time you build the DB at which point you're back to fully dynamic 404 responses again. Likewise the 404 rule in the installer's .htaccess file will always be ignored.
Am I wrong about any of this? As far as I can see, we only have an illusion of static caching of 404s in SilverStripe. If I'm right, I think there's just no good reason for ErrorPage to live in core.

You're not wrong. See http://www.silverstripe.org/blog/route-whitelist/

@sminnee could I get your thoughts on above performance implications? Keen to get this to accepted stage and think the perceived performance limitation is holding this up.

Take a look at the code in ErrorPage::response_for() and tell me in what scenario the static file would be served? The static file is only served as a fallback, and the only way I can see you could get to that point is if you publish a 404 page in the CMS then delete that page from the CMS. But it will be re-generated the next time you build the DB at which point you're back to fully dynamic 404 responses again. Likewise the 404 rule in the installer's .htaccess file will always be ignored.

Look at https://github.com/silverstripe/silverstripe-cms/blob/master/code/logging/ErrorPageErrorFormatter.php; This is the error handler which relies on the static error pages, through ErrorPage::get_content_for_errorcode.
This handler intentionally avoids doing any SQL queries or relying on dataobjects in case it's handling a database error.

If the DB isn't available, it shouldn't be blindly serving a 404, though?
It's mostly true. I don't think this rfc says don't have simple 404s, its just saying this current solution is fragile and should be reconsidered

@tractorcow I don't debate the intention of ErrorPage::get_content_for_errorcode but I just can't seem to produce a scenario where it used. Can you give me some direction here on how to get ErrorPageErrorFormatter to produce some output? I've manually adjusted the content of the static error pages and tried setting up errors all kinds of different ways but have never been successful in seeing the static content on screen. The one exception to this is the static 500 error page, when it gets served up directly by Apache after a fatal error.

I don't think this rfc says don't have simple 404s, its just saying this current solution is fragile and should be reconsidered

That's right. Current approach is flawed and very opinionated, and I don't see a good reason to force this one method of managing error content on all SilverStripe users. I intend for all the existing functionality (however broken it is) to be moved to a module, so I don't see why anyone should be against this being moved out of core? If you like the things the way are just install the ErrorPage module :)
I would really like someone to prove that static error page content is utilised if they can. I've tried and failed, and it seems I really need it confirmed either way before this RFC can get approved as it seems to be a key point of friction. I think it's irrelevant because the module should replicate the current behaviour, but if we can clear up this point maybe I can get a green label and invest time in working on this. Then we can debate the finer details when I've got an implementation ready to share.
The main proposal here is to make ErrorPage optional, so all we should need for this RFC to be accepted is for everyone to agree that it makes sense for it to be moved out of core. I'm not talking about killing ErrorPage. I just want it to be optional.

If the DB isn't available, it shouldn't be blindly serving a 404, though?

Yeah it's mostly for 500 errors, not so much 404s.
If the database crashes mid way through a query, we don't want to re-query it to generate a 404 page. This could be due to high load on the database cluster (e.g. out of memory).

The main proposal here is to make ErrorPage optional, so all we should need for this RFC to be accepted is for everyone to agree that it makes sense for it to be moved out of core. I'm not talking about killing ErrorPage. I just want it to be optional.

It's not the only page I'd like out of core... VirtualPage as well please!

To get to the point, I'm not against moving out of the core, but I still think we need a static file.

Another related edgecase https://github.com/silverstripe/silverstripe-framework/issues/5518

Wow, it's been almost a year! :)
Any movement here?
As a side note, we had a module built that sort-of helps the situation:
https://github.com/Marketo/SilverStripe-UniversalErrorPage
(But it'd be better if it didn't have to override ErrorPage's behavior.)
cc: @stevie-mayhew

@nathanbrauer I haven't had time to dive back in to this and sort of ran out of energy too to be honest. From memory it should be very easy to split off the built in error page functionality out to a separate module, so maybe this could still be done in time for SS4? I haven't been following along too closely on SS4 dev lately so the situation might have changed a bit. If you feel like picking this up please feel free :)

@jonom if you want to do the work yourself I could setup a repo for you and give you admin.

@tractorcow I know I should but I can't fit it in right at the moment sorry :( I could probably do it in a couple of weeks if that won't be too late? Just getting back on deck this week after some time off.

For anyone interested, I've already addressed the speed concerns in my own custom implementation of the ErrorPage by doing a few very specific things:

Generate static pages as a task: Setup an ErrorPageTask which runs as part of my deployment workflow. This is automated such that, along with a dev/build it will also run this task to build out the static pages (will show sample code below). It's roughly as simple as iterating through each ErrorPage instance in the database and calling ->doPublish() on each one.
Reference static files via my CustomErrorPage class when error response is needed. But in order to do so...
Refactor those pesky static calls to point to my custom class via CustomErrorPage::response_for(...) instead of ErrorPage::response_for(...). Something I've brought up in the silverstripe-dev mailing list, since unfortunately core modifications like these have to be made in order to reduce 404 page load overhead (which can be quite considerable for large sites like mine).

Code samples:
CustomErrorPage.php
class CustomErrorPage extends ErrorPage {

	public static function response_for($statusCode) {
		// Try to get cached version *FIRST* before anything else.
		$cachedPath = self::get_filepath_for_errorcode(
			$statusCode,
			class_exists('Translatable') ? Translatable::get_current_locale() : null
		);

		if(file_exists($cachedPath)) {
			$response = new SS_HTTPResponse();

			$response->setStatusCode($statusCode);
			$response->setBody(file_get_contents($cachedPath));

			return $response;
		}

		// Continue with existing functionality.
		return parent::response_for($statusCode);
	}
}

ErrorPageTask.php
NOTE: Here I have special functions in my code base which convert notices/warnings from trigger_error/user_error to catchable exceptions (which have the benefit of also halting execution at the current stack location up to the try/catch unlike  PHP warning messages). The whileAdmin(...) helper function can be found in my SilverStripe database migrations module (specifically here) which facilitates the unfortunate need to escalate to admin permissions (modifying global state at the session level) in order to perform some necessary Versioned object tasks (on SiteTree or Page children).
Also note: In the code below, SSL is forcibly enabled manually since our site is 100% SSL all the time so this ensures links are properly prefixed with https://.
class ErrorPageTask extends BuildTask {

	protected $title = 'Cache Error Pages';

	protected $description = 'Re-publishes error pages to ensure the latest version of the page is saved to the static cache (e.g. assets/error-404.html).';

	protected $enabled = true;

	/**
	 * @param	SS_HTTPRequest $request
	 */
	public function run($request) {
		// Command line only...
		if (!Director::is_cli()) {
			echo "This can only be run from the CLI.<br>";
			exit();
		}

		withExceptions(function() {
			whileAdmin(function() {
				// Before starting, if we're in production, trick the server into thinking we're currently on a secure
				// connection. This way we can at least be sure that references will point to the correct protocol when done.
				if (Director::isLive()) {
					$_SERVER['SSL'] = true;
					$_SERVER['HTTPS'] = 'on';
				}

				echo "Rebuilding static error pages... ";
				foreach(ErrorPage::get()->toArray() as $page) {
					try {
						/** @var $page ErrorPage  */
						$page->doPublish();
					} catch(Exception $e) {
						// Output and log this message, then move on.
						$message = "\n\nError processing page '{$page->getTitle()}': {$e->getMessage()}";
						dump($message);
						logger($message);
					}
				}
				echo "Done.\n\n";
			});
		});
	}
}

p.s. Along with my implementation suggestions above, I'd suggest we adopt @chillu's recommendation from the mailing list about using the Injector to access the static methods at least for the ::response_for(...) method:
// Call static method on injected class.
$obj = Injector::inst()->get('ErrorPage');
$obj::response_for(404);

Digression: Perhaps someday that could be reduced to a simpler API somewhere in core, e.g. injector('ErrorPage')::response_for(404), but alas PHP doesn't allow scope resolution operators tacked immediately onto the end of function calls 😔 In that case you'd be better off with Laravel-esque Facades which would allow for static calls on both instance-level and static level methods (utilizing reflection to determine the correct approach once the injector has determine the correct class to use).

@tractorcow I've taken a first pass at removing ErrorPage from CMS. I realise I'm late to the party here but thought it might be possible to remove ErrorPage from core in time for beta 1, even if 'ErrorPage: the module' isn't available at the same time.
Do you want to create a repo like you offered so I can start setting that up?
CMS PR: https://github.com/silverstripe/silverstripe-cms/pull/1849
Framework PR: https://github.com/silverstripe/silverstripe-framework/pull/7017

@jonom I've setup https://github.com/silverstripe/silverstripe-errorpage and given the core team admin permissions there. Let me know before you push anything to packagist though ok?

@tractorcow will do!

Tracking progress at https://github.com/silverstripe/silverstripe-errorpage/issues/1

Well look, this came together just in time for beta 1! If anyone wants to submit improvements for or is interested in maintaining ErrorPage-the-module head on over to https://github.com/silverstripe/silverstripe-errorpage. Cooking up your own alternative module is an option now too of course. @patricknelson @nathanbrauer @KINKCreative

@jonom Great job! Can you please review/merge my improved upgrading guide? https://github.com/silverstripe/silverstripe-framework/pull/7066. Would be good to add alternative approaches there as they pop up in the community.

I published an alternative to ErrorPage today, if anyone is interested in testing it out: https://github.com/jonom/silverstripe-custom-errors


Set CMake default build type at the beginning of the cmake config

In this case, the default release mode can be properly set at the first
build with the proper compilation optimization level set.

See https://source.cloud.google.com/results/invocations/59e43652-7432-46a5-a0c0-616f8bf75fc0/targets/iree%2Fgcp_ubuntu%2Fcmake%2Fandroid%2Farm64-v8a%2Fpresubmit/log The default set to release and the force disabled NDEBUG

-- No build type selected, default to Release
-- IREE HAL drivers:
--   - local-sync
--   - local-task
--   - vulkan
-- IREE HAL local executable library loaders:
--   - embedded-elf
--   - system-library
--   - vmvx-module
-- IREE_ENABLE_ASSERTIONS force disabled NDEBUG for CMAKE_CXX_FLAGS_RELEASE: '-O3 -DNDEBUG' -> '-O3 '
-- IREE_ENABLE_ASSERTIONS force disabled NDEBUG for CMAKE_C_FLAGS_RELEASE: '-O3 -DNDEBUG' -> '-O3 '

These log entries doesn't exist in main post-submit build log. See https://source.cloud.google.com/results/invocations/7ef7de53-ed19-41d1-8145-701dbeae77d3/targets/iree%2Fgcp_ubuntu%2Fcmake%2Fandroid%2Farm64-v8a%2Fmain/log


epididymis hypertrophy

Please add:
epididymis hypertrophy
DEF: increase in the bulk size of the elongated structure connected to the posterior surface of the testis that transports, stores, and matures spermatozoa between testis and vas deferens due to cell enlargement
SYN: epididymal hypertrophy
SYN: hypertrophic epididymis
J:220689
Thanks
Anna

MP:0014186


Fix Lint Issues #171: Fix numerous errors/warnings

Fixes numerous Lint errors and warnings:

Change Android Log to use Timber
Use apply instead of commit for preference manager as this task runs asynchronously and the boolean returned by commit is not used
Remove redundant semi colons from Kotlin classes
Remove unused imports from Kotlin classes
Clean up XML code in Layout files

Fixes #171 Fix Lint errors/warnings

Codecov Report

Merging #2537 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2537      +/-   ##
=========================================
- Coverage    5.89%   5.89%   -0.01%
=========================================
  Files         258     258
  Lines       12306   12307       +1
  Branches     1103    1103
=========================================
  Hits          726     726
- Misses      11521   11522       +1
  Partials       59      59

Impacted Files
Coverage Δ

...nrw/commons/upload/SimilarImageDialogFragment.java
0% <ø> (ø)
:arrow_up:

.../free/nrw/commons/category/CategoriesRenderer.java
0% <0%> (ø)
:arrow_up:

...w/commons/contributions/ContributionsFragment.java
0% <0%> (ø)
:arrow_up:

...va/fr/free/nrw/commons/filepicker/PickedFiles.java
0% <0%> (ø)
:arrow_up:

...va/fr/free/nrw/commons/utils/SwipableCardView.java
0% <0%> (ø)
:arrow_up:

...free/nrw/commons/campaigns/CampaignsPresenter.java
0% <0%> (ø)
:arrow_up:

...rw/commons/filepicker/FilePickerConfiguration.java
0% <0%> (ø)
:arrow_up:

...ava/fr/free/nrw/commons/filepicker/FilePicker.java
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de9a72d...f87bcab. Read the comment docs.


make launch error 1: Util/BuildTools/Linux.mk:137: recipe for target 'setup' failed make: *** [setup] Error 1

Hello,
CARLA version: 0.9.13
Platform/OS: Ubuntu 18.04
Problem you have experienced: make: *** [launch] Error 1
What you expected to happen: launch Carla in server mode or use the Unreal Engine editor
Steps to reproduce: followed documentation till Build Carla step 2
Other information (documentation you consulted, workarounds you tried):
tried to run without using sudo and getting error:  cp: cannot create regular file '/home/test/carla/PythonAPI/carla/dependencies/lib/libxerces-c.a': Permission denied
when tried to run using sudo getting error: make[1]: Leaving directory '/home/test/carla/Unreal/CarlaUE4'
BuildCarlaUE4.sh: Launching UE4Editor...
Refusing to run with the root privileges.
Util/BuildTools/Linux.mk:7: recipe for target 'launch' failed
make: *** [launch] Error 1
I successfully followed the documentation to run Carla 0.9.13 until the step to compile the server and now getting the following error:
test@test:~/carla$ sudo make launch
[sudo] password for test:
Setup.sh: llvm-8.0 already installed.
Setup.sh: boost-1.72.0-c8 already installed.
Setup.sh: rpclib-v2.2.1_c5-c8 already installed.
Setup.sh: gtest-1.8.1-c8 already installed.
Setup.sh: recast-0b13b0-c8 already installed.
Setup.sh: Libpng already installed.
Setup.sh: Xerces-c already installed.
Setup.sh: Sqlite already installed.
Setup.sh: PROJ already installed.
Setup.sh: Patchelf already installed.
Setup.sh: CARLA version 0.9.13-1-g8854804f4-dirty.
Setup.sh: Generating CMake configuration files.
Setup.sh: Success!
BuildLibCarla.sh: Building LibCarla "Server.Release" configuration.
ninja: no work to do.
[0/1] Install the project...
-- Install configuration: "Server"
BuildLibCarla.sh: Success!
BuildCarlaUE4.sh: Using Unreal Engine at '/home/test/UnrealEngine_4.26'
BuildCarlaUE4.sh: Build CarlaUE4 project.
make[1]: Entering directory '/home/test/carla/Unreal/CarlaUE4'
bash "/home/test/UnrealEngine_4.26/Engine/Build/BatchFiles/Linux/RunMono.sh" "/home/test/UnrealEngine_4.26/Engine/Binaries/DotNET/UnrealBuildTool.exe" CarlaUE4Editor Linux Development  -project="/home/test/carla/Unreal/CarlaUE4/CarlaUE4.uproject"
Running Mono...
Fixing inconsistent case in filenames.
Setting up Mono
~/UnrealEngine_4.26/Engine ~/carla/Unreal/CarlaUE4
Using 'git status' to determine working set for adaptive non-unity build (/home/test/UnrealEngine_4.26).
Target is up to date
Total execution time: 3.03 seconds
make[1]: Leaving directory '/home/test/carla/Unreal/CarlaUE4'
BuildCarlaUE4.sh: Launching UE4Editor...
Refusing to run with the root privileges.
Util/BuildTools/Linux.mk:7: recipe for target 'launch' failed
make: *** [launch] Error 1
test@test:~/carla$ make launch
Setup.sh: llvm-8.0 already installed.
Setup.sh: boost-1.72.0-c8 already installed.
Setup.sh: rpclib-v2.2.1_c5-c8 already installed.
Setup.sh: gtest-1.8.1-c8 already installed.
Setup.sh: recast-0b13b0-c8 already installed.
Setup.sh: Libpng already installed.
Setup.sh: Xerces-c already installed.
cp: cannot create regular file '/home/test/carla/PythonAPI/carla/dependencies/lib/libxerces-c.a': Permission denied
Util/BuildTools/Linux.mk:137: recipe for target 'setup' failed
make: *** [setup] Error 1
Please help as I am confused about what to do next.
Thank you in advance!

same question,sad...

Same problem

Ubuntu 20.04.5 LTS
Python 3.8
The solutions of #3547 don't work.
I guess the same error is in #4650
I followed the steps in https://carla.readthedocs.io/en/0.9.13/build_linux/
UnrealEngine_4.26 is installed, the right branch is used, and in the .bashrc file
export UE4_ROOT=~/UnrealEngine_4.26

The Unreal Engine Editor itself works.
cd ~/UnrealEngine_4.26/Engine/Binaries/Linux && ./UE4Editor

The error happens with
https://github.com/carla-simulator/carla/tree/0.9.13
https://github.com/carla-simulator/carla

For other users that found this Issue.
Try
sudo make launch
You have to use "make launch" after the boots lines, because you will get a UE4_ROOT is not defined later.
For me, it worked.

Is anyone able to solve this issue?
I am installing Carla in Ubuntu 20.04 using UE 4.26 (python 3.9)
when I'm trying to compile the server using the command 'make launch' I get the following error:
make[1]: *** [Makefile:315: CarlaUE4Editor] Error 6
make[1]: Leaving directory '/home/snaray23/carla/Unreal/CarlaUE4'
make: *** [Util/BuildTools/Linux.mk:7: launch] Error 2
can anyone help me out?

Python 3.9 doesn't work.
You have to use 3.7 or 3.8
"Make launch" is using python3 to do stuff.
If you want to change the python that is used, it is better to set it in the setup file.
Util/BuildTools/Setup.sh
If thay didn't work, test if Unreal is installed correctly.

./Setup.sh
./GenerateProjectFiles.sh
make

If it still don't work, try to add this like to your .bashrc
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:/usr/bin/python3.8"
If you may have to change the path to the right python version.

I got the error solved. I was using UEs own 4.26.1 version but we need to use the Carla variant of UE. After changing my UE, giving a clean, PythonAPI and launch solved the issue. I was still using python 3.9 while solving the error.

same question!!!!   I installed 0.9.14 for success,when I changed 0.9.13,I get the error.

I am developing Carla using Ubuntu22.04 and Python3.10. After some adjustments, the build finally succeeded.

My experience is as follows:
If an error is reported:

Util/BuildTools/Linux.mk:101: recipe for target ‘setup’ failed
make: *** [setup] Error 128

Modify line 121 of /home/xiaomi/carla/Util/BuildTools/setup.sh
BOOST_VERSION=1.80.0=>BOOST_VERSION=1.81.0
If an error is reported:

CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
Failed to find XercesC (missing: XercesC_VERSION)
Call Stack (most recent call first):
/usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-3.22/Modules/FindXercesC.cmake:110 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:71 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/xiaomi/carla/Build/libosm2dr-build/CMakeFiles/CMakeOutput.log".
make: *** [Util/BuildTools/Linux.mk:152: osm2odr] Error 1

Modify: /home/xiaomi/carla/Util/BuildTools/setup.sh line 432
XERCESC_VERSION=3.2.3=>XERCESC_VERSION=3.2.4
/home/xiaomi/carla/Util/BuildTools/BuildOSM2ODR.sh lines 108 and 109
xerces-c-3.2.3-install=>xerces-c-3.2.4-install

ok i finally success but idnot know if the solution can help you guys or not
see link“https://link.zhihu.com/?target=https%3A//ftp.cixug.es/apache/xerces/c/3/sources/”for newest version
修改一：~/carla/Util/BuildTools/Setup.sh      428lins change to  XERCESC_VERSION=3.2.5
修改二：~/Carla/Util/BuildTools/BuildOSM2ODR.sh 100lines change to below
-G "Eclipse CDT4 - Ninja"
-DCMAKE_INSTALL_PREFIX=${LIBCARLA_INSTALL_CLIENT_FOLDER}
-DPROJ_INCLUDE_DIR=${CARLA_BUILD_FOLDER}/proj-install/include
-DPROJ_LIBRARY=${CARLA_BUILD_FOLDER}/proj-install/lib/libproj.a
-DXercesC_INCLUDE_DIR=${CARLA_BUILD_FOLDER}/xerces-c-3.2.5-install/include
-DXercesC_LIBRARY=${CARLA_BUILD_FOLDER}/xerces-c-3.2.5-install/lib/libxerces-c.a \
ninja osm2odr
ninja install
and  repeat

make PythonAPI

For other users that found this Issue.

Try sudo make launch
You have to use "make launch" after the boots lines, because you will get a UE4_ROOT is not defined later.
For me, it worked.
but i came across the error like this:
BuildCarlaUE4.sh: Launching UE4Editor...
Refusing to run with the root privileges.
Util/BuildTools/Linux.mk:7: recipe for target 'launch' failed
make: *** [launch] Error 1

I should explain the solution a bit, to make it easier to understand (I switched accounts)
Carla is compiling one package after each other.
If you run the "make launch" command again, it will look for installed packages (installed in the Carla folder), and skip the existing one.
One of the packages had a problem with the "make launch" permission or other problems with non-root permissions.
For that reason, if you use "sudo make launch", it will give the package compiler root permissions, but not every package wants root permissions.
boots want the "UE4_ROOT" set, which is not set on the root account but on your user account.
For that reason you have to use "make launch" for boots
For example:
packages
1: non-root
2: non-root
3: root
4: non-root
5: non-root
With "make launch" it works for packages 1 and 2 and then gives an error.
If you use "sudo make launch" after that, it will skip packages 1 and 2, and work on package 3.
Package 4 needs a non-root again, it will give an error with "sudo make launch", but you can use "make launch" because you are done with package 3.
I edited my Setup.sh file after that and don't have to go in that direction again.
If you don't want to do that, just switch between "make launch" and "sudo make launch" until it works.
If it doesn't work, it could have another problem or the solution is outdated.
These are the changes I made to the setup.sh file until now (0.9.13).
carla/Util/BuildTools/Setup.sh
# line 15 (old)
VERSION_LIST=3
# line 15 (new)
VERSION_LIST=3.8

# line 133 (old)
wget "https://boostorg.jfrog.io/artifactory/main/release/${BOOST_VERSION}/source/${BOOST_PACKAGE_BASENAME}.tar.gz" || true
# line 133 (new)
wget "https://carla-releases.s3.eu-west-3.amazonaws.com/Backup/${BOOST_PACKAGE_BASENAME}.tar.gz" || true

# line 432 (old)
XERCESC_REPO=https://ftp.cixug.es/apache//xerces/c/3/sources/xerces-c-${XERCESC_VERSION}.tar.gz
# line 432 (new)
XERCESC_REPO=https://archive.apache.org/dist/xerces/c/3/sources/xerces-c-${XERCESC_VERSION}.tar.gz

I made some changes in the BuildUtilsDocker.sh file, too
carla/Util/BuildTools/BuildUtilsDocker.sh
# line 16 (old)
BXSDK_URL=https://www.autodesk.com/content/dam/autodesk/www/adn/fbx/2020-0-1/${LIB_NAME}.tar.gz
# line 16 (new)
FBXSDK_URL=https://damassets.autodesk.net/content/dam/autodesk/www/adn/fbx/2020-0-1/${LIB_NAME}.tar.gz

# line 20 (old)
wget -c "${FBXSDK_URL}" -P "${CARLA_DOCKER_UTILS_FOLDER}"
# line 20 (new)
wget -c "${FBXSDK_URL}" -P "${CARLA_DOCKER_UTILS_FOLDER}" --user-agent="Mozilla"

邮件已收到，谢谢！（来自QQ邮箱自动回复邮件）

thank you very much for your reply. As a beginner using the Carla project, many solving methods are still in the exploratory stage. Your reply has taught me a lot. Thank you! i tried 1.use"make launch",get error boost_1_72_0.tar.gz: Insufficient permissions Unable to write to 'boost_1_72_0.tar.gz' (success). Setup.sh: Extracting boost for Python 3. tar (child): boost_1_72_0.tar.gz: cannot open: No such file or directory tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now Util/BuildTools/Linux.mk:137: recipe for target 'setup' failed make: *** [setup] Error 2 2.use"sudo make launch",get error BuildCarlaUE4.sh: Launching UE4Editor... Refusing to run with the root privileges. Util/BuildTools/Linux.mk:7: recipe for target 'launch' failed make: *** [launch] Error 1 so neither can complete launch work,sad(;--;)

so now iam thinking maybe reinstall my UE4.26

I forgot to write down, that you have to set the permissions after the "sudo make launch" command because the files were created with root.
It is good that it works now

The error you encountered, specifically "Error 137" and the "Killed" message, typically indicates that the system ran out of memory (OOM, or Out Of Memory).
Adding more swap space can help if your system runs out of RAM. You can create a swap file to increase the virtual memory.
# Check your current swap space
sudo swapon --show

# Create a swap file (e.g., 8GB)
sudo fallocate -l 8G /swapfile

# Set correct permissions
sudo chmod 600 /swapfile

# Set up the swap area
sudo mkswap /swapfile

# Enable the swap file
sudo swapon /swapfile

# Verify if the swap is active
sudo swapon --show

your email has been received, thank you!邮件已收到，谢谢！


getUserMedia throws TypeError in Chrome for Android

Browsers and versions affected
At least Chrome 37 and Chrome 42 on Android 5.1.1 but I assume Chrome 44 and below in general.
Description
Calling navigator.mediaDevices.getUserMedia throws TypeError: undefined is not a function.
I believe this line is where the error comes from. find seems to be the function that is undefined here, which makes sense to me since Array.prototype.find was first introduced in Chrome 45.
I forwarded this from another repository. For more details you can go to the original issue.
Steps to reproduce
The exact call that produces the error:
navigator.mediaDevices.getUserMedia({
  audio: false,
  video: {
    facingMode: { ideal: 'environment' },
    width: { min: 360, ideal: 1280, max: 1920 },
    height: { min: 240, ideal: 720, max: 1080 }
  }
})

You can also visit this web page where the error occurs.

Well we are using standard API's in the browser to do certain things and its not reasonable to have to shield every API call due to some users using 3 year old browsers which lack common API's like Array.prototype.find .


Fix Travis-CI redis tests

Add missing @expectedException to testBadDisconnect() test so expected warning raised by Redis::connect() wont break test.
Also added sudo: required to .travis.yml because it is not set by default on new repos, so forked repos like mine won't fail to build now.

Coverage remained the same at 95.534% when pulling a88753674448be6ed1dc4bb2a0c30914fcbc6ee8 on thesebas:fixtravisredistests into da0c3daf1c4ffd6d9e89f0c258da457027544fa9 on tedious:master.

This PR together with #342 should fix failing Travis tests.

Coverage remained the same at 95.534% when pulling 402116fdc01abf03adaeb4aae370a05fccf627a0 on thesebas:fixtravisredistests into 5f57ac7c763b0cf6ace8f1f8c216c4250f919ae3 on tedious:master.

LGTM


Add CODE_OF_CONDUCT.md file

Hi there!
I recommend adding a CODE_OF_CONDUCT.md file to your repository. This file would serve as a guide for potential contributors, providing them with a clear set of expectations, guidelines, and behavior standards for participating in your project. It plays an essential role in creating a welcoming and inclusive environment for everyone involved.
I would like to work on this issue. Please assign it to me under GSSoC'23.
Thank you!

Hi @ImranNawar,
Thanks for your interest in making meaningful contribution to this project.
Please go through FAQs & Getting Started Guide for GSSoC students to get started with your first contribution to this project.
Closing this issue as GitHub already provides a template to add CODE_OF_CONDUCT.md
Thanks.


Metadata

Will fix #118

@jonnor added the requested test. Storing metadata to @participants is not the best option, but I would elect to do it that way in this PR, as otherwise we'll get scope creep towards desired/actual. That should be done next, though.


java.lang.NullPointerException at io.github.sleroy.sonar.EsLintSensor.execute(EsLintSensor.java:121)

The issue is similar to https://github.com/sleroy/SonarEsLintPlugin/issues/2
I'm running sonar scanner from jenkins or command line (same issue with both) with this config file:
sonar.projectKey=project
sonar.projectName=project
sonar.projectVersion=0.1
sonar.sources=.
sonar.inclusions=src/**
sonar.scm.provider=git
sonar.eslint.reportPaths=eslint-report.json
sonar.host.url=https://sonar.domain.com
sonar.login=blabla
eslint-report.json does exist and has valid json data. It has been produced by eslint with json output format.
13:20:21.765 DEBUG: Executing EsLint with command: node J:\wks\fa723ab7\node_modules\eslint\bin\eslint.js -f json --output-file J:\wks\fa723ab7\.scannerwork\.sonartmp\5493670960467370263.tmp --config J:\wks\fa723ab7\.eslintrc.json J:/wks/fa723ab7/src/Project/code/Scripts/components/tile/tile-item.vue etc
13:20:25.192 INFO: ------------------------------------------------------------------------
13:20:25.195 INFO: EXECUTION FAILURE
13:20:25.195 INFO: ------------------------------------------------------------------------
13:20:25.195 INFO: Total time: 54.270s
13:20:25.285 INFO: Final Memory: 35M/1373M
13:20:25.285 INFO: ------------------------------------------------------------------------
13:20:25.285 ERROR: Error during SonarQube Scanner execution
java.lang.NullPointerException
        at io.github.sleroy.sonar.EsLintSensor.execute(EsLintSensor.java:121)
        at org.sonar.scanner.sensor.SensorWrapper.analyse(SensorWrapper.java:45)

I don't understand why sonar-scanner still tries to run eslint as i provided the report file in the config file, it should just import the result no?
I tried with or without the sonar.eslint.reportPaths and i get NPE in both cases.
Sonar scanner version is <IP_ADDRESS>7

Working fine with eslint plugin v0.3.0 (had to install it manually though, not visible in sonarqube update center).
Sorry for the noise

SonarQube marketplace will never accept this plugin.


Geometry engine bugfixes

This PR fixes three important bugs in the geometry engine:

The correct growth index was not always being used in the valence System because of rounding errors
Ring-closing torsions were not being added with their appropriate growth index
Hydrogen atoms were being grown before heavy atoms, causing rings to sometimes not close

Geometries look great now, except for intramolecular steric clashes, which I think can be resolved by turning on sterics and exceptions. I think this problem is solved.

I've also added:

Exclusions when sterics are enabled (which should fix the high-energies and NaN issues we were seeing)
(1,4) exceptions are always included in valence terms

The one other thing we may want to do is to change the simple binary use_sterics option to one that allows us to select either intramolecular sterics only (which should be super-fast) or all sterics (which might help increase acceptance rates at the cost of being slower).

I'm going to go ahead and merge this, since tests pass and bugs have been fixed.


Upload Clips Posted on Discord to YouTube

The bot should be able to take clips posted via 3rd party tools such as outplayed to a YouTube channel.

[x] #4
[x] #5

Bot can take an outplayed.tv message in discord, edit the video by finding the first highlight with the neural network, then adding music at the same point, then chancing aspect ratio to 16:9 (youtube shorts are all 16:9), then save the video, open up chromium browser and uploads the video, posting the link to the short in chat.


Update configuring.rst - typo fix

Fixed some typos in the code examples by replacing ':' with '=' in a few places.

Much appreciated! Thanks very much :-)

No problem :)


Add option to change Boolean (checkbox) type

Adds option to change checkbox type.
Options are:

checkbox (default)
slider
toggle

I refactored the code a bit, it is a very nice addition to the package's possibilities!
Thanks for the contribution @richarddewit.

I saw :+1: is it also updated on Atmosphere?

Yes it is.