Restic forget fails with B2 and object locks despite --no-lock (and a crash in forget when remote lock fails)

Output of restic version
restic 0.12.1 compiled with go1.16.6 on linux/amd64
How did you run restic exactly?
restic --no-lock -r b2:restic-backup-2:/ forget e2f7d85
I have set these environment variables (with real values of course):
export B2_ACCOUNT_ID=00000000000000000000000
export B2_ACCOUNT_KEY=K001abcdabdbabcdabcdabcd
export RESTIC_PASSWORD_FILE=/home/user/restic-pw
This is the output of the command:
repository 35b4e101 opened successfully, password is correct
unable to create lock in backend: repository is already locked exclusively by PID 32487 on HOST by user (UID 1002, GID 1003)
lock was created at 2021-08-23 10:53:33 (42m34.573443863s ago)
storage ID 52256a07
the unlock command can be used to remove stale locks
The strange thing is that I have specified --no-lock on all commands, but a lock was made regardless.
What backend/server/service did you use to store the repository?
Backblaze with object locks enabled (with time set to two days as I was testing the feature)
Maybe this is relevant. I did CTRL-C my first backup proces at one point, after which restic has been complaining about "Access denied" (because of object lock I suppose).
Expected behavior
Expected restic being able to forget a snapshot (and also delete the stale files created by interrupted backup after few days as object lock period for the files created on the interrupted backup has expired). I tried to forget a backup created a week ago or so.
Also expected no locks being created on the backend due to "--no-lock" specified to restic.
Also didn't expect restic unlock to crash.
Actual behavior
Restic forget fails. This is the output of the
restic --no-lock -r b2:restic-backup-2:/ forget e2f7d85
repository 35b4e101 opened successfully, password is correct
unable to create lock in backend: repository is already locked exclusively by PID 32487 on HOST by user (UID 1002, GID 1003)
lock was created at 2021-08-23 10:53:33 (42m34.573443863s ago)
storage ID 52256a07
the unlock command can be used to remove stale locks
The restic unlock does not work either now (because the lock cannot be cleared due to the object locks).
This is the output of the
restic --no-lock -r b2:restic-backup-2:/ unlock
repository 35b4e101 opened successfully, password is correct
Remove(<lock/52256a0740>) returned error, retrying after 552.330144ms: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 1.080381816s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 1.31013006s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 1.582392691s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 2.340488664s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 4.506218855s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 3.221479586s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 5.608623477s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 7.649837917s: Delete: b2_delete_file_version: 401: Access Denied
Remove(<lock/52256a0740>) returned error, retrying after 15.394871241s: Delete: b2_delete_file_version: 401: Access Denied
b2_delete_file_version: 401: Access Denied
Delete
github.com/restic/restic/internal/backend/b2.(*b2Backend).Remove
/restic/internal/backend/b2/b2.go:261
github.com/restic/restic/internal/backend.(*RetryBackend).Remove.func1
/restic/internal/backend/backend_retry.go:108
github.com/cenkalti/backoff/v4.RetryNotifyWithTimer
/home/build/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.1/retry.go:55
github.com/cenkalti/backoff/v4.RetryNotify
/home/build/go/pkg/mod/github.com/cenkalti/backoff/v4@v4.1.1/retry.go:34
github.com/restic/restic/internal/backend.(*RetryBackend).retry
/restic/internal/backend/backend_retry.go:46
github.com/restic/restic/internal/backend.(*RetryBackend).Remove
/restic/internal/backend/backend_retry.go:107
github.com/restic/restic/internal/cache.(*Backend).Remove
/restic/internal/cache/backend.go:38
github.com/restic/restic/internal/restic.RemoveStaleLocks.func1
/restic/internal/restic/lock.go:282
github.com/restic/restic/internal/restic.ForAllLocks.func2
/restic/internal/restic/lock.go:336
golang.org/x/sync/errgroup.(*Group).Go.func1
/home/build/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1371
Steps to reproduce the behavior
I can retry and update the reproduction instructions tomorrow, as the old bucket is now unusable for two days do to the object locks.
I now created a new bucket with 1 day of object locks, so I can test things sooner, but will have to wait until tomorrow until I can try delete a snapshot.
But even creating the backup with --no-lock gave the error about not being able to delete the lock file.
Do you have any idea what may have caused this?
restic trying to use locks even if specified it should not. Not sure how working with object locks (or immutability in the backend) should be handled).
Do you have an idea how to solve the issue?
Not atm. Just wait until the object lock time runs out, but before that can't do much as back-end is strict.
Did restic help you today? Did it make you happy in any way?
I like the docs generally, and am just getting into restic, it looks like a well-maintained tool. Just think the operation on object-locked storage is not that well-specified/used.

The strange thing is that I have specified --no-lock on all commands, but a lock was made regardless.

Several commands like forget, prune, check and backup ignore the --no-lock option.

Backblaze with object locks enabled (with time set to two days as I was testing the feature)

restic assumes that it can delete lock files at any time (after 5 minutes to be precise). And to prune data restic must be able to delete any files it wants to. Or to be a bit more precise the files must be marked as deleted and don't show up again when listing files. With AWS S3 and the Object Lock feature buckets must be versioned which allows doing exactly what I've just mentioned. I wonder whether there's a similar feature for B2?

You could also try whether the S3 compatible API of B2 works with a beta release of restic (see https://beta.restic.net/?sort=time&order=desc ). restic 0.12.1 probably won't work as #3246 in not included in any released version so far.

Or you could try whether #2398 works for you.

Thanks. So I gather that working in read-only (or temporarily read-only due to object locked) repositories only works for taking snapshots, but not forgetting them at the moment?
AFAIK the above #2398 just hides the error if the deletion fails, so wont improve the actual behavior, right?. Atm I have also no easy way of building custom binaries based on PRs unfortunately.

#2398 tells B2 to hide a file, when it can't be deleted. That is enough that other restic operations won't process that file again and then forget and prune should work as normal. The only downside is that the hidden files still consume storage space. The hidden files still have to be deleted by e.g. lifecycle rules. And something also has to extend the file lock duration of files that should be kept.
If you check out the code from a PR, then you only need a go compiler and then call go build ./cmd/restic and you're done.

Thanks Michael. That something that could extend the file lock duration I suppose is not in place yet? Can't ask Restic to "touch" the files in B2 (that are still part of the latest snapshot), which I suppose would extend their life?

restic doesn't know anything about file / object locks so far. For now, it should work to (manually) extend the lock duration for all files that haven't been hidden. And to actually make sure that the repository is in a working state you have to run check from time to time.

@MichaelEischer Finally got the restic compiled from the #2398. Was a bit harder as I had an older golang-go version (1.6) which didn't work out of the box. Build failed for missing deps (fuse etc). After realising I do need to update the go instead of trying to manually fulfill the deps, it was an easy build. Maybe a version check of the golang-go to the build command to help future builders in this situation?
I tested the #2398 and it does indeed work better, by hiding files instead of deleting which works for me.
I also actually tried restoring an initially hidden snashots file in the repository side (after forgetting it first with restic), and then restoring the snapshots file manually in the b2 server side.
So, I could achieve nice ransomware safe backups with this using efficient snapshots to go back in time if needed. If the attacker manages to get hold hold of the backup side where restic is run, my backups would be still safe (thanks to object locks), and file versions on the server side (and life cycle rules).
I could actually live with these limitations with the PR, as they do give me truly ransomware safe backups albeit with manual work required in the case attacker has maliciously tried deleting all. The manual work is today required to restore the older versions on server side (especially for older data files, that object locks do not protect anymore). The restore in this case would involve some scripting to get the repo to an earlier state.
I guess the only few remaining optimizations that could be great features

Make #2398 an option in the main restic. Having to maintain an own build is not great.
Add some knowledge on object locks on restic. This would also be helpful with "cold backups" as well so that restic would be "forbidden" to modify files within some time period (to save money/costs). The object lock information could for example be given in the init, or be an backup option. Then in regular use case, user could just be running backup/prune without any scripting required or server side tricks.   If the super motivated attacker tried altering the object lock information, and deleting backups (that are object locked), the operation would just fail but on normal cases the operations would be seamless.
Some way of extending the object locks would be required (as you also point out earlier).

There are few other folks trying also achieve what I am trying:
https://github.com/restic/restic/issues/3195

Thanks for testing and nice to hear that it worked.
The manual and readme already mention that at least Go version 1.14 is required. There's also the possibility to build restic via go run build.go which includes a go version check, although I'm not sure whether that would have worked with Go 1.6. Sorry for not mentioning that a recent go version is required; it's just that nobody had complained so far ;-) .
I agree that object lock support would simplify usage with restic, but that feature request belongs into #3195.

@gabrielwhite Your problem is unrelated to this issue. Please have a look at #3541 instead.

Closing as #2398 has been merged a long time ago.


DNS Unable to resolve to server IP

The DNS of explorabl.es is not resolving for me? Is the site down? Did the domain expire?

Hi, thanks for catching this!  I recently updated the site to HTTPS.  The DNS changes are probably still propagating.  Is the site working for you now?


[BUG] Get-PnPAzureADUser not working anymore in 2.10.0

Reporting an Issue or Missing Feature
After upgrading to 2.10.0, the cmdlet Get-PnPAzureADUser is giving an error. Downgrading to 2.9.0 resolves the issue.
Expected behavior
No error after upgrade.
Actual behavior
The cmdlet gives the following error:
OUTPUT: Error: AADSTS1002012: The provided value for scope User.Read.All is not valid. Client credential flows must have a scope value with /.default suffixed to the resource identifier (application ID URI). Trace ID: c7c52944-7c74-4cf2-9319-7b2114764b00 Correlation ID: d77c02ec-4fec-4bb2-9128-2ba1db41866d Timestamp: 2024-08-30 09:30:20Z
Steps to reproduce behavior
I'm running a powershell script in an Azure Function. Relevant code:
$c = Connect-PnPOnline -Url $TenantAdminUrl -Tenant $TenantDomain -ClientId $ClientId -Thumbprint $CertificateThumbprint -ReturnConnection
$user = Get-PnPAzureADUser -Filter "UserPrincipalName eq '$userId'" -Select "AssignedPlans" -Connection $c

What is the version of the Cmdlet module you are running?
2.10.0
Which operating system/environment are you running PnP PowerShell on?

[ ] Windows
[ ] Linux
[ ] MacOS
[ ] Azure Cloud Shell
[X] Azure Functions
[ ] Other : please specify

Additional information:
Running in an Azure Function, in requirements.psd1 I had 'PnP.PowerShell' = '2.*'. Therefore there was automatically an upgrade to 2.10.0 in the last few days, after which every run of the function gave the error. Changing requirements.psd1 to 'PnP.PowerShell' = '2.9.0' (and adding Import-Module PnP.PowerShell -RequiredVersion '2.9.0' to profile.ps1) all runs ended without errors again.

Hi @RobinDijkstra , apologies for this.
We have fixed this issue , impacts quite a lot of other cmdlets as well.
The fix will be available in tomorrow's nightly builds as well as the next major version


Remove box-shadow from UI, fix dashboard issue

Two tweaks here:

Remove box-shadow from menus and segments. I think it generally looks better this way because most of the UI elements are already flat, and the box-shadowed elements look out-of-place.

Before

After

Fix a issue where the repo list in the dashboard sidebar was 2px wider than the pill menu above it. The cause of the issue looks to be some hacks in Semantic UI and that added artificial width to that repo box.

Before

After

Fun fact: gogs has done a similar change on hiding the box-shadows. I just went ahead and adopted the change as it seemed cleaner.

Codecov Report

Merging #3065 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3065      +/-   ##
==========================================
+ Coverage   33.43%   33.45%   +0.01%
==========================================
  Files         270      270
  Lines       39553    39553
==========================================
+ Hits        13226    13232       +6
+ Misses      24441    24429      -12
- Partials     1886     1892       +6

Impacted Files
Coverage Δ

models/repo.go
37.96% <0%> (-0.19%)
:arrow_down:

modules/indexer/repo.go
63.47% <0%> (+2.6%)
:arrow_up:

models/repo_indexer.go
51.98% <0%> (+2.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9db60e...9a43ffe. Read the comment docs.

LGTM

LGTM

@silverwind please resolve conflicts

Rebased.

Can you please force push?

done


Don't make texture views where not necessary

This is mainly to work around a bug in macOS Sonoma where Radeon Pro 580Xs read black if you attempt to sample a D24S8 texture through a texture view that was previously copied from (so texture goes from copy source → shader read with texture view) [FB13342692]
But less texture views also makes things easier to track in the Xcode shader debugger, so might as well apply it always

Looks like the issue was actually "fixed" by an update to the game that happened while I was testing and slightly changed how the game renders, hiding the issue.  The actual reason for the texture view was that it needed a R001 swizzle, so we'll have to wait for Apple to fix the bug.


[pred-memopts] Add a large file comment to PMOMemoryUtils.h explainin…

…g the history of the utilities and that the code in DIMemoryUtils is similar due to this historical relationship.

@atrick this is the follow on PR for #16774.

@swift-ci smoke test and merge


Migrate operations module to Server Preset (Part 21)

Background
This migrates operations module to Server Preset

@eddeee888 can you please resolve the conflicts?

Done! Thanks for reviewing this @n1ru4l


Use a single worker for cluster spinup time

Currently we're using the default number of workers (i.e. 4) for measuring how fast a cluster spins up. This PR proposes we only spin up a single worker. This will, I think, provide the same rough time estimate we're after, while using fewer AWS resources.
cc @ntabris

Upstream keeps failing but this is unrelated to the PR, I will go ahead and merge this.


feat: deprecate Web search integration

I think it's time to admit we're not going to fix and maintain this feature soon.
Removing it will reduce friction in other development and maintenance of the search experience.
And there's no rule we can't reach into source history and pick up and run with this feature whenever that becomes the right thing to do.

charge us money, but also keep us on the old version of the search api. I'm sure that old version won't live forever, and in the mean time we would be losing out on the newer features. Our next move is to refactor to support the new API.

There's a googleCustomSearchService in the marketplace controller in uportal-home.
Can that come out as well?

wrt to your scope of deprecation question, @Doug-Reed , In the most immediate next step, I'd say the dynamic, JSON-API-integrating aspects of this feature deprecate and drop out, whereas the just-paint-a-hyperlink-launching-external-web-search feature should remain, as that feature could work just fine if we'd put the link on the page where a user could click it.
But in the most most immediate next step, this is just documentation deep, signalling that this is deprecated and seeing whether and what reactions we get. :smile:


Failed to launch Godot instance when trying to Run and Debug Godot 4 projects (non-Mono projects)

OS/device including version:

Windows 10
Godot 4.0-beta1

Issue description:

VS Code always fail to Run and/or Debug a non-Mono Godot 4-beta1 project, with the error:
Failed to launch Godot instance: Error: Command failed: "C:/Dev/Godot/Godot4.exe" --path "P:/Dev/Godot/projects/GameFolder" --remote-debug <IP_ADDRESS>:6031

Note: I only found issues related to Godot 4 and Mono, that's why I'm opening this one, because it's for non-Mono projects.
Setup

To make sure it's not a port conflict, I even changed the default port number.
launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "GDScript Godot",
            "type": "godot",
            "request": "launch",
            "project": "${workspaceFolder}",
            "port": 6031,
            "address": "<IP_ADDRESS>",
            "launch_game_instance": true,
            "launch_scene": false
        }
    ]
}

Screenshots of issue:

Sorry, I posted in the wrong repository.


I would like to reference a config map as a data source for an external secret.

Is your feature request related to a problem? Please describe.
I'm currently using External Secrets to template secrets for PostgreSQL connections:
Format
postgres://{{ .username }}:{{ .password }}@{{ .endpoint }}:{{ .port }}/{{ .dbName }}?sslmode={{ .sslmode }}

But the PostgreSQL helm chart that I'm using only stores the password key in a secret. The rest of these values are only stored in the generated configmap.
Describe the solution you'd like
I'd like a way to reference a value in a configmap the same way we can currently reference values in secrets:
For example:
    - configKey: <name-of-key>
      remoteRef:
        key: <config-map-name>
        property: <config-map-key>

Describe alternatives you've considered
I'm currently hard-coding the non-sensitive values into the template, but this is not ideal because most of these values are generated from other helm charts and can change over time.
Additional context
N/A

hi @vyas-n  - templates already support using configmaps as basis. Can you confirm this is the feature you need?

Hey @gusfcarvalho,
Not in this case, if I understand the existing config map support, it has to have the full go template with variable placeholders that can only be replaced with secrets.
That's not the situation I have. Here's a clearer example of what I'm trying to do:
apiVersion: v1
kind: ConfigMap
data:
  database: <redacted>
  username: <redacted>
  sslmode: <redacted>
  port: <redacted>
  endpoint: <redacted>

apiVersion: v1
data:
password:
kind: Secret

The above Secret & ConfigMap are dynamically generated, so I don't have the ability to hardcode the values into the template.

The feature I'm asking for is to use an ExternalSecret template to pull values from both the above Secret & the ConfigMap:

postgres://{{ .username }}:{{ .password }}@{{ .endpoint }}:{{ .port }}/{{ .database }}?sslmode={{ .sslmode }}

How about this?
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: database
data:
  username: <redacted>
...
---
apiVersion: v1
kind: Secret
metadata:
  name: database
data:
  password: <redacted>
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: externalsecret
spec:
  refreshInterval: 1m
  target:
    template:
      engineVersion: v2
      data:
        DSN: "postgres://{{ .username }}:{{ .password }}@{{ .endpoint }}:{{ .port }}/{{ .database }}?sslmode={{ .sslmode }}"
  data:
    - secretKey: username
      configMapRef:
        name: database
        key: username
    - secretKey: password
      secretRef:
        name: database
        key: password
...

No secretStoreRef or remoteRef included as nothing is from a remote secret store. I'd certainly like to retain the option to pull from all three at the same time though.
configMapRef and secretRef the same as envFrom for containers.
I almost certainly have zero ability to implement any of this, sorry, I just want it.

@gaima8,
Just to be clear, is the example that you provided existing functionality with the External Secrets Operator or a proposed syntax for new functionality?

@gaima8,
Just to be clear, is the example that you provided existing functionality with the External Secrets Operator or a proposed syntax for new functionality?

Sorry, syntax for proposed new.


docs: remove module-metadata.json and commitlint.config.js

/run pipeline

:tada: This PR is included in version 1.0.9 :tada:
The release is available on:

GitHub release
v1.0.9

Your semantic-release bot :package::rocket:


planet habitabilities

For some reason, the game doesn't recognise the names of the planet class preference traits.
Here's an extract from the log:
[15:51:36][persistent.cpp:34]: Error: "Unexpected token: trait_pc_desert_preference, near line: 15
" in file: "common/traits/01_species_traits_habitability.txt" near line: 63
[15:51:36][persistent.cpp:34]: Error: "Unexpected token: trait_pc_arid_preference, near line: 64
" in file: "common/traits/01_species_traits_habitability.txt" near line: 111
This results in blank pc preference traits without modifiers, effectively preventing every race from colonizing other planets.
The fact is, I really can't understand why the game does that, tha trait names are even the same as the original ones.
Any idea?

Found the culprit and fixed it, opened new pull request.

Hey SiceX!
Can't check your fix until next week, but I trust it's good work as usual and I'll join you in updating the mod to the latest version
Cheers,
LuxArdens


Start migrating e2e tests to Go.

Signed-off-by: Vighnesh Shenoy<EMAIL_ADDRESS>
Provide a description of what has been changed
Start migrating e2e tests to Go for code parity.
Sample Logs
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] Tests have been added
[x] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
[x] A PR is opened to update the documentation on (repo) (if applicable)
[x] Changelog has been updated and is aligned with our changelog requirements

Relates to #2737

Sorry for the delay here, but here's a sample PR with E2E tests in Go.
I have also added sample logs from a test run to see how they look. I feel that the logs are a lot cleaner as well compared to the ones from ava.
@tomkerkhove @JorTurFer @zroubalik

Sorry for the delay here, but here's a sample PR with E2E tests in Go.
I have also added sample logs from a test run to see how they look. I feel that the logs are a lot cleaner as well compared to the ones from ava.
@tomkerkhove @JorTurFer @zroubalik

I'm a bit busy atm, but I'll review it ASAP, huge kudos for this improvement! ❤ ❤ ❤ ❤

I'm a bit busy atm, but I'll review

That would be great. If the sample looks okay, I will start working on migrating all the other Azure scalers as a part of this PR as well. After that the community can start working on others, and help with the migration.

Could you please also update the https://github.com/kedacore/keda/blob/main/tests/README.md in this PR as well?

Looking good! Great job :)
We should find a way how to run these in parallel until the full migration is done.

Could you explain what you mean by this? The run-all.sh file does run the tests in parallel, right?
Or do you mean running the go and ts files in parallel until the full migration is done?

Looking good! Great job :)
We should find a way how to run these in parallel until the full migration is done.

Could you explain what you mean by this? The run-all.sh file does run the tests in parallel, right?
Or do you mean running the go and ts files in parallel until the full migration is done?

yeah, but it's more a matter of modifying GH Workflows. @JorTurFer might have better idea. Let's solve this once this PR is ready.

Could you please also update the https://github.com/kedacore/keda/blob/main/tests/README.md in this PR as well?

Currently, I kept these tests under a folder e2e in the root.
I am thinking of moving these to tests/e2e_go until the complete migration.
After the complete migration, we can remove the contents of tests and rename them tests/e2e_go -> tests.
And yes, I will add a readme to tests/e2e_go/README.md explaining the testing.

0

Could you please also update the https://github.com/kedacore/keda/blob/main/tests/README.md in this PR as well?

Currently, I kept these tests under a folder e2e in the root. I am thinking of moving these to tests/e2e_go until the complete migration.
After the complete migration, we can remove the contents of tests and rename them tests/e2e_go -> tests.
And yes, I will add a readme to tests/e2e_go/README.md explaining the testing.

sounds like a good plan!

Looking good! Great job :)
We should find a way how to run these in parallel until the full migration is done.

Could you explain what you mean by this? The run-all.sh file does run the tests in parallel, right?
Or do you mean running the go and ts files in parallel until the full migration is done?

yeah, but it's more a matter of modifying GH Workflows. @JorTurFer might have better idea. Let's solve this once this PR is ready.

The only concern I have is running both the workflows will increase time required. I think what might be a good approach is, letting the main build that gets run on a PR merge not be changed until complete migration is done.
And, for PRs have a temporary separate command such as /run-e2e-go in parallel to /run-e2e. Once the migration is done we can replace the /run-e2e workflow with the other one.

The only concern I have is running both the workflows will increase time required. I think what might be a good approach is, letting the main build that gets run on a PR merge not be changed until complete migration is done.
And, for PRs have a temporary separate command such as /run-e2e-go in parallel to /run-e2e. Once the migration is done we can replace the /run-e2e workflow with the other one.

uff... Adding a second command could be a pain TBH. The command is a super custom workload that I did, and duplicate it could be horrible. Maybe we can just run both all the time due to we are going to move the tests from one to another, so once we have moved the test to Golang, it shouldn't exist for a long time in typescript.
Other option could be just combining both e2e test files in same folder. Thay have different extensions so we can know which one is each one based on the extension, and having all together can simplify the moving process.
When we add a new e2e test in go, we can just delete the ts file.
WDYT?

One more thing, the command make test is also executing the new e2e tests. Maybe we have to add build tags

I would go the simplest way - execute both test suites and always delete overlapping tests.

Agree. The best part of this is that make e2e-test will continue being the way to trigger them, so main-build and nightly build still work without any other change

I am keeping the go tests in scalers_go and the ts tests in scalers.  I have modified the run-all.sh file to run both go and ts files until the migration is complete.
Replaced the setup.test.ts and cleanup.test.ts with setup_test.go and cleanup_test.go.  And I everytime some adds a test for a scaler which already has a ts test, they should delete the ts file and only keep the go file.
I have also added a section in the README with some instructions and a small example. It isn't the best, but I will try to improve it in the future.
@JorTurFer @zroubalik

I will be adding these tests as a part of this PR (hopefully by Monday).

App Insights
Blob
Data Explorer
Key Vault Queue
Log Analytics
Pipelines
Queue Idle Replicas
Queue Pause At N
Queue Pause
Queue Restore Original Replicas
Queue Trigger Auth
Queue
Service Bus Queue Workload Identity
Service Bus Queue
Service Bus Topic

And hopefully once this PR is merged, the community steps up in helping with the migration and takes up the other scalers.

I will be adding these tests as a part of this PR (hopefully by Monday/Tuesday).
* App Insights

* Blob

* Data Explorer

* Key Vault Queue

* Log Analytics

* Pipelines

* Queue Idle Replicas

* Queue Pause At N

* Queue Pause

* Queue Restore Original Replicas

* Queue Trigger Auth

* Queue

* Service Bus Queue Workload Identity

* Service Bus Queue

* Service Bus Topic

And hopefully once this PR is merged, the community steps up in helping with the migration and takes up the other scalers.

@v-shenoy do you think that you can add only a subset of these tests in this PR? There are some new scalers that could already use the golang version. You can add the rest of the Azure related tests in a subsequent PR. WDYT?

The tests folder will have to be skipped during pr and main workflows, else the go files for the e2e tests will also be run.

Please do that in this PR (also include nightly).

I will be adding these tests as a part of this PR (hopefully by Monday/Tuesday).
* App Insights

* Blob

* Data Explorer

* Key Vault Queue

* Log Analytics

* Pipelines

* Queue Idle Replicas

* Queue Pause At N

* Queue Pause

* Queue Restore Original Replicas

* Queue Trigger Auth

* Queue

* Service Bus Queue Workload Identity

* Service Bus Queue

* Service Bus Topic

And hopefully once this PR is merged, the community steps up in helping with the migration and takes up the other scalers.

@v-shenoy do you think that you can add only a subset of these tests in this PR? There are some new scalers that could already use the golang version. You can add the rest of the Azure related tests in a subsequent PR. WDYT?

Alright, I will add the Service Bus (Queue + Topic + Workload) and Key Vault tests. Does that seem good?

@v-shenoy yeah, thanks 🙏

Initially, I wrote the tests in a way that tried to emulate ava, by separating a test for an individual scaler into multiple go test functions such as TestSetup, TestScaleUp, TestScaleDown, etc. This worked since go executes tests within a file sequentially.
However, I have realized that having go tests depend on each other in a non-trivial way, is just not good practice. Instead, having a single TestScaler function is much better. This can call internal sub-routines passing around the t *testing.T  to keep the test itself from getting too big.
Another thing, I was worried about how to deal with panics, because if a test panics the cleanup code may not run. And this might leave the test resources in an unexpected state. But, this is no different than a test in ava with incorrect cleanup code.
Either way, I have added notes in the README.md emphasizing the need to ensure that the test doesn't panic, or if there's a chance it will then to call recover and cleanup.
What I've done overall -

Added 5 tests for now. Will gradually try to add more later. This much should be enough for others to start contributing go tests.
Deleted the ts tests for the ones that have been added.
Added build tags to ensure the tests don't run in the workflow.
Added a gudie in README.md  and moved the previous README content to OLD-README.md

What do you think?
@JorTurFer @zroubalik

/run-e2e

/run-e2e azure*

/run-e2e

/run-e2e Update: You can check the progres here

I only just remembered I could also run tests, haha. Well, it works, and is going to be really helpful :D

/run-e2e Update: You can check the progres here

I only just remembered I could also run tests, haha. Well, it works, and is going to be really helpful :D

@jorturfer

Another thing, I was worried about how to deal with panics, because if a test panics the cleanup code may not run. And this might leave the test resources in an unexpected state. But, this is no different than a test in ava with incorrect cleanup code.

I think that this problem is solved with this deletion "just in case" at the end of every e2e process 😄

I have to recognize that I love the improvement in the loging, I mean, something as simple as like Current  - 0, Target - 1 will help a lot IMO. Super nice improvement!

I'm thinking on what will happen in case of panic, and I'm not sure if we will face with stuck namespaces 🤔
I mean, we will try to delete all e2e nasmapces. but if there is any ScaledObject or ScaledJob at that moment, the namespace will be stuck because the finalizers will block the deletion.
Does Kustomize follow any criteria like 'delete CRDs before the others'?

I'm thinking on what will happen in case of panic, and I'm not sure if we will face with stuck namespaces 🤔 I mean, we will try to delete all e2e nasmapces. but if there is any ScaledObject or ScaledJob at that moment, the namespace will be stuck because the finalizers will block the deletion. Does Kustomize follow any criteria like 'delete CRDs before the others'?

I am not sure about this, but at the very least one work around is to use /run-e2e <regex-that-doesn't-match-anything> this will deploy and undeploy keda and that should resolve hanging namespaces which will then get deleted by make e2e-test clean
Now, obviously, this is not a solution. We need to be a bit more cautious when reviewing the e2e tests in Go to make sure they don't panic. But this might not be a concern, since most libraries return an error instead of panicking. The only other common way to panic will be code accessing nil pointers.

The tests are passing! @JorTurFer @zroubalik

/run-e2e Update: You can check the progres here

The tests are passing!

I'm thinking on what will happen in case of panic, and I'm not sure if we will face with stuck namespaces 🤔 I mean, we will try to delete all e2e nasmapces. but if there is any ScaledObject or ScaledJob at that moment, the namespace will be stuck because the finalizers will block the deletion. Does Kustomize follow any criteria like 'delete CRDs before the others'?

I am not sure if kustomize does this. But maybe you can add delete all scaled objects / jobs in namespaces with label type=e2e in the make undeploy command before the kustomize operation.

I was talking about doing something like this kubectl delete $(kubectl get scaledobjects,scaledjobs -oname) but including all the namespaces (I haven't tested this code below,  I wrote in on-the-fly and maybe we need to adapt it):
while read -r namespace
do
    kubectl delete $(kubectl get scaledobjects,scaledjobs -oname -n "$namespace") -n "$namespace"
done < (kubectl get namespaces)

Doing this as part of the make undeploy before kustomize command should be enough.
WDYT?

I was talking about doing something like this kubectl delete $(kubectl get scaledobjects,scaledjobs -oname) but including all the namespaces (I haven't tested the code below, I wrote in on-the-fly and maybe we need to adapt it):
while read -r namespace
do
    kubectl delete $(kubectl get scaledobjects,scaledjobs -oname -n "$namespace") -n "$namespace"
done < (kubectl get namespaces)

Doing this as part of the make undeploy before kustomize command should be enough to ensure that we delete all the KEDA resources with finalizers. WDYT?

Yup, I was thinking something exactly like this.

Don't forget to include TriggerAuth... :)

Don't forget to include TriggerAuth... :)

TriggerAuth doesn't have finalizers, so I think that we can just delete them deleting the CRD

Don't forget to include TriggerAuth... :) And then finally namespaces.

The namespaces get deleted with make e2e-test clean already because of the e2e label.

Using kubectl get namespaces gives back a tabular response, and -oname gives back namespaces/<ns-name> which cannot be used with kubectl get <resource> -n <ns> as / is not allowed.
while read -r namespace
do
    kubectl delete $(kubectl get scaledobjects -n $namespace -o name) -n $namespace
    kubectl delete $(kubectl get scaledjobs -n $namespace -o name) -n $namespace
done < <(kubectl get namespaces -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}{end}")

I tried this, and this works.

Using kubectl get namespaces gives back a tabular response, and -oname gives back namespaces/<ns-name> which cannot be used with kubectl get <resource> -n <ns> as / is not allowed.
while read -r namespace
do
    kubectl delete $(kubectl get scaledobjects -n $namespace -o name) -n $namespace
    kubectl delete $(kubectl get scaledjobs -n $namespace -o name) -n $namespace
done < <(kubectl get namespaces -o jsonpath="{range .items[*]}{.metadata.name}{'\n'}{end}")

I tried this, and this works.

I am not exactly sure where is the best way to add this. In the makefile? Also this will exit with an error code of 1 because the command may try to delete scaled objects from resources that don't have any. This might cause the make recipe to fail and exit with failure and fail the workflow.
Could you take my comment as a reference and then add it as a separate PR after refactoring?

/run-e2e

/run-e2e Update: You can check the progress here

@JorTurFer @zroubalik I have added a make target e2e-test-clean-crds that deletes any scaled objects, jobs that are on the cluster using a shell script. I have checked that the shell script works with scaled objects and jobs across multiple namespaces.
I have added this new target as a dependency for the undeploy target, and you can see that it's running here.
The tests also seem to be running fine.
Any more changes expected in this PR?


Bug: Github Auth - Unable to exchange external code

Bug report
Describe the bug
Using github as an external provider results in an error during the redirect phase after the sign in attempt.
error: server_error
error_description: Unable to exchange external code: 7cf84a55c55bd5f9ac26

Additional context
Seems to work locally, checking prod to see why its failing
https://github.com/supabase/supabase/discussions/1192

I experienced ghe same with facebook as a provider.

Hi @ElesGergo, at the moment, I can't seem to reproduce the bug, could you please highlight the steps you took to enable facebook as an auth provider? If it helps, we have a tutorial that provides a step-by-step guide on enabling 3rd-party providers on Supabase. Let me know if this helps 👍

a lot of people are reporting getting spaces when they copy-paste the github client secret, might be worth checking for anyone experiencing the same

also there are backend auth logs inside the Supabase dashboard at Auth > Settings (scroll to the bottom), there should be some additional info there


The POT on texture import is resizing to nearest POT rather than next POT

The documentation claims the following:

When importing a texture with the power of two option ticked rather than resizing to the next highest POT they are actually imported to the nearest POT.
e.g.
Imported with POT turned off

Imported with POT turned on

Steps to Reproduce

Ensure Textures POT is turned on under Asset Tasks
Import a texture that has a dimension where the nearest POT is lower rather than higher

Fixed in https://github.com/playcanvas/developer.playcanvas.com/commit/15a6f866e8216bee7b2b01ccc28300ae5a2c6360


Android: Sending a story to Twitter shares the whole story

Just send the title and link. From a tweet: https://twitter.com/afrakt/status/880390479589801986

Nevermind... https://twitter.com/afrakt/status/880391724815847424

My fault for opening this issue, I should have confirmed it first. I feel like this would've been mentioned many times by now if it was actually an issue.


Add CI/CD

A push will trigger all the checks: lint, typescript & cypress.
If you're pushing to the main branch, it will trigger the deployment.

Magiiiiic!!!


Document types of output lines of generate/interactive.py

📚 Documentation
After:
This generation script produces three types of outputs: a line prefixed
with *O* is a copy of the original source sentence; *H* is the
hypothesis along with an average log-likelihood; and *P* is the
positional score per token position, including the
end-of-sentence marker which is omitted from the text.

added:
Other types of output lines you might see are *D*, the detokenized hypothesis,
*T*, the reference target, *A*, alignment info, *E* the history of generation steps.

Merged in adbd89f


created contribution_guidelines file

For the code of conduct link in the contributing guidelines, should I just re-edit it once we’ve merged milestone 1 with our master repo to add the correct link?

That's one option, the other is to put the correct link in now... It's also not the end of the world if we do a first merge to main once we've created all the base repos in the file before working on more specific features.

That's one option, the other is to put the correct link in now... It's also not the end of the world if we do a first merge to main once we've created all the base repos in the file before working on more specific features.

that works for me, I can make a new pull request after we've done our first merge

that works for me, I can make a new pull request after we've done our first merge


Issue while deploying ASP.Net Core Project on AWS Elasticbeanstalk

Tracking issue on our side for https://github.com/aws/aws-toolkit-visual-studio/issues/309
Describe the bug
Step 1:- I created a simple ASP.Net Core Web App using Visual Studio 2022
Step 2:- Installed AWS toolkit
Step3:- Selected Publish to AWS for deploying my application on my free AWS tier using Elasticbeanstalk
Step 4:- I get the following error on the 3rd Step i.e. on Deploying AWS CDK Project :
AWSDeploymentProject: deploying...
[0%] start: Publishing 73645f7c905524d6ca666eae7e1e2a7f1bca1d4062451cd2eb619d2d2f7ec4ec:712088662267-eu-central-1
[0%] start: Publishing b634beaf9192c9e30537653615f0e0f9c3d21b3ab410d6ca555b2ea7fd8ce062:712088662267-eu-central-1
[50%] fail: Socket timed out without establishing a connection
[100%] fail: Socket timed out without establishing a connection
❌ AWSDeploymentProject failed: Error: Failed to publish one or more assets. See the error messages above for more information.
at publishAssets (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\util\asset-publishing.ts:60:11)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at CloudFormationDeployments.publishStackAssets (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\api\cloudformation-deployments.ts:572:7)
at CloudFormationDeployments.deployStack (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\api\cloudformation-deployments.ts:419:7)
at deployStack2 (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\cdk-toolkit.ts:264:24)
at C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\deploy.ts:39:11
at run (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\p-queue\dist\index.js:163:29)
❌ Deployment failed: Error: Stack Deployments Failed: Error: Failed to publish one or more assets. See the error messages above for more information.
at deployStacks (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\deploy.ts:61:11)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at CdkToolkit.deploy (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\cdk-toolkit.ts:338:7)
at initCommandLine (C:\Users\ssaty\AppData\Local\npm-cache_npx\e72b144743208263\node_modules\aws-cdk\lib\cli.ts:364:12)
Stack Deployments Failed: Error: Failed to publish one or more assets. See the error messages above for more information.
AWSDeploymentProject could not be published as ASP.NET Core App to AWS Elastic Beanstalk on Windows: We had an issue deploying your application to AWS. Check the deployment output for more details. Deployment took 124.84s.
AWSDeploymentProject failed to publish to AWS.
Note : The IAM user has administrator privileges.
Any help would be appreciated.
Version used
UNKNOWN
Operating System and version
UNKNOWN

Requested more information from user at https://github.com/aws/aws-toolkit-visual-studio/issues/309

Related issue https://github.com/aws/aws-toolkit-visual-studio/issues/309 closed as stale.


Remove Never return from ParsableCommand.main()

Description
This brings the main() method in line with using the new @main attribute on the root
command type.
Detailed Design
The changes here are to make ParsableCommand.main() a Void-returning function, to break out a separate no-argument version, and to remove the default parameter. The default-argument version doesn't satisfy the () -> Void requirement for @main.
Documentation Plan
No real impact. Documentation for using @main with the library to follow.
Test Plan
All existing tests continue to pass. SwiftPM will need an update to support @main in executable targets.
Source Impact
Typical code should continue to behave as expected, but this is source-breaking if anyone has a strongly-typed usage of ParsableCommand.main(), e.g. let mainFunction: ([String]?) -> Never = MyCommand.main.
Keeping the old version with a deprecation/unavailability annotation isn't possible, since there would be ambiguity between that and the new ones.
Checklist

[x] I've added at least one test that validates that my change is working, if appropriate
[x] I've followed the code style of the rest of the project
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary

@swift-ci Please test


[BUG] cast(9.95 as decimal(3,1)), actual: 9.9, expected: 10.0

Repro
 ~/dist/spark-3.3.0-bin-hadoop3/bin/spark-shell  \
   --conf spark.plugins=com.nvidia.spark.SQLPlugin \
   --conf spark.rapids.sql.test.enabled=true \
   --conf spark.rapids.sql.explain=ALL
   --jars dist/target/rapids-4-spark_2.12-24.06.0-SNAPSHOT-cuda11.jar

scala> val df = Seq(9.95).toDF.coalesce(1)
df: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [value: double]

scala> spark.conf.set("spark.rapids.sql.enabled", true)

scala> df.selectExpr("cast(value as decimal(3,1))").collect()
24/05/13 11:34:42 WARN GpuOverrides:
*Exec <ProjectExec> will run on GPU
  *Expression <Alias> cast(value#1 as decimal(3,1)) AS value#5 will run on GPU
    *Expression <Cast> cast(value#1 as decimal(3,1)) will run on GPU
  *Exec <CoalesceExec> will run on GPU
    ! <LocalTableScanExec> cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.LocalTableScanExec
      @Expression <AttributeReference> value#1 could run on GPU

res2: Array[org.apache.spark.sql.Row] = Array([9.9])

scala> spark.conf.set("spark.rapids.sql.enabled", false)

scala> df.selectExpr("cast(value as decimal(3,1))").collect()
res4: Array[org.apache.spark.sql.Row] = Array([10.0])

Related to #9682

Scope: make casting of floats to decimals feature flag off by default.  Ref: spark.rapids.sql.castFloatToDecimal.enabled.

Filed a cudf issue: https://github.com/rapidsai/cudf/issues/15862

Also take a look at the discussion around https://github.com/NVIDIA/spark-rapids/issues/9682#issuecomment-2126616418 and above. 9.95 is not representable as double
$ jshell
|  Welcome to JShell -- Version 21.0.2
|  For an introduction type: /help intro

jshell> new BigDecimal(9.95)
$8 ==> 9.949999999999999289457264239899814128875732421875

jshell> new BigDecimal(9.949999999999999289457264239899814128875732421875)
$9 ==> 9.949999999999999289457264239899814128875732421875

jshell> new BigDecimal(9.95).setScale(1, BigDecimal.ROUND_HALF_UP)
$11 ==> 9.9

jshell> new BigDecimal("9.95").setScale(1, BigDecimal.ROUND_HALF_UP)
$12 ==> 10.0

Okay after reading through the issue https://github.com/NVIDIA/spark-rapids/issues/9682 then I realize that this issue is also one instance of it.

I tried the float => string => decimal path (it is very easy to implement in plugin), the differences are from the known limits of ryu float to string and a very edge cases in string to decimal https://github.com/NVIDIA/spark-rapids/issues/10890. I will post a pr next week, but not sure if the diffs are acceptable or original way can match it better.

That sounds good. I've also found a way to implement in C++ which is very efficient but not sure if it will pass integration test. I'll verify that and will post a PR too.
If having a chance, please list the related tests that I can run to verify.

@thirtiseven Please test https://github.com/NVIDIA/spark-rapids/pull/10917 to see if you have any test fails. On my local env, all the unit tests and integration tests passed but I'm not sure if I missed anything.

Hi @ttnghia, https://github.com/rapidsai/cudf/pull/15905 has been merged, is it ready to fix this issue by using this new way?

No that is not what we want. We may need something from https://github.com/pmattione-nvidia/cudf/pull/2 which will be added into https://github.com/NVIDIA/spark-rapids-jni/pull/2078, which is what we actually need. I will work on it soon this week.

Thanks @GaryShen2008. The fix in JNI will be picked by spark-rapids in https://github.com/NVIDIA/spark-rapids/pull/10917 thus this issue will be closed by that PR.


Truncated error output

Bug: Notebook Editor, Interactive Window, Python Editor cells

Steps to cause the bug to occur

import numpy as np
np.random.rand(4).astype('f4,f4').astype(np.complex)

Actual & Expected behavior
Actual output is truncated:

Expected: full error message as displayed in standard python prompt
>>> import numpy as np
>>> np.random.rand(4).astype('f4,f4').astype(np.complex)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Cannot cast array from dtype([('f0', '<f4'), ('f1', '<f4')]) to dtype('complex128') according to the rule 'unsafe'

Your Jupyter and/or Python environment
Please provide as much info as you readily know

Jupyter server running:                       Local
Extension version:                               2019.10.44104
VS Code version:                                1.40.0
Setting python.jediEnabled:               true
Python and/or Anaconda version:     3.7
OS:                                                     Mac
Virtual environment:                          conda

Developer Tools Console Output

[Extension Host] Info Python Extension: 2019-11-09 23:05:22: Submitting code for 41fe195c-e86a-4704-92e1-32136d1b7142
console.ts:137 [Extension Host] Info Python Extension: 2019-11-09 23:05:22: Waiting for jupyter server and web panel ...
console.ts:137 [Extension Host] Info Python Extension: 2019-11-09 23:05:22: Wait for sys info for 41fe195c-e86a-4704-92e1-32136d1b7142 0
console.ts:137 [Extension Host] Info Python Extension: 2019-11-09 23:05:22: Finished execution for d96a3ee0-0757-44b4-a902-e29a6998439b

@DonJayamanne @rchiodo @IanMatthewHuff Can you try this?
I can't reproduce it with the same vs code and extension versions.

It did repro for me with our about to be released bits.

@ihnorton, thanks for the feedback! Closing as this is a dupe of #7187


[Feature Request] Refactor Linear algebra service

Is your feature request related to a problem? Please describe.
The LinearAlgebra module is a big mess. Many not implemented functions, naming has to be unified, unmanaged access to LAPACK functions are sparse.
Describe the solution you'd like
Subproject FSharp.Stats.LAPACK (already existing as placeholder project) should contain the Lapack stubs and function access. LinearAlgebra.fs naming has to follow a convention. Drop MKL support and embrace netlib lapack as it is open source. Unify output types of managed and unmanaged versions of linear algebra functions
Describe alternatives you've considered
Support MKL alongside Netlib LAPACK

related: #16


Added command line parameter "--jvm" to pass Java options to the test…

… runs
It can be used to enable remote debug, for example:
./bin/run_tests -s Functional -g functional -d --jvm "-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=50000,server=y,suspend=y"

Since we only have a few main possibilities - debug being one of them, why not provide a flag called --debug or similar which starts test framework in debug mode? We can keep the default values for port address and such. Anyone can tweak them if needed. This way noone needs to actually remember the long debug string.

I wanted to implement is the same way it's implemented in Drill. Please see Paul's comments here: DRILL-6270, https://github.com/apache/drill/pull/1210
Also, you don't need to remember debug string, you can use aliases or functions, for example:
run_with_debug () {
    local cmd=$1
    eval '$cmd --jvm "-Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket,address=50000,server=y,suspend=y"'
}

That way you avoid passing additional parameters like port or suspend.

But you do have to remember and pass the whole string here.. am I missing something? My suggestion was to have meaningful defaults and start in debug mode by a simple flag like --debug. Anyone and open the file and override if they want to change port or something. Where is the debug function you shared above?

My point is that you can store such function in your .bashrc file, and then run tests like this:
run_with_debug bin/run_tests -s Functional -g functional -d

Yes, anyone can open the file and actually add the whole debug string, but that is not very convenient.
Also, general --jvm parameter can be used not only for debugging but any other java options (heap size, timezone etc)
As a compromise I suggest to add both --jvm and --debug with the default string. What do you think?

I see the point for it to be generic, although I don't completely agree with it. So, yes.. please add both options if possible. I'd appreciate it. I think for most cases the --debug flag will be easy to use and sufficient. For advanced / custom scenarios one can use your --jvm parameter.

Done.

I tested it manually and tests were passed.


[WIP] Bump cargo-lock dependency to v10.0.0-pre

Needed for https://github.com/rustsec/rustsec/pull/1224

I should probably just drop that whole codepath and the cargo-lock dependency because it isn't used by anyone, doesn't produce entirely valid lockfiles, and is of little use now that we have conversion to CycloneDX anyway.

Closing in favor of #159, which uses a cargo-lock v10.0.0-pre.0 crate release to crates.io


how to convert command line generated project.json into vs project

Hi, how to convert command line generated project.json into vs project and open it in visual studio?
I would like to develop in windows/vs, and then dotnet run it on ubuntu or docker.thanks

@anguoyang Just open the project.json file directly in VS. VS will generate an xproj file for it.

@livarcocc , wow, I was also looking something like this.

@livarcocc when I opened the json file in VS, it acted as common editor as notepad, no relevant project files created.

@anguoyang , try with "File" > "Open" > "Project/Solution..."

@seanshpark  Thanks a lot, it worked!!!

Kang sa mi da! @seanshpark , it could be opened and compiled successfully in VS, however, it could not be run due to "there is no command in project.json, so it could not be startup"?

@anguoyang , "chun man e yo~", for me, I've copied the output DLL, say "output.dll" file to my Linux(Ubuntu 14.04) and ran with
./corerun output.dll
where required DLLs exist in the same directory, just for testing.

If you can't find the DLL, right click on the "Project" and select "Properties". In the left "Build" tab, check on the "Product outputs on build".  You can see the DLL in "../artifacts/bin/..../Debug/dnxcore50/" folder.

굿 모닝 @seanshpark , I would like to build and run/debug it within visual studio, the IDE is powerful for debugging, after that, I will copy it into ubuntu to run it. My problem is, I could not run/debug the project in visual studio.

@anguoyang , hello. I had the same problem. I'm not sure but please try adding this in project.json.
  commands: {
     "start": "(your project name appearing in VS IDE)"
  },

start string will appear in the debug go button on the toolbar. You can give anything if you want :)
One more thing. When I open the project.json file in the IDE, there appears an URL on top of the window: http://json.schemastore.org/project . If you open this link, it shows the schema of this project.json with some useful information.

This issue is dealing with DNX tooling which has been deprecated as of RC2 of .NET Core. With that I will close this issue. You can use the DNX migration guide to migrate DNX-based projects to RTM projects. RTM was released on 27th of June.
With this, I will close this issue. If you run into any problems with using .NET Core RTM, please file an issue on the dotnet/cli repo. Thanks!


Nullpointer when using webhooks

Please answer the following questions.
Which version of matterbridge are you using?
0.9.0
If you're having problems with mattermost please specify mattermost version.
3.5.1
Please describe the expected behavior.
Slack webhook messages are recieved and relayed to the mattermost bridge and the other way around.
Please describe the actual behavior.
Use logs from running matterbridge -debug if possible.
A nullpointer is thrown here
https://github.com/42wim/matterbridge/blob/master/bridge/slack/slack.go#L162
because there is no username defined with webhooks
DEBU[2017-01-11T14:00:25+01:00] receiving from matterhook (slack) matterhook.IMessage{BotID:"", BotName:"", Token:"<token>", TeamID:"<ID>", TeamDomain:"<domain>", ChannelID:"<Channel>", ChannelName:"<name>", Timestamp:"1484139625.000025", UserID:"U0BDH8N0J", UserName:"sean", PostId:"", RawText:"", ServiceId:"<ID>", Text:"test", TriggerWord:""}  module=slack
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x522d95]

goroutine 9 [running]:
panic(0x903240, 0xc42000e120)
        /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/42wim/matterbridge/bridge/slack.(*Bslack).handleSlack(0xc42007d400)
        /go/src/github.com/42wim/matterbridge/bridge/slack/slack.go:162 +0x255
created by github.com/42wim/matterbridge/bridge/slack.(*Bslack).Connect
        /go/src/github.com/42wim/matterbridge/bridge/slack/slack.go:64 +0x278

Any steps to reproduce the behavior?
setup a slack-mattermost link using just webhooks
Please add your configuration file
(be sure to exclude or anonymize private data (tokens/passwords))
[slack]
[slack.company]
    URL="<URL>"
    BindAddress="<IP_ADDRESS>:9000"
    PrefixMessagesWithNick=true
[mattermost]
[mattermost.company]
    BindAddress                       = "<IP_ADDRESS>:9998"
    URL                                     = "<URL>"
    PrefixMessagesWithNick=true
[[gateway]]
    enable    = true
[[gateway.inout]]
    account  = "mattermost.company"
    channel  = "off-topic"

[[gateway.inout]]
    account  = "slack.company"
    channel  = "#mmtesting"

Thanks for reporting!
Fixed in master, could you test ?

It's now one way traffic, mattermost can send to slack but slack can't send to mattermost.
From what I could see from your code, it might need a webhook user defined instead of only checking for the user when api is used.

I just set up a test-setup, works fine for me.
Could you run matterbridge with -debug sent a message from slack to mattermost and one from mattermost to slack and paste output ? (filtered for private data)

INFO[2017-01-18T12:16:22+01:00] enabling debug
running version 0.9.1-dev
starting gateway "gateway2"
INFO[2017-01-18T12:16:22+01:00] Starting bridge: slack.company
INFO[2017-01-18T12:16:22+01:00] Connecting                                    module=slack
INFO[2017-01-18T12:16:22+01:00] Connection succeeded                          module=slack
INFO[2017-01-18T12:16:22+01:00] slack.company: joining #mmtesting
INFO[2017-01-18T12:16:22+01:00] Starting bridge: mattermost.company
INFO[2017-01-18T12:16:22+01:00] Connecting webhooks                           module=mattermost
INFO[2017-01-18T12:16:22+01:00] mattermost.company: joining off-topic
2017/01/18 12:16:22 Listening on http://<IP_ADDRESS>:9000...
DEBU[2017-01-18T12:16:22+01:00] Choosing API based slack connection: false    module=slack
2017/01/18 12:16:22 Listening on http://<IP_ADDRESS>:9998...
DEBU[2017-01-18T12:16:22+01:00] Choosing API based Mattermost connection: false  module=mattermost
DEBU[2017-01-18T12:16:23+01:00] Start listening for Slack messages            module=slack
DEBU[2017-01-18T12:18:47+01:00] receiving from matterhook (slack) matterhook.IMessage{BotID:"", BotName:"", Token:"TOKEN", TeamID:"ID", TeamDomain:"m2mobi", ChannelID:"ID", ChannelName:"mmtesting", Timestamp:"1484738326.000040", UserID:"ID", UserName:"sean", PostId:"", RawText:"", ServiceId:"ID", Text:"**test**", TriggerWord:""}  module=slack
DEBU[2017-01-18T12:18:47+01:00] Sending message from sean on slack.company to gateway  module=slack

EBU[2017-01-18T12:19:46+01:00] Receiving from matterhook matterhook.IMessage{BotID:"", BotName:"", Token:"TOKEN", TeamID:"ID", TeamDomain:"m2mobi", ChannelID:"ID", ChannelName:"off-topic", Timestamp:"1484738386", UserID:"ID", UserName:"smillernl", PostId:"ID", RawText:"", ServiceId:"", Text:"**and back**", TriggerWord:"and"}  module=mattermost
DEBU[2017-01-18T12:19:46+01:00] Sending message from smillernl on mattermost.company to gateway  module=mattermost
DEBU[2017-01-18T12:19:46+01:00] Sending config.Message{Text:"**and back**", Channel:"#mmtesting", Username:"smillernl", Avatar:"", Account:"mattermost.company", Event:""} from mattermost.company (off-topic) to slack.company (#mmtesting)
DEBU[2017-01-18T12:19:46+01:00] Receiving config.Message{Text:"**and back**", Channel:"#mmtesting", Username:"", Avatar:"", Account:"mattermost.company", Event:""}  module=slack
DEBU[2017-01-18T12:19:46+01:00] receiving from matterhook (slack) matterhook.IMessage{BotID:"ID", BotName:"mattermost-webhook", Token:"TOKEN", TeamID:"ID", TeamDomain:"m2mobi", ChannelID:"ID", ChannelName:"mmtesting", Timestamp:"1484738386.000041", UserID:"USLACKBOT", UserName:"slackbot", PostId:"", RawText:"", ServiceId:"ID", Text:"**and back**", TriggerWord:""}  module=slack
DEBU[2017-01-18T12:19:46+01:00] Sending message from slackbot on slack.company to gateway  module=slack

The issue is probably the slack channel name, you must remove the "#" in your config.
[[gateway.inout]]
    account  = "slack.company"
    channel  = "mmtesting"

I'm an idiot, works now. Thanks


Deprecate this repo

CC @thomashoneyman
This repo's contents were ported to purescript-numbers and purescript-integers. All core, contrib, web, and node libraries imports and dependencies have been updated to remove this package. Moreover, the upcoming 0.15.0 package set builds without this package. https://github.com/purescript/package-sets/pull/1075
We can now deprecate this repo.

This should be done once PureScript 0.15 launches. We can update the README, move to purescript-deprecated, delete all the files, and push to Pursuit.

This is something we can do now.


Update major version from 13 to 14.

Update major version from 13 to 14.
Remove Common issues section.

Hi @jingtong, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

@jingtong, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, MSBOT;

Please make these changes to the original file, and create a forked version get_started_with_rvc_13.md


Error encountered in openshift

Traceback (most recent call last):
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 119, in impl
return await handler(request)
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aiohttp_apispec/middlewares.py", line 45, in validation_middleware
return await handler(request)
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 186, in check_token
return await handler(request)
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aries_cloudagent/admin/server.py", line 199, in apply_limiter
return await task
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 559, in credential_exchange_send
outbound_handler,
File "/home/indy/.pyenv/versions/3.6.9/lib/python3.6/site-packages/aries_cloudagent/protocols/issue_credential/v1_0/routes.py", line 340, in internal_error
err.roll_up, record.connection_id, record.thread_id, outbound_handler
AttributeError: 'ConnectionRecord' object has no attribute 'thread_id'

Hey, you're right. My mistake, I think. If you like I can look into it tomorrow.
There really is no thread_id in ConnectionRecord. There is in other kinds of records - maybe only protocol records?

A little finessing with internal_error() and send_problem_report() in issue-credential v1.0 routes.py ought to fix it, 15 minutes tomorrow first thing in the morning if you can wait that long.

ok @sklump I'll leave it for you

I notice the logic is slightly different in proof request handling, probably makes sense to look at that one at the same time and update to be consistent with the credential error handling.

https://github.com/hyperledger/aries-cloudagent-python/pull/595


PM> Update-Database (error)

Applying migration '20161206204752_Initial' error.
An object named 'AspNetUsers' already exists in the database.

It aways accours to me. Idk why but I can never update the database, I aways need to delete the DB to use the Update-database when I test this solution to learn from Eduardo. I really fear to use dbfirst in big projetcts.

I got it resolved!
The tables were being created in two places.
Equinox.Infra.CrossCutting.Identity\Data\Migrations\00000000000000_CreateIdentitySchema.cs
AND
Equinox.Infra.CrossCutting.Identity\Migrations\20161206204752_Initial.cs


Menus: Update multi-column menu component to render menu based on JSON document as specified

As a Platform.Bible extension developer, I want to be able to add menus to my webviews and contribute to extensible points in platform-provided menus.

This particular task is about having menu components that take a "final" menu JSON document for a multi-column menu and display the menus appropriately.  The menu JSON document definition is in the menu contribution design document.
I'm told the existing mutli-column menu component already does something like this.  It is important for the component to align with the schema from the design document, so some updates are likely needed.
Note that the items in a menu should be considered fixed while the menu is open/drawn on the screen.  Once the menu is closed/hidden, its content should be reloaded before rendering again because extensions might have been enabled/disabled or something else might have changed that would modify the list of available menu items.

You could probably quickly use a JSON-Schema-to-TS-types library to generate TS types for this from the JSON schema in the design quickly just this once :)

Small follow-on PR: https://github.com/paranext/paranext-core/pull/786

Another little PR  to fix an issue in the (for now) hard-coded menu data: https://github.com/paranext/paranext-core/pull/787


All Operation Object's are missing operationId fields

The operationId field is used by generators to make sensible API interface methods.
OpenAPI 3.0.0 description of the operationId field:

Unique string used to identify the operation. The id MUST be unique among all operations described in the API. Tools and libraries MAY use the operationId to uniquely identify an operation, therefore, it is RECOMMENDED to follow common programming naming conventions.

Here are some example function signatures using "/state/{state}/renewables/current" and Apples official OpenAPI generator:

Without operationId:

func get_sol_state_sol__lcub_state_rcub__sol_renewables_sol_current(_ input: Operations.get_sol_state_sol__lcub_state_rcub__sol_renewables_sol_current.Input) async throws -> Operations.get_sol_state_sol__lcub_state_rcub__sol_renewables_sol_current.Output

With "operationId": "getCurrentRenewablesForState":

func getCurrentRenewablesForState(_ input: Operations.getCurrentRenewablesForState.Input) async throws -> Operations.getCurrentRenewablesForState.Output

Adding operationId values seems like a great feature to add for API users.
My suggestions:

getCurrentRenewablesForState
getAllSites
getPricesForSite
getCurrentPricesForSite
getUsageForSite

Opinion?
Carl.

Got an internal PR for this already. Waiting for approval. Should be up tomorrow.

Done


Integrate Applitools

Todo
As a user, I want to add Applitools configuration to the end-to-end test cases so that I can implement visual validation.
Acceptance Criteria

Able to see Vetlog screenshots in Applitools dashboard
Add Applitools configuration and execution information

We will integrate Applitools in another repository


Fix #51 concurrent access to vars not protected - add locking

it's doesn't seems to work, it lock the execution. On travis the build is aborted because it's stale for over 10 min ...

Yes, there's a double lock, my bad. Will look at the issue later - bone now crashes on concurrency protection.


Formatting docs update - April 2024

Category

[X] Content fix

What's in this Pull Request?
Update custom formatter doc to add details about the new embed action.
You can embed maps -

Even youtube videos -

And much more...! 😄

@VesaJuvonen Could you please take a look?

@andrewconnell Thanks for reviewing and merging. What's the process/timeline for these changes to reflect in  ?

Refreshes to the live site usually happen on a weekly basis... that's handled by @VesaJuvonen (I can't publish directly to live being an external)... if you need it to get pushed sooner, I'd suggest contacting him via Teams.

Got it. Thanks  Andrew!


unable to register an account

Since at least August 19th, it has been impossible to successfully register an account.
# ./oreilly-trial --verbose
6:11AM INF oreilly-trial is started! appVersion=2.2.14 buildDate=2023-08-04T18:13:49Z gitCommit=7a1a105 goArch=amd64 goOS=linux goVersion=go1.20.6
6:11AM DBG trying to set request headers
6:11AM DBG sending request with http client url=https://learning.oreilly.com/api/v1/registration/individual/

Same question.

I got same issue.

i can verify that it is the same on my own. on it. thanks for reporting!

still investigating but just a heads up; ive noticed that oreilly changed its API so its timing out actually. but with the new API address, both browser and our app throws 403.

where can I find oreilly api? I only found the search API.
O'Reilly Platform Search API

@by-sknight you can discover using network tool on your browser but there is no API documentation.

@bilalcaliskan Thanks, I forgot about this method. When I see the api in the link, I'm just looking for the api documentation.

any news about this issue?

still no luck. last chance is acting like a browser with a selenium library.

Here it times out:
6:01PM INF oreilly-trial is started! appVersion=2.2.14 buildDate=2023-08-04T18:13:49Z gitCommit=7a1a105 goArch=am                             d64 goOS=linux goVersion=go1.20.6
6:06PM WRN an error occurred while generating email with specific domain error="unable to do http request to remote host https://learning.oreilly.com/api/v1/registration/individual/: Post "https://learning.oreilly.com/api/v1/registration/individual/\": read t            cp <IP_ADDRESS>:42978-><IP_ADDRESS>:443: read: connection timed out" attempt=1 domain=RG9tYWluOjIw<EMAIL_ADDRESS>

still no luck. last chance is acting like a browser with a selenium library.

that is the way to go

couple of news; oreilly is now somehow protecting the API with a cookie and without that cookie, we are encountering the error you all see. sadly i could not find a way to obtain this cookie yet.
contributions are welcomed by the way.

@bilalcaliskan any news from this?


Extract keyvis

extract keyvis as a package
fix circular dependencies warning

Test it in https://github.com/baurine/test-tidb-dashboard-packages , works fine.
Usage: https://github.com/baurine/test-tidb-dashboard-packages/blob/master/src/App.js

All done, @breeswish PTAL.

replace by #254 , closed


Improve the Security Overview

Umbrella issue: #25119
Blocked by: #43176
Collaborators: @tengqm @shannonxtreme @sftim @kubernetes/sig-security
Description
Expand the Kubernetes Security Overview concept page (https://kubernetes.io/docs/concepts/security/) to provide a use-case based, prescriptive guide that various personas can use to do specific jobs at specific phases of their journey.
#43176 refactors the existing Overview of Cloud-native Security page to closely follow the lifecycle phases from the CNCF security whitepaper. We should expand this page to provide additional guidance within each phase for where a reader can go to learn how to achieve specific goals.
What does that look like?
The final structure is up for discussion, but we should consider including the following information:

The shared responsibility model, broken down by persona. In other words, as an application developer, what are the things I need to pay attention to? As a cluster operator, what are the best practices I need to check. As someone who prepares the underlying infrastructure, is there a checklist for me? (thanks @tengqm )
A sensible breakdown of sections in the page. Lifecycle phases is good. What about security "layers" (authn/z, encryption, monitoring/logging, workload security, infrastructure security, etc)?
Whatever the section breakdown, provide use-case/goal oriented prescriptive guidance for the reader to achieve the goal. For example

Control which workloads can deploy in the cluster

Use admission controllers.
Basic: Enforce pre-defined policies based on best practices (link to PSA)
Intermediate: Enforce custom policies using something like Gatekeeper OPA
Advanced: Write your own admission webhook

Identify the key pain points for new users and reinforce that they're complicated and attempt to break them down. For example, networking is probably horrible for folks and TLS/certificate management. Secure multi-tenancy is also probably a scary area. Can we get community feedback?

What's next?

Use a Google Doc for discussion (started one here: https://docs.google.com/document/d/1JYe35tUwTYivBdQ5j2g5bp_qP4Pm5OWlaRDAJNTriLA/edit?usp=sharing)

Discuss goals and presentation of information
Identify pain points and how to address them
Identify "core" information to include no matter what

Create a doc plan to plan the work and scope and outline the actual structure
Create a draft doc and solicit reviews from sig-security
PR
???
Profit!

/sig docs
/sig security
/label priority/important-longterm
/cc @tengqm @sftim

/priority important-longterm

/language en

/triage accepted

/lifecycle frozen
Let's track this 'til it's done, or we explicitly decide to let it rot.

How's this work going?

I think we've improved this somewhat. @shannonxtreme is this OK to close?

/remove-lifecycle frozen

The page has improved, sadly I got too busy to work on the original google doc though. I think we can close this, and if I do find the time to work on it in the future I can always propose reopening!

/close


Verifier presentation failure adjustments

This PR fixes an emerging problem regarding the verifier not doing anything in case the presentation verification fails (a problem report might be generated, but not sent, so the prover would be in the dark).

Codecov Report

Merging #830 (1701c27) into main (07c35cb) will decrease coverage by 40.27%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #830       +/-   ##
==========================================
- Coverage   49.00%   8.73%   -40.27%
==========================================
  Files         410     405        -5
  Lines       33495   32668      -827
  Branches     7436    7200      -236
==========================================
- Hits        16413    2854    -13559
- Misses      11968   29008    +17040
+ Partials     5114     806     -4308

Flag
Coverage Δ

unittests-aries-vcx
8.73% <0.00%> (-40.24%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ocols/proof_presentation/verifier/state_machine.rs
0.00% <0.00%> (-47.27%)
:arrow_down:

... and 301 files with indirect coverage changes


Add InstanceType tip in Classes.md

I personally have a hard time finding this type helper (InstanceType) and would love for it to be in the official docs on Classes.

Not sure I understand why not to include it in docs though --is InstanceType an unstable feature? Assuming it is stable, shouldn't we document it somewhere? And would be great to have it on the page with Classes imho

It's certainly stable, yes. It's defined as:
/**
 * Obtain the return type of a constructor function type
 */
type InstanceType<T extends abstract new (...args: any) => any>
	= T extends abstract new (...args: any) => infer R ? R : any;

I'm not against documenting it, but I'm just saying that if we do, it shouldn't come right at the top; we haven't even introduced what classes are and how they work, let alone that they're objects with a new signature. The first (and only) mention of the new signature is down at the bottom.

good points. Should we maybe add it at the bottom of Constructor section (link), since InstanceOf returns the type of a constructor function?

I think better would be a section about constructor signatures, placed at least before the mention of new () in the abstract constructor signature section. Whether or not that's a full section, or just a part of the constructor section, I'm not sure. I feel like its own section is preferable.

Sure! I updated the PR.

@microsoft-github-policy-service agree


Add cache for @@toStringTag; allow DictionaryTypeHandlerBase into missing-property caches and invalidate when necessary.

Should give us a ~3% improvement in EmberJS on Speedometer 2, as that test involves a lot of toString operations each of which looks up the @@toStringTag property and (as it happens) fails to find it.

Can be a separate change, but seems like it would be a good idea to do the same thing with @@toPrimitive

@dotnet-bot test Windows 7 ci_dev12_x64_debug

@aneeshdk how do we make use this to help @@toprimitive @@isconcatspreadable and @@hasintance  ?


Add Sedona Docker Image Download info to Readme

Expected behavior
Actual behavior
Steps to reproduce the problem
Settings
Sedona version = ?
Apache Spark version = ?
Apache Flink version = ?
API type = Scala, Java, Python?
Scala version = 2.11, 2.12, 2.13?
JRE version = 1.8, 1.11?
Python version = ?
Environment = Standalone, AWS EC2, EMR, Azure, Databricks?

Fixed


Is it possible to apply policies only to associated authentication scheme?

My setup:
 services
            .AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
            .AddJwtBearer(options =>
            {
              //setup
            })
            .AddJwtBearer(AuthenticationSchemeTypes.FileDownload, options =>
            {
                //setup
            });

Now I'm mapping policies to auth scheme
 services.AddAuthorization(options =>
        {
            options.AddPolicy(CustomPolicies.FileDownload, policy => policy
                .AddAuthenticationSchemes(AuthenticationSchemeTypes.FileDownload) <- only for this scheme
                .RequireAuthenticatedUser()
                .RequireClaim(JwtClaimTypes.Id));

            options.AddPolicy(CustomPolicies.SomeOtherPolicy, policy => policy
                .AddAuthenticationSchemes(JwtBearerDefaults.AuthenticationScheme)  <- only for this scheme
                .AddRequirements(new SomeOtherPolicy));
                ...

Now I want to protect all endpoints with SomeOtherPolicy that are using default authentication scheme. I set it globally like this:
 app.UseFastEndpoints(c =>
        {
            c.Versioning.Prefix = "v";
            c.Endpoints.ShortNames = true;
            c.Endpoints.Configurator = ep =>
            {
                    ep.Policies(CustomPolicies.SomeOtherPolicy);
         }

And my file download file endpoint config:
 public override void Configure()
    {
    //setup
        AuthSchemes(AuthenticationSchemeTypes.FileDownload);
        Policies(CustomPolicies.FileDownload);
    }

However when I execute download file endpoint, it validates all default authentication scheme policies and FileDownload.
Is it possible to map policies to a particular authentication scheme and set authentication scheme in endpoint?

i believe something like the following should do the trick:
app.UseFastEndpoints(c =>
{
    c.Endpoints.Configurator = ep =>
    {
        if (ep.PreBuiltUserPolicies?.Contains(CustomPolicies.FileDownload) is not true)
            ep.Policies(CustomPolicies.SomeOtherPolicy);
    };
});

because otherwise, you're adding SomeOtherPolicy to every endpoint.
if that doesn't do it, pls attach a runnable minimalistic repro project and i'll look in to it further.

This would do the job however I was thinking that associating authentication scheme with policy and settings auth scheme in the endpoint would do that automatically for me.

all FE is doing is dynamically creating an [Authorize] attribute and setting it's Policy, AuthenticationSchemes, and Roles properties according to what methods were called in the endpoint Configure().
other than that, all the auth behavior is dictated by how the asp.net middlware is configured.
in other words, calling the AuthSchemes(), Policies() and Roles() inside Configure() is exactly the same as decorating an mvc controller or minimal api endpoint with an [Authorize(...)] attribute. the rest is done by the middlware you configure.


Allow links to accept a string so that curie links work properly

The title says it all: HAL representers should also allow curie links for link collections. Unless I missed something this did not yet seem possible.

:+1:

@edejong thanks!


Add files via upload

@Pro-erzounet file/lib/domains/be/hech/student.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@Pro-erzounet First stage of verification passed, official domain is hech.be as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@Pro-erzounet Thank you for your patience while waiting for a review and processing your request. Pull request merged.


Bugs in the react-tutorial.

I found some discrepancies in the react app tutorial contained in this repo. I would like to point those out:

No Support for HOC : I can directly access the url http://localhost:3000/todo without the need for registering (which is bad). We can implement HOC that will take care of this.
No Alert facility : The signin/signup components do not provide the facility for checking completeness. Although the button does not work, looks good if we have a completeness feature enabled.
Documentation: The Readme is also a one-liner. We can improve upon it.

I can do the needful given some time (probably after my mid-semester exams get over in college).
Please revert the same.

There are quite a few things which are incorrect in this sample, most importantly the client api version is incorrect

It would be good if you could update the sample with the recent client api. Meanwhile I'll focus on improving the docs a little bit.


#131 add blog about wewe

Fixes # .
Make sure these boxes are checked before creating your pull request -- thank you!

[x] Add Blog Header, including title, author, date.
e.g.

---
 layout: post
 title: "'Score Your Face Photo' a ML&Wechaty practice"
 date: 2017-09-18 09:00 +0800
 author: huyingxi
 ---

 > Author: [@huyingxi](https://github.com/huyingxi/wechaty_selfie) enjoying ML&Wechaty at BUPT

[x] Keep all filenames & url as lowercase, and use - to connect words instead of space. e.g. 2017-10-06-wechat-pc-impactor instead of 2017-10-06-WeChat PC Impactor,
[x] Embed the photo & video before publishing, save all external file to the blog /download/2018 directory.
[x] Add <!--more--> section to just show the abstract in blog homepage.

See more

Thanks！ Could you add my wechat:<PHONE_NUMBER>3 I will invite you to Wechaty Contributor Group


Standardize Entities During Extraction

This code update modifies the logic used to created the ENTITIES_LOOKUP_DICT (used for mapping entities to a normalized name) such that for aliases the mapping is the actual Name (non-alias) for the entity.
To test:

Pull down this branch and pip install the gamechanger-data package in your env
Option A: Create a test python scrip including the following:

from common.document_parser.lib import entities
# This should print the standardized `Name` value for the row in Graph Relations
print(entities.ENTITIES_LOOKUP_DICT.get(entities.replace_nonalpha_chars("GCCS-J")))
## `GCCS-J` can be replaced by any alias and this should work

Option B: Run pdf-to-json pipeline and verify that entities are being extracted with their standardized name e.g., if DOD is in the document, this should expand to United States Department of Defense

Where can I get the GraphRelations file?

Where can I get the GraphRelations file?

https://gitlab.advana.boozallencsn.com/advana/gamechanger/gamechanger-ml-source/-/merge_requests/155


No redirect URI actioned on OAuth OneDrive logout

(This is a variation on a closed issue: https://github.com/OneDrive/onedrive-api-docs/issues/35)
The redirect_uri doesn't redirect for me.
I logout the client with (PHP) https://login.live.com/oauth20_logout.srf?client_id=' . urlencode(CLIENT_ID) . '&redirect_uri=' . urlencode(CALLBACK_URI)
... which is all by the book ("redirect_uri ... must match exactly the redirect_uri value used in the get token request"). (Unlike the OP in the closed issue, I do supply the redirect_uri value in the login call.)
The client is duly signed out (and required to sign in again) - but there's no redirection and I need to code an explicit redirect.
So ..
Q.1: Is redirect_uri actually used by oauth20_logout.srf?
Q.2: Is there any downside to doing an explicit redirect myself?

The oauth20_logout page doesn't use the redirect_uri parameter. It clears the auth cookies that are stored in the browser and that's it. You'll need to detect that the page has finished loading and then redirect back to a sign-in experience.


handle licensing

as Fossa is dropped we either search for other available and free alternatives for license management or do manually, which is not a big thing, as the number of deps is still small.

Will be done manually.


refactor: justify readme text

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Hey @caioreius
Thank you for opening this pull request.
This is a standard message notifying you that we've reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you and happy coding.

You haven't tested it locally

in the guideline, there is a mention of opening PRs like this, and what is the process that you should take.
There is no issue discussing the changes, and description in the PR itself discussing the need for the change.

@Sboonny, i fixed. can I give a new Pull?

Discuss it in issue first before opening new PR, other maintainers won't see it in a good light, because of your previous PRs.

@caioreius

Could you please explain to me why the pull request cannot be accepted?

The align attribute should not be used on the p element (technically, the specs don't allow it). If you want to style text in a p element it should be done with CSS. Also, it is considered bad for accessibility to align text in a justified manner, so unless there is a very good reason to do so then it should not be done.

@bbsmooth All right, I understood, I thought the text would be better to understand.
Thanks

@bbsmooth If you look, the changes look very good in the document, I think it looks better to visualize, but that's ok, you know what you do ..

@caioreius This is not my personal opinion. It is documented in a WCAG success criterion. I will post more info in the issue you just opened.


Socks5Proxy and HostKey do not have dial timeout

Whilst investigating https://github.com/cloudfoundry/bosh-cli/issues/432 the BOSH CLI hung indefinitely because the Socks5Proxy and HostKey do not set a Timeout in their ssh.ClientConfig.
Would you accept a PR to timeout after, say, 30 seconds?
Josh and @mirahimage

yes


Website build is broken

I followed the instructions in website/README.md:

Clone jest repo
cd website
npm install
npm start
Go to http://localhost:8080/jest/index.html

Error: Error: ERROR RENDERING PAGE ON SERVER:
<mapped error>
TypeError: Cannot read property '__reactAutoBindMap' of undefined
Please report this to https://github.com/chjj/marked.
    at Constructor (evalmachine.null:null:15)
    at Parser.tok (evalmachine.null:null:14)
    at Parser.parse (evalmachine.null:null:19)
    at Function.Parser.parse (evalmachine.null:null:17)
    at marked (evalmachine.null:null:19)
    at React.createClass.render (evalmachine.null:null:32)
    at ReactCompositeComponentMixin._renderValidatedComponentWithoutOwnerOrContext (evalmachine.null:null:34)
    at ReactCompositeComponentMixin._renderValidatedComponent (evalmachine.null:null:14)
    at wrapper [as _renderValidatedComponent] (evalmachine.null:null:21)
    at ReactCompositeComponentMixin.mountComponent (evalmachine.null:null:30)
</mapped error>

    at <MY HOME>/jest/website/node_modules/react-page-middleware/src/guard.js:27:12
    at onOutputGenerated (<MY HOME>/jest/website/node_modules/react-page-middleware/src/index.js:96:9)
    at Object.renderReactPage.done (<MY HOME>/jest/website/node_modules/react-page-middleware/src/DefaultRouter.js:352:7)
    at renderReactPage (<MY HOME>/jest/website/node_modules/react-page-middleware/src/renderReactPage.js:127:17)
    at renderComponentPackage (<MY HOME>/jest/website/node_modules/react-page-middleware/src/DefaultRouter.js:338:3)
    at routePackageHandler (<MY HOME>/jest/website/node_modules/react-page-middleware/src/DefaultRouter.js:281:5)
    at onComputePackage (<MY HOME>/jest/website/node_modules/react-page-middleware/src/index.js:100:9)
    at <MY HOME>/jest/website/node_modules/react-page-middleware/src/guard.js:29:10
    at onWarmed (<MY HOME>/jest/website/node_modules/react-page-middleware/src/Packager.js:282:9)
    at <MY HOME>/jest/website/node_modules/react-page-middleware/node_modules/async/lib/async.js:116:25

Because of full git paths in website's package.json, the version 0.13.0-rc1 of react-tools is downloaded
and it is probably the reason of the error: https://github.com/facebook/react/issues/2880
I solve the problem by changing the dependency target:
"React": "git://github.com/facebook/react.git#v0.12.2",

FWIW the Flux website (same build) just uses
"React": "~0.12.0",

Hmm strange, React (capitalize) is not registered in npm, resulting a 404 on npm install (not tested with flux's website but I suppose it's the same).
Anyway "react": "~0.12.0", works well.
Tell me what do you prefer, I can amend the commit.

Yea, you probably want lower case and not refer to the git repo.
Filesystems are fun aren't they 😢

Commit amended few days ago. Ping @zpao

Thanks!


Kwxm/plc builtin updates

This updates the syntax for builtin application to avoid ambiguity between type and term variables, hopefully dealing with #1143.  The syntax is implemented in the branch for saturated builtins.
Also, it adds a missing hypothesis about kinds to the typing rule for builtin applications.

While I'm here, I notice that in the typed machines we create a frame for type abstractions and then immediately discard it.  Should we just have
s ▻ {M A} ↦ s ▻ M

instead?  I have a vague memory that we used to do something different here and then changed it.

Should we just have s ▻ {M A} ↦ s ▻ M instead?

There's a corner case: when M is a builtin and it doesn't get fully saturated in the end, we return the full {M A} application as a result. Not sure if it's important, though, as we do not keep track of type variables and stuff in the CEK machine, so that A can be ill-scoped anyway.
I'd perhaps wait until we have saturated builtins and then change the evaluation semantics the way you suggest.

There's a corner case: when M is a builtin and it doesn't get fully saturated in the end, we return the full {M A} application as a result. Not sure if it's important, though, as we do not keep track of type variables and stuff in the CEK machine, so that A can be ill-scoped anyway.
False alarm. I realise now that I was talking nonsense: we do need the {} frame, because otherwise there's no way to know when an abstraction has to be instantiated, and that matters now that we don't have the value restriction.

False alarm. I realise now that I was talking nonsense: we do need the {} frame, because otherwise there's no way to know when an abstraction has to be instantiated, and that matters now that we don't have the value restriction.

Ah, indeed, we need to drop the TyAbs, but for that we need to evaluate M first, which means that we need to put the {} frame onto the stack. And since TyAbs blocks evaluation and we don't have value restriction, we can't just drop it arbitrarily.

The definition of the CEK machine in the document is wrong.  If you give it a term like [(lam n (con integer) (builtin addInteger n n)) (con 5)] then it fails because the environment is empty by the time it gets to the second n when it's trying to apply addInteger.  I think the solution to this is to save the current environment in the frame for builtin and then use that to evaluate all of the arguments.
Since this branch is still open I'll fix that here once I've convinced myself the fix is correct.

I think this should be correct now, but it would be helpful if someone could check.  The problem was that when you evaluate a variable the current environment is discarded and replaced with the one associated with the variable.  When evaluating a builtin it now saves the environment in effect when the evaluation starts and uses it to evaluate all of the arguments (and the builtin itself when the arguments have all been computed, but that's irrelevant at the moment because builtins don't have access to environments).

Nobody's complaining, so I'm just going to merge this. I'd still appreciate confirmation that the update to the way the CEK deals with environments during builtin application looks OK though.

I plan to start implementing the CEK machine following the spec next week.


Performance issues with a high workspace ID open

What I was doing beforehand:
I was doing some testing to see what the highest workspace possible with Hyprland is. It came up to<PHONE_NUMBER>.
Then, when I was done, I just kept a terminal window open to not close off that workspace.
But afterwords, the compositor froze when I used certain methods for switching workspaces.
How to recreate
Doing hyprctl dispatch workspace e+1 would cause a lag spike, and swiping left/right on trackpad also caused the same lag spice.
Image of bottom showing it uses all resources for a small amount of time (it just shows that only 1 core is being used to process this, but this 1 core is halting the entire wm)

Extra note
Another thing to note is that there is no lag when switching from any workspace using the trackpad from 1 to the last workspace from the large gap. So in this image I can switch through all the workspaces just fine except for with workspace 10 which lags even when I try to swipe to the left (to get to workspace 5).

The lag also doesn't happen when using the hyprctl command to switch manually to that workspace (and the sliding animation plays just fine)
Suggestion to fix this
Could there be a list storing all the indexes of the open workspaces, then when switching workspaces you get the ID of the workspace to show by subtracting or adding 1 from the index of the current open index.
This is because what I think you are doing (I haven't checked the code, so this is just a guess) is going changing workspace ID's until you reach one that has a window open.
This is probably a very low priority as I don't think anyone is going to open up<PHONE_NUMBER> workspaces any time soon, but best to have this report in case someone actually needs that many workspaces for some reason.

should be fixed with 0803febac521ae07394eb10a628b9507118335eb