Respecting service supportedGrantTypes [WIP]

I've tried to enable password grant type for OAuth 2.0 and noticed some strange things:

supportedGrantTypes affect nothing. All of the available grant types are supported by all services and nothing can be done about it.
password grant type does not work as intended. username and password fields does not affect anything and I can type anything here or just leave totally leave them out. CAS gives me an access token anyway.
When I use password grant type I can see CAS going to my LDAP Attribute repository fetching attributes for principal id with a value specified in client_id parameter (!)
With an access token got from part 2 I can acquire a profile but it is nearly empty and contains the following:

{
    "id": "client_id_value"
}

Currently I've only managed to deal only with the first problem. Now OAuth grant type validators check if the service support given grant type.
Currently I've not figured out how to deal with the second part (wrong behavior of password grant type) but I'd like to get some feedback now: maybe I am already doing everything wrong.
Also notice that this is a PR to the 5.3.x branch which is more important for me now. It probably should be ported to master one time if it is OK.

Thank you very much. Let's pick this apart:

For the first issue, it looks you may have found an issue. Could I ask you to try to get the build in order and passing, and then when it all checks out, post yet another PR for master?
For the remaining issues, I recommend you try to put together a sample project on Github and share its link here that demonstrates the issue. Show the requests and responses back and we'll try to work through it together. Items 2-4 are likely related, but me personally, would like to see the interactions first to see if you have missed anything obvious. If there are any fixes needed, we'll work through separate PRs together for each change, depending on the impact.

Thanks again.

Regarding the first, I don't understand the reasons for failing test. Guess I'll have to undo my changes to OAuthRegisteredService (bring HashSet back) but I am not sure.

Regarding the example, here it is: https://github.com/gagarski/cas-overlay-template. I've slightly adjusted config locations for this example so you don't have to copy anything to /etc: etc dir in the project root is a root for config for this overlay. To run it you just have to do mvn clean package and java -jar target/cas.war. The YAML service storage is in ./etc/cas/services.
First of all, I've started it and ensured that casuser:Mellon are the only valid credentials.
Then I did the following requests:
$ curl -k -X POST -d "client_id=whatever&client_secret=totallysecrettrustme&grant_type=password&username=casuser&password=Mellon" https://localhost:8443/cas/oauth2.0/accessToken
{"access_token":"AT-3-SJswJRSEd7fwDGEWHpt1Hoe4beuMBBsH","token_type":"bearer","expires_in":28800}%

$ curl -k -X POST -d "client_id=whatever&client_secret=totallysecrettrustme&grant_type=password&username=johndoe&password=helloworld" https://localhost:8443/cas/oauth2.0/accessToken
{"access_token":"AT-4-jaXM33Mm7YRYB9eRJiAmBnNzlqaPmA4A","token_type":"bearer","expires_in":28800}%

$ curl -k -X POST -d "client_id=whatever&client_secret=totallysecrettrustme&grant_type=password" https://localhost:8443/cas/oauth2.0/accessToken
{"access_token":"AT-5-NISzdpVTn09CzQcFs48-UGZZC-pFOtnU","token_type":"bearer","expires_in":28800}%

$ curl -k -X GET "https://localhost:8443/cas/oauth2.0/profile?access_token=AT-3-SJswJRSEd7fwDGEWHpt1Hoe4beuMBBsH"
{
  "attributes" : { },
  "id" : "whatever"
}%

$ curl -k -X GET "https://localhost:8443/cas/oauth2.0/profile?access_token=AT-4-jaXM33Mm7YRYB9eRJiAmBnNzlqaPmA4A"
{
  "attributes" : { },
  "id" : "whatever"
}%

$ curl -k -X GET "https://localhost:8443/cas/oauth2.0/profile?access_token=AT-5-NISzdpVTn09CzQcFs48-UGZZC-pFOtnU"
{
  "attributes" : { },
  "id" : "whatever"
}%

@mmoayyed What should I do to run all the tests locally? It seems like both ./gradlew build and ./gradlew test skip it and using CI to debug my changes step by step is not very convenient.

You can locate the test suite for the module at hand and run the batch. If you want to run everything tagged as a simple unit test, you could take a look at the run-tests-simple.sh in the CI folder and use that command. I should say though that ultimately, the only thing we care about passing tests in the CI. Things might work well on your machine and pass all fine, but CI is the reference :)
It will be some time before I can walk through your scenarios (and thank you very much for those) but I'll be in touch.

Things might work well on your machine and pass all fine, but CI is the reference :)

Yeah, of course. I just meant that I want to debug it on my local machine before pushing again.
Thanks for your response.

Wonderful. I'll probably start to collaborate with you on this mid next week.

Regarding OAuth flows. It seems like the password flow in CAS does the same as client_credentials (but for the latter it's correct behavior). One thing that confuses me a bit is that for client_credentials it returns client_id in the id profile field. It may be the intended behavior, yet I thought it worth mentioning.

Are you including client_secret=totallysecrettrustme in the request because persumeably your application is a confidential client? If not, have you tried dropping that bit from the request?

We just tried to user password flow for quick debugging purposes and included client_secret "because we can". Yeah it seems to work properly without client_secret on my example project (I get right id), I'll check attribute fetching later. But I am still not sure if the behavior with client_secret present in the request is correct. Isn't a confidential client with password flow a valid case?
About this PR: I'll try to finish my work this or next week.

Isn't a confidential client with password flow a valid case?

It would be yes, which would make it a feature request and not exactly a defect, and if I know this correctly, a confidential client is required to pass the client id and the secret as an Authorization header, and not directly in the request. Though it doesn't make sense for an OP to authenticate the request twice (once using username/password and once using client_id/secret), that seems to be the use case. A confidential client scenario isn't supported in CAS 5.x, and given the scope of the issue and the release, that change would be better fitted to master/6.
Also note that in 5.x, the set of supported grant types can be defined as "empty" which means everything is allowed by that service definition. Your patch will need to take this behavior into account so as to not break everyone's integrations and it was done specifically so to not present a breaking change. So, you need to ensure that if the set of supported grant types is left undefined or blank, the request can go through with a warning in the log. If however, at least one grant type is defined then the request may be checked for the supported grant types defined.

I've dug a little more into CAS source code.
Seems like grant_type checking is intended to be implemented in the following classes:

OAuth20ClientCredentialsGrantTypeAuthorizationRequestValidator
OAuth20PasswordGrantTypeAuthorizationRequestValidator
OAuth20RefreshTokenGrantTypeAuthorizationRequestValidator

For some reason the instances of these validators are passed into OAuth20AuthorizeEndpointController. The thing is that /authorize endpoint should never receive grant_type parameter, but token endpoint should have it.
So, my guess for now is to add the logic for grant type checking to GrantTypeTokenRequestValidator as I did (perhaps reusing your logic from OAuth20Utils) and get rid of the classes listed above (they are just useless). Is that right?

Each validator operates on the request/thing it can validate. If it can't, it will ignore it and move on.
I believe some recent changes in this area are pushed to master. Please review and see if those help with this effort.

No. Here is how authorize endpoint controller created in master:
https://github.com/apereo/cas/blob/b90bdf6a224fa4625c38e311e5e47155b7f55dbf/support/cas-server-support-oauth/src/main/java/org/apereo/cas/config/CasOAuthConfiguration.java#L670-L687
And here are oauthAuthorizationRequestValidators:
https://github.com/apereo/cas/blob/b90bdf6a224fa4625c38e311e5e47155b7f55dbf/support/cas-server-support-oauth/src/main/java/org/apereo/cas/config/CasOAuthConfiguration.java#L499-L509
As you can see, all *GrantTypeRequestValidators are added to this controller.
For now I've removed them (but haven't pushed it yet) and trying to make tests work on my machines. But maybe I miss something in OAuth protocol?

A reminder to myself: take a look on OIDC endpoints. The same things can probably happen there.

Didn't manage to make ./run-tests-simple.sh run on my machine It fails with a test unrelated to my changes.

org.apereo.cas.DefaultPrincipalAttributesRepositoryTests > checkDefaultAttributes FAILED
    java.lang.AssertionError: expected:<3> but was:<0>
        at org.junit.Assert.fail(Assert.java:88)
        at org.junit.Assert.failNotEquals(Assert.java:834)
        at org.junit.Assert.assertEquals(Assert.java:645)
        at org.junit.Assert.assertEquals(Assert.java:631)
        at org.apereo.cas.DefaultPrincipalAttributesRepositoryTests.checkDefaultAttributes(DefaultPrincipalAttributesRepositoryTests.java:80)

However, I've managed to run tests only for specific module and they've passed. Hope it will work with CI.

By the way, I apologize if I may have been too brief. I'll set aside some time today hopefully, to review the current design and implementation of validators and will try to post back some findings that would explain how each validators and what might work best without a whole lot of changes. (Worst case, Friday).

Validators operate based on the request that can be operated on and one that can support. If they can't support the request and they don't recognize it, they will skip it and move on, allowing the next validator in the chain to handle the processing.

I understand that. My point is /authorize endpoint never receives grant_type parameter -> supports always returns false -> validate is never called. Since *GrantTypeAuthorizationRequestValidators are not used anywheere else, they are effectively dead code. That's why I've removed them and related tests. But I can restore them if I am wrong.
Regarding *GrantTypeTokenRequestValidators (without Authorize): i found a commit that does the same (20dae32233a84b66e543a197525eecaf8554f676). The code there does quite the same yet a bit differently. It might be better to backport it here, yet I'd fix some issues there (grandType -> grantType and using OAuth20Utils.isAuthorizedGrantTypeForService() instead of reimplementing it).
The rest of changes is tests that verify the new behaviour. I think they could be ported to master as is.

So, that how I see the further work here now:

Fix typos and code duplication for /token validators in master branch (separate PR).
Port /token validators changes from master to this branch, replacing current implementation to reduce the difference in code base.
Copy tests from this PR to master branch (for now they seem more complete) (separate PR).
(?) Remove redundant validators for /authorize endpoint in master (if they are really redundant).
(?) Restore validators for /authorize endpoint in this PR (if they are actually not redundant or if we want to keep then for some other reasons).

Could you please give a feedback on all these points when you are ready?

I understand that. My point is /authorize endpoint never receives grant_type parameter -> supports always returns false -> validate is never called. Since *GrantTypeAuthorizationRequestValidators are not used anywheere else, they are effectively dead code. That's why I've removed them and related tests and I think they should probably removed rom the master too. But I can restore them if I am wrong.

You are absolutely right, and thanks for your patience with me here. Took me a bit to realize this and having looked at this more closely inmaster, I reckon the following classes can be removed in fact:

OAuth20ClientCredentialsGrantTypeAuthorizationRequestValidator
OAuth20PasswordGrantTypeAuthorizationRequestValidator
OAuth20RefreshTokenGrantTypeAuthorizationRequestValidator

The rest should be able to stay as they are.

Fix typos and code duplication for /token validators in master branch (separate PR).

Sounds good. Sure.

Port /token validators changes from master to this branch, replacing current implementation to reduce the difference in code base.

Absolutely.
Copy tests from this PR to master branch (for now they seem more complete) (separate PR).
Wonderful.

(?) Remove redundant validators for /authorize endpoint in master (if they are really redundant).

Yup. The 3 above are good candidates.

(?) Restore validators for /authorize endpoint in this PR (if they are actually not redundant or if we want to keep then for some other reasons).

Same as above, really. Essentially those that operate on grant_type can be removed here.
As always, and while not necessarily a requirement really, it would be best to do these in separate PRs (as you note) and try to keep changes focused and nuclear. Makes it easier for everyone to track, modify and merge quickly, but in the end, do what would be most comfortable for you and I'd would try to accomodate.
Thanks again.

Now this PR is basically a combination of the following ported to 5.3 branch:

20dae32
#3471
#3472
#3473
#3474

So, I've done everything I've planned here and these changes are ready for the review (seems like tomorrow this PR will be marked as stale).


Fix/gateway amount quantize

Before submitting this PR, please make sure:

[x] Your code builds clean without any errors or warnings
[x] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:
This mainly fixes the issue with quantizing amount for gateway connectors.
Tests performed by the developer:

Run LP strategy with WETH-USDC and confirmed LP positions are created successfully
Reason for using that pair is that USDC had 6 decimal places, hence, a good one to test this fix.

Tips for QA testing:

Test LP and AMM strategies with any pair having less decimals to confirm this fix.

related to https://github.com/hummingbot/hummingbot/issues/5642

@nikspz
Can you post a link to the issue page you're describing for Kovan and Optimism?

@vic-en https://github.com/hummingbot/hummingbot/issues/5668

Tests Performed:

arbitrum

checking amm_arb

using https://rpc.ankr.com/arbitrum nodeURL

could check balance
Bot return huge traceback after start of amm_arb strategy

using [https://arbitrum-mainnet.infura.io/v3/8e7e…………………………..e0927](https://arbitrum-mainnet.infura.io/v3/8e7ede0dd2034cb0b058d9c6338e0927) nodeURL

Failed to check balance

Bot returns an exceptions.TimeoutError

start the strategy - bot failed to start

Call to https://localhost:30806/network/status failed
asyncio.exceptions.TimeoutError

previously I was able to start the strategy and do the trades using this nodeURL (now doesn’t work)

optimism

checking amm_arb
https://ropsten.infura.io/v3/ nodeURL - failed to connect wallet
https://rpc.ankr.com/optimism node URL

Failed to check balance

Bot returns an exceptions.TimeoutError

Failed to trigger no arb opportunity

UniswapLP

Blocked with https://github.com/hummingbot/hummingbot/issues/5657
tried Delete gateway_connections.json file in the humminbot/conf dir

issue still persists

@arnelhbot @nikspz

None of those issues reported are related to the changes I've made.
I'm unable to reproduce any of the issues that have been reported

Fix/gateway amount quantize #5664

Currently blocked with  https://github.com/hummingbot/hummingbot/issues/5657 (pending check for UniswapLP)
Tests Performed:

Cloned and installed branch
Created gateway image manually
Gateway generate certs/crete successfully

UniswapLP
Blocked with https://github.com/hummingbot/hummingbot/issues/5657

Amm arbitrage:

Mainnet:

stuck into uniswap_ethereum_mainnet not ready: waiting for allowances.

Arbitrum one:

Connected key - ok
Checked balance - ok
Create/started strategy successfully - ok
Triggered no arb opportunity
Set up min profitability and trigger trades - ok
Checked history/CSV - ok

Optimism:

Connected key - ok
Checked balance - ok
Create/started strategy successfully - ok
Triggered no arb opportunity
Set up min profitability and trigger trades - ok
Checked history/CSV - ok

Kovan:

Connected key - ok
Checked balance - ok
Create/started strategy successfully - ok
Triggered no arb opportunity
Set up min profitability and trigger trades - ok
Checked history/CSV - ok

Pangolin:

Connected key - ok
Checked balance - ok
Create/started strategy successfully - ok
Triggered no arb opportunity

@nikspz @arnelhbot
What else is blocking this pr?

@vic-en Pending dev review from @Faouzijedidi1


Automobilista 2 support

Hi Markus, Hoping you can add Automobilista 2 to your supported sim racing apps. I tried and it didn't work.  It works fine with projects cars 2 setting.


[variables.js] minify variable values

CSS property values stored in JavaScript in material-react-components are not minified. This can lead to warnings from React when using server side rendering. See console output below:

Coverage remained the same at 97.483% when pulling 7c7dcc5322d9cda7a3556f01f1e58b46e9387619 on variables-patch into ecfef30ad58ed8218c70a6341809845126bd1ec1 on master.


fix(go): add redundancy to null-ls config

That's not needed. You should always import LazyVim first, followed by extras, followed by your own plugins


refinery_icon_tag method no longer working so converted to action_icon method

icons were no longer loading because of changes to the refinery-core refinery_icon_tag method.  So went ahead and switched over to action_icon.  I couldn't find a "zoom" equivalent in refinery-core _icons.scss.  So used "preview".

Thanks @dianaq12 !


test-backend: Rename '--processes' option to '--parallel'.

This change is being made to simply match up our naming conventions
with django's test framework.

I would have thought we had references to this in our docs, but apparently we don't.  Merging.


Escape special characters in strings to render as expected for json

Fixes escape sequence characters not rendering correctly in strings/keys.
If a json string contains "\n" then it would be expected that the display of the json string would show "\n" rather than an actual new line (within a <pre> tag) or a space (within most other html tags). This updates how these values are displayed (but not the underlying value).
As an aside, it is possible to edit a string and add such characters into the string however this can't be done by directly typing them (AutosizeTextarea automatically escapes these characters meaning typing "\n" is stored in the json as "\n" which is not what we want). But pressing enter creates a "\n" and other special escape sequences can be copied in.
Note: A couple of the files have a lot of formatting changes but this was due to using prettier with the project's included prettier config.
This is my first PR for this project so let me know if I am following the process correctly.

Hello, I took the opportunity to improve the setup of the project related to conventions with whitespaces, etc.
You're seeing a lot of conflicts because I merged this ahead of your PR
https://github.com/microlinkhq/react-json-view/pull/41
Can you open a new PR, just promoting the changes related with the original bug report?
The new setup should format the code properly 🙏


Hotfix for 'hoveredMarkerId' is undefined

*Lightweight Charts Version: v3.7.0
I've been wanting to target markers on hover, but hoveredMarkerId always returns undefined .
Example use:
const  myCrosshairMoveHandler = (param) => {
	console.log(param.hoveredMarkerId) //undefined
}
chart.subscribeCrosshairMove(myCrosshairMoveHandler)

I don't know Typescript and therefor won't create a pull request, but I've managed to hotfix the source code so it returns the marker's index in the series.
In the compiled version changing the following lines of code does the trick.
@line:10326, from:
var  hoveredObject = hoveredSource !== null && hoveredSource._internal_object !== undefined
? hoveredSource._internal_object._internal_externalId
: undefined;

To:
var  hoveredObject = hoveredSource !== null && hoveredSource._internal_object !== undefined
? hoveredSource._internal_object._internal_hitTestData
: undefined;

Result: console.log(param.hoveredMarkerId) //123
So I took a quick dive into the TS files and changed the following lines:
File: chart-widget.ts
@line32 – from: hoveredObject?: string; -> to: hoveredObject?: any;
@line611 – from hoveredSource.object.externalId -> to hoveredSource.object.hitTestData
I'm not sure what _internal_externalId is supposed to return initially, and I haven't thoroughly tested it, but it works for me.

param.hoveredMarkerId will return undefined if you didn't defined the id when you set the markers.
Are you defining the markers with an id like below?
setMarkers([
    { id: 'id1', time: '2017-04-10', position: 'inBar', color: 'orange', shape: 'circle' }
]);
// hoveredMarkerId will be 'id1'

Here is an example you can try to see the id being returned:
const chart = LightweightCharts.createChart(container, {});
const line = chart.addLineSeries();
line.setData([
    { time: '2017-04-11', value: 80.01 },
    { time: '2017-04-12', value: 96.63 },
    { time: '2017-04-13', value: 76.64 },
    { time: '2017-04-14', value: 81.89 },
    { time: '2017-04-15', value: 74.43 },
    { time: '2017-04-19', value: 81.89 },
    { time: '2017-04-20', value: 81.89 },
    { time: '2017-04-21', value: 81.89 },
    { time: '2017-04-22', value: 81.89 },
    { time: '2017-04-23', value: 81.89 },
]);

line.setMarkers([
    { time: '2017-04-10', position: 'inBar', color: 'orange', shape: 'circle', id: 'id1' },
    { time: '2017-04-16', position: 'inBar', color: 'orange', shape: 'circle', id: 'id2' },
    { time: '2017-04-17', position: 'inBar', color: 'orange', shape: 'circle', id: 'id3' },
    { time: '2017-04-18', position: 'inBar', color: 'orange', shape: 'circle', id: 'id4' },
    { time: '2017-04-24', position: 'inBar', color: 'orange', shape: 'circle', id: 'id5' },
]);

chart.subscribeCrosshairMove(param => {
    console.log(param.hoveredMarkerId);
})

@CreativeSau ^^

@Ffloriel No I didn't. That totally makes sense. Thanks for clarifying!
Maybe adding an 'id' key in the example in the docs could point others into the right direction in the future.


[QE] fix e2e test failure of podman

Test case Pull image locally, push to registry, deploy sometimes failed:

stderr:
Cannot connect to Podman. Please verify your connection to the Linux system using podman system connection list, or try podman machine init and podman machine start to manage a new Linux VM
Error: unable to connect to Podman socket: failed to connect: dial tcp <IP_ADDRESS>:22: i/o timeout
But this error never appear in nightly run. And I checked the machine, podman work well.

this is b/c somehow it is trying to use podman remote instead of podman.

this is b/c somehow it is trying to use podman remote instead of podman.

Can you point out the feature file for that? Also I think we want to use podman remote instead podman on system since it is not default available for mac/windows.

Tests should be using podman-remote for win/mac and podman for linux

@praveenkumar , This failure all happened on tests of Linux.

Tests should be using podman-remote for win/mac and podman for linux

@adrianriobo Does that mean for win/mac it is using podman socket which is running inside the crc VM or using podman-machine vm?

Yeah, for win and mac any scenario making use of podman is relying on "our" podman (crc-VM)

Yeah, for win and mac any scenario making use of podman is relying on "our" podman (crc-VM)

@adrianriobo so in this case we should also do same for linux right?

Well, we are not making it for testing our podman setup, but b/c we rely on some container engine.  In the end we somehow would like to test the interaction between podman / crc.
I would suggest keep the same strategy, as in future we remove the podman socket  or make more tricky to make use of it ..we may end on a situation where we need to use podman machine on win and mac and keep using podman on linux.
dunno...WDYT?

What we want to have our test case follow same steps on all platform to have reliable metrics in case of this https://github.com/crc-org/crc/blob/main/test/e2e/features/story_openshift.feature#L55-L66 (which is causing issue).

Well, we are not making it for testing our podman setup, but b/c we rely on some container engine. In the end we somehow would like to test the interaction between podman / crc.

In this case I would suggest to have a different test case only for linux which use the podman installed on the system instead using podman-socket from the VM.

From source:
https://github.com/containers/podman/blob/7567cc1fcd0764b60386ecbeacaaf575a067e926/cmd/podman/root.go#L120-L124
https://github.com/containers/podman/blob/9b8b2bf684aeb76199467d351d02a014befdcb0b/cmd/podman/registry/remote.go#L21-L23
If we just call on purpose podman remote, should we avoid that on linux?
https://github.com/crc-org/crc/blob/6652e57dbb6d3d317db49dd9ba71edf4de86c043/test/e2e/features/story_openshift.feature#L55-L66

It is because we have https://github.com/crc-org/crc/blob/main/test/extended/crc/cmd/cmd.go#L214C2-L217C44 logic so what I suspect that on linux since we are running podman-remote command we have to make sure we execute crc podman-env command.

Ohh my bad from top of my head I thought that should be using podman not podman-remote...about the crc podman-env this is kind of executed with line 57 on that scenario: Given podman command is available
That line goes to https://github.com/crc-org/crc/blob/d9f83bcc27dc8717469091553c90d9b97c2b5105/test/e2e/testsuite/testsuite.go#L942

After investigation, the podman-remote command failed because CONTAINER_HOST is set as default network system-mode, but actual network is user-mode.
The network mode is changed to user-mode in @story_microshift case, and didn't change back to default.


How to return different promise in a then closure?

firstly {
    UIDevice.microphoneAuthorized()
}.then {
    UIDevice.cameraAuthorized()
}.then {
    UIDevice.checkNetworkOnWifi()
}.then { isOnWifi in
    if isOnWifi {
        return Promise.value(true)
    } else {
        return viewController.showAlertView(content: "Are you sure?", cancelTitle: "exit", confirmTitle: "confirm")
    }
}.done { success in
   .....
}.catch { error in
   .....
}

The code above compiled error.
As you can see, in the third then closure after having checked whether the device is on Wifi. If is  on Wifi, fine, continue to next closure. On the opposite, I just want to show an alertView to let user confirm. The showAlertView method will also return a Promise. After user selects an option, I still want to continue the chain.
Throw an Error may seem not to be a solution in this case.
Could you pls give some advice on how to achieve this feature? Thx very much.

firstly {
    UIDevice.microphoneAuthorized()
}.then {
    UIDevice.cameraAuthorized()
}.then {
    UIDevice.checkNetworkOnWifi()
}.then { isOnWifi -> Promise<Bool> in
    if isOnWifi {
        return .value(true)
    } else {
        return viewController.showAlertView(content: "Are you sure?", cancelTitle: "exit", confirmTitle: "confirm")
    }
}.done { success in
   //…
}.catch { error in
   //…
}

Yes, it works. Thanks.

Great 👍🏻
If you found my work helpful, please consider supporting its development.


Test the Crowdin integration and workflow in a staging environment

Crowdin is a cloud-based platform that helps businesses manage the localization (translation) of their software, websites, and other content. It allows teams to collaborate on translations, track progress, and ensure that their products are available in multiple languages.
We are looking to support three languages with this integration. DLT Science will be responsible for proof reading the transactions that are auto generated by Crowdin and respond to issues from developers in those supported languages.
We need a way to do end to end testing for this workflow.
Tasks

[ ] Create a Hedera docs staging environment
[ ] The Hedera docs staging environment will be assigned to the staging branch in GitHub
[ ] The staging branch will be linked to the Hedera Staging GitBook space
[ ] The Hedera Staging space will need to go through the steps of creating a collection as we expect to do with the production space
[ ] The GitBook staging site has its own URL we can view the implemented changes
[ ] Crowdin instance will need to point the Hedera docs repo and linked to the staging branch
[ ] The Crowding translations need to be written to the staging branch
[ ] Once the directories for each language are present, we need to submit a PR to staging and observe the translation process
[ ] Crowdin also needs to be updated with the proof reader role add the individuals from DLT Science that will be responsible
[ ] We should schedule a meeting with DevRel/DLT Science/Docs to go through the end to end work flow
Push a new change to the staging branch
Crowdin should pick up the change and do the translations
DLT Science proof readers will read an approve
The PR gets opened to the staging branch
A PR is made from the staging branch to the master branch
The PR is merged

Once we validate the end to end process, address open questions, and have assigned clear ownership for which teams owns what part of the process we can schedule the work to complete this on the production site.

Staging branch can be found here: https://github.com/hashgraph/hedera-docs/tree/staging


add german template file

@iamNoah1, so glad to see you're interested in helping to translate the glossary into German!  We are having a kickoff meeting during our Glossary Working Group meeting this Thursday. If you can, please join us. You can find the invite here  by searching for glossary.
Also, we started a localization doc based on what the Kubernetes docs are doing.

@CathPag I will try to attend :)

@iamNoah1, sorry you couldn't make it to the Glossary Working Group. If you're still interested, please join the #glossary-localizations and #glossary-localization-german on the CNCF Slack


[Question]: unable to launch the RAGFlow backend service while launching service from source

Describe your problem
I have run the ragflow sucessfully through docker and the nginx server can be started up normally. BUT when I tried use the Launch Service from Source, I met some problems.
I modified the entrypoint.sh file as follows:

when I run the follow scripts:
"chmod +x ./entrypoint.sh
bash ./entrypoint.sh"
problems happend:
[WARNING] [2024-06-11 18:25:32,265] [_node_pool.mark_dead] [line:249]: Node <Urllib3HttpNode(http://es01:9200)> has failed for 1 times in a row, putting on 1 second timeout
[WARNING] [2024-06-11 18:25:32,265] [_node_pool.mark_dead] [line:249]: Node <Urllib3HttpNode(http://es01:9200)> has failed for 1 times in a row, putting on 1 second timeout
[WARNING] [2024-06-11 18:25:32,266] [_transport.perform_request] [line:395]: Retrying request after failure (attempt 0 of 3)
Traceback (most recent call last):
File "/home/ja/anaconda3/envs/ragflow/lib/python3.11/site-packages/elastic_transport/_transport.py", line 328, in perform_request
meta, raw_data = node.perform_request(
^^^^^^^^^^^^^^^^^^^^^
File "/home/ja/anaconda3/envs/ragflow/lib/python3.11/site-packages/elastic_transport/_node/_http_urllib3.py", line 202, in perform_request
raise err from None
elastic_transport.ConnectionError: Connection error caused by: NameResolutionError(<urllib3.connection.HTTPConnection object at 0x7fb3057e0510>: Failed to resolve 'es01' ([Errno -2] Name or service not known))
[WARNING] [2024-06-11 18:25:32,265] [_transport.perform_request] [line:395]: Retrying request after failure (attempt 0 of 3)
Traceback (most recent call last):
File "/home/ja/anaconda3/envs/ragflow/lib/python3.11/site-packages/elastic_transport/_transport.py", line 328, in perform_request
meta, raw_data = node.perform_request(
^^^^^^^^^^^^^^^^^^^^^
File "/home/ja/anaconda3/envs/ragflow/lib/python3.11/site-packages/elastic_transport/_node/_http_urllib3.py", line 202, in perform_request
raise err from None
elastic_transport.ConnectionError: Connection error caused by: NameResolutionError(<urllib3.connection.HTTPConnection object at 0x7f720e772650>: Failed to resolve 'es01' ([Errno -2] Name or service not known))
[WARNING] [2024-06-11 18:25:33,337] [_node_pool.mark_dead] [line:249]: Node <Urllib3HttpNode(http://es01:9200)> has failed for 2 times in a row, putting on 2 second timeout
Please give some guidance and help, Thank you!

RAGFlow can't connect with ES.

check if ES docker server is up.
check if the port is set correctly.

FAQ document also mention this problem.

Check every host in conf/service_conf.yml. Change them to <IP_ADDRESS>. And mind the ports in conf/service_conf.yml. Make sure these ports are available outside the docker container.


Static link ffmpeg

How can I statically link ffmpeg into executable when compiling in windows inside msys2 ucrt64?
(windows-gnu rustc)

You can check this: https://github.com/CCExtractor/rusty_ffmpeg?tab=readme-ov-file#fine-grained-usage.
In short, set FFMPEG_PKG_CONFIG_PATH if you have pkg-config files; set FFMPEG_LIBS_DIR  + FFMPEG_INCLUDE_DIR vise versa.

@ldm0
I'm not sure where I can find ready to use statically linked ffmpeg libraries,
and if I should compile it (maybe it will take long time to compile...)
Did you tried it in msys2?

@ldm0 I'm not sure where I can find ready to use statically linked ffmpeg libraries and if I should compile it.

It's not recommended to use prebuilt statically linked FFmpeg libraries due to the verbosity of build options and license issues(usage of different build options result in libraries with different open source licenses). You should compile it locally and tweak the build options according to your project requirements.

(maybe it will take long time to compile...)

People only need to compile FFmpeg once as long as they don't change FFmpeg source code & version. BTW, FFmpeg compilation won't take that long time if you have disable the artifacts you don't want.

Did you tried it in msys2?

Nope. I compile FFmpeg for Windows with MinGW. But it should work fine as rsmpeg only needs to link the final static libraries.
Here are some references you may need to compile FFmpeg with MSYS2

https://trac.ffmpeg.org/wiki/CompilationGuide/MinGW#Option2:MSYS2
https://gist.github.com/crossle/43e356468d160902f03ffd25d6a045ba

Feel free to file an issue here if you encounter any problem.


executor: fix report generation

closes #15

Codecov Report

Merging #16 (52da648) into master (011a74f) will decrease coverage by 0.36%.
The diff coverage is 0.00%.

@@            Coverage Diff            @@
##           master     #16      +/-   ##
=========================================
- Coverage    6.19%   5.83%   -0.37%
=========================================
  Files           5       5
  Lines         113     120       +7
=========================================
  Hits            7       7
- Misses        106     113       +7

Impacted Files
Coverage Δ

reana_workflow_engine_snakemake/config.py
0.00% <0.00%> (ø)

reana_workflow_engine_snakemake/executor.py
0.00% <0.00%> (ø)


Use pseudo comments for older systems

At some lines of the source code, there are pseudo comments or pragmas used. Pseudo comments should be used for older SAP systems. These pragmas are unknown according to my SE80:

SUBRC_OK
WARN_OK
ENH_OK

In my opinion these warnings on older systems are fine. There's no negative effect from a user's point of view. From a developer's point of view pragmas have the advantage that an existence checked is performed on compilation / syntax check (and otherwise that raises a warning).
With the AS ABAP Developer edition it is easy to develop abapGit on a recent release so I don't see the benefit of replacing the pragmas.

@flaiker agree, closing


Evaluation and Implementation of the Xwalk (Crosswalk) Feature in AEM

We need to evaluate the Crosswalk (Xwalk) feature and subsequently implement a starter template in our boilerplate for Xwalk projects. see also here

Boilerplate: https://github.com/diva-e-aem/xwalk-boilerplate

The doc based and xwalk boilerplates are quite similar with just some differences in a few lines of code and files (see pull request). We want to extend the cli so that we can generate either a xwalk or a doc based boilerplate.

[ ] branching from eds-boilerplate-generator
[ ] rewrite the cli so that it can be possible to generate a xwalk or a doc based project

hi @mbehzad i've heard that you would like to continue on this topic. If you want have a look in this branch https://github.com/diva-e-aem/eds-boilerplate-generator/tree/18-xwalk-integration and get in touch to me if you need help. Thank you!

Hi @samuelecalamita , in doc-based, jest is using jsdom, in the xwalk boilerplate, @happy-dom/jest-environment is used. Does the two boilerplates also need to have different jest setup (+shared part)? Or is currently one jest setup outdated and a single one should work for both? (in the branch is see only jsdom)

Hi @samuelecalamita , in doc-based, jest is using jsdom, in the xwalk boilerplate, @happy-dom/jest-environment is used. Does the two boilerplates also need to have different jest setup (+shared part)? Or is currently one jest setup outdated and a single one should work for both? (in the branch is see only jsdom)

Hi @mbehzad! One jest setup is definitely outdated. Up to you to choose the right one.


Can I instantiate more than one device-mqtt containers in the same docker-compose?

Hi all,
I'm currently working on a project where I have two types of devices (type A and type B) that will send some data asynchronously over MQTT to an EdgeX deployment. Both types of device publish to their own topic on the same broker:

/data/topicA -- for device type A
/data/topicB -- for device type B

I would like to deploy one device-mqtt container for each type of device, to loosen the coupling. My docker-compose file has the following:
# TYPE A
  device-mqtt-type-A:
    image: edgexfoundry/docker-device-mqtt-go:1.0.0
    ports:
      - "49999:49999"
    container_name: edgex-device-mqtt-type-A
    hostname: edgex-device-mqtt-type-A
    networks:
      - edgex
    volumes:
      - ../edgex-device-drivers/device-mqtt-type-A:/custom-config
    environment:
      <<: *common-variables
      Service_Host: edgex-device-mqtt-type-A
    entrypoint:
      - /device-mqtt
      - --registry=consul://edgex-core-consul:8500
      - --confdir=/custom-config
    depends_on:
      - consul
      - data
      - metadata

# TYPE B
  device-mqtt-type-B:
    image: edgexfoundry/docker-device-mqtt-go:1.0.0
    ports:
      - "49998:49998" # I changed the port here to avoid conflicts
    container_name: edgex-device-mqtt-type-B
    hostname: edgex-device-mqtt-type-B
    networks:
      - edgex
    volumes:
      - ../edgex-device-drivers/device-mqtt-type-B:/custom-config
    environment:
      <<: *common-variables
      Service_Host: edgex-device-mqtt-type-B
    entrypoint:
      - /device-mqtt
      - --registry=consul://edgex-core-consul:8500
      - --confdir=/custom-config
    depends_on:
      - consul
      - data
      - metadata

There seems to be some conflicts with the two containers. When I docker-compose up, the two device-mqtt containers start up, but after a few seconds, the container B catches an error and crashes. Here is the error message:
panic: start incoming data Listener failed, please check MQTT broker settings are correct, Connection lost before Subscribe completed

I'm not sure what to look for... Both my devices are connected to the HiveMQ public broker using port 1883, so everything looks pretty standard to me. Here are some exerpts  from the configuration.toml files for both containers:
Device type A:
# Driver configs
[Driver]
IncomingSchema = "tcp"
IncomingHost = "broker.hivemq.com"
IncomingPort = "1883"
IncomingUser = ""
IncomingPassword = ""
IncomingQos = "0"
IncomingKeepAlive = "3600"
IncomingClientId = "IncomingDataSubscriber"
IncomingTopic = "/data/topicA"
ResponseSchema = "tcp"
ResponseHost = "broker.hivemq.com"
ResponsePort = "1883"
ResponseUser = ""
ResponsePassword = ""
ResponseQos = "0"
ResponseKeepAlive = "3600"
ResponseClientId = "CommandResponseSubscriber"
ResponseTopic = "ResponseTopic"

Device type B:
# Driver configs
[Driver]
IncomingSchema = "tcp"
IncomingHost = "broker.hivemq.com"
IncomingPort = "1883"
IncomingUser = ""
IncomingPassword = ""
IncomingQos = "0"
IncomingKeepAlive = "3600"
IncomingClientId = "IncomingDataSubscriber"
IncomingTopic = "/data/topicB"
ResponseSchema = "tcp"
ResponseHost = "broker.hivemq.com"
ResponsePort = "1883"
ResponseUser = ""
ResponsePassword = ""
ResponseQos = "0"
ResponseKeepAlive = "3600"
ResponseClientId = "CommandResponseSubscriber"
ResponseTopic = "ResponseTopic"

I guess my first question is: can two device-mqtt containers coexist within a same deployment? If so, does anyone have any idea what I'm doing wrong?
Thanks,
Sam

Yes, just need some more configuration.

Add SERVICE_PORT: "49998" to environment section so it service on the port you are exposing
Add EDGEX_INSTANCE_NAME: "device-mqtt-type-B" to environment section so they don't share config and registration in Consul. This is probably causing your issue since once first config is in Consul the second one will ignore local config and just pull from Consul.
Make sure to do clean start by removing stopped containers and volumes so Consul and DB are clear and start fresh.

Thanks for the quick reply.
I edited the device-mqtt-type-b section in my docker-compose file like you said:
# TYPE B
  device-mqtt-type-B:
    image: edgexfoundry/docker-device-mqtt-go:1.0.0
    ports:
      - "49998:49998"
    container_name: edgex-device-mqtt-type-B
    hostname: edgex-device-mqtt-type-B
    networks:
      - edgex
    volumes:
      - ../edgex-device-drivers/device-mqtt-type-B:/custom-config
    environment:
      <<: *common-variables
      Service_Host: edgex-device-mqtt-type-B
      SERVICE_PORT: "49998" # Added this
      EDGEX_INSTANCE_NAME: "edgex-device-mqtt-traverse-pieton" # Added this
    entrypoint:
      - /device-mqtt
      - --registry=consul://edgex-core-consul:8500
      - --confdir=/custom-config
    depends_on:
      - consul
      - data
      - metadata

After that, I removed all my containers and deleted the volumes, etc. Then I restarted everything using docker-compose up and I got another error. The device-mqtt-type-B still crashes, with the following message:
error loading config file: could not register service with Registry: unable to register service with consul: Service information not set

Please attach (not copy paste) your configuration.toml files.

Here are the configuration.tml files for both A and B devices.
configuration-toml-files.zip

Other issue is you are using a very old version (1.0.0).
Hanoi (1.3.1) is the latest supported release. With Hanoi you can accomplish all the config setting changes via environment overrides in the compose files environment sections.
Attach is non-secure Hanoi base compose file with 2 instances of Device MQTT configured for broker.hivemq.com
You will need to uncomment the lines that override ProfilesDir  and add your volume mounts for the /custom-config so your custom profiles are picked up.
I don't see a DeviceList on your TOML files, so assume you are adding the device via REST API?
docker-compose-yml.txt

Thank you very much for this - I was able to launch the two mqtt drivers using the file you provided.
I am indeed provisioning the devices using the REST API. In the POST call I make, I need to specify the name of the service. I used the names "edgex-device-mqtt1" and "edgex-device-mqtt2" for the two types of devices. However, for both device types, I got errors when POSTing:
no item found for supplied key: invalid device service: edgex-device-mqtt1

What Service name should I provide in my POST call? Does it correspond to any environment variable provided in the docker-compose file?
See for reference the provisioning script in Python:
endpoint = "http://localhost:48081/api/v1/device"

headers = {
    'Content-Type': 'application/json',
}

device = {
    "name": "DEVICE_TYPE_A_01",
    "profile": {"name": "device_type_a.mqtt.device.profile"},
    "description": "test device of type A",
    "labels": ["MQTT"],
    "protocols": {
        "mqtt": { }
    },
    "adminState": "UNLOCKED",
    "operatingState": "ENABLED",
    "service": {"name": "edgex-device-mqtt1"}  # This is where I specify the Service name
}

response = requests.post(endpoint, headers=headers, data=json.dumps(device))

It depends on the EDGEX_INSTANCE_NAME you set
please try to query the device service name when your services start running:
http://localhost:48080/api/v1/deviceservice
I guess the name is "device-mqtt_device-mqtt2".
The name would be original name + "_" + EDGEX_INSTANCE_NAME

Ahh, yes the EDGEX_INSTANCE_NAME  adds to the service name, doesn't override the whole name. Thanks @cloudxxx8 !!
@stlaperriere , you may want to tweak the EDGEX_INSTANCE_NAME  values to just 1 & 2, but leave container/hostname as is.

Ok I just made it work. Like @cloudxxx8 said, the name was edgex-device-mqtt_device-mqtt2. I edited my provisioning script and it seems to work now!
Thanks a lot!


Update scala3-library to 3.1.2-RC1-bin-20220124-0e4e5d5-NIGHTLY

Updates org.scala-lang:scala3-library from 3.1.2-RC1-bin-20220121-7ce60e1-NIGHTLY to 3.1.2-RC1-bin-20220124-0e4e5d5-NIGHTLY.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-lang", artifactId = "scala3-library" } ]

labels: library-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #1114.


docs: Fix a few typos

There are small typos in:

module/src/main/cpp/Dobby/source/InterceptRouting/InterceptRouting.cpp
module/src/main/cpp/Dobby/source/InterceptRouting/RoutingPlugin/FunctionWrapper/intercept_routing_handler.h
module/src/main/cpp/Dobby/source/Interceptor.h
module/src/main/cpp/Dobby/source/UserMode/MultiThreadSupport/ThreadSupport.h

Fixes:

Should read utility rather than utilty.
Should read replace rather than relpace.
Should read multiple rather than mutipl.
Should read instructions rather than insturctions.

Semi-automated pull request generated by
https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

Any questions about Dobby, please go here to submit.
https://github.com/jmpews/Dobby


mergin tracking  sern_simpcw_2023

New issue to deal seperately with sern_simpcw_2023
v23 fix duplicate site name for what was erroneous duplicate 13905538
v24 sort and rename photos to site directories on mergin.  assumes all photos were tracked in mergin field form which is not always the case.

v25 - Cleaned form_pscis_2023, ready for QA now.

V26 - QA half of the pscis sites. Second half coming tomorrow.

v27 Update
sf::st_read("/Users/airvine/Projects/gis/sern_simpcw_2023/background_layers.gpkg", layer = "rfp_tracking")
#> Reading layer `rfp_tracking' from data source
#>   `/Users/airvine/Projects/gis/sern_simpcw_2023/background_layers.gpkg'
#>   using driver `GPKG'
#> Warning: no simple feature geometries present: returning a data.frame or tbl_df
#>          timestamp                                  content
#> 1 2024-06-25 07:23           whse_fish.pscis_assessment_svw
#> 2 2024-06-25 07:23      whse_fish.pscis_design_proposal_svw
#> 3 2024-06-25 07:23 whse_fish.pscis_habitat_confirmation_svw
#> 4 2024-06-25 07:23          whse_fish.pscis_remediation_svw
#> 5 2024-06-25 07:28    bcfishobs.fiss_fish_obsrvtn_events_vw
#> 6 2024-06-25 07:28                  bcfishpass.crossings_vw
#> 7 2024-06-25 07:28                    bcfishpass.streams_vw
#>                                                        watershed_groups
#> 1 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 2 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 3 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 4 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 5 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 6 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#> 7 'ADMS' 'MORK' 'UFRA' 'CANO' 'LNTH' 'UNTH' 'MURT' 'CLWR' 'CLRH' 'REVL'
#>                  source
#> 1 rfp_source_bcdata.txt
#> 2 rfp_source_bcdata.txt
#> 3 rfp_source_bcdata.txt
#> 4 rfp_source_bcdata.txt
#> 5    rfp_source_aws.txt
#> 6    rfp_source_aws.txt
#> 7    rfp_source_aws.txt
#>                                                                                description
#> 1                             This view defines the most current assessment for a stream crossing location. An assessment provides the field survey data required to determine whether a stream crossing requires remediation efforts in order to make it passable to fish habitat.
#> 2                                                                                This view defines the stream crossing locations in which design proposals have been received to remediate a stream crossing that is defined as not passable.
#> 3 PSCIS_HABITAT_CONFIRMATION_SVW represents Habitat Confirmation of Stream Crossings. The Habitat Confirmation is carried out by a biologist who does an office and field review of the fish habitat to be potentially gained by remediation of an identified failed road crossing.
#> 4                                                                                This view defines all the stream crossing locations that have had remediation activities take place.
#> 5
#> 6
#> 7

Created on 2024-06-25 with reprex v2.1.0
plus
path_gpkg <- "/Users/airvine/Projects/gis/sern_simpcw_2023/background_layers.gpkg"

layers_remove <- c("streams", "crossings", "fiss_fish_obsrvtn_events_vw")

purrr::map(layers_remove, ~ gdalraster::ogr_layer_delete(path_gpkg, .x))

v28 - more pscis QA. Synching so I can get updated layers from v27

v32 - add id_camera when missing .  commit before running 01b_HACK_pscis_refresh.R
v33 - reburn to project after cleaning with 01b_HACK_pscis_refresh.R


REST API /chain/blocks/ returns empty blocks (other than block 0)

This issue is observed in conjunction with another error case, see #920. When a test program makes the REST API call /chain/blocks/, it gets a normal HTTP response but the data is NULL, i.e., the block has no transactions. I don't think this makes sense except for the genesis block. This issue is observed in
commit 92cc78029d04736928c3c07b7c3336023de2d747
Merge: c4b31f9 96210ef
Author: Sheehan Anderson<EMAIL_ADDRESS>Date:   Sun Apr 10 21:17:43 2016 -0400

    Merge pull request #1044 from ghaskins/rebrand

and is easy to reproduce if required.
@jyellick offered the following opinion in #920:
My suspicion is that the peer is currently performing state transfer and so his block height does not    necessarily match the number of blocks he has. This is probably fooling the error checking that exists at that layer which assumes the chain is always unbroken.
I think we need to think about a policy for more clearly handling requests when a replica knows its state is not complete. Do we make a best effort to return a correct result? Do we reject all queries? This seems like it should be a well studied problem, maybe chetmurthy could weigh in?

I can confirm that I am seeing this, see #920. What is being returned is null (not sure is this is really the empty string, or something that prints as "null")
vagrant@hyperledger-devenv:v0.0.9-0c353b1:/opt/gopath/src/github.ibm.com/bcbrock/busywork/scripts$ curl http://<IP_ADDRESS>:5000/chain/blocks/71
null

@bcbrock I wouldn't rule out that this is state transfer related, it's possible that in this scenario, state transfer is being invoked so many times, it is producing some gaps that are for whatever reason not being filled in, I'll investigate this possibility. (Though I'll agree, this should reply with an error, rather than an empty block if it is missing)

@jyellick I was a bit optimistic when I said in #920 that the runs were going to completion. I'm seeing now that as often as not they actually fail in various ways. I assume it is best to wait until this issue (1091) is fixed before posting the other issues, would you agree?
I can also add here that the evidence suggests that if peer 0 has empty blocks in its blockchain, peer 0  also appears not to have executed every transaction.

This issue is still active, even after the numerous updates made yesterday.

@bcbrock By this issue do you mean the 'empty block' problem? If so, I think @srderson may be the most qualified person to investigate. (Or, @srderson if you'd like me to try to track this down, let me know)
In my stress tests, I'm unable to even force a view change via DOS anymore, if you could upload your logs at DEBUG (or at least with consensus/obcpbft and consensus/statetransfer at debug), I would like to see how it is that one or more of your replicas are being forced into statetransfer.

@jyellick @tuand27613 Yes, I mean the 'empty block' problem. This may have something for @srderson to look at, however it is clear that the peer with the corrupted blockchain is not executing every transaction.
This 10K transaction run generates 1GB of logs even with the limitations suggested, so I think it will be best for you to try to recreate it yourselves. This is how:

Install https://github.ibm.com/bcbrock/busywork
cd busywork/scripts; make images
cd busywork/counters; make issue1091

FWIW, messages like this always appear
/home/bcbrock/vp2_logs/stderr:15:17:33.995 [consensus/statetransfer] stateThread -> ERRO 98025 name:"vp2"  could not retrieve blocks as recent as 2240 as the block hash advertised
/home/bcbrock/vp2_logs/stderr:15:17:33.997 [consensus/statetransfer] stateThread -> ERRO 9803c name:"vp2"  believed its state for block 2240 to be valid, but its hash (7e73a9c972a8566e04f84547c3f1277f94f5350d4ce54393818af529f272b067a1e28ac453bdc698c9c837a64af5e8529fe0d07bcb8a9c155e64293fc13f517e) did not match the recovered blockchain's (b9b4e2ee523d16d9cf54d2181d3947f0932ed6572588e7f756095cff5622b07dfb21d47afeff70e209f98681ab41055d1a918fd71ed35c5815162c5ebd5be7ba)

@bcbrock Agreed, it sounds like there is some other PBFT bug lurking here.  I was no longer able to reproduce the stress bugs with my local testing setup, so thanks for the instructions to configure yours.  I will do that, and post back here once I have results.

I'm seeing the same or similar issue in PBFT Batch. Here the block data is not NULL; Instead the block has no transactions as you can see from visual inspection of the JSON response. (I need to improve the error reporting of my code).
fabricLogger:ERROR: Block 1386 JSON response does not parse
fabricLogger:ERROR: HTTP 'ncode' = '200'
fabricLogger:ERROR: Full HTTP token dump below; Aborting
fabricLogger:ERROR:     sock sock3a8b9b0
fabricLogger:ERROR:     http HTTP/1.1 200 OK
fabricLogger:ERROR:     querylength 0
fabricLogger:ERROR:     -queryblocksize 8192
fabricLogger:ERROR:     state body
fabricLogger:ERROR:     charset iso8859-1
fabricLogger:ERROR:     type application/json
fabricLogger:ERROR:     -binary false
fabricLogger:ERROR:     url http://<IP_ADDRESS>:42278/chain/blocks/1386
fabricLogger:ERROR:     -keepalive 0
fabricLogger:ERROR:     -timeout 0
fabricLogger:ERROR:     socketinfo <IP_ADDRESS>:42278
fabricLogger:ERROR:     queryoffset 0
fabricLogger:ERROR:     binary 1
fabricLogger:ERROR:     -strict 1
fabricLogger:ERROR:     totalsize 329
fabricLogger:ERROR:     -validate 0
fabricLogger:ERROR:     -protocol 1.1
fabricLogger:ERROR:     connection close
fabricLogger:ERROR:     -queryprogress
fabricLogger:ERROR:     -headers
fabricLogger:ERROR:     -blocksize 8192
fabricLogger:ERROR:     coding
fabricLogger:ERROR:     status ok
fabricLogger:ERROR:     body {"stateHash":"8fXq0WPlgjERRs/fhMpz8TIp9vCtHlBH+7y1N8V3ihcD5o3sNNpRdCO6W83tfJv0JycexOObeS5N+jSpG+IeJQ==","previousBlockHash":"j1L3tebPQYlzq4f+ef96t9+ZIn6AXxgWiT6Oe1kOddyvccjgtTcOnzUozkRoqUV9OufApkWhBr3pfs7/Xb8spw==","consensusMetadata":"COoK","nonHashData":{"localLedgerCommitTimestamp":{"seconds":1464022054,"nanos":791764525}}}

fabricLogger:ERROR:     currentsize 329
fabricLogger:ERROR:     meta Access-Control-Allow-Headers {accept, content-type} Access-Control-Allow-Origin * Content-Type application/json Date {Mon, 23 May 2016 16:47:34 GMT} Content-Length 329 Connection close
fabricLogger:ERROR:     -type application/x-www-form-urlencoded

@bcbrock Just wanted to mention this is on the radar, and we are working through it.  The above PR should make it more transparent from a query perspective when a replica is out of date.  Changes to prevent this scenario from occurring are still being actively worked.

@jyellick Thanks

What happens if the replica crashes during state transfer?  On restart, will it reply with empty blocks?  Why does it reply with empty blocks in the first place?  It seems there is some sort of transaction missing in the ledger/state transfer.

In general, state transfer works from the front backwards, ie, you tell it the latest block hash and height, it grabs these, and attempts to go back to its most recent block.  However, if something goes wrong during state transfer, then it might be given a new target hash, and it starts over again.  So, once state transfer 'completes', you can have a blockchain which looks like:
0, 1, 2, 3, _, _, 6, 7, 8, 9, _, _, _, 13, 14, 15
Those are not empty blocks, they are missing blocks, and, generally speaking (with the exception of blocks which contained a deploy transaction), the peer can operate just fine with missing blocks, so long as its current state and lead block are 'current'.  State transfer continues to work in the background, to fill in these gaps, and should eventually catch up.
My strong suspicion is that the ledger, when queried assumes that if the block height is 16, that all blocks below 16 are present, and so probably does not do error checking, and allocates an empty block, shoves the content of block 10 (which is missing) into it, and returns it, and voila, you get your empty block.  This is another case where we probably need the API enhanced to give more sophisticated error reporting. (Though returning an empty block is still unacceptable, and should be fixed)
The other half, and I think the way you are more likely to see this, is the scenario where the network is processing tons of transactions, and then stops.

Replica falls behind, starts eavesdropping for what 'current is'
Tells state transfer, "go to block 20", state transfer starts
Tells state transfer, "block 30 would be fine too"
Tells state transfer, "or block 40"
State transfer comes back and says "Alright, we got to block 20"
PBFT is waiting for the checkpoint for block 50, to tell state transfer to go there, but it never comes
The peer is left with blocks up to 20, and none beyond.

I'm planning to put together a PR so that at after step 5, PBFT will tell state transfer "go to block 40", which should help to more quickly and completely recover.  And the null requests from #1454 would equivalently, though more slowly fix this problem.
And in quick explanation to the natural question of "why does state transfer not simply automatically go to block 40?" the answer is briefly, that for a consensus protocol like PBFT, which has a 'window' of requests, we can recover to anywhere in that window, and still be fine.  So, imagine a different case.

Replica falls behind, starts eavesdropping for what 'current is'
Tells state transfer, "go to block 20", state transfer starts
Tells state transfer, "block 30 would be fine too"
State transfer finishes, and says "Alright, we got to block 20"
PBFT sees that it still has what it needs to make blocks 21-30 in its store, so simply executes those requests and catches up.

@bcbrock After some investigation, it looks to me like this is indeed a consensus problem.  Although the investigation discovered some problems in the ledger access patterns (see #1608) that does not seem to be the root of this issue.  Instead, and the reason is unclear, it seems that we must be invoking Begin/Exec/Commit with no transactions, and empty blocks are being created.  This seems likely to be yet another race, so I'll be focussing to continue the patch series from #1586 and #1595 to eliminate the multi-threading from PBFT and hopefully summarily squash all of these bugs.

@jyelick I checked yesterday to see if maybe a transaction was failing with an error, which might have created an empty block (?), but they all looked OK.

@jyellick Here are errors we're getting now running busywork test stress2s (4-node Sieve)
/home/bcbrock/.busywork/vp3/stderr:15:55:52.860 [consensus/obcpbft] executeVerifySet -> ERRO 1930 Replica 3 executing stale request , this indicates a bug
/home/bcbrock/.busywork/vp3/stderr:15:55:53.856 [peer] handleChat -> ERRO 19d3 Error handling message: Peer FSM cannot handle message (DISC_GET_PEERS) with payload size (0) while in state: created
/home/bcbrock/.busywork/vp3/stderr:15:55:55.042 [consensus/statetransfer] stateThread -> ERRO 1b1c name:"vp3"  has a state for block 6907 which is too far out of date to play forward to block 7309, max deltas are 200, invalidating
/home/bcbrock/.busywork/vp3/stderr:15:55:55.042 [consensus/statetransfer] stateThread -> ERRO 1b1f name:"vp3"  believed its state for block 7309 to be valid, but its hash (8314fb9a88d5e2afa978d46ae19f871b373eb5bdea168e52c1c89f74c320aed9977363194fcc2b39746e272cfa5d39534f9ca79a5abeba0b83275a65a64c491a) did not match the recovered blockchain's (f3a30798e9acde537083383eb837df6fa46d2538c6bd5565dde74f340bccaa868429a0f6fdbaf27dbbf1768fb6199bb67d298b4f6230b1e45c822c2d0469ea41)
/home/bcbrock/.busywork/vp3/stderr:15:55:57.190 [consensus/statetransfer] stateThread -> ERRO 1cd7 name:"vp3"  has a state for block 7349 which is too far out of date to play forward to block 7719, max deltas are 200, invalidating
/home/bcbrock/.busywork/vp3/stderr:15:55:57.190 [consensus/statetransfer] stateThread -> ERRO 1cd9 name:"vp3"  believed its state for block 7719 to be valid, but its hash (dc58abaa239b2308e04b00ee4b981b002a6df5d9d63842323ab3a80e1a8d334ceee9a5b7bb59c7771309ea5d3ec4520e7db45cf5939e4f0da01bb24c01c0cb14) did not match the recovered blockchain's (e0c6e28b2da9c4b4b2cb000723a3f50bd1f85c1a864fe1020413d6f7d212f6ebb7caf4994056bd02b303e4fe37a5b563861d0f3663d7a8725fb95ad4b3bc3e1e)
/home/bcbrock/.busywork/vp3/stderr:15:55:58.856 [peer] handleChat -> ERRO 1d7e Error handling message: Peer FSM cannot handle message (DISC_GET_PEERS) with payload size (0) while in state: created
/home/bcbrock/.busywork/vp3/stderr:15:55:59.310 [consensus/obcpbft] executeVerifySet -> ERRO 1de0 Replica 3 executing stale request , this indicates a bug
/home/bcbrock/.busywork/vp3/stderr:15:55:59.310 [consensus/obcpbft] executeVerifySet -> ERRO 1de1 Replica 3 executing stale request , this indicates a bug
/home/bcbrock/.busywork/vp3/stderr:15:56:03.856 [peer] handleChat -> ERRO 1de2 Error handling message: Peer FSM cannot handle message (DISC_GET_PEERS) with payload size (0) while in state: created

@bcbrock For the next few weeks, the focus is on hardening the batch PBFT implementation, so any Sieve related bugs are likely to be left for later investigation.  I would recommend focusing your testing away from Sieve.

@bcbrock Just as a quick update, thanks to your new busywork integration, I'm now able to recreate this using stress2b.  The two PRs above should partially address this, and after I finish the PR series which removes the race potentials, I'll be investigating to try to find the remaining issues.


RustyCode not detecting PATH correctly

So it looks like RustyCode/VSCode isn't correctly detecting environment variables. Here's evidence that the PATH is set correctly for all tools and when setting the location for racer, it is unable to read the SRC location.
$ echo $PATH
/home/b3nw/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

$ which racer
/home/b3nw/.cargo/bin/racer

$ which rustfmt
/home/b3nw/.cargo/bin/rustfmt

$ which rustsym
/home/b3nw/.cargo/bin/rustsym

$ which cargo
/usr/bin/cargo

$ echo $RUST_SRC_PATH
/home/b3nw/.multirust/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src

These can be executed fine within the integrated terminal but that likely does not matter. I have completely quit VSCode (Check process list for any orphaned node processes too).
I'm at a loss really. I'm using zsh, however this issue persists even if I set PATH in .bashrc too.
Any ideas?

While I'm unable to verify this fixed worked, I did manage to fix the issue. I don't know which of these variables managed to fix it but I'm swaying towards the second.

Specified the absolute paths again rather than relying on the path (This previously didn't work, in fact it didn't work this time until I did the second step).

Ran the node repl (Just type "node" in the terminal).

I believe maybe running the node repl populates something which then picks up the environment variables in the future. shrugs
cc: @Enet4 @chipshorter
Keeping open so this can be explored a little by the devs, feel free to close otherwise.

I'm having trouble understanding your steps. Once the absolute paths are configured, then it no longer relies on environment variables and should definitely work after a window reload. Are you also suggesting to remove the settings made in step 1 after doing step 2?

Enet, it didn't work with absolute paths either, it was only once I ran the node repl that everything started working.

Hi @GTB3NW,
I have fixed your problem in my extension. It is named Rust.


Adding test files to ghas ignore list

Hey, I just made a Pull Request!
Adding a regex for test files that can be ignored by GHAS. To see if this will get rid of the false positives.
:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[ ] All your commits have a Signed-off-by line in the message. (more info)

This is our public fork of Backstage and we have another repo for the internal site.
@DM-sb is it a good idea to disable CodeQL checks on all the test code?

Hi @addersuk It depends really on how many false positives you are seeing. If you are inundated with FP because of test files, I think it makes sense to avoid analysing  them so you can focus on true positives on the rest of your code instead. With static analysis,  it really is a cost-benefit decision imo @ttt23

Yes, I don't see much risks in leaving out test files from the CodeQL scans. Alternatively we can mark out these as "Used in tests". I think Github might learn a pattern from such input. We ask Github to see if the workflow supports patterns of test secrets to ignore (like it does for secret scanning).


Update webactions.md

Fix response in demo

@csantanapr was the intention to show the input arguments to the action?
the way it's written now - thanks @cfjedimaster for pointing this out - it suggests that's the response from the action which is of course wrong.
should we show both? input + output explicitly (two different sections)?

I read this more carefully. The response was correct. But I suggest we clarify this text as suggested in this inline patch:
diff --git a/docs/webactions.md b/docs/webactions.md
index fbffd30..e8a51ff 100644
--- a/docs/webactions.md
+++ b/docs/webactions.md
@@ -109,19 +109,55 @@ Web actions bring some additional features that include:
 5. `Activation via multiple HTTP verbs`: a web action may be invoked via one of four HTTP methods: `GET`, `POST`, `PUT` or `DELETE`.

-The example below briefly sketches how you might use these features in a web action. Given an action `/guest/demo/hello` with the following body:
+The example below briefly sketches how you might use these features in a web action. Consider an action `/guest/demo/hello` with the following body:
 ```javascript
 function main(params) {
-    return { 'response': params};
+    return { response: params };
 }

-and invoking the action with a query parameter name, using the .json extension to indicate a JSON response, and projecting the field /response will yield the following HTTP response:
+
+When this action is invoked as a web action, you can alter the response of the web action by projecting different paths from the result.
+For example, to return the entire object, and see what arguments the action receives:
+
-$ curl https://${APIHOST}/api/v1/experimental/web/guest/demo/hello.json/response?name=Jane
+$ curl https://${APIHOST}/api/v1/experimental/web/guest/demo/hello.json
{
-    "name": "Jane"
+  "response": {
+    "__ow_meta_verb": "get",
+    "__ow_meta_headers": {
+      "host": "<IP_ADDRESS>",
+      "connection": "close",
+      "user-agent": "curl/7.43.0",
+      "accept": "*/*"
+    },
+    "__ow_meta_path": ""
+  }
}
-```
+```
+
+and with a query parameter:
+```bash
+$ curl https://${APIHOST}/api/v1/experimental/web/guest/demo/hello.json?name=Jane
+{
+  "response": {
+    "name": "Jane",
+    "__ow_meta_verb": "get",
+    "__ow_meta_headers": {
+      "host": "<IP_ADDRESS>",
+      "connection": "close",
+      "user-agent": "curl/7.43.0",
+      "accept": "*/*"
+    },
+    "__ow_meta_path": ""
+  }
+}
+```
+
+and to project just the name (as text):
+```bash
+$ curl https://${APIHOST}/api/v1/experimental/web/guest/demo/hello.text/response/name?name=Jane
+Jane
+```

Ah ok. So - do I do anything now? Seems like your change should just take over.

K i'll open a pr.

Superseded by #1934.

I was confuse on what was @cfjedimaster looking to clarify.
I did the example in a way to cover many concepts in one , I agree siting into multiples look like good approach by #1966

@csantanapr the original text had the wrong result for the call that was made - that was the confusion.

It has the correct result give it a try, that's what @rabbah also pointed out
The examples returns all parameters regardless of the query parameter you pass down

Ok - not sure what to say here then. My original PR did fix a bug in the docs, @rabbah agreed. The result was NOT right, I stand by that, but @rabbah has changed the docs to be more appropriate I think and my mod no longer applies. If you really want me to explain why it was wrong, at the time, I can, but it's a moot point.

Looking again - maybe I'm wrong too. But again - moot point, @rabbah has made it better, so let's just go with his stuff. :)

Yes I agree moot point, let's not discuss how I'm right 😜


[Feature]: Custom Off States for Entities

Is your feature request related to a problem?
Some media player devices have more "off" states instead of on/off.  For example, my AppleTV has the state of "standby" when it is actually off or sleeping.  Because of this, the mushroom card still colors the icon for the AppleTV as if the device was still on.
Describe the solution you'd like
Include a configuration option for entity/media mushroom cards that allow you to set custom "off" states per entity so that the card will properly appear on or off.
Describe alternatives you've considered
No response
Additional context
No response

I think it's better to consider standby as inactive for every media player.

I think it's better to consider standby as inactive for every media player.

Sure, that works too

Thank you

Hello, my AppleTV device is still showing as turned on when in standby on the mushroom media card.

No worries, it happens, thank you!


Wayland scanner cannot parse KDE server decoration protocol

Issue found here.
EDIT: Reproduced here:
thread 'main' panicked at 'Ill-formed protocol file: Some(Ok(CData(
    Copyright (C) 2015 Martin Gräßlin

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Lesser General Public License as published by
    the Free Software Foundation, either version 2.1 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Lesser General Public License for more details.

    You should have received a copy of the GNU Lesser General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
  )))', /home/cosm/.cargo/registry/src/github.com-1ecc6299db9ec823/wayland-scanner-0.12.2/src/parse.rs:70:40
note: Run with `RUST_BACKTRACE=1` for a backtrace.

This might be an issue with xml-rs, because I think the XML should be valid.

Ok looks like the issue is here where it expects a bunch of characters but in this case got CData.
Not sure how you want to refactor this, since the macro panics instead of returning an error. I assume doing a next and matching on the result (e.g bypassing the macro entirely?) I'll try to make a patch doing that, but if you have a better solution that would probably be better since you know your code better.

Out of curiosity, how are you using wayland-scanner for wlroot-rs?
I'm wondering because it was never designed to be used without wayland-client/wayland-server.


[tr] Translated chapter1/2.mdx

Translate to Turkish #41

@lewtun , sorry for totally forgetting the toctree! I have just pushed the translation of the relevant chapter details into the file.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


port changes to m97

Approved

master PR : https://github.com/Microsoft/vso-agent-tasks/pull/1360


Fix documented type for the cleanUrl option

Motivation
The documented type was wrong, I think.
Have you read the Contributing Guidelines on pull requests?
Well, I have now.
Test Plan
I must admit I haven't verified it because it's so trivial, but build the docs and visit /docs/en/site-config#cleanurl-string.
Related PRs
-

That doesn't seem to work: https://deploy-preview-1775--docusaurus-preview.netlify.com/docs/en/site-config#cleanurl-string
I'm not sure why though?


[Example] Flutter - Uploading Videos

Hi @colbyfayock ,
I noticed there isn't any example regarding uploading videos to Cloudinary with Flutter. I'd like to contribute by creating an example that demonstrates how to upload videos using the Cloudinary Flutter SDK. Can I work on that?
Looking forward to your feedback!

@aashish-g03 as commented in the other Issue, let's start with 1 at a time. Once you complete that you can work on this too. Thanks for your input!


Adding python-tblib

Saucy doesn't have this package... I think this might break the deb build on packages that depend on this, is there a way to fix it ?

Generally we want all dependencies to be available on target platforms.
Typically for something like this we'd consider importing a backport from a ppa if the dependency is necessary and not available otherwise.
We also plan to turn off Saucy builds since it's reached EOL. Once that has happened builds will work again. But until then we want to keep compatibility.

So this will be mergeable when saucy is dropped ? any ETA ?
I would like to avoid doing a backport for an obsolete distro...

Actually, how about I do a third-party release of  https://github.com/ionelmc/python-tblib ?
I recently found out that the trusty version was too old and didn't work as expected... I might need to use a more recent version, but I am not sure how the dependency will be resolved if there is both a rosdep key, and a rosdistro key...
Doing the thirdparty release would also probably satisfy saucy builds...

Yes a 3rdparty build would satisfy saucy and hopefully get you the needed version.

In the end I might eventually do this a different way...
Anyway I don't need it at the moment.
I ll reopen if needed.


Methods for Evaluating Performance on the Test Set of PDBBind

Thank you for your excellent work!
I am looking for the code to reproduce the results presented in Tables 1 and 2 of the paper. However, I have been unable to locate the script or instructions for this. Could you please provide me with the script or instructions for evaluating the test set of pdbbind? Thank you!

Thank you for raising this issue. We have not committed this part of the code yet. However, the metrics should be pretty straightforward. If you need any help implementing them, feel free to raise specific concerns.


Skype download

Hello,
I have a question from an observation I had. When having Skype session on Franz, the download doesn't work. To download a file (pdf,etc) from a discussion it redirects me to a browser (which mine is Safari) and it downloads instead of the file something else : with the name "original", coming from " https://weu1-api.asm.skype.com/v1/objects/0-weu-d7-712dd4382cb933aa30212409204ba1c3/views/original " .
Can this be resolved by a user such as me or is this a work in progress ?
Desktop :

OS: version 10.14.6
Browser: safari version 13.1
Franz version 5.3.3
bests,
Yannick

Same on OSX 10.15.4 in Franz.


Compare TextDB with Lingpipe

Lingpipe can extract named entity and part of speech from given data. We would like to compare these features with our NLP extractor.

Put lingpipe code under sandbox package.
Comparison between stanford NLP / Lingpipe Do research about the differences, what's lingpipe's advantage versus stanford NLP?
Go through a training process and make a model for Lingpipe (if lingpipe is good).

Standford NLP is open source while Lingpipe is a commercial product.
Standford NLP has higher popularity rate and activity rate than Lingpipe.       https://java.libhunt.com/project/corenlp/vs/lingpipe

Create a wiki page to explain the experience with Lingpipe
Write a README file to introduce how to use Lingpipe

Solved in PR #262


Open to LAN option not working on customized worlds

When loading a world with customized generation the Open to LAN setting will not properly work. The world will load normally but no LAN server will be started. I assume this is due to the warning screen that gets shown when loading a customized world. A current way to get around this issue is to use a mod that removes this warning.

Fixed in version 1.7, thanks for reporting.


New release ready for review: v3.14.10

Please consider if this is ready for the public crowd! If so, push the green button.

Denne var for å fikse warnings i C#


Check for updates to major versions other than the latest major version

@owenjames Here's a screenshot of a solution, let me know if this remedies the problem at hand. We can notify the user about other available versions by a) making the update into a dropdown [like in install], b) adding a badge next to version badge with the number of available versions, and c) amending the microcopy underneath the button to tell the user how many versions they are behind main.
Screenshot:

Loom video:
https://www.loom.com/share/58f73fe082d6438eb48091a1afefae76?sid=75eb0b5b-5cb1-4a66-9b34-4d45f72fa8b3


Bug: back button not working

Type
UI-related issues
Bug description
after using search tag from details page of image
Steps to reproduce

search anything
then try to going back to previous page

it dont let me go to previous page
Android version
11
App version
latest
Screenshots or videos
No response
Solution
if pressed back button ,i should go to previous post
Additional context
No response
Acknowledgements

[X] I have searched the existing issues and this is a new and no duplicate or related to another open issue.
[X] I have written a short but informative title.
[X] I filled out all of the requested information in this issue properly.

Duplicate of #47.
Currently it's not possible to "fix" this due to current data flow design that only focus on single instance search.


Add docker & openshift version collection script

based on https://github.com/openshift/openshift-tools/pull/1650
Adding a script that collects docker and openshift versions and sends them to hawkular & zabbix

Rebased of prod

rebased after #1792 was merged.
@joelddiaz @twiest please review

ok to test

@twiest I made the changes you requested:

Getting docker version using docker api
Getting openshift version using REST

I tested in the container in my environment.
Please note the first commit fixes a bug in hawk_client of supporting unicode metrics and cherry-picked from a PR that should be merged first: https://github.com/openshift/openshift-tools/pull/1830

@twiest changed to get oc version using oc binary.
please approve.


Add regression model to Shiny lesson 016

Run a regression model on bill length vs. bill depth and add the details to the plot via tableOutput or textOutput functions.

closed by 3926e1a


Finetuning Starcoder2 for python copilot

What does this PR do?
This adds a new article on how to finetune starcoder2 for a python copilot
At the moment, please tag @MKhalusova

Hi @Exorust! Thank you for your contribution. We already have a notebook in the cookbook on fine-tuning the StarCoder model on custom code - https://huggingface.co/learn/cookbook/fine_tuning_code_llm_on_single_gpu
How is that notebook that you are adding different from it?


Cant find module/var/task/index?

Tried literally downloading this and put onto lambda and i get this error when trying to test the function:
{
"errorMessage": "Cannot find module '/var/task/index'",
"errorType": "Error",
"stackTrace": [
"Function.Module._load (module.js:276:25)",
"Module.require (module.js:353:17)",
"require (internal/module.js:12:17)"
]
}

I got the same error , and after changing my functions file name to index.js ,
its works fine for me.

I am still getting the same error. My file name is same as mentioned above.

@cristal2431   please ensure you have All the three files,
node_modules
index.js
packaje.json
secondly , please check whether the node_modules have packages.. because i was using other folder that is having readme files...
third , check u are zipping the content of the folder i.e selecting all the three and zipping into one file..
and not zipping the folder.
this is works fine for me.


CAM-KP returning no results on D.4

Should CAM return results in this query?
https://arax.ncats.io/?source=ARS&id=f2886569-5ebb-49c3-aa35-a047d40f6d9b
https://github.com/NCATSTranslator/minihackathons/blob/main/2021-12_demo/workflowD/D.4_tryptophan-kynurenine-with-curie.json

I can't get to the query anymore.


[DOCS] Fix table inconsistencies detected by mdl

This includes a fix in Configurations/unix-Makefile.tmpl to allow
running make md-nits from an out-of-source build tree as well.

Okie, thanks, merging now

Merged
5995dc37197dd85baa749325ba23f5aa8ccbe1f6 [DOCS] Fix table inconsistencies detected by mdl
0c74339c730dc74f9f7aeac866f02c780f39b3a9 Configurations/unix-Makefile.tmpl: Ensure that md-nits always works


index.xsd:143-144: Index.xsd: add xsd:documentation as...

The puzzle<PHONE_NUMBER>0 from #199 has to be resolved:
https://github.com/yegor256/jpeek/blob/dbc5c0552c1636eade98c9e5a05a9c9753bc85a5/src/main/resources/org/jpeek/xsd/index.xsd#L143-L144
The puzzle was created by rultor on 17-Dec-18.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@paulodamaso/z please, pay attention to this issue

Job #282 is now in scope, role is DEV

The job #282 assigned to @thomas-melville/z, here is why; the budget is 30 minutes, see §4; please, read §8 and §9; if the task is not clear, read this and this; there will be a monetary reward for this job

@thomas-melville/z this job was assigned to you 10days ago. It will be taken away from you soon, unless you close it, see §8. Read this and this, please.

The user @thomas-melville/z resigned from #282, please stop working. Reason for job resignation: It is older than 10 days, see §8

Resigned on delay, see §8: -30 point(s) just awarded to @thomas-melville/z

The job #282 assigned to @uurl/z, here is why; the budget is 30 minutes, see §4; please, read §8 and §9; if the task is not clear, read this and this; there will be a monetary reward for this job

Resigned on delay, see §8: -30 point(s) just awarded to @uurl/z

The user @uurl/z resigned from #282, please stop working. Reason for job resignation: It is older than 10 days, see §8

The job #282 assigned to @rajanamistry/z, here is why; the budget is 30 minutes, see §4; please, read §8 and §9; if the task is not clear, read this and this; there will be a monetary reward for this job

The user @rajanamistry/z resigned from #282, please stop working. Reason for job resignation: It is older than 10 days, see §8

The puzzle<PHONE_NUMBER>0 has disappeared from the source code, that's why I closed this issue.

@fabriciofx/z please review this job completed by @gnusinpavel/z, as in §30; the job will be fully closed and all payments will be made when the quality review is completed

The job #282 is now out of scope

@0crat status

@fabriciofx ping, please finish qa review

@0crat status  (here)

@GnusinPavel This is what I know about this job in C7JGJ00DP, as in §32:

The job #282 is not in scope
The job is waiting quality review verdict by @fabriciofx/z for 30days
@gnusinpavel/z will receive 30 minutes after quality review
Job footprint (restricted area)

@0crat quality good

Quality review completed: +4 point(s) just awarded to @fabriciofx/z

Order was finished, quality is "good": +35 point(s) just awarded to @gnusinpavel/z


Learning of Python basics with MongoDB

import pymongo
client = pymongo.MongoClient('mongodb://localhost:27017/')
mydb=client['NewConnection']
try:
conn= client.server_info()
print (f'Connected to MongoDB {conn.get("version")}')
except Exception:
print("Unable to connect to the MongoDB server.")
for db in client.list_databases():
print(db)
information=mydb.information
records5=[{
"EMPNAME":"RAghu",
"Empno":14,
"college":"dwdmuhubli"
},{
"CITY":"DHARAWAD",
"CAMPUS":"KUD"
}]
information.insert_many(records5)

Learning completed with sample code


Changing Padding on a Scene throws off the fog mask

Repro Steps

Create scene with default 0.5 padding.
Add Simplefog mask, do some operations on the mask.
Change scene padding.

Expected: Fog Operations stay positioned where they were on the map.
Actual: Fog shifts unexpectedly.
The sightLayer's fog somehow compensates for this, and I'm not sure how much of a pain it'd be to figure out how to do something like that for simplefog.

Just providing an update on all open issues with Vance's module. This has been taken over by the League of Foundry Developers. This will help prevent lapses in updates. I have taken over as main contributor. All new releases and updates will be managed through the new forked version. We will get the link updated on the Foundry VTT website, but you may want to go ahead and switch. We have already updated and added some new features.
To install this fork, you first will have to uninstall the old module, then install based on this manifest url. You should not lose your fog on maps you have used previously.
https://raw.githubusercontent.com/League-of-Foundry-Developers/simplefog/master/module.json


Integration of REST/API in client #4

Many client fixes:

Labels with white spaces and commas
Defiant
Splitting
Grouping

Repaired qunit tests for client
Many codacy issues have been resolved

@bsekachev , in the past we decided to use "full" version of js libraries because it is easy to debug problems. Now you are adding "min" versions. What is the reason for that? Does our "minimizer" work bad?

@nmanovic
We don't have any minimizer yet, a bundler only. Now I don't see any problems in 3rd party libraries. If bugs occur, than they occur in our code. I believe to use minimized versions of js in production is right way and I am striving to it.

@bsekachev , if we don't have minimiser yet than it is OK. The right way is to use non-minimised version of all libraries and minimise them together with our code. Using minimised libraries is good only if you have a static page and don't have any pipeline to preprocess js. We have the pipeline but in the past we disabled minimiser because it worked incorrectly.

@nmanovic
Okay, then we will use non-minimized versions in future.


app core bundles have crappy groupIds

https://github.com/adobe/adobe-dx/packages/166511
https://github.com/adobe/adobe-dx/packages/166514

this is has been fixed part of #121


Add Lending API

We need to add Lending api (same as staking)
We currently needs to support Compound for lending.
Also there is some research about other lending providers that can be integrated.

It is still issue for open discussion, we need specification for clients and action flow for lending

@catenocrypt can you take it after  current issue? What do you think?

I suggest:
https://aave.com/
https://compound.finance/
https://iearn.finance/earn
https://idle.finance/#/dashboard/best

Lending API prototype is being worked on here: #1116 .

Not relevant anymore


NORMALMAP doesn't work if the mesh has no UVs

Godot 3.0.2
I ran in circles for about half an hour only to realize, that Godot is probably ignoring NORMALMAP in shaders if the mesh has no UVs...
Which is the case in my terrain plugin, since I get my UVs from VERTEX instead.

Is Godot doing such a thing even with ShaderMaterials, and why?

It needs the UV in order to know what pixel to sample in your normal map,
so yeah, no UV, no normal map.
You also need to calculate tangent, binormal and normals in your vertex
shader.
Without any of those Godot won’t be able to apply your normals. Have a look
at my terrain shader on github. I’m on my phone right now so don’t have the
url handy but google BastiaanOlij and terrain editor and you’ll find it
Cheers,
Bas
On Mon, 28 May 2018 at 12:18 pm, Marc<EMAIL_ADDRESS>wrote:

Godot 3.0.2
I ran in circles for about half an hour only to realize, that Godot is
probably ignoring NORMALMAP in shaders if the mesh has no UVs...
Which is the case in my terrain plugin, since I get my UVs from VERTEX
instead:
[image: image]
https://user-images.githubusercontent.com/1311555/40594191-ee7b1aca-622d-11e8-9246-df0f9701345e.png
[image: image]
https://user-images.githubusercontent.com/1311555/40594198-050bc924-622e-11e8-9621-b78f0f7dc227.png
Is Godot doing such a thing even with ShaderMaterials, and why?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/19212, or mute the thread
https://github.com/notifications/unsubscribe-auth/AB2vabzgvDM2KrVVIS_rqTh-hO6G-cPeks5t215rgaJpZM4UPiev
.

--
Kindest regards,
Bastiaan Olij
https://www.facebook.com/bastiaan.olij
https://twitter.com/mux213
https://www.youtube.com/channel/UCrbLJYzJjDf2p-vJC011lYw
https://github.com/BastiaanOlij

Ah, my phone is showing images now:) yeah your just missing binormal,
tangent and normal calculations. Normal mapping doesn’t just work with flat
planes so the normal from the normal map is always adjusted by the normal
of the plane it’s applied to, even if that is overkill for terrain.
If you’re normalmap is already adjusted to the slopes of your terrain you
can probably get away with adding:
NORMAL = vec3(0.0, 1.0, 0.0);
BINORMAL = vec3(1.0, 0.0, 0.0);
TANGENT = vec3(0.0, 0.0, 1.0);
To your vertex shader.
I might have swapped the tangent and binormal and one of those might need
to be negative 1:)
On Mon, 28 May 2018 at 12:18 pm, Marc<EMAIL_ADDRESS>wrote:

Godot 3.0.2
I ran in circles for about half an hour only to realize, that Godot is
probably ignoring NORMALMAP in shaders if the mesh has no UVs...
Which is the case in my terrain plugin, since I get my UVs from VERTEX
instead:
[image: image]
https://user-images.githubusercontent.com/1311555/40594191-ee7b1aca-622d-11e8-9246-df0f9701345e.png
[image: image]
https://user-images.githubusercontent.com/1311555/40594198-050bc924-622e-11e8-9621-b78f0f7dc227.png
Is Godot doing such a thing even with ShaderMaterials, and why?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/19212, or mute the thread
https://github.com/notifications/unsubscribe-auth/AB2vabzgvDM2KrVVIS_rqTh-hO6G-cPeks5t215rgaJpZM4UPiev
.

--
Kindest regards,
Bastiaan Olij
https://www.facebook.com/bastiaan.olij
https://twitter.com/mux213
https://www.youtube.com/channel/UCrbLJYzJjDf2p-vJC011lYw
https://github.com/BastiaanOlij

It needs the UV in order to know what pixel to sample in your normal map, so yeah, no UV, no normal map.

But I sample the pixel already. I'm just not using UV to do it, and I was using NORMAL before without any problem.

You also need to calculate tangent, binormal and normals in your vertex shader.

So far I've been computing normalmaps myself, and I didn't need tangents or binormals either.
But now I'm trying to use NORMALMAP because I believe it can help simplifying my shader.
I'll try to compute these anyways to see if something comes up...

@BastiaanOlij  thing is, I need the same normals in the vertex AND fragment shader, because of LOD, so I cannot get rid of the normalmap (the terrain one). What I do then, is to transform those normals according to the transform of the terrain, so that it shades correctly when scaled.
I understand it's needed on classic 3D models, but with a terrain based on textures I try to avoid fetches as much as I can, because the impact is often huge. I'm not keen on adding 4 more texture fetches in the vertex shader...
I'll eventually do that if there if I can't find a faster solution for a heightmap, then...

Yup, like I said, it's overkill :) I need it because I'm blending terrains including the normal map from the terrain so my normal maps aren't directly usable without adjusting to the orientation of the faces.
You can always do the necessary conversion on the normal map and use it directly, the problem is that I'm not sure if a normal in the normal map of (0.0, 1.0, 0.0) will be up but in theory the following should work if that is the case:
NORMAL = texture(u_ground_bump_0, ground_bump, grounduv).rgb * 2.0 - 1.0;

It's very likely that (0.0, 0.0, -1.0) is deemed to point outwards of the plane in which case you'll need to transpose it.
Unless the normal is a two channel normal map but I think Godot converts that automatically (seeing all normals are unit length you can get away with storing just the x and z and calculating y with pythagoras and save memory).
Did you try doing:
NORMAL = vec3(0.0, 1.0, 0.0);
BINORMAL = vec3(1.0, 0.0, 0.0);
TANGENT = vec3(0.0, 0.0, 1.0);

in your vertex shader? should result in Godot doing the right conversion on the normal map in theory if I understand correctly how you're building your normal map.

So I ended up with something that appears to work:
I simply keep the existing shader, but instead of using WORLD_MATRIX to transform normals, I multiply my normals by basis.inverse().transposed() (calculated in GDScript and sent with a uniform). After doing this both in vertex and fragment shaders for terrain normals AND normal maps, I got it working with everything in NORMAL.
So in the end I'm still not using NORMALMAP and still not calculating BINORMAL and TANGENT... I'm not sure what this implies even though the result looks correct.

Sounds good to me. Basically you're just skipping a few steps. The NORMAL/BINORMAL/TANGENT simply enables the shader to create the correct rotation matrix by which to re-orientate the vector it obtains from your normal map.
If you're doing that rotation yourself, well there you go:)
Yeah I ran into mat3 support being a little weird too. I had to use it when I implemented the panoramic stereoscopic sky, can't remember what the trick was though. It does work.

@BastiaanOlij actually rotating isn't that trivial, I still need at least two other samples in the heightmap to get it right...

This is not a bug, you can't use normal mapping (NORMALMAP) without tangent space, which obviously includes UV. Write to NORMAL instead.