How can I help?

Hi Dimitri,
I've just discovered Solid and followed the trail here.  I've been looking into Ethereum, Akasha, ceptr, IPFS.io and a bunch of other distributed technologies and found Solid.  After figuring out that the databox.me WebID uses the certificate method that now doesn't work due to changes in the browsers I found OIDC and I'd really like to help out.
Cheers
Phil

@philipbeadle thanks for reaching out!
One thing that would be helpful is - try out the OIDC stack (write a small Solid application with it), and let us know if there are any questions or problems.

You can find a solid-client OIDC build over at https://github.com/solid/releases
Instructions for auth-related functions are available in the readme of https://github.com/solid/solid-auth-oidc
Use https://solidtest.space as a server, or you can install the dz_oidc branch of solid-server (don't forget to run the init cli command)

@philipbeadle Also, there's a sample OIDC Auth app at https://github.com/solid/solid-auth-oidc/blob/master/demo/index.html
You can either run it directly at github.io at https://solid.github.io/solid-auth-oidc/demo/
or npm install http-server, check out a copy of the solid-auth-oidc repo, and http-server demo/.

Ok, got some of this working.  Ive got the solid server running on localhost:8443 and the sample app on 8080.
When I enter my localhost address I get this
'''
solid-client.min.js:80 Getting provider from default popup UI
(index):90 onLoginSuccess() callback! webId:  undefined
solid-client.min.js:80 Auth client received event:  MessageEvent {isTrusted: true, data: Object, origin: "http://<IP_ADDRESS>:8080", lastEventId: "", source: Window…}
solid-client.min.js:80 Provider selected:  https://localhost:8443/
solid-client.min.js:80 Registering public client...
solid-client.min.js:80 GET https://localhost:8443//.well-known/openid-configuration 401 (Unauthorized)
value @ solid-client.min.js:80
(anonymous) @ solid-client.min.js:80
solid-client.min.js:80 Error in rp.discover() while fetching provider config
(anonymous) @ solid-client.min.js:80
solid-client.min.js:80 Error in RelyingParty.register(): SyntaxError: Unexpected token A in JSON at position 0
(anonymous) @ solid-client.min.js:80
solid-client.min.js:80 Uncaught (in promise) TypeError: Cannot read property 'serialize' of undefined
at e.value (solid-client.min.js:80)
at solid-client.min.js:80
'''
The local app logs in with my solidtest.space webid
I'm using the dz_oidc branch too.  Any clues as to what's going wrong here?

@philipbeadle thanks! Looking into it. (ping me on Gitter when you have a chance, I have some questions)

@philipbeadle I suspect the issue might be - do you have "auth": "oidc" enabled in your configuration? A 401 Unauthorized as a response to GET https://localhost:8443/.well-known/openid-configuration suggests the server is running in webid-tls mode.

I opened an issue for this, https://github.com/solid/solid-auth-oidc/issues/5

@dmitrizagidulin  Ive got this as my config
{   "root": "/Users/philipbeadle/Projects/SoLiD",   "port": "8443",   "serverUri": "https://localhost:8443",   "webid": true,   "mount": "/",   "sslKey": "/Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem",   "sslCert": "/Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem",   "idp": false,   "proxy": "/proxy",   "auth": "oidc" }
I added the "auth": "oidc" and restarted the server and now when I go to https://localhost:8443/ I get
Access to https://localhost:8443/ requires authorization (401), even if i now remove that line.
I'm in Melbourne Aus UTC+10 lets organise a gitter chat soon.

Ok, that's closer. (the 401 Unauthorized response is expected when you access / when not logged in).
Have you gone through the node bin/solid.js init setup wizard? I'm not seeing the configPath and dbPath settings.

Yep, just ran it again to make sure. There was no question about dbpath or config path.

@philipbeadle It looks like there's some confusion between a globally installed Solid server, and a locally installed one. Running solid init most likely hits the globally installed one (which is why you're not seeing the config db questions). If running the local init gives you that error, it looks like you may have forgot to run npm install locally.

Ok, fixed that with npm install however there is no change.
Philips-MBP-2:node-solid-server philipbeadle$ node bin/solid.js init
? Path to the folder you want to serve. Default is /Users/philipbeadle/Projects/SoLiD/node-solid-server
? SSL port to run on. Default is 8443
? Solid server uri (with protocol, hostname and port) https://localhost:8443
? Enable WebID authentication Yes
? Serve Solid on URL path /
? Path to the SSL private key in PEM format /Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem
? Path to the SSL certificate key in PEM format /Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem
? Enable multi-user mode (users can sign up for accounts) No
? Do you want to have a CORS proxy endpoint? Yes
? Serve proxy on path /proxy
? Do you want to set up an email service? No
config created on /Users/philipbeadle/Projects/SoLiD/node-solid-server/config.json
Still creates the same config.
{
"root": "/Users/philipbeadle/Projects/SoLiD/node-solid-server",
"port": "8443",
"serverUri": "https://localhost:8443",
"webid": true,
"mount": "/",
"sslKey": "/Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem",
"sslCert": "/Users/philipbeadle/Projects/SoLiD/certificate/localhost.pem",
"idp": false,
"proxy": "/proxy"
}

So that still looks like an earlier version, judging by the config wizard. Can you do a git fetch origin / git checkout dz_oidc / git rebase origin/dz_oidc?

Ok got the node server working with OIDC, but now theres no demo folder in this repo to try it out.  Is there another demo now?

The demo folder is in another repo, https://github.com/solid/solid-auth-oidc
(The names are very similar -- this one is a server-side auth manager, and that one is the client-side).

Ah thanx :)  Still trying to get my head around all this.  Although it may look like it, i'm not a noob :)  My main claim to fame is http://www.dnnplatform.com which I built with a bunch of others.  I'd really like to get solid.social up and running with the OIDC auth but cant figure out what the user object is supposed to look like that gets returned in the HEADERS in the current code.  Are you familiar with that project at all?

DNN looks pretty cool (and no, no worries, this branch definitely still needs more documentation/explanations)!
I'm not quite familiar with solid.social, I know it's Melvin's project and I've seen screenshots, but haven't looked at the code. If it's still using WebID-TLS auth, then I think you're referring to the User: header, yeah? That should just be a WebID URI string. Also, lets take the conversation to gitter, so as not to spam this thread.

Closing - let me know if you have any other questions.


Fixed default OpenSsl cipher suites

Motivation:
The default enabled cipher suites of the OpenSsl engine are not set to
SslUtils#DEFAULT_CIPHER_SUITES. Instead all available cipher suites are
enabled. This should happen only as a fallback.
Modifications:
Moved the line in the static initializer in OpenSsl which adds the
SslUtils#DEFAULT_CIPHER_SUITES to the default enabled cipher suites up
before the fallback.
Result:
The default enabled cipher suites of the OpenSsl engine are set to the
available ones of the SslUtils#DEFAULT_CIPHER_SUITES.
The default enabled cipher suites of the OpenSsl engine are only set to
all available cipher suites if no one of the
SslUtils#DEFAULT_CIPHER_SUITES is supported.

@SgtSilvio good catch!
Can you sign our ICLA and maybe also add a unit test:
https://netty.io/s/icla

cc @rkapsi @slandelle @CodingFabian

Nice catch, +1 for unit test.

Cherry-picked into 4.1(3c8f4b81d7c5dd7ac0c221585623ccf5346380cd) and 4.0 (6583b5271c)

Also added a test-case.

Thank you for your quick response, sorry that I did not find time for writing the unit test myself.

@SgtSilvio no worries... thanks for the fix :)

oof ... good catch!


[Bug]: BigQueryIO getFailedInserts fails when using Storage APIs

What happened?
Hello,
I am having issues when using getFailedInserts / getFailedInsertsWithErr from the WriteResult of a BigQueryIO write using methods STORAGE_WRITE_API or STORAGE_API_AT_LEAST_ONCE.
The message leads us to think there's something wrong with the usage of the API, but there's no combination that can make it work:
.getFailedInserts():
Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInserts as this WriteResult uses extended errors information. Use getFailedInsertsWithErr instead
  at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
  at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInserts(WriteResult.java:167)

.getFailedInsertsWithErr():
Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInsertsWithErr as this WriteResult does not use extended errors. Use getFailedInserts instead
  at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
  at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInsertsWithErr(WriteResult.java:185)

.withExtendedErrorInfo().getFailedInsertsWithErr():
Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInsertsWithErr as this WriteResult does not use extended errors. Use getFailedInserts instead
  at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
  at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInsertsWithErr(WriteResult.java:185)

.withExtendedErrorInfo().getFailedInserts():
Exception in thread "main" java.lang.IllegalArgumentException: Cannot use getFailedInserts as this WriteResult uses extended errors information. Use getFailedInsertsWithErr instead
  at org.apache.beam.sdk.util.Preconditions.checkArgumentNotNull(Preconditions.java:68)
  at org.apache.beam.sdk.io.gcp.bigquery.WriteResult.getFailedInserts(WriteResult.java:167)

It appears that they are mutually exclusive, but upon debugging, both failedInserts and failedInsertsWithErr are initialized as null.
This error started after upgrading from 2.38.0 to 2.40.0, and I couldn't find a workaround.
I'm trying to get more details on what's causing it and will update here if I see anything interesting.
Issue Priority
Priority: 1
Issue Component
Component: io-java-gcp

Ok, I found out that there's a specific tag/PCollection for Storage API, which can be obtained through getFailedStorageApiInserts.
Not closing the issue because I feel there's room for improvement on the error message (as there are more exit points for errors now).
Also, it would be good to have it listed as a breaking change -- I am not sure if it is still possible.


Implement colors from figma in _colors.scss

Colors

https://www.figma.com/design/bMs7rEHGfr5h0ZfOX36wHU/PokemonApp?node-id=166-118&node-type=frame&t=hrbB9O3k97aVdZnO-0


I don't really know if this is a issue but it is for me, that it doesn't have support for picom for animations and round corners

Doesn't have support for picom for animations and round corners

Pretty sure picom is an X11 compositor, so it is not relevant for macOS?

wait you can use picom on Mac OS???

wait you can use picom on Mac OS???

Nop. See #148 for animations.


Connection behind a proxy

Since 8.0, due urlliib3 change, telepot fails to connect when this connection is through a proxy. With telepot 7.1, no probems.
Regards

I think you have to change the underlying class with which requests are made. By default, I use PoolManager. To go through a proxy, you need ProxyManager.
import urllib3
import telepot.api

proxy_url = 'http://localhost:3128/'

telepot.api._pools = {
    'default': urllib3.ProxyManager(proxy_url=proxy_url, num_pools=3, maxsize=10, retries=False, timeout=30),
}

telepot.api._onetime_pool_spec = (urllib3.ProxyManager, dict(proxy_url=proxy_url, num_pools=1, maxsize=1, retries=False, timeout=30))

You may also need to put in a custom header. Read urllib3 docs for details.
I have not tested the above code because I don't have a proxy, but it should not be far off.

Yesterday I tried "play" with urllib3 with no success... I will try your suggestion and I´ll tell you...
Thank you!!!

Perfect!!!! The connection was successful!.
If the proxy has autentication, then:
proxy_url='http://user:pass@proxyurl:port'
Thousand of thanks for your quickly support and solution!!!!

No problem :smile:

just in case
for aio version of bot you can use
`proxy_url = "http://proxy.server:3128"
telepot.aio.api._pools = {
'default': aiohttp.ProxyConnector(proxy=proxy_url, limit=10)
}
telepot.aio.api._onetime_pool_spec = (aiohttp.ProxyConnector, dict(proxy=proxy_url, force_close=True))
`
it works fine on pythonanywhere

Hi guys,
I'm having troubles making proxy run with latest versions of aiohttp. That's because aiohttp.ProxyConnector has been deprecated.
Any suggestions?


Update puppet.ts

Implement toContact() method

When I add one PR link in Wechaty issue, it will generate an issue link here. Maybe I have not understand what u mean exactly.
I have created a new method messageContact() in Puppet which it doesn't have before, and then PuppetTest extends Puppet, maybe it's the reason that CI can not pass.

@huan

When I add one PR link in the Wechaty issue, it will generate an issue link here.

We need to do the reverse operating as you described above:
I mean we should add the Issue link to your PR here, then this PR will generate a PR link under the Issue over there (in Wechaty repo).

I have created a new method messageContact() in Puppet which it doesn't have before, and then PuppetTest extends Puppet, maybe it's the reason that CI can not pass.

Yes, you are right. Please fix it.

Get it, thank you very much.

@windmemory No, you should not approve this PR because it had not passed the CI test.
Please make sure the CI turns green before you approve any PR for the Wechaty project, thank you very much.

@su-chang You are welcome.
Please follow my reviews in the previous post: they are action requirements, which need us to take action over them before we can continue the discussion.

I have implemented toContact() method both in puppet.ts and puppet.spec.ts already, and I can run toContact() method successfully in local env. But the CI still thinks that I have not implemented it, I do not know where the bug is.
Maybe we should to make CI try again... How do u think about it? @huan

Related issues:
https://github.com/Chatie/wechaty/issues/1627
https://github.com/Chatie/wechaty/issues/1855

Thanks for the fix, the CI passed now!
All code looks good to me. Could you please bump the patch version up, so that we can get a new version of the NPM package after this PR was merged?
I'll merge this PR after you bumped the version up. Cheers!

@windmemory You are 6 seconds behind me... ;-D

@huan after you :P

BTW: It would be better if we can have a more meaningful subject for this PR.
For example: Update puppet.ts -> Implement toContact() method

@su-chang Thank you very much for enhance the puppet abstract class!


split train/test by instrument

Currently, when splitting data intro train/test, we sample from each year, so profiles are split in the same proportion for each year. We do not take instrument into account when doing the split. We should aim to also sample from each instrument. SWo the function should iterate through each year/instrument value pair, and sample n profiles based on the train/test fractions.

This has now been implemented.

This has now been implemented in the dataset class and demomnstrated in the decision tree notebook, so closing the issue.


🧪  test: add help test

tests

添加 help test

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.9%) to 1.294%

Totals

Change from base Build<PHONE_NUMBER>1:
0.9%

Covered Lines:
22

Relevant Lines:
1700

💛  - Coveralls


How to get the ATT estimate from the eventstudyinteract command ?

How to get the ATT estimate from the eventstudyinteract command in STATA.

You can find an example in the updated help page as well as this past issue.


[T5A3][W10-B2] Alex Fong Jie Wen

Accepting this PR without a review because another of your PRs for this tutorial was reviewed in detail.


API: Implements RoutingService

This PR implements two new gRPC APIs according to the proposals in https://github.com/v2fly/v2ray-core/issues/105:

Subscribe to routing statistics
Test route with manually constructed routing context

The APIs are implemented as a new service "RoutingService", where all the code is put in a new package app/router/command.
NOTE: All the file changes in this PR is only scoped in app/router/command, so this PR just implements the core service support. Routing service currently isn't exposed to user through JSON config, nor will any app actually publishs routing statstics to routing service. These features will be implemented in next PR.

Build failed occurred again in FreeBSD like this https://github.com/v2fly/v2ray-core/pull/181

@Loyalsoldier I figured out that I forgot to place "// +build !confonly" in one go file, so this file will be included during build of v2ctl, and will fail when importing a module of confonly.

The Stats Channel needs some more refactoring for a cleaner Routing Service implementation, so this PR would still keep as a draft.
Btw, @Loyalsoldier @Robot-DaneelOlivaw @kslr, regarding the name of the Test route manually API, which one do you prefer?

GetRoute
TestRoute

TestRoute

Interested in adding  addroute and delroute ? Just like adding or deleting users, for example, the following three blocked tags are added to the routing rules in real time and take effect.
[tag:"blocked" protocol:"bittorrent" tag:"blocked"<EMAIL_ADDRESS>tag:"blocked" source_geoip:{cidr:{ip:":\xf1\xd6\x23" prefix:32}}]

@ifworldok Currently, Routing app does not provide an elegant way to manipulate its rules after initialization. The routing has a long way to go with much refactoring works, and at some point when related interfaces are ready, these rule-related APIs would be implemented then.
@kslr Do you think https://github.com/v2ray/v2ray-core/issues/2024 should be reopen and transferred to v2fly? (Is it possible for issues to be trasferred between organizations?)

@ifworldok Currently, Routing app does not provide an elegant way to manipulate its rules after initialization. The routing has a long way to go with much refactoring works, and at some point when related interfaces are ready, these rule-related APIs would be implemented then.
@kslr Do you think v2ray#2024 should be reopen and transferred to v2fly? (Is it possible for issues to be trasferred between organizations?)

Cannot cross team


Tooltip not always points to the center of the component

When we have a truncated element, we could centralize the tooltip arrow based on the width of the visible part of the element.

Another example of this, a more subtle one:


Added repagh as a maintainer.

Checklist

[ ] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

pypi is returning a 504 for https://pypi.io/packages/source/s/slycot/slycot-0.3.3.tar.gz


match references to ambient variables

Sometimes we want to refer to the contents of a dynamic variable during a match (whether locally or externally defined).  It would be useful if there was support for this without having to resort to guards.
e.g.
eqTest x y d = match x y with
| z z -> "both are equal"
| _ d -> "at least y is equal to the given d"
| _ _ -> "who knows?"

This example involves a non-linear match which can be desugared by using a fresh variable and a guard with equality predicate. Similarly it would be useful to allow a named constant in a match:
one = 1
match x with | one -> ...

fails because "one" not a pattern, in particular, it isn't a constructor. Felix allows
match x with | $(expr) ->

which desugars to a pattern variable with an equality predicate. I think this is slightly different to a non-linear match (where the pattern variable is repeated).


Use Nginx reverse proxy server for the Application ?

This way, people can use SSL certificates in the Nginx config for HTTPS. We could actually provide an automated way to get LetsEncrypt certificates too, as an option.

Okay, looks like you have mentioned it even in the planned features

@karuppiah7890 There's some dev overhead at present required to deploy this app. We've tried to minimise this with EC2 instances with elastic IPs, which use TLS by default. However your point is valid, we can add Nginx to our docker-compose setup & expose the resulting port.
My only concern is that I think we'll inevitably need separate guides for separate services. Adding SSL in a dockerised container to this process will not necessarily make this easier, unless we can use LetsEncrypt with a lib such as Certbot run as a cron job to automate this. All a little speculative at this point.

Yeah, I agree. We need to do quite some automation. Also, actually, there are some more ways to use reverse proxies and get SSL certificates or use own certificates and also prevent from DDoS. For example, people could use a cloud service like cloudflare for this.


Issue Reporter

This is a proposal for an issue reporter feature. Thorium Classic had one which posted issues directly to Github, which was really nice.

A few things I don't like about it:

These issues sometimes require discussions with the OP, but if they only provide their name it makes it hard to follow up. Email is better, but then the question becomes "Who will do the following up?" It's much better if the whole conversation can happen in Github.
The body field supports markdown. It'd be great if we had some kind of rich text options, much like the Github comment field.
 Frankly, I don't care what priority the issue submitter thinks it is. The developers working on Thorium Nova should decide the priority of things.
Attachment uploads never worked. It should be possible to upload multiple attachments.

Describe the solution you'd like

For handling discussions, let's literally move it to Github. This adds some constraints: In order to submit an issue report in-app, the user must first sign in to Github. Which means they need a Github account. Not ideal, but it's the default I'm going to go with when I eventually implement this. I would love other suggestions for how to address this.
The body field can use the same input field that I built for the admin pages on Thoriumsim.com. It's really nice, you'll like it, I promise.
Remove the priority field. Easy peasy.
Attachments can also work the same way as attachments for the Thoriumsim.com input field. I think it only supports images, but we can look into supporting other file types later.

As for implementation, I think it makes sense for the issue form to be tied directly to Thoriumsim.com. So when the form is submitted, it sends it to Thoriumsim.com, which then sends it to Github.com.
Alternative Proposal
Instead of sending the issue directly to Github, we build out an issue tracker directly in Thoriumsim.com which links back to Github. I actually started building something like this last year.
This might address the discussion problem, since folks could be required to sign in with their Thoriumsim.com account when submitting issues, which makes way more sense than signing in with a Github account. Discussion happens on Thoriumsim.com, and then once the issue is understood well enough, a Github issue can be created and triaged.

An issue submission mechanism inside Thorium Nova is an excellent idea. It should hide the complexities of GitHub.
I suggest making a contact method required. It could be email, it could be Discord name. By "required" I don't necessarily mean enforced on the submission form. Perhaps a note stating that including a contact method significantly improves the chance of action being taken on the issue reported.
Yes, remove priority from the submission form. Priority can still be set in GitHub by the contribution team, right?
Please define OP
Owner/Proprietor?
Original Poster? <-- I assume something like this
Attachments would be nice, but anything is better than nothing

I suggest making a contact method required. It could be email, it could be Discord name.

In the issue, I mentioned that

Email is better, but then the question becomes "Who will do the following up?"

Having them share their email or Discord means that someone has to get in touch with them to get the follow-up information. I want this to be a community project, where everyone and anyone can take care of tasks like following up. So who does it? How do we decide who does it? and how do we make sure that follow-up information is made available to everyone else so they can help with addressing the issue?
Not an insurmountable problem, but certainly one that requires a bit of coordination on the part of the community. I'd much rather force issue writers to sign in with a Github account and take advantage of the great notification and commenting features that Github already provides. Bonus: By encouraging them to sign up, they are one step closer to becoming more involved contributors. 😉

Priority can still be set in GitHub by the contribution team, right?

Yep, anyone with contribution access to the repo can change the priority setting on an issue. (If anyone doesn't have contribution access, file an issue or ask on Discord and we'll send you an invite to the Thorium-Sim org.)
OP = Original Poster. The person who authored the issue. I've updated the description to remove the "OP" term.

since folks could be required to sign in with their Thoriumsim.com account when submitting issues,

I actually think this is the ideal approach, for a number of reasons:

Folks will hopefully be signing into their Thorium account anyway for tracking their flights, downloading and submitting plugins. See more about what that account does here: https://github.com/Thorium-Sim/thorium-nova/discussions/21#user-content-thorium-accounts
We can make it so they can create a Thorium account from within Thorium Nova.
Make it possible to connect a Thorium account to Github - ideally this also can happen within Thorium Nova.
Only allow folks to submit issues if they've connected their Github account.
No discussion or storing the issues on ThoriumSim.com - it just proxies the submitted issue to Github.

Anyone have any thoughts about that?


Align logo, edit PR template

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[ ] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[ ] Is your initial contribution a single, squashed commit?

[X] Does gradlew build run cleanly?

[X] Have you written or updated unit tests to verify your changes?

[X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

@pivotal-amurmann @upthewaterspout could you possibly have a look at this.

Hi @upthewaterspout , and thanks for the question. Sorry, missed this in my inbox. I commented out, not removed, these sections, so that the guidance appears in the PR when contributors write the content for the PR description, but that the content doesn't appear when displaying after the PR has published. The content is still available when re-editing the PR.

@upthewaterspout have your concerns now been addressed?  If so please change your -1 to a +1 and merge this, since submitter is not a committer.  If not, please restate what still needs to be changed.


wasm support

I'm looking for a socket io client crate that works on both wasm and native. Is this something you'd want to add?

What exactly do you mean by working with Wasm? Right now you could compile the crate to Wasm using Emscripten's runtime: cargo build --target wasm32-unknown-emscripten. What you mean is probably interoperability with JS via wasm_bindgen. The problem here is, that you then usually compile to the target wasm32-unknown-unknown, which doesn't compile with stdio things like threads and networking (stuff that this crate uses extensively). A possible solution would be to add #![cfg(target_arch = "wasm32")] flags to those functions and sub it with the bindings from web-sys. I generally would like to see that this crate works with Wasm, also because I think that this would be a use case.

Yes, the use case I have is that I want to be able to run the same code without modifications both in the browser and as a native application.

Ups sorry! I think in order to get this started one could look at how reqwest (the http library used here) implemented Wasm support (mainly in src/wasm or src/lib.rs).  We would also need to somehow factor out the thread that starts the polling cycle and threads that start the callback (currently through the spawn_scoped! macro). Async support before this (see Roadmap.md) would come in handy because async is supported by the browser. A possible issue I see is, that the websocket library used (websocket) doesn't support the wasm32 target for now. But we could first start with polling support.

If looked further into this and WASI could be an option as well. WASI provides sort of a normal standard library for WebAssembly and is both supported by the browser and the most common runtimes. It is accessible via the wasm32-wasi target. It provides a std implementation, but yields unimplemented!() calls on the functions unavailable in the browser.

A build error related to openssl is found while I am trying to use it in an app built on wasm and rust.

Yes, this is not yet implemented. There is no support for the WASM target yet.

Yes, this is not yet implemented. There is no support for the WASM target yet.

Got it :)  You have aldeady done great job.
It 's nice to see more and more frontend frameworks wrintten in rust.

any update on this? i would like to use socketio client within leptos but got the same error with wasm

I am ok if only can compile to wasm only so I can use it with my browser extension.
currently getting error for getrandom crate when compiling with wasm-pack.

wants to be added js feature

WASM support would be excellent, it's definitely a nice use case. Rust end to end is powerful. It's been a few years since this ticket was created, I wonder what the limitations are now?
I'm not sure where to start if I were to help with this


How to consider impedance control and position control at the same time?

I'm conducting some grasping experiments where the gripper may contact the desk a little. But I've noticed that in the grasping process, after the gripper closed, the program just stuck and finally aborted (core dumped).
my code is like:
self.panda.move_to_joint_position(pre_grasp_joints, speed_factor=self.move_speed)
time.sleep(0.03)

self.panda.move_to_joint_position(grasp_joints, speed_factor=self.move_speed)
self.gripper.move(width=0.0, speed=self.gripper_speed)
time.sleep(0.03)

self.panda.move_to_joint_position(after_grasp_joints, speed_factor=self.move_speed)
self.gripper.move(width=self.max_gripper_width, speed=self.gripper_speed)
time.sleep(0.03)

here is the output:
....(previous ignored)
INFO - 2023-11-08 22:57:14,710 - grasp_panda - Stopping active controller (Trajectory).
INFO - 2023-11-08 22:57:14,712 - grasp_panda - Initializing motion generation (moveToJointPosition).
INFO - 2023-11-08 22:57:14,724 - grasp_panda - Computed joint trajectory: 1 waypoint, duration 5.72 seconds.
INFO - 2023-11-08 22:57:14,724 - grasp_panda - Starting new controller (Trajectory).
INFO - 2023-11-08 22:57:20,850 - grasp_panda - Stopping active controller (Trajectory).
INFO - 2023-11-08 22:57:20,853 - grasp_panda - Initializing motion generation (moveToJointPosition).
INFO - 2023-11-08 22:57:20,860 - grasp_panda - Computed joint trajectory: 1 waypoint, duration 2.17 seconds.
INFO - 2023-11-08 22:57:20,861 - grasp_panda - Starting new controller (Trajectory).
  File "grasp_panda.py", line 136, in _execute_pick_and_place
    self.gripper.move(width=0.0, speed=self.gripper_speed)
RuntimeError: libfranka gripper: Command failed!
INFO - 2023-11-08 22:59:13,746 - core - signal_shutdown [atexit]
Aborted (core dumped)

I also used another package (named frankapy and franka-interface), its goto_pose function will consider impedance as default. But there might be bugs in that repository because the trajectory it planned always met joint limits in the way. So I refer to your repo and the trajectory is very impressive with no similar "joint limit" status.
Besides, how can we add impedance control into position control?

Sorry to bother, I found I should use gripper.grasp() function to grasp something, instead of move(width=0.0)

Sorry to bother, I found I should use gripper.grasp() function to grasp something, instead of move(width=0.0). Now it will not jump into core dump! But it failed to grasp the object.

The gripper functions are the same as in the libfranka class (cf. here). Regarding the gripper, panda-py is only a thin wrapper, all functionality comes from libfranka and unfortunately there is no way to extend the gripper's behavior. However, I may add more documentation and explanations about the gripper in this repository soon.
You should be able to use both move and grasp without any exceptions. Maybe check that you use sensible parameters, i.e. the maximum speed is around 0.2m/s, width should be between 0m and 0.08m etc. The difference between moving and grasping is that the latter continues to apply a force after the call has returned. Both calls are blocking, but move expects that the fingers can freely move to the desired position and will block until it succeeded (this is just the behavior of the gripper). Additionally, the grasp function requires you to specify a desired force (this is fairly inaccurate and only works somewhat if the width of the grasp is known, i.e. a feed-forward term) as well as allowed deviation from the desired width. If the grasp is without the allowed deviation the grasp will be considered a failure and the gripper won't apply any force. If you're still having trouble try to run homing before using the gripper and maybe verify your gripper is set up properly by controlling it through Desk.

Additionally, the grasp function requires you to specify a desired force (this is fairly inaccurate and only works somewhat if the width of the grasp is known, i.e. a feed-forward term) as well as allowed deviation from the desired width. If the grasp is without the allowed deviation the grasp will be considered a failure and the gripper won't apply any force.

Thank you! I didn't noticed that before. I checked my code and the default value of epsilon_inner and epsilon_outer, then I guess that is the problem.
My robot is FER and I found it failed to grasp because the after the gripper closed and seized the object, it suddenly let it go a little.


wavemon: 0.9.3 -> 0.9.4

Motivation for this change
Update to latest upstream release 0.9.4
Change log: https://github.com/uoaerg/wavemon/releases/tag/v0.9.4
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 138579 run on x86_64-linux 1

1 package built:

wavemon


Pireps page: 'Pending' doesn't show pending PIREPs

Describe the bug
In the admin section of the site, the Pirep page doesn't filter on pending PIREPs.
Version
version 7.0.0 (build 191030.70d43e)
To Reproduce
Steps to reproduce the behavior:

Make sure there is a PIREP with a status of 'Pending'
Log in as admin in phpvms
Click on 'Pireps'
Click on 'Pending' in the top right corner.
See the Pending PIREP disappear

Expected behavior
The pending PIREP should still be shown.
Screenshots
Added.
Additional context
The URL shown is https:///admin/pireps?search=status:0
I have tried with 1 and -1 as well, no change. Haven't looked at the code yet, sorry.
URL: https:///admin/pireps
Result:

URL: https:///admin/pireps?search=status:0 (after clicking on Pending)
Result:

Hmm, that's strange, it seems to be showing up for me:

Lemme investigate the code. It uses the Repository search functionality (https://github.com/andersao/l5-repository#example-the-criteria)

Is this still open?


Access bucket size to correct calculation / riemann sum / integrate / integral / area under graph

Checklist

[x] I am running the latest version of the card
[x] I checked if there isn't another feature request opened with the same request
[x] I checked that the feature is not already available in the dev branch

Is your feature request related to a problem? Please describe.
It is related to #205. I would be great to access the number of states in a bucket so that one can transform the grouped value to display the correct data. For example if monitor the power consumption of a device and group it by 30 minutes, the result will not display the actual consumption over 30 min, but the sum of all occurred states. If there was just one state in 30 minutes, the sum will consist of one state. If there where 300 states  in 30 minutes, the sum will be far higher. This does not represent the reality.
Describe the solution you'd like
If it would be possible to access the number of states in a bucket, one could transform the grouped result. For example: transform: return x / $BUCKET_SIZE;
Or far better, add function to calculate the riemann sum / area under the graph. That would be perfect.
Describe alternatives you've considered
I have tried all different combination of statistics and group_by, but was unable to fix this.
Additional context
Here is the sum per 20 minutes of the power consumption of my server. The server uses between 27 and 50 watts. The graph shows values around 2000 watts. That is ridiculous.

type: custom:apexcharts-card
graph_span: 24h
update_interval: 20min
header:
  show: true
  title: Falsch, da bucket size unbekannt
all_series_config:
  stroke_width: 2
  group_by:
    func: sum
    duration: 20min
    start_with_last: true
  float_precision: 0
series:
  - entity: sensor.wohnzimmer_server_stecker_aubess_tuya_power

Why do you use sum, the right function is avg for your use case, which is also the default.

No, sum is what I need. I want the total amount of consumed energy, not the average.
But, doesn't matter. Access to bucket size would still be great.

Mistakenly closed.

Still a good thing.

Would still be great.


SSL doesn't work when using OTP 19

I've found that the functionality of OTP has changed in version 19, as a result ssl options do not appear to work.

I haven't figured out exactly what has changed between OTP 18.2 and 19 but I have found that it appears to be an issue with TLS 1.2. If I disable TLS 1.2, it connects with TLS 1.1 and works correctly.
Current solution: Disable TLS 1.2 on Domain Controller

This appears to be fixed in the latest OTP release


fix build failure

STFTSpectrogram is not part of Keras 3.6 yet and causing build failure, commenting the part till it will be available as part of next stable release.

This is good to merge @fchollet


Error: Unexpected state in afterSpecHandler: step-started

Current behavior
Getting Unexpected state in afterSpecHandler: step-started error, when the all other remaining tests has skipped due to any error as shown in image. If the tests doesn't skipped then I am not seeing this error. Getting this error for the version 20.0.0 and above, not for the lower versions.

Test code to reproduce
I have given the scenario where the issue is. I hope test code is not required.
Versions

Cypress version: v13.6.5
Preprocessor version: v20.0.1
Node version: v16.17.0

Checklist

[x] I've read the FAQ.
[x] I've read instructions for logging issues.
[x] I'm not using<EMAIL_ADDRESS>(package name has changed and it is no longer the most recent version, see #689).

I hope test code is not required.

It is. You need to provide a minimal, reproducible example that illustrates the issue.

Closing due to lack of a reproducible example. Feel free to open up another issue if anything changes in this regard.


how to retrieve multiple image in php

how to retrieve multiple image in php?

I think you have to refer PHP sites for that.

@farhadsaberi http://stackoverflow.com/a/12006515/1318946 Try to look at this. Thanks :)


Data upload

Great project. However, I can't get the data uploader to work properly. No matter the size of the csv, if I ask for the row count it always returns 4.
For example, I uploaded the mtcars.csv file from here. The bot says it has 4 rows. So I then asked it what the average value of the cycl variable was and the answer was 5.5. When I followed up with a request to tell me how it calculated that, the bot told me that it summed  4, 6, 4, and 8 and then divided by 4.

Not sure how to go about correcting this. My file I'd like to use is only 13.2mb.

Yes, this problem has already been reported to me, that's why I added csv agent but it's true that it's not explained on the chatbot. Basically the main chatbot is designed to answer questions based on the content of the csv to answer questions from the user like: how much money did this company make during the month of may? it's mostly to test the use case of a chatbot for a company on its financial reports for example, the chatbot outputs the answer according to the indexes that best fit the question. for questions on the csv that requires calculations or questions on many data table it should use the button csv agent present on chatbot csv, it has quite a lot of default but it can run python on the csv file to make calculations, count the number of occurrences ...


Performance regression for gluon dataloader with large batch size

Description
gluon's dataloader performs terrible compared to mx.io.NDArrayIter
Environment info (Required)
----------Python Info----------
Version      : 3.6.6
Compiler     : MSC v.1900 64 bit (AMD64)
Build        : ('v3.6.6:4cf1f54eb7', 'Jun 27 2018 03:37:03')
Arch         : ('64bit', 'WindowsPE')
------------Pip Info-----------
Version      : 19.1.1
Directory    : d:\program files\python36\lib\site-packages\pip
----------MXNet Info-----------
Version      : 1.4.1
Directory    : d:\program files\python36\lib\site-packages\mxnet
Commit hash file "d:\program files\python36\lib\site-packages\mxnet\COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library      : ['d:\\program files\\python36\\lib\\site-packages\\mxnet\\libmxnet.dll']
Build features:
No runtime build feature info available
----------System Info----------
Platform     : Windows-10-10.0.17758-SP0
system       : Windows
node         : Neutron
release      : 10
version      : 10.0.17758
----------Hardware Info----------
machine      : AMD64
processor    : Intel64 Family 6 Model 158 Stepping 10, GenuineIntel
Name
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz

----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0249 sec, LOAD: 2.2043 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.5834 sec, LOAD: 1.2048 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0020 sec, LOAD: 0.9903 sec.
Error open FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)>, DNS finished in 0.1266636848449707 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.2234 sec, LOAD: 4.6954 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.3620 sec, LOAD: 0.7789 sec.

Minimum reproducible example
Steps to reproduce
import mxnet as mx
def data_xform(data):
    """Move channel axis to the beginning, cast to float32, and normalize to [0, 1]."""
    return mx.ndarray.moveaxis(data, 2, 0).astype('float32') / 255

train_data = mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
train_loader = mx.gluon.data.DataLoader(train_data, shuffle=False, batch_size=10000)
a=train_loader.__iter__()
from time import time
t=time()
_=next(a)
print(time()-t)

It took 3.6711745262145996 seconds to execute a next(a), roughly 20s for total 6 next(a)
For mx.io.NDArrayIter, it finish iter almost immediately.
What have you tried to solve it?
I find the problem occurs here, but I cannot fix it.
What's more, with changing batch_size to 10000, NDArray API failed to optimize the MNIST model:
......
>>> batch_size = 10000
>>> train_iter = mx.io.NDArrayIter(mnist['train_data'], mnist['train_label'], batch_size, shuffle=True)
>>> val_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'], batch_size)
......
>>> import logging
>>> logging.getLogger().setLevel(logging.DEBUG)  # logging to stdout
>>> # create a trainable module on compute context
... mlp_model = mx.mod.Module(symbol=mlp, context=ctx)
>>> mlp_model.fit(train_iter,  # train data
...               eval_data=val_iter,  # validation data
...               optimizer='sgd',  # use SGD to train
...               optimizer_params={'learning_rate':0.1},  # use fixed learning rate
...               eval_metric='acc',  # report accuracy during training
...               batch_end_callback = mx.callback.Speedometer(batch_size, 100), # output progress for each 100 data batches
...               num_epoch=10)  # train for at most 10 dataset passes
INFO:root:Epoch[0] Train-accuracy=0.107000
INFO:root:Epoch[0] Time cost=0.123
INFO:root:Epoch[0] Validation-accuracy=0.113500
INFO:root:Epoch[1] Train-accuracy=0.112367
INFO:root:Epoch[1] Time cost=0.183
INFO:root:Epoch[1] Validation-accuracy=0.113500
INFO:root:Epoch[2] Train-accuracy=0.112367
INFO:root:Epoch[2] Time cost=0.158
INFO:root:Epoch[2] Validation-accuracy=0.113500
INFO:root:Epoch[3] Train-accuracy=0.112367
INFO:root:Epoch[3] Time cost=0.504
INFO:root:Epoch[3] Validation-accuracy=0.113500
INFO:root:Epoch[4] Train-accuracy=0.112367
INFO:root:Epoch[4] Time cost=0.142
INFO:root:Epoch[4] Validation-accuracy=0.113500
INFO:root:Epoch[5] Train-accuracy=0.112367
INFO:root:Epoch[5] Time cost=0.164
INFO:root:Epoch[5] Validation-accuracy=0.113500
INFO:root:Epoch[6] Train-accuracy=0.112367
INFO:root:Epoch[6] Time cost=0.471
INFO:root:Epoch[6] Validation-accuracy=0.113500
INFO:root:Epoch[7] Train-accuracy=0.112367
INFO:root:Epoch[7] Time cost=0.167
INFO:root:Epoch[7] Validation-accuracy=0.113500
INFO:root:Epoch[8] Train-accuracy=0.112367
INFO:root:Epoch[8] Time cost=0.241
INFO:root:Epoch[8] Validation-accuracy=0.113500
INFO:root:Epoch[9] Train-accuracy=0.112367
INFO:root:Epoch[9] Time cost=0.307
INFO:root:Epoch[9] Validation-accuracy=0.113500

compared to NDArrayIter
import mxnet as mx

# Select a fixed random seed for reproducibility
mx.random.seed(42)

def data_xform(data):
    """Move channel axis to the beginning, cast to float32, and normalize to [0, 1]."""
    return nd.moveaxis(data, 2, 0).astype('float32') / 255

train_data = mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
val_data = mx.gluon.data.vision.MNIST(train=False).transform_first(data_xform)

import gzip,struct
import numpy as np
#I cannot connect to the original MXNET dataset provides by mxnet.utils.get_mnist, thus I tried to download data first and "get" it using this function.
def get_mnist():
    """Download and load the MNIST dataset
    Returns
    -------
    dict
        A dict containing the data
    """
    path ='C:/Users/Neutron/AppData/Roaming/mxnet/datasets/mnist/'
    def read_data(label_url, image_url):
        with gzip.open(label_url) as flbl:
            struct.unpack(">II", flbl.read(8))
            label = np.frombuffer(flbl.read(), dtype=np.int8)
        with gzip.open(image_url, 'rb') as fimg:
            _, _, rows, cols = struct.unpack(">IIII", fimg.read(16))
            image = np.frombuffer(fimg.read(), dtype=np.uint8).reshape(len(label), rows, cols)
            image = image.reshape(image.shape[0], 1, 28, 28).astype(np.float32)/255.
        return (label, image)
    (train_lbl, train_img) = read_data(
        path+'train-labels-idx1-ubyte.gz', path+'train-images-idx3-ubyte.gz')
    (test_lbl, test_img) = read_data(
        path+'t10k-labels-idx1-ubyte.gz', path+'t10k-images-idx3-ubyte.gz')
    return {'train_data':train_img, 'train_label':train_lbl,
            'test_data':test_img, 'test_label':test_lbl}

mnist=get_mnist()

batch_size = 10000
#Even with shuffle=False, the performance of DataLoader is terrible with batch_size=10000
#train_loader = mx.gluon.data.DataLoader(train_data, shuffle=False, batch_size=batch_size)
#val_loader = mx.gluon.data.DataLoader(val_data, shuffle=False, batch_size=batch_size)

train_iter = mx.io.NDArrayIter(mnist['train_data'], mnist['train_label'], batch_size, shuffle=True)
val_iter = mx.io.NDArrayIter(mnist['test_data'], mnist['test_label'], batch_size)

from mxnet import nd, gluon, autograd
from mxnet.gluon import nn

net = nn.HybridSequential(prefix='MLP_')
with net.name_scope():
    net.add(
        nn.Flatten(),
        nn.Dense(128, activation='relu'),
        nn.Dense(64, activation='relu'),
        nn.Dense(10, activation=None)  # loss function includes softmax already, see below
    )

ctx = mx.gpu(0) if mx.context.num_gpus() > 0 else mx.cpu(0)
net.initialize(mx.init.Xavier(), ctx=ctx)

trainer = gluon.Trainer(
    params=net.collect_params(),
    optimizer='sgd',
    optimizer_params={'learning_rate': 0.1},
)
metric = mx.metric.Accuracy()
loss_function = gluon.loss.SoftmaxCrossEntropyLoss()

num_epochs = 10

from time import time
for epoch in range(num_epochs):
    for data in train_iter:
        inputs, labels=data.data[0],data.label[0]
        # Possibly copy inputs and labels to the GPU
        inputs = inputs.as_in_context(ctx)
        labels = labels.as_in_context(ctx)
        # The forward pass and the loss computation need to be wrapped
        # in a `record()` scope to make sure the computational graph is
        # recorded in order to automatically compute the gradients
        # during the backward pass.
        with autograd.record():
            outputs = net(inputs)
            loss = loss_function(outputs, labels)
        # Compute gradients by backpropagation and update the evaluation
        # metric
        loss.backward()
        metric.update(labels, outputs)
        # Update the parameters by stepping the trainer; the batch size
        # is required to normalize the gradients by `1 / batch_size`.
        trainer.step(batch_size=inputs.shape[0])
    train_iter.reset()
    # Print the evaluation metric and reset it for the next epoch
    name, acc = metric.get()
    print('After epoch {}: {} = {}'.format(epoch + 1, name, acc))
    metric.reset()

#found it much faster than DataLoader.

Any idea @zhreshold?

Data loaders are different from Data iterators.
Data iterators first load all the data in to the memory before iterating it, while data loaders load data into memory as they iterate the data, hence they are slower than data iterators.
So why we use data loaders if data iterators are faster?
Some times when we deal with massive datasets that can't be loaded into memory, so for these data sets data iterators wouldn't work.
In practice we should use data iterators if our dataset can be loaded into memory (for example small datasets), otherwise we have to use data loaders.
For more info regarding Data iterators Vs Data loaders check this out.

Data loaders are different from Data iterators.
Data iterators first load all the data in to the memory before iterating it, while data loaders load data into memory as they iterate the data, hence they are slower than data iterators.
So why we use data loaders if data iterators are faster?
Some times when we deal with massive datasets that can't be loaded into memory, so for these data sets data iterators wouldn't work.
In practice we should use data iterators if our dataset can be loaded into memory (for example small datasets), otherwise we have to use data loaders.
For more info regarding Data iterators Vs Data loaders check this out.

Maybe you are right.
BUT... Did you test how slow the DataLoader could be?
It tooks 18s to finish a loop
import mxnet as mx
from mxnet import nd
def data_xform(data):
    """Move channel axis to the beginning, cast to float32, and normalize to [0, 1]."""
    return nd.moveaxis(data, 2, 0).astype('float32') / 255

train_data = mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
val_data = mx.gluon.data.vision.MNIST(train=False).transform_first(data_xform)
batch_size = 100#set to 10000 produce the same result.
train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size)
val_loader = mx.gluon.data.DataLoader(val_data, shuffle=False, batch_size=batch_size)
for i,j in train_loader:
  pass

It took 18s for me to finish the final loop. (with both mxnet-cu100mkl and mxnet-mkl tested)
It is not the data loader slower than data iter

@Neutron3529
Unfortunately for DataLoader, it has to allocate additional memory as you iterate through the dataset, and it will involve mx.nd.stack operator to batch images, which means mxnet engine will take control. In comparison, NDArrayIter or pure numpy array iteration won't trigger additional overheads.
This problem is rather visible to small workloads, i.e., for MNIST. However, for large network training, several seconds is merely nothing compared to per epoch training or validation time (mins).
In fact if you have multi-core cpu you can speed up the process by utilizing multi_worker in this case
import mxnet as mx
from mxnet import nd
import time

def data_xform(data):
    """Move channel axis to the beginning, cast to float32, and normalize to [0, 1]."""
    return nd.moveaxis(data, 2, 0).astype('float32') / 255

def bench_time(num_workers=0):
    print('-----\nnum_workers:', num_workers)
    tic = time.time()
    train_data = mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
    val_data = mx.gluon.data.vision.MNIST(train=False).transform_first(data_xform)
    t1 = time.time() - tic
    tic = time.time()
    batch_size = 100#set to 10000 produce the same result.
    train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size, num_workers=num_workers)
    val_loader = mx.gluon.data.DataLoader(val_data, shuffle=False, batch_size=batch_size, num_workers=num_workers)
    for i,j in train_loader:
        pass
    t2 = time.time() - tic
    print('t1', t1, 't2', t2)

if __name__ == '__main__':
    bench_time(0)
    bench_time(4)

-----
num_workers: 0
t1 0.35317301750183105 t2 8.19723916053772
-----
num_workers: 4
t1 0.2771739959716797 t2 3.3613219261169434

Thanks for your reply.
Maybe I should choose Linux rather than windows since windows has no fork and it will take even longer time to execute the benchmark..
-----
num_workers: 6
t1 0.628288745880127 t2 28.964709043502808
-----
num_workers: 5
t1 0.6462414264678955 t2 26.153035402297974
-----
num_workers: 4
t1 0.6172983646392822 t2 23.17400550842285
-----
num_workers: 3
t1 0.608367919921875 t2 19.690324306488037
-----
num_workers: 2
t1 0.6791801452636719 t2 18.883481979370117
-----
num_workers: 1
t1 0.6682109832763672 t2 25.414013624191284

I close this issue since with small dataset, I could store it in my GPU directly and my GPU could not support a larger dataset since it is quite slow..
Anyway, I hope I could find a faster dataloader in the future.
BTW, I find it is too slow to import mxnet pack since the mxnet in .whl file contains a huge amount of CUDA_ARCHes. I asked how to remove the unnecessary ARCH, and find nvprune could do such thing.
The question is, the libmxnet.dll I have is not relocatable so nvprune failed to strip the unnecessary ARCHes. I know that libmxnet.lib store all the symbols of libmxnet.dll, but I don't know how to merge them together.
I wonder if it is possible using nvprune and get a smaller libmxnet.dll from a .whl file.
If it is possible, we will import mxnet much faster.

The next gen dataloader is under discussion so yes it will get faster in the future.
To reduce the size of libmxnet.dll you can specify which arch and build from source on your desktop.  The libmxnet.dll on windows has trade-offs choosing relocatable archs and to be honest I am not familiar with that.

@zhreshold
I find the bottleneck:
>>> import mxnet as mx
>>> def data_xform(data):
...     """Move channel axis to the beginning, cast to float32, and normalize to [0, 1]."""
...     return mx.nd.moveaxis(data, 2, 0).astype('float32') / 255
...
>>> train_data = mx.gluon.data.vision.MNIST(train=True).transform_first(data_xform)
>>> train_data_NO_TRANSFORM = mx.gluon.data.vision.MNIST(train=True)
>>>
>>> batch_size = 10000
>>> train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size)
>>> train_loader_NO_TRANSFORM = mx.gluon.data.DataLoader(train_data_NO_TRANSFORM, shuffle=True, batch_size=batch_size)
>>>
>>>
>>> from time import time
>>> t=time()
>>> for data, label in train_loader:
...   pass
...
>>> print(time()-t)
22.582587480545044
>>> t=time()
>>>
>>> for data, label in train_loader_NO_TRANSFORM:
...   pass
...
>>> print(time()-t)
1.8590285778045654
>>> batch_size=100
>>> train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size)
>>> t=time()
>>> for data, label in train_loader_NO_TRANSFORM:
...   data=data_xform(data)
...
>>> print(time()-t)
2.0953640937805176
>>> batch_size=10
>>> train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size)
>>> t=time()
>>> for data, label in train_loader_NO_TRANSFORM:
...   data=data_xform(data)
...
>>> print(time()-t)
2.348717451095581
>>> batch_size=1
>>> train_loader = mx.gluon.data.DataLoader(train_data, shuffle=True, batch_size=batch_size)
>>> t=time()
>>> for data, label in train_loader_NO_TRANSFORM:
...   data=data_xform(data)
...
>>> print(time()-t)
2.3497142791748047
>>>

.transform_first spends so much time than it is expect.
Change the time to execute data_xform (e.g., in a batch) could help improve the performance


Addressing Duplicates Question and Incorrect 'Ground Truth' in RAG Evaluation with Ragas

I am currently using Ragas to evaluate my RAG application, which is built using llama index . I've encountered a few issues in the generated results:
1- When generating queries using TestsetGenerator, I've noticed that many of the questions are duplicated.
2- In several instances, the 'ground_truth' is incorrectly marked as "The answer to the given question is not present in the context," even though the context is actually available. This leads to a 'context_precision' and 'context_recall' of 0, negatively impacting the overall score.
Could you advise on how to address these issues?

@dosu  How can i handle such cases where it is incorrectly classifying "The answer to the given question is not present in the context"


Hangman Game

Description
Please include a summary of the change and which issue is fixed. List any dependencies that are required for this change.
Fixes #(issue_no)
Replace issue_no with the issue number which is fixed in this PR
Have you read the Contributing Guidelines on Pull Requests?

[ ] Yes
[ ] No

Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Documentation Update

Checklist:

[ ] My code follows the style guidelines(Clean Code) of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have created a helpful and easy to understand README.md
[ ] My documentation follows Template for README.md
[ ] My changes generate no new warnings
[ ] I have added tests/screenshots(if any) that prove my fix is effective or that my feature works.

@avinashkranjan plss review it this is pull for the issue no #245

@avinashkranjan plss review it this is pull for the issue no #245

Please fill the PR template first.

Please fill the PR template first.

@atarax665 bro i am deleted some files but some files are not deleted plsss tell me why plss help me bro

@atarax665 bro i am deleted some files but some files are not deleted plsss tell me why plss help me bro

@atarax665 bro i am deleted some files but some files are not deleted plsss tell me why plss help me bro

You have deleted the branch from which you raised this PR. I will advice you to raise a new PR from a new branch with proper README, Screenshots and source code related to the issue #245

@atarax665 bro i am deleted some files but some files are not deleted plsss tell me why plss help me bro

You have deleted the branch from which you raised this PR. I will advice you to raise a new PR from a new branch with proper README, Screenshots and source code related to the issue #245


Allow more Hessian set types than just Set{Tuple{Int,Int}}

At the moment, using TracerSparsityDetector{G,H} with H something other than Set{Tuple{Int,Int}} will fail. But one might like to try:

G = MySet{Int} and H = MySet{Tuple{Int,Int}} (definitely)
G = MySet{Int} and H = MyOtherSet2Tuple{Int,Int}} (maybe)

To remedy this, here are the steps to take:

Allow MySet to accept things beyond Number (easy)
Implement union! for MySet (easy except for RecursiveSet)
Make sure that the result of the Cartesian product x(::MySet1{Int}, ::MySet1{Int}) can be union-ed with MySet{Tuple{Int,Int}}. This can be solved in two non-mutually-exclusive ways:

Make x(::MySet{Int}, ::MySet{Int}) return a MySet{Tuple{Int,Int}}. That way, the existing homogeneous union(::MySet{T}, ::MySet{T}) will work
Implement heterogeneous unions union(::MySet{T}, ::AbstractSet{T}). I don't like this option because it will presumably be very slow, in a way that is not directly visible to the user.

I've spent a lot of time thinking about this and think that the cleanest solution is to implement
×(::Type{H}, grad1::G, grad2::G)

for HessianTracer{G,H} / TracerSparsityDetector{G,H}.
It solves options 1 and 2 and avoids some of the pitfalls of heterogeneous Hessian-set unions, since H is specified prior to computing a Hessian-set union.

okay, but I'll start with the homogeneous methods

And for maximum performance, we might even end up needing
×!(hessian_buffer::H, grad1::G, grad2::G)

in the long term to avoid allocations.


license

Without a license, this project can't grow without legal issues.
I advocate for the MIT license cause it's simple, but it's ur choice @en-GB

yes MIT seems reasonable


Unusable for non-interactive accounts

There does not appear to be a way for a script running as a non-interactive user to create/interact with a local Secret Store.  As the password is set interactively on fist usage, a non-interactive user cannot set the password of a newly-created store.  And as the stores are per-user, there is no way to create and configure a store manually.  Adding a means to specify a password on store registration or through a secondary action would be very useful.

You can use Reset-SecretStore after creating it. This removes all SecretStore data! But if you just created it, there should be no problem with that. At least in my mind.. I am doing this in scripts and it works for me. You could use something like the following:

Interactively
[securestring] $SecretStorePassword = Read-Host -Prompt "Please enter a new password for the SecretStore" -AsSecureString
$SecretStorePassword | ConvertFrom-SecureString | Set-Content -Path C:\myPasswordFile -Force
Non-interactively
<Create SecretStore here>
Reset-SecretStore -Scope CurrentUser -Authentication Password -Password $(Get-Content -Path C:\myPasswordFile | ConvertTo-SecureString) -PasswordTimeout 300 -Interaction Prompt -Confirm:$true

Hops that helps.

Please also have a look at Issue #46 where something similar is discussed. Set-SecretStoreConfiguration provides another way to accomplish what you are looking for. You will find it in my post there.

The latest RC3 release now lets Set-SecretStoreConfiguration to run non-interactively.
https://devblogs.microsoft.com/powershell/secretstore-release-candidate-3/
Also, please see:
https://github.com/PowerShell/SecretStore/issues/46#issuecomment-801212012

Thank you.  I had not seen the new release, issue resolved.


Rewamp DebugBreak handling

Expand Debug.DebugBreak as IL intrinsic
Implement RhDebugBreak JIT helper for IL break instruction
Switch assembly code to use RhDebugBreak to avoid C++ name mangling goo
Change PalDebugBreak to be inline to avoid extra frames under debugger
Cleanup some left-over cruft in Unix PAL

👍


Could you update Docker Hub with the latest changes

Could you update Docker Hub with the latest changes. Seems like it was last updated 3 years ago - https://hub.docker.com/r/johnktims/folding-at-home

Done. The build is now automated.


added eips and spot instance to the default yaml config

These could be used as defaults, which would optimize on cost and also clean up the custom config yamls in the examples.

Yeah I like this as well. Merging in.

Yup like this idea and will be merging. Change looks good to me


Can you support for landscape mode? Recording video with landscape.

Maybe later🙂

Please try that yourself first.
I'm busy doing other things, so I this may delay for weeks.

Hi @wysaid , Thanks, I got it. But when i increase resolution to 1280 x 720 the frame rate decrease. 25/30 fps without filter and about 20/ 30 fps when add filter. Can we control it enough 30 frame?

Please compare with other apps using the same device, such as snapchat and facebook.(using the same )
If their fps is not better than this demo, maybe  your device's performance is a limit.

Let me check. But I use Samsung note 5


Feature extractions on 2D tiff images

Is it possible to use extract first order features on 2D tiff images using the built in class?

@JoostJM I have a similar question , whether the DCM image (2D)  also needs to convert to 3D?

@QiChen2014 Yes

Try this:
import SimpleITK as sitk
import numpy
from radiomics import featureextractor

# Transform input
im_tiff = sitk.ReadImage('path/to/image.tiff')
im_vect = sitk.JoinSeries(im_tiff)  # Add 3rd dimension
im = sitk.VectorImageSelectionCast(im_vect, 0, sitk.sitkFloat64)  # Select first color channel (if image is grayscale, all channels are equal)

# Build full mask
im_arr = sitk.GetArrayFromImage(im)
ma_arr = numpy.ones(im_arr.shape)
ma = sitk.GetImageFromArray(ma_arr)
ma.CopyInformation(im)

# Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor('path/to/config')  # optional supply parameter file)

# extractor = featureextractor.RadiomicsFeaturesExtractor()  # Default settings

# Enable just first order
extractor.disableAllFeatures()
extractor.enableFeatureClassByName('firstorder')

# Extract features
results = extractor.execute(im, ma)

How to convert tif image (RGB) into grayscale image？

@destinybuilt, PyRadiomics has since been upgraded and now also supports 2D input. Therefore, the addition of the 3rd dimension is no longer necessary.
However, PyRadiomics does still require a scalar datatype. Your example constitutes one way of dealing with this (extracting features for channels separately). You can also put this in a loop to extract for all channels:
import SimpleITK as sitk
import numpy
from radiomics import featureextractor

# Transform input
im_vect = sitk.ReadImage('path/to/image.tiff')
# im_vect = sitk.JoinSeries(im_vect)  # Add 3rd dimension, NO LONGER NECESSARY

# Build full mask
im_size = numpy.array(im_vect.GetSize())[::-1]  # flip x, y, z to z, y, x
ma_arr = numpy.ones(im_size)
ma = sitk.GetImageFromArray(ma_arr)
ma.CopyInformation(im_vect)

# Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor('path/to/config')  # optional supply parameter file)

# extractor = featureextractor.RadiomicsFeaturesExtractor()  # Default settings

# Enable just first order
extractor.disableAllFeatures()
extractor.enableFeatureClassByName('firstorder')

# Extract features
results = []
selector = sitk.VectorIndexSelectionCastImageFilter()
for i in range(im_vect.GetNumberOfComponentsPerPixel()):
  selector.SetIndex(i)
  im = selector.Execute(im_vect)  # Select first color channel (if image is grayscale, all channels are equal)
  channel_result = extractor.execute(im, ma)
  channel_result["channel"] = i
  results.append(channel_result

Alternatively, you can merge the channels prior to extraction (yielding a scalar volume). For this, you'd need to decide how to aggregate the information from the separate channels. E.g. take the mean, max, min...

@destinybuilt, PyRadiomics has since been upgraded and now also supports 2D input. Therefore, the addition of the 3rd dimension is no longer necessary.
However, PyRadiomics does still require a scalar datatype. Your example constitutes one way of dealing with this (extracting features for channels separately). You can also put this in a loop to extract for all channels:
import SimpleITK as sitk
import numpy
from radiomics import featureextractor

# Transform input
im_vect = sitk.ReadImage('path/to/image.tiff')
# im_vect = sitk.JoinSeries(im_vect)  # Add 3rd dimension, NO LONGER NECESSARY

# Build full mask
im_size = numpy.array(im_vect.GetSize())[::-1]  # flip x, y, z to z, y, x
ma_arr = numpy.ones(im_size)
ma = sitk.GetImageFromArray(ma_arr)
ma.CopyInformation(im_vect)

# Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor('path/to/config')  # optional supply parameter file)

# extractor = featureextractor.RadiomicsFeaturesExtractor()  # Default settings

# Enable just first order
extractor.disableAllFeatures()
extractor.enableFeatureClassByName('firstorder')

# Extract features
results = []
selector = sitk.VectorIndexSelectionCastImageFilter()
for i in range(im_vect.GetNumberOfComponentsPerPixel()):
  selector.SetIndex(i)
  im = selector.Execute(im_vect)  # Select first color channel (if image is grayscale, all channels are equal)
  channel_result = extractor.execute(im, ma)
  channel_result["channel"] = i
  results.append(channel_result

Alternatively, you can merge the channels prior to extraction (yielding a scalar volume). For this, you'd need to decide how to aggregate the information from the separate channels. E.g. take the mean, max, min...

Thank you for your reply!
Maybe I should explain to you the purpose of applying pyradiomics to tif images: I want to extract all the features of the tif image...not just the first-order features; I use matlab to read the tif file (info = Tiff('path /to/image.tif', 'r');)， read my image and mask (I already have a mask file so I don't need numpy to generate the mask) are 2D RGB files; see this post Previous content, so I thought I would like to increase the third dimension of my two tif files (set to 1), and I found that after using the sitk.JoinSeries function, the third dimension of my original tif file did not Was changed (I have never used simple itk and pyradiomics, so I guess maybe I don't know how to save the changed file overwrite to the original path...)
The following is the code I used to run pyradiomics for extraction. Can you help to modify the code to meet the purpose of extracting all the features of the tif file (about twenty patients)? thank you very much!
import radiomics
import radiomics.featureextractor as FEE
import SimpleITK as sitk
file name
main_path = r'D:\1Transition\1Trans\Test'
ori_name = r'\tissueSlide.tif'
lab_name = r'\mask.tif'
para_name = r'\Params.yaml'
File path
ori_path = main_path + ori_name # image path
lab_path = main_path + lab_name # mask path
para_path = main_path + para_name
print("originl path: " + ori_path)
print("label path: " + lab_path)
print("parameter path: " + para_path)
Initialize the feature extractor using a configuration file
extractor = FEE.RadiomicsFeaturesExtractor(para_path)
print("Extraction parameters:\n\t", extractor.settings)
print("Enabled filters:\n\t", extractor._enabledImagetypes)
print("Enabled features:\n\t", extractor._enabledFeatures)
Extract features
result = extractor.execute(ori_path, lab_path)  # Extract features
print("Result type:", type(result))  # result is returned in a Python ordered dictionary
print("")
print("Calculated features")
for key, value in result.items():  # Output features
print("\t", key, ":", value)

You can apply the following code, it will extract features for each channel in the image, and use the first channel in mask to get the mask (I assume all 3 channels in the mask are identical).
If your image is grayscale, you can extract from 1 channel (e.g. channel 0), as all channels are the same.
Finally, I'd advise using a parameter file. The documentation for this is here.
You need a list of python tuples, which each tuple consisting of 3 values: p_name (identifier for your case), string pointing to the location of the image file and string pointing to the location of the mask file.
e.g.:
get_patients = [
  ('case_1', r'path\to\case_1\image.tiff', r'path\to\case_1\mask.tiff'),
  ('case_2', r'path\to\case_2\image.tiff', r'path\to\case_2\mask.tiff'),
  ...,
  ('case_n', r'path\to\case_n\image.tiff', r'path\to\case_n\mask.tiff'),
]

You can then use this piece of code:
import SimpleITK as sitk
import numpy
from radiomics import featureextractor

param_file = r'path\to\config.yml'

# Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor('path/to/config')  # optional supply parameter file)

results_dict = {}

# Extract features for each case
for p_name, im_path, ma_path in get_patients:
  # Transform input
  im_vect = sitk.ReadImage(im_path)

  # Extract features
  results = []
  selector = sitk.VectorIndexSelectionCastImageFilter()
  for i in range(im_vect.GetNumberOfComponentsPerPixel()):  # extract features for each channel
    selector.SetIndex(i)
    im = selector.Execute(im_vect)  # Select first color channel (if image is grayscale, all channels are equal)
    channel_result = extractor.execute(im, ma_path, label_channel=0)  # label_channel = 0 selects the first channel in the mask tiff image
    channel_result["channel"] = i
    results.append(channel_result
  results_dict[p_name] = results

This will give you a python dictionary (results_dict), with 1 entry for each case.
each entry value is a list (length = number of channels), where each item in the list is another python dictionary, giving you the feature values for that case for that channel:
results_dict = {
  'case_1': [
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value}
  ],

  'case_2': [
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value}
  ],

  ...,

  'case_n': [
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value},
    { 'feature_1': value, 'feature_2': value, ..., 'feature_n': value}
  ]
}

@JoostJM Thank you very much, I will run your code a bit, but it will still report an error. The error is:
RuntimeError: Exception thrown in SimpleITK VectorIndexSelectionCastImageFilter_Execute: c:\d\vs14-win64-pkg\simpleitk\code\common\include\sitkDualMemberFunctionFactory.hxx:214:
Sitk::ERROR: Pixel type: 8-bit unsigned integer is not supported in 2D byclass itk::simple::VectorIndexSelectionCastImageFilter
Here is my code:
import SimpleITK as sitk
import numpy
from radiomics import featureextractor
param_file = r'D:\1Transition\1Trans\Test\Params.yaml'
Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor(r'D:\1Transition\1Trans\Test\Params.yaml')  # optional supply parameter file)
get_patients=[('case_1',r'D:\1Transition\1Trans\Test\tissueSlide.tif',r'D:\1Transition\1Trans\Test\mask.tif'),('case_2',r'D:\1Transition\1Trans\Test\tissueSlide.tif',r'D:\1Transition\1Trans\Test\mask.tif')]
results_dict = {}
Extract features for each case
for p_name, im_path, ma_path in get_patients:
Transform input
im_vect = sitk.ReadImage(im_path)
Extract features
results = []
selector = sitk.VectorIndexSelectionCastImageFilter()
for i in range(im_vect.GetNumberOfComponentsPerPixel()):  # extract features for each channel
selector.SetIndex(i)
im = selector.Execute(im_vect)  # Select first color channel (if image is grayscale, all channels are equal)
channel_result = extractor.execute(im, ma_path, label_channel=0)  # label_channel = 0 selects the first channel in the mask tiff image
channel_result["channel"] = i
results.append(channel_result)
results_dict[p_name] = results
I can't upload more than 10MB files (even if I use zip files). Can you give me one of your emails? I can pass you a tif file and the corresponding mask so that you can debug it.

Try this
import SimpleITK as sitk
import numpy
from radiomics import featureextractor

param_file = r'path\to\config.yml'

# Instantiate extractor
extractor = featureextractor.RadiomicsFeaturesExtractor('path/to/config')  # optional supply parameter file)

results_dict = {}

# Extract features for each case
for p_name, im_path, ma_path in get_patients:
  # Transform input
  im_vect = sitk.ReadImage(im_path)

  # Extract features
  results = []
  if im_vect.GetNumberOfComponentsPerPixel() == 1:  # scalar image
    results.append(extractor.execute(im_vect, ma_path, label_channel=0)
  else:  # vector image
    selector = sitk.VectorIndexSelectionCastImageFilter()
    for i in range(im_vect.GetNumberOfComponentsPerPixel()):  # extract features for each channel
      selector.SetIndex(i)
      im = selector.Execute(im_vect)  # Select first color channel (if image is grayscale, all channels are equal)
      channel_result = extractor.execute(im, ma_path, label_channel=0)  # label_channel = 0 selects the first channel in the mask tiff image
      channel_result["channel"] = i
      results.append(channel_result
  results_dict[p_name] = results

@JoostJM It still reports an error....
TypeError: execute() got an unexpected keyword argument 'label_channel
The following is the image and mask I shared with Google Cloud Drive：
https://drive.google.com/file/d/1tEj9AwFzrh_j-rjQtXeb_7bf5HN8BP5U/view?usp=sharing
https://drive.google.com/file/d/1uvrwiKjDuhiKWpLoHnO8wZsdLRZn2jOY/view?usp=sharing
https://drive.google.com/file/d/1Xfycu5EXFezEq6DXVu1THHbPKh-Jnp9V/view?usp=sharing

@JoostJM  The following is the image and mask I shared with Google Cloud Drive:
https://drive.google.com/file/d/1tEj9AwFzrh_j-rjQtXeb_7bf5HN8BP5U/view?usp=sharing
https://drive.google.com/file/d/1uvrwiKjDuhiKWpLoHnO8wZsdLRZn2jOY/view?usp=sharing
https://drive.google.com/file/d/1Xfycu5EXFezEq6DXVu1THHbPKh-Jnp9V/view?usp=sharing


Template 10 : Cross-platform support

As discussed in #684, Template 10 should be cross-platform (with support for Xamarin). As much of it as possible should be cross-platform.

This issue seems too broad at first glance. Let's try to zero down into the problem. What components of Template10 need to be cross platform? First and foremost, ViewModels. Those aren't portable because NavigationService and it's friends rely on UWP specific classes. To remedy this, we need to make interfaces abstract enough so that they do not rely on platform specific code. Currently, I have a working solution but requires implementing INavigable on each individual ViewModel explicitly. This results in a lot of repeated code like this example..
Let's start with following rough checklist of what needs to be done. We can start working once @JerryNixon approves of the design.

[ ] Restructure INavigable to make it PCL compatible.
[ ] Restructure INavigationService to make it PCL compatible.
[ ] Supply platform dependent implementations of the interfaces above. This is important to avoid creating bridges between different navigation interfaces (for example, MVVMLight navigation service + T10 interface). All platforms can stick to using the T10 implementation.
[ ] Supply platform dependent ViewModelBase implementation to make use of platform specific state management, etc.
[ ] Make IStateItems platform independent. I think the interface and it's implementation both already are PCL compatible. We just need to integrate them in VMs in platform agnostic way.

Let me know if I missed something.

Very good

Are there any plans for cross platform support? If there is, any eta?

@syaifulnizamyahya I think #1444 should answer your question.


Plugin Support in Docker

Once https://github.com/inventree/InvenTree/pull/2074 is merged, we need to consider how to support plugins in a docker configuration.
In a "production" docker setup, the python environment is setup internal to the docker container as part of the dockerfile.
Installing any further python plugins will not be "persistent" as they are stored temporarily within the container instance.
The "development" container, on the other hand, places the python environment in the mounted volume external to the docker container. This means that installation of plugins in the "development" container is persistent...
Maybe we can move to storing the python environment externally for the "production" docker setup also? Or some other solution.
Whatever the solution we will need to add documentation for installing plugins under docker

@SchrodingersGat already working on that; installed plugins can be saved to a text file (like the secret) and the system checks on startup if all plugins in the file are there. This requires an install mechanism which I already have running with PyPI but not custom registries - which is a must for private plugins.
Also live - reload should be comming today or tomorrow, I am working out the kinks at the moment.

@matmair OK I'll wait to see what you have worked out :)

@matmair what's the status of this? If you have not done this, I have an idea for implementation.

Add a new file called "plugins.txt" (similar to "requirements.txt") which contains a snapshot of required plugins
Can be stored anywhere, specified as PLUGIN_REQUIREMENTS_FILE in settings.py
In docker setup, this would be in "/home/inventree/plugins.txt"
Add a new invoke command "invoke plugins" which installs the plugins (into the virtual env) based on this file

Let me know if you have any objections otherwise I'll work on implementing something.


Determine fields we need to deal with in database / interface(s)

Note that different fields may be relevant to different "user stories"
From Tuan:

After much thought, I realized the only fields to, ideally, map to each other/combine are under reading level, where we  have different entries to translate, see:

https://www.readinga-z.com/learninga-z-levels/level-correlation-chart/

Hi Dav,

Can we please get a sample data on mercury to so that we can explore postgresql more.
Also how do you think we can best implement a database search functionality using Django. As a very basic starting point, we would like to have search by author or title working.

Sorry - I'm a bit overwhelmed with beginning-of-semester activities... so I'll be slow.
For (2) - do you understand this stuff? This is the core logic:
https://docs.djangoproject.com/en/1.8/topics/db/queries/
You'll use a form to generate the query, and a template to display the results list - each element can link to a view of an individual book.
For (1) I think this is the sample data from Tuan: https://drive.google.com/open?id=0B2JP4TBQGyuXcXlTNDVfZ1BzT2QxOC1CRWQ2STF3NjJFZGlB

The model should be updated to have all the columns as fields.

I'm pretty sure this is done, but it is a good opportunity for Code Review!
@yadelab can you have a look at the model code that @tonyshenyy wrote and verify that it matches the book fields and generally think about whether the fields are well-specified? You can provide feedback to @tonyshenyy either here, or via the GitHub code commenting interface (ask me how to use that if you'd like).


Can't get tests to work

I was working on a fix for #74 and I wanted to run the tests you have in the repository but as far as I can tell they don't work. Is this something you are working on?

@griff sorry I'm just now getting back to you. I've been out of commission for the last few days.
I'm really glad you started by running the test suite, most people don't. :smile:
All of the tests do pass, but you have to jump through a few hoops to run them. I'll start be showing you how to run them. This example uses the elasticsearch-embedded test:
$ cd ./1.4/test
$ make test-elasticsearch-embedded

That will fire up the container for testing. You then need to docker exec to get inside of it.
$ docker exec -it elasticsearchembedded_logstash_1 /bin/bash

Then fire up the bats test suite:
$ ./test.bats
 ✓ Logstash is running
 ✓ Logstash.config contains '{ elasticsearch { embedded => true }'
 ✓ Elasticsearch.yml contains 'script.disable_dynamic: true'
 ✓ Kibana's elasticsearch server is 'http://+window.location.hostname+:9200'
 ✓ Elasticsearch is listening on port '9200'
 ✓ Elasticsearch is listening on port '9300'
 ✓ Kibana is listening on port '9292'
 ✓ Elasticsearch is reachable at '/_status'
 ✓ Kibana dashboard reachable at '/index.html'

9 tests, 0 failures

The workflow definitely needs to be tweaked. I more or less pushed up a minimal viable test suite to get as much coverage as possible. A better strategy would be to execute the test suite in the fig.yml file.
Let me know if anything else comes up. Thanks for the help. I really appreciate it.


Fix GitHub Actions failures due to redirection to macos-10.15 (close #583)

@paulboocock FYI: I've merged it directly as there are just minor changes in the workflow needed to fix failing builds.


using Github Actions as alternative ci

Do you have plans for using Github Actions as alternative ci in future? If yes, can I prepare PR with ci file and implementaion travis ci in github actions ci?

Ok, i try to implement it

https://github.com/spiral/roadrunner/pull/200


Need sample for HDKF (SHA256) using latest OpenSSL

I am trying to use HKDF API from OpenSSL v3.1 (Source which i build from master branch).
I am trying to build [sample] (https://wiki.openssl.org/index.php/EVP_Key_Derivation ) from this site.
Certainly this is because of incompatibility between OpenSSL latest and old sample.
I could not find any alternative sample for new API for same.
Can you please help me with sample or how can i use HDKDF with latest OpenSSL
I am using VS2017 and Windows environment.

I've fixed the example at the Wiki page to make it work. It was just a trivial change - addition of the NULL params argument to the EVP_KDF_derive() function.

I am sorry, still I am facing lots of compile errors

Unable to include #include <openssl/params.h>


About epic games rdr2

Hello,
I installed rdr2 inside epic games and when i try to launch, it says your gpu driver is old, you need have to update it. (steam deck)
Can you help about it?

@satchelkartalmis I have no idea, sounds like it can be fixed through a launch options command maybe -vulkan or -force-vulkan

Hello @moraroy,
I have to do it directly from the steam launcher, right?

@satchelkartalmis OK then put it in the end of the launch options

@satchelkartalmis maybe try this? https://www.reddit.com/r/SteamDeck/comments/y6gc67/solution_red_dead_redemption_2_epic_games_version/

@satchelkartalmis also maybe try updating proton GE to the latest 9-1 see if that helps

Any news on here?
I run into exactly same issue :(
Tried with Variable
WINDLLOVWRIDE="volkan-1=n,b"
also with different Proton Versions.
But the Game won't start :(
Here is a screenshot of my error Message.
Any help?
I really want to play RDR2 on my SteamDeck :(

@Kamikaze01git unfortunately I don't know how to fix this, I have the same game except I bought it from steam and it runs perfectly fine with the latest proton GE. So maybe the problem lies in the epic games launcher somehow not communicating to the Rockstar launcher that the graphics are ok... just got a wild idea though.... maybe you could download Rockstar Games Launcher using NSL, from there grab the launch options, aka the compatability data command, and go into the epic games launcher and go to the settings... and on the bottom lists command lines for each game you have installed... maybe from there.. you can paste the compatdata command to point to the new Rockstar launcher? I'm not sure if that would work but it's worth a shot!

Hello @moraroy
Thank you for your quick response and help.
I also downloaded the RockstarLauncher with the NSL anyway. And I have already tried to add the variable “WINEDLLOVERRIDE” to the RDR2 game data in EPIC.
What could a command look like to use the other RockstarLauncher?!? Unfortunately I don't know enough about this :(
I already own the "normal Version" RDR2 regularly via the Rockstar Launcher. So every time I start the launcher, I have the game ready to install. But as told, this is the "normal" version... Because it wasn't possible to play via the SteamDeck (I've realy tried a lot, downloading many Proton versions and the 120GB game data very often, etc...) I have bought the "Ultimate Version" of RDR2 again via EPIC.
And now that doesn't work either :(
My last hope was your work with this truly wonderful script. But apparently the big companies just don't want to work together and are giving us customers a very, very unpleasant experience.
I'm not going to buy the game a third time on Steam (especially since it's disproportionately expensive there and I already have two versions oft the game (normal via RockStar and the ultimate version via EPIC).
More than frustrating!!
But your wonderful work is innocent of that! Thank you for your effort and the great script! Hang in there and keep going. Extremely many SteamDeck players love your work ;-)

Hello @moraroy
Thank you for your quick response and help.
I downloaded the RockstarLauncher with the NSL anyway.
When i start it, the R* launcher recognise the game ask me to start the game via EPIC !! This is great and a way better then without your script (wher game and launchers are in seperate proton folders).
When i do so and start the R* launcher, which then starts the Epic launcher, which then starts the game (LoL), IT REALY STARTS.... but crashed at loading and freezes my whole Deck (LoL again).
This way it will look to my frieds as i am just playing "R* Launcher". And additional the Steam Overlay ist just for the R* launcher.
My last hope for normal playing was your work with this truly wonderful script. But apparently the big companies just don't want to work together and are giving us customers a very, very unpleasant experience.
I'm not going to buy the game a third time on Steam (especially since it's disproportionately expensive there and I already have two versions oft the game (normal via RockStar and the ultimate version via EPIC).
More than frustrating!!
But your wonderful work is innocent of that! Thank you for your effort and the great script! Hang in there and keep going. Extremely many SteamDeck players love your work ;-)

@Kamikaze01git if your steam deck is freezing like that, you are running out of ram, I recommend that you install cryoutilities and do all the recommended settings, including a 16 GB page file, this will help allot. Also don't forget to do the 4GB virtual ram in the bios. Pet me know if that fixes your issue

Hello and thank you again for your help and effort.
There seem to be a lot of people having trouble with this game on SteamDeck.
Thanks to your help and a helpful post on Reddit, I have now managed to get the game running stably.
It's an ugly workaround, but at least I can play it.
First: If i just install the RDR2 game via Epic, it installs the R* launcher at the first start and then wants me to update the GPU driver... this doesn't work.
Second: If i install the R* Launcher as non-Steam and then install the EPIC game separately, the R* Launcher cannot find the EPIC installed game... this doesn't work.
BUT:
Because your wonderfull script creates all launchers in a unique proton prefix folder, in this way I can start the R* Launcher, which recognizes the game installed via EPIC and starts the EPIC Launcher. This in turn then starts the game and the R* Launcher no longer asks for an update of the GPU drivers,... but crashes at loading.
So this doesn' work either (but we are near... ^^)
AND NOW:
With the help of Cyro-Utilitis (recommended settings) the game now runs stable.
For my Steam friends I just play "Rockstar Launcher" all the time... and the Steam overlay is also only for the launcher and not the game. Additional the SteamDeck shows me that I have 3 (!!) windows open (the R* Launcher, the EPIC Launcher and the game).
 But as I said - the main thing is that I can play it 👍
Maybe I'll jump on the next Steam sale and buy the game a third time so I can play it directly via Steam without any worries... :(
Or maybe Valve will provide us with new GPU drivers, so i can directly start it via EPIC.
So be it! THANK YOU SO MUCH FOR THE HELP AND A GREAT JOB!!!
I have to go and play RDR2 now... hehe


Validate the artifact(s) download URL's

Automate the validation of download Url's for OpenSearch and Opensearch-Dashboards during Opensearch releases.
-Verifies if the Tar, Rpm and Yum distributions can be downloaded from the Opensearch website.

Closed by https://github.com/opensearch-project/opensearch-build/pull/2725


completion on git aliases not working

The following does not work :
gpo fea<TAB><TAB>
while this work :
git push origin fea<TAB><TAB>
Any ideas ?
thanks

tested it in a vagrant machine for debugg purpose & everything seems alright.
The behavior above seems to be an anomaly


Renaming Items causes Recent Actions to stop updating/not work

If you rename an item, the (technically) old item stays in the Recent Actions list, but clicking on it gives you the popup: "Your controlled Actor does not have an item named _____". This is fine, but for some reason using the renamed item by going into your character sheet and clicking on it does not cause it to update the Recent Actions list.
So for instance, if I change an item from the name Longsword to Greatsword, Longsword will remain in the Recent Actions list, and going into the sheet and clicking on Greatsword, even a bunch of times, will not cause the Recent Actions list to update with it. Other items will update it, but it even getting Longsword to eventually cycle off the list will not replace it with Greatsword ever.

Combat booster does not use item names - those are only for rolling. the recent items are stored by ID so i'm not even sure what your issue is. It's not happening on my end. Did you follow the instructions and test with all other mods disabled?

Yes, I'm 100% certain no other modules are running and the error is still happening.
https://user-images.githubusercontent.com/72000497/152654737-a4be3df0-ff8f-4d95-b690-faef60ec187e.mp4

I have the same issue : attack with weapons are not showing up in the HUD

made some changes that should fix these issues


./datasets/ILSVRC2012/images

how to get images train file?

The dataset we used is ImageNet, you can apply for the dataset from http://www.image-net.org/ for academic research.


Passive event handler

Could you make the scroll event handler passive? It would make the page more responsive and get rid of the [Violation] Handling of 'wheel' input event was delayed for 283 ms due to main thread being busy. Consider marking event handler as 'passive' to make the page more responsive. warning.
https://developers.google.com/web/updates/2016/06/passive-event-listeners

Sorry for delay.
Thanks and good advice!
Maybe you can use throttle or debounce to relieve the problems temporarily, and I'll enhance the project as soon as i can in the future.

Already did that. Thank you for the future enhancement.
Never mind about the delay.

Any news on this?

Sorry for delay replying. I've no time to consider to enhance the project recently.
It's the best if you can open a pr. I'll appreciate it. : )


Fix cherry-pick and put files in right place

commit 18ad46297dcd44c1ac6c9f7f2f966e19c4d45e91 ended up with some files in the top-level doc directory, not the right subdirectory.

thanks!


device/disk: Don't use RBD for FS custom volumes

This fixes an issue where a filesystem custom volume on ceph would fail
to be added to a VM as filesystem volumes would falsely use RBD support.
With this commit, only block volumes will use the RBD support, and
filesystem volumes will be mounted and passed into the VM through
9p/virtio-fs.
Fixes #11351
Signed-off-by: Thomas Hipp<EMAIL_ADDRESS>

LGTM, thanks.
Please can you update https://github.com/lxc/lxc-ci/blob/master/bin/test-lxd-storage-volumes-vm to test for this change. Thanks

https://github.com/lxc/lxc-ci/pull/707