DID AuthN for the Interop Project (due Aug 11th)

Please note that the DID AuthN review period will end by August 11th. The goal is to decide whether to use the SIOP spec in the course of the interop project.
The current proposal/ paper can be found here: https://github.com/decentralized-identity/papers/blob/master/did-authn/siop/did-authn-siop-profile.md
This will allow us to bring the decision to the next interop call.
If no substantial objections will be brought up and we can reach consensus, we will use the DID AuthN SIOP profile.

I strongly support this proposal, it would be a major milestone in interoperability just to implement this part, and because of the success of OIDC, this is the natural starting point for integrations with legacy systems.

A few questions about this:

I don't see any reference to verifiable credentials in this and assume
that is out of scope - e.g. that this is limited to DID authentication.
Not raising this as issue but just to clarify the use case.  Assuming that
is the case, is there any thought to how verifiable credential
authentication could be used? I'm guessing that would be a custom
post-authentication flow by the RP?

It looks to me like this must use a publically resolvable DID, and
hence, did:peer could not be used as the authenticated DID. Is that correct?

I think this does support the use case of the authenticating party
creating a DID for every site for which they are using this kind of
DID-Auth. On first visit, the user (via the Identity Wallet) would create a
new DID (or would have a set of unused ones available) and authenticate
with that. On subsequent accesses to the same site, they would use the same
DID each time.  Is that correct?

What needs to be implemented for this to work?  Since as noted most OIDC
clients don't implement SIOP, is the effort going to be take an existing
OIDC client and add SIOP support?  Again, not an issue - I'm just curious.

Thanks
On Mon, Aug 5, 2019 at 8:55 AM Orie Steele<EMAIL_ADDRESS>wrote:

I strongly support this proposal, it would be a major milestone in
interoperability just to implement this part, and because of the success of
OIDC, this is the natural starting point for integrations with legacy
systems.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/decentralized-identity/interop-project/issues/9?email_source=notifications&email_token=AAHYRQRTUBIEFJBYNDLPS7LQDBEOJA5CNFSM4IJMFQUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3SIIIA#issuecomment-518292512,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHYRQU5VT7CXDB5ZAA7YQTQDBEOJANCNFSM4IJMFQUA
.

@swcurran these are great questions. See my answers/ comments below:

I don't see any reference to verifiable credentials in this and assume that is out of scope - e.g. that this is limited to DID authentication. Not raising this as issue but just to clarify the use case.  Assuming that is the case, is there any thought to how verifiable credential authentication could be used? I'm guessing that would be a custom post-authentication flow by the RP?

Yes, for DID AuthN this was considered to be out-of-scope. But I guess, we have to address this for the interop project as well @OR13 ? While nobody would prevent people from using the flow to include self-attested claims (in the same fashion as OIDC) or W3C Verifiable Presentations in the id_token, I'm personally in favour of defining a sub-protocol for credentials exchange for service endpoints. After the RP gets the  and the DID Document, one could obtain the service endpoint and start the credentials flow. Does this make sense?

It looks to me like this must use a publically resolvable DID, and hence, did:peer could not be used as the authenticated DID. Is that correct?

Please correct me if I was wrong, but as far as I understood, the did:peer requires to exchange the DID Document in the DID Exchange protocol. This is supported in the SIOP flow. We have a did_doc or did_comm claim in the  to include the DID Document or potentially other information that is required. I recognized that there are some unintended gaps in the spec for non-public DIDs, e.g., no validation rules if the DID Document was included in the response; claim format of the DID Document in the response. If we addressed these gaps, would the spec work for peer DIDs?

I think this does support the use case of the authenticating party creating a DID for every site for which they are using this kind of DID-Auth. On first visit, the user (via the Identity Wallet) would create a new DID (or would have a set of unused ones available) and authenticate with that. On subsequent accesses to the same site, they would use the same DID each time.  Is that correct?

Yes, that is correct. That should be fine, right?

What needs to be implemented for this to work?  Since as noted most OIDC clients don't implement SIOP, is the effort going to be take an existing OIDC client and add SIOP support?  Again, not an issue - I'm just curious.

It is a simple challenge/response protocol based on JWT. IMO, the implementation could be very slim. Indeed, the adoption of SIOP is not very big, as outlined in the "8 OIDC Considerations" section. I believe supporting OIDC makes sense and could help in increasing the adoption of SSI. Otherwise, we would have to fight slow adoption as OIDC had to do in the beginning. We would have to convince OIDC RP providers like Auth0, or other help/ contribute to OIDC RP implementation providers (e.g., https://www.npmjs.com/package/openid-client) to natively support that flow in their libraries.

Thanks for the note, Oliver - much appreciated.  We've had a number of discussions about this with Drummond (@talltree) in Victoria this week.  A couple of thoughts to consider:

I recommend that the VC use case be thought through a bit.  This is a necessary component for our use case (for our so called "Lawyer Use Case" we are building) and at this point, we think it's pretty hard to use SIOP with that without extending the RP to be a proper verifier, including the mapping of the verified claims into the OIDC token.  If the VC handling is post-authn, then it again means putting a SSI agent capabilities into the RP. In that case, we prefer the idea of just using an OP that talks OIDC to the RP and DIDComm to the Identity Wallet.

We think there might be more than planned complexity in the use of a non-public DID. Specifically, either the key for the DID could not be rotated, or there would need to be support in the data exchange for rotating the key during an authn interaction. Without one of those, a third party finding out the DID (the string) being used could authenticate as the user with a different keypair.

@swcurran The VC Format is tracked here: https://github.com/decentralized-identity/interop-project/issues/5
It sounds like what we are both wanting is claims about an identity and some id_token for that identity, you are suggesting that it would be nicest if those claims came in the id_token.
Its 100% correct that if the OP attaches claims to an id_token they must verify them first, there is a standard interface for doing this in OpenID:
https://openid.net/specs/openid-connect-core-1_0.html#DistributedExample
However, SIOP only provides an id_token, and its probably not a good idea to try and add distributed claims to it without showing it working first.
With the id_token, an entity can be authenticated, and provide whatever credentials they want, in whatever format they want, JWTs, JSON-LD, etc.. the verifier will use the id_token to authenticate and the credentials to make authorization decisions.
In the future if SIOP supports Distributed Claims, perhaps these can be combined, but the claims would have to be in OIDC format.
This proposal is to provide an end to end implementation of SIOP for use with the interop project.
I want to unpack:
"In that case, we prefer the idea of just using an OP that talks OIDC to the RP and DIDComm to the Identity Wallet."
SIOP is for authenticating a DID, the use of the term identity wallet is again confusing things here.
I think what you are saying is that you want to exchange credentials with DIDComm. If thats the case, then this proposal will work for you, especially if SIOP does not support Distributed Claims.
If SIOP supported Distributed Claims and they were provided by DIDComm that sounds like your ideal solution.
I'd like to understand more about all the places that DIDComm could be used, can we get a ticket opened that tries to solve all the interop challenges with DIDComm? That would help me understand it a bit better.

@swcurran It sounds like you might have some valuable insight for this issue as well:
https://github.com/decentralized-identity/interop-project/issues/6
Especially about how DIDComm might be used to request credentials.

@OR13 - I thought this group was using the term identity wallet, hence my use.  What term is being used for the thingy that the user has that has the private keys of the DIDs and (optionally) the credentials?  Anyway, that's what I meant.
I'm not sure of the relevance of the cited example. For distributed claims, it seems to be using a "call home" model where the claims from issuers are retrieved by contacting the issuer. That's a model we're not using.
Regard the comment I think what you are saying is that you want to exchange credentials with DIDComm.  If thats the case, then this proposal will work for you, especially if SIOP does not support Distributed Claims.
I don't think that's right.  What we want is an RP that ONLY talks OIDC to be able to use/trust verifiable credentials. Based on the examples we've seen, we think the best way to do that is to create an OP that talks to the RP using OIDC and talks to the Holder/Prover (the thing I don't know how to name) with DIDComm since that is it's intended purpose.  That allows us to introduce verifiable credential based-authn/authz without changing the RP, other than to add configuration. The OP is where the verification of the VC occurs and so must be trusted by the RP - hence likely on the same infrastructure/controlled by the same entity (in our case, run on BC Gov HW/run by BCGov ops).
A question is whether this thread impacts the Interop Project and the use of SIOP. I'm guessing not, but for us SIOP is not great choice because isn't supported by the IAM vendors, and even if it was, it doesn't support VCs easily, and certainly not without extensions that will be hard for the OIDC community to move on (notably - being a verifier). We think that a dual-personality OP will get us further faster, and for the long term.

@swcurran I feel like we are saying the same thing :)
what we want is an RP that ONLY talks OIDC to be able to use/trust verifiable credentials.

You can have that, the RP checks id_token, and then uses DIDComm to use/verify credentials.
Its up to the RP to manage the linkage if credentials and auth are split, in other words:
If id_token does not contains claims, then RP needs to match sub from id_token to claims (whever they are stored).
We think that a dual-personality OP will get us further faster, and for the long term.

Does this mean you wish to obtain an id_token for a DID which contains VC, because the OP is a verifier?
Does DIDComm plan to integrate with OIDC?

It seems like both OIDC SIOP and DIDComm are defining challenge response based protocols that rely on URIs and JWTs.
Here is a credential exchange spec for did comm: https://github.com/hyperledger/aries-rfcs/tree/master/features/0023-did-exchange
I propose the following:

DIDComm and OIDC SIOP agree to an id_token only auth flow based on JWTs, URIs and QRCodes.
We document how RP can leverage this id_token with VC.

In order to do this, we need to understand how DIDComm plans to handle auth, how it leverages existing authN technology, whether it supports JWTs, OIDC.
If most users of DIDComm prefer a custom id_token that is not OIDC compliant but contains VC and enables faster development, I think that is excellent, but for the purposes of the interop project we need to separate auth and credential exchange in order to support all the different member organization credential mgmt systems.... Hubs, Agent, Wallet Apps...
Put another way...
The interop issuer wants:

to authenticate DIDs
2.to deliver credentials to them.

depending on the services listed in the DID, the issuer should attempt to deliver credentials in different ways.
While its easy to support authenticated credential exchange if everyone agrees to a protocol for it, that seems harder to accomplish then splitting auth and delivery of credentials, especially if we want to support OIDC.

Not sure where to take this convo, but this isn't working :-).  Perhaps on a future call.  Bottom line for us is that SIOP is not particularly helpful given the current state of IAM and the difficulty in using VCs in the authn process.
To clarify your confusion above (I think using the term RP but planning to not support OIDC is confusing me) - from the perspective of the RP, in the approach we are planning, the only protocol will be OIDC.

The core contract of OIDC is that the RP trusts the Provider to make assertions about the party that's trying to authenticate, on the Provider's own authority. When I login to a website with Google or Facebook, the website is trusting Google or Facebook to A) assert that I'm the legitimate owner of an identity they should trust; B) assert on their own authority attributes like my email address or name.
SIOP doesn't change this. However, it does change what are reasonable assertions for the provider to make, because the provider is now the same as the party being challenged--all assertions are now self-asserted, and what the provider asserts on their own authority is now suspect. It turns out that it's fine to self-assert that you control a DID, because this can be trivially proven. And it's fine to self-assert your email address, in many use cases. But it's not fine to self-assert that you have a valid passport or driver's license. Yes, those fact can be proved--but not with the simple token-granting mechanism that OIDC offers. To prove something complex like that, you'd have to tunnel sophisticated, possibly interactive exchanges of nonces, requests for proof, and offers to respond, through a channel that wasn't built for it. And unless the channel has cryptographic guarantees that are just as strong as the ones that created the credentials, you might degrade trust by doing so. Exchanging credential-based proof is a job for a different protocol, not for OIDC. Remember that the goal of OIDC--the very reason it was created--was to authenticate (produce a binary yes/no about whether someone matches identity), not to facilitate a rich give-and-take with nuances and a potentially large corpus of evidence flowing.
Now, in a use case where an institution has backend systems, and access to all of them is governed by LDAP or AD or something similar, it can make total sense to tackle the question, How could we use credentials to grant people access? But think about the dynamics: there's no risk involved in the institution trusting the OIDC Provider, because that provider is their own server. If they want to configure that server to engage in a super complex credential flow, or a trivial one, they can--it's just a configuration and trust choice that they make. The provider is inside their own sovereign domain; they can therefore trust all of its assertions. The provider is free to base those assertions on the results of a sophisticated, DIDComm-based protocol that involves credential exchange. But the relying parties don't care and don't need to know. They just need to ask the provider to do the right thing. I think this is @swcurran 's use case, and it makes total sense.
This doesn't work with SIOP, because there's no good way to do reach evidence exchange, and even if there were, the burden of evaluating that evidence now falls on the RP.
Bottom line for me: The SIOP approach is interesting. It solves a particular problem, if the standard for authentication is as simple as proving you control a DID. But if the standard for authentication involves proof with verifiable credentials, I think the other approach--where the Provider is a server run by the same org as the RP--is required. They solve different problems, and we shouldn't try to fit square pegs into round holes by conflating them.
@swcurran @tplooker

@OR13, perhaps viewing this repo https://github.com/bcgov/vc-authn-oidc will provide more clarity to the approach we are taking with VC authn? As Daniel eloquently described above we are building an OP that supports vc authn over OIDC. So that existing RPs that are OIDC clients can just ask an OP that supports vc authn to use that as the method of authentication. The OP in this case acts as the verifier of credentials from a holder and uses didcomm as the messaging protocol to achieve communication with the holder. The OP then takes the information disclosed in the verifiable presentation and returns an ordinary id token back to the RP

Thanks @dhh1128 - way better explanation than I was able to provide.  A slight addition is that the simple case - prove DID ownership - is pretty easily done with the approach proposed in https://github.com/bcgov/vc-authn-oidc - although we might need to add a new DIDComm protocol.

@OR13 @swcurran @tplooker @dhh1128 I will try to compare these two approaches and IMO, both approaches serve a different purpose:
DID AuthN SIOP:

Two parties involved, i.e., RP and SIOP
RP does not have to register with the SIOP
Simple challenge/response flow
SIOP does not have to be known to the RP because it uses a standard URL
Backward compatibility with SIOP
Focus is on proofing control of a DID
Does not have to be integrated with every OP vendor, e.g., keycloak, auth0, etc.
RP requires additional business logic to cross-verify with DID Document

VC AuthN OIDC:

At least three parties involved, i.e., RP, OP and User
RP has to register with the OP (dynamic/static)
VC AuthN is a simple challenge/response protocol integrated into the Authorization Code Flow
OP URL has to be known to the RP
Full backward compatibility with all OIDC flows
Focus is on proofing of VCs
Has to be integrated with every OP vendor using it, e.g., keycloak, auth0. Note, OIDC does not have a standard extension point for authentication part. This is out-of-scope of the OIDC spec.
RP does NOT have to implement additional business logic as it won't know any DID was involved at all.

To find a solution for the interop project, we need both:
1 DID AuthN
2 Exchanging VCs or presentations
I think we agreed that we want to decouple 1 from 2.
For 1, we could use SIOP. After doing the SIOP exercise and the conversations that we had, I think it could make sense to have a dedicated OAuth2/OIDC grant type for DID Auth, or even for exchanging the VCs. The new grant type could be very similar to SIOP but without the extra backward compatibility with SIOP, i.e., no sub_jwk required, iss does not have to be selfi-ssued.me etc.
Why even do the extra work of having OAuth2/OIDC? Existing web apps, PWAs, native apps use the Auth2/OIDC approach and respective libraries. Even Apple uses OIDC for their Apple login. Because Apple also mandates their login if multiple login modalities are provided, we should make the life of the devs easier and contribute to RP libs that are widely used.

@swcurran In case the above statements made a different impression, IMO the OP authN DID Comm integration that you are doing is very valuable in case you keep it open to all types of VCs.

@awoie I (strongly) agree with virtually everything you summarized. The only thing I would tweak is  in your summary of the VC AuthN OIDC. You describe the RP and the OP as different parties. I agree that they are different servers, but I think they are very often owned by the same org. This is what makes it practical, in the use cases that that approach targets, to assume that the OP and RP are configured in mutually compatible ways, know one another's URIs, etc.
That small tweak has no impact on the larger analysis. I think the SIOP stuff is a great direction for this interop project, for all the reasons you've pointed out. We need both approaches, for different use cases.

Nice job, @awoie!  That captured things well.  I'm not sure that the integration with IAMs is as challenging as it sounds in that list, but we'll let you know with an implementation.
The idea of the separation of Auth-DID/VC from VC presentation is interesting and we've been thinking about it as well.  A big use case for us is using this approach to eliminate user ID/Password in Gov't with VCs across the 1000's of unchanged apps BC Gov operates.  For that, the VC for authn/authz is sufficient.
However, as you and others have pointed out in this issue, a subset of those apps will transition to using VCs (and more generally, DIDComm) for non-authz purposes. For those apps, the connection handling is interesting.  Do the apps use (or perhaps just bootstrap off) the auth-DID/VC mechanism, or do the apps establish their own connection directly with the user's component?  It's an interesting question that we are exploring with this work.  Again - implementing will help us learn.

The due date for this is past. I'm going to assume we have adopted DID AuthN with OIDC SIOP, and we can tackle credential exchange next.

I'm going to close this issue, and open an implementation ticket:
OIDC SIOP Implementation:
https://github.com/decentralized-identity/interop-project/issues/14
Since there has been a lot of credential exchange discussion here, I'm going to open a new ticket regarding credential delivery post AuthN:
https://github.com/decentralized-identity/interop-project/issues/15

Lets continue the discussion for adding specific support for vc-authn-oidc here:
https://github.com/decentralized-identity/interop-project/issues/16


Rancher support?

Rancher has a nice API, would it be possible to add support for it, similar to the other SD configurations currently supported?
It would be cool to be able to configure Prometheus based on Rancher labels for example.

In general, absolutely.
It mostly needs someone who writes the implementation (with guidance of course) and agrees to maintain it in the future.

@jmreicha basic support can be used through the following. It should expose host/service/stack and label metrics so far... it's quite basic at the moment but open to any pull-requests if you have the time.
https://github.com/Rucknar/prometheus-rancher-exporter

That's a rancher exporter (which isn't listed on http://prometheus.io/docs/instrumenting/exporters/ - could you send a PR?), this issue is about service discovery support.

@Rucknar Cool, I will check it out.  Thanks for the update!

@brian-brazil my mistake, will send a PR!

@Rucknar Getting around to testing this I am running in to an issue getting the prometheus-rancher-exporter Docker image to play with the Prometheus Rancher stack but it looks like issues are closed for the project?  What's the best way report and get hlep issues?

@fabxc: I am willing to take this assignment since we are trying to go into production with Rancher & Prometheus and we need a better solution.
I clearly need some guidance - I will try to use consul_sd code as a starting point but I'm not sure if it's enough.
For rancher metadata I am planning to use the go client: https://github.com/rancher/go-rancher-metadata. If additional feature would be needed, I will PR this code also.
Do you have any advice to start working on this feature?

The Go client unfortunately lacks documentation but AFAICS there is no polling for changes. In that case one would just refresh every x seconds. For that case the DNS SD is more suitable as an example than the Consul one. It's also easier.
The biggest challenge will be providing testing.
After accepting SD integrations without tests in the past, due to recent regressions I decided to no longer accept such PRs.
I would say just start implementing as far as you get and open a pull request to walk through missing steps together.

Does anyone have any new about that? I'm thinking about implementing Rancher SD for Prometheus but wonder if there is anybody who already started to do something.

Hello!
I took a different route, please see: https://github.com/brutus333/prom-rancher-sd
The idea was borrowed from https://github.com/DanielDent/prom-rancher-sd which used json file for Prometheus integration.
I have also a rancher catalog which uses this:
https://github.com/brutus333/rancher-extra-catalog
It is not the most elegant solution, but it was an easy workaround.
A real rancher_sd implementation would be clearly better.

@brian-brazil Given rancher 2.0 moving to a kubernetes base, suggest this one is closed off?
DNS service discovery works as-is, kubernetes should be usable once 2.0 is released.

That sounds reasonable to me.


Allow hexadecimal string input

It would be useful to copy bytecode directly as a hexadecimal string (for example, from the block explorer) and feed it in from the command line

Hi.  I've moved on from Fuel and am not actively working on this crate any longer.  I have no idea if it still works, TBH.  So, as they say, happy to take PRs.


[ownership] Convert ValueOwnershipKind to have an Invalid state instead of using optional.

At Andy's request. If it creates too much noise in covered switches, we may go
back to the original.

@swift-ci smoke test

Going to fix Andys comments in a follow up.


fix: compilation on nightly

This fixes a bug when casting a *const _ to *mut _ on nightly, it's being phased out as acceptable and is now a hard error :grimacing:
I could be completely wrong in my fix, but the compiler is happy - also, I noticed in spi.rs that the comment about writing 8 bits vs 16 bits mattered - I couldn't find anything about that in any manual for the f769 (board I have) but I could have easily missed something there.
Any feedback is appreciated!

In RM0385 for example (F74x, F75x) in 32.5.9 it says (emphasis mine):

The handling of FIFOs depends on the data exchange mode (duplex, simplex), data frame
format (number of bits in the frame), access size performed on the FIFO data registers (8-bit
or 16-bit), and whether or not data packing is used when accessing the FIFOs (see
Section 32.5.13: TI mode).

All data to be written is written into the DR register, and it then goes through a small FIFO inside the SPI peripheral before it gets transmitted. For efficiency, when the SPI peripheral is configured to do 8-bit words, you can write 16 bits to the FIFO and it's treated as two sequential words to transmit.
What that means is that performing a strb instruction (store a single byte) will do something very different from strh (store halfword, i.e. 16-bit) or str (32-bit). The svd2rust write() function you call in this PR always performs a 32-bit access, which would cause two bytes to be written to the SPI FIFO. In this case, that means every byte will get send as [byte, 0x00] on the wire. The only correct thing to do is use write_volatile() directly with a *mut u8 to cause Rust to emit a strb. I guess you'll need a more roundabout conversion in this case, like ptr as *const u32 as u32 as *mut u8 or something.

Thanks! But casting it to *const u32 is also invalid as well, the only way would be to first cast as its own type as *mut T, which...feels a bit annoying

Ugh, yea, that is a pain.
It looks like you could do something like (USART::ptr().offset(10) as u32) as *mut u8, though it's a bit annoying. Could replace the 10 with offset_of!(RegisterBlock, tdr) but even so..

that works for me, but that's not great-looking I'll agree, a special note for the relevant spots can be added explaining why that was done

Is there any open issue for that on stm32-rs?

Hi @maximeborges,
I do not see an open issue for it on stm32-rs but it is true that warnings/errors do pop up when trying to build this project.
For example, if I check out the current version and build "cargo build -F stm32f722" I see 3 warnings. Sometimes these warnings are seen as errors, which is confusing. I suppose this merge request is targeting all of those cases.

The fix in https://github.com/probe-rs/hs-probe-firmware/pull/45 looks rather simple. I wonder if it just hides the compiler error? :thinking:
But wouldn't core::ptr::addr_of! and core::ptr::addr_of_mut! to at least ensure, that we can obtain a valid address of the register without invoking undefined behavior?

Hey, sorry about the delay - I was planning to use core::ptr::addr_of, just rebased as well for the USART changes @adamgreig

You should probably change that invocation as well:
https://github.com/stm32-rs/stm32f7xx-hal/blob/08e313a37cc8d0c8698848278285cff09cc91916/src/spi.rs#L439

Thanks!

thanks, sorry for the delay :)


Debugging

This quote "To assist you in plug-in development, the Dynamics 365 for Customer Engagement apps Data SDK contains the Plugin Profiler and Debugger; for more information see [TBD]." seems to have been the case for over a year. Will real debugging be coming soon?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4b16d564-d8b3-6088-fa8c-ceb0434ccc9e
Version Independent ID: 64b9e3cd-3240-9b59-3c69-92c868ae36ad
Content: Custom virtual entity data providers (Developer Guide for Dynamics 365 for Customer Engagement)
Content Source: ce/developer/virtual-entities/custom-ve-data-providers.md
Service: crm-online
GitHub Login: @JimDaly
Microsoft Alias: jdaly

Information about debugging plug-ins can be found here.
https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/tutorial-debug-plug-in
It should apply to virtual entity plug-ins as well as regular plug-ins.
This topic will eventually be moved to https://docs.microsoft.com/en-us/powerapps/developer/common-data-service/virtual-entities/custom-ve-data-providers. I'll update that topic to refer to the debugging topic

I do note that the instructions for using plug-in profiler require selecting a step within PRT. But with virtual entities you do not create a step using PRT but instead using data in the EntityDataProvider and EntityDataSource entities. I don't know for certain whether creating the step registrations in this way will enable to you to launch Plug-in Profiler as described in the instructions.
Please re-open this if you are still blocked.

Hi, I had the same issue and did not see a way to directly debug the custom Virtual Entity Data Provider by hooking the Plug In Registration tool into Visual Studio.
What I have been doing is enabling the Plug In Trace Log inside CRM and writing trace information via C#. e.g. via
ITracingService tracingService = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
then..
tracingService.Trace("Tracing xyz: " + dataToLookAt);
It is a little clunky but helps get to values inside the Plug In Trace Logs.  Would be great to be able to hook in the debugger in future though.

Thanks guys, I have used the trace log to the extent that I can but that gets really messy really fast. I did submit a Microsoft Support Ticket on this issue too, there solution was to simply remove the part of the documentation I quoted above.
Quote from MS:
"I was able to work with the dev and here is what I found out :
Unfortunately that line was a mistake , which has been taken down now . You can check the same link : https://docs.microsoft.com/en-us/dynamics365/customer-engagement/developer/virtual-entities/custom-ve-data-providers  .
However we  have the plans to get the functionality of debugging for “Custom Virtual Entity Data providers” Unfortunately this is long term plan. Apologies for the inconvenience caused."

Hi guys.
Can you lease inform me if the above issue is already fixed? I still cannot debug virtual entity data provider on online v9.0!

Reopening issues since it seems multiple people are blocked by this.

@JimDaly no so much a documentation issue, but rather a platform and tooling issue: CDS debugging needs to be greatly improved. That's a higher level discussion I'm trying to push forward.

The Product suggestion portal is the place to drive that awareness.
This is the documentation feedback channel.
Of course, feature capabilities (or lack of them) get reflected in the documentation.
But the scope of issues here are things that can be addressed via documentation.
This is why we have the two different buttons:

Not saying you shouldn't raise issues here. Just saying that the documentation issue won't remain open awaiting a product fix.

Agree that lack of debugging is an issue for VE and for Custom API plug-ins running in stage 30.
I know that the tools team is aware of this, but it isn't a documentation issue. so closing this.
Please create or vote for ideas on the ideas site: https://powerusers.microsoft.com/t5/Power-Apps-Ideas/idb-p/PowerAppsIdeas


Fixing the issue with the source path for issue.json and checklist.json

Full path to issue.json and checklist.json should be defined to work properly.
Line 556 for hunt_scanner.py and line 161,171 for hunt_methodology.py should be edit by the user with full paths.
Here is the fix:
hunt_scanner.txt
hunt_methodology.txt

@JPmonglis Mind submitting this as a pull request? Thanks.

@swagnetow I'm getting permission errors for some reason , after talking to Jason hes guide me to submitted as an issue.

He gonna merge it tonight as he told me!

Do you mind pasting the error message you're getting? It should be in the Burp error window in Extender.

@swagnetow The error was about issue.json and checklist.json cant be found.
The issue seems to be the use of relative path, after change it to full path everything worked fine

@Chilico I assume everything is working now with the current code? If so, I'll go ahead and close this issue out.

@swagnetow Even with the latest updates I'm still getting the same error
"IOError: (2, 'No such file or directory', './conf/issues.json')"
I have tested on different systems and getting the same error.
The only solution I found which work for me is to add the full path as you can see on my fixes.
If you dont face the same error feel free to close this issue.

Are you by chance using Windows? I feel like that may be the issue.

@Chilico Hey, I updated the code to use absolute paths. Let me know if that works for you now.

@swagnetow No I only use linux , ubuntu and arch specifically and still getting the same error.
If this is only me I'm not sure if worthing the troubleshooting for a single person as I am able to solve the problem for my self.

@Chilico I'll go ahead and close the issue out then. It's weird that you're getting path errors even after changing the code to use absolute paths.


Support custom management commands

Change nanodjango manage ... to check inside the script - maybe we need an @app.command class decorator?
Can we get it to work with django-click as well?
Inspired by https://www.reddit.com/r/django/comments/1hhotil/nanodjango_enthusiasts_assemble/

hii @radiac assign this to me


WIP: test GPU Operator must-gather PR

/test ?

/test gpu-operator-commit

/test gpu-operator-commit

/test gpu-operator-commit

PR got merged, closing
https://gitlab.com/nvidia/kubernetes/gpu-operator/-/merge_requests/358
/close


Use variables from helm env instead of hardcoded values

Hello!
When using helmper on a Mac it uses ~/.cache/helm, but it's configured to use ~/Library/helm. I suggest to use the built-in values from helm env instead of assuming where the files are located.

Hello :wave:
Great suggestions, will look into implementing shortly

This issue should have been fixed in the PR above. I will release v0.1.2 shortly, please confirm the issue is indeed closed :man_technologist:


Sort the settings in the same order in both web and app UI

Please let's sort the pop characteristics in the same order in both the web UI and the app UI.
Please follow the order shown in the data dictionary: https://docs.google.com/spreadsheets/d/1dRYeuq8rPXNcAmqFfXlSeMC1gm_ejsvl76gQM39zxm8/edit#gid=892568965 (see Pop Charact. tab)

This order seems a little arbitrary to enforce all of the time. I was wondering in my other issue whether the web can just have sortable columns like the other admin tables.


Add influx CLI 2.1 release notes

Closes https://github.com/influxdata/docs-v2/issues/2939

@danxmoran Concerning the OSS-only changes, will they work with the existing 2.0 or will it only work with the upcoming 2.1.0? Thank you!

@mluu12 they will work with the existing 2.0


unescape paths

Wanted to see your opinion on this.
In a scenario where we use path helpers to generate a swagger path:
swagger_path some_resource_path("{some_id}") do
   ...
end

The desire is for the swagger param interpolation to work as expected. However, a path helper is likely to escape the { and } characters, resulting in %7Bsome_id%7D in the swagger json.
Do you think the fix for this fits rightly in the swagger json generation (e.g. this commit), or is this is a swagger ui (or equivalent downstream client) responsibility?

Why don't you just unescape it before you pass it in?
swagger_path URI.unescape(some_resource_path("{some_id}").to_s) do

@fotinakis That is what I was doing to work around this, but it wasn't very DRY. This seemed like a normal thing a ruby dev might do (use a path helper that very well might escape the params passed to it), so I figured I would pass it up the chain.

Understandable. You could dry it up by wrapping in other methods yourself if you really wanted to. I can see the use case for this, but I don't really want to introduce this behavior as our default for two reasons: 1) swagger-blocks is a Ruby library, and what you're talking about is a more Rails-centric thing, and 2) I haven't heard any other requests for this to be the default behavior. Going to close for now, but if it comes up again I'll come back to this. Thanks for checking!


Checking of KEY_EXCHANGE_RSP's SlotIDParam is incorrect for SPDM 1.2

And maybe 1.1. SPDM 1.2 specification says

Bit [3:0]. Shall be SlotID. Slot number of the Requester certificate chain that shall be used for mutual authentication, if MutAuthRequested Bit 0 is set. The value in this field shall be between 0 and 7 inclusive, or 0xF if the public key of the Requester was provisioned to the Responder through other means. All other values Reserved.
For any other value of MutAuthRequested this field shall be set to 0 and ignored by the Requester.

while SPDM 1.1 says

Bit[3:0] = SlotID. The slot number of the certificate chain of the Requester to be used for mutual authentication. The value in this field shall be between 0 and 7 inclusive, or 0xF if the public key of the Requester was provisioned to the Responder through other
means. All other values Reserved. If MutAuthRequested = 0x00 this field shall be set to 0 and ignored by the Requester.

libspdm currently implements the 1.1 check : https://github.com/DMTF/libspdm/blob/8a5f88b4425539a0b3b5aa22bdd8c91c89fd6068/library/spdm_requester_lib/libspdm_req_key_exchange.c#L494-L519 so it needs to be updated for 1.2. However I think the 1.1 wording is a bug in the specification and that the 1.2 wording should be in 1.1. I'll file an issue against the specification.

Issue has been filed against the specification : https://github.com/DMTF/SPDM-WG/issues/2280

After discussing this with the SPDM working group we will make 1.1.3 the same as 1.2. So there's no need for a version check.


Update VSTest task doc for UWP tests

Task doc at https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/test/vstest?view=azure-devops needs to be updated for UWP.
The following addition to the FAQ section will help.
How can I run tests for UWP apps?
Tests for UWP apps can be run by providing the .appxrecipe input to the test assembly input.
For e.g.,
testAssemblyVer2: |
\Release\UnitTestProject1.build.appxrecipe
!*TestAdapter.dll
!**\obj**
If you use the /p:UapAppxPackageBuildMode=StoreUpload option when building the projects, an additional copy of the unittestproject appxrecipe is generated, which cannot be used to run tests.
Microsoft hosted agents are Server OS and cannot be used to run UWP tests. To run pipelines containing UWP workloads, you must use self hosted agents.

@steved0x - can you help with this, pls? I have included the information about what needs to change above.

@vijayma @PBoraMSFT Thanks for driving this!
Highly recommend getting the documentation in ; spent hours researching online about what and how to do and tried billion of things without success just to find out that you need to do some magic by specificing ".appxrecipe" pattern for mstest build task; yet that is not enough because in some weird scenarios your output would contain multiple ".appxrecipe" files; so then it's even better to specify smth like "Release*.appxrecipe" etc.
But even with these changes there was no easy explanation that if you want to automate testing in VSTS you need a private agent customized with Developer mode on, etc..
Good luck using VSTS Hosted agents like msvc2017-win2016 etc.
My instant reaction was: leave UWP testing aside (although I have everything working on my dev machine) since everything is "magic" and try a cppwinrt test :|
Originally posted by @mateialexandru in https://github.com/MicrosoftDocs/vsts-docs/issues/6240#issuecomment-564612717

Also spent a while to find this. Docs could really use a callout for UWP.


Update support.md

Missing Rev. for uppercover hands. Verify the inserted value

As documented here
we decided not to put revision in ukit pages
I forgot to remove the dedicated columns from the ukit elio did
I'm going to fix this ASAP


Allow to be rename the label of paths

That would be interesting to work with large projects where we have many subprojects, because often the directory name is not necessarily friendly.

The sample

Also it might be required to be virtually naming the folders, to make any sense. See https://github.com/atom/atom/issues/5728
Mr @mrameezraja has the problem he has multiple projects all named "www" he can't add them all to project because they would be all named "www".
Instead the paths should be allowed to have object:
'my solution'
  'paths':
    'my project' : 'C:\some\www'
    'other project : 'C:\other\www'

this way would be beautiful

The jump cat!!!
Make symbolic links with label name

The result is:

More its not beautiful

Interesting idea, but I think the tree view package would have to add some support for this to work.

inheritance and overwritten could not solve this?
Or a pull request with the code for add this feature.

The issue is open in tree-view project
https://github.com/atom/tree-view/issues/453

@danielbrodin can you see this project https://github.com/subesokun/atom-project-view

A related feature seems to be also discussed on the main Atom project.
Otherwise, in addition to the Project View package mentioned above, maybe it could be useful to consider project-alias package, which announces to be able to "rename projects and (shows) a tooltip with the original name when the user hovers with the cursor over a renamed project".


Do not allow for the same scheduler ips to be set in 2 different hosts.

Rancher Versions: v1.4.0-rc8
We are allowed to add same shceduler ip to be set in 2 different hosts.
Ideally this  should be prevented.

@cjellick need to discuss with @cjellick. Not sure what to do with two same private ip.

if it is doable, the fix should not be in the scheduler. Is it possible to prevent it in UI? I think UI knows all the scheduler ips for all the hosts and can prevent user putting the same IP? @vincent99 any idea about this?

While not common, there are cases where the same IP is actually available on multiple hosts (anycast, vrrp).  So it's not clear that we should really enforce anything here.
Anyway if you want validation, that goes in the API; the UI is only one possible client.  (And this is where I again argue that this, like most things, should have been a real API field instead of a label).


Target Instance screen fails to load

Upon a specific configuration of an H3 Target's profile overrides, the database gets into a bad state, and searches of Target Instances that include a TI for this Target, fail and throw this error
org.springframework.orm.hibernate3.HibernateSystemException: exception setting property value with CGLIB (set hibernate.cglib.use_reflection_optimizer=false for more info) setter of org.webcurator.domain.model.core.ProfileOverrides.setOverrideH3DocumentLimit; nested exception is org.hibernate.PropertyAccessException: exception setting property value with CGLIB (set hibernate.cglib.use_reflection_optimizer=false for more info) setter of org.webcurator.domain.model.core.ProfileOverrides.setOverrideH3DocumentLimit

Haven't managed to reproduce, but have narrowed down what is causing the error. A Target Instance that ran and completed on an H3 harvester, is somehow linked to an H1 profile and profile override. In the attached spreadsheet it is Target Instance 22052864. If I update the profile override ID in the Target Instance row and set it to another one used by an H3 Profile, then the error goes away.
wct_v2_wrong_profile_override.xlsx
I'm not sure how it got into this state, but I was doing a lot of testing on this one Target Instance, updating the profile overrides, switching between H3 and H1 and doing test crawls. I think its best to park this for now unless it occurs again.


Highlighting of module names

Would it be possible to add highlighting for the module and end module name such that it is the same as with the use command?
That would make it consistent with subroutine and call.

You mean something like this?

Instead of this...

Yes, that's exactly it.

Added a Pull-Request #53 to fix this minor highlight issue. Will close the issue as soon as the fix is in master branch.


Incorrect rendering of the sequence یتے in Arabic Typesetting

In the font Arabic Typesetting, the sequence یتے is rendered as نتے :

Screenshot from Microsoft Word for Microsoft 365 MSO (Version 2210 Build 16.0.15726.20188), on Windows 10 21H1 (Build 19043.1706)

Thank you reporting this. Confirm it is a bug in Arabic Typesetting font.


[ios][engine][impleller]crash on entity_pass.cc assertion failure

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce

pull latest main branch
compile engine

gclient sync -D && ./tools/gn --ios --unoptimized --simulator --no-goma --runtime-mode=debug --simulator-cpu=arm64 && ./tools/gn --unoptimized --no-goma --runtime-mode=debug && ninja -C ../out/ios_debug_sim_unopt_arm64 && ninja -C ../out/host_debug_unopt/

under framework project flutter/dev/integration_tests/ios_platform_view_tests, run
 flutter build ios --local-engine ios_debug_sim_unopt_arm64 --local-engine-host host_debug_unopt --config-only

then open in xcode and hit run

 open ios/Runner.xcworkspace

choose "Platform View Focus Tests", tap on the "Flutter Text Field", and make some changes. Observe the crash.

Expected results
Should not crash
Actual results
Crash in impeller:
[FATAL:flutter/impeller/entity/entity_pass.cc(812)] Check failed: clip_coverage_stack.back().clip_depth == clip_coverage_stack.size() - 1.
Code sample
Code sample
[Paste your code here]

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel main, 3.16.0-11.0.pre.80, on macOS 13.6 22G120 darwin-arm64, locale en)
! Unknown upstream repository.
Reinstall Flutter by following instructions at https://flutter.dev/docs/get-started/install.
[!] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
✗ cmdline-tools component is missing
Run path/to/sdkmanager --install "cmdline-tools;latest"
See https://developer.android.com/studio/command-line for more details.
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[!] Xcode - develop for iOS and macOS (Xcode 15.0)
✗ CocoaPods not installed.
CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on
the Dart side.
Without CocoaPods, plugins will not work on iOS or macOS.
For more info, see https://flutter.dev/platform-plugins
To install see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2023.1.2)
[✓] VS Code (version 1.83.0)
[✓] Connected device (5 available)
! Error: Browsing on the local area network for Huan’s iPad Pro. Ensure the device is unlocked and attached with a
cable or associated with the same local area network as this Mac.
The device must be opted into Developer Mode to connect wirelessly. (code -27)
[✓] Network resources

looks like it's fixed by https://github.com/flutter/engine/pull/46912 which landed 4 minutes ago. good timing.


Error during egen build on IBM power ppc64le

dbt5-build-egen tool is throwing error while building the egen. Followed instructions in https://github.com/osdldbt/dbt5/blob/main/doc/user-guide.rst to build the tool
dbt5-build-egen /tmp/egen
ERROR: missing 'series' file in '/usr/share/dbt5/patches', try "dbt5-build-egen --help" for more information.
tried another way but I see different error
dbt5-build-egen --build-only /tmp/egen
../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::SECURITY_ROW]': ../src/../inc/SecurityTable.h:143:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::SECURITY_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:58,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETradingFloor.h:50,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/TableRows.h:302:16: note: 'struct TPCE::SECURITY_ROW' declared here  typedef struct SECURITY_ROW                 ^~~~~~~~~~~~ In file included from ../src/MEETickerTape.cpp:42: ../src/../inc/MEETickerTape.h: In constructor 'TPCE::CMEETickerTape::CMEETickerTape(TPCE::CMEESUTInterface*, TPCE::CMEEPriceBoard*, TPCE::CDateTime*, TPCE::CDateTime*, const TPCE::DataFileManager&)': ../src/../inc/MEETickerTape.h:81:25: warning: 'TPCE::CMEETickerTape::m_pCurrentTime' will be initialized after [-Wreorder]      CDateTime*          m_pCurrentTime;                          ^~~~~~~~~~~~~~ In file included from ../src/MEETickerTape.cpp:42: ../src/../inc/MEETickerTape.h:70:36: warning:   'const StatusTypeDataFile_t& TPCE::CMEETickerTape::m_StatusType' [-Wreorder]      const StatusTypeDataFile_t&    m_StatusType;                                     ^~~~~~~~~~~~ ../src/MEETickerTape.cpp:84:1: warning:   when initialized here [-Wreorder]  CMEETickerTape::CMEETickerTape( CMEESUTInterface* pSUT, CMEEPriceBoard* pPriceBoard, CDateTime* pBaseTime, CDateTime* pCurrentTime, const DataFileManager &dfm )  ^~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETradingFloor.h:50,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::DAILY_MARKET_GEN_ROW]': ../src/../inc/DailyMarketTable.h:138:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::DAILY_MARKET_GEN_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_stdafx.h:71,                  from ../src/../inc/MEEPriceBoard.h:47,                  from ../src/../inc/MEETradingFloor.h:53,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/DailyMarketTable.h:67:20: note: 'struct TPCE::DAILY_MARKET_GEN_ROW' declared here      typedef struct DAILY_MARKET_GEN_ROW                     ^~~~~~~~~~~~~~~~~~~~ In file included from ../src/MEETickerTape.cpp:42: ../src/../inc/MEETickerTape.h: In constructor 'TPCE::CMEETickerTape::CMEETickerTape(TPCE::CMEESUTInterface*, TPCE::CMEEPriceBoard*, TPCE::CDateTime*, TPCE::CDateTime*, TPCE::RNGSEED, const TPCE::DataFileManager&)': ../src/../inc/MEETickerTape.h:81:25: warning: 'TPCE::CMEETickerTape::m_pCurrentTime' will be initialized after [-Wreorder]      CDateTime*          m_pCurrentTime;                          ^~~~~~~~~~~~~~ In file included from ../src/MEETickerTape.cpp:42: ../src/../inc/MEETickerTape.h:70:36: warning:   'const StatusTypeDataFile_t& TPCE::CMEETickerTape::m_StatusType' [-Wreorder]      const StatusTypeDataFile_t&    m_StatusType;                                     ^~~~~~~~~~~~ ../src/MEETickerTape.cpp:100:1: warning:   when initialized here [-Wreorder]  CMEETickerTape::CMEETickerTape( CMEESUTInterface* pSUT, CMEEPriceBoard* pPriceBoard, CDateTime* pBaseTime, CDateTime* pCurrentTime, RNGSEED RNGSeed, const DataFileManager &dfm )  ^~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETradingFloor.h:50,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::NEWS_ITEM_AND_XREF_ROW]': ../src/../inc/NewsItemAndXRefTable.h:122:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::NEWS_ITEM_AND_XREF_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_stdafx.h:76,                  from ../src/../inc/MEEPriceBoard.h:47,                  from ../src/../inc/MEETradingFloor.h:53,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/NewsItemAndXRefTable.h:57:20: note: 'struct TPCE::NEWS_ITEM_AND_XREF_ROW' declared here      typedef struct NEWS_ITEM_AND_XREF_ROW                     ^~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETradingFloor.h:50,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::LAST_TRADE_ROW]': ../src/../inc/LastTradeTable.h:95:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::LAST_TRADE_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:58,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETradingFloor.h:50,                  from ../src/MEETradingFloor.cpp:42: ../src/../inc/TableRows.h:266:16: note: 'struct TPCE::LAST_TRADE_ROW' declared here  typedef struct LAST_TRADE_ROW                 ^~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::COMPANY_ROW]': ../src/../inc/CompanyTable.h:126:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::COMPANY_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:58,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableRows.h:114:16: note: 'struct TPCE::COMPANY_ROW' declared here  typedef struct COMPANY_ROW                 ^~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::FINANCIAL_GEN_ROW]': ../src/../inc/FinancialTable.h:196:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::FINANCIAL_GEN_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_stdafx.h:64,                  from ../src/../inc/MEEPriceBoard.h:47,                  from ../src/../inc/MEETickerTape.h:53,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/FinancialTable.h:87:20: note: 'struct TPCE::FINANCIAL_GEN_ROW' declared here      typedef struct FINANCIAL_GEN_ROW                     ^~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::SECURITY_ROW]': ../src/../inc/SecurityTable.h:143:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::SECURITY_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:58,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableRows.h:302:16: note: 'struct TPCE::SECURITY_ROW' declared here  typedef struct SECURITY_ROW                 ^~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_common.h:59,                  from ../src/../inc/CustomerAccountsAndPermissionsTable.h:44,                  from ../src/../inc/TxnHarnessStructs.h:48,                  from ../src/../inc/MEETickerTape.h:50,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/TableTemplate.h: In instantiation of 'void TPCE::TableTemplate<T>::ClearRecord() [with T = TPCE::DAILY_MARKET_GEN_ROW]': ../src/../inc/DailyMarketTable.h:138:25:   required from here ../src/../inc/TableTemplate.h:144:32: warning: 'void* memset(void*, int, size_t)' clearing an object of type 'struct TPCE::DAILY_MARKET_GEN_ROW' with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]      void ClearRecord() { memset(&m_row, 0, sizeof(m_row)); }                           ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from ../src/../inc/EGenTables_stdafx.h:71,                  from ../src/../inc/MEEPriceBoard.h:47,                  from ../src/../inc/MEETickerTape.h:53,                  from ../src/MEETickerTape.cpp:42: ../src/../inc/DailyMarketTable.h:67:20: note: 'struct TPCE::DAILY_MARKET_GEN_ROW' declared here      typedef struct DAILY_MARKET_GEN_ROW

We're using RHEL version 8.9
[root@p182n147 ~]# cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.9 (Ootpa)"
Currently we're using C++ v8.5.0
[root@p182n147 ~]# g++ --version g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)

Ah, I didn't read this closely enough while I was on the road.  I don't believe this is actually compiler related after all:

ERROR: missing 'series' file in '/usr/share/dbt5/patches', try "dbt5-build-egen --help" for more information.

Barring a problem with the install rules, I'm guessing the kit wasn't installed onto the system.  That's ok if you're comfortable just editing your PATH environment.  So hopefully all you need to do is use --include-dir, --patch-dir and --source-dir, with patches/, src/include and src directories, resp.

We're successful in building egen.
When we tried to run the benchmark test
[root@p182n147 run-data]# dbt5 run --tpcetools=/opt/egen -d 600 -u 1 pgsql /tmp/results
# DBT-5

Workload consists of 4 stages:

1. Start of the Brokerage House server
2. Start of the Market Exchange server
3. Test (Start of the Customer Emulator)
4. Processing of results

Results will be found in: /tmp/results

## 1. Starting Brokerage House server

Giving BrokerageHouseMain 1 second to start up.

## 2. Starting Market Exchange server

## 3. Starting Customer driver(s)

* 1 user starting every 1000 milliseconds.

1 user(s) started.

* User ramp up to finish in 2 s.

* Test expected to finish in 600 s.

## 4. Processing data
/root/dbt5/builds/release/dbt5-run: line 703: 3898518 Killed                  /opt/egen/bin/MarketExchangeMain -c 5000 -t 5000 -i /opt/egen/flat_in -o /tmp/results/mee > /tmp/results/mee/mee.out 2>&1
/root/dbt5/builds/release/dbt5-run: line 629: 3898516 Killed                  /opt/egen/bin/BrokerageHouseMain -d dbt5 -o /tmp/results/bh > /tmp/results/bh/bh.out 2>&1

## Primary Metric

7.83 TRTPS

Complete results are in: /tmp/results

in the above output at 4th stage 2 commands are getting killed. Is it ok to ignore or something is wrong with our configuration?

Glad to hear it's building successfully again.
The kill messages are expected. It's not elegant for at least a couple of reasons. So there is an opportunity for improvement. :)
Thus if you are getting meaningful results then it's safe to ignore.

@markwkm we have done couple of runs using DBT-5, We have 2 questions based on our run results

What's the difference between TRTPS vs TPS, if there is a difference how to convert TRTPS number to TPS?
How to change read and write transactions mix ratios using dbt5 command


what is the performance characterstics of template evaluation?

Can this handle high volumes of data processing? At times we do get 100k messages from various different instruments, external internal systems. Is there any measurement or benchmark for this implementation?

@kesavkolla Could you please clarify the context of your question?

The way I understand the code is working based on yaml configuration and dyanmic mapping to create the FHIR resource. Wondering would this scale when there are many many messages that need to be transformed?

@kesavkolla  -- The template is loaded once per instance of HL7ToFHIRConverter. The HL7ToFHIRConverter does not need to be initialized for each request. Also I am planing on adding support for batch Hl7 file in future.


push.on('registeration', callback); gives wrong registration id;

Expected Behaviour
The older deprecated version used to return a registration id aka device token for android device using GCM. The new version should also be returing the similar value on the regitration event.
Actual Behaviour
The current version return the registration id which is quit similar to the older one but with a change that there are some extra characters and digits before the actual id i.e. dKXLm: After the colon, the actual device token starts.
Due to this behavior, the device is unable to receive notifications which it was doing very fine on older version on android devices. (not on iOs)
Platform and Version (eg. Android 5.0 or iOS 9.2.1)
Android 5.1.1
(Android) What device vendor (e.g. Samsung, HTC, Sony...)
Google Nexus 7
Cordova CLI version and cordova platform version
cordova --version                                   6.2.0
cordova platform version android                     5.1.1

Plugin version
cordova plugin version | grep phonegap-plugin-push   1.7

@salman-ghauri this is expected behaviour. The format of registration ID you are seeing follows the Google Instance ID specification that Google wanted everyone to move to. You should still be able to send messages to devices with the full registration ID.

@macdonst that's the thing, when I split the instance Id on ":" and use the actual device token, it works but when I use the full instance Id, it does not. Also I am receiving a single messages three times on the notification event.

@salman-ghauri I can't really debug this as I have no idea how you are sending the message. I'm pretty confident if you use the node-gcm scripts I use as samples in the docs this will work fine.


Side Panel Search/Replace doesn't hit multiple instances on one line

Issue Type: Bug
When performing a search/replace for a directory in VS Code using the side panel and regex, it's only replacing one instance of the searched regex on a line per time it's run.
For example:
I'm wanting to find all instances of where I have a link with a leading slash after the parenthesis ((/) and ends with a .md, I would search for: (/(.*).md)
I want to remove that .md at the end, so I replace with: (/$1)
However when there are multiple instances of this on a line: "Go here or here the search tool only finds and replaces one. You then have to hit "refresh" and search again to find another instance, replace again, rinse and repeat. Shouldn't this find every instance and replace them, even if they're on the same line?
Thanks!
VS Code version: Code 1.23.1 (d0182c3417d225529c6d5ad24b7572815d0de9ac, 2018-05-10T16:03:31.083Z)
OS version: Darwin x64 17.4.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz (4 x 3100)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledvideo_decode: enabledvideo_encode: enabledvpx_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (2.27GB free)

Process Argv
/Applications/Visual Studio Code.app/Contents/MacOS/Electron

Screen Reader
no

VM
0%

Extensions (1)

Extension
Author (truncated)
Version

python
ms-
2018.4.0

Closing this - I think it behaves normally I just goofed on the regex.


fix: Register.java for GetFreeAtsign

Issue: NullPointerException while generating Free AtKey
Solution: Updated class GetFreeAtsign to fix the bug

Hi @JayUpadhyay-8 Thank you for your PR. :)
Would it be reasonable to add result.apiCallStatus... in the catch block above that as well?


partical fix for serializer (Keras 1.1.1)

I spent some time fixing up the serializer. Fully connected layer/relu/softmax now works w/ models exported from Keras 1.1.1. I did not attempt other layers, but the patch should be a good starting point.
Seems the owner is no longer active. But here is the pull request in case anybody wants to save some time.
https://github.com/scienceai/neocortex/pull/12

Worth pointing out that since this looks abandoned; kerasjs may be a better alternative which while still having some issues; is more active.
https://github.com/transcranial/keras-js

Agreed. When I looked at keras-js, it didn't support NPM, so ....


Access proxy: the parameters in the documentation is not consistent with the parameters in CloudFormation

Describe the issue

Links
https://aws-solutions.github.io/centralized-logging-with-opensearch/en/implementation-guide/domains/proxy/#using-the-cloudformation-stack
https://docs.aws.amazon.com/solutions/latest/centralized-logging-with-opensearch/access-proxy-1.html#create-a-proxy

Fixed in Version 2.1


Reform 2.0 breaks ActiveRecord uniqueness validation

The follwing code works with Reform 1.2.6 but fails in Reform 2.0.1 with the message: ArgumentError (Unknown validator: 'UniquenessValidator')
validates :email, presence: true, uniqueness: { case_sensitive: false }

In Reform 2.0.1 I have to write the following:
validates :email, presence: true
validates_uniqueness_of :email, case_sensitive: false

http://trailblazerb.org/gems/reform/validation.html#uniqueness-validation
I will drop ActiveModel support soon, so please keep that in mind when opening tickets.

Damn I'm sorry to hear that, Reform is the best Form object gem out there but I prefer working with the validations I've used for years.
Are you dropping it totally or allowing people to choose which validations they want to use?

Sorry!!
I am not sure if I will keep providing both as people will assume that I support ActiveModel, which I don't. Also, the new features like removing validations in subclasses, cascaded validations, etc. are impossible to implement with ActiveModel, so it's probably best to say "Good-bye!".

BTW, you already put the most sensitive part of an app - validation - in the hands of Reform. Replacing rather primitive validations with another implementation is the part you should be the least worrying part! :stuck_out_tongue_winking_eye: :

Yeah that's totally fair of course. I'll continue using 1.2.6 for now as there are too many breaking changes for now (which is also fair when you bump to a major new version) and reevaluate when I start the next project.


Is it possible to reset value

How can I reset a value which is already selected ?

I just created a new method called "resetValue" in the dropdownController.
You can check in the example.


ES6 support with depcheck-es6

If your code has JSX, or uses stuff that acorn or node-soure-walker cannot
parse, such as static, then it's still going to say that you haven't included
something.
Not sure what the right course of action is, as this is really sort of an
interesting problem for the parsing libraries that we use.
Anyway, if you're using plain ol' ES6 without anything fancy, this should
resolve your imports for you just fine.
#17

Quick note, this follows a fork of depcheck called depcheck-es6 which has ES6 support. After the ES6 support PR sat without response on depcheck for a while, the author decided to fork it and publish it on npm under depcheck-es6.
This PR uses this fork to add ES6 support.

Thanks for doing this @woodb and thanks @lijunle for your continued work on the fork!


Add linting while writing code?

I'm not sure if this issue goes into this repo. Please let me know if it belongs to a different one and I'll be happy to move it.
The idea is that we would get linting warnings and errors while writing code on the notebooks. Initially this could be style agnostic and just show warnings or errors in code that would not compile or that is known as bad practice (typeof null, non-strict comparisons, etc.), but it could also support getting an .eslintrc by fetching or uploading a file.
I'd be more than happy working in getting this feature available.

Hi @dgrcode - yep, this repo probably isn't the best for this conversation. notebook-inspector inspects the output values of Observable notebooks, like allowing you to explore objects and arrays - it doesn't have any role in terms of authoring those notebooks. We have an active forum for other discussions.
Per linters - well, our current focus is on adding prettier, which is nearly complete except for a performance fix that we’d like to implement for prettier. We might spend some more time making some helpful suggestions or auto-fixers, but the philosophy isn't so much about bringing more JavaScript/editor features into Observable than imagining whether we can implement something simpler and more consistent. Prettier, for example, is only available with the default settings and will have no configuration - it’s just on or off.
I think some checkers are useful, but for newcomers to JavaScript and especially for short chunks of code, best practices aren't necessarily worthwhile - if the code runs, it works. Best practices are super useful for larger applications, but often they don't scale down and they have the unfortunate side effect of making projects and such less accessible to newbies.


Improving indoor performance

Hello guys, I've been using LIO-SAM with decent success outdoors, but when it comes to indoors I struggle to find the right params to run it with and get very mixed results, sometimes it's okay but most of the time the odometry is a lot more unstable. I've tried reducing the voxel leaf size as suggested, and I've also played around with the loop closure params and mapping interval but they don't seem to have huge impact. I am curious to find out what the reason is for the performance difference and what makes indoor mapping so much more difficult, but also what would be some ways to potentially improve the algorithm to also work well in indoor environments as it does outdoors. Thanks in advance!

Hi Jaetriel,
What is your IMU rate? Me and my team were having trouble using this algorithm @55Hz, it became unstable very easily. Hopefully, we could increase to 200Hz and it all goes better now. Do you have videos of the unstabilization? Here are results of our indoor output:

@Huguet57 Hi, thanks for your answer. My IMU runs at 500hz raw data output, it can go up to 1KHz but I didn't see much improvement going from 500 to 1khz. My main issues right now are scanning smaller indoor rooms, it seems like the odometry isn't as good and starts becoming unstable and inconsistent, sometimes it drifts or tilts, sometimes even crashes if I rotate or move the sensors too quickly indoors. I will attach a few screenshots of maps we have, I might try to get a short video as well if I can find the time.

Hi both,
Have you checked changing the parameters at lines 62-65 of config/params.yaml? Those are what helped me the most going from outdoor to indoor.
We use a Vectornav VN-300.
Andreu

@AHuguetUPC hey, thanks for you suggestion! I have tried fiddling around with the voxel filter parameters, and in some cases I have seen noticeable improvement but I find that it's just kind of unstable overall when working indoor, sometimes I get decent results, sometimes LIO-SAM just crashes for me and I cannot really figure out a way to make it work consistently indoors, and maybe it is not really meant to, my knowledge doesn't go so far so I'm not sure

Using a branded LiDAR (Ouster, Velodyne, HESAI) I guess all the unstability of the algorithm comes from the IMU. When we were comparing our IMU output with the ones on the public datasets we noticed that our raw output was way noiser, have you tried applying a low pass filter beforehand?
We also found that LIO-SAM really outperformed others when outside going at 10m/s doing hairpin turns with our autonomous car. That's where the IMU really helped serving as an initial solution. Judging for your application, if you do not have a drone or anything similar that can go that fast, have you tried a loosely-coupled approach? Personally, we love the stability of A-LOAM that for localizing the robot with lower speeds (<10m/s) delivers outstanding results. We have only tried it outdoors, however.

@AHuguetUPC I've noticed that the public datasets are really stable, but I'm not sure how I could efficiently compare the output of the IMU used there versus the one we use, are there any tools that can quickly help with that?

@AHuguetUPC I've noticed that the public datasets are really stable, but I'm not sure how I could efficiently compare the output of the IMU used there versus the one we use, are there any tools that can quickly help with that?

I guess there's not much more you can do than comparing the output data (using rosrun rqt_plot rqt_plot) of the two signals and how they behave on similar situations. Noisy IMU data with a slow moving vehicle on the inside can be a real pain.

Hi @Huguet57 , @Jaetriel
What kind of imu are both of you using? I am facing the same issue as Jaetriel, when come to indoor, the mapping result is similar to what Jaetriel show. I am using Ouster Lidar, therefore i use the built-in imu of Ouster lidar and pointcloud from ouster lidar to feed LIO-SAM. Ouster IMU gives 100Hz data. I also would like what parameters shall i set to get a good result in indoor.
Best, Samuel

Hi @HappySamuel I am also working on using Ouster internal IMu with LIO-SAM, can you guide me with it?
Thank you.


Add dynamic GOROOT setting for fish

This PR adds dynamic GOROOT setting for fish

Very cool. Thanks @rdoois!


Does Pytorch-Lightning support the task of federated learning?

Does Pytorch-Lightning support the task of federated learning? If so, any example provided? Thanks!

@tchaton

Hey @marsggbo,
We worked on a first integration with PySyft. Here is the notebook: https://github.com/OpenMined/PySyft/blob/dev/packages/syft/examples/duet/mnist_lightning/MNIST_Lightning_Syft_Data_Scientist.ipynb
Only works for CPU right now.
Best,
T.C


Resend dependency broke Jest

@react-email/render uses pretty that has a common js dependency. It breaks Jest. It'd be great to know how to fix it. Thanks.

@DawChihLiou Do you have a minimal reproducible repository for us to test?


panda/can_send: remove pos++

I don’t understand why pos++ is here,  it might be a mistake, right?

We add counter(1 byte) to send vector. Pos represents byte count(length) of send vector that we pass to usb_bulk_write later.

https://github.com/commaai/openpilot/blob/70c1f19304cf6c81baddee2f67d64e48b7891d3f/selfdrive/boardd/panda.cc#L398-L406
@briskspirit should we call send.clear() after usb_bulk_write? otherwise in the next cycle, the old data will be sent again. (The counter will also be re-inserted into the old data, which should also be a problem).  Is my understanding correct ?


slang-test/WGPU: IR opcode during code emit

This output is generated by
slang-test -bindir %BINDIR% -category full -api wgpu tests/autodiff/custom-intrinsic.slang
...
error: ERROR:
EXPECTED{{{
result code = 0
standard error = {
}
standard output = {
}
}}}
ACTUAL{{{
result code = 1
standard error = {
}
standard output = {
abort compilation: tests/autodiff/custom-intrinsic.slang(21): internal error 99999: unimplemented feature in Slang compiler: unexpected IR opcode during code emit
    T myexp(T x);
      ^~~~~
}
}}}

error: ERROR:
EXPECTED{{{
type: float
7.389056
7.389056
0.0
0.0
0.0}}}
ACTUAL{{{
}}}

...

Tests:

tests/autodiff/custom-intrinsic.slang

Test needs to be disabled.

Fixing might be easy.


Create an new release

I propose the creation of a new release. It's been two and a half years since the last release, and unfortunately with the last release the code is running on .NET 4.5.2 which is no longer supported.
Let me know how I can help!
Edit: removed a bunch of incorrect information per @sebastienros. See his comment below for what the release(s) should be.

from the subset above

1.10.x branch is the base for 1.10.4 release
dev is the base for 1.11 release
These are two distinct releases. We can start releasing 1.10.4, then merge it to dev and release 1.11.

Is there a release checklist for O1 like this one for Orchard Core?

All the issues assigned to 1.10.x and closed can be assigned to 1.10.4.
If an 1.10.x issue is closed but fixed in the dev branch then it should be assigned to dev.
The 1.10.x branch should be merged into dev to ensure all the fixes in 1.10.4 are reflected in dev.


Format isn't working

When I set a numeric field (say it NOMINAL) with format #,##0;[Red]-#,##0, it's display Red numbers in Datasheet view.
But when is exploring the field on a query with Build option with same format as above, we don't have a red numbers nor the thousand separator.
Kindly please advise

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c1af879b-24d6-a97a-a019-3936ecf628fa
Version Independent ID: 84c630e4-bae0-7c8c-72bb-d5baeaeacea1
Content: Format Property - Number and Currency Data Types
Content Source: api/Access.format.propertynumber.and.currency.md
Product: access
GitHub Login: @o365devx
Microsoft Alias: o365devx

@HarryFuna Thanks for your feedback. We use Github issues for documentation bugs, not for developer support. Please try the VBA developer community on Stack Overflow (https://stackoverflow.com/questions/tagged/vba) where you can get help from actual developers. If you find a bug in our documentation, please let us know. Thanks!


Cleaning up HAL buffer transfer usage.

This cleans up places using the iree_hal_buffer_* API in an effort to move more toward device-based transfers. In order to make the requirement of the buffer functions acting on mappable buffers more explicit "map" was added to their name.

Failing test was the asan flake in scalar.mlir.


Observation shows black images/screen

Hi,
When I run the evaluation python main.py --split val --eval 1 --load pretrained_models/sem_exp.pth -v 1the output visualization shows black screen for observations and the semantic map does not show any objects. [See reference image below]

I am using python 3.7.0, pytorch==1.9.0, torchvision==0.10.0, torchaudio==0.9.0, cudatoolkit=11.1.
My data seems to be in place:
The .glb and .navmesh files are under ~/Object-Goal-Navigation/data/scene_datasets/gibson_semantic/
The .glb.json.gz files are under ~/Object-Goal-Navigation/data/datasets/objectnav/gibson/v1.1/{split}/content/
The .gz and .pbz2 files are under ~/Object-Goal-Navigation/data/datasets/objectnav/gibson/v1.1/{split}/
Running the python test.py --agent random -n1 --num_eval_episodes 1 --auto_gpu_config 0 script shows Test successfully completed
Following is the console output for running python main.py --split val --eval 1 --load pretrained_models/sem_exp.pth -v 1
`Auto GPU config:
Number of processes: 1
Number of processes on GPU 0: 1
Number of processes per GPU: 0
Dumping at ./tmp//models/exp1/
Namespace(agent='sem_exp', alpha=0.99, auto_gpu_config=1, camera_height=0.88, cat_pred_threshold=5.0, clip_param=0.2, collision_threshold=0.2, cuda=True, du_scale=1, dump_location='./tmp/', entropy_coef=0.001, env_frame_height=480, env_frame_width=640, eps=1e-05, eval=1, exp_name='exp1', exp_pred_threshold=1.0, floor_thr=50, frame_height=120, frame_width=160, gamma=0.99, global_downscaling=2, global_hidden_size=256, hfov=79.0, intrinsic_rew_coeff=0.02, load='pretrained_models/sem_exp.pth', log_interval=10, lr=2.5e-05, map_pred_threshold=1.0, map_resolution=5, map_size_cm=2400, max_d=100.0, max_depth=5.0, max_episode_length=500, max_grad_norm=0.5, min_d=1.5, min_depth=0.5, no_cuda=False, num_eval_episodes=200, num_global_steps=20, num_local_steps=25, num_mini_batch=1, num_processes=1, num_processes_on_first_gpu=1, num_processes_per_gpu=0, num_sem_categories=16, num_train_episodes=10000, num_training_frames=10000000, ppo_epoch=4, print_images=0, reward_coeff=0.1, save_interval=1, save_periodic=500000, seed=1, sem_gpu_id=-1, sem_pred_prob_thr=0.9, sim_gpu_id=1, split='val', success_dist=1.0, task_config='tasks/objectnav_gibson.yaml', tau=0.95, total_num_scenes=5, turn_angle=30, use_gae=False, use_recurrent_global=0, value_loss_coef=0.5, version='v1.1', vision_range=100, visualize=1)
Scenes per thread:
Thread 0: ['Collierville.glb', 'Corozal.glb', 'Darden.glb', 'Markleeville.glb', 'Wiconisco.glb']
2022-07-12 22:17:44,765 Initializing dataset PointNav-v1
2022-07-12 22:17:44,766 initializing sim Sim-v0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0712 22:17:44.778574 252548 Simulator.cpp:96] Loading navmesh from data/scene_datasets/gibson_semantic//Collierville.navmesh
I0712 22:17:44.778640 252548 Simulator.cpp:98] Loaded.
I0712 22:17:44.778653 252548 SceneGraph.h:92] Created DrawableGroup:
Renderer: Mesa Intel(R) UHD Graphics (TGL GT1) by Intel
OpenGL version: 4.6 (Core Profile) Mesa 21.2.6
Using optional features:
GL_ARB_ES2_compatibility
GL_ARB_direct_state_access
GL_ARB_get_texture_sub_image
GL_ARB_invalidate_subdata
GL_ARB_multi_bind
GL_ARB_robustness
GL_ARB_separate_shader_objects
GL_ARB_texture_filter_anisotropic
GL_ARB_texture_storage
GL_ARB_texture_storage_multisample
GL_ARB_vertex_array_object
GL_KHR_debug
Using driver workarounds:
no-layout-qualifiers-on-old-glsl
mesa-implementation-color-read-format-dsa-explicit-binding
mesa-forward-compatible-line-width-range
I0712 22:17:44.809857 252548 ResourceManager.cpp:1380] Importing Basis files as ASTC 4x4
I0712 22:17:46.590839 252548 Simulator.cpp:167] Loading house from data/scene_datasets/gibson_semantic//Collierville.scn
I0712 22:17:46.590855 252548 Simulator.cpp:188] Not loading semantic mesh
I0712 22:17:46.591197 252548 simulator.py:146] Loaded navmesh data/scene_datasets/gibson_semantic//Collierville.navmesh
I0712 22:17:46.591398 252548 simulator.py:158] Recomputing navmesh for agent's height 0.88 and radius 0.18.
I0712 22:17:46.602424 252548 PathFinder.cpp:375] Building navmesh with 108x202 cells
I0712 22:17:46.676290 252548 PathFinder.cpp:643] Created navmesh with 246 vertices 126 polygons
I0712 22:17:46.676309 252548 Simulator.cpp:609] reconstruct navmesh successful
2022-07-12 22:17:46,678 Initializing task ObjectNav-v1
/home/arhanni/anaconda3/envs/ogn/lib/python3.7/site-packages/torchvision/transforms/transforms.py:281: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
"Argument interpolation should be of type InterpolationMode instead of int. "
[07/12 22:17:46 detectron2]: Arguments: Namespace(confidence_threshold=0.9, config_file='configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['input1.jpeg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl', 'MODEL.DEVICE', 'cuda:0'], output=None, video_input=None, webcam=False)
Changing scene: 0/data/scene_datasets/gibson_semantic//Collierville.glb
Loading episodes from: data/datasets/objectnav/gibson/v1.1/val/content/Collierville_episodes.json.gz
/home/arhanni/anaconda3/envs/ogn/lib/python3.7/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at  /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/native/BinaryOps.cpp:467.)
return torch.floor_divide(self, other)
/home/arhanni/anaconda3/envs/ogn/lib/python3.7/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /opt/conda/conda-bld/pytorch_1623448265233/work/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Loading model pretrained_models/sem_exp.pth
/home/arhanni/anaconda3/envs/ogn/lib/python3.7/site-packages/torch/nn/functional.py:4044: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
 Please do not use them for anything important until they are released as stable. (Triggered internally at  /opt/conda/conda-bld/pytorch_1623448265233/work/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
QObject::moveToThread: Current thread (0x557aa5d2f580) is not the object's thread (0x557b53203880).
Cannot move to target thread (0x557aa5d2f580)
QObject::moveToThread: Current thread (0x557aa5d2f580) is not the object's thread (0x557b53203880).
Cannot move to target thread (0x557aa5d2f580)
QObject::moveToThread: Current thread (0x557aa5d2f580) is not the object's thread (0x557b53203880).
Cannot move to target thread (0x557aa5d2f580)
......
QObject::moveToThread: Current thread (0x557aa5d2f580) is not the object's thread (0x557b53203880).
Cannot move to target thread (0x557aa5d2f580)
Time: 00d 00h 00m 00s, num timesteps 0, FPS 0,
Rewards:
Losses:
Time: 00d 00h 00m 01s, num timesteps 10, FPS 8,
Rewards:
Losses:
Time: 00d 00h 00m 02s, num timesteps 20, FPS 9,
Rewards:
Losses:
Time: 00d 00h 00m 03s, num timesteps 30, FPS 9,
Rewards:
Losses:
Time: 00d 00h 00m 04s, num timesteps 40, FPS 9,
Rewards:
Losses:
...........
`
Following is what I have tried so far and have been unsuccessful.
I found the following issue reported on habitat-sim https://github.com/facebookresearch/habitat-sim/issues/1080.
I downloaded the 3DSceneGraph_tiny.zip and ran the tools/gen_gibson_semantics.sh script as specified here: https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md. This script generated .ids and .scn files which I copied to ~/Object-Goal-Navigation/data/scene_datasets/gibson_semantic/ but the visualization still shows a blank screen. The gen_gibson_semantics.sh script gave the following error in generating _semantic.ply files Allensville wrote 2557798 bytes wrote 27718 bytes ESP_CHECK failed: esp::logging::LoggingContext: No current logging context. tools/gen_gibson_semantics.sh: line 17: 237742 Aborted                 "${TOOLS_DIR}"/../build/utils/datatool/Datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply because it couldn't find the mesh.obj files.
The gibson_habitat_trainval.zip does not contain the .obj files.
I need the RGB observations to extract the semantic predictions and build my project on top of this. Any support would be appreciated!

The code does not require the 3DSceneGraph to be built as it does not require access to ground-truth semantics.
Could you try saving the images obtained from habitat directly here:
https://github.com/devendrachaplot/Object-Goal-Navigation/blob/master/envs/habitat/objectgoal_env.py#L380

I figured out why that was happening. I was running this code on a PC with RTX 3080 GPU. I believe there is a Cudatoolkit compatibility issue for this GPU. Even the docker installation had the same issue. However, I got through by using a docker installation on a PC with RTX 1080 GPU.
Thank you for the response! But I cannot try that as I'm no longer working on that project. Please feel free to close the issue.


fix Added missing capability schema properties

Added additional properties found in capability schemas while making CLI enhancements:
properties?: { [name: string]: CapabilityJSONSchema }
nested properties
items?: CapabilityJSONSchema | [CapabilityJSONSchema]
items as a schema object
items as an array of schema objects
title?: string
title of type (definition)

:tada: This PR is included in version 0.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Support defining world deployments in manifest

Developers should be able to easily deploy worlds to different chains, for example, devnet, testnet, mainnet etc.
Currently, we have a attribute defined in the Scarb tool manifest:
https://github.com/dojoengine/dojo/blob/8d5daf11e4b55e648d232d97d3fd0ad895f0c331/examples/Scarb.toml#L17
It's not particularly well thought out though. We probably want to namespace it by env:
[tool.dojo.env.testnet]
rpc = "https://rpc.testnet.starkware.co"

and allow users to configure various parameters to handle the deployment to that env, for example, rpc, private_key_env, account_address etc.
From the command line, the developer should be able to define an env as an arg, for example: dojo migrate --env testnet

Can the env name be arbitrary, or we check for reserved keywords like testnet, mainnet, devnet only?


Add support for esp32c3 hardware RS485 half duplex mode

Changes required in the HAL to support half duplex mode for the esp32c3 only.
Signed-off-by: Dean Sellers<EMAIL_ADDRESS>Draft
This change only applies to the esp32C3, I am not familiar with the other SOC in this platform and haven't implemented or tested on these.

As other ESP SOCs also support direct RS485, would you mind adding the pinctrl defs and ll funcs for them as well please?

@marekmatej I can, although I won't be able to test it as I don't have any of the other SOC on hand. We are currently developing 3 products based on the ESP32C3, so that is the only hardware I have available to test on. If I can assume that the other variants work the same way, happy days! :)

Adapting the changes here to all ESP SOC got messy. I have closed this PR in favour of creating a new one on a different branch that includes all SOC. See #185.