Consider using chex in validation and/or testing

https://github.com/google-deepmind/chex
Could be helpful for improve, e.g. assertions about shapes:
https://github.com/CDCgov/multisignal-epi-inference/blob/4f203361a75be79c06f990b781194bece48d1f26/model/src/test/test_first_difference_ar.py#L29-L30

Beyond its formality and conciseness, it looks like it might be slightly faster, but I don't know if boost is trivial, or even if it's meaningful, given that this will be used in tests. Anyway, nice find.


Check for empty body before splitting lines

If a body is empty, splitlines will return an empty list. This
the remaining content before trying to split it.
Fixes #1661

Codecov Report

Merging #1662 into develop will increase coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #1662      +/-   ##
===========================================
+ Coverage    92.36%   92.36%   +<.01%
===========================================
  Files           53       53
  Lines         9797     9798       +1
===========================================
+ Hits          9049     9050       +1
  Misses         748      748

Impacted Files
Coverage Δ

botocore/response.py
92.75% <100%> (+0.1%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d67f197...51bcaca. Read the comment docs.


Pushe library increases signed app's size

my apk's size (by proguard) is Almost 5mb,  but when I added pushe library size became 15mb(sometimes 10mb), how to fix that?

15 MB doesn't seem sensible to me since there aren't much code in Pushe + All its deps. 10MB does though.
There's nothing to fix. If you have nothing in common with Pushe dependencies, you'll get a larger pack at the end.
Libraries otherthan Pushe that will be added to classpath are quite large:

Firebase-messaging
WorkManager
A few AndroidX jetpack such as life-cycle related ones
Huawei PushKit (if you add :hms module for huawei)
RxJava2

Together they make the final pack quite larger (specially for an almost empty APK)
If there's gonna be a remake to the project, it will be much smaller. However, next release will only be bugFix and the size issue still remains intact.

Update to this. Since deletion of RxJava2 from the library is really complex and time-consuming, after some minor updates to the library I'll be working on a solution to reduce the final AAR size of all modules.
Unfortunately, I can not specify a deadline for now.
Will notify in the issue as soon as I have new updates.

hi again, any update?

Not yet. The focus is on automation development and the improvements have lower priority. So tge process is in paused state temporarily.


In-memory inputs for column split and vertical federated learning

We've recently added support for column-wise data split (feature parallelism) and vertical federated learning (#8424), but the user interface in python is limited to text inputs and numpy arrays (#9365) only. We'd like to support other in-memory formats such as scipy sparse matrix, pandas data frame, cudf, and cupy.
One question is the meaning of passing in data_split_mode=COL. There are potentially two interpretations:

We assume each worker has access to the full dataset, passing in data_split_mode=COL would load the whole DMatrix, then split it by column according to the size of the cluster. The columns are split evenly into world_size slices, with each worker's rank determining which slice it gets. This is the approach currently used by the text inputs for feature parallel distributed training, but not for vertical federated learning.
We assume each worker only has access to a subset of the total number of columns, with column indices starting from 0 on every worker. The whole DMatrix is a union of all the columns from all the workers, with column indices re-indexed starting from worker 0. This is the approach currently used for vertical federated learning.

Now we want to support more in-memory inputs, it probably makes more sense to standardize on the second approach, since it seems wasteful to construct a DMatrix in memory and then slice it by column.

@trivialfis

Helps with #9472

Let's focus in the federated learning use case and remove the data splitting in XGB entirely

Sounds good. We'll standardize on the second approach, i.e. each worker only provides its own set of columns that are 0-indexed, and the global DMatrix is a union of all worker columns, re-indexed based on worker ranks.

Excellent!
On 9/30/23 03:23, Rong Ou wrote:

Sounds good. We'll standardize on the second approach, i.e. each
worker only provides its own set of columns that are 0-indexed, and
the global DMatrix is a union of all worker columns, re-indexed based
on worker ranks.
—
Reply to this email directly, view it on GitHub
https://github.com/dmlc/xgboost/issues/9619#issuecomment-1741379036,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD7YPKMYGFQ3WTB7PN5XXFTX44N33ANCNFSM6AAAAAA5LT2I7Y.
You are receiving this because you were mentioned.Message ID:
@.***>

--------------9PPYtvmG7ybnc0U29Ggd0Hgz
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

Excellent!

On 9/30/23 03:23, Rong Ou wrote:

Sounds good. We'll standardize on the second
    approach, i.e. each worker only provides its own set of columns
    that are 0-indexed, and the global DMatrix is a union of all
    worker columns, re-indexed based on worker ranks.

—

    Reply to this email directly, view it on GitHub, or unsubscribe.

    You are receiving this because you were mentioned.Message
      ID: <dmlc/xgboost/issues/9619/1741379036@github.com>


Mast: Watch wellness.cbd

Mast requests the watch of the watch_keyword wellness.cbd. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

wellness.cbd has been seen in 17 true positives, 0 false positives, and 0 NAAs.

Superseded by a slightly stricter blacklist regex.


Créer le lab pratique pour comment utiliser Github

Context
At CEDILLE, one of our aspiration is to create a learning environment that is accessible to everyone regardless of their level of expertise as long as they are willing to learn. We are very much aware of the reality that a number of student start their journey at ETS with next to no prior programming knowledge. Truth to be told, each of us once started from nothing and the road is certainly step at the beginning. That is why we want to develop a set of resources that will allow our new members that are just starting their software and IT journey to get a foot in the door and find their bearing.
That said, we think one of the most useful ways we could help kickstart new members and speed up their integration into our community is by introducing them to our collaboration tools of choice: git and GitHub. These tools are going to be fundamental to the future labs in the onboarding process and to their professional career in general. Therefore, we need an introductory lab on the topic to ensure everyone can follow along.
Objective
Develop a beginner-friendly lab to introduce new members to Git and GitHub, providing them with the essential skills they need to set up their development environment, create a GitHub account if they do not have one, and manage repositories. This lab should be especially helpful for students without prior programming experience.
Git and GitHub are obviously topics that have been extensively documented on the internet for anyone that cares to look it up. Our main objective with this lab is not necessary to reinvent the wheel but rather to synthesise all this often overwhelming information into an intuitive tutorial that is digestible for someone who is just getting started.
If we keep things goal oriented then by the end of this lab new members should have achieved the following:

successfully set up and integrated git into their dev environment of choice
successfully created and set up a GitHub account
successfully linked git to their GitHub account
successfully created and customized their own profile repository from a given a template (e.g. https://github.com/SonOfLope/SonOfLope)

Acceptance criteria

[ ] The lab should arm new members with a basic understanding of version control, Git and GitHub.
[ ] Members should be walked through setting up their development environment (probably at minimum git and vscode)
[ ] Members should be instructed to set up a GitHub account if they don't have one already.
[ ] Members should be instructed on how to link git with their GitHub account.
[ ] Members should be instructed on how to set up their own profile repository. Bonus point if there is a section explaining why it will be beneficial for their professional career to have a clean github profile that they slowly grow.
[ ] Members should be instructed on how to make edits on said profile repository by modifying the provided template. Through this process of customization members should be thaught how to perform git basic operations (e.g. clone, add, commit, push, etc) and basic GitHub operations (e.g. what is an issue, how can I create a branch from an issue, what is a PR, how can I merge a PR, etc).

https://github.com/ClubCedille/cedille-web-track repo to be used for the track guideline @etienneoh

See #263
First draft (only in english) of the Git onboarding
@Endlessflow @SonOfLope


Conditional rendering of fields breaks when more than one field referencing another

I would like a radio button with yes no to drive 2 following fields. A yes causes the first field to display and the second to hide. A no causes the first to hide and the second to display. I set this up as follows:

When i come to save or render I get the following error in the js console:
demo.min.js:24 Uncaught TypeError: Cannot read property 'parentElement' of null
at Object.isVisible (demo.min.js:24)
at Object.equals (demo.min.js:24)
at t.execResult (demo.min.js:24)
at demo.min.js:24
at Array.forEach ()
at demo.min.js:24
at Array.forEach ()
at demo.min.js:24
at Array.forEach ()
at demo.min.js:24
If i remove the second fields conditional statement (referencing the first radio button) the save and render returns to working

got it working, fixes below:
renderer.js: :~238
          isNotVisible: () => {             // fix             let parent = elem.parentElement;             while (!parent.parentElement.classList.contains('formeo-column')) {                 parent = parent.parentElement;             }             parent.setAttribute('hidden', true);             elem.required = false // Hidden input cannot be required.           },           isVisible: () => {             //fix             let parent = elem.parentElement;             while (!parent.parentElement.classList.contains('formeo-column')) {                 parent = parent.parentElement;             }             parent.removeAttribute('hidden')             elem.required = elem._required           },
      and :270

      `  getComponent = address => {
const componentId = address.slice(address.indexOf('.') + 1)
const component = isExternalAddress(address)
  ? this.external[componentId]
  : this.renderedForm.querySelector(`[name^=f-${componentId}]`) // fix
return component

}`


Please create a method to list all users in a group or that are connected to the service.

The API would benefit greatly from a call to get a list of users and connections connected to a group or hub.
When a client first joins a group, they may want to know who is connected.
Also I may want to poll a particular group to see who is online at that time.
This is more robust since it's tracked inside PubSub which already is maintaining such lists internally.
There is already a UserExists method so this would be a good comliment.
I propose you add a method such

listUsers(optional group name)
listConnections(optional group name)
listUserGroups(userID)

In the current Web PubSub service, we actually don't preserve such a list. And the AddUserToGroup is actually a quick way to add current connections for this user to the group, which means, if a connection for userA connects after the AddUserToGroup(GroupA) call, this new connection is not joined to GroupA automatically. So actually the user can have a situation where some connections for userA belong to groupA while other connections for userA are not.
For the scenarios you have:

When a client first joins a group, they may want to know who is connected
poll a particular group to see who is online at that time

So if it is some group chat scenario, I feel that such user-group map contains application logic, and sounds fair to have a database to store such map for your application instead.

Could you please elaborate.

When a user is joined to a group, you are actually joining the connection to the group.
When I make a call to send a message to a group, it goes to the connections.
When I make a call to a user, does it message all the connections associated to the user?

Because you must keep a record of what connection goes with which user, can you not get a list of connections for the group and then join that list to the list of connections for the app to get the userIDs?
What are the technical reasons that the userID is not kept with the connection on the group?  Is it because the userID is not a required parameter?
Given the above assumptions, I'd recommend that userIDs be kept with your connections on the groups and that you still add the methods mentioned above.
At a minimum, please provide a list of connections for a group and a method to lookup that connection to get any associated userID that was given for it.  if you store the connection as [connectionID]|[userID] then you can still do any kind of checks for the connection IDs without degrading performance and if you are hashing them then just add them to a dictionary and track how many connections are opened or closed.  You kind of do this already I'm sure.
I'm aware that a database can track the information, however that assumes that there is never an outage or disconnect between pubsub and the api endpoints.  In that scenario the database will be stale compared to pubsub and out of synch.  It is far better for pubsub to do that logic on it's end as it already has that basic data.  You don't really know how the service can be used, but tracking variables that are associated with a user or connection is important.
Also I may need to know what connections or users are still in a group, or what groups are assigned to a user.
In my opinion the service ought to be able to report that.

Consider a chat room with users joining a group, I think the source of truth should be your database that maintains the user-group mapping instead of relying on the Web PubSub service to store the user-group mapping. Web PubSub helps you to manage the alive connections but does not store any data persistently. The Group in Web PubSub is a logical concept, it is some kind of "group session" that starts when some connection joins it and ends when there is no connection in it. Consider the chat room scenario, when there is a group chat, and everyone joins the group but offline, you don't want the group to disappear but still want the user-group mapping there, right? From this perspective, your "group" is not the same concept as the "group" in Web PubSub.
I'd also like to understand your concern about "stale data" in your database. Could you provide me an example of what would be out of sync?

Actually I already received a notice from Azure that there was a billing issue related to virtual machines this month that they detected.  I’ll look for the credit and then touch base if for some reason it does not come through.
This appears to be a known issue.
Thanks,
Andrew
From: Liangying.Wei @.>
Sent: January 27, 2022 2:13 AM
To: Azure/azure-webpubsub @.>
Cc: Andrew Middleton @.>; Author @.>
Subject: Re: [Azure/azure-webpubsub] Please create a method to list all users in a group or that are connected to the service. (Issue #325)
Consider a chat room with users joining a group, I think the source of truth should be your database that maintains the user-group mapping instead of relying on the Web PubSub service to store the user-group mapping. Web PubSub helps you to manage the alive connections but does not store any data persistently. The Group in Web PubSub is a logical concept, it is some kind of "group session" that starts when some connection joins it and ends when there is no connection in it. Consider the chat room scenario, when there is a group chat, and everyone joins the group but offline, you don't want the group to disappear but still want the user-group mapping there, right? From this perspective, your "group" is not the same concept as the "group" in Web PubSub.
I'd also like to understand your concern about "stale data" in your database. Could you provide me an example of what would be out of sync?
—
Reply to this email directly, view it on GitHubhttps://github.com/Azure/azure-webpubsub/issues/325#issuecomment-1022912184, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APXZTM7TSPMWO4YS7RCGR53UYDV6RANCNFSM5MWN47PQ.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

I think the issue is that you are assuming everyone is making a chat interface, but I am using this to move data notifications across a platform to connected clients that may want to know when a job is done, data has changed, or to indicate that they can take a phone call.  What if I want to revoke security for a user but don't have their connection info?  At the moment I literally have to check if they are in the group to do that.  What happens if for my phone system I don't get the even to say they disconnected and now I have a customer alive in my queue waiting for an answer that won't come.  Basically you have to broaden your understanding of how this can be used.  It's a pub sub system, not a group chat system.
Also you just said that you keep a list of active connections, that's all an app needs you to return to make a decision.  I want to be able to ask what all the active connections are for a group or the service as a whole.
You already have the data ready to go, it should be easy as pie to add a method to return that information.
And as a customer of Microsoft, really you should be heeding your customer needs even if you disagree with them.
We are the ones paying for the service so clearly we have a need for these kinds of features.

Sorry for the late response, (just back from holiday).
Could you share more about your user scenario? We can also discuss more through email lianwei(at)microsoft.com or through meetings (we can arrange that through email too) on how to fit Web PubSub into your scenario.

Quote here:

Scenario: User A, B & C joined a group (group is modifying a common resource/document). User A makes a change to the document which needs to be notified to the group (except User A).
Question: How can the developer avoid sending the notification to "User A"?
Comment: The developers do not intend to handle this scenario through UI code.
If there were a function to retrieve list of users subscribed to a group, this situation could have been addressed.

Hi @abymsft, if you are using the subprotocol, there is a property called noEcho:

For this scenario, I think it is actually not sending to "UserA" but not sending to the connection modifying the document. The same user but opened another browser is expecting to receive the updates too.
If you are not using subprotocol but REST API(server SDK), if we provide an "exclude" option when sending messages to a group,will that help?

Hi @Johno-ACSLive , thanks for the details. Some follow up questions on the scenarios:

In your scenario, your server add/remove users to the group and add/remove permissions to the users while your users send messages to the group, right?
Are there needs that your server dynamically add/remove permissions for a user? We do have an API to "Remove connections/users from groups" https://learn.microsoft.com/rest/api/webpubsub/dataplane/web-pub-sub/remove-connections-from-groups?view=rest-webpubsub-dataplane-2023-07-01&tabs=HTTP , but we do not yet have an API to "Remove connections/users group permissions". We could add this one if needed.

Hi @vicancy,
The flow in my scenario is that a user will create a "session" (group in Web PubSub by essentially being the first to join it). They then own control over who can join the "session" (group). My custom backend service handles this (client sends user events to enact the appropriate action e.g. createsession, deletesession, joinsession,unjoinsession, bootuser etc.). All users who are in the "session" (group) can communicate with each other only. No one can join the session and publish messages unless the owner allows them to join.
The reason for this is that the user creating the "session" (group) will limit the number of users who will join and thus limit who is allowed to join the group as described above. The owner of the "session" (group) can also boot users from the "session" (group). The owner does not know what their userid or connectionid is. It only knows certain other information. The service I wrote to handle events doesn't store or track these details either since Web PubSub already does. It's just that the information is not available to query.
I've been mocking up to test PubSub's suitability but running into this limitation (and found this has already been raised but not addressed) in that Web PubSub knows which user has joined which group but we can't query the service to find out. I'd rather not duplicate that data in a database. At the moment the only thing I need to track is who is the owner (I'm not sure if Web PubSub tracks which userid first joined - if it does this would be useful to retrieve as well).
Ably provides this capability - their occupancy  under the heading - "Retrieve presence members". They also have a heading further down called "Synced presence set" which is partly what we have to do now (although we need to use events 100% of the time to build and maintain the list for Azure Web PubSub at this point in time). Ably advise against this because they are already tracking everything - same as Web PubSub.
What I described above from Ably is just one example for what has been requested here but hopefully demonstrates why as well.
To answer your questions, for point 1 - yes. For point 2, this is already part of the Client SDK (both - e.g. I can call RemoveConnectionFromGroup or RemoveUserFromGroup as well as related async methods). There is also a call I can make to RevokePermission.
The problem with both GrantPermission and RemovePermission is that I can only specify a connection when I should also be able to specify a user. This is because I can specify both a user or connection when calling AddUserToGroup/RemoveUserFromGroup so should have the same options.
However, as described I'm not tracking which user has been added to a group nor which connection has been granted what permission to which group. Web PubSub does track this hence the request of capabilities I and others have mentioned. That way we are not duplicated data (potentially having stale data) as others have mentioned and as per Ably's point of view with their equivalent service.
I hope this helps move this forward.

Thanks for the detailed explanation @Johno-ACSLive I understand your needs now that you'd like a way to remove connections/users and also the permission from a group except the "owner" user.
IMHO, having a method to "list all users in a group“ might not be a simple enough solution consider race conditions when there are new users coming in between "getting all the users in a group" and "removing users and permissions from the group". Some atomic method call to directly remove these users/permissions might fit more into your scenario.
We actually have REST API ready for such need to specify some filters when remove connections from a group https://learn.microsoft.com/en-us/rest/api/webpubsub/dataplane/web-pub-sub/remove-connections-from-groups?view=rest-webpubsub-dataplane-2023-07-01&tabs=HTTP, the SDK support is on-going, when SDK is supported, the method call would be similar to ServiceClient.RemoveConnectionsFromGroup(group: "group1", filter: "userId ne 'owner'"). We apply OData syntax to the filter as described here:https://learn.microsoft.com/en-us/azure/azure-web-pubsub/reference-odata-filter.
In your scenario another API is also needed, sth. like ServiceClient.RevokePermissions(permission: "permission", target: "group1", filter: "userId ne 'owner'")
How do you feel about the proposed APIs?

Thanks for the detailed explanation @Johno-ACSLive I understand your needs now that you'd like a way to remove connections/users and also the permission from a group except the "owner" user.

@vicancy I'd like to query groups for their list of current user id's, associated connectionid and permissions. I can already remove them via the Server SDK. It would be nice to know who "created" the group i.e. first user/connection joined but I don't need to excempt removal of the "owner".

IMHO, having a method to "list all users in a group“ might not be a simple enough solution consider race conditions when there are new users coming in between "getting all the users in a group" and "removing users and permissions from the group". Some atomic method call to directly remove these users/permissions might fit more into your scenario.

Should be simple enough, the service already knows and can just provide what it knows at that point in time. If there is a change then I can use event notifications if it's critical for whatever reason but in most situations I don't see if would be required. If for example the service returns a permission but it had been removed between me obtaining the response and actioning it (very unlikely to occur), the Server SDK has checks (GroupExists, CheckPermission, ConnectionExists and UserExists). If there is a check that doesn't exist then Web PubSub will respond as such. I would typically call those checks first but even then I could check a connection exists and the client disappears if I want to remove them from a group etc. so I need to handle such responses either way.

We actually have REST API ready for such need to specify some filters when remove connections from a group https://learn.microsoft.com/en-us/rest/api/webpubsub/dataplane/web-pub-sub/remove-connections-from-groups?view=rest-webpubsub-dataplane-2023-07-01&tabs=HTTP, the SDK support is on-going, when SDK is supported, the method call would be similar to ServiceClient.RemoveConnectionsFromGroup(group: "group1", filter: "userId ne 'owner'"). We apply OData syntax to the filter as described here:https://learn.microsoft.com/en-us/azure/azure-web-pubsub/reference-odata-filter.

That gets us part way there, however, doesn't address all cases. What if an event is missed, how do I signal to users in the group that a user has gone if it gets missed for whatever reason? Again in the case that the Client SDK doesn't recover but reconnects I need to check which group they were allocated to. There is currently no API for me to send a query to.
What if I want to build a management dashboard and provide statisics like how many groups there are and how many users and connections there are per group. What if we need to manage connections through a management interface to provide some facility that requires requesting information.
To summarise, we need API's and SDK updates that will allow the following:

Delete all user id's, connection id's and associated permissions from a group - this could be two calls like serverClient.RemoveAllUsersFromGroup(<group_name>); and serverClient.RemoveAllConnectionsFromGroup(<group_name>); - now there might also be a 3rd call to add or in place of the previous two called serverClient.RemoveGroup(<group_name>);
Query a group for its user id members and/or connection id members (because you can add both user or connection id to a group) and associated permissions - this could be a call like serverClient.GetGroupUsers(<group_name>); and serverClient.GetGroupConnections(<group_name>);. The difference being the former listing userid's with sub properties including the connectionid and permissions. The later just containing permission sub properties (unless it's worth including the userid as well).
Query a user for all it's connection id's, associated groups and permissions - The call here could be serverClient.GetUserDetails(<user_id>);
Get a list of all users - The call here could be serverClient.GetAllUsers();
Get a list of all groups - The call here could be serverClient.GetAllGroups();
Update the Server SDK GrantPermission and RemovePermission (and async variants) to accept a userid or connectionid (instead of just connectionid) - The current call parameters are for example public virtual async Task<Response> RevokePermissionAsync(WebPubSubPermission permission, string connectionId, string targetName = null, RequestContext context = default) so adding another for both Grant and Remove containing public virtual async Task<Response> RevokePermissionAsync(WebPubSubPermission permission, string userId, string targetName = null, RequestContext context = default)

Is there really no way to list all users for a hub today?
I have a use case where I would like to use Azure WebPubSub for a home automation scenario.
The users would be Home Automation controllers running on physical hardware at various locations. The controllers would connect to the server and a web application should be able to send messages to the controllers. The web application should be able to list the connected controllers and the user of the web application should be able to select a controller for management. The web application would then send messages and get responses from the controller.
Today I use Azure IoT Hub for this scenario, but I would like to have alternative implementations for this scenario.

Is there really no way to list all users for a hub today?

You won't find a call for it in the Server SDK and I havn't seen anything for it in the API documentation for the data plane.

Ok, then I can't use this service for my scenario.

Hi, we plan to add some REST APIs to address your requirements in the issue. Please check #799 for proposal to list users and connections in a group. Feel free to leave your comments.

@Y-Sindo its a start but more of what's been requested needs to be included. I've commented on 799.

@Johno-ACSLive
In your scenario where you need to:

Query groups for their list of current user id's, permissions.
Recover the groups and permissions when the Client SDK doesn't recover but reconnects.

I believe additional persistent storage is necessary. For example, use a database to record the permissions, and groups of users. As our service currently doesn't store the customer data persistently, such data would be lost if a user doesn't have an online connection for a long time. If such kind of requirement is common, we've also considered a new feature called "Bring your own storage" that allows our customers to provide custom storage for our service to read and write.

@Y-Sindo since my backend service is handling management of connections to the service and permissions to groups, the Client SDK doesn't need to recover anything so point 2 can be ignored.
However, for point 1 I mentioned earlier that the service already knows about and is tracking these items so why shouldn't I be able to query it? I shouldn't need persistent storage if the service already knows what's been configured.
There is also some other queries I should be able to make as well since the service is already tracking these items.

Ably provides this capability - their occupancy  under the heading - "Retrieve presence members". They also have a heading further down called "Synced presence set" which is partly what we have to do now (although we need to use events 100% of the time to build and maintain the list for Azure Web PubSub at this point in time). Ably advise against this because they are already tracking everything - same as Web PubSub.

@Y-Sindo this is the info relating to another product that offers the same service.


No separation incidents while locked on ILS

While on the ILS planes should not cause a separation incident, e.g. same height on parallel runways

I'm sure it depends on specialized technology that the airports are using to make these things possible, for example: http://www.airservicesaustralia.com/precision-runway-monitor/a-pilots-guide-to-ils-prm-approaches/.
Though I personally know nothing about how this is done in practice.

The vertical separation should be 1000 feed, the horizontal 3NM in minimum
at approach heights. But if on ILS that rule does not or cannot apply. E.g.
on Munich Airport the two parallel runways are less than 1NM apart and
there are planes landing simultaneously. So my suggestion is: no separation
incidents while on (different, parallel) ILS.
2015-05-12 18:55 GMT+02:00 Anthony Van de Gejuchte<EMAIL_ADDRESS>
:

I'm sure it depends on specialized technology that the airports are using
to make these things possible, for example:
http://www.airservicesaustralia.com/precision-runway-monitor/a-pilots-guide-to-ils-prm-approaches/
.
Though I personally know nothing about how this is done in practice.
—
Reply to this email directly or view it on GitHub
https://github.com/zlsa/atc/issues/101#issuecomment-101346983.

Yes, though safety is still a concern.
Simply disabling it and then being in a situation that a plane is flying into another plane is simply undesired. It's way more complicated than just disabling the rules.
So the question now is, what procedures should one follow.

As i said: as long as two planes ILS locked on two parallel strips there is
no separation incident
2015-05-12 19:36 GMT+02:00 Anthony Van de Gejuchte<EMAIL_ADDRESS>
:

Yes, though safety is still a concern.
Simply disabling it and then being in a situation that a plane is flying
into another plane is simply undesired. It's way more complicated than just
disabling the rules.
So the question now is, what procedures should one follow.
—
Reply to this email directly or view it on GitHub
https://github.com/zlsa/atc/issues/101#issuecomment-101360777.

According to the answer here, the minimum runway centerline separation for IFR is 750 feet; I seem to recall that you cannot land wingtip-to-wingtip in those conditions, though.

Since this is a simulation and there are so many factors that can influence the possibility of a parallel approach . Maybe we should reduce the use case to 'perfect condition' for now (good weather, no technical issues, etc... ). These things can always be added later separately and it will simplify the implementation for now.
(uh I should have read the title while thinking of a separation strategy)
We might take the rule that separation of airplanes is lowered when planes are locked to ILS and reduce the distance before having a separation incident event. Since there are multiple standards (thanks for the link!) maybe it should be configurable by map.
Though given [some information about safety related issues (yeah, I'm using that link again :-))], a serious separation incident should be happening at 150 meters (about 492.1 foot?).

It's actually good to gather and read the information, its so easy to find and there are a lot of details to be considered...


Update README.md

added 2 Brett Cannon posts and Sebastian McKenzie

ooh, hadn't seen this one from Sebastian! good stuff 🙇♀


Add gatling scale out .sh script and update doc

There is a small issue with your PR :
Linking the script properly from the documentation may prove a bit difficult, and does not work in its current state.
Could you please, for the moment, inline the script in the "Scaling Out" page ?
We'll sort it out after ;)
Could you also rebase your commits after that ?

add inline script + rebase, you can change later on and remove the inline script/ add link
thanks

Thanks !


bruig: pm tab after receiving an invite greys out

I have setup a brand new account and have exchanged an invite to add my first contact to my addressbook.
The user appears in my user-list in the gui and i start interacting. As soon this user invites me to his group chat (welcome) the complete private message tab greys out and becomes unreadable. Newly received messages appear at the bottom of the tab. As soon i write an answer the tab becomes grey again.
Looks like a conflict with the "accept invite" popup. From an UX perspective having a modal could be a solution instead injecting the popups in the messaging-tabs directly. if they get injected into the message-list it should be in the message-flow like a regular message, this would even enable to have a history at what point in time the invite was received.

Is this still occurring after the recent updates to chat lists in v0.1.4 ?

Should be fixed.


Automate workload benchmarks

Signed-off-by: Shyam Jesal<EMAIL_ADDRESS>

please rebase

closed to avoid redundant CI runs. to be re-opened after the deadline


Stringified json in error message

Expected Behavior
Show an error message "unknown
Actual Behavior
Displays the error message "unknown flag" and a stringified json in the end of output.

Steps to Reproduce the Problem

Open terminal.
Go to kics folder.
Run: 'kics --any'

Specifications

Version: 1.2.4
Platform: any
Subsystem: n/a

@cxlucas also reproduced this problem when using unknown commands, as can be seen on following image:

If a flag must have arguments, but not receives, it will reproduce the bug, e.g.: go run ./cmd/console/main.go scan -p valid_path --fail-on


Intake subsystem

for tracking or something idk ask luke

@lmaxwell24 Can you merge main into this branch.  git pull origin main .  I tested and that creates a clean merge commit.  Next PR we can try to do things differently, but that is the fastest way to clean up this branch and make it easy to review the PR.


No module found Pythoncom

I use Python 3.4, Django1.8.18, django-mssql 1.8 and django-sqlserver 1.11. But when I run server, an error message comes up 'No module found Pythoncom'. Could you please help?

can you provide a stack trace?

many thanks for the prompt response.
what I want to achieve is to use django-mssql but without pywin32 as I will need to use it in a Linux environment.
I am new to django so I am not sure what you mean by stack trace.
but please see attached a screen shot of my Command Prompt:

at the top, you could see the structure of my project folder (for the testing purpose, I have just created a new django project without any app)
then you could see all the packages I have install with their version
lastly you could see the error message

I did try django-sqlserver with python 3.4, django 1.6 and django-mssql 1.6 and I did not received the error message. however, django 1.6 is no longer supported so I will need django 1.8.18 at least.
it will be great if you could have a look! Thanks again!
Doc1.docx

I think you are using django-mssql database driver in your Django configuration file.  If this is so you should switch to django-sqlserver driver.  If this is not the case please upload your Django configuration file.


Parent does not reflect changes in children

Godot Version:

Currently when you make changes to the children of Panel Container (NB: This also applies to other nodes that make changes to children, but for sake of this post I will use PanelContainer as an example) the parent does not adjust unless I make any changes inside it after making changes in a child.
Scene Tree:

Before making changes to child

After making changes to child

As you can see, the parent do not wrap the children as it is set to do.

However if I just click inside one of the parents' margin, and press enter;

It changes to the way it should have been after making these changes to child.

While this is not anything "serious", it would be nice to have it looked at - and adjusted so that It do as expected :)

This was actually a bug in GridContainer not letting the parent know that it's minumum size may have changed when the cells property changed.

Neat, glad to see this is fixed :) :+1:


Cookie Issues, Episode 3?

Hello again!
It's been a while and over at @risetechnologies we have been doing pretty well with Meteor-Files and the ?xmtok=... authentication approach, as necessitated by cordova-plugin-meteor-webapp.
Recently I have been researching the topic again because we faced a similar issue on another front - the AWS loadbalancer only supports 'sticky sessions' using Cookies, but as we know, these are tricky with cordova.
In the process of researching this (details in cordova-plugin-meteor-webapp#64), I have found two things:

all non-XHR requests (e.g. img.src etc) could totally be authenticated with Cookies, it's just that ostrio:cookies sets the Cookie from the client side, and so it ends up being set on the localhost:12008 proxy server and not on the alternate-origin ROOT_URL. Clientside JS is disallowed from reading or manipulating cookies there but server responses can read and write cookies.
XHR requests by default exclude Cookies and Auth information, but if .withCredentials is set on the XHR before sending they behave just like described above. Most web packages that use either expose this option or are easily modified to set it. This is the case for all our bundled asset-viewers (threejs, hlsjs, pdfjs).

With the above information, the only missing piece to fix 'The Cookie Experience' is a serverside route, e.g. ${downloadRoute}/_cookie, that responds with the correct Set-Cookie: x_mtok=... header. On cordova (or in general), the client bundle can then trigger a .withCredentials XHR there when the connection is established, and all further requests, browser- or XHR-initiated, will be authenticated.
We are thinking about implementing this ourselves, but it sounds mostly trivial to add to Meteor-Files and I am sure others could be interested in this approach.
A more widely-scoped solution would change the way ostrio:cookies works on Cordova, to something like this: (pseudocode)
client:
Meteor.call('cookies.set', { [name]: value }, (err, res) => {
  if (res) {
    const xhr = new XmlHTTPRequest();
    xhr.withCredentials = true;
    xhr.open('GET', res.pickupURL);
    xhr.send(null);
  }
});

server:
Meteor.methods({
  'cookies.set': (to_set) => {
    // store cookies temporarily
    const pickupId = Random.ID();
    return `${pickupRoute}?id=${pickupId}`;
  }
});

....onRequest(pickupRoute, (req, res) => {
    const pickupId = req.params.id;
    // retrieve cookies from temp storage
   res.set_cookies(cookies);
})

Hey @s-ol ,
Great Idea. I've been thinking to move all XHR and WebSockets logic to WebWorkers, your contribution is good reason to stop thinking and implement it, or maybe we shall finally jump to v2.0?
As I know there is no issues running WebWorkers on Cordova, right?

@dr-dimitru do you mean the logic concerning uploads? This is more about a one-time setup request that can establish your x_mtok cookie on the correct origin domain.
I have some time today allocated to investigating a possible implementation, expect a PR later :)

@s-ol you know what? I think you're right.
We should just extract hostname out of ROOT_URL or MOBILE_ROOT_URL and pass it to Cookies constructor. wdyt?

@dr-dimitru please check out the new PRs VeliovGroup/Meteor-Cookies#11 and #676.

@dr-dimitru @s-ol
how do we want to continue here? I think this issue is related to the Meteor core (cordova) as well.
I followed the discussion on https://github.com/VeliovGroup/Meteor-Cookies/pull/11 and want to put some questions here:

Does it make sense to get in touch with MDG about the underlying issue in cordova and security implications? I'd like to know if they are aware or not and also, if they care or not.
Is it possible to come up with a more decoupled solution, that does not break the current cookies structure?
Does it make sense to rewrite this package and the cookies package in a way, that cordova-specific code / behavior can be handled by adding some additional packages like for example ostrio:files-cordova and ostrio:cookies-cordova. Those package could provide additional logic that is only related to cordova to fix these issues while keeping the basic structures of both packages backwards-compatible.

@jankapunkt

Does it make sense to get in touch with MDG about the underlying issue in cordova and security implications?

I tried, it seems they don't care:
meteor/cordova-plugin-meteor-webapp#64

Is it possible to come up with a more decoupled solution, that does not break the current cookies structure?

This solution does not break the current cookies structure - my implementation may have introduced a bug, but it is 100% fixable and non critical. None of the changes I made (should) affect the API the plugin exposes in any way.

Does it make sense to rewrite this package and the cookies package in a way, that cordova-specific code / behavior can be handled by adding some additional packages...

Splitting is probably possible, but also highly likely to be an over-engineered solution, it probably takes a lot more time getting the packages to cooperate rather than just having a boolean option in ostrio:cookies that enables the cordova specific api endpoint.

@s-ol @jankapunkt

Does it make sense to get in touch with MDG about the underlying issue in cordova and security implications?

Well, MDG team went against cookies since day zero of Meteor. You can esily find a lot of articles why they aren't using and not going to support Cookies in any way.

This solution does not break the current cookies structure - my implementation may have introduced a bug, but it is 100% fixable and non critical. None of the changes I made (should) affect the API the plugin exposes in any way.

It does break current auto-tests, and who knows what is it going to break in current implementations around existent apps. We should keep in mind security and stability as this is one of the most popular community packages used in many production apps.

@s-ol @jankapunkt made first step with releasing<EMAIL_ADDRESS>Please, review its changes and let me know if any further work is required, if not — I'm ready to integrate with Meteor-Files library

@dr-dimitru I currently can't test the mobile part (@s-ol how about you?) but I could test the default browser-server integration.

Hello @jankapunkt
Yes, please, test as much as you can

@s-ol @jankapunkt friendly ping, any news on your end?

@dr-dimitru sorry for the delay, I'm having a busy week. I hope that I can take a look at everything tomorrow.

Okay @dr-dimitru, finally found the time to take a look and try it. The changes in meteor-cookies:4.2.0 are not enough - withCredentials is activated as necessary in cookies.set() but the cookies are never set on $METEOR_URL; they are only set via document.cookie which saves them for localhost:12*** but does not save them for $METEOR_URL.
Therefore when the cookies.set() request arrives at $METEOR_URL, even though withCredentials: true is set, since the browser hasn't seen any credentials/cookies for $METEOR_URL it also doesn't send any.
https://github.com/VeliovGroup/Meteor-Cookies/blob/2.4.0/cookies.js#L506
Here you set the Set-Cookie response-header value to what the Cookie request-header was.
This has no effect! The client already has these cookies, he just sent them to you!
If the client wanted to set cookies, but can't (since he is on cordova and on a different origin), then **he cannot send the cookies via the Cookie header (or any other header)! That is why I sent the cookies via the URI parameters in my PR. The server then has to turn the parameters into a Set-Cookie header. That is the only case where a Set-Cookie header is necessary.

If the client wanted to set cookies, but can't (since he is on cordova and on a different origin), then he cannot send the cookies via the Cookie header (or any other header)! That is why I sent the cookies via the URI parameters in my PR. The server then has to turn the parameters into a Set-Cookie header. That is the only case where a Set-Cookie header is necessary.

Got that part. How we'll be able to provide security over it? For example do not execute hooks when cookies passed via get query?

For example do not execute hooks when cookies passed via get query?

which hooks? This is supposded to happen in Meteor-Cookies on the '/cookies/setendpoint. Then ifSet-Cookieis properly used,Meteor-Files` will get actual cookies and not care.

How we'll be able to provide security over it? For example do not execute hooks when cookies passed via get query?

Regular clients can already set Cookies as they want - Cookies can never be trusted. Passing cookies via the query parameter is not insecure, as long as the same origin policy is properly set up to prevent malicious JS on other domains from overriding the cookies (or theoretically reading them, if a bug exposes them).

Thank you guys (@s-ol and @jankapunkt) for contribution, finally it's here 🥳
Released as v1.12.1.
Feel free to close it in case if the issue is solved on your end.

Hello everyone! v1.13.0 is out as I understood it is finally solves Cordova issue.
Feel free to close it in case if the issue is solved on your end.

@menelike are you able to use stock meteor-files + meteor-cookies in production on Cordova now?

@menelike friendly ping 🖖

@s-ol sorry I have missed your question.
@dr-dimitru Thanks for the heads up.
This issue has been closed as this works now (tested against 1.13.0).
Thanks @s-ol @dr-dimitru for all the effort you've put into this! 🚀❤

@menelike no worries.
Thank you for testing and confirming this one as fixed.
This is our team effort guys!
Thank you all @s-ol @menelike @jankapunkt
Feel free to reopen it in case if the issue is still persists on your end.


XD-3482 Add jdbc sink

based on Eric's PR #40

change to use JdbcMessageHandler from Spring Integration

Why drop the batching support? Could have our own, at least till SI provides what we need, WDYT?

I thought we decided to wait for it to be in SI -- https://jira.spring.io/browse/INT-3530

I thought we decided to wait for it to be in SI

Guys, what would you like to see there?
Any thought how to address?
Thanks

Merged as 9643875 alongside https://github.com/spring-cloud/spring-cloud-dataflow/commit/16c9db1a2138ca597c8a8dbd13966e3cece26c98


Write a section for "Correcting data based on NDA QC Report"

See title.

Can you clarify Eric? Thanks!

@kweldon1 If we chat over Slack that kind of defeats the purpose of open source Issue discussions. The issue title is suggesting there be a section on how to handle working with NDA QC reports if that matters for users of this tool.
Imagine you just finished using this tool and NDA comes back with corrections for you. A little troubleshooting doc on what to change and what to re-run would be good.

repo is no longer being maintained!


BFE User Templates - Add Property bug

The "Add Property" button has bugs when being used with templates.

Fixed with #67


Add themeColor option to metadata

Option to update the theme-color meta tag.
<meta name="theme-color" content="#ffffff">
Just check https://github.com/Polymer/pwa-helpers/commit/66878a75d363e913edcedf6719e79149b4f66e98. When https://github.com/Polymer/pwa-helpers/pull/47 is merged, I can rebase it to master.

Added setMetaTag() to set arbitrary meta tag in #58


allow for some broken links

In this PR, I changed the command that gets evaluated on the Travis a little bit, such that green builds can have a maximum number of 10 broken links (we can tweak the actual number). This is to avoid the (current) situation in which builds are always failing and as a results, the build status is no longer something people look at.
I'd be interested in people's opinions on whether this PR is actually an improvement or that we should enforce zero broken links all the time.
Inspired by the discussion here https://github.com/NLeSC/guide/pull/153.

Instead of this, would it be possible to test only the files that are actually changed for broken links? That would also reduce the number of false positives and still provide an accurate check. With this test you can add up to ten broken links and still pass all tests if you're "lucky".
We could then add a nightly build which allows a few broken links, or even better, links that are broken for less than a week.

Also, the test result would be much more readable if only the broken links were reported. Now you have to scroll through hundreds of working links to find them.

"test only the files that are actually changed for broken links"
is it possible? probably. but tricky because we change markdown files, but then ask the link checker to check whatever is being served on localhost:4000.
"broken less than a week"
indeed that would be preferable, but requires keeping data somewhere which is a hassle I wasnt willing to get in to for the moment. Same for counting the number of times a link was reported broken.
"reporting only broken links"
should be possible with either some advanced grepping or a python script of sorts. AFAIK, not possible with blc.

So I've made some changes such that there is a summary of broken links printed at the end of the travis output. Should cover the "reporting only broken links" part.
@sverhoeven suggested using travis cache for storing data on when links were broken

The addition of the summary is good, but I still think the test generates too much output: the occasional user will likely fail to spot the summary at the very end.
Indeed you can cache arbitrary directories with travis as described here: https://docs.travis-ci.com/user/caching/#Arbitrary-directories

"test only the files that are actually changed for broken links"
is it possible? probably. but tricky because we change markdown files, but then ask the link checker to check whatever is being served on localhost:4000.

The reason I suggested this is that I think it will really increase the feeling of ownership people have for the problem. With the list of files that have changed (something like git diff ...master --name-only) it should be fairly easy to write a small python script that post processes the output of blc to select only those files that have changed. E.g. if I would change the file best_practices/version_control.md, that would match broken links below 'Getting links from: http://localhost:4000/best_practices/version_control.html' in the blc output.

So I added folding to the travis output, which seems to work nicely.  Also summary is now even more compact.
I don't disagree on the points you make about ownership, but I won't be implementing that any time soon. I'll leave this PR for what it is now, feel free to merge or reject as you see fit.


Proposed Refactor Including Updates to Support Hugo v0.25

Refactor folder structure to reflect that of the default Hugo site structure to improve initial user ease of use, hopefully increasing contributions
Update Pygments config values to support Hugo v0.19+
Fix codefence short names (yml -> yaml, curl -> bash)
Update deprecated $.Now. to now
Implement Go block templates for simpler layout management
Move all checkout specific temporary files to .local directory
Update dockerfile to use Hugo v0.25
Move scripts to bin directory
Update swagger to 1.3.1
Remove sed command in vendor.sh that was causing asciidoctor errors and breaking Hugo v0.19+
Move static images from content directory to static directory
Fixed RSSLink var
Commit the API adoc files so you can have a near production docs site without having to generate the adoc files every time you checkout / clone the project
Remove unused Highlight.js
Replace GCSE with DocSearch
Simplify menus and remove redundant information

I am going to close this and resubmit it in smaller chunks so it is easier to merge.


Input type copied incorrectly

Heyo!
I have a failing test for copying over an input type:
https://github.com/kristoferjoseph/nanomorph/blob/master/test.js#L45
I made sure that this isn't an issue with bel already.

OK I was wrong! Turns out this was a bug in raynos/global/min-document which is a dependency of bel. It only happens when the code is not run in the browser which is why the bel tests were passing and not here.
I chased it down and got PRs opened for the fixes will PR here once bel merges.

Sweet, thanks for chasing it down! :D


Fix SecretScanning API by switching arguments from url to json

Closes: #2871.
This PR fixes the endpoint:
<EMAIL_ADDRESS>by sending its parameters in the body instead of as URL query parameters.
I was unable to edit #2871 directly, so this is a replacement.
Thank you, @jentfoo for the bug report and initial PR!

Yikes, the commit history is causing problems.  Starting fresh by closing.


Contact form in portfolio mobile project

This is the contact form for my portfolio project mobile version. I worked with the figma template number 1.
This is the third milestone of the project, where you can find:

A contact form added to my portfolio.
My coding partner @hafedEfheij and me worked on this project.

Please review my project and let me know if you have any comments.

Thank you @Shisui6!!!👏👏👏👌


Malformed presets will crash the game

latest.log
If you paste a prefix with improper json whatever when you press done nothing will happen, and pressing cancel takes you to an empty screen with the dirt texture background.

It looks like the game crashed here, but didn't shut down correctly. This will be fixed in next update.


Fails to parse UTF8 strings with BOM.

What steps will reproduce the problem?
1. create UTF8 file using ordinary text editor (Nodepad++). By default it is
created with BOM.
2. load file as raw binary file
3. pass to rapidjson Document to parse

Parsing fails with error "Expect either an object on array at root".

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 8 Jun 2012 at 8:16

I cloned repository from github just yesterday. Is it need to checkout from
google-code?

Original comment by<EMAIL_ADDRESS>on 8 Jun 2012 at 8:49

Ok, got it - not to use github's version.

Original comment by<EMAIL_ADDRESS>on 8 Jun 2012 at 11:11

BOM is not supported in 0.1x. It is not a "bug" but a missing feature.
Encoding conversions with BOM detection was implemented in the newer version in
the trunk.

Original comment by<EMAIL_ADDRESS>on 8 Jun 2012 at 8:21

Changed state: Fixed

Rapidjson does not have an official repository on github. I think it may be the
first released version 0.1, copied to github by someone.

Original comment by<EMAIL_ADDRESS>on 8 Jun 2012 at 9:31

Sorry, but I cannot understand what I must see there. What is "utf8.json"?
Anyway, I skipped first 3 bytes of UTF8+BOM text file within my project and
Rapidjson parsed it well.
Did you meant by that link that UTF8 with BOM is not supported and will not be
ever or that it is already supported?
Thank you.

Original comment by<EMAIL_ADDRESS>on 15 Jun 2012 at 1:58

Trunk SVN (from code.google.com) version does not support it either.

Original comment by<EMAIL_ADDRESS>on 14 Jun 2012 at 4:06

Hum. The examples and documentation is not ready yet.

I think this example may help you with the new encoding feature, which also
handles BOM.

http://code.google.com/p/rapidjson/source/browse/trunk/example/prettyauto/pretty
auto.cpp

Original comment by<EMAIL_ADDRESS>on 16 Jun 2012 at 1:36

Please refer to
http://code.google.com/p/rapidjson/source/browse/trunk/test/unittest/encodedstre
amtest.cpp

Original comment by<EMAIL_ADDRESS>on 15 Jun 2012 at 2:07


uniutils: init @ 2.27

Present in 9 repo's at the time of writing:
https://repology.org/project/uniutils/versions

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

nixpkgs-review pr 85540
1 package built:
uniutils

Result of nixpkgs-review pr 85540 run on x86_64-linux 1

2 packages built:

ascii2binary
uniutils

Closing due to inactivity from author.


A typo in builtin function name

In CodeGenerators.swift, this line defines some builtin functions.
var candidates = Set(["getPrototypeOf", "setPrototypeOf", "isExtensible", "preventExtension", "getOwnPropertyDescriptor", "defineProperty", "has", "get", "set", "deleteProperty", "ownKeys", "apply", "call", "construct"])

Shouldn't the preventExtension be preventExtensions according to the spec?

Ah yes correct, thanks! Fixed with https://github.com/googleprojectzero/fuzzilli/commit/4cf15dc993c634c18a3d76df6eeac5927fee7add


Function tutorial in documentation not working as intended.

Which Category is your question related to?
Lambda Layer
Amplify CLI Version
4.31.1
What AWS Services are you utilizing?
Lambda
Provide additional details e.g. code snippets
Hi all, thank you for your hard work.
I was following the tutorial outlined in the documentation here:
https://docs.amplify.aws/guides/functions/appsync-operations-to-lambda-layer/q/platform/js#set-up-your-layer
After running the script to generate graphql queries/mutations/subscriptions, no file is generated inside the graphql directory of the appsyncOperation. My guess is that babel has nothing to transpile because my codegen target is typescript. Do you guys have any additional info/way to fix this issue?
Thank you very much :D

Hi @9710willy I think you will need to add the typescript plugin to your babel config and add --extensions ".ts" to the babel command: https://babeljs.io/docs/en/babel-preset-typescript. You can also try out the --copy-files flag: https://babeljs.io/docs/en/babel-cli#copy-files


Modify logo's URL for Diffusion Electrolyte Mixture app

I merged PR #53. However, when checking the page https://big-map.github.io/big-map-registry/, the logo for the new app Diffusion Electrolyte Mixture is not rendered.

From Simon:
That’s because they are linking it to the GitHub page: https://github.com/saientan/Diffusion_Electrolyte_Mixture/blob/main/Diffusion.png instead of the actual image: https://raw.githubusercontent.com/saientan/Diffusion_Electrolyte_Mixture/main/Diffusion.png


Support ansible pre and post hook

Issue: https://github.com/stolostron/backlog/issues/19368

Add support for Ansible pre and post hook
Fixed bug with AppSet topology not display when one of the managedcluster that an app is deployed to is offline
Fixed the cluster count not showing when switching from the Overview tab to Topology

/test pr-image-mirror-mce

/retest-required

/retest-required


Add --rm or --keep on the run command (docker)

Currently it's possible to run a command in a specific container, but that container will stick around when the connection ends.
It'd be good having a --rm command to automatically remove it, like docker already does. Or otherwise having it as the default behavior and a --keep flag to keep the container.
Something like cx run -s Stack --server web --service web --keep 'rails c'

We'd love to add support for this, but it isn't currently possible due to the way that ContainerNet (underlying weave) creates and attaches to containers. We will keep this open to re-visit when weave has made some advancements in that area (https://github.com/weaveworks/weave/issues/47)

This is handled with the latest version of weave whereby we can include the -rm option


Remove Max from the list of people that automatically get invited to an incident

Right now, the security folks including Max get automatically invited to an incident when the incident is created. As Max is leaving on February 9, make sure that Calvin (if he wants to) or whoever else instead of Max (if any) gets automatically invited. Otherwise, an error most likely will occur once his account is deactivated.

This has been done and I am closing this ticket.


Editor not scrolling to centre when cursor out of view

Issue Type: Bug
Steps to reproduce:

Open a fairly long file
Place the cursor somewhere in the middle
Keeping the cursor active on the editor, scroll with mouse wheel up to another part of the code so the cursor is out of sight
Press → to move the cursor
Observe that the cursor comes back into view but stays at the very bottom of the editor window
Now scroll down so the cursor is out of view again
Press → to move the cursor
Observe that the cursor comes back into view but now stays at the very top of the editor window

Expected behaviour:
For the cursor to come back to the middle/centre of the editor, much like when you press CMD + G (find next) and you browse through the search results, it also centres the search item in the centre of the editor, which is helpful to quickly locate the cursor/active item.
Now I often find myself losing the cursor and not seeing where it is, because it's all the way at the top or bottom of the screen.
I believe this used to be centered in the previous version of Code, as I only started noticing this behaviour after I updated to this version.
VS Code version: Code 1.49.0 (e790b931385d72cf5669fcefc51cdf65990efa5d, 2020-09-10T17:39:53.251Z)
OS version: Darwin x64 18.7.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz (8 x 3400)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmetal: disabled_offmultiple_raster_threads: enabled_onoop_rasterization: unavailable_offopengl: enabled_onprotected_video_decode: unavailable_offrasterization: unavailable_offskia_renderer: disabled_off_okvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
24.00GB (0.97GB free)

Process Argv
-psn_0_1749419

Screen Reader
no

VM
0%

Extensions (17)

Extension
Author (truncated)
Version

project-manager
ale
11.3.0

vscode-eslint
dba
2.1.8

gitlens
eam
10.2.2

EditorConfig
Edi
0.15.1

file-icons
fil
1.0.26

gc-excelviewer
Gra
3.0.40

name-that-color
gui
0.1.2

json2csv
kha
0.0.1

l13-diff
L13
0.28.1

prettify-json
moh
0.0.3

vetur
oct
0.27.3

multi-command
ryu
1.4.0

vscode-fileutils
sle
3.0.1

control-snippets
svi
1.9.1

reopenclosedtab
uyi
1.1.0

highlight-matching-tag
vin
0.10.0

vscode-todo-highlight
way
1.0.4

This was always the behaviour. Cursor revealing when moving the cursor left / right was always configured to be the minimal amount of scrolling necessary to reveal the cursor.

So why is this the opposite of how search term revealing works, which shifts to the centre?
Seems to me like these two cases should be consistent, or at least configurable to allow us to make them consistent.

@alexdima can we please get a 2nd pair of eyes on this issue, as I don't believe this is a feature request but a bug as the behaviour is quite inconsistent at the moment.

@adamreisnz This is the behavior of the editor since its inception. You can verify this by downloading an old VS Code version (e.g. from 2017 - https://code.visualstudio.com/updates/v1_10 ) and checking it there. This behavior (to do minimal revealing when pressing arrow left or arrow right) is consistent with other code editors. I have just tried XCode and SublimeText. This is not a bug.

I'll just repeat my question then:

So why is this the opposite of how search term revealing works, which shifts to the centre?
Seems to me like these two cases should be consistent, or at least configurable to allow us to make them consistent.

So why is this the opposite of how search term revealing works, which shifts to the centre?

Because pressing arrow left/arrow right is really not the same as clicking on a search result. Yes, they both cause scrolling in the editor to reveal a position, but the use-cases are different:

when clicking on a search result, it is very likely that the user didn't look at the code before (or maybe since a while). Also, it is not clear what other code around the search result is relevant. It could be some code above, it could be some code below. The safest bet is to reveal the position in the center vertically, rendering both some code above and some code below. In other words, this is the method which most likely minimizes post-reveal scrolling.
when pressing arrow left/arrow right, the user has intentionally scrolled away from a position which they were looking at. They were looking at that position and then chose to intentionally scroll in a certain direction, let's say up. This is because it is very likely there was something up that they needed to look at in order to continue working at the current position. So when they press arrow left/arrow right we want to do the minimal amount of scrolling to reveal the position, as that might leave the code they were looking at still in the viewport. In other words, this is the method which most likely minimizes post-reveal scrolling.

Seems to me like these two cases should be consistent, or at least configurable to allow us to make them consistent.

Our goal is to minimize post-reveal scrolling and not to reveal in the same way in all scenarios. This issue has been left open and marked as a feature request to track the feature request to add a configuration option to be able to tweak how revealing works when pressing arrow left/arrow right.


Use cli to run text extraction for CCLW corpus using Adobe Extract API and Embedded Text parser

Use the pdf2text cli to extract the text from the CCLW corpus and generate the json output files in an s3 bucket.
The text and positional data extracted from the corpus can then be used in the next sprint to:

Support the investigation and development of post-processing heuristics
Develop the storage of passages in the database
Support the investigation of search databases

This should be done on the set of English language PDFs as identified in the prototype.

Hey team! Please add your planning poker estimate with ZenHub @eurolife @kdutia @opyate

@ingemar-cpr @stefl14 and I talked about doing a one-off data cleaning exercise for CCLW to get the document title from each PDF. Before doing this data load we should check that the data is suitable for this.

I emailed adobe today regarding outstanding BAD_PDF and 500 errors, and the timeout and variable page splitting limit issues we were having.


Loop exceptionally stopped + Trying to access offset on value of type null

using amphp/http-client: 4.4.0 and php 7.4.0
Reproducable script:
$builder = (new HttpClientBuilder);

        $client = $builder->build();

        $promise = call(
                function () use ($client) {
                    $url = 'https://samsung.com';

                    $request = new Request($url);
                    $response = yield $client->request($request);
                }
            );
        wait($promise);

Result:
Loop exceptionally stopped without resolving the promise {"exception":"[object] (Error(code: 0): Loop exceptionally stopped without resolving the promise at vendor/amphp/amp/lib/functions.php:232)
.....
[previous exception] [object] (ErrorException(code: 0): Trying to access array offset on value of type null at vendor/amphp/socket/src/Internal/functions.php:131)
[stacktrace]

I've also posted a similar issue in the amphp/socket repo, since the exception seems to be originating there: https://github.com/amphp/socket/issues/76

It outputs a proper error message for me: PHP Fatal error:  Uncaught Amp\Socket\TlsException: TLS negotiation failed: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages: error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small. I guess it can be resolved by reducing the security level on the TLS context.
I've applied a patch to amphp/socket that should now report Unknown error to you if there's no available message.

Hm, thats weird, do you have any idea why i might be getting different results?

Something like that might be the case, yes.

reducing the security level to 1 works, using OpenSSL 1.1.1d  10 Sep 2019
Now to figure out why i needed to reduce it :D

@duronrulez This means we should probably switch to using an error handler instead of error_get_last().

can confirm that if i put restore_error_handler() before the stream_socket_enable_crypto() call the error is being received in error_get_last(). In my case the handler is the default laravel exception handler: https://github.com/laravel/framework/blob/0b12ef19623c40e22eff91a4b48cb13b3b415b25/src/Illuminate/Foundation/Bootstrap/HandleExceptions.php
Obviously this doesn't solve anything, but at least i know whats breaking it.

Looks like its working 👍
All 3 contained exceptions are now being reported properly and catchable.
Not sure what is the process exactly, but when do you think an update will be available?

Tagged as v1.1.3. ;-)


Naming is confusing

There is already an existing tool named Scipio with a similar function that was started a full year before this one. A simple Google search would've yielded this. https://github.com/evandcoleman/Scipio

Hi.
Honestly, I didn't know your project when it was started.
It seems we happened to have the same idea.
There are no commits for your project over one year. So, I thought it was not active project.
But your opinion makes sense. You took the name first.
So, I'll consider renaming this project. It may take time.

Thank you, I very much appreciate that.

I apologize for the confusion about naming.
Repeatedly, I didn't mean to steal your idea.
I promise you to rename this project in a month. Please moment.


Smoketest failure: install and enable vscode-smoketest-check extension

https://dev.azure.com/monacotools/Monaco/_build/results?buildId=123090&view=logs&j=4d2898ab-dfbe-557e-92e7-aaac158fdd2f&t=aaa04083-6c0b-5a33-e907-3d845b07f0b5
 1) VSCode Smoke Tests (Web)
       VSCode Smoke Tests (Web)
         Extensions
           install and enable vscode-smoketest-check extension:
     Error: Timeout: get element '.extension-editor .monaco-action-bar .action-item:not(.disabled) .extension-action.uninstall' after 20 seconds.
      at poll (/__w/1/s/test/automation/out/code.js:176:19)
      at async Code.waitForElement (/__w/1/s/test/automation/out/code.js:257:16)
      at async Extensions.installExtension (/__w/1/s/test/automation/out/extensions.js:39:9)
      at async Context.<anonymous> (out/areas/extensions/extensions.test.js:19:13)

Please note that I just pushed a change that will disable the getting started page (which is a complex iframe). It is possible that this editor has caused a couple of issues.

I don't reproduce it anymore. Hence closing it.


Migrate infrastructure password management to BitWarden

We are currently using LastPass. Switching to BitWarden will allow us to be in sync with what the Eclipse Foundation uses.

From discussions yesterday Eclipse consider the BitWarden organisation to be purely for internal Eclipse staff use, so (with EF Infra team's support) we intend to persue creating our own organisation which we will manage ourselves, and allow the EF webmaster to be a member of that organisation to ease sharing of credentials.

Collections as follows with appropriate groups:

"Internal Services" will be things like AWX etc. that we run ourselves.
External services will be services we get from elsewhere e.g. Apple accounts, CloudFlare, JFrog etc. (Do we need to tier those for access? Things that releasers or API developers may require vs others)
Anything with - has no explicit permissions so is admin only (Currently myself, George, Martijn, and Tim)

(Not Changed colletions contain items that have been migrated but not yet updated. Once updated, they will be moved into one of the other collections)

Collection
Read access
Read-write access

AdoptOpenJDK (Not Changed)
AdoptOpenJDK
-

Infrastructure (Not Changed)
-
infrastructure

Cloud Providers
infrastructure
-

External Services
external-accounts
-

Internal Services
internal-accounts
-

Marketing
Marketing
-

Most passwords updated and categorised. A few outliers remain that I couldn't get into/change or were unused, and there are a few missing that I'll get around to. This list is mostly a "memo to self" so I complete them

[ ] Linaro (They started a new system so the old account doesn't work)
[ ] OSUOSL/aarch64
[ ] Marist (Now on self-service)
[ ] MacInCloud (Old account is empty)

Bumping to April to continue this work - GA/MV to progress the next steps for some of the other accounts.


Fix global linkages

Greetings!
First of all, thank you for your library, it's awesome and useful.
But here's a tiny fix I dare to propose: as soon some of the headers contain definitions without static specified, could you add it to them?
Currenlty those definitions have global linkage, which means that in certain cases someone is likely to get "multiple definitions" error, I can provide you a MCVE of that here or in a separate issue if you like.
The fix is small and should not affect performance, tests pass fine.
@yurymalkov , what'd you say?

Hi @MasterAler,
Thanks! Yeah, I do not think it might cause any problem. Will merge in a few days.


Dev/ markov read write data

Complete the implementation of markov read write data and test pass.
Realisation

pre-define a dictionary in scenario_tree.py -> __allocate_data() and return num_data_values of the dictionary.
In scenario_tree.py -> set_data_at_node(), write the data by input node_idx and data_dict, match the position and replace previous values. Raise an error when Data length wrong.
In scenario_tree.py -> set_data_at_node(), return the dictionary of all data information.

@Gczmy, I simplified your code. The main problem was that you assumed that the data has a very specific format. This would not be extensible in the future and would not allow us to easily store other information (e.g., s, y, etc). Now we can just store any information at the nodes of the tree.


[Messaging] No Retry for HostNotFound

Summary
The focus of these changes adjusts the retry policy to consider socket errors where the host is not found or not reachable to be terminal errors.  Previously, all socket errors were considered transient and would be retried.

API change check
API changes are not detected in this pull request.

/azp run net - eventhub - tests

/azp run net - servicebus - tests

/check-enforcer override

Failures in Service Bus live test suite are unrelated and have been occurring the last couple of days.  I didn't see an obvious test error to fix and didn't want to set them to ignore, so I'm going to just force the merge for these changes.


Fix handling of managed dependencies edge cases

The existing logic for filtering the dependency hierarchy misses some
edge cases, including the case where a dependency is declared without a
version (e.g. :dependencies [[foo]]) and the corresponding managed
dependency is declared in another project.clj file (as can be the case
when using plugins like lein-parent).
This change addresses these edge cases by delegating dependency
resolution to Leiningen.
Based on the commit history it appears that leiningen.nvd.deps was
was added to filter the classpath to only include dependencies declared
in a :dependencies or :managed-dependencies list. In particular,
there was a desire to exclude :plugins from the scanned classpath.
Based on #1 I believe there was also a desire to exclude certain base
profiles (presumably :user and :base).
Current versions of Leiningen do not appear to include :plugins in
get-classpath computation, but do include some other directories which
are irrelevant in this case (e.g. the local src and resource
directories). Rather than reverting back to using get-classpath this
change uses resolve-managed-dependencies to exclude these extraneous
classpath elements. The :base, :system and :user profiles are also
unmerged so that only dependencies of the project itself are included.

I will create a release at some point later on today.

Thank you!


Python install has no recommended package...

Related to #4109
$ sudo apt-get install libgrpc-dev python-dev python-pip
$ sudo pip install grpcio
Downloading/unpacking grpcio
  Could not find a version that satisfies the requirement grpcio (from versions: 0.10.0a0, 0.11.0b0, 0.11.0b1, 0.4.0a0, 0.4.0a1, 0.4.0a13, 0.4.0a14, 0.4.0a2, 0.4.0a3, 0.4.0a4, 0.4.0a5, 0.4.0a6, 0.4.0a7, 0.4.0a8, 0.5.0a0, 0.5.0a1, 0.5.0a2, 0.9.0a0, 0.9.0a1)
Cleaning up...
No distributions matching the version for grpcio
Storing debug log for failure in /root/.pip/pip.log

The natural thing to do is look at the end of the list... Things count up, right? Wrong! The highest version is actually 0.11.0b1!
When I do take the one from the end of the list (0.9.0a1), I get lots of errors.
Further, I think most people are going to be like "Wait, wtf? I can't just pip install grpcio ?"
Can we make something a default version on PyPI so that people don't have to figure out this ==? stuff?

Just FYI, here's what happens when I install the libgrpc-dev package and then try to use grpcio==0.9.0a1 :
$ sudo apt-get install libgrpc-dev python-dev python-pip
$ sudo pip install grpcio==0.9.0a1
... snip ...
       from grpc/_adapter/_call.c:34:

/usr/include/grpc/grpc.h:525:17: note: declared here

 grpc_call_error grpc_call_start_batch(grpc_call *call, const grpc_op *ops,

                 ^

grpc/_adapter/_call.c: In function ‘pygrpc_call_status’:

grpc/_adapter/_call.c:354:16: error: too few arguments to function ‘grpc_call_start_batch’

   call_error = grpc_call_start_batch(self->c_call, &op, 1, c_tag);

                ^

In file included from ./grpc/_adapter/_call.h:38:0,

                 from grpc/_adapter/_call.c:34:

/usr/include/grpc/grpc.h:525:17: note: declared here

 grpc_call_error grpc_call_start_batch(grpc_call *call, const grpc_op *ops,

                 ^

grpc/_adapter/_call.c: In function ‘pygrpc_call_cancel’:

grpc/_adapter/_call.c:363:38: error: too few arguments to function ‘grpc_call_cancel’

   return pygrpc_translate_call_error(grpc_call_cancel(self->c_call));

                                      ^

In file included from ./grpc/_adapter/_call.h:38:0,

                 from grpc/_adapter/_call.c:34:

/usr/include/grpc/grpc.h:580:17: note: declared here

 grpc_call_error grpc_call_cancel(grpc_call *call, void *reserved);

                 ^

grpc/_adapter/_call.c:364:1: warning: control reaches end of non-void function [-Wreturn-type]

 }

 ^

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-TGyR1A/grpcio/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-eNXxu2-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-TGyR1A/grpcio
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 248, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 39: ordinal not in range(128)

I'm not sure what you mean by 'things count up' --> 'wrong'. Why is 0.9.x considered more towards the 'end' than 0.11.x?

It looks like PIP does alphabetical sorting when showing you the list of available options, but our versions count numerically for each "group" (separated by dots):
For example, let's say we have versions: 0.1, 0.9, 0.11, 1.0 (for the illustration, I'm ignoring non-numeric characters)
PIP sorted (note that it's just saying "let's sort the numbers")
>>> versions = ['0.1', '0.9', '0.11', '1.0', '0.5']
>>> sorted(versions)
['0.1', '0.11', '0.5', '0.9', '1.0']

In other words, sorting this way treats `0.5
"What a human actually wants" (which compares each number between the dots)
>>> def compare_versions(a, b):
...     pieces_a, pieces_b = a.split('.'), b.split('.')
...     pieces_a, pieces_b = map(float, pieces_a), map(float, pieces_b)
...
...     # Make sure they're the right length
...     length_diff = len(pieces_a) - len(pieces_b)
...     if length_diff > 0: pieces_b += [0]*length_diff
...     elif length_diff < 0: pieces_a += [0]*-length_diff
...
...     for (piece_a, piece_b) in zip(pieces_a, pieces_b):
...         if piece_a < piece_b: return -1
...         elif piece_a > piece_b: return 1
...
...     return 0
>>>
>>> print sorted(versions, compare_versions)
['0.1', '0.9', '0.11', '1.0']

Forgive the long comment -- I got all geeky on this...
It looks like PIP does alphabetical sorting rather than using distutils.version sorting. This shows up as (not rendering as code here so you can see everything without scrolling):
$ sudo pip install grpcio
Downloading/unpacking grpcio
Could not find a version that satisfies the requirement grpcio (from versions: 0.10.0a0, 0.11.0b0, 0.11.0b1, 0.4.0a0, 0.4.0a1, 0.4.0a13, 0.4.0a14, 0.4.0a2, 0.4.0a3, 0.4.0a4, 0.4.0a5, 0.4.0a6, 0.4.0a7, 0.4.0a8, 0.5.0a0, 0.5.0a1, 0.5.0a2, 0.9.0a0, 0.9.0a1)
Cleaning up...
Note that the last one in the list is 0.9.0a1, but the "most recent" (if PIP was sorting based on distutils.version) would be 0.11.0b1. I figure the last one is the highest number (I see it's certainly bigger than 0.5.0a2...) so I just go with that, which then leads me to compile errors.
We can't fix how PIP does things in the scope of this issue, but we can set a default version so that this choice isn't required...

If you're interested, it looks like PIP is just calling sorted on the versions when showing you the list of available options, but that isn't really how versions should be sorted.
For example, let's say we have versions: 0.1, 0.9, 0.11b1, 1.0
PIP sorted (note that it's just saying "let's sort the strings")
>>> versions = ['0.1', '0.9', '0.11b1', '1.0', '0.5']
>>> sorted(versions)
['0.1', '0.11b1', '0.5', '0.9', '1.0']

What we expect here is what distutils.version provides, where we're treating each "section" (separated by dots) as it's own piece:
>>> from distutils.version import StrictVersion
>>> versions = ['0.1', '0.9', '0.11', '1.0', '0.5']
>>> print map(str, sorted(map(StrictVersion, versions)))
['0.1', '0.5', '0.9', '0.11b1', '1.0']

In other words, PIP is sorting in a way that treats version 0.5 and 0.50 as the same thing, whereas most versions treat 0.5 and 0.50 numerically as comparing 0.05 and 0.50 (and read it as "oh dot five" and "oh dot fifty")

I think we 'solved' (as best one might) this a while back by hiding older versions by default on PyPI. Feel free to re-open if my claim is bonkers.


Fix typos, indentations, and tailing spaces

Some people (like me) push commits that have typos, space indentations, and tailing spaces.
I can check my commits with editorconfig-checker and codespell now.

I think this can even be done online via github, right? I remember having done so before on kojix2' libui-ng bindings, doing typo corrections. That's quite easy to do that way. :)
Perhaps cody can assign matyalatte to do so. :D
(I can also go through the above list; I won't modify any code, though, as my C++ knowledge is way too limited to change things. But typos are easy to fix.)

Hmm one thing sounds weird though:
    322: Wrong indentation type(spaces instead of tabs)

Isn't it more common to use spaces these days? At the least
I always use spaces, after realising that mixing tabs with
documentation is not good. Spaces don't have that problem.
That was when I abandoned tabs completely.

Isn't it more common to use spaces these days?

See CONTRIBUTING.md for the existing style guide.

Some people (like me) push commits that have typos, space indentations, and tailing spaces.

Yes the plan is to eventually enforce strict consistency on all new PRs using clang-format.
https://developers.redhat.com/articles/2022/02/25/enforce-code-consistency-clang-format

@rubyFeedback

I think this can even be done online via github, right? I remember having done so before on kojix2' libui-ng bindings, doing typo corrections. That's quite easy to do that way. :)

Yeah, there should be a code check in the CI workflow.
But Idk if we should fix all existing typos with it.
Some of them might conflict with existing PRs.

Perhaps cody can assign matyalatte to do so. :D
(I can also go through the above list; I won't modify any code, though, as my C++ knowledge is way too limited to change things. But typos are easy to fix.)

Contributions are very welcome.
I don't plan to work on this issue (and the CI workflow) for now.
@cody271

Yes the plan is to eventually enforce strict consistency on all new PRs using clang-format.
https://developers.redhat.com/articles/2022/02/25/enforce-code-consistency-clang-format

Thanks for the info. Looks better than editorconfig-checker. I'll try it later.


Generate QC for sct_label_utils

Fixes #2703

The PR is ready for review.
The QC looks something like this:

If you want to try it you can do:
sct_qc -i /Volumes/sct_testing/large/sub-amuPAM50002/anat/sub-amuPAM50002_T1w.nii.gz -s /Volumes/sct_testing/large/derivatives/labels/sub-amuPAM50002/anat/sub-amuPAM50002_T1w_labels-disc-manual.nii.gz -p sct_label_utils

It would be nice to have the value of the label appear next to it
Do you mean pixel intensity for "the value" ?
Why didn't you use a vectorial matplotlib object (e.g. a dot), instead of using overlaid pixels?
I could implement the dot/circle considering the coordinates of the pixel location of the mask.

It would be nice to have the value of the label appear next to it

Do you mean pixel intensity for "the value" ?

yes

Why didn't you use a vectorial matplotlib object (e.g. a dot), instead of using overlaid pixels?

I could implement the dot/circle considering the coordinates of the pixel location of the mask.

no, what i meant was: instead of overlaying an actual pixelated image, use the vectorized display (if that's even possible?)

Something funky it happens to some labels for example sub-amuPAM50002_T2w_labels-disc-manual.nii.gz

While it renders well for sub-amuPAM50002_T1w_labels-disc-manual.nii.gz.

My understanding is that current implementation of QC looks into the mask (2d image) on the middle slice, I think the labels are in a different slice.
@jcohenadad @lrouhier is it a problem with the labels ? How should I approach this problem ?

@jcohenadad @lrouhier is it a problem with the labels ? How should I approach this problem ?

how do you retrieve the label coordinates to display it? if you only grab the AP and SI coordinates (sagittal orientation), then you don't care about where the label is defined in the RL coordinates, right? Or am I missing something?

I do results_mask_pixels = np.where(mask > 0) to find the coordinates, but this give me a 2D result. Maybe I should do a recursive np.where or for the complete 3D volume.

I do results_mask_pixels = np.where(mask > 0) to find the coordinates, but this give me a 2D result. Maybe I should do a recursive np.where or for the complete 3D volume.

can you point us to the code?

Here is the line : https://github.com/neuropoly/spinalcordtoolbox/pull/2734/files#diff-cd0ed16d3a599345c58ea8bbd2fe675dR183

Here is the line : https://github.com/neuropoly/spinalcordtoolbox/pull/2734/files#diff-cd0ed16d3a599345c58ea8bbd2fe675dR183
I even added a line print(mask.ndim) and I get 2, that means it's not passing the complete image.

mask is already processed by the qcslice.Sagittal() method, and it tries to find a center of mass here, which in your case is not relevant. You probably want to consider adding a method under the Sagittal() class that outputs the XY coordinates of interest. Does that make sense?

@jcohenadad thank you so much for the tip. I'll look into it.

@jcohenadad I tried to understand and implement the retrieve of the mask coordinates using the Sagittal() method, but I'm not familiar enough with the complete implementation of the class & method.
@lrouhier did you use these methods  ? are you more familiar with them ? do you think you can help me ?

I didn't use it previously but I might be able to help you. I will look into it!

@lrouhier you can use:
sct_qc -i /Volumes/sct_testing/large/sub-amuPAM50002/anat/sub-amuPAM50002_T1w.nii.gz -s /Volumes/sct_testing/large/derivatives/labels/sub-amuPAM50002/anat/sub-amuPAM50002_T1w_labels-disc-manual.nii.gz -p sct_label_utils
But the problem is that the current implementation fails for  sub-amuPAM50002_T2w_labels-disc-manual.nii.gz as it showed here.

One of the solution is to figure out a condition to put here
I am currently trying with:
elif np.argwhere(image.data).shape[0] < 25:
            print(np.argwhere(image.data))
            return [np.argwhere(image.data)[0][2]] * image.data.shape[2]

which seems to work but poses two problems:

if all label are not on the same slice we will have a problem --> we will need to create a function to project all label in one slice (I can do that)
If you do a spinal segmentation of a small image (let's say 2 or 3 slices you might have less than 25 pixels in the mask) --> we might need to check if the label are "discrete" in the IS direction.

IIUC the problem described in https://github.com/neuropoly/spinalcordtoolbox/pull/2734#issuecomment-637754548 is when the label is on an other slice than the medial slice (which is showed). Why not simply grab the I-S and A-P coordinate, and display the dot with these coordinates?

I think I misunderstood your previous comment. I thought that Qcimage() methods (used [here] (https://github.com/neuropoly/spinalcordtoolbox/blob/bc4c0e99e557cc4f654d2d919dd47ee86f9fa0f4/spinalcordtoolbox/reports/qc.py#L674) ) was expecting a 2D image produced by the slice method previously. Therefore we can create this image in slice() (by adding a projection method) or generate it with another function.

Therefore we can create this image in slice()

which image, and what is "slice()"?

(by adding a projection method)

why do you want to do a projection?

which image, and what is "slice()"?
Sorry if that was not clear I meant Sagittal method, which is used as a slicer (if I understand the code correctly). The created image would be a 3D image with all the label on the same slice .

why do you want to do a projection?
Because this will allow us to have the same workflow as before, we just modify the fname_seg image.

which image, and what is "slice()"?
Sorry if that was not clear I meant Sagittal method, which is used as a slicer (if I understand the code correctly). The created image would be a 3D image with all the label on the same slice .

why do you want to do a projection?
Because this will allow us to have the same workflow as before, we just modify the fname_seg image.

ok got it. yes, it makes sense (i think)

ok, this implementation is needed urgently (see https://github.com/spine-generic/spine-generic/pull/146)
who's on it? @lrouhier @alexfoias?

I Added the projection in qc not sure if it is the best option (WIP)

@alexfoias @lrouhier i'm onto something. i'll find another way to support sct_label_utils with the QC, without the need to do the projection. stay tuned

Example of QC output (with e7f1809fe13aa8174464d04f2531ee2588a80a69):

Tested  os sct_testing/large and it works well.