[Security][Twistlock][oss_high]: Need to upgrade jackson-databind to higher verison in gremlin jar

We have found high sev issues with the com.fasterxml.jackson.core:jackson_databind in the gremlin jar dependency used by Egeria using twistlock tool.
Below is the table that shows the current package version used and the last column is the version that needs to be upgraded to.
Vulnerability issue can be looked at here: https://nvd.nist.gov/vuln/detail/CVE-2020-24616 (can replace last CVE parameter with ones in the below table)

Image
Package
Version
Path
Type
CVE
CVSS
Severity
Status

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-24616
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-24750
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-25649
7.5
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-35490
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-35491
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-35728
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36179
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36180
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36181
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36182
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36183
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36184
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36185
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36186
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36187
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36188
8.1
high
fixed in <IP_ADDRESS>

omag
com.fasterxml.jackson.core_jackson-databind
<IP_ADDRESS>
/igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar/gremlin-shaded-3.4.8.jar
jar
CVE-2020-36189
8.1
high
fixed in <IP_ADDRESS>

Can this dependency be upgraded in current and previous Egeria versions?
Can we assert if any untrusted data (end-user provided data) is flowing through the gremlin-shaded-3.4.8.jar?

Judging by the jar file name (igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar) I believe this is the connector offered through Information Server / Watson Knowledge Catalog directly, not the open source connector...  @mlottesen ?

Judging by the jar file name (igc-omrs-server-<IP_ADDRESS>-SNAPSHOT.jar) I believe this is the connector offered through Information Server / Watson Knowledge Catalog directly, not the open source connector...  @mlottesen ?

@cmgrote Indeed it would be good to have clarification on this -- though it does seem we have a gremlin shaded jar in base egeria in any case

@cmgrote Indeed it would be good to have clarification on this -- though it does seem we have a gremlin shaded jar in base egeria in any case

@cmgrote Yes, that's the connector that's offered directly through IS/WKC.  For the above release we are using egeria-release-2.4 which has the org.apache.tinkerpop:gremlin-core:3.4.8 (https://github.com/odpi/egeria/blob/egeria-release-2.4/build.gradle#L129)
gremlin-core:3.4.8
|----gremlin-shaded:3.4.8
       |----com.fasterxml.jackson.core:jackson-databind:<IP_ADDRESS>

Link to gremlin-shaded:3.4.8 https://repo1.maven.org/maven2/org/apache/tinkerpop/gremlin-shaded/3.4.8/gremlin-shaded-3.4.8.pom
Looking at other tinkerpop dependencies: https://github.com/odpi/egeria/blob/egeria-release-2.4/build.gradle#L130-L131
Both dependencies
            implementation("org.apache.tinkerpop:gremlin-driver:3.4.8")
            implementation("org.apache.tinkerpop:tinkergraph-gremlin:3.4.8")

use gremlin-core:3.4.8.
I think these dependencies need to be updated to version 3.4.9 or above.
The gremlin-core 3.4.9 and above version uses
com.fasterxml.jackson.core:jackson-databind:2.11.3 or above versions.

@cmgrote Yes, that's the connector that's offered directly through IS/WKC.  For the above release we are using egeria-release-2.4 which has the org.apache.tinkerpop:gremlin-core:3.4.8 (https://github.com/odpi/egeria/blob/egeria-release-2.4/build.gradle#L129)
gremlin-core:3.4.8
|----gremlin-shaded:3.4.8
       |----com.fasterxml.jackson.core:jackson-databind:<IP_ADDRESS>

Link to gremlin-shaded:3.4.8 https://repo1.maven.org/maven2/org/apache/tinkerpop/gremlin-shaded/3.4.8/gremlin-shaded-3.4.8.pom
Looking at other tinkerpop dependencies: https://github.com/odpi/egeria/blob/egeria-release-2.4/build.gradle#L130-L131
Both dependencies
            implementation("org.apache.tinkerpop:gremlin-driver:3.4.8")
            implementation("org.apache.tinkerpop:tinkergraph-gremlin:3.4.8")

use gremlin-core:3.4.8.
I think these dependencies need to be updated to version 3.4.9 or above.
The gremlin-core 3.4.9 and above version uses
com.fasterxml.jackson.core:jackson-databind:2.11.3 or above versions.

@cryptox31 In 2.4 that is correct - we only updated this level in the current release (master) which is about to start the release process in the next few days. In fact 3.4.9 was only released on Dec 11th. Since then there's a 3.4.10, but that is likely to be added in Feb as we like to run through much of the monthly dev work with the new dependencies before packaging, time to catch any oddities.
So a version of the connector built with the next version of egeria (2.6) should address the concern with gremlin version

@cryptox31 In 2.4 that is correct - we only updated this level in the current release (master) which is about to start the release process in the next few days. In fact 3.4.9 was only released on Dec 11th. Since then there's a 3.4.10, but that is likely to be added in Feb as we like to run through much of the monthly dev work with the new dependencies before packaging, time to catch any oddities.
So a version of the connector built with the next version of egeria (2.6) should address the concern with gremlin version

@planetf1, I agree with you that we only make updates to master because we offer monthly releases.  The cost of retesting combinations of new dependencies on back-level versions is not affordable for our community.   @cryptox31  Sorry that we can not retrofit the fix to earlier releases.
I am also pleased that the issue and fix was being picked up and processed automatically as part of our monthly release cycle.  The cadence of the monthly release is tough at times but it does keep our dependencies at the latest available level.

@planetf1, I agree with you that we only make updates to master because we offer monthly releases.  The cost of retesting combinations of new dependencies on back-level versions is not affordable for our community.   @cryptox31  Sorry that we can not retrofit the fix to earlier releases.
I am also pleased that the issue and fix was being picked up and processed automatically as part of our monthly release cycle.  The cadence of the monthly release is tough at times but it does keep our dependencies at the latest available level.

@mandy-chessell @planetf1 Thank you for your comments. We will upgrade our egeria version to 2.6 or above in a later release, but for now, can we assert if user data flows from the
gremlin-core:3.4.8
|----gremlin-shaded:3.4.8
       |----com.fasterxml.jackson.core:jackson-databind:<IP_ADDRESS>

dependency?

@mandy-chessell @planetf1 Thank you for your comments. We will upgrade our egeria version to 2.6 or above in a later release, but for now, can we assert if user data flows from the
gremlin-core:3.4.8
|----gremlin-shaded:3.4.8
       |----com.fasterxml.jackson.core:jackson-databind:<IP_ADDRESS>

dependency?

@cryptox31 I'm afraid I don't have the information to be able to assert that.
It depends on what you mean by user data flows. Any data provided by a 'user' (ie API) is typically stored in the graph, so in that definition it absolutely does flow
If instead you mean whether the user input data like search queries are passed unfiltered, that's probably not the case - but I think still needs further analysis.
Additionally might guess jackson is only being used remotely but that needs further analysis
ie what user? what data?
Therefore my safe response is to say 2.6 has fixed the vulnerability & until demonstrated otherwise there is a potential exposure in prior releases, so you should upgrade

@cryptox31 I'm afraid I don't have the information to be able to assert that.
It depends on what you mean by user data flows. Any data provided by a 'user' (ie API) is typically stored in the graph, so in that definition it absolutely does flow
If instead you mean whether the user input data like search queries are passed unfiltered, that's probably not the case - but I think still needs further analysis.
Additionally might guess jackson is only being used remotely but that needs further analysis
ie what user? what data?
Therefore my safe response is to say 2.6 has fixed the vulnerability & until demonstrated otherwise there is a potential exposure in prior releases, so you should upgrade

I think we have done what we can here. closing

I think we have done what we can here. closing


Where is the self-attentive logic

i could not find the self-attention encoder. I found a LSTM that encodes the embeddings, but nothing else.

https://github.com/kaushalshetty/Structured-Self-Attention/blob/84ed64f9cee75f84a5d871a46508a308182e4a6d/attention/model.py#L107


Fix dvorak command

Dvorak command had left and right inverted. This simple fix swaps them back to be the correct way around.
Contains
Fix dvorak command (left and right were inverted)
How to test
Set keyboard to be a dvorak layout. Open up a game world, and run the dvorak command. The movement should have the same keys as a normal qwerty keyboard.

Can one of the admins please verify this patch?

From testing, this works as intended. Good work.
Merging in.


Empty nested object returned when not included in result

Hi everyone
This is the schema I have
var UserSchema = new Schema({
    firstName:          {type: String, required: true},
    name:               {type: String, required: true},
    email:              {type: String, required: true},
    birthday:           {type: Date, required: true},
    phone:              {type: String, required: true},
    picture:            {type: String},
    address:            {
                            street:         {type: String},
                            number:         {type: String},
                            zipcode:        {type: String},
                            city:           {type: String},
                            country:        {type: String}
                        }
});

If I execute the following query
User.find().select('firstName name birthday').exec(function(err, users) {
    if(err) {
        // something happened
    }

    console.log(users);
});

It returns the following results
{
    "_id": "551161373ddc2b97abbeeef5",
    "firstName": "Foo",
    "name": "Bar",
    "birthday": "Sun Apr 16 1972 00:00:00 GMT+0100 (CET)",
    "address": {}
}

Why is it returning the address? I don't want it because I don't ask for it. If I run the same query directly in mongo it does not return the address field so this is mongoose related.
Thanks in advance

Duplicate of #2503. Known limitation in how mongoose builds docs - it builds all nested paths because it's not designed to know whether that path will be empty or not. Use the minimize option if you want to get rid of the empty object in toObject() or toJSON() output.

Searched for this issue but didn't find it. Thanks for pointing me to #2503.

I've got the opposite: nested object is undefined (with no defaults) even if minimize: false is set and it's included in query. Is it intended?

Seems to be specific to Single Embedded Subdocs™ — probably worth creating a separate issue.

@inca yes please :)


Refactor JavaScript code for Signup.js ✨

Closes
Closes #1535
Description
1. Use named imports for better readability and maintainability.
2. Separate the functions for signup and Google signup for clarity and reusability.
3. Move the input field values retrieval and validation to separate functions.
4. Use more descriptive variable names.
5. Extract the password toggle logic into a function.
Checklist:

[x] I have linked the PR to the correct issue.
[x] I have read the Contribution Guidelines
[x] I have built and tested the changes, and they do not break or show any errors.

Hey @Divesh2003  Please Review and Merge the PR


⚠ Bofrost.es has degraded performance

In ded9344, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1495 ms

Resolved: Bofrost.es performance has improved in 13d1783 after 8 minutes.


Suppressions for checks, more unit tests

closes #85

PR-Codex overview
Detailed summary

Added VeryLongClassName class with @SuppressWarnings("OOP.LongClassNameCheck") annotation in two different locations.
Added maxClassNameLen configuration in pom.xml.
Renamed parsesClassNamesInRightFormat method to parsesInvalidClassNamesInRightFormat in ClassNamesTest class.
Renamed check parameter to checks in IsSuppressedTest class.
Added @ExtendWith(SuppressedLongClassName.class) annotation in passesWhenSuppressed method in LongClassNameCheckTest class.
Added @ExtendWith(ManySuppressions.class) annotation in passesWhenManySuppressed method in MethodChangesStateTest class.
Added CodeClass interface and Code, Declaration, and Default classes in the ru.l3r8y.parser package.
Added license header in the CodeClass, Code, Declaration, and Default classes.

The following files were skipped due to too many changes: src/main/java/ru/l3r8y/parser/Default.java, src/main/java/ru/l3r8y/parser/ClassNames.java, src/main/java/ru/l3r8y/parser/IgnoresSuppressed.java, src/test/java/ru/l3r8y/extensions/SuppressedLongClassName.java, src/test/java/ru/l3r8y/parser/DeclarationTest.java, src/test/java/ru/l3r8y/extensions/ParserDeclaration.java, src/test/java/ru/l3r8y/extensions/ManySuppressionsDeclaration.java, src/test/java/ru/l3r8y/parser/DefaultTest.java, src/test/java/ru/l3r8y/parser/IgnoresSuppressedTest.java, src/test/java/ru/l3r8y/parser/SuppressedChecksTest.java

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@l3r8yJ please review this Pull Request. Deadline (when it should be merged or closed) is 2024-01-08T12:10:03.968245.
You should check if the requirements have been implemented (partially or in full), if there are unit tests covering the changes and if the CI build passes. Feel free to reject the PR or ask for changes if it's too big or not clear enough.
Estimation here is 30 minutes, that's how much you will be paid. You will be paid even if this PR gets rejected.

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (c662b88) 76.99% compared to head (2c26838) 77.40%.
Report is 1 commits behind head on master.

Files
Patch %
Lines

...c/main/java/ru/l3r8y/parser/IgnoresSuppressed.java
77.77%
1 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff              @@
##             master     #125      +/-   ##
============================================
+ Coverage     76.99%   77.40%   +0.41%
- Complexity       70       78       +8
============================================
  Files            18       21       +3
  Lines           226      239      +13
  Branches          8       10       +2
============================================
+ Hits            174      185      +11
- Misses           46       47       +1
- Partials          6        7       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@l3r8yJ take a look, please

@rultor merge

@rultor merge

@l3r8yj OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@l3r8yj Done! FYI, the full log is here (took me 2min)

@l3r8yJ thank you for resolving this ticket. I've just added it to your active invoice. You can always check all your invoices and more on the Contributor Dashboard.


fix unstable tests

Signed-off-by: xufei<EMAIL_ADDRESS>What problem does this PR solve?
Issue Number: ref #5708, ref #5095
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[x] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/merge


Migrate to poetry

pipfile seems to be a dead-end as it did not received commit in years.
I suggest migrating to poetry, that has a very active development. I can provide a PR if this is OK with you.

I'm using pipenv which seems to be sort of the follow up project and is also pretty active. I have though about poetry but to me it seems like it's poetry or nothing as it doesn't seem to allow multiple dependency managers in the same python installation.
Though I did maybe want to go for poetry in a python 3.10. compatible version.

Closed by ee0c3896dfb0adb96ebd5a9367f9fe588da3ca6f


Add support for  elements

You can learn about what exactly this element is, and what sort of props it has here: click me!
What we want:

builder variants for each tag
element variants for each tag
All props defined in the above link are supported
It follows standard practices for this repo
It has proper KDoc annotation, in a similar style to the others
The code is idiomatic and not verbose

@daymxn You can assign this task  to me.


Instances stuck in "Initializing" state forever in one of the hosts.

Server version - v0.57.0-rc1
Validation test scripts were executed on a fresh install of v0.57.0-rc1with 3 hosts.
All of LB instances and health check enabled instances in one of the host in the setup are stuck in "Initializing" state forever.
LB instances and health check enabled instances in other 2 hosts are fine.

@ibuildthecloud
the reason for the failure - haproxy-monitor (host-api process) wasn’t running, but by some reason haproxy-monitor.pid was not null. And as both monit and cattle apply.sh rely on pid file to signal that the process is alive, it was never restarted. I believe the process was killed around the time @sangeethah observed connectivity issues: around 19:35
But the most concerning thing is, monit seems to report process statuses from 2 hours back:
date
Fri Jan 29 22:29:40 UTC 2016

root@5f0b3f3ec3e5:/# monit status
The Monit daemon 5.6 uptime: 2h 51m

Process 'rancher-net'
  status                            Running
  monitoring status                 Monitored
  pid                               1000
  parent pid                        1
  uptime                            58m
  children                          0
  memory kilobytes                  12088
  memory kilobytes total            12088
  memory percent                    0.3%
  memory percent total              0.3%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

Process 'rancher-metadata'
  status                            Running
  monitoring status                 Monitored
  pid                               555
  parent pid                        1
  uptime                            59m
  children                          0
  memory kilobytes                  29556
  memory kilobytes total            29556
  memory percent                    0.7%
  memory percent total              0.7%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

Process 'rancher-dns'
  status                            Running
  monitoring status                 Monitored
  pid                               546
  parent pid                        1
  uptime                            59m
  children                          0
  memory kilobytes                  6808
  memory kilobytes total            6808
  memory percent                    0.1%
  memory percent total              0.1%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

Program 'logrotate'
  status                            Waiting
  monitoring status                 Waiting
  data collected                    Fri, 29 Jan 2016 19:36:43

Process 'haproxy'
  status                            Running
  monitoring status                 Monitored
  pid                               12881
  parent pid                        1
  uptime                            0m
  children                          0
  memory kilobytes                  2968
  memory kilobytes total            2968
  memory percent                    0.0%
  memory percent total              0.0%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

Process 'haproxy-monitor'
  status                            Running
  monitoring status                 Monitored
  pid                               954
  parent pid                        1
  uptime                            58m
  children                          0
  memory kilobytes                  9396
  memory kilobytes total            9396
  memory percent                    0.2%
  memory percent total              0.2%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

Process 'charon'
  status                            Running
  monitoring status                 Monitored
  pid                               964
  parent pid                        1
  uptime                            58m
  children                          0
  memory kilobytes                  4112
  memory kilobytes total            4112
  memory percent                    0.1%
  memory percent total              0.1%
  cpu percent                       0.0%
  cpu percent total                 0.0%
  data collected                    Fri, 29 Jan 2016 19:36:43

System '5f0b3f3ec3e5'
  status                            Running
  monitoring status                 Monitored
  load average                      [0.78] [0.84] [0.79]
  cpu                               31.6%us 20.7%sy 0.0%wa
  memory usage                      589560 kB [15.5%]
  swap usage                        0 kB [0.0%]
  data collected                    Fri, 29 Jan 2016 19:36:43

I see the same same issue happen again when validation run was run on master builds from mar 17.

We need to switch to process regexp based checking to fix this.

https://github.com/rancher/cattle/pull/1526

Have not encountered this issue when testing with  v1.0.0-rc1


SA1110 (OpeningParenthesisMustBeOnDeclarationLine) reporting false positives

SA1110 is getting reported for the ( token following TriviaList in the following code:
private static SyntaxTriviaList XmlLineStart()
{
    return SyntaxFactory.TriviaList(
        SyntaxFactory.ElasticMarker,
        SyntaxFactory.DocumentationCommentExterior("/// "));
}

Ok, I see the problem.


Nested CSS Syntax highlighting inconsistency

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.89.0
OS Version: Ubuntu 23.10

Steps to Reproduce:

Try the following

a {
  &[href*="/abc"],
  &[href*="/def"] {
    color: #f50;
  }

  &[href*="/ghi"],
  &[href*="/jkl"] {
    color: #00f;
  }
}

Inconsistency in the resulting syntax highlighting

/duplicate of https://github.com/microsoft/vscode-css/issues/9

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


Break out WebsiteStatusesRefreshJob

Hopefully lessens these duration(s):

I'd say that's an improvement!


hackagebot keeps repeating itself

Hackagebot has been spamming the same line at least 5 times in the #haskell channel (and there are other duplications).
hackagebot cabal2nix 2.3.1, crypt-sha512 0, safecopy-migrate <IP_ADDRESS>

It would be great if this could be fixed since it becomes annoying very quickly.

I have quietened it on the #haskell channel. Please contact the #haskell-ops

Sorry about that 😞 I've fixed this in 0ada22bcbd2770a7190dd25d1a9c8209531b3501, and am currently verifying the fix.
The bug was due to a regression in 254fb9e7981302e4e2e35489585aeef78809c04a. In that commit, I refactored the bot to preserve the list of pending announcements across iterations, so that network failures do not cause the bot to lose this state.
That isn't a bad idea in and of itself. The problem was that I forgot to also clear this list after a successful iteration. This meant that the bot would announce an ever-growing set of packages, over and over again.
Ironically, if I had stuck with my initial release process and run it in a staging environment first, this bug would have been caught very quickly. It seems fitting that the one time I don't test before a deployment is also when it bursts into flames. Finagle's law in action...

I have removed the +q on hackagebot.

Looks like it's working okay now.


feat: enable native JDBC connections

Enables connections to PgAdapter from the native PostgreSQL JDBC driver. This requires a couple of minor tweaks to the startup protocol:

Some older versions of the JDBC driver require the encoding values to be in upper case (i.e. UTF8 instead of utf8).
The date style should also include the year-month-day order.
The value for standard_conforming_strings should be on or off (not true or false).

Overall LGTM. Maybe we should add in more pg-jdbc testing, but it doesn't have to be done in this PR.

Yeah, I'll bring in a lot more test cases in separate PRs. #23 already contains a lot of test cases, but I decided to split it into several smaller PRs so it would be easier to review. Many of the test cases in #23 will only succeed once #27 has been merged, as the tests use parameterized queries and the pg-jdbc driver sends parameter values in binary format as opposed to the default text format used by psql.


Textfield not working correctly with Strict Mode

If root component is wrapped in React.StrictMode, TextFields components don't work properly.
Ex. Label doesn't float on focus.

RMWC Version: 14.0.7
React Version: 18.2.0

Thank you for making an issue. We will keep this here, but this is a known issue. https://github.com/rmwc/rmwc/issues/815


[Bug]: Persistent OutOfMemoryError error when using speculative decoding

Your current environment

The output of `python collect_env.py`
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-1067-aws-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A10G
Nvidia driver version: 535.161.07
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.7
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.7
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        48 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               16
On-line CPU(s) list:                  0-15
Vendor ID:                            AuthenticAMD
Model name:                           AMD EPYC 7R32
CPU family:                           23
Model:                                49
Thread(s) per core:                   2
Core(s) per socket:                   8
Socket(s):                            1
Stepping:                             0
BogoMIPS:                             5600.00
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save rdpid
Hypervisor vendor:                    KVM
Virtualization type:                  full
L1d cache:                            256 KiB (8 instances)
L1i cache:                            256 KiB (8 instances)
L2 cache:                             4 MiB (8 instances)
L3 cache:                             32 MiB (2 instances)
NUMA node(s):                         1
NUMA node0 CPU(s):                    0-15
Vulnerability Gather data sampling:   Not affected
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; untrained return thunk; SMT enabled with STIBP protection
Vulnerability Spec rstack overflow:   Mitigation; safe RET
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Retpolines; IBPB conditional; STIBP always-on; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Not affected

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.5
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] optree==0.12.1
[pip3] sentence-transformers==2.7.0
[pip3] torch==2.3.1+cu121
[pip3] torcheval==0.0.7
[pip3] torchvision==0.18.1+cu121
[pip3] transformers==4.41.2
[pip3] triton==2.3.1
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: N/A
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	0-15	0		N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

🐛 Describe the bug
I get an out of memory error when using speculative decoding (SD). This happens in the middle of generation or after several batches completed successfully.
I have tried reducing max_model_len, max_num_batched_tokens and gpu_memory_utilization, but to no avail.
Usage
llm = LLM("nvidia/Llama-3.1-Minitron-4B-Width-Base", enable_prefix_caching=True, block_size=3, gpu_memory_utilization=0.8, max_num_batched_tokens=11000,  max_model_len=11000, speculative_model="ibm-fms/llama3-8b-accelerator", num_speculative_tokens=1, use_v2_block_manager=True)
sampling_params = SamplingParams(temperature=0.5, top_p=0.2, max_tokens=1)
formatted_prompts = ["array of len 10000 to 400000, each around 100-200 tokens"]
from outlines import models, generate
model = models.VLLM(llm)
generator = generate.choice(model, ["yes", "no"])
predictions = generator(formatted_prompts, sampling_params=sampling_params)

In contrast, when SD is disabled, runs always complete even at high max_model_len, max_num_batched_tokensandgpu_memory_utilization`.
llm = LLM("nvidia/Llama-3.1-Minitron-4B-Width-Base", enable_prefix_caching=True, block_size=3,max_num_batched_tokens=20000, gpu_memory_utilization=0.95, max_model_len=20000, use_v2_block_manager=True)

Error trace
OutOfMemoryError: CUDA out of memory. Tried to allocate 1.86 GiB. GPU 0 has a total capacity of 21.99 GiB of which 337.38 MiB is free. Process 30125 has 21.64 GiB memory in use. Of the allocated memory 18.20 GiB is allocated by PyTorch, with 27.10 MiB allocated in private pools (e.g., CUDA Graphs), and 2.26 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
File <command-1108086693486187>, line 1
----> 1 predictions = generator(formatted_prompts, sampling_params=sampling_params)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/outlines/generate/api.py:503, in SequenceGeneratorAdapter.__call__(self, prompts, max_tokens, stop_at, seed, **model_specific_params)
    497         return self.format_sequence(sequences)
    499 generation_params = self.prepare_generation_parameters(
    500     max_tokens, stop_at, seed
    501 )
--> 503 completions = self.model.generate(
    504     prompts,
    505     generation_params,
    506     self.logits_processor,
    507     self.sampling_params,
    508     **model_specific_params,
    509 )
    511 return format(completions)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/outlines/models/vllm.py:105, in VLLM.generate(self, prompts, generation_parameters, logits_processor, sampling_parameters, sampling_params)
    102 if sampler == "beam_search":
    103     sampling_params.use_beam_search = True
--> 105 results = self.model.generate(
    106     prompts, sampling_params=sampling_params, lora_request=self.lora_request
    107 )
    108 results = [[sample.text for sample in batch.outputs] for batch in results]
    110 batch_size = len(results)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/utils.py:1030, in deprecate_kwargs.<locals>.wrapper.<locals>.inner(*args, **kwargs)
   1023             msg += f" {additional_message}"
   1025         warnings.warn(
   1026             DeprecationWarning(msg),
   1027             stacklevel=3,  # The inner function takes up one level
   1028         )
-> 1030 return fn(*args, **kwargs)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/entrypoints/llm.py:345, in LLM.generate(self, prompts, sampling_params, prompt_token_ids, use_tqdm, lora_request, prompt_adapter_request, guided_options_request)
    336     sampling_params = SamplingParams()
    338 self._validate_and_add_requests(
    339     inputs=inputs,
    340     params=sampling_params,
    341     lora_request=lora_request,
    342     prompt_adapter_request=prompt_adapter_request,
    343     guided_options=guided_options_request)
--> 345 outputs = self._run_engine(use_tqdm=use_tqdm)
    346 return LLMEngine.validate_outputs(outputs, RequestOutput)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/entrypoints/llm.py:686, in LLM._run_engine(self, use_tqdm)
    684 total_out_toks = 0
    685 while self.llm_engine.has_unfinished_requests():
--> 686     step_outputs = self.llm_engine.step()
    687     for output in step_outputs:
    688         if output.finished:
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/engine/llm_engine.py:1325, in LLMEngine.step(self)
   1315     finished_requests_ids = self.scheduler[
   1316         0].get_and_reset_finished_requests_ids()
   1317     execute_model_req = ExecuteModelRequest(
   1318         seq_group_metadata_list=seq_group_metadata_list,
   1319         blocks_to_swap_in=scheduler_outputs.blocks_to_swap_in,
   (...)
   1323         running_queue_size=scheduler_outputs.running_queue_size,
   1324         finished_requests_ids=finished_requests_ids)
-> 1325     output = self.model_executor.execute_model(
   1326         execute_model_req=execute_model_req)
   1327 else:
   1328     output = []
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/executor/gpu_executor.py:129, in GPUExecutor.execute_model(self, execute_model_req)
    126 def execute_model(
    127     self, execute_model_req: ExecuteModelRequest
    128 ) -> Optional[List[Union[SamplerOutput, PoolerOutput]]]:
--> 129     output = self.driver_worker.execute_model(execute_model_req)
    130     return output
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/torch/utils/_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    113 @functools.wraps(func)
    114 def decorate_context(*args, **kwargs):
    115     with ctx_factory():
--> 116         return func(*args, **kwargs)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/spec_decode/spec_decode_worker.py:402, in SpecDecodeWorker.execute_model(self, execute_model_req)
    398 self._maybe_disable_speculative_tokens(
    399     disable_all_speculation, execute_model_req.seq_group_metadata_list)
    401 if no_spec:
--> 402     return self._run_no_spec(execute_model_req,
    403                              skip_proposer=disable_all_speculation)
    404 return self._run_speculative_decoding_step(execute_model_req,
    405                                            num_lookahead_slots)
File /usr/lib/python3.11/contextlib.py:81, in ContextDecorator.__call__.<locals>.inner(*args, **kwds)
     78 @wraps(func)
     79 def inner(*args, **kwds):
     80     with self._recreate_cm():
---> 81         return func(*args, **kwds)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/spec_decode/spec_decode_worker.py:494, in SpecDecodeWorker._run_no_spec(self, execute_model_req, skip_proposer)
    491         self.previous_hidden_states = HiddenStates(
    492             hidden_states, execute_model_req.seq_group_metadata_list)
    493     else:
--> 494         self.previous_hidden_states.update(
    495             hidden_states, execute_model_req.seq_group_metadata_list)
    497 if not skip_proposer:
    498     # We prepare the prefill hidden states here so that there no
    499     # additional complexity in worker for spec_decode vs non_spec_decode
    500     # flow and execute_model doesn't need additional modifications.
    501     execute_model_req.previous_hidden_states = \
    502         prepare_prefill_hidden_states(
    503             sampler_output.prefill_hidden_states)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-6e87dd7d-c6c7-4a06-8001-d206e2d3c391/lib/python3.11/site-packages/vllm/sequence.py:1217, in HiddenStates.update(self, hidden_states, seq_group_metadata_list, second_last_token_hidden_states)
   1215 assert len(seq_group_metadata_list) == len(hidden_states)
   1216 self._seq_ids.extend(get_all_seq_ids(seq_group_metadata_list))
-> 1217 self.hidden_states = torch.cat([self.hidden_states, hidden_states])
   1219 if self.second_last_token_hidden_states is not None:
   1220     # Adding dummy hidden_states to this to maintain same shape
   1221     self.second_last_token_hidden_states = torch.cat([
   1222         self.second_last_token_hidden_states,
   1223         torch.zeros_like(hidden_states)
   1224         if second_last_token_hidden_states is None else
   1225         second_last_token_hidden_states
   1226     ])```

### Before submitting a new issue...

- [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.

Same issue here with Qwen 2.5 models (7b and 0.5b).
vllm version: 0.6.2
GPU: 2x3090 (tensor_parallel_size=2)
llm = vllm.LLM(
    model=main_model_name,
    tokenizer=main_model_name,
    tensor_parallel_size=2,
    dtype="bfloat16",
    speculative_model=speculative_model_name,
    num_speculative_tokens=5,
    use_v2_block_manager=True,
    gpu_memory_utilization=0.8,
    enforce_eager=True,
    max_seq_len_to_capture=2048,
)

Same issue with n-gram speculative decoding. I've tried everything to go around this, but still happens. This happens with every version of vLLM from at least 0.6.1 to current, with Pytorch 2.5.1 and 2.4.0, and different vLLMs options for enable-eager, prefix caching etc.
In some cases the engine processes hundred or more requests ok, sometimes the crash happens soon after the first requests. Doesn't seem to happen on the first request.
This makes speculative decoding not usable for production, as the servers would throw errors and require frequent restarts.

reducing max_num_seqs works for me


Build SLURM client from source, instead of the one provided by CycleCloud.

Build SLURM client from source, instead of the one provided by CycleCloud.
Using 20.11.9 as 20.11.7 (and earlier versions) are not available from download.schedmd.com due to security vulnerability.

Allows using newer versions of SLURM.
Includes PMIx and Enroot/pyxis for CentOS, AlmaLinux and Ubuntu.

What about the SLURM server and client ansible playbooks ? they use version 20.11.7 by default and should be upgraded to 20.11.9 by default I guess.

What about the SLURM server and client ansible playbooks ? they use version 20.11.7 by default and should be upgraded to 20.11.9 by default I guess.

Now building SLURM server as well by default (with job_submit cyclecloud plugin).
Changed slurm version to 20.11.9 (option build_rpms=true)
Would make sense now to have slurm version as a setting in config.yml, do you agree?

Yes it make sense to have slurm version in the config file


Use enum as namespace

Given those structs/classes only have static function, they should be enum, that makes impossible initialise them.
They basically becomes simple namespaces

Warnings

:warning:

Any changes to library code should be reflected in the Changelog.
Please consider adding a note there and adhere to the Changelog Guidelines.

:warning:

Sources/RunnerLib/Runtime.swift#L33 - TODOs should be resolved (Support running from a fork?). (todo)

  Generated by :no_entry_sign: Danger Swift against d75d4b33e4c5f3e91c1cf61e1e25a5f1e19de44c


[DNM] stdlib: Add reasync variants of '&&', '||' and '??'

Fixes rdar://problem/72770687.
We're not ready to merge this just yet but I wanted to re-run source compat to see if the debug build issues were fixed.

@swift-ci Please test source compatibility

@swift-ci Please test source compatibility

@swift-ci Please test

Curious about the roadmap or any progress on different fronts regarding this issue. cc @DougGregor

Would you mind sharing why this is closed and where I can continue to follow resolution for this problem?


Change by_text to use inner text for searching

by_text was searching for individual text nodes but this makes it
difficult to use with templating frameworks that build an elements text
using multiple text nodes.

bors r+


DeepGraphInfomax prediction problem

❓ Questions & Help
Hello!
I tried to use DeepGraphInfomax example here to embed nodes on multiple graphs. Model converges good, but i get weird results then i try to predict embeddings for all graphs in training set: first graph embeddings are making sense and do actually group similar nodes together on UMAP projection, but every other graph after that looks like it`s index was shuffled. To test, i removed first graph on learning phase, and trained on all others - same, first graph (which was not present in training data) have sensible embeddings, all other - completely random on UMAP projection. I removed first graph completely, now second graph shows sensible embeddings, and all others are shuffled. I feel like this may be connected to how data is stored right now, but cant quite understand whats wrong.
I create Data objects like that:
from sklearn.preprocessing import OneHotEncoder

data_type_encoder = OneHotEncoder(sparse=False)
data_type_encoder.fit(results['type'].values.reshape(
    -1, 1))

dataset_list = []

for group_id,  group_data in tqdm_notebook(results.groupby(['group'])):

    print(group_id, group_data.shape)

    group_adjacency = groups_adj[group]

    graph = nx.Graph()

    for _, data in group_data.iterrows():
        graph.add_node(data['index'])

    for i, near_nodes_idx in enumerate(group_adjacency):
        if i not in graph.nodes():
            continue

        if near_nodes_idx.size == 0:
            continue

        for node_idx in near_nodes_idx:
            if node_idx not in graph.nodes():
                continue

            graph.add_edge(i, node_idx)

    graph = nx.to_scipy_sparse_matrix(graph,
                                      nodelist=group_data['index'].to_list())

    edge_indices = from_scipy_sparse_matrix(graph)[0]

    sample_data = torch.Tensor(
        data_type_encoder.transform(group_data['type'].values.reshape(
            -1, 1)))

    sample_data_object = Data(edge_index=edge_indices,
                              x=sample_data)

    dataset_list.append(sample_data_object)

Then is pass this list to dataloader and train, just like in example, except on multiple graphs, so i set cached to False in GCNConv:
import torch
import torch.nn as nn
from torch_geometric.nn import GCNConv, DeepGraphInfomax
from torch_geometric.data import Data, DataLoader

dataloader = DataLoader(dataset_list, batch_size=32, shuffle=True)

class Encoder(nn.Module):
    def __init__(self, in_channels, hidden_channels):
        super(Encoder, self).__init__()
        self.conv = GCNConv(in_channels, hidden_channels, cached=False)
        self.conv_2 = GCNConv(hidden_channels, hidden_channels, cached=False)
        self.conv_3 = GCNConv(hidden_channels, hidden_channels, cached=False)
        self.conv_4 = GCNConv(hidden_channels, hidden_channels, cached=False)

        self.prelu = nn.PReLU(hidden_channels)

    def forward(self, x, edge_index):
        x = self.conv(x, edge_index)
        x = self.conv_2(x, edge_index)
        x = self.conv_3(x, edge_index)
        x = self.conv_4(x, edge_index)
        x = self.prelu(x)

        return x

def corruption(x, edge_index):
    return x[torch.randperm(x.size(0))], edge_index

device = torch.device('cuda:1' if torch.cuda.is_available() else 'cpu')

model = DeepGraphInfomax(
    hidden_channels=64,
    encoder=Encoder(14, 64),
    summary=lambda z, *args, **kwargs: torch.sigmoid(z.mean(dim=0)),
    corruption=corruption).to(device)

optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

def train():
    model.train()
    optimizer.zero_grad()

    mean_loss = []

    for data in dataloader:
        data = data.to(device)
        pos_z, neg_z, summary = model(data.x, data.edge_index)

        loss = model.loss(pos_z, neg_z, summary)
        loss.backward()

        mean_loss.append(loss.item())

        optimizer.step()

    return np.mean(mean_loss)

for epoch in range(700):
    loss = train()
    print('Epoch: {:03d}, Loss: {:.4f}'.format(epoch, loss))

Then i preform prediction like this:
group_embeddings = []

dataloader = DataLoader(dataset_list, batch_size=32, shuffle=False)

model.eval()
for data in dataloader:
    data = data.to(device)

    pos_z, _, _ = model(data.x, data.edge_index)

    node_embeddings = pd.DataFrame(pos_z.detach().cpu().numpy())

    group_embeddings.append(node_embeddings)

group_embeddings = pd.concat(group_embeddings)
group_embeddings = group_embeddings.reset_index(drop=True)

After that, first graph in list is always shows sensible embeddings, all others completely random. What can be the case here?

Example, graph is 2D map of nodes with different type, correct embeddings:

Now shuffled embeddings, characteristics stripes, like indexes are off:

That's interesting, as it seems to not generalize to unknown graphs at all. Note that in the paper, the inductive learning setup is quite different to the one applied to transductive tasks. It uses a different GNN operator, and uses a different corruption function (randomly sampled training graphs as negative examples). This might fix your problems.

Well the weird thing is, that first graph in example is unseen by network on training. And it produces correct embeddings. I expected overfit and informative embedings on graphs on which network is trained, and random embeddings on test data, here the situation is opposite. It is as if somehow order of graphs on prediction matters, batch shuffles nodes, or i create dataloader incorrectly.

Dataset consists of 5 graphs, now i tried to take this situation to extreme, i take only last 5 graph to train on and overfit for 2000 epochs, and then predict embeddings for all of them. Again graph 1 which is first in data list produces correct embeddings, all other, including 5th, on which model is fitted give shuffled embeddings. This reproduces the i remove graph 1 from data list completely, and graph 2 starts to produce correct embeddings, all others look random.

Yes, i figured it out! It was indeed error on indexing on my part, when i concatenated the data on prediction phase, probably i have different graph order in original dataframe before groupby operation. In case some one have similar problem with 2d graphs, image of graph with characteristics "stripes" pattern can help to guide your way in solving this bug.
As for model, it generalizes to new graphs quite good, thanks for quick response)

Awesome :)


Using indexed accessors for DataGridColumn.SortMemberPath doesn't work

Assuming i have the following object for a DataGrid row which contain most of the values to display in a dictionary called Properties:
interface IObject
{
    string Name {get;set;}
    IDictionary<string,object> Properties {get;set;}
}

When using "Name" for DataGridColumn.SortMemberPath, it works as expected. However, if i use "Properties[ColumnId]" for DataGridColumn.SortMemberPath, the DataGrid renders this column as not sortable. Looks like that indexed property accessors are not working.

Also it is not possible to sort by array item with property path like "Person[0].FirstName".
Although sorting by dictionary with path like "Dict[Key]" works for me now.


--no-tweets gets stuck when plotting route

Hey there & awesome use of OpenData!!
rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:        22.04
Codename:       jammy
rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ poetry --version
Poetry (version 1.2.0)
rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ python3 --version
Python 3.10.4

rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ poetry run nopeusbotti   --speed-limit 10   --north 60.2246 --south 60.2211 --east 24.7987 --west 24.7842 --route 114 --route 215 --route 200 --route 214 --route 239 --route 532 --route 224 --route 227 --no-tweets
2022-09-13 14:12:06,207 Run with --no-tweets: only producing figures, will not send any tweets
2022-09-13 14:12:06,360 Client connected (rc = 0)
2022-09-13 14:12:06,986 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2114/#
2022-09-13 14:12:07,474 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2215/#
2022-09-13 14:12:07,986 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2200/#
2022-09-13 14:12:08,504 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2214/#
2022-09-13 14:12:09,092 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2239/#
2022-09-13 14:12:09,606 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2532/#
2022-09-13 14:12:10,230 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2224/#
2022-09-13 14:12:10,778 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2227/#
2022-09-13 14:13:28,906 (6, 454, '2022-09-13', '13:29') has entered the monitored area
2022-09-13 14:13:50,843 (22, 1154, '2022-09-13', '13:34') has entered the monitored area
2022-09-13 14:14:01,785 (22, 1128, '2022-09-13', '13:45') has entered the monitored area
2022-09-13 14:15:02,031 (6, 454, '2022-09-13', '13:29') has left the area, plotting route
^C
Aborted!
rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$

Nothing happens for several minutes, no image is created (at least in any of nopeusbotti folders) and no further tracking is done. Tried this several times. Also tried with the code in the pending pull request -> the --csv-only option works and the csv log file is created successfully so the used parameters should be OK, I assume.
Any pointers what could cause this?
r

Ah, seems to work. Just takes a long time at least on this virtual machine :D
rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ poetry run nopeusbotti   --speed-limit 20   --north 60.2246 --south 60.2211 --east 24.7987 --west 24.7842 --route 114 --route 215 --route 200 --route 214 --route 239 --route 532 --route 224 --route 227 --no-tweets
2022-09-13 14:21:25,687 Run with --no-tweets: only producing figures, will not send any tweets
2022-09-13 14:21:25,859 Client connected (rc = 0)
2022-09-13 14:21:26,500 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2114/#
2022-09-13 14:21:27,161 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2215/#
2022-09-13 14:21:27,767 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2200/#
2022-09-13 14:21:28,306 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2214/#
2022-09-13 14:21:28,888 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2239/#
2022-09-13 14:21:29,539 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2532/#
2022-09-13 14:21:30,090 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2224/#
2022-09-13 14:21:31,181 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2227/#
2022-09-13 14:21:31,578 (12, 1918, '2022-09-13', '13:53') has entered the monitored area
2022-09-13 14:21:31,747 (22, 900, '2022-09-13', '13:44') has entered the monitored area
2022-09-13 14:21:41,861 (6, 450, '2022-09-13', '14:18') has entered the monitored area
2022-09-13 14:21:57,742 (22, 900, '2022-09-13', '13:44') has left the area, plotting route
2022-09-13 14:26:18,484 Saved plot to 64863248-c259-4a8f-b15f-e30b088a099e.png
2022-09-13 14:26:18,485 (12, 1918, '2022-09-13', '13:53') has left the area, plotting route
2022-09-13 14:32:51,685 Saved plot to 674318f9-f11c-44bc-a960-c76cab2bdb47.png
2022-09-13 14:32:52,820 Client connected (rc = 0)
2022-09-13 14:32:53,361 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2114/#
2022-09-13 14:32:53,978 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2215/#
2022-09-13 14:32:54,626 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2200/#
2022-09-13 14:32:55,205 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2214/#
2022-09-13 14:32:55,780 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2239/#
2022-09-13 14:32:56,351 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2532/#
2022-09-13 14:32:56,955 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2224/#
2022-09-13 14:32:57,553 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2227/#
2022-09-13 14:32:58,362 (12, 1402, '2022-09-13', '14:08') has entered the monitored area
2022-09-13 14:32:58,770 (22, 1166, '2022-09-13', '13:54') has entered the monitored area
2022-09-13 14:32:58,809 (6, 450, '2022-09-13', '14:18') has left the area, plotting route
2022-09-13 14:32:59,000 Saved plot to 6ddec04c-0aaf-42c8-8733-c588042a98b3.png
2022-09-13 14:33:29,792 (6, 462, '2022-09-13', '13:54') has entered the monitored area
2022-09-13 14:34:07,198 (12, 1402, '2022-09-13', '14:08') has left the area, plotting route

Reopening. There's propably something that someone would want to have a look at.
Looks like that it took about 70 minutes to create a single plot - and then receiving error 500 from digitransit was too much and the bot crashed.
``rhk@rhk-VirtualBox:~/nopeusbotti/nopeusbotti$ poetry run nopeusbotti   --speed-limit 20   --north 60.2246 --south 60.2214 --east 24.8010 --west 24.7842 --route 114 --route 215 --route 200 --route 214 --route 239 --route 532 --route 224 --route 227 --no-tweets
2022-09-13 15:10:42,664 Run with --no-tweets: only producing figures, will not send any tweets
2022-09-13 15:10:42,823 Client connected (rc = 0)
2022-09-13 15:10:43,415 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2114/#
2022-09-13 15:10:43,999 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2215/#
2022-09-13 15:10:44,881 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2200/#
2022-09-13 15:10:46,289 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2214/#
2022-09-13 15:10:46,967 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2239/#
2022-09-13 15:10:47,470 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2532/#
2022-09-13 15:10:48,008 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2224/#
2022-09-13 15:10:48,742 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2227/#
2022-09-13 15:10:49,024 (6, 445, '2022-09-13', '14:28') has entered the monitored area
2022-09-13 15:10:49,883 (22, 1234, '2022-09-13', '14:38') has entered the monitored area
2022-09-13 15:11:59,942 (22, 1234, '2022-09-13', '14:38') has left the area, plotting route
2022-09-13 16:21:53,727 Saved plot to 215929c5-98da-4262-82a2-06237daddf2f.png
2022-09-13 16:22:55,316 Client connected (rc = 0)
2022-09-13 16:22:55,986 Subscribing to /hfp/v2/journey/ongoing/vp/+/+/+/2114/#
Traceback (most recent call last):
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/transport/aiohttp.py", line 304, in raise_response_error
resp.raise_for_status()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 500, message='Internal Server Error', url=URL('https://api.digitransit.fi/routing/v1/routers/hsl/index/graphql')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 1, in
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/rhk/nopeusbotti/nopeusbotti/cli.py", line 69, in main
client.loop_forever()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
rc = self._loop(timeout)
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 1164, in _loop
rc = self.loop_read()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 1556, in loop_read
rc = self._packet_read()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 2439, in _packet_read
rc = self._packet_handle()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 3039, in _packet_handle
return self._handle_connack()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/paho/mqtt/client.py", line 3138, in _handle_connack
on_connect(
File "/home/rhk/nopeusbotti/nopeusbotti/bot.py", line 71, in on_connect
topic = self.get_mqtt_topic(route)
File "/home/rhk/nopeusbotti/nopeusbotti/bot.py", line 55, in get_mqtt_topic
result = client.execute(query)
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/client.py", line 388, in execute
data = loop.run_until_complete(
File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
return future.result()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/client.py", line 285, in execute_async
return await session.execute(
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/client.py", line 1220, in execute
result = await self._execute(
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/client.py", line 1126, in _execute
result = await asyncio.wait_for(
File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/transport/aiohttp.py", line 326, in execute
await raise_response_error(resp, 'No "data" or "errors" keys in answer')
File "/home/rhk/.cache/pypoetry/virtualenvs/nopeusbotti-Q_9Pjy1C-py3.10/lib/python3.10/site-packages/gql/transport/aiohttp.py", line 306, in raise_response_error
raise TransportServerError(str(e), e.status) from e
gql.transport.exceptions.TransportServerError: 500, message='Internal Server Error', url=URL('https://api.digitransit.fi/routing/v1/routers/hsl/index/graphql')
``

For what it's worth, I had no issue running your command:

...and yeah, it looks like a good use case for the bot.
Since there are no CPU or RAM woes, I suspect something may be off with getting the basemap for the map plot over the internet.

At some point it started to run smoothly:
2022-09-13 21:05:37,079 (6, 881, '2022-09-13', '20:30') has entered the monitored area
2022-09-13 21:05:50,532 (6, 461, '2022-09-13', '21:01') has left the area, plotting route
2022-09-13 21:05:50,823 Saved plot to 0b6b814a-115a-4a27-8aaf-d0491b02b6e1.png
2022-09-13 21:06:21,543 (6, 457, '2022-09-13', '21:03') has entered the monitored area
2022-09-13 21:06:22,360 (6, 776, '2022-09-13', '21:01') has left the area, plotting route
2022-09-13 21:06:22,656 Saved plot to 74c956f4-43c4-4756-b821-18f792780619.png
2022-09-13 21:06:37,937 (22, 1035, '2022-09-13', '21:03') has left the area, plotting route
2022-09-13 21:06:38,297 Saved plot to 7c9cecb4-6147-4344-a850-730e887a0ca1.png
2022-09-13 21:06:44,288 (6, 881, '2022-09-13', '20:30') has left the area, plotting route
2022-09-13 21:06:44,642 Saved plot to 9ee554f9-89f1-4abd-b5a5-4c3859fce4be.png
2022-09-13 21:07:31,565 (6, 457, '2022-09-13', '21:03') has left the area, plotting route
2022-09-13 21:07:31,881 Saved plot to 78bbd39e-171b-450e-bb5e-c81bb9739476.png
and has so far been running smoothly. So it's possible, that there's been some glitches in the connection / background services?
Oh yeah, and the speed limit there - luckily - is 60kmh, not 20 - just threw 20 there as an example :D

Hah, alright.
From the logs it looks like your MQTT connection dies while the plot takes ages to do, after which the application starts to fetch the topic names again and encounters another (probably unrelated and temporary) error with the GraphQL API there. Technically we wouldn't need to get the topic names here again, so I could add some caching for those.
But that's really unrelated to the slowness issue here, which most probably was just related to temporary issues with either your internet connection or something in the background services that provide the maps.


messages: Use user ID for sender and group-pm-with operators.

Both frontend and backend migration is done. The only thing left is stream:; I was done with it but ran into a issue where for some reason get_cache_with_key isn't happy for reason. It throws error saying AttributeError: 'int' object has no attribute 'strip'.
Once we get done with stream: we can provide a better example in the docs which we have been meaning to do.
Testing Plan:

Check that user ID is sent for group-pm-with and sender operators. Also check for sender:me case.
Try out incorrect emails just to make sure no error is thrown.
Unit Tests

Also would doing a rewrite so that get_user_including_cross_realm supports both user ID and email be better? Currently we have to check that lot of time that I think it almost would be better if the function could handle it.

I haven't looked at all the callers, but I agree it'd be nice to have a version of get_user_including_cross_realm that's maybe get_user_by_email_or_id.  It's possibly all the uses of the existing function should migrate to the new function + name; or possible we'll have a couple call points that we intend to only support email addresses.
We definitely want to encapsulate this complexity in a less code-duplicating heavy way.

BTW, this looks great, merged, thanks @priyank-p!


If the compiler output is a JSON stream, don't restrict chunk naming

With JSON streams, it's the responsibility of the caller to create the file structure.
Allows chunk names to contain path segments and other non-identifier characters.

Imported and sent for review

Chad, one other internal review comment - do you know if there are any tests for this? If not, is it reasonable to add a unit test?

Review issue addressed. Test added. Commits squashed.


[CS2113T-W17-3]-mong-nus-B-Reminders

This commit contains the reminder function created in
ReminderCommand.java file.

Looks good to me


Git Hub

I have created a anew folder and .git is also created. first time made changes in the visual code are getting captured when checked on Git status.
Once add and commit were made, 2nd time changes are not getting captured.
After making changes  git status message is giving below msg.
$ git status
On branch QA
nothing to commit, working tree clean
Even I cant see color change in visual code after making 1st change.
Please help me in understanding this

Hi Aamukta,
Basically git doesn't identify empty folders it identifies only files so first you create some files and then try.


Monday source documentation

What do these changes do?

Added documentation for Monday source

Type of documentation

[x] New documentation
[ ] Documentation update
[ ] Hotfix

@yashasvibajpai do we have an update on this? When is the source expected to be released?

@yashasvibajpai do we have an update on this? When is the source expected to be released?

@ameypv-rudder we are still caught with server release. Until then this would be on hold


Auto Shooting/Aligning

This Should:

Automatically find the nearest target
Automatically align to the target
Automatically adjust shooter speed to hit the target

All code should be written, but is untested


Temperature arithmetic is wrong

According to https://en.wikipedia.org/wiki/Celsius#Temperatures_and_intervals

What is often confusing about the Celsius measurement is that it follows an interval system but not a ratio system; that it follows a relative scale not an absolute scale. This is put simply by illustrating that while 10 °C and 20 °C have the same interval difference as 20 °C and 30 °C the temperature 20 °C is not twice the air heat energy as 10 °C. As this example shows, degrees Celsius is a useful interval measurement but does not possess the characteristics of ratio measures like weight or distance.

and further more at https://de.wikipedia.org/wiki/Grad_Celsius#Temperaturdifferenz (german)

delta t = 2°C - 1°C  is equivalent to delta T = 2K - 1K
delta t = 1K (= 1°C) is equivalent to delta T = 1K (not equal -272.15°C)

And that will fail here
var delta_t = 2.DegreesCelsius( ) - 1.DegreesCelsius( );
var delta_T = 2.Kelvins( ) - 1.Kelvins( );

System.Diagnostics.Debug.Assert( delta_t == delta_T );
System.Diagnostics.Debug.Assert( delta_t.Kelvins == delta_T.Kelvins );
System.Diagnostics.Debug.Assert( delta_t.DegreesCelsius == delta_T.DegreesCelsius );
System.Diagnostics.Debug.Assert( delta_t.Kelvins == 1 );
// next will fail because delta_t.DegreeCelsius is -272.15
System.Diagnostics.Debug.Assert( delta_t.DegreesCelsius == 1 );

I follow what you say. I think in this case, the arithmetic of subtracting one temperature from the other cannot be thought of as an interval, but rather a new temperature.
So in your example, subtracting 2K - 1K is not an interval of 1K, but a new temperature of 1K.
I don't think the implementation is wrong, at least the way I think about it, but rather what you expect when subtracting two temperatures.
To achieve this, I propose a new method TemperatureInterval GetInterval(Temperature other) and a new type TemperatureInterval having the same units, but a different semantics. You can then maybe add/subtract temperatures from temperature intervals. I think a .NET analogy would be TimeSpan vs DateTime. You can combine the two types and perform arithmetic between them, but they mean different things.

The example shows what happens if you subtract a measured temperature from another measured temperture: You get a delta and not a new temperature.
TemperatureDelta t = TemperatureMeasure.FromKelvin( 2 ) - TemperatureMeasure.FromKelvin( 1 );

The TemperatureDelta can be handled as any other dimension (adding delta, multiply by value) and can be added to a TemperatureMeasure.
// will evaluate to 2°C / 275.15K
TemperatureMeasure t = TemperatureMeasure.FromCelsius( 1 )
  + TemperatureDelta.FromCelsius( 1 );

// will evaluate to 3°C / 276.15K
TemperatureMeasure t = TemperatureMeasure.FromCelsius( 1 )
  + 2 * TemperatureDelta.FromCelsius( 1 );

with the current implementation I will get
// will evaluate to 275.15°C / 548.3K
Temperature t = 1.DegreeCelsius() + 1.DegreeCelsius();

I apologize, I finally got my head the right way around this.  You are absolutely correct.
In the analog of time, Temperature is DateTime and when subtracting two DateTimes you get TimeSpan, which would be our TemperatureDelta. You can add/subtract/multiply/divide a Temperature with a TemperatureDelta. You cannot do DateTime + DateTime, in the same way Temperature + Temperature don't make sense. I suppose the confusing part was that we talk about degrees Celsius as both a measurement and an interval/delta, while in time we have different units for this.
So back to the implementation proposal:

Remove current generated arithmetic for Temperature (breaking change)
Add new unit TemperatureDelta with same units as Temperature, ideally our code generator supported some inheritance or aliasing, so we could define TemperatureDelta to be the same as Temperature, except different naming.
Add custom arithmetic between Temperature and TemperatureDelta

Would you be interested in doing a pull request on this?

Temperature is used in speech and writing for both meanings Delta and Absolute/Measured and we had to read between the lines what is really mentioned here.
I am thinking about two new types named TemperatureDelta and TemperatureMeasured (or TemperatureAbsolute) to avoid conflicts with the current implementation of Temperature type.
Extensions can be done for both:
TemperatureDelta td = 35.DegreeCelsius();
TemperatureMeasured tm = 35.MeasuredDegreeCelsius();
// or
TemperatureAbsolute ta = 35.AbsoluteDegreeCelsius();

I am not quite sure what naming convention we should follow here and what would be the one most people will know at first sight what is going on and what does it mean.
Once we are both satisfied with the kids name I would like to offer a pull request to you :o)

I am leaning towards keeping Temperature, to mean a measured temperature. That was always what it was meant to be. The only confusing part is that it currently has arithmetic that does not make sense, such as adding two temperatures and that subtracting two temperatures returns a new temperature instead of a delta.
It would be a breaking change, but on the other hand, the behavior/semantics are simply wrong, so I would be fine doing this change without bumping the major version of the lib.
I am currently looking into extending the generator scripts to support a InheritsUnitsFrom: [ "Temperature" ] property, with array of names of units to inherit units from, so TemperatureDelta can copy the unit definitions of Temperature.
Then I will look into adding a property to omit the generated arithmetic, since that needs to be done manually in custom code.

Well thats fine for me.
How would the naming convention look like for the number extensions (which I like very much). Somethig like:
Temperature t = 34.DegreeCelsius();
TemperatureDelta td = 10.DelteDegreeCelsius();

Temperature newTemperature = 34.DegreeCelsius() - 10.DeltaDegreeCelsius();

Hm, I think the plural form DeltaDegreesCelsius() rings best in my ear? The current form is plural DegreesCelsius().

My german ears are more accurate on german language - so we best follow your ears in this case ;o)

Another possible option would be DegreesCelsiusDelta which would maybe useful with IntelliSense as a reminder. Typing DegreesCelsius will show you both options to mark it as a temperature or a delta.
Well typing Delta will show you all options related to delta temperatures.
Do you feel we need both or would be Delta... enough?

I was thinking the same thing, sure, DegreesCelsiusDelta is fine by me.
I would not add both. At least the ReSharper intellisense would give you all the hits when typing Delta, not sure about the Visual Studio intellisense.

I am making progress on the InheritsUnitsFrom property, got a working prototype here now. Will push later today.

Also a property for GenerateArithmetic: bool, which defaults to true.


mysql pooled connection error

this example route from the 3.0-rc documentation (https://docs.vapor.codes/3.0/mysql/core/#query) will not compile
    return req.withPooledConnection(to: .mysql) { conn in
        return try conn.query("select @@version as v;").map(to: String.self) { rows in
            return try rows[0].firstValue(forColumn: "v")?.decode(String.self) ?? "n/a"
        }
    }
}

xcode complains with the error Member 'mysql' in 'DatabaseIdentifier<_>' produces result of type 'DatabaseIdentifier<MySQLDatabase>', but context expects 'DatabaseIdentifier<_>'

I was having this issue too. It seems you have to be a little more specific to help out the compiler:
return req.withPooledConnection(to: .mysql) { (conn: MySQLDatabase.Connection) in     return conn.simpleQuery("select @@version as v").map(to: String.self) { rows in         return try rows[0].firstValue(forColumn: "v")?.decode(String.self) ?? "n/a"     } }
Once I did that, it worked as expected.

I'll make sure this gets fixed in the docs, thanks!


Rename project names to what they do :lipstick:

ProjectExample1.dpr: OFX simple text
ProjectExample2.dpr: OFX to DataSet

Fix


[TextServer] Add ICU Unicode security and spoofing detection.

Addition for #53956
Adds and exposed ICU spoof detection based on Unicode Technical Report #36 and Unicode Technical Standard #39. Includes extra 45 KiB of ICU data and additional 155 KiB of ICU source files.
Adds two functions to TextServer:

int is_confusable(String, Vector<String>)
Returns index of the first string which is visually confusable with provided string. e.g. Harvest and Ηarvest (second string starts with the Greek capital letter Eta).

bool spoof_check(String)
Checks if provided string is likely to be an attempt at confusing the reader. e.g. user (with a zero-width space added at the end) or usеr (Cyrillic Е in the middle).

Note: Currently, it's not integrated into the GDScript, but the idea for its usage is the following:

Check each string and identifier using spoof_check and add a warning.
Check each identifier against the list of all known identifiers, using is_confusable and add a warning if it's similar but not equal to some.

This looks ok, but my only concern is whether this can result in some sort of security risk? If users send text over the network is there any kind of measure that should be taken?

but my only concern is whether this can result in some sort of security risk? If users send text over the network is there any kind of measure that should be taken?

I'm not sure how it's related to networking, the purpose of these functions is to check if identifier strings are visually confusable (looks the same, but not the same text). ICU can do similar checks for URLs, but it's not exposed here.

Thanks!


polkadot: 0.8.28-1 -> 0.8.29

Motivation for this change
Version bump
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 115190 at 0e33ee4c run on aarch64-linux 1

1 package built:

polkadot

1 suggestion:

warning: unclear-gpl
gpl3 is a deprecated license, check if project uses gpl3Plus or gpl3Only and change meta.license accordingly.
Near pkgs/applications/blockchains/polkadot/default.nix:38:5:
   |
38 |     license = licenses.gpl3;
   |     ^

@r-rmcgibbo's suggestion has been fixed.

@r-rmcgibbo's suggestion has been fixed.

That is a bot and will probably not answer you.


add double-evil.tar and double-evil.zip inside testdata/testarchives/evilarchives

#247

When will it get accepted @coolaj86 ?

@kross9924 Will you update the PR to include the code used to create these as well?
I think putting them in testdata/create-evil-tar.go and testdata/create-evil-zip.go might be a good choice.


Add Neural Network project

Community Contribution
Apply Neural Network for detecting handwritten digits:
https://mortis-grimreaper.ddns.net/projects/HandwrittenDigit/
Sharing

[ ] I would be happy to have my project shared on The Coding Train's social media!

Instagram:
Twitter:

This is awesome!

This is awesome!


improve trace logging in packages autoroller

Add debugging to diagnose: https://github.com/flutter/flutter/issues/154151

Windows build_tests_1_7 failure was this: https://github.com/flutter/flutter/issues/154453

Reverting. The signal is strong that this broke the tree.


Bump version and update changelog

Version 1.16.0

Server: Add 'tag' parameter to list-message for go,js,python.
Server: backport expired message cleaner updates.
Libs/Javascript: Fix thrown error message when API status code is unknown.
Server: expired message cleaner cleanup.
Libs: fix descriptions in package repositories.
Add download badges to README.

Don't merge yet, I need to add a fix in the JS library


fix UnicodeDecodeError

There is a UnicodeDecodeError when I run the build script. I run the build script on my Windows system which system language is Chinese.
I think it's an encoding problem. Add encoding=utf-8 to avoid encoding issues.

Hey @Case3y, thanks so much for this helpful and very concise PR! I’ve tested it, and it seems to work without issue on my end. If it makes this work on your system, that is great! I’ve actually just added another th open(filepath) as file: statement to the project in (https://github.com/arrowtype/recursive-code-config/pull/25, so I’ll go specific utf-8 there, as well.


成功安装musicbox却不能运行？ImportError: No module named 'menu'不支持windows？

(py35) C:\Users\shu>pip install musicbox
Collecting musicbox
Downloading musicbox-<IP_ADDRESS>.tar.gz
Requirement already satisfied: requests in c:\anaconda3\envs\py35\lib\site-packages (from musicbox)
Requirement already satisfied: BeautifulSoup4 in c:\anaconda3\envs\py35\lib\site-packages (from musicbox)
Building wheels for collected packages: musicbox
Running setup.py bdist_wheel for musicbox ... done
Stored in directory: C:\Users\shu\AppData\Local\pip\Cache\wheels\0b\8e\69\5ab88b956db7a15795b348889e6acfd2ef3e6cc1e34c694653
Successfully built musicbox
Installing collected packages: musicbox
Successfully installed musicbox-<IP_ADDRESS>
(py35) C:\Users\shu>
(py35) C:\Users\shu>musicbox
Traceback (most recent call last):
File "c:\anaconda3\envs\py35\lib\runpy.py", line 184, in run_module_as_main
"main", mod_spec)
File "c:\anaconda3\envs\py35\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Anaconda3\envs\py35\Scripts\musicbox.exe_main.py", line 5, in
File "c:\anaconda3\envs\py35\lib\site-packages\src_init.py", line 8, in
from menu import Menu
ImportError: No module named 'menu'

pip install错了。
pip install NetEase-MusicBox
支不支持主要看mpg123能不能支持Windows
2017年4月23日(日) 16:33 dAxIOnG8<EMAIL_ADDRESS>
(py35) C:\Users\shu>pip install musicbox
Collecting musicbox
Downloading musicbox-<IP_ADDRESS>.tar.gz
Requirement already satisfied: requests in
c:\anaconda3\envs\py35\lib\site-packages (from musicbox)
Requirement already satisfied: BeautifulSoup4 in
c:\anaconda3\envs\py35\lib\site-packages (from musicbox)
Building wheels for collected packages: musicbox
Running setup.py bdist_wheel for musicbox ... done
Stored in directory:
C:\Users\shu\AppData\Local\pip\Cache\wheels\0b\8e\69\5ab88b956db7a15795b348889e6acfd2ef3e6cc1e34c694653
Successfully built musicbox
Installing collected packages: musicbox
Successfully installed musicbox-<IP_ADDRESS>
(py35) C:\Users\shu>
(py35) C:\Users\shu>musicbox
Traceback (most recent call last):
File "c:\anaconda3\envs\py35\lib\runpy.py", line 184, in
run_module_as_main "main", mod_spec) File
"c:\anaconda3\envs\py35\lib\runpy.py", line 85, in run_code exec(code,
run_globals) File "C:\Anaconda3\envs\py35\Scripts\musicbox.exe_main.py",
line 5, in File "c:\anaconda3\envs\py35\lib\site-packages\src_init.py",
line 8, in
from menu import Menu
ImportError: No module named 'menu'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/darknessomi/musicbox/issues/531, or mute the thread
https://github.com/notifications/unsubscribe-auth/AD_RSu0GDr8YV9AqUS5eCbbjUYJqTJZbks5rywzAgaJpZM4NFV74
.

Send me $1.00.
Sent from my iPhone

On Apr 23, 2017, at 9:49 PM, Weiliang Li<EMAIL_ADDRESS>wrote:
Closed #531.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Two dubbo calls, kryo serialization failed

Environment

Dubbo version: 2.6.2
Operating System version: Red Hat Enterprise Linux Server release 6.6 (Santiago)
Java version: 1.8

The two dubbo's SerializationOptimizer registered the bean separately, and the kryo serialization problem occurred. The problem is as follows:

hi, could you please provider a demo to reproduce this problem? Thanks a lot.

@diecui1202 ，The problem is that two dubbo's SerializationOptimizer registers the bean separately, and dubbo uses the map to store the bean class. The key is in numeric order and the value is the path class of the bean. Therefore, the bean cannot be found in the morning. The solution is in each case. The Service's SerializationOptimizer uses the same number of collections. I have tried it, or not. Is there any other solution?

would you mind to attach a simple project on github to reproduce so that we could understand this issue better? @llx206

@llx206, I did a little bit investigation today, and cannot reproduce it.
I tried to configure two optimizer like this:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dubbo="http://code.alibabatech.com/schema/dubbo"
       xmlns:context="http://www.springframework.org/schema/context"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://code.alibabatech.com/schema/dubbo http://code.alibabatech.com/schema/dubbo/dubbo.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
    <context:property-placeholder location="classpath:benchmark.properties" system-properties-mode="OVERRIDE"/>
    <dubbo:application name="dubbo-kyro-client"/>
    <dubbo:reference id="userService1" check="false"
                     interface="org.apache.dubbo.benchmark.service.UserService"
                     url="dubbo://${server.host}:${server.port}?optimizer=org.apache.dubbo.benchmark.serialize.SerializationOptimizerImpl&serialization=kryo"/>
    <dubbo:reference id="userService2" check="false"
                     interface="org.apache.dubbo.benchmark.service.UserService"
                     url="dubbo://${server.host}:${server.port}?optimizer=org.apache.dubbo.benchmark.serialize.SerializationOptimizerImpl&serialization=kryo"/>
    <dubbo:consumer client="netty4" filter="-default"/>
</beans>

I can successfully invoke the calls twice with no exception. I put the project at https://github.com/beiwei30/issue2116. Pls. take a look and submit pull request to modify if my test case is incorrect.

I could reproduce the issue. If the optimizer is not globally known and kept consistent to all participated parties, then the serialization exception gets happen, for example:
service-a (registered with optimizer 1) -> service-b (registered with optimizer 1 + 3) -> service-c (registered with optimizer 3)

In order to solve the issue, optimizer must be registered on service level, which in fact disobeys the current design. One particular serialization optimizer must be bound together with serialization mode onto a protocol definition, for example:
<dubbo:protocol name="dubbo" host="${server.host}" server="netty4" port="${server.port}" serialization="kryo" optimizer="foo.SerializationOptimizerImpl"/>

Because of this, I will simply close this issue and mark it as 'will not fix'. What I suggest to work around this particular situation is to provide the same optimizer implementation among your services, or not to use optimizer at all.


Release v7.0.22

🐛 Bug Fixes

fix: Edit button appear for the side of header (#9270) @yuki-takei
fix: Collaborative editing occurs unstable behavior (#9267) @yuki-takei

@mergify queue


How to connect flutter app with flask server because i want to excess rethinkdb

How to connect python-flask server with flutter app.Please help me hot connect server with flutter app.

https://flutter.io/cookbook/networking/fetch-data/

so in this way i will send and get data from flask server

yup

I assume this issue is obsolete.
Please don't hesitate to comment on the bug if you think it should be reopened.
Thanks for your contribution.


OCPBUGS-34137, OCPBUGS-34460, OCPBUGS-34113, OCPBUGS-35462: sync upstream on release-1.30/go1.22

Commands used:
git clone https://github.com/openshift/cloud-provider-openstack.git
cd cloud-provider-openstack
git remote add openshift https://github.com/openshift/cloud-provider-openstack.git
git remote add kubernetes https://github.com/kubernetes/cloud-provider-openstack.git
git fetch kubernetes release-1.30
git fetch openshift master
git checkout -b sync-master openshift/master
git rebase -i kubernetes/release-1.30
# Drop any CARRY: Vendoring commit
# Drop any conflicting CARRY commit related to upstream CI not required downstream

Then apply patches from:

https://github.com/openshift/cloud-provider-openstack/pull/283
https://github.com/openshift/cloud-provider-openstack/pull/282
https://github.com/openshift/cloud-provider-openstack/pull/281

Use k3s with containerd (#2529)
CI: Fetch DevStack and k3s logs (#2527)
Ingress: Fix listener timeout updates (#2518)
Add support to only run selected CSI services (#2316)
[ci] Bump golangci-lint for go 1.22 (#2544)
cinder-csi: Adds support for managing backups (#2473) (#2480)
fix: add missing labels to resources (#2519)
[octavia-ingress-controller] Add annotations to keep floating IP and/or specify an existing floating IP (#2166)
Pass KeyId to EncryptResponse (#2535)
Improve CCM docs in dual-stack context (#2563)
Update KMS API to v2 (#2561)
Remove enforcement of IPv6 LB as internal (#2557)
Convert outstanding commands to cobra (#2384)
Update the release procedure (#2565)
Allow changing cluster-name on existing deployments (#2552)
added route section to cloud-config secret (#2570)
[cinder-csi-plugin] define availability zone for snapshot backup (#2569)
Set --use-service-account-credentials=false (#2572)
Update Cinder and Manila CSI charts maintainers (#2583)
Bump K8s to v1.30.0 (#2581)
[occm] KEP-1860: Add support for LoadBalancer ipMode (#2587)
Bump versions for 1.30 (#2589)
[release-1.30] [occm] add a node selector support for loadbalancer services (#2603)
CARRY: Add DOWNSTREAM_OWNERS
CARRY: Add .snyk configuration file
CARRY: Add images
CARRY: More snyk ignores
Updating ose-openstack-cinder-csi-driver-container image to be consistent with ART for 4.17 Reconciling with https://github.com/openshift/ocp-build-data/tree/0a1310d4df1f00e3e218da6e356585997c7c0178/images/ose-openstack-cinder-csi-driver.yml
Remove dulek from shiftstack-team
UPSTREAM: : Updating ose-openstack-cloud-controller-manager-container image to be consistent with ART for 4.17
UPSTREAM: : Updating ose-openstack-cinder-csi-driver-container image to be consistent with ART for 4.17
UPSTREAM: : Updating csi-driver-manila-container image to be consistent with ART for 4.17

/jira refresh

/test e2e-openstack-csi-cinder e2e-openstack-csi-manila

/override ci/prow/e2e-openstack
The only failure is fixed via https://github.com/openshift/openstack-test/pull/175

/test e2e-openstack-csi-manila
timeout :/

/test e2e-openstack-csi-manila

/override ci/prow/e2e-openstack-csi-cinder
The results indicates a failure on a LOW impact test that fails all the time (we should fix or skip it).
https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cloud-provider-openstack/287/pull-ci-openshift-cloud-provider-openstack-master-e2e-openstack-csi-cinder/1802697896110329856

/override ci/prow/e2e-openstack-csi-manila
The error is low


[Merge once lowest k8s is 1.17] Remove conversion of webhook calls

What this PR does / why we need it:
Removed the admission and conversion webhook call conversion which we added in v1.0 as we now have v1 in all versions
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Part of #3413
Special notes for your reviewer:
So i have no idea why i called the branch that...
Release note:

Use admissionReview and conversionReview version v1

/test pull-cert-manager-e2e-v1-16
/test pull-cert-manager-e2e-v1-17
/test pull-cert-manager-e2e-v1-18

/kind cleanup

/test pull-cert-manager-e2e-v1-16
/test pull-cert-manager-e2e-v1-17
/test pull-cert-manager-e2e-v1-18

/hold

As per https://github.com/kubernetes/kubernetes/issues/82023
only allowing v1 to be sent was only allowed in Kubernetes 1.17

Can we just allow v1beta1 only to be used, remove the conversion logic and get this merged? :)

@munnerz we can but that causes the v1 upgrade issue to happen... again

What's the v1 upgrade issue? 🤔

Webhook pod being replaced in upgrade that when you now send a v1 request to it it will not work

I'm not sure I follow - if we specify the admission review version to be v1beta1, the apiserver will only ever send v1beta1 AdmissionReview objects.
Webhooks typically shouldn't need any conversion code because they can explicitly state which AdmissionReview version they support - the only time conversion is needed is if you are supporting two k8s versions that do not share a common AdmissionReview supported version.
Kubernetes 1.20 still supports v1beta1 AdmissionReview, right? So v1beta1 is the version supported in all our supported kubernetes versions today

I totally follow there but if we now ship one that only does v1beta1 we get a race condition on upgrade where the v1 version is still enabled but the new webhook pod is already running denying the admission requests as not correct

Anyhow closing this PR for now
/close


Broken links in documentation page

Github.io and three different versions links are broken, it redirects to /todo

Hello @pradiplimbu,
I think this does not exist anymore.

This is referencing
https://www.stirlingtools.com/
https://www.stirlingpdf.com/
I never got round to updating the website though and been focusing on GitHub currently
I will eventually get to it..

Is the website also open source ? @Frooodle

https://github.com/Frooodle/Stirling-PDF-Docs

@hostile-advisor you are welcome to change update or add however much you want
It's missing quite a few things here and there and I never got round to even making the homepage look that good 😂

@Frooodle Thanks mate!

Can be closed after merge of https://github.com/Frooodle/Stirling-PDF-Docs/pull/2


Fix or silence lint errors

yarn lint in the root directory shows 967 problems 😱

:tada: This issue has been resolved in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Allow pub server to run a secure connection / pub serve fails if HTTP Strict Transport Security (HSTS) is enabled

E.g. with python you can specify a certfile and a keyfile then it runs a secure connection
    import http.server
    import socketserver
    import ssl

    PORT = 7000

    # <IP_ADDRESS> allows connections from anywhere
    httpd = http.server.HTTPServer(('<IP_ADDRESS>', PORT), http.server.SimpleHTTPRequestHandler)
    httpd.socket = ssl.wrap_socket (httpd.socket, certfile='./jetty.crt', keyfile='./jetty.key', server_side=True)

    print("serving at port", PORT)
    httpd.serve_forever()

We need something like this for pub (and also for tests via content-shell or dartium)

If HTTP Strict Transport Security is enabled on the server side the server tells the client to switch to https - if pub server cannot handle this request it brings a 404

It's quite annoying! I have to turn off https on the server side because pub is not able to serve a secure connection.

This issue was moved to dart-lang/pub#1484