Classpasth Conflict Problem with Spring Boot 3.3.2 and Flyway 10.17.1 for MongoDB 7.0.12

Which version and edition of Flyway are you using?
flyway-database-mongodb 10.17.1
mongo-jdbc-driver 1.19.0
If this is not the latest version, can you reproduce the issue with the latest one as well? (Many bugs are fixed in newer releases and upgrading will often resolve the issue)
This is the latest version
Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)
Java API (Maven, Spring Boot)
Which database are you using? (Type & version)
MongoDB 7.0.12
Which operating system are you using?
MacOS
What did you do? (Please include the content causing the issue, any relevant configuration settings, the SQL statement(s) that failed (if any), and the command you ran)
pom.xml

4.0.0

org.springframework.boot
spring-boot-starter-parent
3.3.2

com.testmongoflyway
migrationpoc
0.0.1-SNAPSHOT
migrationpoc
migrationpoc

<spring.boot.version>3.3.2</spring.boot.version>

org.springframework.boot
spring-boot-starter
${spring.boot.version}

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
  <version>${spring.boot.version}</version>
</dependency>

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-jdbc</artifactId>
  <version>${spring.boot.version}</version>
</dependency>

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-test</artifactId>
  <version>${spring.boot.version}</version>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>

  <groupId>org.flywaydb</groupId>
  <artifactId>flyway-database-mongodb</artifactId>
  <version>10.17.1</version>
  <exclusions>
    <exclusion>
      <groupId>com.github.kornilova203</groupId>
      <artifactId>mongo-jdbc-driver</artifactId>
    </exclusion>
  </exclusions>
</dependency>
  <dependency>
    <groupId>com.github.kornilova203</groupId>
    <artifactId>mongo-jdbc-driver</artifactId>
    <version>1.19</version>
    <scope>system</scope>
    <systemPath>${project.basedir}/lib/mongo-jdbc-standalone-1.19.jar</systemPath>
  </dependency>
</dependencies>

org.springframework.boot
spring-boot-maven-plugin

org.flywaydb
flyway-maven-plugin

jdbc:mongodb://localhost:27017/catalog
.js

V1__schema.js
db.chai.insert({type: "Adrak", cost: 10});
Application.yaml
`spring:
application:
name: migrationpoc
datasource:
url: jdbc:mongodb://localhost:27017/catalog
driver-class-name: com.dbschema.MongoJdbcDriver
flyway:
url: jdbc:mongodb://localhost:27017/catalog
driver-class-name: com.dbschema.MongoJdbcDriver
environment: "mongodb"
fail-on-missing-locations: true
locations: classpath:db/migration/
sqlMigrationSuffixes:
- ".js"
enabled: true
server:
port: 8089
`
What did you expect to see?
Script run successfully.
What did you see instead?
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _ | \ \ \
\/  )| |)| | | | | || (| |  ) ) ) )
'  || .__|| ||| |_, | / / / /
=========||==============|/=///_/
:: Spring Boot ::                (v3.3.2)
2024-08-14T11:08:24.583-04:00  INFO 61637 --- [migrationpoc] [           main] c.s.m.MigrationpocApplication            : Starting MigrationpocApplication using Java 17.0.12 with PID 61637 (/Users/dbisht/Git/pdh/migrationpoc/target/classes started by dbisht in /Users/dbisht/Git/pdh/migrationpoc)
2024-08-14T11:08:24.585-04:00  INFO 61637 --- [migrationpoc] [           main] c.s.m.MigrationpocApplication            : No active profile set, falling back to 1 default profile: "default"
2024-08-14T11:08:24.937-04:00  INFO 61637 --- [migrationpoc] [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8089 (http)
2024-08-14T11:08:24.941-04:00  INFO 61637 --- [migrationpoc] [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2024-08-14T11:08:24.941-04:00  INFO 61637 --- [migrationpoc] [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.26]
2024-08-14T11:08:24.956-04:00  INFO 61637 --- [migrationpoc] [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2024-08-14T11:08:24.957-04:00  INFO 61637 --- [migrationpoc] [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 342 ms
[To redirect Truffle log output to a file use one of the following options:

'--log.file=' if the option is passed using a guest language launcher.
'-Dpolyglot.log.file=' if the option is passed using the host Java launcher.
Configure logging using the polyglot embedding API.]
[engine] WARNING: The polyglot context is using an implementation that does not support runtime compilation.
The guest application code will therefore be executed in interpreted mode only.
Execution only in interpreted mode will strongly impact the guest application performance.
For more information on using GraalVM see https://www.graalvm.org/java/quickstart/.
To disable this warning the '--engine.WarnInterpreterOnly=false' option or use the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property.
2024-08-14T11:08:25.236-04:00  WARN 61637 --- [migrationpoc] [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: null
2024-08-14T11:08:25.237-04:00  INFO 61637 --- [migrationpoc] [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2024-08-14T11:08:25.240-04:00  WARN 61637 --- [migrationpoc] [           main] o.a.c.loader.WebappClassLoaderBase       : The web application [ROOT] appears to have started a thread named [MongoShell ExecutorService] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
app//com.oracle.truffle.api.library.LibraryFactory.dispatch(LibraryFactory.java:539)
app//com.oracle.truffle.api.library.LibraryFactory.getUncached(LibraryFactory.java:396)
app//com.oracle.truffle.api.library.DynamicDispatchLibraryGen$UncachedDispatch.dispatch(DynamicDispatchLibraryGen.java:300)
app//com.oracle.truffle.api.library.LibraryFactory.dispatch(LibraryFactory.java:541)
app//com.oracle.truffle.api.library.LibraryFactory.getUncached(LibraryFactory.java:396)
app//com.oracle.truffle.polyglot.PolyglotValueDispatch$PrimitiveValue.(PolyglotValueDispatch.java:1666)
app//com.oracle.truffle.polyglot.PolyglotValueDispatch.addDefaultValue(PolyglotValueDispatch.java:1652)
app//com.oracle.truffle.polyglot.PolyglotValueDispatch.createDefaultValues(PolyglotValueDispatch.java:1647)
app//com.oracle.truffle.polyglot.PolyglotLanguageInstance.(PolyglotLanguageInstance.java:105)
app//com.oracle.truffle.polyglot.PolyglotLanguage.createInstance(PolyglotLanguage.java:171)
app//com.oracle.truffle.polyglot.PolyglotSharingLayer.allocateHostLanguage(PolyglotSharingLayer.java:334)
app//com.oracle.truffle.polyglot.PolyglotLanguageContext.ensureCreated(PolyglotLanguageContext.java:574)
app//com.oracle.truffle.polyglot.PolyglotContextImpl.createContextArray(PolyglotContextImpl.java:679)
app//com.oracle.truffle.polyglot.PolyglotContextImpl.(PolyglotContextImpl.java:516)
app//com.oracle.truffle.polyglot.PolyglotEngineImpl.createContext(PolyglotEngineImpl.java:1750)
app//com.oracle.truffle.polyglot.PolyglotEngineDispatch.createContext(PolyglotEngineDispatch.java:162)
app//org.graalvm.polyglot.Context$Builder.build(Context.java:1861)
app//com.mongodb.mongosh.MongoShellContext.(MongoShellContext.kt:14)
app//com.mongodb.mongosh.MongoShell.(MongoShell.kt:10)
app//com.dbschema.mongo.mongosh.PrecalculatingShellHolder.lambda$submitNewShell$0(PrecalculatingShellHolder.java:31)
app//com.dbschema.mongo.mongosh.PrecalculatingShellHolder$$Lambda$788/0x000000a00148e140.call(Unknown Source)
java.base@17.0.12/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base@17.0.12/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
java.base@17.0.12/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
java.base@17.0.12/java.lang.Thread.run(Thread.java:840)
2024-08-14T11:08:25.245-04:00  INFO 61637 --- [migrationpoc] [           main] .s.b.a.l.ConditionEvaluationReportLogger :

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-08-14T11:08:25.251-04:00 ERROR 61637 --- [migrationpoc] [           main] o.s.b.d.LoggingFailureAnalysisReporter   :

APPLICATION FAILED TO START

Description:
An attempt was made to call a method that does not exist. The attempt was made from the following location:
com.dbschema.mongo.MongoClientWrapper.lambda$new$3(MongoClientWrapper.java:100)

The following method did not exist:
'com.mongodb.connection.SocketSettings$Builder com.mongodb.connection.SocketSettings$Builder.connectTimeout(int, java.util.concurrent.TimeUnit)'

The calling method's class, com.dbschema.mongo.MongoClientWrapper, was loaded from the following location:
jar:file:/Users/dbisht/Git/pdh/migrationpoc/lib/mongo-jdbc-standalone-1.19.jar!/com/dbschema/mongo/MongoClientWrapper.class

The called method's class, com.mongodb.connection.SocketSettings$Builder, is available from the following locations:
jar:file:/Users/dbisht/.m2/repository/org/mongodb/mongodb-driver-core/5.0.1/mongodb-driver-core-5.0.1.jar!/com/mongodb/connection/SocketSettings$Builder.class
jar:file:/Users/dbisht/Git/pdh/migrationpoc/lib/mongo-jdbc-standalone-1.19.jar!/com/mongodb/connection/SocketSettings$Builder.class

The called method's class hierarchy was loaded from the following locations:
com.mongodb.connection.SocketSettings.Builder: file:/Users/dbisht/.m2/repository/org/mongodb/mongodb-driver-core/5.0.1/mongodb-driver-core-5.0.1.jar

Action:
Correct the classpath of your application so that it contains compatible versions of the classes com.dbschema.mongo.MongoClientWrapper and com.mongodb.connection.SocketSettings$Builder`

I got flyway-database-mongodb to work in my Spring Boot app with the following setup:
Snippet from pom.xml
<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.1.12</version>
    <relativePath/>
</parent>

<groupId>com.example.demo</groupId>
<artifactId>demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>demo</description>

<properties>
    <flyway.version>10.21.0</flyway.version>
    <java.version>17</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-mongodb</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- Dependencies needed for Flyway migrations -->
    <dependency>
        <groupId>org.flywaydb</groupId>
        <artifactId>flyway-core</artifactId>
        <version>${flyway.version}</version>
    </dependency>
    <!-- This dependency is in preview, so future versions may include breaking changes.
        See https://documentation.red-gate.com/flyway/flyway-cli-and-api/supported-databases/mongodb -->
    <dependency>
        <groupId>org.flywaydb</groupId>
        <artifactId>flyway-database-mongodb</artifactId>
        <version>${flyway.version}</version>
        <exclusions>
            <!-- Exclude this so it can be manually included from local jar -->
            <exclusion>
                <groupId>com.github.kornilova203</groupId>
                <artifactId>mongo-jdbc-driver</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- We have to manually include this jar, since Flyway uses a JDBC driver, even though it's MongoDB. -->
    <dependency>
        <groupId>com.github.kornilova203</groupId>
        <artifactId>mongo-jdbc-driver</artifactId>
        <version>1.19</version>
        <scope>system</scope>
        <systemPath>${project.basedir}/lib/mongo-jdbc-standalone-1.19.jar</systemPath>
    </dependency>
    <!-- This is necessary because Flyway uses a JDBC driver, even though it's MongoDB.
        Without this, the Flyway initializer bean doesn't get created. -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-jdbc</artifactId>
    </dependency>
</dependencies>

Snippet from application.yml:
spring:
  application:
    name: demo
  data:
    mongodb:
      uri: mongodb://localhost:27017
      database: demo

  datasource:
    url: jdbc:mongodb://localhost:27017/demo
    driver-class-name: com.dbschema.MongoJdbcDriver
  flyway:
    url: ${spring.datasource.url}
    driver-class-name: ${spring.datasource.driver-class-name}
    baseline-on-migrate: true
    sql-migration-suffixes: .js

I was able to run it. It was due to dependency coming from flywaymongodb and starter jdbc had one common class with same method name but different return type.
Thank you for your response.


default rendering shows no dataframe contents

This used to work, but now when the result is a dataframe, the default rendering shows no content. widgets.display() still works as expected.

This is with formattedShaVersion-0.6.2-SNAPSHOT-097a54bb62a16e4810d63648a9384922eef68171

arf indeed, I saw that once, I think this is a javascript cache crap.
You can try to clear the outputs (in the menu cell)
save the page
then reload the page

Clearing the output appears to work, but it's non-obvious and inconvenient. Is there a fix possible, like clearing some data structure used by the widget?


[YUNIKORN-204] Basic controller logic

This is a WIP PR. There are a couple of missing thins to to: adding tests, fix the TODO's, but I am creating it because some feedback it would be very helpful regarding the main idea. (especially because there are some things what smells a bit hacky for me, but I had no better idea).
@wilfred-s, @yangwwei, @HuangTing-Yao can you please check it when you will have some time?

@yangwwei I have addressed your review comments. I still have to add the unit tests and try to set the ownerReferrence for the pods belonging to an application.
@wilfred-s  can you please take a look to this changes as well?

hi @kingamarton changes look good to me. But could you please check the travis job failure? Thanks.

@yangwwei, the actual failure is because I need to change the go.mod file to point to the latest interface and also to the latest core. For updating it we need to merge the core side changes first otherwise the shim and the core will point to different interface versions.

hi @kingamarton  can you at least fix the codecov/project error? Thanks

hi, @kingamarton thanks for getting this done. Now it looks good.
The latest job shows only the codecov/project is failing, but I don't think that is caused by your patch. We will need to improve the code coverage in the shim repo to get those places, such as application_events.go. We can do that in follow up tasks, I have created https://issues.apache.org/jira/browse/YUNIKORN-383 for this.


BARF Comparison in the Demo

Hi @Nemo1999 @chenhsuanlin @szymanowiczs @martinarroyo et al.,
May I ask what the diff between BARF in the demo & the original BARF? Cuz I found my BARF takes > 50K iters to obtain a reasonable rendering. Thx & rgds,

Hi, for BARF we need to run 200k for complete result.


Added test table teardown

Dropping tables at the end of the test facilitates repeated runs of the tests.

Thanks.
FWIW, I think we want to fix up the code so it's more like this: https://examples.javacodegeeks.com/core-java/junit/junit-setup-teardown-example/
and then we can @Before and @After.


Add flutter support please

This project is currently only targeting HTML based UI's.  I imagine someone else is working on flutter, likely Google themselves.


Update the kubecon date

This issue follows https://github.com/kubernetes/website/pull/24880 but for Japanese

/lgtm

/approve


support embedded arrays

string and number arrays currently aren't supported.
this would be particularly helpful for evaluated / referenced R.

This issue was moved to att/rcloud.dcplot#19


fix: Enable ISO action once archive.org login & write permission works

It's currently disabled because archive.org was cyber-attacked.

Login is working now, but write permission isn't enabled yet

Uploading ISO now works. Deleting the old ones doesn't at the moment, but I think it will be fixed soon, before 1st day in the month.


BelongsToMany does not support prefix

https://github.com/adonisjs/adonis-lucid/blob/02f5c977da4e736326fed2c32554546478c30a79/src/Lucid/Relations/BelongsToMany.js#L79
I configured the prefix in the database, but this code makes the spelling database name not use the prefix, it will report the database error.

After digging more into it, I realized this is just the behaviour with the pivot table. You can simply define your own custom table name, when defining the relationship.


Rewrite /jobs/{{id}} to Go

I'm submitting a ...
improvement request (usability, performance, tech debt, etc.)
Traffic Control components affected ...

Documentation (possibly)
Traffic Ops

Current behavior:
/jobs/{{id}} is handled by Perl
Expected / new behavior:
/jobs/{{id}} should be handled by Go.
Minimum API Version
1.1
Request Methods

[ ] GET

Minimal reproduction of the problem with instructions:
N/A

deprecate as the new GET job endpoint supports ?id=


How to forward backend api route to frontend?

Hi thanks alot for amazing free pynecone software!
How could I forward a route on the backend api to the frontend?
Following the example in the docs https://pynecone.io/docs/advanced-guide/api-routes gets a result at localhost:8000/items/23. Is there a way to forward this same result to localhost:3000/items/23?
Right now I can serve a jinja template and mount a gradio app on the fastapi backend object, and they work fine running alongside the main pynecone app, but they are only exposed on the backend localhost:8000 port. It would be really great to have a way to get them to the frontend.

For production use, I have the pynecone app behind nginx reverse proxy
    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection Upgrade;
    }
    location ~ ^/(event|items)/ {
	proxy_pass http://localhost:8000;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection Upgrade;
    }


drivers/amazonec2: adds flag to prevent mutating security groups

This adds a workaround for issues like #4093, #3810. The user can
set a boolean flag to prevent the driver from editing their security groups.
The default behavior is kept the same for compatibility reasons.
Signed-off-by: André Carvalho<EMAIL_ADDRESS>

LGTM


Unable to run the script lightrag_oracle_demo.py because there is an error inserting records in a table

I am using lightrag with oracle database running the script lightrag_oracle_demo with the dicken's book and I got the following error message: An error occurred: ORA-12899: value too large for column "LIGHTRAGUSER3"."LIGHTRAG_GRAPH_NODES"."SOURCE_CHUNK_ID" (actual: 468, maximum: 256)
Help: https://docs.oracle.com/error-help/db/ora-12899/
This is the trace:
Inserting entities:  93%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎          | 102/110 [00:45<00:01,  6.07entity/s]INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
Inserting entities:  96%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋     | 106/110 [00:46<00:00,  6.89entity/s]INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
ERROR:lightrag:Oracle database error: ORA-12899: value too large for column "LIGHTRAGUSER3"."LIGHTRAG_GRAPH_NODES"."SOURCE_CHUNK_ID" (actual: 468, maximum: 256)
Help: https://docs.oracle.com/error-help/db/ora-12899/
MERGE INTO LIGHTRAG_GRAPH_NODES a
USING DUAL
ON (a.workspace = :workspace and a.name=:name and a.source_chunk_id=:source_chunk_id)
WHEN NOT MATCHED THEN
INSERT(workspace,name,entity_type,description,source_chunk_id,content,content_vector)
values (:workspace,:name,:entity_type,:description,:source_chunk_id,:content,:content_vector)
{'workspace': 'company', 'name': '"SCROOGE"', 'entity_type': '"PERSON"', 'description': '"Ebenezer Scrooge is the main character who undergoes a transformation from being miserly to generous, especially towards Bob Cratchit and his family.""Scrooge is a character who appears in the story alongside the Phantom, and his presence is acknowledged by the old man in the shop.""Scrooge is a character who is accompanied by the Spirit in observing the Christmas festivities.""Scrooge is a character who is visited by the Spirit and is led on a journey to change his ways.""Scrooge is a character who is visited by the Spirit and taken on a journey to various locations, where he witnesses people celebrating Christmas.""Scrooge is the main character of the story, a bitter old man who is visited by a Spirit and forced to confront his past.""Scrooge is the main character, a man of the worldly mind, who is visited by the Ghost and is forced to confront his own life choices.""Scrooge is the main character, a miserly and bitter old man who despises Christmas and is dismissive of his nephew's cheerfulness.""Scrooge is the main character, who is being shown various scenes by the Spirit to make him understand the importance of kindness and compassion.""Scrooge is the main character, who is initially portrayed as a grumpy and strict boss, but later shows a softer side.""Scrooge is the protagonist who is observing the business men and their reactions to the news of the death."', 'source_chunk_id': 'chunk-aa72b3229693bfae1ce0b0bdd0938a3dchunk-f2e4092f0c96452716187b590c0522a3chunk-6149859b58e3b7a8d8b3d13cf2bd062bchunk-643a9924a517b20b8ffa53501ae5f2c6chunk-696879398f43d6bcd6eafe44fc165187chunk-9de0190c942e4d6705c0feb2964ee52dchunk-8066fe69a8e40f713eb9e6b92dce9308chunk-f8190162edaa30ec159fba6987567d1cchunk-5b96d79538ade7a657f4574eb1d42bf5chunk-d86f26201cda83e54d77b0f1536b480fchunk-023e23590cd66795e8a95d96bd46b36e', 'content': '"SCROOGE""Ebenezer Scrooge is the main character who undergoes a transformation from being miserly to generous, especially towards Bob Cratchit and his family.""Scrooge is a character who appears in the story alongside the Phantom, and his presence is acknowledged by the old man in the shop.""Scrooge is a character who is accompanied by the Spirit in observing the Christmas festivities.""Scrooge is a character who is visited by the Spirit and is led on a journey to change his ways.""Scrooge is a character who is visited by the Spirit and taken on a journey to various locations, where he witnesses people celebrating Christmas.""Scrooge is the main character of the story, a bitter old man who is visited by a Spirit and forced to confront his past.""Scrooge is the main character, a man of the worldly mind, who is visited by the Ghost and is forced to confront his own life choices.""Scrooge is the main character, a miserly and bitter old man who despises Christmas and is dismissive of his nephew's cheerfulness.""Scrooge is the main character, who is being shown various scenes by the Spirit to make him understand the importance of kindness and compassion.""Scrooge is the main character, who is initially portrayed as a grumpy and strict boss, but later shows a softer side.""Scrooge is the protagonist who is observing the business men and their reactions to the news of the death."', 'content_vector': array([ 0.02348328,  0.0236969 ,  0.00622559, ...,  0.01704407,
-0.02807617, -0.00294685])}
Inserting entities:  96%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▋     | 106/110 [00:46<00:01,  2.28entity/s]
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:Node and edge data had been saved into oracle db already, so nothing to do here!
An error occurred: ORA-12899: value too large for column "LIGHTRAGUSER3"."LIGHTRAG_GRAPH_NODES"."SOURCE_CHUNK_ID" (actual: 468, maximum: 256)
Help: https://docs.oracle.com/error-help/db/ora-12899/

The SOURCE_CHUNK_ID column should have be longger in DDL. This was an oversight in programming. You can modify the table manually right now.

I modified the table setting the SOURCE_CHUNK_ID with 1024 characters and also I modified the table LIGHT_RAG_DOC_CHUNKS setting the SOURCE_CHUNK_ID with 1024 characters. But now I got the following message after running the example with the Dicken's book:
Inserting relationships: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 70/70 [00:30<00:00,  2.27relationship/s]
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:Node and edge data had been saved into oracle db already, so nothing to do here!
==================== naive ====================
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:Truncate 60 to 7 chunks
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
Unfortunately, you didn't provide a story for me to analyze. Please share the story, and I'll be happy to help identify the top themes.
==================== local ====================
An error occurred: AsyncCompletions.create() got an unexpected keyword argument 'keyword_extraction'

solve or not?

No. I think that the lightrag framework is excellent, but the lightrag team didn't test the oracle integration exhaustively, However I want to thank the development team for their great effort and commitment to generate this excellent tool.

I updated the github repository today and I ran the lightrag demo again, but I got the following messages:
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:full doc and chunk data had been saved into oracle db!
INFO:lightrag:Node and edge data had been saved into oracle db already, so nothing to do here!
==================== naive ====================
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:Truncate 60 to 7 chunks
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
Unfortunately, you didn't provide the story. Please share the story, and I'll be happy to help you identify the top themes!
==================== local ====================
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
INFO:lightrag:kw_prompt result:
{
"high_level_keywords": ["Story themes", "Narrative analysis", "Literary elements"],
"low_level_keywords": ["Plot structure", "Character development", "Symbolism", "Tone", "Setting"]
}
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:Local query uses 60 entites, 34 relations, 7 text units
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
This conversation has just started. You haven't shared a story with me yet. Please go ahead and share the story, and I'll be happy to help identify the top themes!
==================== global ====================
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
INFO:lightrag:kw_prompt result:
Unfortunately, I don't see a story provided. Please provide the story text so I can assist you in identifying the top themes.
Once you provide the story, I'll respond with the high-level and low-level keywords in JSON format, as instructed.
(If you provide the story, I'll respond in the correct format, for example:
{
"high_level_keywords": [...],
"low_level_keywords": [...]
})
JSON parsing error: Expecting value: line 2 column 27 (char 28) {
"high_level_keywords": [...],
"low_level_keywords": [...]
}
Sorry, I'm not able to provide an answer to that question.
==================== hybrid ====================
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
INFO:lightrag:kw_prompt result:
{
"high_level_keywords": ["Themes", "Story"],
"low_level_keywords": ["Plot", "Characters", "Setting", "Literary devices"]
}
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:Local query uses 60 entites, 34 relations, 7 text units
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/embeddings "HTTP/1.1 200 OK"
INFO:lightrag:Global query uses 26 entites, 60 relations, 7 text units
INFO:httpx:HTTP Request: POST http://localhost:8088/v1/chat/completions "HTTP/1.1 200 OK"
Unfortunately, I don't see a story here. Could you please paste the story, and I'll be more than happy to help identify the top themes for you?

It seems that prompt context is not generated.
Can you print the prompt, and see what is missig?


Grantee of a access right is not recorded

Thank you very much for pg-deps-management!
We set access rights to a admin role  "xxx":
create view phpunit.aa_view as ( select a,b from phpunit.a_view union select b,a from phpunit.b_view);
grant select, insert, update, delete on phpunit.a, phpunit.a_view, phpunit.b_view, phpunit.aa_view to xxx with grant option;
set session authorization xxx;
grant select on phpunit.a_view to "Just_for_fun";
set session authorization default;

the acls are set like this:
grantee           privileges  grantor
====================================
Just_for_fun      r           xxx
xxx               a*r*w*d*    postgres
...

After  deps_restore_dependencies() the rights are alle granted by postgres.

@ehansmair issue accepted, will fix in 3 days

DDL generation is incorrect in current algo.
This could take more time to fix.

Okay, it is only left to resolve grant order now


Add possibility to add review to a company.

 Add the possibility to add a review to a company.

@lgnstn updated.


Add mssql-cli to "Projects using prompt_toolkit"

mssql-cli is using prompt_toolkit.
Please add mssql-cli to "Projects using prompt_toolkit".

Added in this commit: https://github.com/jonathanslenders/python-prompt-toolkit/commit/6976e0c91d3f115d16aaf4430880bf905954a0c9
Thanks for sharing. Another amazing project!


tests: refactoring

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+51.0%) to 77.381%

Totals

Change from base Build<PHONE_NUMBER>0:
51.0%

Covered Lines:
130

Relevant Lines:
168

💛  - Coveralls


Landing page of "Adopt Me" button.

Description:
The landing page once you click the "Adopt Me" button under any particular animal is not properly styled.
Screenshot:

Please assign this issue to me as part of GSSoC '23.

Hey! I think I can design the landing page of the Adopt Me button. I would like if this issue is assigned to me.
Please assign this issue under GSSoC,23

some contributor is working on this


Fix MediaWiki.index()

The old code fetched non existing fields, resulting in an aborted iteration after the first "batch" of articles.
This was probably due to a change in the underlying MediaWiki API.
Reference: https://meta.wikimedia.org/w/api.php?action=help&modules=query%2Ballpages

Coverage increased (+0.004%) to 67.464% when pulling 8cf3454d2a67782cc45b9635f2403e8566d58d2f on maxmunzel:master into d25511f9ca7ed9356b801d8663b8b5168464e68f on clips:master.


libkbfs: avoid rekey loops on incomplete rekeys

If a reader handled a rekey request by setting the rekey bit, but
there was work it couldn't do on behalf of itself (like incrementing
the key generation due to a revoked device), it could enter into a
loop of continuous Put/SendRekeyNeeded messages back and forth with
the server.
Instead, detect if the rekeyer made any useful revisions, and if not,
don't bother putting a new revision.
Issue: KBFS-1119

Ok, I added a commit that tries to be more correct about when a reader rekey was actually done.  Please take another look.

👍


Nexus plugin is slow to load docker artifacts

Describe the bug
When loading the "Build Artifacts" tab for the first time (in each browser), with a nexus-repository-manager/docker.image-name: annotation, the tab takes around 20 seconds to load. This is because of the requests for the docker manifests, which take 6-7 seconds each, on our Nexus. This is past the point where people will click away because it is too slow.
To Reproduce
Steps to reproduce the behavior:

Initialize plugin
Add a nexus-repository-manager/docker.image-name: annotation to a catalog item
Click on the "Build Artifacts" tab (may need to open devtools and disable cache)
See time to load

Expected behavior
Loading to be near-instant. If possible, load times to be shared among users, rather than repeated per user. Thoughts on how to do this:

Add a backend to the plugin, with either a request-level or data-level cache. Refresh this on some schedule
Add config to only fetch the latest N artifacts of each type
Find a way to not request the full manifests, or at least not block on it -- we should be able to display a list of tags without waiting for the requests to complete. I think only artifact size requires the manifest?
Suggest adding Varnish or similar in-between backstage and nexus (we'd rather not do this, if possible)
Two or more of the above

Screenshots

Additional context
My company was planning on creating our own Nexus plugin, but delayed long enough that y'all shipped an OSS one. Since we've budgeted time for this already, we'd like to spend it working to contribute back to the work you're doing. I'm reporting this issue to see if there's something you already have planned for this/start a discussion, so we don't duplicate work you're already doing, or go down a different route than you envisioned.

Hey @rmartine-ias,
Thank you for filing an issue here! 🙂
If you have more ideas or if you would like to steer this plugin in a specific direction or design, please share it! We welcome it, we are open to any suggestions. 🙂
If you'd like, you can also join us on Slack and for our bi-weekly community calls.
Back to the issue:
What if we paginate on the searchServiceFetcher (we can use the continuationToken for pagination)? We can use this to lazy load additional entries and populate the rows on the fly. I don't think this would be too complex to implement and it would certainly be better than limiting the list of displayed entries. WDYT?

Hello @tumido and @schultzp2020! We definitely have some thoughts that we'll want to share -- I'll make a post in the slack, and add GH issues for discoverability as well. Will not be able to make it to the meetings (at least regularly), unfortunately; they're three hours before I'm typically awake.
Partially good news -- I figured out why our nexus was being slow. Requesting a manifest without certain accept headers makes it think only manifest schema 1 is supported, and on-the-fly rewriting to schema 1 is slow. (That's where I stopped looking. But I do wonder why it's slow, and why nexus doesn't cache those files.) I have a code change in the works for the nexus plugin to send those headers when requesting docker manifests. That significantly speeds up requests, but for resources with hundreds of images, it still takes a long time, so I don't want to close this issue.
I like the idea of paginating, but I think it can be one step faster, maybe. I don't know exactly the right words for things, but this is what I'm picturing: each page of search dumps the manifest download URLs into a queue. That queue is fetched from in groups of <pagination amount> (see image) (at least for the first request), and so we can draw the table as soon as the first <pagination amount> have returned, and keep fetching in the background. This would be faster than waiting for the data from a full page of search.

This could even be snappier by displaying the data from the search results immediately (version, repo, name, sha, modified) and async loading the size field as above.
A few other thoughts:

I think default sort should be date modified, descending (which needs a little bug fix, clicking the table's modified field seems to sort alphabetical by human-readable date -- will open an issue). This needs a sort=version parameter added to the nexus search API call. This would make the "first five results" much more likely to be meaningful. I can make this change if you all agree.

Something we're looking to add is the ability to have multiple artifact types per Component (maybe an OpenAPI spec, two maven artifacts, and a docker image). It would be ideal to handle this so that on first paint, the latest of all types are interleaved, limited to <pagination amount> in total. (Or maybe it would be better to add them to the table as they come?) This might be a "cross the bridge when we come to it" type thing, but I wanted to bring it up in case it would influence the design.
If we want to be able to let users sort/filter the table, lazy loading the rest of the results on page navigation introduces a lot of latency at that point. If everything is fetched in the background though, that's potentially a lot of unneeded API calls if the user never navigates past the first page, sorts, or filters. I'd lean towards the second, I think, for a better user experience.

@tplass-ias, anything I'm missing?

From what I've heard from our users, they are mostly interested in quickly checking that their latest version has arrived in nexus, so limiting how many entries are displayed is actually preferred imo
For browsing through the full list of versions of an artifact, I think that's what the Nexus UI itself is for, and ideally the Backstage UI would have a convenient link to click for that, e.g. ${NEXUS_URL}/#browse/search/docker=attributes.docker.imageName%3D${IMAGE_NAME} for docker images

For the quickly checking the latest version, I was thinking the main way they'd do that would be a home-tab card with the latest asset of each type. Saves a click over going to the dedicated tab.
Not to derail, but I'm still sort of unclear on the more general philosophical point of what Backstage plugins should do, and what they should delegate to the underlying tool via click-through. Do you have a heuristic you're using?


[SPARK-8121] [SQL] Fixes InsertIntoHadoopFsRelation job initialization for Hadoop 1.x

For Hadoop 1.x, TaskAttemptContext constructor clones the Configuration argument, thus configurations done in HadoopFsRelation.prepareForWriteJob() are not populated to driver side TaskAttemptContext (executor side configurations are properly populated). Currently this should only affect Parquet output committer class configuration.

cc @yhuai

Build triggered.

Build started.

Test build #34264 has started for   PR 6669 at commit c46b044.

Merged build triggered.

Merged build started.

Merged build triggered.

Merged build started.

Test build #34267 has started for   PR 6669 at commit 910b8aa.

Merged build finished. Test FAILed.

Test build #34264 has finished for   PR 6669 at commit c46b044.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test FAILed.

Test build #34267 has finished for   PR 6669 at commit 910b8aa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Test build #34344 has started for   PR 6669 at commit e1ede07.

Test build #34344 has finished for   PR 6669 at commit e1ede07.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #34355 has started for   PR 6669 at commit 1340f25.

Test build #34355 has finished for   PR 6669 at commit 1340f25.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Test build #34390 has started for   PR 6669 at commit b6f78a6.

Test build #34390 has finished for   PR 6669 at commit b6f78a6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Should we log the output committer used by parquet?

Merged build triggered.

Merged build started.

Test build #34407 has started for   PR 6669 at commit fce089c.

Test build #34407 has finished for   PR 6669 at commit fce089c.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
      logInfo(s"Using user defined output committer class $
      logInfo(s"Using output committer class $

Merged build finished. Test PASSed.

Merged build triggered.

Merged build started.

Test build #34431 has started for   PR 6669 at commit 73819e8.

Test build #34431 has finished for   PR 6669 at commit 73819e8.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
      logInfo(s"Using user defined output committer class $
      logInfo(s"Using output committer class $

Merged build finished. Test PASSed.

LGTM. I am merging it to master.


False negative with Lint/SymbolConversion in combination with string interpolation

With code like this I expect Lint/SymbolConversion to trigger an offence. Interpolation isn't currently considered.
foo = "bar"
"#{foo}_baz".to_sym
=> :bar_baz

Autocorrect would be the following which is equivalent:
:"#{foo}_baz"

RuboCop version
1.55.1 (using Parser <IP_ADDRESS>, rubocop-ast 1.29.0, running on ruby 3.2.2) [x86_64-linux-musl]
  - rubocop-erb 0.3.0
  - rubocop-rails 2.20.2

以下の2つのlintに引っかかっている

Style/RedundantParentheses

https://github.com/rubocop/rubocop/issues/12311

Lint/SymbolConversion

https://github.com/rubocop/rubocop/issues/12373


getStandardDeviationLast(uint16_t count)

Would you consider implementing this?
I'm working on a project based on average and standard deviation at different buffer lengths. This would save me lots of RAM.
Thanks!

Thanks for the request,
A non optimized version should be not too difficult to implement. Need to find some time to give it a try.

A quick implementation, not tested.
Add this in your .cpp file
float RunningAverage::getStandardDeviationLast(uint16_t count) const
{
  uint16_t cnt = count;
  if (cnt > _count) cnt = _count;
  if (cnt <= 1) return NAN;

  float temp = 0;
  float average = getAverageLast();
  uint16_t idx = _index;
  for (uint16_t i = 0; i < cnt; i++)
  {
    if (idx == 0) idx = _size;
    idx--;
    temp += pow((_array[idx] - average), 2);
  }
  temp = sqrt(temp/(cnt - 1));
  return temp;
}

and a signature in your .h file
float getStandardDeviationLast(uint16_t count) ;

please give it a try if this works for you

@alvaro-oliver
Added the above function to the develop branch of for the 0.4.6 version.
Please verify if it meet your needs.

Added function to version 0.4.6
Thanks again for the idea!

Thanks a lot, Rob!.
I'm sorry for the delay, but I've just tested and it works great, just needed to add "const" to the .h file prototype to match your .cpp function declaration.
This is the output for my test sketch:
Added: 2.000
Added: 3.000
Added: 4.000
Added: 5.000
Added: 6.000
Added: 7.000
Added: 8.000
Added: 9.000
Added: 10.000

Total Average: 5.500
Total Stdev. (sample): 3.028

Last five Average: 8.000
Last five Stdev. (sample): 1.581
Ended...

No need to apologize, it is good to hear that you can confirm that it works as intended.
I will check your note on const asap.
Will not make a new release for that now.

@alvaro-oliver
Can you share you test code?


Error: workers.api.error.unknown

Getting below error while deploying partykit
npx partykit deploy
🎈 PartyKit v0.0.100
Preparing 2 assets for upload...
Uploading 2 assets...
Uploaded dist/client.js.map
Uploaded dist/client.js
Updating asset index...
Deploying...
file:///Users/user/Documents/GitHub/project-hub/packages/partykit/node_modules/partykit/dist/bin.mjs:91369
throw err;
^
Error: workers.api.error.unknown
at fetchResult (file:///Users/user/Documents/GitHub/project-hub/packages/partykit/node_modules/partykit/dist/bin.mjs:84791:11)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async deploy (file:///Users/user/Documents/GitHub/project-hub/packages/partykit/node_modules/partykit/dist/bin.mjs:90818:21)
at async _Command. (file:///Users/user/Documents/GitHub/project-hub/packages/partykit/node_modules/partykit/dist/bin.mjs:91458:3)
Node.js v20.15.1

Are you still getting this error?

Yes, still getting same error.
Here's the latest logs
🎈 PartyKit v0.0.100
---------------------
Deploying...
file:///home/ubuntu/party/node_modules/partykit/dist/bin.mjs:91369
    throw err;
    ^

Error: workers.api.error.unknown
    at fetchResult (file:///home/ubuntu/party/node_modules/partykit/dist/bin.mjs:84791:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async deploy (file:///home/ubuntu/party/node_modules/partykit/dist/bin.mjs:90818:21)
    at async _Command.<anonymous> (file:///home/ubuntu/party/node_modules/partykit/dist/bin.mjs:91458:3)

Node.js v20.17.0

I'm getting this error as well, on PartyKit v0.0.110, and only when i have
  "placement": { "mode": "smart" },
set

I don't have any placement key, but still getting same error,
Here's the files
package.json
{
  "name": "partykit",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "dev": "partykit dev --live",
    "deploy": "partykit deploy"
  },
  "dependencies": {
    "partysocket": "^1.0.1"
  },
  "devDependencies": {
    "partykit": "^0.0.100",
    "typescript": "^5.4.2"
  }
}

partykit.json
{
  "$schema": "https://www.partykit.io/schema.json",
  "name": "hub-party",
  "main": "src/server.ts",
  "compatibilityDate": "2024-07-08",
  "serve": {
    "path": "public",
    "build": "src/client.ts"
  }
}

src/server.ts
import type * as Party from "partykit/server";

interface User {
  userId: string;
  avatar: string;
  name: string;
  cursor?: { x: number; y: number };
}

interface ConnectionState {
  rooms: Set<string>;
}

export default class Server implements Party.Server {
  private onlineUsers = new Map<string, Map<string, User>>();

  constructor(readonly room: Party.Room) {}
  async onStart() {
    console.log("Server started");
  }
  async onConnect(connection: Party.Connection, ctx: Party.ConnectionContext) {
    console.log(`Client connected: ${connection.id}`);
    connection.setState({ rooms: new Set<string>() } as ConnectionState);
  }
  async onClose(connection: Party.Connection) {
    console.log(`Client disconnected: ${connection.id}`);
    this.removeUserFromAllRooms(connection);
  }
  async onMessage(message: string | ArrayBuffer, connection: Party.Connection) {
    if (typeof message !== "string") return;
    const parsedMessage = JSON.parse(message);
    switch (parsedMessage.event) {
      case "joinRoom":
        this.handleJoinRoom(parsedMessage.data, connection);
        break;
      case "leaveRoom":
        this.handleLeaveRoom(parsedMessage.data, connection);
        break;
      case "cursorPosition": // Handle cursor position updates
        this.handleCursorPosition(parsedMessage.data, connection);
        break;
      default:
        console.log("Unknown event");
    }
  }
  private handleJoinRoom(
    data: { room: string; userId: string; avatar: string; name: string },
    connection: Party.Connection
  ) {
    const { room, userId, avatar, name } = data;
    // Add the user to the room
    if (!this.onlineUsers.has(room)) {
      this.onlineUsers.set(room, new Map());
    }
    this.onlineUsers.get(room)!.set(connection.id, { userId, avatar, name });
    // Add the room to the connection state
    const connectionState = connection.state as ConnectionState;
    connectionState.rooms.add(room);
    connection.setState(connectionState);
    // Broadcast the updated online users to the room
    this.broadcastToRoom(room, {
      event: "onlineUsers",
      data: Array.from(this.onlineUsers.get(room)!.values()),
    });
  }
  private handleLeaveRoom(
    data: { room: string; userId: string },
    connection: Party.Connection
  ) {
    const { room } = data;
    // Remove the user from the room
    if (this.onlineUsers.has(room)) {
      this.onlineUsers.get(room)!.delete(connection.id);
      // Broadcast the updated online users to the room
      this.broadcastToRoom(room, {
        event: "onlineUsers",
        data: Array.from(this.onlineUsers.get(room)!.values()),
      });
    }
    // Remove the room from the connection state
    const connectionState = connection.state as ConnectionState;
    connectionState.rooms.delete(room);
    connection.setState(connectionState);
  }
  private removeUserFromAllRooms(connection: Party.Connection) {
    const connectionState = connection.state as ConnectionState;
    connectionState.rooms.forEach((room) => {
      if (this.onlineUsers.has(room)) {
        this.onlineUsers.get(room)!.delete(connection.id);
        this.broadcastToRoom(room, {
          event: "onlineUsers",
          data: Array.from(this.onlineUsers.get(room)!.values()),
        });
      }
    });
    connectionState.rooms.clear();
    connection.setState(connectionState);
  }
  private broadcastToRoom(room: string, message: any) {
    for (const conn of this.room.getConnections()) {
      conn.send(JSON.stringify(message));
    }
  }
  private handleCursorPosition(
    data: { room: string; userId: string; cursor: { x: number; y: number } },
    connection: Party.Connection
  ) {
    const { room, userId, cursor } = data;
    if (!this.onlineUsers.has(room)) return;
    const user = this.onlineUsers.get(room)!.get(connection.id);
    if (user) {
      user.cursor = cursor;
      this.broadcastToRoom(room, {
        event: "cursorPosition",
        data: { userId, cursor },
      });
    }
  }
}

Thanks for the repro and the reports folks, annoyingly I'm not able to reproduce this here, which is confusing me. I can follow up with the cloudflare team early next week.

actually hmm I'm able to reproduce it now for smart placement, digging in some more

Ok I figured out an edge case for the smart placement error and fixed it, @MaxBittker you should be unblocked now, no noeed to update, just try deploying again with the placement config.
@RamDurgaSai I'm still struggling to reproduce your error. Could you try updating your partykit version and trying? If possible can you make a standalone repro that has the error and that I can check out here locally?


Add an include path for cppcheck

Not adding Boost not deal.II include paths because checking time goes through
the roof when I do
NOTE: please wait for me to check on Jenkins before you merge

[Current coverage][cc-pull] is 91.69%

Merging [#178][cc-pull] into [master][cc-base-branch] will decrease coverage by 0.82%

@@             master       #178   diff @@
==========================================
  Files            70         67     -3
  Lines          4184       3900   -284
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           3871       3576   -295
- Misses          313        324    +11
  Partials          0          0

3 files (not in diff) in python/test were deleted. more
File ...rce/end_criterion.py (not in diff) was modified. more

Misses +11
Partials 0
Hits -11

[][cc-pull]

Powered by Codecov. Last updated by [413a6ab...77cfade][cc-compare]
[cc-base-branch]: https://codecov.io/gh/ORNL-CEES/Cap/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/ORNL-CEES/Cap/compare/413a6ab4a4fadba7c98eafca484cb0235d39e44e...77cfadeaccda8dfa2b02e70151917d64a345560c
[cc-pull]: https://codecov.io/gh/ORNL-CEES/Cap/pull/178?src=pr

Ready for merge


Add volleyball environment

A Human VS AI arcade style volleyball mini-game example trained with a single policy using sb3.
https://github.com/edbeeching/godot_rl_agents_examples/assets/61947090/713b28e1-cad3-431e-9c00-5661d9504c31

[!WARNING]
This env currently uses https://github.com/edbeeching/godot_rl_agents_plugin/pull/34 which is currently in testing. It's possible to decouple by adding overrides if needed, but the features from the PR allow for an easy way to set a specific AIController as human.

Updated the video to match the newest onnx included in the files.


Project 2: Components.

Project 2: Components.
In this PR, I have done :-

Delete all the boilerplate from CRA (text, images, styles).
Create a directory called components and Calculator.js file.
Create a React component that matches the calculator design.

Thank you @Bondok6 for your feedback and approval.


Can't get it to work

Hi I just recently discovered this mod and as of version 1.6.13 when I try to go to Pierre's to get diapers, it just wouldn't load and there's a bunch of error codes in the SMAPI, does anyone else have the same issue? I followed all the instructions but the diapers just won't show up at Pierre's store

Copy pasting this from my last response:
I'm not sure if you can get the mod to work with the latest version of Stardew, but I've got it working on version 1.6.8
ContentPatcher 2.0.6
Smapi 4.0.8
And PyTk is no longer required if you read the latest release
https://github.com/zippity21/Stardew_Valley_Regression_Mod/releases/tag/v1.3.0
It works pretty well, the only issue I'm having is that sometimes notifications don't show up properly, not a huge deal but can be a bit inconvenient sometimes
Hoping we get an update sometime soon though 😅


Make it easier to search for unresolved topics

To make it easier to search for unresolved topics, we should add "Unresolved topics" to the search typeahead.

Suggested for: "unresolved", "not resolved"
Inserts a -is:resolved pill

This might be a blocker for #24200, depending on the approach taken for that issue.

CZO thread

@zulipbot claim

Hey @Joelute, are you still working on this issue?

Hey @opmkumar, yes I am however I've been busy for the past week so I haven't had the chance to work on it. If you would like to take over, feel free to. Otherwise, I'll continue working on it in the following days.

Hey @alya, could I pick up this issue since there has been no activity over the past 2 months? Once this blocker is resolved, I can proceed to work on #24200.
Thanks!

Hey @Joelute , can I pick up this issue since there has been no activity over the past 2 months? Once this blocker is resolved, I can proceed to work on #24200.
Thanks!


[WIP] Add blank line to test ci

Signed-off-by: Dmitry Surnin<EMAIL_ADDRESS>

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Can one of the admins verify this patch?

ok to test

Can one of the admins verify this patch?

ok to test

retest this please

Can one of the admins verify this patch?

Can one of the admins verify this patch?

test this please

Close as it was for testing purposes


Python js-beautifier fails when script contains UTF-8, and output is piped

$ cat test.js
var s = "åäö";
$ js-beautify test.js
var s = "åäö";
$ js-beautify test.js | cat -
'ascii' codec can't encode characters in position 9-11: ordinal not in range(128)
$ js-beautify --version
1.6.4
$ python --version
Python 2.7.12

@tobbez - I had a similar issue crop up here:
https://github.com/beautify-web/js-beautify/commit/a0bc8c3ecb8b3abcba5d8be2c9a3412171a25ef3#diff-319aee52866e5899a11a7f49987984b4R2302
It would be great help in understanding this if you could do any or all of the following:

Update to js-beautify 1.6.8 and test again.
Downgrade to Python 2.7.10 and test again.
Upgrade to Python 3.x and test again.

$ ./venv27/bin/python --version
Python 2.7.12
$ ./venv27/bin/js-beautify --version
1.6.8
$ ./venv27/bin/js-beautify test.js | cat -
'ascii' codec can't encode characters in position 9-11: ordinal not in range(128)

$ ./venv35/bin/python --version
Python 3.5.2
$ ./venv35/bin/js-beautify --version
1.6.8
$ ./venv35/bin/js-beautify test.js | cat -
var s = "åäö";

$ ./py279/bin/python --version
Python 2.7.9
$ ./py279/bin/js-beautify --version
1.6.8
$ ./py279/bin/js-beautify test.js | cat -
'ascii' codec can't encode characters in position 9-11: ordinal not in range(128)

I don't have easy access to 2.7.10, but I don't think this is a regression in Python 2.7 anyway

@tobbez - Thanks for your fast response.

@tobbez - If you can provide any help fixing this, that would be great.

After some research, the best way is likely be to add this at the start of the cli program:
import sys
import codecs
import locale

# Wrap stdout when output from the program is redirected
if sys.stdout.encoding is None:
    try:
        sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout.detach())
    except:
        sys.stdout = codecs.getwriter(locale.getpreferredencoding())(sys.stdout)

In Python 2, we create a writer connected to sys.stdout that encodes to the preferred encoding, and assigns that to sys.stdout.
In Python 3, we need to additionally retrieve the underlaying raw stream (using detach), and connect the writer to that. This is because sys.stdout in Python 3 expects str, while the writer writes bytes.
The Python 3 approach is tried first, because detach was first introduced in Python 3.1, so a raised exception means the code is running on Python 2.
(The code above should obviously never be included in a library.)

If you include it inside a library, users of that library will have their stdout replaced behind their backs just by importing the library, which may lead to unintended side effects and/or errors that are hard to track down.

@tobbez - Oh, you mean putting that if block right at the top of the file.  Got it.
No, I'd make that the last step right before writing to stdout.  😄

Putting it right at the top of the file is fine as long as it's an application (and was actually intended for that).
Anyway, I didn't look too closely at the existing code before writing the above, but doing so makes it clear that the above code is much more general than is needed here. That solution is useful when writing to standard out in many places, but seeing as js-beautifier only writes output in a single place, the following should suffice:
diff --git a/python/jsbeautifier/__init__.py b/python/jsbeautifier/__init__.py
index 9eecb59..3697513 100644
--- a/python/jsbeautifier/__init__.py
+++ b/python/jsbeautifier/__init__.py
@@ -7,6 +7,7 @@ import re
 import string
 import errno
 import copy
+import locale
 from jsbeautifier.__version__ import __version__

 #
@@ -2316,7 +2317,16 @@ def main():
                 import msvcrt
                 msvcrt.setmode(sys.stdout.fileno(), os.O_BINARY)

-            sys.stdout.write(pretty)
+            if sys.stdout.encoding is None:
+                # sys.stdout.encoding will be None if stdout is redirected (at
+                # least on Python 2), leading to encoding errors when stdout is
+                # redirected and the beautified content contains unicode
+                try:
+                    sys.stdout.buffer.write(pretty.encode(locale.getpreferredencoding()))
+                except:
+                    sys.stdout.write(pretty.encode(locale.getpreferredencoding()))
+            else:
+                sys.stdout.write(pretty)
         else:
             if isFileDifferent(outfile, pretty):
                 mkdir_p(os.path.dirname(outfile))

You're about 90% of the way having written the PR at this point. 😄
All that's needed adding file with the contents you've shown to
js/test/resources
And then adding a couple tests test to
python/jsbeautifier/tests/shell-smoke-test.sh
and
js/test/shell-smoke-test.sh
It's cool if you don't have time to do the last bit - I'll get to it soon. Thanks again!

Hey @bitwiseman, did this get fixed by chance? I looked through recent changes here https://github.com/beautify-web/js-beautify/commits/master but didn't find any relevant ones.
I'm on:
EditorConfig (0.12.1)
jsbeautifier (1.6.12)
I tested it on Python 2.7.9 thru 2.7.13 on both brew and pkg's to try to get it to work, only to find that the bug seems to manifest always when redirecting the output, regardless of version.
It works as intended if I do not redirect the output (and the program outputs to stdout), but as soon as I pipe or redirect the output to anything, the program shatters (the output does not get correctly redirected) and emits to stderr a message such as:
'ascii' codec can't encode character u'\u....' in position 376942: ordinal not in range(128)
This makes using js-beautify an excellent adventure in copy/paste when beautifying larger files.
And just now reading @tobbez's above proposed change which I somehow failed to read before debugging it, that solution should fix it exactly.
Bump :)

the bug is still there:
js-beautifier -o test.b.js test.js  works
js-beautifier >test.b.js test.js  gives the error.


test-branch/execute-job/quoted-authorization/97c2fed6-9fd7-4240-9040-f3ecfa108c41

PR for testing c61d86337661d3976ca9dba68322e5051c4d4baf

/canonical/self-hosted-runners/run-workflows c61d86337661d3976ca9dba68322e5051c4d4baf


Update mode.md

Update mode docs to be in line with current module.exports function signature.
At least for me a 4.8.3 → 4.12.0 upgrade removed the second parameter, so I'm guessing this should be reflected in the docs?

In the example, default module export is a function, you can get rest of parameters in args object.
env is a string parameter, not an object.

How exactly are you calling your webpack configuration?

OK, so apparently it's a webpack-command issue / inconsistency, see https://github.com/webpack/webpack/issues/6460#issuecomment-397322772

As mentioned in the linked issue thread, having no env but only argv returned is apparently an explicit design choice in webpack-command / config-loader: https://github.com/webpack-contrib/config-loader#gotchas
So what should be in the official Webpack docs now that webpack-cli is to be deprecated this month?
cc @shellscape

We could change it once webpack-command is official, until then, it works the way it is now.

PR is not longer valid. Thanks for your contribution.


Create EightSumsUp.py

Description
function that gets every pair of numbers from an array that sums up to eight and returns it as an array of pairs
Issue No.- #31
Checklist

[ ] Yes, this task assigned to me.
[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] My Filename is in camelCase format
[x] I have added Problem statement, Problem link in file.

one line solution

Invalid Task, PR without Assign. Marking Invalid


Remove unnecessary leftover cURL calls

Don't need it!

whoops never noticed this. closing as its no longer valid


PDF controls control the wrong PDF

I have two PDFs open side-by-side in Pycharm. Clicking the controls of the first one works fine. But the controls of the second one control the first one instead.
For example, clicking "hide sidebar" in the second PDF closes the sidebar of the first one.

I have

I am also having this issue! more specifically, it seems that there is only one set of controls -- selected options are the same across instances, but only the first-opened instance respects the options. For the moment I am working around this by changing the default view in the plugin settings, then closing and reopening the documents.


Cannot read property 'newPage' of undefined

Hello,
I've been trying to get this to work but I'm unable to. I'm just following along the getting started tutorial and attempting to scrape my own page like so:
const scrapedin = require('scrapedin')

const options = {
    email: <email>,
    password: <pass>,
}

scrapedin(options)
    .then((profileScraper) => profileScraper('https://www.linkedin.com/in/karan-rajpal/'))
    .then((profile) => console.log(profile))
    .catch(e => console.log(e))

But I get the following error:
➜ node index.js
TypeError: Cannot read property 'newPage' of undefined
    at module.exports (/mnt/d/projects/email-automation-node/node_modules/scrapedin/src/openPage.js:14:30)
    at module.exports (/mnt/d/projects/email-automation-node/node_modules/scrapedin/src/login.js:6:22)
    at module.exports (/mnt/d/projects/email-automation-node/node_modules/scrapedin/src/scrapedin.js:21:13)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Any idea what's going wrong? Am I doing something wrong?

Hi,  I've just had this, then changed the login to use proxyauthentication and cookies and it's working again.

Hi,
Just tried it again, I'm getting the error if I try to login with username/password, but if I use cookies it works fine.

Hi @karanrajpal14  and @neverton1970 it was a bug on v1.0.13, I fixed that
Please try again using v1.0.14 and provide feedback if it's working now, thanks!

Hi, that's working for me, many thanks for the quick reply and fix :)

thanks for the feedback!

Yes the issue is fixed now. Thank you!


Relative path for redirection

Hello
I'm trying to configure my Client (Google2Client) with a relative callback URL because my application can be called with different URLs. I don't think this is possible for now. Am I right ? What can I do to bypass this problem ?

A concept of contextual callback already exists, but it is not used in most clients like the google one. I plan to replace this limited implementation by a new feature in version 1.9. I guess the best option is currently to subclass the google client and compute the callback url your way.

Closing. Will be addressed by: https://github.com/pac4j/pac4j/issues/198


fix broken MNIST link

Fixes broken tutorial link.
I believe this is the correct link, but reviewer please confirm.

/cc @shannonbradshaw

/approve


Custom assert messages

I think it would be helpful to be able to fail tests with custom assert messages. Sometimes value equality isn't descriptive enough.

I'm a bit lost on this. All assertions have an optional context string you can pass in.
asserts.is_true(value, "expected to be true because whatever")
asserts.is_equal(a, b, "a == b because the same factory method called")
Do you mean something slightly different? If you could give me a code use example it would be great.

I didn't realize this feature was was already available, more or less. I guess what's needed is more explicit mention of it in the documentation.
One nice addition might be to have a separate pass and fail message if you specify two additional strings or a list containing two strings as the context.

Alright.
I know the documentation isn't up to par so that's my main focus right now. Still going be a while though.


WIP: retain all reported EIA sector codes for harvesting

PR Overview

PR Checklist

[ ] Merge the most recent version of the branch you are merging into (probably dev).
[ ] All CI checks are passing. Run tests locally to debug failures
[ ] Make sure you've included good docstrings.
[ ] For major data coverage & analysis changes, run data validation tests
[ ] Include unit tests for new functions and classes.
[ ] Defensive data quality/sanity checks in analyses & data processing functions.
[ ] Update the release notes and reference reference the PR and related issues.
[ ] Do your own explanatory review of the PR to help the reviewer understand what's going on and identify issues preemptively.

The main problem I've had is the inconsistency of field naming. It appears that the sector IDs come in labeled eia_sector and go out labeled sector_id_eia. The latter is also what's used in pudl/metadata/fields.py, which looks like something I shouldn't mess with. I finally decided the best way to deal with this was to rename the columns when the data comes into the transform step, such as in the generation() function in pudl/transform/eia923.py. I also changed the field names in pudl/metadata/resources/eia923.py.  But the code in this commit then hits an SQL error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table generation_fuel_eia923 has no column named eia_sector
[SQL: INSERT INTO generation_fuel_eia923 (data_maturity, eia_sector, energy_source_code, fuel_type_code_aer, plant_id_eia, prime_mover_code, sector_name, fuel_consumed_for_electricity_mmbtu, fuel_consumed_for_electricity_units, fuel_consumed_mmbtu, fuel_consumed_units, fuel_mmbtu_per_unit, net_generation_mwh, fuel_type_code_pudl, report_date) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 266.0, 0.0, 266.0, 0.0, 0.0, 30.317, 'wind', '2020-01-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 272.0, 0.0, 272.0, 0.0, 0.0, 31.076, 'wind', '2020-02-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 273.0, 0.0, 273.0, 0.0, 0.0, 31.133, 'wind', '2020-03-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 272.0, 0.0, 272.0, 0.0, 0.0, 30.983, 'wind', '2020-04-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 258.0, 0.0, 258.0, 0.0, 0.0, 29.431, 'wind', '2020-05-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 274.0, 0.0, 274.0, 0.0, 0.0, 31.203, 'wind', '2020-06-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 202.0, 0.0, 202.0, 0.0, 0.0, 23.001, 'wind', '2020-07-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 204.0, 0.0, 204.0, 0.0, 0.0, 23.325, 'wind', '2020-08-01')  ... displaying 10 of 351128 total bound parameter sets ...  ('final', 1.0, 'NG', 'NG', 65768, 'IC', 'Electric Utility', 126.0, 122.0, 126.0, 122.0, 1.032, 9.293, 'gas', '2021-11-01'), ('final', 1.0, 'NG', 'NG', 65768, 'IC', 'Electric Utility', 160.0, 155.0, 160.0, 155.0, 1.032, 11.785, 'gas', '2021-12-01'))]
The backtrace shows this happening under dfs_to_sqlite().When I grep for eia_sector, I still see usage in some files like pudl/package_data/eia923/column_maps/generation_fuel.csv. I can work on tracking down where the particular reference to eia_sector that's causing the problem is coming from, but before I do that it would be good to know if this general approach sounds right, or if the eia_sector/sector_id_eia distinction should be bridged somewhere else.

@knordback Don't worry, we use Draft PRs to get feedback on stuff we know isn't working all the time! And won't merge stuff without all the CI passing. Sometimes folks use a WIP prefix on the PR title to indicate it's a work in progress.
I changed this PR to be  a Draft and gave it a more descriptive title.

A relatively quick way to test the basic ETL without blowing away your local PUDL databases is running just the following test from the main repo directory:
$ pytest test/integration/etl_test.py::test_pudl_engine

The devtools/eia-etl-debug.ipynb notebook is also helpful for just running one table or step at a time for debugging EIA ETL stuff.

The main problem I've had is the inconsistency of field naming. It appears that the sector IDs come in labeled eia_sector and go out labeled sector_id_eia. The latter is also what's used in pudl/metadata/fields.py, which looks like something I shouldn't mess with. I finally decided the best way to deal with this was to rename the columns when the data comes into the transform step, such as in the generation() function in pudl/transform/eia923.py. I also changed the field names in pudl/metadata/resources/eia923.py.  But the code in this commit then hits an SQL error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table generation_fuel_eia923 has no column named eia_sector
[SQL: INSERT INTO generation_fuel_eia923 (data_maturity, eia_sector, energy_source_code, fuel_type_code_aer, plant_id_eia, prime_mover_code, sector_name, fuel_consumed_for_electricity_mmbtu, fuel_consumed_for_electricity_units, fuel_consumed_mmbtu, fuel_consumed_units, fuel_mmbtu_per_unit, net_generation_mwh, fuel_type_code_pudl, report_date) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 266.0, 0.0, 266.0, 0.0, 0.0, 30.317, 'wind', '2020-01-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 272.0, 0.0, 272.0, 0.0, 0.0, 31.076, 'wind', '2020-02-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 273.0, 0.0, 273.0, 0.0, 0.0, 31.133, 'wind', '2020-03-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 272.0, 0.0, 272.0, 0.0, 0.0, 30.983, 'wind', '2020-04-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 258.0, 0.0, 258.0, 0.0, 0.0, 29.431, 'wind', '2020-05-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 274.0, 0.0, 274.0, 0.0, 0.0, 31.203, 'wind', '2020-06-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 202.0, 0.0, 202.0, 0.0, 0.0, 23.001, 'wind', '2020-07-01'), ('final', 2.0, 'WND', 'WND', 1, 'WT', 'NAICS-22 Non-Cogen', 204.0, 0.0, 204.0, 0.0, 0.0, 23.325, 'wind', '2020-08-01')  ... displaying 10 of 351128 total bound parameter sets ...  ('final', 1.0, 'NG', 'NG', 65768, 'IC', 'Electric Utility', 126.0, 122.0, 126.0, 122.0, 1.032, 9.293, 'gas', '2021-11-01'), ('final', 1.0, 'NG', 'NG', 65768, 'IC', 'Electric Utility', 160.0, 155.0, 160.0, 155.0, 1.032, 11.785, 'gas', '2021-12-01'))]
The backtrace shows this happening under dfs_to_sqlite().When I grep for eia_sector, I still see usage in some files like pudl/package_data/eia923/column_maps/generation_fuel.csv. I can work on tracking down where the particular reference to eia_sector that's causing the problem is coming from, but before I do that it would be good to know if this general approach sounds right, or if the eia_sector/sector_id_eia distinction should be bridged somewhere else.

Turns out it's easier to fix code if someone tells you exactly what to do.
I requested re-review. I hope that's the right thing. Please look at the top of the bug-2193 branch.

@knordback If you want to grab a pristene "before" version of the PUDL DB, you can download the most recent successful nightly build output from AWS Open Data (a 1GB DB file) (see the collection of links in the README for other outputs if you need them).
To check what's different between the before and after databases, I'd probably run the full ETL, without any of the FERC or EPA CEMS datasets, which will save a lot of time, and then in a Jupyter Notebook run something like...
import sqlalchemy as sa
import pandas as pd
import pudl

pudl_settings = pudl.workspace.setup.get_defaults()

pudl_engine_before = sa.create_engine("path/to/your/old/sqlite.db")
pudl_engine_after = sa.create_engine(pudl_settings["pudl_db"])

plants_before = pd.read_sql("plants_eia860", pudl_engine_before)
nosector_before = plants_before.set_index(["report_date", "plant_id_eia"])
nosector_before = nosector_before.loc[nosector_before.sector_id_eia.isna()]

plants_after = pd.read_sql("plants_eia860", pudl_engine_after)
nosector_after = plants_after.set_index(["report_date", "plant_id_eia"])
nosector_after = nosector_after.loc[nosector_after.sector_id_eia.isna()]

plants_filled_idx = nosector_before.index.difference(nosector_after.index)
plants_after.loc[plants_filled_idx]

Or simpler, just count the difference in null values:
import sqlalchemy as sa
import pandas as pd
import pudl

pudl_settings = pudl.workspace.setup.get_defaults()

pudl_engine_before = sa.create_engine("path/to/your/old/sqlite.db")
plants_before = pd.read_sql("plants_eia860", pudl_engine_before)
na_before = plants_before.sector_id_eia.isna().sum()

pudl_engine_after = sa.create_engine(pudl_settings["pudl_db"])
plants_after = pd.read_sql("plants_eia860", pudl_engine_after)
na_after = plants_after.sector_id_eia.isna().sum()

@knordback generally looks good!
import sqlalchemy as sa
import matplotlib.pyplot as plt

pudl_engine = sa.create_engine(pudl_settings["pudl_db"])
op = pd.read_sql("plants_eia860", pudl_engine)
np = out_dfs["plants_eia860"]

for plants, label in zip([op, np], ["old", "new"]):
    plants_gb = (
        plants
        .assign(has_sector_id=lambda x: x.sector_id_eia.notna())
        .loc[:,["report_date", "has_sector_id"]]
        .groupby("report_date")
    )
    df = plants_gb.sum() / plants_gb.count()
    plt.plot(df, label=label)
    # display(df)
plt.legend()
plt.show()

@knordback you should accept my invitation to be a Catalyst collaborator so the CI can run on your PRs.

Great. And I accepted the invitation.

@knordback I think now the CI is failing because this PR is coming from a fork.  Now that you're part of the Catalyst org you should be able to make a PR from a branch within the PUDL repo.


Fixed PS version in manifest

This was causing the module to not import in VSCode with PS Core as the shell.  I think PowerShellVersion was intended but PowerShellHostVersion was used by mistake.

Hey, great pick up. You are totally correct, we should only specify PowerShellVersion.


Enhance "JSON" documentations with new columns: "requirement level", "data type"

Whenever we describe JSON files in the BIDS specification, we usually do it using a FieldName and a Definition column (see e.g., dataset_description.json):

FieldName
Definition

Authors
OPTIONAL. List of individuals who contributed to the creation/curation of the dataset.

In terms of JSON, the FieldName is relatively uncontroversial. It must be a key in the JSON file, formatted like this:
{
  "Authors":
}

However, what's happening in Definition is a lot more to handle. I suggest to change the way we describe JSON files to the following:

FieldName
Req. Level
Datatype
Definition

Authors
OPTIONAL
array of string
List of individuals who contributed to the creation/curation of the dataset.

This would make it easier to parse the requirement level (optional, required, recommended)
This would make clear how to format the field ... solving and preventing issues like #516 --> also, bids-validator devs can more easily adopt the spec in terms of JS code

I propose this enhancement "as is" ... but for the future, we may also consider turning these tables into YML format, like @tsalo is currently doing with the entity table in https://github.com/bids-standard/bids-specification/pull/475 (BTW: please review that PR!)

I think specifying the required data type in the schema and documentation would eliminate the need for a field specifying type on a file-by-file basis in the jsons, right?

you mean in the JSON schemas that the bids-validator uses? https://github.com/bids-standard/bids-validator/tree/master/bids-validator/validators/json/schemas
and with "specifying in the documentation" I assume that you mean doing what I hint at in the original post of this thread?
If I understand you correctly, then yes I would agree that these two aspects are sufficient and we can close #221

@sappelhoff Yes, sorry for the poor wording. That's exactly what I mean, although bids-validator JSON schema will of course become bids-specification schema if and when we have that implemented. If your recommendation is accepted (as I hope it will be), we can also close #350.

Once this PR is done, we can also transfer the benefits over to the validator --> e.g., here: https://github.com/bids-standard/bids-validator/issues/851


Add symbol containing process ID

Reported by steven c hankin on 4 Jan 2011 18:56 UTC
In the main Ferret program there are several calls that set symbols at startup - the date and time time counters, Ferret's version, the version of the NetCDF library, so this would fit right in there.
It is a simple C function call - getpid() - so probably the best place for it.  But it is not ANSI C standard (but is POSIX standard).
Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/1779

Modified by ansley.b.manke on 5 Jan 2011 18:25 UTC

Comment by andrew.wittenberg on 7 Mar 2012 21:25 UTC
See also #1016 regarding the need for this.

Comment by ansley.b.manke on 7 Mar 2012 23:10 UTC
This is done in Ferret v6.73 and higher, using the gfortran getpid call.
> ferret
        NOAA/PMEL TMAP
        FERRET v6.73
        Linux 2.6.18-274.12.1.el5PAE 32-bit - 03/07/12
         7-Mar-12 15:09

yes? sh sym session_pid
SESSION_PID = "3488"


DMRG norm_tol convergence criterion

Jack pointed out to me that there is an inconsistency in the norm_tol parameter of DMRG:

In DMRGEngine._canonicalize, the default norm_tol is 1e-5, while the test for whether an MPS is canonical or not in Tenpy seems to be 1e-10 (e.g. in MPOTransferMatrix .init). This means that one cannot, for example, naively use the output of DMRGEngine as the initial state for a new ground_state_search simulation from file without risking a "not in canonical form" error. Of course, the problem is easily solved for a finite MPS -- one can just call .canonical_form after DMRG -- while for iMPS we can just set the norm_tol to 1e-10 appropriately and possibly also raise norm_tol_iter. I guess my question is, why are these values for the norm tolerance so different and which should I prefer for iDMRG simulations? It naively seems to me that setting the norm_tol to 1e-5 might lead to an MPS quite far from canonical form.

That's not at all a naive question, and it raises an issue in about the DMRG convergence criteria in TenPy....
The norm_tol is used during DMRG, where one expects that the state is not yet in canonical form until DMRG fully converges. Especially for infinite systems with long correlation lengths, this convergence is sometimes quite slow, so we allowed a very moderate value of 1.e-5 just to get DMRG to converge at all for these cases.
On the other hand, I've required a quite good precision for the MPOTransferMatrix because I found that any error of the MPS not being in canonical form propagates to wrong environments, which can screw up all of the following DMRG run - and one can simply circumvent this by calling psi.canonical_form() before initializing the MPOTransferMatrix.
Calling psi.canonical_form is

bad if you want to re-use the environments, because it renders the environments invalid - so we try to avoid it whenever one might need to reuse the environments
but good whenever you want to re-use psi in another way: expectation values, MPOTransferMatrix etc assume canonical form, and any error in the canonical form propagates to wrong expectation values.

Historically, we only added the MPOTransferMatrix with the ability to recover the environments from psi in version 0.9.0 last summer.
This allows us to be a bit more on the "safe" side, and explicitly call psi.canonical_form at the end of DMRG, and rather re-generate environments for the next DMRG run, if necessary. However, right now, TeNPy is still using the "old" mode trying to avoid rendering the environments invalid...
Given that we have the MPOTransferMatrix now, I think we should add "split" the norm_tol parameter up into one "norm_tol" to be acchieved for DMRG convergence, and one norm_tol_final to be used for the state returned by DMRG after convergence by calling psi.canonical_form, even if this renders environments invalid. The norm_tol_final could then be significantly more precise than the norm_tol for convergence...

Thanks for explaining this! Just to be clear, the idea is to run DMRG for norm_tol_iter steps to attempt to converge to norm_tol, as before, but then if the new parameter norm_tol_final has not been reached, simply call psi.canonical_form()?

Yes, I would keep the norm_tol_iter around at least for backwards compatibility. However, we might actually let it default to 0 now.
FYI: I've implemented a replacement psi.canonical_form_infinite2 in the last few days, which is now the new default for infinite MPS.


Loading data raises CUDA error

🐛 Bug
I have a GraphSAINTRandomWalkSampler data loader. When I put data on CPU, everything runs smoothly. When I put the tensors onto cuda, it raises the following bug:
Traceback (most recent call last):
  File "main.py", line 58, in <module>
    main()
  File "main.py", line 40, in main
    loss = training.train(model, data_loader, optimizer)
  File "training.py", line 16, in train
    for data in loader:
  File "lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 435, in __next__
    data = self._next_data()
  File "lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 475, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "lib/python3.8/site-packages/torch_geometric/data/graph_saint.py", line 93, in __getitem__
    node_idx = self.__sample_nodes__(self.__batch_size__).unique()
  File "lib/python3.8/site-packages/torch_geometric/data/graph_saint.py", line 214, in __sample_nodes__
    node_idx = self.adj.random_walk(start.flatten(), self.walk_length)
  File "lib/python3.8/site-packages/torch_sparse/rw.py", line 8, in random_walk
    return torch.ops.torch_sparse.random_walk(rowptr, col, start, walk_length)
RuntimeError: start.device().is_cuda() INTERNAL ASSERT FAILED at "/home/travis/build/rusty1s/pytorch_sparse/csrc/cuda/rw_cuda.cu":35, please report a bug to PyTorch. start must be CUDA tensor

To Reproduce
Steps to reproduce the behavior:
# data is a tensor on GPU
def get_minibatch_loader(data):
    sampler = GraphSAINTRandomWalkSampler(
        data,
        batch_size=TrainingConfig.training_batch_size,
        walk_length=TrainingConfig.walk_length,
        num_steps=TrainingConfig.num_steps,
        sample_coverage=0,
        save_dir=Config.dataset_dir
    )
    return sampler

loader = get_minibatch_loader(data)
for d in loader:
    # The above raises the bug

Environment

OS: Linux 18.04
Python version: 3.8.5
PyTorch version: 1.7.1
CUDA/cuDNN version: 9.0.176
GCC version: 7.5.0
Any other relevant information:

Additional context

Yes, GraphSAINT data loading routines are designed to work on CPU, while each mini-batch is then passed to the GPU for further processing. I added a check that input data to GraphSAINT needs to be stored on the CPU.

What about other samplers like RandomNodeSampler and RandomEdgeSampler?


Have to manually define attributes once avsc is created - need help

This library is recognising elements and attributes in the same manner. Need to be differentiated.
For example the XSD contains :
<Student Index="1">
        <Surname>Aaron</Surname>
 </Student>

And the corresponding avsc contents are :
{
    "name" : "Student",
    "type" : [ "null", {
      "type" : "record",
      "name" : "rspStudent",
      "fields" : [ {
        "name" : "Surname",
        "type" : [ "null", "string" ],
        "default" : null,
        "source" : "element Surname"
      }, {
        "name" : "Index",
        "type" : "string",
        "source" : "attribute Index"
      } ]

Now I am modifying all the attributes and inserting an "_" before it's name to differentiate from elements.
Any way to resolve this ?

just added the option
below option under XSD config, check ReadMe.md, also make sure to take latest code
attributePrefix: "_"          # Optional, will assign the specified prefix for attributes in the avsc schema

Thanks @roycem90 . Will be taking the new jar for any further processing of XML


Test suite should unaware of infra details

The Cilium test suite currently holds many references to infrastructure details, namely Vagrant. The test suite should not have any awareness of how infrastructure is built.

To begin with, it would be good to know if e.g. the following functions is used for anything:
https://github.com/cilium/cilium/blob/4bdb3617786dd7c3a6f2de78923f5981cdbfc8a6/test/helpers/vagrant.go#L35
Also, there are flags like -cilium.provision and -cilium.provision-k8s , are these used?

-cilium.provision is definitely used often when running locally (to speed up subsequent test runs once provisioned). Don't know for the second. Maybe asking on the #development channel would be best?

@pchaigno is it -cilium.provision=false that you are actually using?

@errordeveloper I use both, but I think -cilium.provision=true is actually the default so I shouldn't need to explicitly specify it.

I thought there was a bash script that handles provisioning. In any case, this is a more general issue to say that we need to draw clearer boundaries in that area. I think it makes sense to have a mode where test suite invocation involves some setup steps, but whatever it does shouldn't be tied into a particular implementation of infrastructure, right now it is directly tied into vagrant. I am intending to run a  survey soon to find out what setups people have and what is important for them.

According to the survey Ray did recently, most folks use ginkgo to run tests locally.


Some part in the sample code didn't work.

I copied exactly the same code from the example. but some of them didn't work and I don't know why. The following code didn't working (with error explanation)
input.KeyDelay = 1;
// 'Input' does not contain a definition for 'KeyDelay' and no accessible extension method 'KeyDelay' accepting a first argument of type 'Input' could be found (are you missing a using directive or an assembly reference?)

input.SendKeys(Keys.Enter, KeyState.Down);
//cannot convert from 'Interceptor.KeyState' to 'Interceptor.Keys'

input.SendKeys(Keys.Enter, KeyState.Up);
// cannot convert from 'Interceptor.KeyState' to 'Interceptor.Keys'

input.Unload();
// not support on this platform

Hi,
It's an open source library.  You can check the code.
They merely change the names of some variables.

        input.KeyPressDelay = 10;
        input.SendKey(Interceptor.Keys.Enter, KeyState.Down);
        Thread.Sleep(10);
        input.SendKey(Interceptor.Keys.Enter, KeyState.Up);

KeyDelay > KeyPressDelay
SendKeys(Key, State) >SendKey(Key, State)


[windows client] [cliprdr] Translate local to remote format IDs in cl->sv requests

Situation
Copying any rich text from the remote desktop and pasting it in the local desktop results in a sudden but graceful disconnection of the FreeRDP client on windows.
(however, pasting unformatted text works fine, and pasting rich text from the local desktop into the remote desktop works fine too! Perplexing, right?)
Background
Windows' clipboard API allows for the same data to be stored in multiple formats. It supports several builtin clipboard formats, but also allows for new clipboard formats to be registered by applications.
The FreeRDP windows client receives a list of names of clipboard formats and their respective remote clipboard format IDs whenever something is copied in the remote desktop. Subsequently, it registers those clipboard formats locally using RegisterClipboardFormatW. This function returns a local clipboard format ID.
Assessment
The local clipboard format ID is not guaranteed to be equal to the remote clipboard format ID (unless it's a standard clipboard format, see above).
While FreeRDP sends a valid-looking request to the server for a remote->local paste, the format ID included in that request corresponds to the local format ID. When this is received by the server, the server is perplexed and lets the client know that it has requested a clipboard format that is unsupported by the server. The client sees this message from the server and gracefully shuts down without any logs to indicate why this occurred.
Recommendations
This PR resolves the above by ensuring that we include a remote format ID (translating it in cliprdr_send_data_request via get_remote_format_id) when requesting clipboard data from the server.

@freerdp-bot test


custom connection delay added

https://github.com/thoov/mock-socket/issues/346

@bvedad please add this property to ServerOptions interface to declaration file


add docker healthcheck for Microsoft SQL Server on Linux

While this is neat, I don't think this repo is an appropriate place to put it -- this is exclusively for the official images.

I believe you are not supporting ms products - but that is ok ;)

But wait I'm working on official ms docker image - should I clean my dockerfile ?

The name is a bit unfortunate, but I'm referring specifically to images from Docker's "Official Images" program (https://github.com/docker-library/official-images and https://hub.docker.com/explore/).  There are some Windows-based image tags there, but the images under microsoft/... (or any other user/organization, for that matter) are not included.


Adding extra options in pickadate.js

How do we specify extra options available in pickadate.js. Options like selectMonths, selectYears etc.
Thanks.

It's not in the bower version right now, it's been merged but testing didn't seem conclusive:
https://github.com/Textalk/angular-schema-form-datepicker/pull/16

Okay, great.

The current bower version solved my issue. Thanks.


⚠ Bofrost.be has degraded performance

In 3d22012, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1200 ms

Resolved: Bofrost.be performance has improved in e22defa.


CLI wrapper to create Unix daemons

Closes #15

Codecov Report

Merging #16 into master will increase coverage by 0.62%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
+ Coverage   96.60%   97.21%   +0.62%
==========================================
  Files           3        4       +1
  Lines         264      322      +58
  Branches       53       58       +5
==========================================
+ Hits          255      313      +58
  Partials        9        9

Impacted Files
Coverage Δ

src/sdsstools/logger.py
94.36% <ø> (ø)

src/sdsstools/daemonizer.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7f23b7...ea4eea3. Read the comment docs.


Junit5 allow switch off slow or flaky tests

Category
This change is exactly one of the following (please change [ ] to [x]) to indicate which:

[ ] a bug fix (Fix #...)
[ ] a new Ripper
[ ] a refactoring
[ ] a style change/fix
[ x] a new feature

Description
allow switching off slow tests via command line by adding a tag, description on how to do it in commit message. slow tests are run by default. the same principle is used to switch off flaky tests. they are not run by default to get a green build. as jdk-9 died out, replace it with jdk-14 for the build matrix. number of failing tests and reasons for failure stay the same.
Testing
Required verification:

[x ] I've verified that there are no regressions in mvn test (there are no new failures or errors).
[ x] I've verified that this change works as intended.

[x ] Downloads all relevant content.
[x ] Downloads content from multiple pages (as necessary or appropriate).
[ x] Saves content at reasonable file names (e.g. page titles or content IDs) to help easily browse downloaded content.

[ ] I've verified that this change did not break existing functionality (especially in the Ripper I modified).

Optional but recommended:

[ ] I've added a unit test to cover my change.

@cyian-1756 i left out again a couple of tests again to get a green build.

one wordpresscomicrippertest failed, mark it as flaky and changed all other failed from disabled to flaky

LGTM


How to show lsp server is started like in the readme

The title says it all. I followed the code in the readme. There is a very long error

Can you show the error you're getting?

ah wait. can u tell where should i be putting that func in? and will it appear everytime i enter a new file? like i am in init.lua, it pop up for the first time. when i enter mappings.lua, it appears again. will that happen?

The code on the wiki makes the implicit assumption that you replaced vim.notify by the notification function provided by this plugin. I.e., it assumes that you also have the line
vim.notify = require("notify")

in your init.lua
That's probably something worth changing in the wiki...

@rcarriga

@vogelsgesang Thank you yes this is the problem here, I've updated the wiki to note the assumption.
@bryant-the-coder Please don't tag people unnecessarily, they will see the notifications already and it just acts as spam
Closing this as the problem is solved


Remove: IvanG.BrowserTamer version 3.1.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: codeflow:open?pullrequest=https://github.com/microsoft/winget-pkgs/pull/109405&drop=dogfoodAlpha

Closing as duplicate of #109407