Language Server appears to be out of sync with command line gradle

Hi Kotlin Language Server developers,
Many thanks for a great project. I have been trying to set it up against emacs, but I must be doing something very silly because I get errors with KLS that I do not see when I manually run gradle. Let me first explain my setup. I am using Debian testing on x64 and I installed kotlin using SDK Man [0]. I have tried two distinct setups though, sadly, with more or less the same outcome.
Setup using Java 11
I have read through the many posts detailing how it is not possible to use Java 11 to build KLS - e.g. [1, 2, 3] - but it was still  unclear to me if this limitation also applied to building user projects or only to building KLS itself [4]. So I started first by installing openjdk-11 and using the binaries supplied on GitHub [5]. Though there is no out of the box support on emacs for KLS [6], thanks to the help of the emacs lsp community, I managed to get emacs to start KLS and report errors on my kotlin code.
The weird thing was that it all worked well at first, but as I started to add more dependencies and change the project files, I started to see errors in KLS that I do not see in gradle. In particular, I see:
 Server.kt    20  27 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Server.kt    27  33 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)

However, my gradle file states:
tasks.withType<KotlinCompile> {
    kotlinOptions.jvmTarget = "1.8"
}

In addition, my kts (gradle Kotlin script files) are riddled with errors:
 build.gra…     1  29 error    UNRES… kotlin: Unresolved reference: gradle (lsp-ui)
 build.gra…     3   1 error    UNRES… kotlin: Unresolved reference: plugins (lsp-ui)
 build.gra…     4   5 error    UNRES… kotlin: Unresolved reference: application (lsp-ui)
 build.gra…     5   5 error    UNRES… kotlin: Unresolved reference: id (lsp-ui)
...

The first error stems from this line:
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

To the untrained eye, it seems like the gradle kotlin DSL files are just not being picked up correctly.
Setup using Java 8
I then decided to investigate two questions:

check if there have been fixes in master that address my problem
check if Java 11 is also not supported in user projects.

For this I relied on openjdk-8, but sadly it is not (entirely) available on Debian testing. The missing packages were downloaded manually from their FTP server [7]:
openjdk-8-jdk_8u212-b03-2_deb9u1_amd64.deb
openjdk-8-jdk-headless_8u212-b03-2_deb9u1_amd64.deb

Once installed I selected the java version via alternatives:
update-alternatives --config java

And set it to Java 8. I confirmed that my binaries was running from Java 8, as well as my local build using gradle. I then git cloned KLS and built it as per my understanding of the instructions:
./gradlew server:build
./gradlew server:installDist

It all worked fine. I then installed the resulting binaries, restarted emacs and picked up the new KLS. However, I am getting exactly the same errors as I had before, as per section above.
Conclusions
So it seems to me my problems are actually not related to the java version, but to my project. The weird thing is that when I run gradle build (or via the gradle wrapper) it all works. So I'm trying to figure out what KLS is doing differently. Can you point out to mistakes I have done on my setup, or if nothing obvious can be spotted, on ways to get more debug information about what KLS is doing?
Thanks very much for all your help.
Marco
[0] https://kotlinlang.org/docs/tutorials/command-line.html
[1] https://github.com/fwcd/KotlinLanguageServer/issues/125
[2] https://github.com/fwcd/KotlinLanguageServer/issues/108
[3] https://github.com/fwcd/KotlinLanguageServer/issues/114
[4] https://github.com/fwcd/KotlinLanguageServer/pull/124
[5] https://github.com/fwcd/KotlinLanguageServer/releases/tag/0.1.13
[6] https://github.com/emacs-lsp/lsp-mode/issues/900
[7] http://ftp.debian.org/debian/pool/main/o/openjdk-8/

Actually, an important omission in the previous commentary is that there likely are some problems in the setup of my kotlin project. I have created as a way to learn how kotlin projects work, (mostly) by hand, and with the objective of supporting multi-project builds [1]. Maybe I have done something that is confusing KLS.
My file structure is as follows (files that were generated by gradle are marked as [generated]):
./gradlew.bat                                                 [generated]
./build.gradle.kts
./gradle.properties
./settings.gradle.kts
./gradlew                                                     [generated]
./gradle                                                      [generated]
./gradle/wrapper                                              [generated]
./gradle/wrapper/gradle-wrapper.properties                    [generated]
./gradle/wrapper/gradle-wrapper.jar                           [generated]
./projects
./projects/cli
./projects/cli/build.gradle.kts
./projects/cli/src
./projects/cli/src/test
./projects/cli/src/test/kotlin
./projects/cli/src/test/kotlin/abc
./projects/cli/src/test/kotlin/abc/cli
./projects/cli/src/test/kotlin/abc/cli/HelloTest.kt
./projects/cli/src/main
./projects/cli/src/main/kotlin
./projects/cli/src/main/kotlin/abc
./projects/cli/src/main/kotlin/abc/cli
./projects/cli/src/main/kotlin/abc/cli/Server.kt
./projects/core
./projects/core/build.gradle.kts
./projects/core/src
./projects/core/src/test
./projects/core/src/test/kotlin
./projects/core/src/test/kotlin/abc
./projects/core/src/test/kotlin/abc/core
./projects/core/src/test/kotlin/abc/core/HelloTest.kt
./projects/core/src/main
./projects/core/src/main/kotlin
./projects/core/src/main/kotlin/abc
./projects/core/src/main/kotlin/abc/core
./projects/core/src/main/kotlin/abc/core/Hello.kt

All my targets (build, test, etc) work from the gradle CLI. I will try to clean up the project and put it on a stand alone GitHub project if it helps.
[1] https://docs.gradle.org/current/userguide/multi_project_builds.html

Yet another interesting point: I see a few errors in *kotlin-ls::stderr* that look puzzling:
ERROR: Exception while analyzing expression at (1,492) in /dummy.kt
org.jetbrains.kotlin.utils.KotlinExceptionWithAttachments: Exception while analyzing expression at (1,492) in /dummy.kt
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.logOrThrowException(ExpressionTypingVisitorDispatcher.java:233)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.lambda$getTypeInfo$0(ExpressionTypingVisitorDispatcher.java:211)
	at org.jetbrains.kotlin.util.PerformanceCounter.time(PerformanceCounter.kt:91)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.getTypeInfo(ExpressionTypingVisitorDispatcher.java:161)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.getTypeInfo(ExpressionTypingVisitorDispatcher.java:145)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingServices.getTypeInfo(ExpressionTypingServices.java:114)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingServices.getTypeInfo(ExpressionTypingServices.java:93)
	at org.javacs.kt.Compiler.compileExpression(Compiler.kt:148)
	at org.javacs.kt.CompiledFile.bindingContextOf(CompiledFile.kt:39)
	at org.javacs.kt.CompiledFile.referenceAtPoint(CompiledFile.kt:53)
	at org.javacs.kt.hover.HoversKt.hoverAt(Hovers.kt:37)
	at org.javacs.kt.KotlinTextDocumentService$hover$1.invoke(KotlinTextDocumentService.kt:80)
	at org.javacs.kt.KotlinTextDocumentService$hover$1.invoke(KotlinTextDocumentService.kt:26)
	at org.javacs.kt.util.AsyncExecutorKt$sam$java_util_function_Supplier$0.get(AsyncExecutor.kt)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: Should not be called
	at org.jetbrains.kotlin.resolve.lazy.NoTopLevelDescriptorProvider.shouldNotBeCalled(TopLevelDescriptorProvider.kt:36)
	at org.jetbrains.kotlin.resolve.lazy.NoTopLevelDescriptorProvider.getPackageFragment(TopLevelDescriptorProvider.kt:39)
	at org.jetbrains.kotlin.resolve.DeclarationResolver.getTopLevelDescriptorsByFqName(DeclarationResolver.kt:107)
	at org.jetbrains.kotlin.resolve.DeclarationResolver.checkRedeclarationsInPackages(DeclarationResolver.kt:86)
	at org.jetbrains.kotlin.resolve.LazyTopDownAnalyzer.analyzeDeclarations(LazyTopDownAnalyzer.kt:214)
	at org.jetbrains.kotlin.types.expressions.LocalClassifierAnalyzer.processClassOrObject(LocalClassifierAnalyzer.kt:106)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorForStatements.visitObjectDeclaration(ExpressionTypingVisitorForStatements.java:102)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorForStatements.visitObjectDeclaration(ExpressionTypingVisitorForStatements.java:62)
	at org.jetbrains.kotlin.psi.KtObjectDeclaration.accept(KtObjectDeclaration.kt:63)
	at org.jetbrains.kotlin.types.expressions.ExpressionTypingVisitorDispatcher.lambda$getTypeInfo$0(ExpressionTypingVisitorDispatcher.java:172)
	... 16 more
Jul 16, 2019 8:16:13 PM org.eclipse.lsp4j.jsonrpc.RemoteEndpoint fallbackResponseError

And
SEVERE: Internal error: org.javacs.kt.util.KotlinLSException: Error while analyzing: object Server {
...
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1592)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.javacs.kt.util.KotlinLSException: Error while analyzing: object Server {

The second error is related to plain kotlin code, the first I'm not too sure.

As a control test, I've decided to clone the KotlinQuickStart repo and validate KTS from there. I experience the same problems with the Kotlin DSL. Upon further investigation, it seems the kotlin DSL is actually not yet supported [2], so I will switch off LSP for these files. On the other hand, I do seem to get good error checking on the regular kotlin files. I will try to bring across code from my project and see if I can get it to work...
[1]https://github.com/fwcd/KotlinQuickStart
[2] https://github.com/fwcd/KotlinLanguageServer/issues/6

OK, I've managed to reproduce my problems using the KotlinQuickStart repo. I modified the code as follows:
diff --git a/build.gradle b/build.gradle
index 936ec53..679d111 100644
--- a/build.gradle
+++ b/build.gradle
@@ -16,14 +16,29 @@ version = '0.1.1'
 mainClassName = 'MainKt'
 description = 'My Application'

-sourceCompatibility = 1.8
-targetCompatibility = 1.8
-
 repositories {
 	mavenCentral()
+    jcenter()
+
 }

 dependencies {
 	implementation 'org.jetbrains.kotlin:kotlin-stdlib'
 	testImplementation 'junit:junit:4.12'
+
+    compile 'io.vertx:vertx-core:3.7.1'
+    compile 'io.vertx:vertx-web:3.7.1'
+    compile 'io.vertx:vertx-jdbc-client:3.7.1'
+
+}
+
+compileKotlin {
+    sourceCompatibility = JavaVersion.VERSION_1_8
+    targetCompatibility = JavaVersion.VERSION_1_8
+
+    kotlinOptions {
+        jvmTarget = "1.8"
+        apiVersion = "1.1"
+        languageVersion = "1.1"
+    }
 }
diff --git a/src/main/kotlin/Main.kt b/src/main/kotlin/Main.kt
index 46dfdfc..a0ecc39 100644
--- a/src/main/kotlin/Main.kt
+++ b/src/main/kotlin/Main.kt
@@ -1,3 +1,34 @@
+import io.vertx.core.AsyncResult
+import io.vertx.core.Future
+import io.vertx.core.Vertx
+import io.vertx.core.json.Json
+import io.vertx.core.json.JsonArray
+import io.vertx.core.json.JsonObject
+import io.vertx.ext.jdbc.JDBCClient
+import io.vertx.ext.sql.ResultSet
+import io.vertx.ext.sql.SQLConnection
+import io.vertx.ext.web.Router
+import io.vertx.ext.web.RoutingContext
+import io.vertx.ext.web.handler.BodyHandler
+import java.util.*
+import kotlin.reflect.KClass
+
+object Server {
+    @JvmStatic fun main(args: Array<String>) {
+
+        val vertx = Vertx.vertx()
+        val server = vertx.createHttpServer()
+        val port = 9000
+        val router = Router.router(vertx)
+
+        router.get("/").handler { it.response().end("Hello world!") }
+        server.requestHandler { router.accept(it) }.listen(port) {
+            if (it.succeeded()) println("Server listening at $port")
+            else println(it.cause())
+        }
+    }
+}
+
 fun main(args: Array<String>) {
 	println("Hello world")
 }

I had to add jvmTarget etc to get the code to compile with gradle. However, KTS is now complaining of unresolved references:
 Main.kt     1   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     2   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     3   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     4   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     5   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     6   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     7   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     8   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt     9   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    10   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    11   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    12   8 error    UNRES… kotlin: Unresolved reference: io (lsp-ui)
 Main.kt    19  21 error    UNRES… kotlin: Unresolved reference: Vertx (lsp-ui)
 Main.kt    22  22 error    UNRES… kotlin: Unresolved reference: Router (lsp-ui)
 Main.kt    24  35 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    25  47 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    26  17 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)
 Main.kt    27  26 error    UNRES… kotlin: Unresolved reference: it (lsp-ui)

These are all of the vertx imports. It seems I need to "resync" KTS after gradle downloaded the new jars?

OK, I now realised that by restarting my workspace I can get KLS to pick up the dependencies (lsp-restart-workspace in emacs). However, the only snag is that I still can't get it to recognise my Java 1.8 settings:
 Main.kt    19  27 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Main.kt    23  29 error    INTER… kotlin: Calls to static methods in Java interfaces are prohibited in JVM target 1.6. Recompile with '-jvm-target 1.8' (lsp-ui)
 Main.kt    26  40 warning  DEPRE… kotlin: 'accept(HttpServerRequest!): Unit' is deprecated. Deprecated in Java (lsp-ui)

For good measure, I installed VS code just to see if this was an emacs problem but it also complains about the same error.

OK right so this ticket is quite long and rambling-y, so I think its worth summarising my findings:

As stated in many other places, you need Java 8 to build KLS itself. (#125, #108, #114)
Kotlin DSL files are not supported by KLS. (#124)
It seems building user projects using Java 11 gives the exact same behaviour as using Java 8 - e.g. they are supported by KLS.
Adding new dependencies (jars) to a project appears to require a restart of KLS when used under emacs ( (lsp-restart-workspace).

Which means that the only real problem I have is this one:

Warnings related to using Java 8 features appear even though the project has jvmTarget = "1.8" set in the gradle files. This behaviour is present in both in v0.1.13 and master.

To avoid confusion, I think I will close this ticket and open a new one focused only on this problem.

This ticket already exists [1], so I'll move the discussion there.
[1] https://github.com/fwcd/KotlinLanguageServer/issues/72

Thanks for investigating and providing such a detailed description of the issue!
I have to admit that I haven't got around to test the compiled KLS on Java 11 projects yet, though this will definitely be part of #109, since KLS should always support its own codebase.


Foreign keys

Hello!
I have to say Spot is a very neat library. I set up my database schema with it and everything was fine until I realized that the foreign keys are missing from the db. Am I blind or something? If I create the foreign keys by hand the next migration removes them. Do Spot support foreign keys? If not then why not?
What should I do? Switch to a different orm (datamapper) or extend Spot's functionality?
PS.: I need foreign keys for my university project.

Do not create them by hand. Create them with the migrations. For example.
public static function fields()
{
    return [
        "id"        => ["type" => "integer", "primary" => true, "autoincrement" => true],
        "parent_id" => ["type" => "integer"],
        ...
        "created_at" => ["type" => "datetime", "value" => new \DateTime()],
        "updated_at" => ["type" => "datetime", "value" => new \DateTime()]
    ];
}

That's okay. I have an Entity like this:
public static function fields()
  {
    return [
      'id' => ['type' => 'integer', 'primary' => true, 'autoincrement' => true],
      'course_id' => ['type' => 'integer', 'index' => true],
      'student_id' => ['type' => 'integer', 'index' => true]
    ];
  }
  public static function relations(Mapper $mapper, Entity $entity)
  {
    return [
        'course' => $mapper->belongsTo($entity, 'Entity\Course', 'course_id'),
        'student' => $mapper->belongsTo($entity, 'Entity\Student', 'student_id')
    ];
  }

The field part is okay, Spot created the table from it.
But the thing is the 'relations' part is not mapped to the database. I need constraints like this:
CREATE TABLE example (
    id INT,
    course_id INT,
    student_id INT,
    INDEX course_id_fk (course_id),
    INDEX student_id_fk (student_id),
    FOREIGN KEY (course_id)
        REFERENCES course(id)
        ON DELETE CASCADE,
    FOREIGN KEY (student_id)
        REFERENCES student(id)
        ON DELETE CASCADE
) ENGINE=INNODB;

But Spot only did this:
CREATE TABLE example (
    id INT,
    course_id INT,
    student_id INT,
    INDEX course_id (course_id),
    INDEX student_id (student_id)
) ENGINE=INNODB;

Foreign keys are something that Spot simply doesn't have yet - not because of any specific decision that was made to exclude them. It would be great if you could fork and modify Spot to include them by default (and optionally omit them in case of errors).
The file you will want to look at to achieve this is here:
https://github.com/vlucas/spot2/blob/master/lib/Spot/Query/Resolver.php#L78

I need the foreign keys too:
Here is a draft of a solution for belongsTo relations, I might do a fork when I'm satisfied with my solution. Insert the code into the migrateCreateSchema() method.

        //TODO Should move that to the entityManager
        $relations = call_user_func(array($entityName, 'relations'), $this->mapper, new $entityName);
        foreach ($relations as $relation_name => $relation) {
            if ($relation instanceof \Spot\Relation\BelongsTo) {
                $parent_entity = $relation->entityName();

                $foreignTable = call_user_func(array($parent_entity, 'table'));
                //TODO Should be customisable, sometimes you want to delete other times you don't want to
                $table->addForeignKeyConstraint($foreignTable, array($relation->localKey()), array($relation->foreignKey()), array("onDelete" => "CASCADE"));

            }
            //TODO Other types of relations
        }

Thanks @nebulousGirl - that's a great start.

Bumping here.
I've decided to put some work on this issue, because having foreign keys supported would be really a good thing. Following the example from above everything should be working just fine when it comes to BelongsTo and HasOne relations and I've added some code to my fork of this project with the ability to define yourself what should happen when onUpdate and onDelete occurs. But, when I ran the tests, I noticed a strange behaviour for HasOne relation.
The relation itself is added at the inverse side of the relation, and it should be at the owning side. Because when we execute addForeignKeyConstraint the code is expecting for the column with our foreign key to be defined in this table, which would eventually result to Invalid Foreign key Consrtraint exception. At the very least I can say that currently, if this feature is enabled, this test will fail.
You can see for a reference, the way Doctrine handles this for better understanding of what I'm trying to say. Anyway, any thoughts on this @vlucas ?

Do we have something here? Is @nebulousGirl's solution any good or do I need to look into it?
I invested a lot of time in Spot during the past few months. I built my own library around it, but now I have a lot of problems caused by the lack of foreign keys. I can't migrate my tables properly and now it's a really big pile of painful mess. I don't like the idea of dropping the database constraints.

I tested @nebulousGirl code and fixed it: 3057994

The relations variable was always an empty array for me, it didn't do anything.
Added alias support.

It works for me but further testing is required.
I'm using my fork now but If you want I'll create a PR.

#77 and #156 have been merged.


[flutter_plugin_tests] Split analyze out of xctest

To prep for making a combined command to run native tests across different platforms, rework xctest:

Split analyze out into a new xcode-analyze command:

Since the analyze step runs a new build over everything with different flags, this is only a small amount slower than the combined version
This makes the logic easier to follow
This allows us to meaningfully report skips, to better notice missing tests.

Add the ability to target specific test bundles (RunnerTests or RunnerUITests)

To share code between the commands, this extracts a new Xcode helper class.
Part of https://github.com/flutter/flutter/issues/84392 and https://github.com/flutter/flutter/issues/86489
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the relevant style guides and ran the auto-formatter. (Note that unlike the flutter/flutter repo, the flutter/plugins repo does use dart format.)
[x] I signed the CLA.
[x] The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
[x] I listed at least one issue that this PR fixes in the description above.
[ ] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy.
[x] I updated CHANGELOG.md to add a description of the change.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test exempt.
[x] All existing and new tests are passing.

Add the ability to target specific test bundles (RunnerTests or RunnerUITests)

Both types of tests will run on the same build artifacts, so it should be faster to run them at the same time.  It's a fine feature to add, just keep it in mind if you intend to split them into separate CI tests.

Both types of tests will run on the same build artifacts, so it should be faster to run them at the same time. It's a fine feature to add, just keep it in mind if you intend to split them into separate CI tests.

Yes, I had tested that at the same time I tested splitting out analyze and as expected it is much slower. This is intended for local use (and is prep for the new combined PR that will have options to select unit or integration tests for local use, and maybe for use in CI on platforms where combining them doesn't help).


Feature/more rework adj jac apply

I converted the last few functions that use adj_jac_apply to the new thing.
I'm gonna put a couple questions inline.
Edit: It was very easy. I think we should deprecate adj_jac_apply for the next release.
Also we could have called this tadj_jac_apply.

Technically this operation is transpose(adjoint) * Jacobian, so that is a more precise name
Tadej -> Tadj -> tadj -> tadj_jac_apply is a good bit of easter-egg history

It's a shame reverse_pass_callback is a good name.

I'm happy with this code now. You can either merge it into your branch or we'll do it as a separate pull req. into Stan. Might be simpler to do the second.

The current adj_jac PR is pretty small so I think it would be fine to merge into there. Though again because I goofed the merge earlier it may be better to wait for the adj_jac PR based on develop, then just squash this branch and cherry pick the squashed commit onto the adj_jac branch based off develop
Also I'm unironically okay with calling it tadj_jac_apply. I love the easter egg and if we are doing transpose(adjoint) * Jacobian it really is a more informative name


Update the initial IR statement construction and improve the logic to compute the innermost locations

This PR makes initial IR construction to explicitly use the precomputed topological order + uses the order to compute innermost locations of the functions without explicit compute_at.
The high-level structure of the code:

compute topological order for the functions in the pipeline.
using the order from 1) build loop nest tree and compute undefined compute_at locations.
construct the initial IR statement using 1) and 2).

This addresses the issues described in #146 and some other bugs. For the test from #146, the final IR is nice and clean:
intm2 = allocate(heap, 2, {
  {[min(buffer_min(intm4, 0), buffer_min(intm3, 0)), max(buffer_max(intm4, 0), buffer_max(intm3, 0))], 2, <>},
  {[min(buffer_min(intm4, 1), buffer_min(intm3, 1)), max(buffer_max(intm4, 1), buffer_max(intm3, 1))], (((max(buffer_max(intm4, 0), buffer_max(intm3, 0)) - min(buffer_min(intm4, 0), buffer_min(intm3, 0))) * 2) + 2), <>}
}) {
 call(<fn>, {in1}, {intm2})
 call(<fn>, {intm2}, {intm3})
 call(<fn>, {intm2}, {intm4})
}

All existing tests pass with  bazel test //....
Unrelated change: I've added a name field to the func to use it for easier debugging. I can move it to another PR or remove it altogether if people think it's not needed.

Only a few minor nits left. You should also run clang-format on the changed files, there are a few minor formatting issues I noticed.

Done.

It would also be worth skimming the output of pipeline_test with SLINKY_VERBOSE=1 to see if the output still looks good. I have tried to cover as much as I could with actual checks, but I still sometimes catch regressions just by looking at the IR for changes like this.

Yes, I've doing this all along while working on this (especially, when trying to fix the tests which were failing at the moment).


When I use the keyboard to comment out more than 15 lines of code (excluding blank lines) in VSCode, it crashes.

Type: Bug
When I use the keyboard to comment out more than 15 lines of code (excluding blank lines) in VSCode, it crashes. However, if I use the comment button in the menu bar, it doesn’t crash. I haven’t found any useful information in the VSCode editor logs and this has been bothering me for years. I don’t know how to solve it.
This issue occurs on both my work and home computers.

Please use the Start Extension Bisect command to investigate whether your problem is being caused by an extension.

Version: 1.77.1 (user setup)
Commit: b7886d7461186a5eac768481578c1d7ca80e2d21
Date: 2023-04-04T23:21:11.906Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19044
Sandboxed: No

I have already uninstalled all the plugins but it still crashes. I will try to reinstall it later.

请使用 Start Extension Bisect 命令来调查您的问题是否是由扩展引起的。

if I use the comment button in the menu bar

Maybe this is coming from an extension.
Please use the Start Extension Bisect command to investigate whether your problem is being caused by an extension.

Thanks for your reply  .I have already uninstalled all the plugins but it still crashes. I will try to reinstall it later.

Version: 1.77.1 (user setup) Commit: b7886d7 Date: 2023-04-04T23:21:11.906Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: <IP_ADDRESS>-electron.0 OS: Windows_NT x64 10.0.19044 Sandboxed: No

Extension Bisect is active and has disabled 0 extensions. Check if you can still reproduce the problem and proceed by selecting from these options.

After adding the editor.accessibilitySupport: on property in the settings.json file, VSCode crashes when commenting multiple lines of code. I’m not sure which plugin carries this configuration item. I found that it is a command only available in Visual Studio.

This problem has been bothering me for many years and I finally solved it today. Thank you for your guidance.

@shiyuanGame I cannot reproduce a crash by configuring editor.accessibilitySupport: on and commenting out more than 15 lines of code. Could you please follow the steps at https://github.com/microsoft/vscode/wiki/Native-Crash-Issues to obtain a crash dump?

I am commenting on my settings line by line. I have restored my backup extensions. If you want to reproduce this, you can find an empty VSCode and install it. If I reinstall or use my original configuration, this situation will occur.
vscode setting.zip

@shiyuanGame I cannot reproduce a crash by configuring editor.accessibilitySupport: on and commenting out more than 15 lines of code. Could you please follow the steps at https://github.com/microsoft/vscode/wiki/Native-Crash-Issues to obtain a crash dump?

This is the backup I made before solving the problem.

My English is not very good, so I need to use a translation tool. I will try to provide feedback in this way tomorrow: https://github.com/microsoft/vscode/wiki/Native-Crash-Issues.  thank

@shiyuanGame I cannot reproduce a crash by configuring editor.accessibilitySupport: on and commenting out more than 15 lines of code. Could you please follow the steps at https://github.com/microsoft/vscode/wiki/Native-Crash-Issues to obtain a crash dump?

This time I imported the previous configuration without importing the extension. Even after commenting out 50 lines of code, it still crashes.
The .dmp file has been generated and I have attached it to the current conversation.
reports.zip

@alexdima
If there is anything else I can do, please feel free to contact me. I am willing to contribute my own strength for the advancement of technology. Thank you to the developers for using code to change the world.

@deepak1556 @rzhao271 Can you please take a look at the attached .dmp file? I cannot reproduce the crash.

watcher.node appears a lot in the log file, but I'm not sure how to obtain symbols for it.
Also, Frame 25 is TerminateProcessOnMemoryExhaustion, so I wonder whether there was a memory leak before the crash.
symbolicated-e9.log

Do you need me to record a video demonstration?

Actually yeah, a recording or example file would be helpful, especially if it contains non-ASCII characters.

版本: 1.77.3 (system setup)
提交: 704ed70d4fd1c6bd6342c436f1ede30d1cff4710
日期: 2023-04-12T09:16:02.548Z
Electron: 19.1.11
Chromium: 102.0.5005.196
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19044
沙盒化: No
Hey guys, I just tested it and the issue seems to be gone. Has the bug been fixed?

I don't believe that the bug was directly fixed between 1.77.1 and 1.77.3.
I'll close this issue for now, but feel free to comment if the bug appears again.


Two fixes for our spa's integration with the GoodData library.

Rxjs library update to match the version of GoodData and adding a require to reflect-metadata so our unit tests are able to run and produce code coverage.

Codecov Report

Merging #298 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #298   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          58     58
  Lines        1358   1358
  Branches      205    205
=====================================
  Hits         1358   1358

Flag
Coverage Δ

#builder
100% <ø> (ø)
:arrow_up:

#runtime
100% <ø> (ø)
:arrow_up:

#srcapp
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92dfe68...1f43fc3. Read the comment docs.


[Solved] UnsatisfiedLinkError on Arch Linux

As we discussed in this reddit thread, Arch doesn't seem to have libtinfo.so.5 but an easy solution is to symlink the newer one to it:
sudo ln -s /usr/lib/libtinfo.so.6 /usr/lib/libtinfo.so.5

Thank you very much, for me this solved the error
Execution error (UnsatisfiedLinkError) at org.lwjgl.system.linux.LinuxLibrary/loadLibrary (LinuxLibrary.java:32).
Failed to dynamically load library: /tmp/lwjglch/3.2.3-build-13/libvim.so(error = null)
on Arch Linux


WEB-1980: Clear unused files from fetch cache to reduce OOM errors

This ups the cache expiration to 24 hours (as the files themselves shouldn't be changing) and then uses the new logic for clearing the cache of unused files every 15 minutes. I think we'll want to test this in practice to see if there's a duration that makes more sense but I think this is a good starting point.
Related PR: https://github.com/Khan/cache-service-cache-module/pull/1
Issue: https://khanacademy.atlassian.net/browse/WEB-1980
Test plan:
yarn test passed. I think the rest we'll want to test in production to see how it works!

@kevinbarabash Yeah, having ways of tweaking these variables would be pretty cool. At the very least this change shouldn't make things any worse than before (hitting uncached files) and it should make it slightly better for the common case (the current live deploy's files should stay in the cache for longer than before).


WIP: rewrite SCIP wrapper with Clang.jl to support a MIP solver through MOI

do not support callbacks or nonlinear expressions :disappointed:

LQOI wasn't designed to work with nonlinear. What's the motivation for using it instead of directly writing an MOI wrapper? SCIP's API is rich enough that LQOI might not be a good fit (it was designed to wrap APIs that look like Gurobi/CPLEX/Xpress).
@odow @joaquimg

Hmm yeah LQOI isn't a good fit for SCIP. Best to wrap directly.

Rich enough, or too rich? :smirk:
Well, that decision about LQOI was premature, I only looked briefly at the GLPK impementation. What I mean is that at first I will only support SCIP as a MIP solver.
I plan to drop the MPB support, and with it the support for nonlinear constraints and also callbacks. That would be added later. I'm not sure whether it's actually a good idea to do this on the current project, rather than start a new one. But I guess nobody will maintain the old code anyways, and the last version of SCIP.jl to work with JuMP 0.18 is version-tagged, so it should be fine?
cc @fserra

A reason not to drop MPB support yet is that the SCIP package will immediately stop working with JuMP 0.18 and users will have to pin it to an old version to get it working. You'll probably spend more time handling these support questions than you would maintaining the MPB interface for another few months until lots of people are using JuMP 0.19.

OTOH given that it's a complete rewrite I see why you wouldn't want to put the effort into re-enabling MPB support. I didn't consider this in my comment 30 seconds ago :)

Well, yes, I definitely wanted to get rid of the CSIP dependency, so it was a rewrite.
I figure the current release of the code has an upper bound on MathProgBase while the new release would have a lower bound on MathOptInterface, so the package manage should pick the appropriate version depending on the installed JuMP automatically? Maybe that's wishful thinking.

@mlubin Feel free to review and comment. I'm not done yet, but basic LP solving should work now.
In particular, you could suggest additional MOI tests to activate.
I still want to add MIP stuff (variable types, SOS), quadratic constraints and maybe deletion of variables and constraints.
Maybe I'll also expose some dual solution things for LP, but I'm not up-to-date on how easy that is nowadays. In the past, it required going deep into the SCIP-LP-solver interface, but I heard that this has changed.

dual lp information is not really possible. we tried to make it work, but
there are fundamental issues, like constraints with one variable get
deleted even if presolve is off. so we have to rely on reduced costs to get
the multipliers, and if there are copies of the same constraint, they will
all get the same multiplier.
On Sat, Dec 22, 2018, 22:49 Robert Schwarz<EMAIL_ADDRESS>wrote:

@mlubin https://github.com/mlubin Feel free to review and comment. I'm
not done yet, but basic LP solving should work now.
In particular, you could suggest additional MOI tests to activate.
I still want to add MIP stuff (variable types, SOS), quadratic constraints
and maybe deletion of variables and constraints.
Maybe I'll also expose some dual solution things for LP, but I'm not
up-to-date on how easy that is nowadays. In the past, it required going
deep into the SCIP-LP-solver interface, but I heard that this has changed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SCIP-Interfaces/SCIP.jl/pull/76#issuecomment-449599594,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFNG9tkM78XPPx1ecohLHaHnVic0PXAMks5u7qkDgaJpZM4ZYUfz
.


1.21 Support Needed

Hello there, is this Mod still maintained, and will eventually updated to 1.21?

It works with 1.21


Release build failed with new 3.0.0

Short description of the problem:
After upgrading to new ionic-angular 3.7.0 and ionic-app-scripts 3.0.0 on building a release bundle with
ionic cordova build android --release
we get this output with error:
ionic cordova build android --release
[INFO] Running app-scripts build: --platform android --target cordova

[12:30:22]  build dev started ...
[12:30:22]  clean started ...
[12:30:22]  clean finished in 7 ms
[12:30:22]  copy started ...
[12:30:22]  deeplinks started ...
[12:30:22]  deeplinks finished in 265 ms
[12:30:22]  transpile started ...
[12:30:26]  transpile finished in 3.86 s
[12:30:26]  preprocess started ...
[12:30:26]  copy finished in 4.23 s
[12:30:26]  preprocess finished in 80 ms
[12:30:26]  webpack started ...
[12:30:39]  webpack finished in 12.92 s
[12:30:39]  sass started ...
[12:30:40]  sass finished in 1.35 s
[12:30:40]  postprocess started ...
[12:30:41]  postprocess finished in 33 ms
[12:30:41]  lint started ...
[12:30:41]  build dev finished in 18.70 s
> ionic cordova prepare android
> cordova prepare android

> cordova build android --release
✖ Running command - failed!
[ERROR] An error occurred while running cordova build android --release (exit code 1):

        (truncated) ... s scheduled to be removed in Gradle 4.0.
        Incremental java compilation is an incubating feature.
        The TaskInputs.source(Object) method has been deprecated and is scheduled to be removed in Gradle 4.0. Please
        use TaskInputs.file(Object).skipWhenEmpty() instead.
        :preBuild UP-TO-DATE
        :preReleaseBuild UP-TO-DATE
        :checkReleaseManifest
        :preDebugBuild UP-TO-DATE
        :CordovaLib:preBuild UP-TO-DATE
        :CordovaLib:preDebugBuild UP-TO-DATE
        :CordovaLib:checkDebugManifest
        :CordovaLib:prepareDebugDependencies
        :CordovaLib:compileDebugAidl UP-TO-DATE
        :CordovaLib:compileDebugNdk UP-TO-DATE
        :CordovaLib:compileLint UP-TO-DATE
        :CordovaLib:copyDebugLint UP-TO-DATE
        :CordovaLib:mergeDebugShaders UP-TO-DATE
        :CordovaLib:compileDebugShaders UP-TO-DATE
        :CordovaLib:generateDebugAssets UP-TO-DATE
        :CordovaLib:mergeDebugAssets UP-TO-DATE
        :CordovaLib:mergeDebugProguardFiles UP-TO-DATE
        :CordovaLib:packageDebugRenderscript UP-TO-DATE
        :CordovaLib:compileDebugRenderscript UP-TO-DATE
        :CordovaLib:generateDebugResValues UP-TO-DATE
        :CordovaLib:generateDebugResources UP-TO-DATE
        :CordovaLib:packageDebugResources UP-TO-DATE
        :CordovaLib:processDebugManifest UP-TO-DATE
        :CordovaLib:generateDebugBuildConfig UP-TO-DATE
        :CordovaLib:processDebugResources UP-TO-DATE
        :CordovaLib:generateDebugSources UP-TO-DATE
        :CordovaLib:incrementalDebugJavaCompilationSafeguard UP-TO-DATE
        :CordovaLib:compileDebugJavaWithJavac UP-TO-DATE
        :CordovaLib:processDebugJavaRes UP-TO-DATE
        :CordovaLib:transformResourcesWithMergeJavaResForDebug UP-TO-DATE
        :CordovaLib:transformClassesAndResourcesWithSyncLibJarsForDebug UP-TO-DATE
        :CordovaLib:mergeDebugJniLibFolders UP-TO-DATE
        :CordovaLib:transformNative_libsWithMergeJniLibsForDebug UP-TO-DATE
        :CordovaLib:transformNative_libsWithSyncJniLibsForDebug UP-TO-DATE
        :CordovaLib:bundleDebug UP-TO-DATE
        :CordovaLib:preReleaseBuild UP-TO-DATE
        :CordovaLib:checkReleaseManifest
        :CordovaLib:prepareReleaseDependencies
        :CordovaLib:compileReleaseAidl UP-TO-DATE
        :CordovaLib:compileReleaseNdk UP-TO-DATE
        :CordovaLib:copyReleaseLint UP-TO-DATE
        :CordovaLib:mergeReleaseShaders UP-TO-DATE
        :CordovaLib:compileReleaseShaders UP-TO-DATE
        :CordovaLib:generateReleaseAssets UP-TO-DATE
        :CordovaLib:mergeReleaseAssets UP-TO-DATE
        :CordovaLib:mergeReleaseProguardFiles UP-TO-DATE
        :CordovaLib:packageReleaseRenderscript UP-TO-DATE
        :CordovaLib:compileReleaseRenderscript UP-TO-DATE
        :CordovaLib:generateReleaseResValues UP-TO-DATE
        :CordovaLib:generateReleaseResources UP-TO-DATE
        :CordovaLib:packageReleaseResources UP-TO-DATE
        :CordovaLib:processReleaseManifest UP-TO-DATE
        :CordovaLib:generateReleaseBuildConfig UP-TO-DATE
        :CordovaLib:processReleaseResources UP-TO-DATE
        :CordovaLib:generateReleaseSources UP-TO-DATE
        :CordovaLib:incrementalReleaseJavaCompilationSafeguard UP-TO-DATE
        :CordovaLib:compileReleaseJavaWithJavac UP-TO-DATE
        :CordovaLib:processReleaseJavaRes UP-TO-DATE
        :CordovaLib:transformResourcesWithMergeJavaResForRelease UP-TO-DATE
        :CordovaLib:transformClassesAndResourcesWithSyncLibJarsForRelease UP-TO-DATE
        :CordovaLib:mergeReleaseJniLibFolders UP-TO-DATE
        :CordovaLib:transformNative_libsWithMergeJniLibsForRelease UP-TO-DATE
        :CordovaLib:transformNative_libsWithSyncJniLibsForRelease UP-TO-DATE
        :CordovaLib:bundleRelease UP-TO-DATE
        :prepareComAndroidSupportSupportCompat2600Alpha1Library UP-TO-DATE
        :prepareComAndroidSupportSupportCoreUi2600Alpha1Library UP-TO-DATE
        :prepareComAndroidSupportSupportCoreUtils2600Alpha1Library UP-TO-DATE
        :prepareComAndroidSupportSupportFragment2600Alpha1Library UP-TO-DATE
        :prepareComAndroidSupportSupportMediaCompat2600Alpha1Library UP-TO-DATE
        :prepareComAndroidSupportSupportV42600Alpha1Library UP-TO-DATE
        :prepareOrgApacheCordovaCordovaLib623ReleaseLibrary UP-TO-DATE
        :prepareReleaseDependencies
        :compileReleaseAidl UP-TO-DATE
        :compileReleaseRenderscript UP-TO-DATE
        :generateReleaseBuildConfig UP-TO-DATE
        :generateReleaseResValues UP-TO-DATE
        :generateReleaseResources UP-TO-DATE
        :mergeReleaseResources
        /home/michi/software/workspace/web-client-ionic/platforms/android/res/drawableXXX/icon.png:1:1: Error: Content
        ist nicht zulässig in Prolog.
        :mergeReleaseResources FAILED

        FAILURE: Build failed with an exception.

        * What went wrong:
        Execution failed for task ':mergeReleaseResources'.
        > /home/michi/software/workspace/web-client-ionic/platforms/android/res/drawableXXX/icon.png:1:1: Error: Content
        ist nicht zulässig in Prolog.

        * Try:
        Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

        BUILD FAILED

        Total time: 1.095 secs
        Error: /home/michi/software/workspace/web-client-ionic/platforms/android/gradlew: Command failed with exit code
        1 Error output:
        FAILURE: Build failed with an exception.

        * What went wrong:
        Execution failed for task ':mergeReleaseResources'.
        > /home/michi/software/workspace/web-client-ionic/platforms/android/res/drawableXXX/icon.png:1:1: Error: Content
        ist nicht zulässig in Prolog.

        * Try:
        Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
....
....
....
....
....

My environment:
cli packages: (/usr/lib/node_modules)
    @ionic/cli-utils  : 1.12.0
    ionic (Ionic CLI) : 3.12.0

global packages:
    cordova (Cordova CLI) : 7.0.1

local packages:
    @ionic/app-scripts : 3.0.0
    Cordova Platforms  : android 6.2.3
    Ionic Framework    : ionic-angular 3.7.0

System:
    Android SDK Tools : 26.0.2
    Node              : v6.11.3
    npm               : 5.4.2
    OS                : Linux 4.10

Misc:
    backend : legacy

If you're doing a --release build, you should also pass --prod for an optimized build. This seems like a cordova issue, so we'll move this issue to our CLI repo.
Thanks,
Dan

--prod --release has the same issue. We always do this, but we tested on which condition we have the issue and it appear only on --release parameter.
We had this error only from today on updating app-scripts to latest, for that reason I opened the issue here.
But now I found the real issue!!!!
/drawableXXX/ isn't a valid folder for cordova!
Sorry for wasting your time!!


New example: adversarial attacks on small CNN

Hello,
This is my first commit to TinyGrad: a new entry to the list of example applications. The code:

Trains a simple CNN network on MNIST
Generates a small batch (128 test set elements) of adversarially-attacked* inputs

* Adversarial attacks are purposefully-crafted perturbations to a neural network's inputs such that they are misclassified.
The example fixes the weights of a pre-trained model and generates a buffer between the inputs' Tensor and the model. The buffer will accumulate adversarial perturbations by computing the additive inverse of the gradients of each input image's pixels with respect to the network's output loss (hill climbing exercise).
Implemented attacks: L2 and Linf FGSM, R+FGSM, PGD attacks [1]
A stand-alone implementation of this example, tested on the current master version of Tinygrad, can be found here along with output image examples.
[1] Adversarial Attacks and Defences: A Survey

This should be something in another repo and not in the main tinygrad repo.


PUB-1533 - Cancelled password reset

JIRA link
https://tools.hmcts.net/jira/browse/PUB-1533
Change description
Added cancelled password reset page

Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x] No

Code looks good. I think we need to update the password reset URLs in flux config that we send with notify to reflect this new change.

Good spot! I've added PR's for sds flux + pub services


Update Wally

Add Wally to Foreman
Add Include and Exclude Folder to reduce size.

I also recommend publishing the latest version as a beta on Wally. wally publish

Wally ignores any Excludes if Includes is specified. If Includes is specified, everything not in it is automatically excluded.

Wally ignores any exclude if include is specified. If include is specified, everything not in it is automatically excluded.

Now I removed the exclude tag, but it would not matter, since the exclude tag said that he should exclude everything.


Improved testing in development workspace

It's clear to contributors how to add tests to the development workspace. Immediate goal here is not high test coverage, just getting the systems in place in the development workspace.
Tests should include at least one example of:

[ ] Unit testing supporting modules
[ ] Unit testing an internal API
[x] Browser testing components and/or a page in the dev workspace.

Playwright for browser testing, and Vitetest for unit testing seem popular in other Svelte projects, but no strong preference here.
Tests should run in CI/CD along with existing suite.
Additional considerations

We should have a mechanism for tagging a subset of tests to run with different database connections (different environment variables set in the test runner)

I will be looking 👀  into the unit testing & integration test next week.

We will have to wait on the SvelteKit upgrade to the new unit tests as  Vitest requires Vite >=v3.0.0
(https://vitest.dev/guide)

We should cover [ ] Best practices locating elements under browser testing components


Data Foundation tab should not be blocked on StorageCluster presence

Before:

After:

/approve
/lgtm

/cherrypick release-4.11

/cherrypick release-4.11-compatibility


Crash loop when deleting only profile of a game

What are you experiencing an issue with?

[x] Aurora
[ ] Aurora supported game, (Fill in the game)
[ ] Aurora Wrapper
[x] Aurora Profile
[ ] Aurora supported device, (Fill in problematic devices)
[ ] Aurora Scripts
[ ] Aurora Updater
[ ] Aurora Lighting issue
[ ] Other, (Fill in another issue here)

What kind of an issue is it?

[x] Crash
[ ] Performance problems
[ ] Visual bug
[ ] Localization issue
[ ] Enhancement suggestion
[ ] Device issue
[ ] Other, (Fill in another type of issue here)

Expected behavior:
Deleting aurora profile shouldn't cause a crash
Actual behavior:
Crash and unable to reopen without crash
How often does this problem occur?
Every time
Please describe the steps to reproduce this issue:

Delete only profile for a game (I did CSGO in particular)

What is the version of Aurora you are experiencing this issue with?
Newest as of writing this (can't check since I can't open)
Does the previous version of Aurora have this issue?
Unknown
Please include any logs, screenshots, or files that are related to this issue:
https://pastebin.com/rCNM0Yk0

You shouldn't be able to delete the last profile as at least one needs to exist for stuff to work. Try deleting %appdata%/Aurora/Profiles/csgo, that should recreate everything for that and set it back to default. I'll make it so it doesn't allow you to delete the last profile, and if you delete it manually it will just recreate it.

Thanks for the temporary solution. The long term solution also sounds solid.

Fixed with above commit


.desktop file doesn't have the MimeType line

At least some supported file formats do have MIME types. See this for example.

duplicate of #58


test-pod fails to mount rdb disk

following setup as described - everything looks fine - and it gets an active volumeclaim (pvc state = Bound) - but test-pod fails with
  Warning  FailedMount             1m (x10 over 21m)  kubelet, p-boa-kube-cbw01.example.org  Unable to mount volumes for pod "test-pod_default(02345395-ef1e-11e8-9a42-005056af66a5)": timeout expired waiting for volumes to attach or mount for pod "default"/"test-pod". list of unmounted volumes=[pvc]. list of unattached volumes=[pvc default-token-p6kmb]
  Warning  FailedMount             1m (x19 over 23m)  kubelet, p-boa-kube-cbw01.example.org  MountVolume.WaitForAttach failed for volume "pvc-0e15d135-ef19-11e8-9a42-005056af66a5" : fail to check rbd image status with: (executable file not found in $PATH), rbd output: ()

any hints on how to debug that? It seems the command to "check rbd image status" is missing?
Running rbd ls kube on ceph cluster looks fine:
 rbd  --cluster t-boa-cephA ls kube
kubernetes-dynamic-pvc-0e1dc2ae-ef19-11e8-be6e-064df804900d

ahh.. I installed ceph-common on alle kubernetes nodes - and voila :)
Perhaps the code should check for dependencies such as the rdb binary being there - and report more precisely whats missing :)


feat: implement pattern matching to program

Description
This PR is on charge of implementing **Pattern Matching ** functionality to the main program that parses yamil code.
Type of Change
Put an x in all the boxes that apply:

[x] ✨ New Feature (non-breaking change which adds functionality)
[ ] 🛠 Bug Fix (non-breaking change which fixes an issue)
[ ] ❌ Breaking Change (fix or feature that would cause existing functionality to change)
[ ] 🧹 Code Refactor
[ ] ✅ Build Configuration Change
[ ] 📝 Documentation
[ ] 🗑 Chore

Pull Request Checklist
Put an x in all the boxes that apply:

[x] My commit messages are detailed.
[x] My code follows the code style of this project.
[ ] No existing features have been broken without good reason.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests pass.

Add the end of the fix please do it a squash since most commits contain small changes

Okay! I'll do it after I gain your review.

Add the end of the fix please do it a squash since most commits contain small changes

Okay! I'll do it after I gain your approve again.


Fix getting AZ from subnet id to check mismatch

QE figured out that this wasn't actually doing what we expected. This fix makes sure we are able to compare these 2 values and if mismatched, log a message informing the user of such event.

/retest

/retest


[doc/pwrmgr] Fix incomplete functional spec

The pwrmgr ip functional spec needs to mention CSR "escalate_reset_status".
Note this CSR is present in the autogen pwrmgr.hjson file, but not in the
corresponding ip pwrmgr.hjson file.

i believe this has already been added.


Remove getSpaces

@harshmaur @bonustrack please approve if there isn't any more issues

Has to be approved by @bonustrack


orbitdb version 0.20.0-rc3 event issues in React

@haadcode @aphelionz  I have just used this new verison but still problems are same. Even  orbitdb.open function is also not working  ".open is not a function".
Other events like
db.events.on('ready', (dbname, heads) => ... )

db.events.on('replicated', (address) => ... )

These events are also not working and there is no data sync between peers .
I just want to know when you guys will release stable version for react ? Your feedbacks will be highly appreciated

Hi @AqeelKazmi, might there be a repo for your react app that I can load locally to reproduce your error in place and help debug?

Thank you @aphelionz . There is no problem in sharing my repo and sure i will share it, but please let me know about the availability of stable version for react? I just noticed even  orbitdb.keyvalue is also having problems.
So for the time i just want to make sure when you guys will release the stable version for react?

@AqeelKazmi thanks for reporting the issues and helping us to test the upcoming release! 👍❤
Based on the issue description, it sounds like something went wrong while creating the db instance (thus the error "open is not a function"). The events not being emitted may be due to some connectivity problem. Not knowing how exactly your code is setup, it's hard to provide the right answers as the reasons for the error may be various. It would help us to find the answer if we'd see the full code, or at least the parts where OrbitDB (and IPFS) are setup/create/initialized.
Rest assured everyone's working hard to land the new release as soon as possible, and opening issues and reporting problems are extremely valuable to reach a stable release. Thank you for doing so 🙏 If you're in urgent need of a stable version, the latest release on npm, 0.19.9 would be it.

Actually 0.19. 9 was not working with react app. Therefore adopted unstable versions.
Anyways bundle of thanks for your replies and again I will try to make things work if not then obviously i will share repo with you guys 👍

@haadcode i have tested all versions from 0.19.9 to rc3. Following are the results
version                                                Issues
"orbit-db": "0.19.9":                     Arguments to path.join must be strings
"orbit-db": "0.20.0-rc.1":              orbitdb.keyvalue is not a function
"orbit-db": "0.20.0-rc.1.1":           orbitdb.keyvalue is not a function
"orbit-db": "0.20.0-rc2":               orbitdb.keyvalue is not a function
"orbit-db": "0.20.0-rc3":               orbitdb.keyvalue is not a function

@haadcode @aphelionz  i have add you guys in my react based repo, that has only the connection code for ipfs and orbitdb, Which i have tested and above mentioned issues i faced for each and every version i used...I dont know but why it is not even creating database now?

Hi @AqeelKazmi Just wanted to check in here. We've done a lot of work with performance and reliability and now have an RC for version 0.22 on next, if you want to try and are still working on this app.

Closing due to inactivity.


Add new Endpoints and refactor some duplicate lines.

Hi @meshfields,
I've added two new endpoints /namespaces and /{NAMESPACE}/deployments.
I could also fix the hardcoded default namespace to use the requested namespace parameter instead.
Please take a look and let me know, if you see any issues.
Thanks!
Marcel

i've added an commit to fix the merge conflict.

Looking good. Thanks!

me noob

Me noob too. My fork was behind your repo. Could fix that, but npm test fails. I'll try to align my code.


Hygieia 2.0

Merge changes to upstream for Sprint 1 of Hygieia product/pipeline dashboard.

@jayhogan , please be aware that i am also refactoring for code quality on some existing collectors.  Let me know offline or on here if you expect to have any package collisions that we can avert.  You can check my most recent pull request here:  https://github.com/capitalone/Hygieia/pull/284


GM Vision and roofs

related to https://github.com/foundryvtt/foundryvtt/issues/11032
With GM Vision active, Overhead tiles that use vision based occlusion don't fade when a token is selected and the GM has to hover over the tile to see the tokens in a room.
I hope that a Core Foundry change will fix this incidentally, but it's best to raise it here too in case we can get an earlier fix for the whole behavior or if GM Vision needs special handling even after core vision gets changed.

based on a quick test in v11 tokens with vision used to actually use vision type occlusion for this roof even with GM vision active, so they were different from visionless tokens in that regard. So I'd say it's an actual regression in the system/not caught up with the core changes.


Bug/measure download protocol

Hi Mick,  John and Katrin noticed an issue where the 'action' was returning as 'undefined' for download links.  I think the couple of updates below might resolve it?
Cheers,
S

@sarah-storm Thank you for picking this up.
I've made a small change to add the category to the documentType Object that has been extracted from the settings Object, rather than taking a copy of the settings Object and adding an action to it. I hope you don't mind.
This module is still sitting on this branch and unpublished because we don't have sufficient test coverage (or documentation) but it has escaped into the wild on a few projects so we should prioritise releasing it.


PackfileMaintenanceStep should dynamically resize the batch size

The PackfileMaintenanceStep expires prefetch packs where every object appears in a new pack. It then condenses many packs into one pack using git multi-pack-index repack --batch-size=<size> where "size" is currently "2gb".
We cannot make "2gb" much larger due to issues with unsigned long on Windows. This also limits the amount of work we would do in one step. With this batch size, we anticipate the users with ~100 GB of prefetch data will get to a steady state in 30-60 runs of the step.
Once in a steady state, we anticipate that it will take a long time before the repack command has enough data for a full batch of data.
We can predict if the batch size is too large by examining the pack directory and counting the number of files smaller than 2gb. We could even predict a size that would group all packs into a single pack by adding their sizes and subtracting 1. If this size is larger than 2 * 1024 * 1024 * 1024, then we can stick with the default.

This became a bit more important now that we updated how git multi-pack-index repack works:
We select a pack for repacking based on its "expected output size" which is calculated by counting the number of objects the multi-pack-index references in that pack, dividing by the total number of objects in the pack, and multiplying by the pack size:
(pack size) * (num referenced objects)
----------------------------------------- = (expected output size)
         (total objects in pack)

Using this as the measurement for the batch size, the pack we write should be closer in size to our batch size than before. This allows the expire command to delete more data. In particular, if someone had two "big" packs because they did not prefetch for 30+ days, the old method would never repack the super-old big pack if there was at least one object in that pack and no other pack. This calculation allows repacking that pack.
The issue is that now we need much more than 2gb of "input" data to create a full "batch".
I'll start working on a way to leave a reminder that says "we did not meet the batch size last time, please lower the batch size for next time" and maybe we drop the batch size as follows: { "2g" , "512m", "128m" }.

I talked with @jeffhostetler about this a little and thought of a different approach.
If we simply stop checking that we have a "full" batch, then (in the "steady state") we would repack all of the small packs every day and keep the pack count low. It has an extra "cost" that our step will do something almost every time it runs, but that is already happening with the expire step.
Before jumping into implementation, we should watch the current behavior more and see what we can learn about this steady state.


Update CHANGELOG.md

Changes in this pull request
For the sake of completion I added commit/pr numbers to the rest of the changelog
Finishes https://github.com/Instagram/IGListKit/issues/278
Pull request checklist

[x] All tests pass. Demo project builds and runs.
[x] I added tests, an experiment, or detailed why my change isn't tested.
[x] I have reviewed the [contributing guide]
(https://github.com/Instagram/IGListKit/blob/master/.github/CONTRIBUTING.md)
[x] Have a real need to complete minor things

I dig it! @jessesquires?

🎉 💯  thanks @Sherlouk !

Coverage remained the same at 97.268% when pulling 393eadc8231a5ee8219939044af938b796e0988e on Sherlouk:patch-1 into 4aae4edbf89c80c5e624ce2d66306dba59a5e48f on Instagram:master.

Looks like this merged, but the bot screwed up somehow. I think we can close since https://github.com/bmeverett/IGListKit/commit/ea98e1b41ceb62d57509f73704bed53e38a97be2 is in the commit history.


Different Node.js version

Summary of Problem
When using Electron 5 with Serialport 7.1.5 I am getting the following error.
I have a postinstall task that runs electron-builder install-app-deps.
As far as I know, this should re-compile the native dependencies against the appropriate Electron version.
When using Electron 4, it's working without any issues.
App threw an error during load
Error: The module<EMAIL_ADDRESS>was compiled against a different Node.js version using
NODE_MODULE_VERSION 47. This version of Node.js requires
NODE_MODULE_VERSION 70. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).
    at process.func [as dlopen] (electron/js2c/asar.js:155:31)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:850:18)
    at Object.func [as .node] (electron/js2c/asar.js:155:31)
    at Module.load (internal/modules/cjs/loader.js:677:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:609:12)
    at Function.Module._load (internal/modules/cjs/loader.js:601:3)
    at Module.require (internal/modules/cjs/loader.js:715:19)
    at require (internal/modules/cjs/helpers.js:14:16)
    at bindings (/Users/Client/node_modules/bindings/bindings.js:112:48)
    at Object.<anonymous> (/Users/Client/node_modules/@serialport/bindings/lib/darwin.js:1:36)

I am not sure, if this is a problem of Serialport or Electron or Electron-Bulider?
Versions, Operating System and Hardware

SerialPort@? v7.1.5
Node.js v? v12.4.0
Electron v5.0.4
Mac

@phhoef  try this :
https://github.com/song940/node-escpos/issues/214#issuecomment-502806807
It worked for me, but gave me a strange behavior after.

Thanks for your reply - but your answer also confuses me.
Ad far as I know, running electron-builder install-app-depsˋ should completely replace the ˋelectron-rebuildˋ.
Anyhow, I tried running ˋelectron-rebuildˋ, but with no success.

I'm experiencing the same issue.  Did older versions work?  I'm actually doing both because I didn't know what was necessary:
    "postinstall": "yarn rebuild && electron-builder install-app-deps",
    "rebuild": "electron-rebuild -f -o node-hid && electron-rebuild -f -o serialport",

Yes, my configuration is working, when using electron 4.
as mentioned, I have a postinstall task for rebuilding the native dependencies: electron-builder install-app-deps
I‘ll try your rebuild task, maybe this is working for me ...

The rebuild worked for me using node-hid but doesn't seem to work for this library:

Same here, rebuild does not work for serialport. :-(

I also encountered the same problem, using the latest version of electronic (5.0.4) node (12.4.0), serialport (7.1.5), and electronic back to 4.2.4.

MedAmineBenyettou comment solution worked for me. Rebuild worked for Zigbee.

./node_modules/.bin/electron-rebuild appended at the run script worked for me.

I also encountered this problem, it still doesn't work in all solution above.

the same problem

I also encountered the similar issue when I want to create a vscode plugin with serialport.
My env:
windows 10 x64
serialport: 1.37.0
nodejs: 12.7.0
Activating extension undefined_publisher.helloworld failed:  The module<EMAIL_ADDRESS>was compiled against a different Node.js version using
NODE_MODULE_VERSION 72. This version of Node.js requires
NODE_MODULE_VERSION 69. Please try re-compiling or re-installing
the module (for instance, using npm rebuild or npm install).
If I change the nodejs version to 10.11.0, also has this issue. But the error information is different:
Activating extension undefined_publisher.helloworld failed:  The module<EMAIL_ADDRESS>was compiled against a different Node.js version using
NODE_MODULE_VERSION 64. This version of Node.js requires
NODE_MODULE_VERSION 69. Please try re-compiling or re-installing
the module (for instance, using npm rebuild or npm install).

Hi guys, I have fixed this issue: use serialport in vscode plugin.
steps:

install electron-rebuild
npm install --save-dev electron-rebuild

install serialport
npm install serialport

rebuild
./node_modules/.bin/electron-rebuild --version 4.2.7
NOTE: 4.2.7 is electron version, perhaps you need to change for different vscode version.


Update graph template content for TSTool 13.

The handling of graph templates needs to be updated for TSTool 13 user files folder structure, changing from .tstool/template-graph to .tstool/13/template-graphs.  The change to plural is consistent with other files such as datastores and plugins.

I updated the documentation.  Close the issue.


[MB-3553] New office user unauthorized on first callback from login.gov

Description
This PR attempts to fix the longstanding bug where when creating an office user from the admin interface would result in an unauthorized error coming back from login.gov on first sign in.  The admin made it seem like you could assign roles but this was silently failing because the underlying user record was never created until the login.gov account was created so the office user would exist with no roles.
I've changed the flow to drop the not null requirement on the login_gov_uuid column allowing us to create a users table record when we first create the office_users record from the admin.  This allows the roles to be saved and assigned properly.  Now on first login to login.gov we now perform an update to fill in the login_gov_uuid value instead of needing to create the whole user record.
I added the roles checkbox group to the office user creation screen instead of having to save and then go back and update the record for easier use.  Or if we wanted to use the create api in the future for adding bulk records creation it won't require multiple queries.
Finally there is a migration that looks for office_users and admin_users that have never logged in. We'll create users table records for them so if they ever do log in there will be a user record that can be updated with the login.gov id.
Reviewer Notes
I probably could have been more thorough in handling admin error cases, we haven't been good about displaying informative error messages there.
There is probably more cleanup to be done here but I was trying to keep this PR narrow and then leave some refactoring later.
One thing that occurred to me while making these changes is we don't handle if a login.gov user changes their email (or has multiple emails) at some point and we'll need to update that value in our records.
Setup
Add any steps or code to run in this section to help others prepare to run your code:
make db_dev_e2e_populate
make server_run
make admin_client_run

Login as an admin user (devlocal is fine) to adminlocal:3000
Click on the Office Users tab
Click the + Create button to make a new office user
Fill in values for all of the office user fields (with an email<EMAIL_ADDRESS>including the roles and save the record
Logout of the admin and go to officelocal:3000 and choose sign in with login.gov
Create a new login.gov account with the email used previously<EMAIL_ADDRESS>On completion you should be brought back to the office app and redirected depending on your role to PPM/TOO/TIO queue instead of seeing an unauthorized error
In the database the user record should have been updated with the login_gov_uuid value as well.

These changes affect admin user creation as well.
Code Review Verification Steps

[ ] If the change is risky, it has been tested in experimental before merging.
[ ] Code follows the guidelines for Logging
[ ] The requirements listed in
Querying the Database Safely
have been satisfied.
Any new migrations/schema changes:

[ ] Follow our guidelines for zero-downtime deploys (see Zero-Downtime Deploys)
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] User facing changes have been reviewed by design.
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

References

Jira story for this change

Screenshots

Warnings

:warning:
This PR does not include changes to unit tests, even though it affects app code.

:warning:

Files located in legacy directories (src/shared or src/scenes) have
been edited. Are you sure you don’t want to also relocate them to the new file structure?
View the frontend file org ADR for more information

Messages

:book:
:link: MB-3553

  Generated by :no_entry_sign: dangerJS against c587436e0be2f38841ce271ae8ea509940ad8f04

I noticed that when we create a new admin user via devlocal, it automatically gets a login_gov_uuid, and I verified that when I signed in with that admin user via login.gov, my login_gov_uuid was updated to the one from login.gov. Is there a reason why we wouldn't do the same for office users? That would be better than removing the NOT NULL constraint.
Also, not related to this PR, but I noticed we're not enforcing uniqueness of the user email. I wonder why that is. Seems like we should.

I noticed that when we create a new admin user via devlocal, it automatically gets a login_gov_uuid, and I verified that when I signed in with that admin user via login.gov, my login_gov_uuid was updated to the one from login.gov. Is there a reason why we wouldn't do the same for office users? That would be better than removing the NOT NULL constraint.
Also, not related to this PR, but I noticed we're not enforcing uniqueness of the user email. I wonder why that is. Seems like we should.

Right, a devlocal user is supposed to be one that never logs in from login.gov though.  A new admin user doesn't get a login_gov_uuid though if you create it through the admin ui just like an office user.  I don't think we ever update a login_gov_uuid except on initial account creation when authorizeUnknownUser gets called when the openID doesn't have a database match but does have a matching email.  After that it will match the openID with the login_gov_uuid and call authorizeKnownUser instead.

Right. What I'm saying asking is why can't we create a random login_gov_uuid as part of creating the user for the admin user or office user, and then when they sign in for the first time, it will automatically update it to the correct value from login.gov.

Right. What I'm asking is why can't we create a random login_gov_uuid as part of creating the user for the admin user or office user, and then when they sign in for the first time, it will automatically update it to the correct value from login.gov.

@monfresh Gotcha, I think that would fundamentally alter the core logic which uses the login_gov_uuid search to know if the user exists in the system.  If that value is populated with a dummy value we'll have no way of knowing if they've ever logged into the app.  The FetchUserIdentity function in the user model is doing most of the work here.  We would have to then switch to matching the email and then checking to see if the login_gov_uuid differed on every sign in instead of just the first one.  I mentioned this in the reviewer notes too that in the future you should be able to change your login.gov email and that uuid will be stable so we can discover that change.  We don't want to start relying heavily on that email.

I realized I have to fix this for Service Member account creation now 🤦

@duncan-truss Here is the way I understand the flow:

New office user is created in the admin interface, along with a corresponding user in the users table
office user signs in via login.gov
We check the users table for an entry that matches the UUID provided by login.gov
If we don't find that UUID, then that means this is the first time they are signing in via login.gov, and so we call authorizeUnknownUser
We look up the office user by the email in the session (which comes from login.gov), and get the corresponding user
We update the login_gov_uuid field in the users table to match the one provided by login.gov
Going forward, they will always have the same login.gov UUID, even if they change their email, so from now on, they are known users

What about this current flow prevents us from setting a dummy UUID in step 1, since it will get updated with the correct UUID in step 6?

@duncan-truss Here is the way I understand the flow:
1. New office user is created in the admin interface, along with a corresponding user in the users table

2. office user signs in via login.gov

3. We check the users table for an entry that matches the UUID provided by login.gov

4. If we don't find that UUID, then that means this is the first time they are signing in via login.gov, and so we call `authorizeUnknownUser`

5. We look up the office user by the email in the session (which comes from login.gov), and get the corresponding user

6. We update the `login_gov_uuid` field in the users table to match the one provided by login.gov

7. Going forward, they will always have the same login.gov UUID, even if they change their email, so from now on, they are known users

What about this current flow prevents us from setting a dummy UUID in step 1, since it will get updated with the correct UUID in step 6?

@monfresh What you outline matches my understanding.  I was wrong to say that it would require changing to use emails.  I stand by my point that having that value be null allows us to know who has never signed in and filling it in obscures it's purpose.  If you just want to make the test factories simpler I think we can do that.

In what context do we need to know if a user has every signed in? Is it for informational or auditing purposes? As far as I can tell, it doesn't affect the authentication flow. If a field is meant to be required, it should have a DB constraint. That's my main issue. I don't think we should lose that safety if there is a way around that. If we need to be able to find out if a user has ever signed in, there are other ways to do that, such as adding a last_signed_in_at field. This is actually a STIG requirement that says users need to be able to see for themselves when they last signed in, but because the login.gov profile provides that information, we think that might be enough to satisfy that requirement.

In what context do we need to know if a user has ever signed in? Is it for informational or auditing purposes? As far as I can tell, it doesn't affect the authentication flow. If a field is meant to be required, it should have a DB constraint. That's my main issue. I don't think we should lose that safety if there is a way around that. If we need to be able to find out if a user has ever signed in, there are other ways to do that, such as adding a last_signed_in_at field. This is actually a STIG requirement that says users need to be able to see for themselves when they last signed in, but because the login.gov profile provides that information, we think that might be enough to satisfy that requirement.

@monfresh For something that gets audited like user records, I don't think having garbage data in that uuid field is the right route and isn't inherently more safe than a null.  We had it not null before and this is the state we got to of silent errors. I think this reflects the business logic more explicitly whereas having that field with a nonexistent value will lead to confusion.  I am all for adding first class tracking of last signed in and other features.

How about instead of leaving it null, we populate it with a zero UUID? Would that be a good compromise? That would preserve the data integrity knowing that we would never end up with an empty UUID in the database, and at the same time, it makes it easy to differentiate between real UUIDs and placeholders ones for users that were newly created via the admin interface.

How about instead of a random UUID, we populate it with a zero UUID? Would that be a good compromise? That would preserve the data integrity knowing that we would never end up with an empty UUID in the database, and at the same time, it makes it easy to differentiate between real UUIDs and placeholder ones for users that were newly created via the admin interface.

That sounds like we're reinventing NULL but now can't add a uniqueness constraint if we want.

Oh yeah! I didn't think about that! I think it helped to talk it through in Slack, so we should be fine with your approach.