Language Server Protocol is not forgetting old errors?

It seems that the LSP implementation is reporting outdated diagnostics. For example: this code snippet generates a warning about a redundant do keyword.

If I backspace once and save, leaving it with just the d from the do, I now have a syntax error from ghc-mod, and the previous warning.

Finally, if I delete the last part of the keyword, which should fix both messages, they both remain.

This was tested with this very simple VSCode extension:
'use strict';

import * as path from 'path';

import { workspace, Disposable, ExtensionContext } from 'vscode';
import { LanguageClient, LanguageClientOptions, SettingMonitor, ServerOptions, TransportKind } from 'vscode-languageclient';

export function activate(context: ExtensionContext) {
	let serverOptions: ServerOptions = { command: "nix-shell", args: ["--run", "hie --lsp"], options: { cwd: workspace.rootPath } }
	console.log("test");
	let clientOptions: LanguageClientOptions = { documentSelector: ['haskell'] }
	let disposable = new LanguageClient('languageServerExample', 'Language Server Example', serverOptions, clientOptions).start();
	context.subscriptions.push(disposable);
}

export function deactivate() {
}

And in the root directory of the project, my shell.nix adds hie to the path via this expression, but rest assured that this is the same as having hie installed on my system and having the VSCode extension simply use hie --lsp without nix-shell.

FWIW, I think LSP should be the primary protocol going forward. It's going to be much, much easier to integrate with arbitrary editors if we use this common protocol. I was able to implement an Atom version of that VSCode extension in a similarly small amount of code due to the LSP. And of course, the Atom implementation displayed the same issue.

I agree on it being the primary protocol. We currently have one user (that I am aware of) on the JSON/stdio protocol, so will support that too, but I think the feature set is going to be driven by LSP.
In terms of this problem, there should be a log file in /tmp/hie-logs giving the LSP message interaction. Can you post it somewhere for when you do the sequence you describe.

Also, the engine and languageclient versions are quite important, see that you match the ones in https://github.com/alanz/vscode-hie-server/blob/master/package.json

hie.log
2017 Jun-03 08:13:06.827974 [INFO] Setting home directory:/Users/willfancher
2017 Jun-03 08:13:06.828471 [INFO] run entered for HIE Version <IP_ADDRESS> x86_64
2017 Jun-03 08:13:06.828667 [INFO] Current directory:/Users/willfancher
2017 Jun-03 08:13:06.852959 [DEBUG] run:top of loop
2017 Jun-03 08:13:06.853073 [DEBUG] main loop:got:CReq {cinPlugin = "applyrefact", cinReqId = 0, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:06.853336 [DEBUG] doDispatch:desc=PluginDescriptor ApplyRefactapply-refact applies refactorings specified by the refact package. It is currently integrated into hlint to enable the automatic application of suggestions.[(Command CommandDesc {cmdName = "applyOne", cmdUiDescription = "Apply a single hint", cmdFileExtensions = [".hs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "applyAll", cmdUiDescription = "Apply all hints to the file", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Run hlint on the file to generate hints", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "FileDiagnostics", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:06.853672 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:07.012021 [INFO] lint:res=Right [/Users/willfancher/Documents/programming/test/src/Main.hs:4:8: Warning: Redundant do
Found:
  do putStrLn "hello world"
Why not:
  putStrLn "hello world"
]
2017 Jun-03 08:13:07.023491 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "applyrefact", cinReqId = 0, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("uri",String "file:///Users/willfancher/Documents/programming/test/src/Main.hs"),("diagnostics",Array [Object (fromList [("severity",Number 2.0),("range",Object (fromList [("start",Object (fromList [("line",Number 3.0),("character",Number 7.0)])),("end",Object (fromList [("line",Number 4.0),("character",Number 24.0)]))])),("source",String "hlint"),("message",String "Redundant do\nFound:\n  do putStrLn \"hello world\"\nWhy not:\n  putStrLn \"hello world\"\n")])])]))
2017 Jun-03 08:13:07.023691 [DEBUG] run:top of loop
2017 Jun-03 08:13:07.024168 [DEBUG] main loop:got:CReq {cinPlugin = "ghcmod", cinReqId = 1, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:07.024269 [DEBUG] doDispatch:desc=PluginDescriptor ghc-modghc-mod is a backend program to enrich Haskell programming in editors. It strives to offer most of the features one has come to expect from modern IDEs in any editor.[(Command CommandDesc {cmdName = "check", cmdUiDescription = "check a file for GHC warnings and errors", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Check files using `hlint'", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "info", cmdUiDescription = "Look up an identifier in the context of FILE (like ghci's `:info')", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [ParamDesc {pName = "expr", pHelp = "The EXPR to provide info on", pType = PtText, pRequired = Required}], cmdReturnType = "Text", cmdSave = SaveNone}),(Command CommandDesc {cmdName = "type", cmdUiDescription = "Get the type of the expression under (LINE,COL)", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "TypeInfo", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:07.02442 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:07.491702 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "ghcmod", cinReqId = 1, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("ok",String "")]))
2017 Jun-03 08:13:07.491889 [DEBUG] run:top of loop
2017 Jun-03 08:13:12.094196 [DEBUG] main loop:got:CReq {cinPlugin = "applyrefact", cinReqId = 2, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:12.09439 [DEBUG] doDispatch:desc=PluginDescriptor ApplyRefactapply-refact applies refactorings specified by the refact package. It is currently integrated into hlint to enable the automatic application of suggestions.[(Command CommandDesc {cmdName = "applyOne", cmdUiDescription = "Apply a single hint", cmdFileExtensions = [".hs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "applyAll", cmdUiDescription = "Apply all hints to the file", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Run hlint on the file to generate hints", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "FileDiagnostics", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:12.094575 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:12.137836 [INFO] lint:res=Right []
2017 Jun-03 08:13:12.145265 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "applyrefact", cinReqId = 2, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("uri",String "file:///Users/willfancher/Documents/programming/test/src/Main.hs"),("diagnostics",Array [])]))
2017 Jun-03 08:13:12.145389 [DEBUG] run:top of loop
2017 Jun-03 08:13:12.145569 [DEBUG] main loop:got:CReq {cinPlugin = "ghcmod", cinReqId = 3, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:12.145674 [DEBUG] doDispatch:desc=PluginDescriptor ghc-modghc-mod is a backend program to enrich Haskell programming in editors. It strives to offer most of the features one has come to expect from modern IDEs in any editor.[(Command CommandDesc {cmdName = "check", cmdUiDescription = "check a file for GHC warnings and errors", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Check files using `hlint'", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "info", cmdUiDescription = "Look up an identifier in the context of FILE (like ghci's `:info')", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [ParamDesc {pName = "expr", pHelp = "The EXPR to provide info on", pType = PtText, pRequired = Required}], cmdReturnType = "Text", cmdSave = SaveNone}),(Command CommandDesc {cmdName = "type", cmdUiDescription = "Get the type of the expression under (LINE,COL)", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "TypeInfo", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:12.145815 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:12.390169 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "ghcmod", cinReqId = 3, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("ok",String "src/Main.hs:4:8:Variable not in scope: d :: (String -> IO ()) -> [Char] -> IO ()\n")]))
2017 Jun-03 08:13:12.391429 [DEBUG] run:top of loop
2017 Jun-03 08:13:14.006068 [DEBUG] main loop:got:CReq {cinPlugin = "applyrefact", cinReqId = 4, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:14.006197 [DEBUG] doDispatch:desc=PluginDescriptor ApplyRefactapply-refact applies refactorings specified by the refact package. It is currently integrated into hlint to enable the automatic application of suggestions.[(Command CommandDesc {cmdName = "applyOne", cmdUiDescription = "Apply a single hint", cmdFileExtensions = [".hs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "applyAll", cmdUiDescription = "Apply all hints to the file", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "HieDiff", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Run hlint on the file to generate hints", cmdFileExtensions = [".hs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "FileDiagnostics", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:14.006323 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:14.050847 [INFO] lint:res=Right []
2017 Jun-03 08:13:14.058715 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "applyrefact", cinReqId = 4, cinReq = IdeRequest {ideCommand = "lint", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("uri",String "file:///Users/willfancher/Documents/programming/test/src/Main.hs"),("diagnostics",Array [])]))
2017 Jun-03 08:13:14.058857 [DEBUG] run:top of loop
2017 Jun-03 08:13:14.058921 [DEBUG] main loop:got:CReq {cinPlugin = "ghcmod", cinReqId = 5, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)}
2017 Jun-03 08:13:14.05924 [DEBUG] doDispatch:desc=PluginDescriptor ghc-modghc-mod is a backend program to enrich Haskell programming in editors. It strives to offer most of the features one has come to expect from modern IDEs in any editor.[(Command CommandDesc {cmdName = "check", cmdUiDescription = "check a file for GHC warnings and errors", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "lint", cmdUiDescription = "Check files using `hlint'", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [], cmdReturnType = "Text", cmdSave = SaveAll}),(Command CommandDesc {cmdName = "info", cmdUiDescription = "Look up an identifier in the context of FILE (like ghci's `:info')", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxFile], cmdAdditionalParams = [ParamDesc {pName = "expr", pHelp = "The EXPR to provide info on", pType = PtText, pRequired = Required}], cmdReturnType = "Text", cmdSave = SaveNone}),(Command CommandDesc {cmdName = "type", cmdUiDescription = "Get the type of the expression under (LINE,COL)", cmdFileExtensions = [".hs",".lhs"], cmdContexts = [CtxPoint], cmdAdditionalParams = [], cmdReturnType = "TypeInfo", cmdSave = SaveAll})] [] []
2017 Jun-03 08:13:14.059377 [DEBUG] doDispatch:req=IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}
2017 Jun-03 08:13:14.298084 [DEBUG] sendResponse (req,resp)=(CReq {cinPlugin = "ghcmod", cinReqId = 5, cinReq = IdeRequest {ideCommand = "check", ideParams = fromList [("file",ParamValP {unParamValP = ParamFile "/Users/willfancher/Documents/programming/test/src/Main.hs"})]}, cinReplyChan = (TChan ChannelResponse)},IdeResponseOk (fromList [("ok",String "")]))
2017 Jun-03 08:13:14.298632 [DEBUG] run:top of loop

hie-wire.log
Content-Length: 298

{"result":{"capabilities":{"textDocumentSync":2,"hoverProvider":true,"completionProvider":{"resolveProvider":true},"documentHighlightProvider":true,"codeActionProvider":true,"renameProvider":true,"executeCommandProvider":{"commands":["applyrefact:applyOne","hare:demote"]}}},"jsonrpc":"2.0","id":0}Content-Length: 36

{"result":[],"jsonrpc":"2.0","id":1}Content-Length: 371

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs","diagnostics":[{"severity":2,"range":{"start":{"line":3,"character":7},"end":{"line":4,"character":24}},"source":"hlint","message":"Redundant do\nFound:\n  do putStrLn \"hello world\"\nWhy not:\n  putStrLn \"hello world\"\n"}]}}Content-Length: 271

{"result":[{"arguments":[{"file":{"textDocument":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs"}}},{"start_pos":{"position":{"line":3,"character":7}}}],"title":"Apply hint:Redundant do","command":"applyrefact:applyOne"}],"jsonrpc":"2.0","id":2}Content-Length: 371

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs","diagnostics":[{"severity":2,"range":{"start":{"line":3,"character":7},"end":{"line":4,"character":24}},"source":"hlint","message":"Redundant do\nFound:\n  do putStrLn \"hello world\"\nWhy not:\n  putStrLn \"hello world\"\n"}]}}Content-Length: 271

{"result":[{"arguments":[{"file":{"textDocument":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs"}}},{"start_pos":{"position":{"line":3,"character":7}}}],"title":"Apply hint:Redundant do","command":"applyrefact:applyOne"}],"jsonrpc":"2.0","id":3}Content-Length: 557

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs","diagnostics":[{"severity":1,"range":{"start":{"line":3,"character":7},"end":{"line":3,"character":7}},"source":"ghcmod","message":"Variable not in scope: d :: (String -> IO ()) -> [Char] -> IO ()\n"},{"severity":2,"range":{"start":{"line":3,"character":7},"end":{"line":4,"character":24}},"source":"hlint","message":"Redundant do\nFound:\n  do putStrLn \"hello world\"\nWhy not:\n  putStrLn \"hello world\"\n"}]}}Content-Length: 271

{"result":[{"arguments":[{"file":{"textDocument":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs"}}},{"start_pos":{"position":{"line":3,"character":7}}}],"title":"Apply hint:Redundant do","command":"applyrefact:applyOne"}],"jsonrpc":"2.0","id":4}Content-Length: 557

{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///Users/willfancher/Documents/programming/test/src/Main.hs","diagnostics":[{"severity":1,"range":{"start":{"line":3,"character":7},"end":{"line":3,"character":7}},"source":"ghcmod","message":"Variable not in scope: d :: (String -> IO ()) -> [Char] -> IO ()\n"},{"severity":2,"range":{"start":{"line":3,"character":7},"end":{"line":4,"character":24}},"source":"hlint","message":"Redundant do\nFound:\n  do putStrLn \"hello world\"\nWhy not:\n  putStrLn \"hello world\"\n"}]}}Content-Length: 477

{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":1,"message":"haskell-lsp:parse error. Object (fromList [(\"jsonrpc\",String \"2.0\"),(\"method\",String \"shutdown\"),(\"id\",Number 5.0)]) \"When parsing the record RequestMessage of type Language.Haskell.LSP.TH.DataTypesJSON.RequestMessage the key params was not present.\" `stack update` and install new haskell-lsp. Or check information on https://marketplace.visualstudio.com/items?itemName=xxxxxxxxxxxxxxx"}}

vscode-languageclient v2.6.3. haskell-ide-engine is built from master: a4759cd3b8f3245cfafcf8beebada53273c12b77. Although I was using Nix instead of Stack, so it's possible the versions of hie's dependencies are breaking things. I should try it with Stack.

I have also managed to reproduce it locally.

Oh good, I don't have to wait for Stack to build dozens of dependencies =P

I put a mechanism in to cache the error reports from the different sources (hlint, ghc-mod) and send them together, as the client is spec'd to overwrite. Obviously it is buggy.

Ah. How significant is the performance cost of not caching?

Its more the problem of running an async backend, so all the reports are not available at the same time.
So rather send the diagnostics through as they come, than have the hlint appear then disappear when ghc-mod comes

So accumulation is a better word than caching

Ah. Seems like it needs to be transactional in a sense. I wonder if there's any clever way to use STM to solve this for free.

Well, it is simple enough, the LSP client does send a document version, so indexing is trivial. Smells like a bug to me.

And because there is different amounts of work being done, potentially, having the results come through as available is a good thing. IMHO.

And it seems haskell-lsp supports the version management, but it is not being used in HIE. Rectifying it.

https://github.com/haskell/haskell-ide-engine/pull/235 should sort this out, it no longer happens for me locally when using it.

Seems to fix it for me.

Great.  Will merge to master.

I was able to implement an Atom version of that VSCode extension in a similarly small amount of code due to the LSP. And of course, the Atom implementation displayed the same issue.

@ElvishJerricco would you happen to have that Atom implementation somewhere? I started on an initial attempt at setting up hie for atom-languageclient in https://github.com/Tehnix/atom-haskell-hie, but I don't won't to duplicate efforts here.

@Tehnix see https://github.com/atom-haskell/hie-lsp-adapter

@wz1000 thanks! Seems the only difference we had was very minor :)


[KYUUBI #2576][FOLLOWUP] Bump Hudi 0.11.0

Why are the changes needed?

Resolve WARNING prompt for IntelliJ IDEA Build Panel
Re-adjusted the order of several hudi moudles to make it consistent in the order of server and engine

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

IntelliJ IDEA Build Panel has this WARNING prompt after #2576 was merged.
Thanks for reporting the problem to me. @pan3793

The problem is introduced by hbase related dependencies.
In the research and debugging of IntelliJ IDEA RemoteMavenServer and maven-dependency-plugin, it is found that it is caused by specifying a range of dependent versions.
        <dependency>
            <groupId>org.glassfish</groupId>
            <artifactId>javax.el</artifactId>
            <version>[3.0.0,)</version>
        </dependency>

https://repo1.maven.org/maven2/org/glassfish/javax.el/3.0.0/javax.el-3.0.0.pom
(repository jvnet-nexus-release)
maven-resolver-provider first obtains the pom through 3.0.0, then obtains the repository described by the pom, and finally through the central repo and the repository described by the pom, summarizes the version list, and then obtains the latest version.
Because several versions corresponding to the repository described by pom (https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/maven-metadata.xml) do not exist, such a prompt appears.
REF: https://issues.apache.org/jira/browse/HBASE-18831

Thanks for digging the root cause, I verified the PR locally and it works!

Codecov Report

Merging #2659 (afb4f0a) into master (3187885) will decrease coverage by 0.08%.
The diff coverage is n/a.

:exclamation: Current head afb4f0a differs from pull request most recent head 3e86bc3. Consider uploading reports for the commit 3e86bc3 to get more accurate results

@@             Coverage Diff              @@
##             master    #2659      +/-   ##
============================================
- Coverage     64.37%   64.29%   -0.09%
  Complexity       80       80
============================================
  Files           385      385
  Lines         18628    18628
  Branches       2523     2523
============================================
- Hits          11992    11977      -15
- Misses         5499     5509      +10
- Partials       1137     1142       +5

Impacted Files
Coverage Δ

...ain/scala/org/apache/kyuubi/engine/EngineRef.scala
77.35% <0.00%> (-8.50%)
:arrow_down:

...rg/apache/kyuubi/engine/trino/TrinoSqlEngine.scala
52.63% <0.00%> (-7.02%)
:arrow_down:

...rg/apache/kyuubi/engine/trino/TrinoStatement.scala
66.66% <0.00%> (-2.30%)
:arrow_down:

...in/scala/org/apache/kyuubi/config/KyuubiConf.scala
96.62% <0.00%> (-0.12%)
:arrow_down:

.../org/apache/kyuubi/operation/KyuubiOperation.scala
69.33% <0.00%> (+1.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3187885...3e86bc3. Read the comment docs.

Thanks, merging to master


Fixing the README.md file

The last line for the license in README.md file has been corrected

Merging the 2 branches


Miscellaneous minor bug fixes

Fixed a number of minor bugs handling empty data, wrong variable names and partial dimension overlap.

Fixes look good and the tests are green. Ready to merge?

Yes let's merge.


Fix fuse shell logging

What changes are proposed in this pull request?
Fix the logging of fuse shell
Why are the changes needed?
Otherwise the first layer of the exception information will not be shown to users and users won't know the actual failure.
Does this PR introduce any user facing changes?
Improve fuse shell logging

@Xenorith For Alluxio 2.8, FYI

Error: 9.874 [ERROR] alluxio.client.cli.job.DistributedCommandsStatsTest.testAsyncPersistCancelStats  Time elapsed: 90.171 s  <<< ERROR!
java.util.concurrent.TimeoutException: Timed out waiting for job<PHONE_NUMBER>798 to be one of status [CANCELED] options: WaitForOptions{interval=20, timeout=45000} last value: false

alluxio-bot, merge this please

@Xenorith Please help cherry-pick to 2.8 please

alluxio-bot, cherry-pick this to branch-2.8 please


Received issue; j2net.runtime could not be found

Hi! I tried to integrate JDBC.NET to my .NET Core 6 application. However, after following the steps in Getting Started, I ran the sample codes and after hitting connection.Open(), I received the above error.
The full message would be "j2net.runtime could not be found. please check if the runtime is installed for your platform."
I have double-checked in the config as well and it seems fine.
Worth to take note is I upgraded the version from .NET Core 2.1 to 6 prior to integrating the code.
Thank you!

I noticed the issues always happens when I create .NET Core Web API project, but this issue didn't happen in console application

Originally, installing the J2NET Runtime Nuget package installs the Java runtime in the same path as the executable.
However, the Web Core API approach doesn't seem to properly read the relative path to the Java runtime.
We will check the issue. Thanks you!

Noted on that, thank you!
May I know if there's a workaround for this? Is it possible to copy the Java runtime manually to the same path as executable?

That's right. The logic to find the Java Runtime path is here.
First, check the execution path while running the Web API.
Then copy the contents of J2NET's "runtimes" folder to that execution path and it should work.

Thank you! We managed to execute connection.Open() method now after did the above steps. But, it seems we need to copy the .dll files as well apart from "runtimes" folder else it didn't work:

谢谢你！完成上述步骤后，我们现在设法执行了 connection.Open() 方法。但是，除了“运行时”文件夹之外，我们似乎还需要复制 .dll 文件，否则它不起作用：

I also encountered the same problem. I am grateful if you can provide a solution


Add support to sub comments in TimberComment & TimberPost

Hello ! This is my suggestion for #182
This add an attribute to TimberComment class (an array of his sub comments).

Coverage decreased (-0.08%) to 76.82% when pulling 5dc58135d789714a862511d9d66e4996fa5c2f79 on VincentLoy:master into 7f00115dae9e30a1da8230a623aadcc2b397705c on jarednova:master.

Coverage decreased (-0.08%) to 76.82% when pulling 5dc58135d789714a862511d9d66e4996fa5c2f79 on VincentLoy:master into 7f00115dae9e30a1da8230a623aadcc2b397705c on jarednova:master.

Coverage decreased (-0.08%) to 76.82% when pulling 5dc58135d789714a862511d9d66e4996fa5c2f79 on VincentLoy:master into 7f00115dae9e30a1da8230a623aadcc2b397705c on jarednova:master.

Thanks @VincentLoy -- I'm pulling this down to add some tests before mergin

Nice ! :)


Git Rootfs

It looks like most of the effort in setup_wargame.sh and external_tools.sh is spent doing things which could be automated with a simple Git clone into the root directory.
root $ cd /
root $ git init
root $ git remote add origin https://github.com/RPISEC/MBE-rootfs
root $ git checkout -f -t origin/master

This would automate the creation of all of the files and folders, and everything would be "ready to go".  The only scripting necessary would be to install a list of packages and fix permissions.
For an example of what this looks like, check out https://github.com/zachriggle/defcon22.

There is only one cp command (see line 320) to place the levels in the specified directory. Most of the work in setup_wargame.sh is locking down the machine, installing tools, creating users, and setting permissions.


auth-backend: add plugin export for new backend system

Hey, I just made a Pull Request!
Added authPlugin export for the auth backend. Since we want to move auth providers to be implemented in separate modules instead the plugin no longer installs all built-in providers by default.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Needs API reports.


Tasky

Title: Tasky  project and Main README.md changes
template of the project :
I have created a unique task app, you might ask if it's a standard app but, look into its feature which makes it unique & unforgettable. 🌟✌🤩
Description :

This is a Tasky  web application it will help others to make there task daily easy.
please accept this request.

Made some changes in README  that will help new contributors to how to make PR in the browser as well as from the local system

Do check the repo 🤩

Thanks for the contribution


Place the text in center

At the moment the numbers seems to be left aligned, is it possible to put them in center?

@kannaiah, I think it is possible. I will look into it...

@kannaiah, I think it is possible. I will look into it...

@kannaiah, now numbers should be center aligned.

@kannaiah, now numbers should be center aligned.

Thank you.

Thank you.


Add external-dns(v0.14.0) as a dependency

Adds external-dns (latest released version v0.14.0) as a go module dependency and updates the dnsrecord controller and dns provider logic to align with an external-dns approach to dns management.
Updates
dnsrecord changes:

Replace v1alpha1.Endpoint with externaldns Endpoint
Add GetRootDomain method to DNSRecord, returns the shortest domain that is shared across all spec.Endpoints dns names.

provider changes:

Remove Ensure and Delete and instead include externaldnsprovider.Provider in provider.Provider interface.
Add provider.Config struct to hold common provider configuration, currently it only handles filters. Update provider Factory constructors accept the config as input.
Set DomainFilter, ZoneTypeFilter and ZoneIDFilter in provider implementations from provider.Config.

AWS provider changes:

Implement AdjustEndpoints to change our generic provider specific fields for weight and geo into aws ones understood by external-dns.

Google provider changes:

Copy external-dns google provider
Adds implementations of funtions required to translate endpoints and resourceRecordSets into the different states.

endpointsToProviderFormat - converts a list of endpoints into a google specific format.
endpointsFromResourceRecordSets - converts a list of ResourceRecordSet into endpoints (google format).
resourceRecordSetFromEndpoint - converts an endpoint(google format) into a ResourceRecordSet.

controller changes:

Update DNSRecord controller to use applyChanges which follows an external-dns process to determine a set of changes(Create/Update/Delete) that are required to be made against the provider.
Use noop registry, no owner is currently set or checked for any records.
Setup provider filters:

DomainFilter = root domain for endpoints determined by calling dnsRecord.GetRootDomain()
ZoneTypeFilter = "", public or private zones considered.
ZoneIDFilter = set to single hosted zone id (managedZone.Status.ID)

Use sync policy, all changes will be applied (Create, Update and Deletes)
Use plan with above and using the current zone(plan.Current) and spec endpoints(plan.Desired) as input.

additional changes:
downgrade gatewayapiv1 to v0.7.1. This is required since the external dns aws provider is including the external dns source package which has a dependency on Gateway API v0.7.1. Downgrading is less than ideal, but dns-operator shouldn't have a dependency on gateway api, and the health check code that is currently causing it will likely be removed soon anyway in favour of provider health checks. There is also an opportunity to change external-dns to prevent it including the source package since it's including here for no reason other than getting a version
https://github.com/kubernetes-sigs/external-dns/blob/master/provider/aws/session.go#L72
Validation
As part of the POC several example dnsrecords were created to run through the various different scenarios on Google and AWS. These were all specific to my own accounts so are not added here. I did add them to the poc branch though so can be used, with modification, for testing here.
A test script is also available in this branch to step through each of the scenarios manually.
A test suite based on these will be created in the very near future,
Note: The tests for "multicluster with no control plane" are to expected to work
closes #47

Could you put the samples you showed us in the demo somewhere, so I can try this out locally?

Added a "Validation" section to the end of the description. Has links to the example DNSRecords i was using.


window recording is clipped

I'm testing with tt.display.resolution=1920x1200 resolution but
recording is done with 1920x1080 in such a way, that 120 pixels from the bottom are clipped.
I assume that 1920x1200 window should be scaled to 1920x1080, or recording resolution should match tt.display.resolution.

workaround : set
tt.display.resolution=1920x1080
in test.properties
we are working on a better default behaviour. thanks for bringing this to our attention.


[Rector] Register CODE_QUALITY_STRICT set to rector parameter set

All checks have passed 🎉 @TomasVotruba it is ready for review.

All checks have passed 🎉 @TomasVotruba it is ready for review.

Thank you. It's great to see Rector in practise. I would never spot there places :)

Thank you. It's great to see Rector in practise. I would never spot there places :)

Thank you. It's great to see Rector in practise. I would never spot there places :)

Thank you. It's great to see Rector in practise. I would never spot there places :)


how can i update the runtime asset

all tranlate data are runtime asset
i want update the data for runtime json when the  locale json has changed
I can now update the json when restarting the service, but I expect that no restart is required

all tranlate data are runtime asset
This library complies with i18n expected by Qwik. The runtimeAssets have been added to allow you to handle specific cases where it is not possible to know the translation key at compile time (meta tags, values returned from a server, etc.) and not to build the entire app.
Then your issue lacks all relevant information. Please provide the use case, and an example or detailed description of what you are trying to do or want to achieve.
Thanks

Closing due to lack of information.

I think that this use-case is similar to backend-call updated translation data to insert into json then call translation interactive webapps.
For SEO, marketing websites, current Qwik-Speak covers well cases.


macOS tabris build error

npm version : 6.10.3
node version: v12.8.0
cordova version: 9.0.0
can this cause error?

@gacal Could you provide more details? This is not a known issue.
What Tabris version do you specify in your package.json?
Which Android SDK do you have installed?
Are you using the tabris-cli?
Can you execute the build with our online build service?

Please reopen if you still have this issue.


texts with multiple weight are not respected

help, when I have text with multiple weights, the plugin homologates everything and the weight styles are lost.
Thanks.
your work is wonderful!!!!

Added support! https://github.com/schwa23/cleanupsfui/commit/9be6cc8d469d594c869c998ebc4172047f23e6d7


added jck logic to jck.mk (for issue #1754)

see discussion in PR #2474
I'm going to refer to this PR in the comments to PR #2474

@Mesbah-Alam - may I ask you to run a few internal Grinders to verify this change please?

We might want to have a more human-readable name for this PR that expresses what change it contains.

Also, just wondering, are you creating this PR as a replacement for https://github.com/AdoptOpenJDK/openjdk-tests/pull/2474 ?

@ellschi - which pull request is valid?  this one or https://github.com/AdoptOpenJDK/openjdk-tests/pull/2474?  If one is now outdated, please close it or indicate in a comment so it can be closed (it will tell us which you are using to fix https://github.com/AdoptOpenJDK/openjdk-tests/issues/1754

@Mesbah-Alam - may I ask you to run a few internal Grinders to verify this change please?

Absolutely, @ShelleyLambert ! I'd run some grinders once the PR is ready.

Hi @Mesbah-Alam and @smlambert - thank you for the feedback. I closed PR #2474. So this one is the valid one. I also changed the name of this PR. @Mesbah-Alam: Is that what you had in mind?

Tested the change internally :
jdk 16 : Internal/Grinder/14786/
jdk 8 : Internal/Grinder/14785/
The change looks good.

FYI: We are now in code freeze, so this will not be merged until after next week's release is completed.

@ellschi : I suggest that we change the title of the PR to a more meaningful one. Something like "Transition JCK logic in testJobTemplate into jck.mk".
Thanks.

@ellschi any progress on this PR? Do you have any questions?

I will close this one as stale / stalled.  Feel free to reopen with updates if I was wrong to have closed it.


What is the Sampling method "xLabs sampler" using? How to achieve roughly same result using Ksampler?

Hi, I m currently trying to migrate a old flux workflow to use xlab controlnet. Currently the two sampling method have huge difference in terms of style and composition. I have NOT even plug any controlnet stuff into the sampler yet
I wonder how to get roughly the same result of xlab sampler when using ksampler? Which sampling method is xlabs Sampler is using?
Can you maybe give us some guide regarding this? Thank you very much.
Flux Dev - xLab ControlNet .json

the same question. I have so different results. more realism and details on the sampler custom advanced in my case.

能解释一下和参数的意义吗？cfg值设置多少为宜？图生图强度和降噪强度有什么区别？


CB-9235 Adds more checks based on the windows-target-version

This is an implementation for CB-9235
The logic of check_all method was updated to include additional checks for Visual Studio and  Windows/Windows phone SDKs.
Also logic of checks has been improved to checks for versions depending on the windows-target-version and windows-phone-target-version preferences from platform’s config.xml.

Why are there AppVeyor failures

I've slightly changed Version module to allow to accept incomplete version strings, but forgot ro check  test results :( However, if the test for incomplete strings is failing, this might be done intentionally. @robpaveza, could you please confirm that?

@robpaveza Can you please take a look?

I'm checking out how it works with #102 .

When I run this branch on my local, I get an npm test failure for check_reqs.spec.js, failing to find config.xml in cordova-windows root.

build.spec.js spec.13 fails.

I'm supposing build.spec.js spec.13 fails because it doesn't mock findAllAvailableVersions (or was also findAvailableVersions in your code).  I wrote a use case for this in #102 in spec.1; take a look at that and ensure the right code paths.  Otherwise, the merge of #92 and #102 is pretty straightforward.  Before merge, make sure your AppVeyor passes - I think it's a problem with check_reqs.

When I run this branch on my local, I get an npm test failure for check_reqs.spec.js, failing to find config.xml in cordova-windows root.

Sorry, that was my bad. Fixed this here

build.spec.js spec.13 fails because it doesn't mock findAllAvailableVersions

I can't repro this. build specs are all passing on my local machine. Probably merge issues?

make sure your AppVeyor passes - I think it's a problem with check_reqs

AppVeyor tests was failing because AppVeyor machines has UAC disabled (I already mentioned that in one of my PRs, can't find it now), so the run command was failing. See https://github.com/apache/cordova-windows/blob/master/template/cordova/lib/run.js#L38. I've updated run command and its tests, so this condition is now mocked, and tests are passing even under the elevated prompt.

AppVeyor tests was failing because AppVeyor machines has UAC disabled (I already mentioned that in one of my PRs, can't find it now)

Found it: https://github.com/apache/cordova-windows/pull/99

lgtm :+1:

This was merged in 9be50cd43a893f6af532b585baa7fa00e71772c9 but not auto-closed. Closing it now.


Old images?

Question:
Portainer 2 (CE) is leaving old images on system after a new deployment/pull?
I believe that the prior version(s) removed the old image(s) unless I'm mistaken?
For example, if I (or Portainer) pull an image (such as traefik:latest)  - I will have two images:
traefik:latest
traefik:none
So Portainer is untagging the image when it stops being used, but not removing it?
Is that by design or an issue?

That is the behaviour of Docker. Always has been, and its always been the behaviour of Portainer too.
We pull the new, we dont auto-delete old.


Prediction drops to 0 after certain number of epochs

I'm using Neon for my deep Q-learning code - https://github.com/tambetm/simple_dqn. Recently I noticed an issue, that prediction of my network drops to 0 after certain number of epochs. This can be seen from Q-value graph:

Normally it would look like this:

This plot was produced using Neon commit hash 7a56fa9645a51e97c05f2e5afbbd1df7057ae832 from October 30th. My code is exactly the same.
The most plausible explanation would be, that weights are truncated to 0 at some point. Because my code hasn't changed, I suspect something in Neon code related to saving and loading weights repeatedly. In my code I need to clone a model, and simplest (and most compatible) way of doing that is to just save and load the model. I do this ~45 times before network prediction drops (it doesn't drop always at the same moment).
Any ideas what change could have resulted in such a behavior and how to debug it?

Do you have access to the serialized model files for epoch 6 and 7?  If so, we can directly check to see if the weights are going to zero.  The weights are saved via pickle so I don't know how that would truncate on serialization.  Maybe there's an issue with another part of the model initialization from a serialized file.
Is 7a56fa9 a working commit (meaning linked to the lower plot)?  Which commit is the other plot run on?
Are you serializing and deserializing every epoch?  Can we maybe add a logging line to print the average Q-value for each iteration (mini-batch) between the epochs so see if this is sudden of slowly developing over the course of an epoch?
Thanks

Thanks @nervetumer for suggestions! I checked the serialized models and weights are not zeros. But all the serialized models after epoch 7 are exactly the same (at least means of weights of all layers).
Current hypothesis is, that it might be something related to RMSProp optimizer and state handling. While weight means seem ok, mean state values for all layers are 1.26117e-44 starting from epoch 7. Possibly related to commit https://github.com/NervanaSystems/neon/commit/85a478edb63552edc396cce17ab80ae8c730ab2d, because that's the only commit that touched optimizers in between working and not working version. I also tried Adam, and while it doesn't have the exact same problem of predicting zero, it still fails to learn the way it used to.
Yes, https://github.com/NervanaSystems/neon/commit/7a56fa9645a51e97c05f2e5afbbd1df7057ae832 is the working commit. Upper plot is done with latest GitHub HEAD, probably https://github.com/NervanaSystems/neon/commit/e479ce351ce7149e566cb4be1d8101c1c3e4d179. I'm serializing-deserializing several times during epoch.
Debugging it takes time, I will keep you informed.

Have you tried running neon right before  85a478e and after to isolate if it is the optimizer?
Are you using gradient_clip_value or gradient_clip_norm in your model? Before that commit 85a478e, it was by clip_gradients set to True, and gradient_limit set to some value.
If the states all of sudden become almost 0 from epoch 7, are you able to see the "grad" value or the "state" from epoch 6? As states are updated by:
state[:] = decay * state + self.be.square(grad) * (1.0 - decay)
Then we can trace further down where causes this issue.

I played with serialization test in test_model.py and I couldn't replicate the exact error, but found another weird result:
print mlp.eval(train_set, Misclassification())

# Serialize model
save_obj(mlp.serialize(keep_states=True), tmp_save)

# Load model
mlp = Model(layers=layers)
mlp.load_weights(tmp_save)

print mlp.eval(train_set, Misclassification())

This outputs:
[ 0.74373335]
[ 0.88763332]

Shouldn't the misclassification rates be exactly the same, if evaluated on the same dataset and with the same model? The difference is even more evident, when you increase n_test variable and let it train longer - after loading the model the result is always the same - 0.8876. Amusingly outputs and weights match...
Full code is here (just run python test_model.py):
https://gist.github.com/tambetm/65542362cb24256350d8

I'm looking into the serialization issue.

Thanks for noting this issue with the test.  It is not a serialization issue but actually has to to with a buffer mismatch that happens when the the second instance of the Model object is generated.  Since the layers passed to Model (i.e. Model(layers=layers)) have already been instantiated, they do not get new buffers but a new shared buffer is generated for the MergeMultistream layer container.  So when the model is run the second time the layers in MergeMultistream output to the buffer used the first time around but the MergeMultistream  layer container is expecting those outputs in a new location.  This causes the outputs from the MergeMultistream layer on the second run to remain 0.
That said, we will try to push a fix for this.  In the meantime, I'm wondering if you can try it out to see if this is the root of your problem as well - if your model touch on the affected code then maybe this is your issue as well.
The code to try is here:
https://gist.github.com/nervetumer/7a220708967b88cbdcf4
So this should only affect container layers that inherit from the MergeBroadcast class.

Did you mean to post only model.py? There is only one minor change there and it didn't really affect the result - the second evaluation still produces 0.8876.
[ 0.73153335]
[ 0.88763332]

I'm not using MergeMultistream in my code, so I guess the fix doesn't apply directly. But once we get this fixed, hopefully I can modify the test code to reproduce my error as well (so far no luck).
I also noticed, that version 1.1.3 (e479ce351ce7149e566cb4be1d8101c1c3e4d179) tends to have worse performance than version 1.1.0 (7a56fa9645a51e97c05f2e5afbbd1df7057ae832) - average score in Breakout is 305 vs 319. I'm using the exact same model snapshot (breakout_77.pkl), not doing training at all, only prediction.

Sorry that was the wrong file.  The one i meant to give you layers/containter.py :
https://gist.github.com/nervetumer/7c538802bad3f7a60576

Give me a few days for this. All my Maxwell GPU-s are busy at the moment :).

As expected, this container fix didn't fix my problem:

Back to testing...

I can assure, that the problem is not related to saving-loading the model. When I turned off saving-loading, then 7a56fa9645a51e97c05f2e5afbbd1df7057ae832 still learned, but 367651890e646c6861dbdc93c1bf4788c51a363d didn't. Currently testing if https://github.com/NervanaSystems/neon/issues/171#issuecomment-168426500 fixes the problem.

Hi @tambetm,  Have you had a chance to test #171 with this?  I'm hoping to get some time to work on this soon.

Nope. Tested with 64 threads as outlined here: https://github.com/NervanaSystems/neon/issues/171#issuecomment-168426500. Currently I'm out of ideas.

Hi @tambetm  can you speak to @yinyinl comment above with respect to isolating the commit that caused this.  Is it for sure that 85a478e caused the break?

  I think it is https://github.com/NervanaSystems/neon/commit/adab385dace2ef75f58852157b55788cb56b8647.  That said I have no idea why this would affect you at all.  Will need to look at it closely.

Thanks @nervetumer for tracking that down! I've been really busy last week
and this week doesn't look any better... Will test it ASAP.
On Mon, Feb 22, 2016 at 8:03 AM, nervetumer<EMAIL_ADDRESS>wrote:

I think i tracked down the commit that is causing your problem. I think it
is adab385
https://github.com/NervanaSystems/neon/commit/adab385dace2ef75f58852157b55788cb56b8647.
That said I have no idea why this would affect you at all. Will need to
look at it closely.
—
Reply to this email directly or view it on GitHub
https://github.com/NervanaSystems/neon/issues/153#issuecomment-187025434
.

If you can confirm that that is the commit causing the issue, that would help.  I'm looking at the commit and working with people here to see why it may be causing you an issue.

@tambetm.  I think I have verification that this is the commit that led to your model breaking. I believe the problem may be coming from line 143-145 of neon/layers/layer.py. I'm testing it now and will report as soon as enough epochs run to have some confidence.   Still not sure why this would cause a problem.

Below is the Q plot for the commit SHA adab385

If I comment out lines 143 and 144 in neon/layers/container.py
       if self.next_layer is None or self.next_layer.parallelism != self.paral
            self.owns_delta = True

I get a good run:

What is happening is that this code is switching the self.owns_delta from False to True for the Linear layers in your model.   Still not exactly sure why this is causing a strange, random error like this.  We will try to put a fit for this asap.  If you want comment those lines out for now.

I guess you meant neon/layers/layer.py, no?
I'm trying to reproduce the error here with adab385d, but no luck yet (but likely my fault). Starting again from scratch; will leave it running overnight and report back tomorrow.

I can also confirm, that there is no error with c8003191bb309b6df9208631dbef74a4b2388203 and the error occurs with 257b5d5aa8e154dce43dbe718d26f19aa1f3871a (which is one commit after adab385dace2ef75f58852157b55788cb56b8647,  adab385dace2ef75f58852157b55788cb56b8647 had a weird bug and I had to test with the next commit).
The lines that cause the error are beyond my depth with Neon, so I'm relying on you here. I wonder if it would be possible to create a simpler test case to reproduce the issue? I tried it once, based on test_model_serialize() in tests/test_model.py, but failed. Reproducing the issue currently takes couple of hours and that's the main bummer.

We have a PR staged to fix this.

We're delaying the fix until we're sure that it won't affect anything else, but in the meantime, adding the following lines to your existing code should accomplish the same thing:
Ensure same parallelism mode all the way through the network as follows
Prior to model initialization for self.model here
for l in self.model.layers.layers:
    l.parallelism = 'Disabled'

And prior to model initialization for self.target_model here
for l in self.target_model.layers.layers:
    l.parallelism = 'Disabled'

I should also note that you can now get_description from the model to accomplish your deep copy without serializing to disk (will still transfer weights from device to host and back) by doing:
pdict = self.model.get_description(get_weights=True)
self.target_model.deserialize(pdict, load_states=False)

This should be fixed now in 1.5.1.


[SUGGESTION] Add DimStorage Support

DimStorage is a bit like EnderStorage. They basically have the same concept but I personally use DimStorage because rendering Ender Storage items in JEI crashes my client. Would it be possible to add support for the DimChests from DimStorage mod?
Here's the curseforge page for DimStorage: https://www.curseforge.com/minecraft/mc-mods/dimstorage

I will look into it before the next update

Support added in v1.2.0


Switch OIDC controller to use service network to read from KAS

controlPlaneEndpoint is not resolvable/reachable from the management cluster for private clusters.
This PR switches the OIDC controller to read from the HCP KAS over the service network rather than using the controlPlaneEndpoint, which is the endpoint nodes use to contact the KAS.

/test e2e-aws


[BULK] - DocuTune remediation - Sensitive terms with GUIDs

Applying sensitive terms with GUID changes as part of Content SFI and outlined in Overview - Writing content securely - Platform Manual. Changes are part of the Microsoft-wide SFI effort. Point of contact: @CelesteDG
DocuTune v<IP_ADDRESS>
CorrelationId: ac15aa43-4e2b-437f-ab1c-fdd7e79cd4db
#docutune

Changes here are for an allowed GUID. We should close this PR.
#please-close

#please-close

@MicrosoftDocs/apex-docs-pr-reviewers, can you please close this PR? No changes are required.


Migrate build tooling to PNPM and NX

Migrate build tooling to PNPM and NX
Purpose

The current monorepo build tooling relies on Yarn, which, while effective, does not offer the same performance optimizations, workspace management, and task orchestration provided by newer tools like PNPM and NX. Additionally, the current release process lacks automation for managing versioning and changelogs, which can be improved with Changesets. As the project scales, upgrading the tooling will enhance the development workflow and ensure smoother releases.

Goals

[ ] Replacing Yarn with PNPM to leverage faster installs and workspace management.
[ ] Integrating NX for improved build processes and incremental builds across the monorepo.
[ ] Incorporating Changesets to automate version control and changelog updates.
[ ] Updating scripts and CI configurations to support the new tooling.
[ ] Verifying compatibility and functionality across all packages.

CC: @pavinduLakshan
The migration to PNPM is complete. Could you please review it?


returning URL.empty instead of null. with this approach the route wil…

…l match only with root URL, as susually is used as catchAll type of route.

Codecov Report

Merging #1306 (212f486) into main (c8f1437) will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1306      +/-   ##
==========================================
+ Coverage   59.64%   59.68%   +0.04%
==========================================
  Files          70       70
  Lines        2453     2453
  Branches       71       71
==========================================
+ Hits         1463     1464       +1
+ Misses        990      989       -1

Impacted Files
Coverage Δ

...io-http/src/main/scala/zhttp/service/Handler.scala
73.91% <ø> (+8.69%)
:arrow_up:

...ain/scala/zhttp/service/ServerResponseWriter.scala
83.72% <0.00%> (-4.66%)
:arrow_down:

zio-http/src/main/scala/zhttp/http/Response.scala
82.85% <0.00%> (-1.43%)
:arrow_down:

zio-http/src/main/scala/zhttp/http/HttpData.scala
83.33% <0.00%> (+1.51%)
:arrow_up:

...http/service/server/ServerChannelInitializer.scala
82.60% <0.00%> (+4.34%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8f1437...212f486. Read the comment docs.


BUG: ModuleNotFoundError when SciPy exist and Installation of 1.3.0 doesnt work

Describe your issue.
I am trying to install scipy==1.3.0 on a virtual environment for a personal project but it seems to throw an ModuleNotFoundError when I run my code as shown below.
Traceback (most recent call last):
  File "C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\main.py", line 1, in <module>
    from scipy.spatial import distance
ModuleNotFoundError: No module named 'scipy'

When I use !pip list, it shows a version of  scipy == 1.11.4 ... I try to install using requirements.txt and it returns the error as shown in the error message.
Reproducing Code Example
from scipy.spatial import distance

Error message
Collecting scipy==1.3.0 (from -r requirements.txt (line 4))
  Using cached scipy-1.3.0.tar.gz (23.6 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [302 lines of output]
      Ignoring numpy: markers 'python_version == "3.5"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.6"' don't match your environment
      Collecting wheel
        Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/c7/c3/55076fc728723ef927521abaa1955213d094933dc36d4a2008d5101e1af5/wheel-0.42.0-py3-none-any.whl.metadata
        Using cached wheel-0.42.0-py3-none-any.whl.metadata (2.2 kB)
      Collecting setuptools
        Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/55/3a/5121b58b578a598b269537e09a316ad2a94fdd561a2c6eb75cd68578cc6b/setuptools-69.0.3-py3-none-any.whl.metadata
        Using cached setuptools-69.0.3-py3-none-any.whl.metadata (6.3 kB)
      Collecting Cython>=0.29.2
        Obtaining dependency information for Cython>=0.29.2 from https://files.pythonhosted.org/packages/85/7d/58635941dfbb5b4e197adb88080b9cbfb230dc3b75683698a530a1989bdb/Cython-3.0.8-cp311-cp311-win_amd64.whl.metadata
        Downloading Cython-3.0.8-cp311-cp311-win_amd64.whl.metadata (3.2 kB)
      Collecting numpy==1.14.5
        Using cached numpy-1.14.5.zip (4.9 MB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'done'
      Using cached wheel-0.42.0-py3-none-any.whl (65 kB)
      Using cached setuptools-69.0.3-py3-none-any.whl (819 kB)
      Downloading Cython-3.0.8-cp311-cp311-win_amd64.whl (2.8 MB)
         ---------------------------------------- 2.8/2.8 MB 6.3 MB/s eta 0:00:00
      Building wheels for collected packages: numpy
        Building wheel for numpy (setup.py): started
        Building wheel for numpy (setup.py): finished with status 'error'
        error: subprocess-exited-with-error

        python setup.py bdist_wheel did not run successfully.
        exit code: 1

        [244 lines of output]
        Running from numpy source directory.
        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\misc_util.py:464: SyntaxWarning: "is" with a literal. Did you mean "=="?
          return is_string(s) and ('*' in s or '?' is s)
        blas_opt_info:
        blas_mkl_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries mkl_rt not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        blis_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries blis not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        openblas_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries openblas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
        get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
        customize GnuFCompiler
        Could not locate executable g77
        Could not locate executable f77
        customize IntelVisualFCompiler
        Could not locate executable ifort
        Could not locate executable ifl
        customize AbsoftFCompiler
        Could not locate executable f90
        customize CompaqVisualFCompiler
        Could not locate executable DF
        customize IntelItaniumVisualFCompiler
        Could not locate executable efl
        customize Gnu95FCompiler
        Found executable C:\msys64\ucrt64\bin\gfortran.exe
          NOT AVAILABLE

        atlas_3_10_blas_threads_info:
        Setting PTATLAS=ATLAS
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries tatlas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        atlas_3_10_blas_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries satlas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        atlas_blas_threads_info:
        Setting PTATLAS=ATLAS
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries ptf77blas,ptcblas,atlas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        atlas_blas_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries f77blas,cblas,atlas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\system_info.py:624: UserWarning:
            Atlas (http://math-atlas.sourceforge.net/) libraries not found.
            Directories to search for the libraries can be specified in the
            numpy/distutils/site.cfg file (section [atlas]) or by setting
            the ATLAS environment variable.
          self.calc_info()
        blas_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries blas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\system_info.py:624: UserWarning:
            Blas (http://www.netlib.org/blas/) libraries not found.
            Directories to search for the libraries can be specified in the
            numpy/distutils/site.cfg file (section [blas]) or by setting
            the BLAS environment variable.
          self.calc_info()
        blas_src_info:
          NOT AVAILABLE

        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\system_info.py:624: UserWarning:
            Blas (http://www.netlib.org/blas/) sources not found.
            Directories to search for the sources can be specified in the
            numpy/distutils/site.cfg file (section [blas_src]) or by setting
            the BLAS_SRC environment variable.
          self.calc_info()
          NOT AVAILABLE

        'svnversion' is not recognized as an internal or external command,
        operable program or batch file.
        'svnversion' is not recognized as an internal or external command,
        operable program or batch file.
        non-existing path in 'numpy\\distutils': 'site.cfg'
        F2PY Version 2
        lapack_opt_info:
        lapack_mkl_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries mkl_rt not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        openblas_lapack_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries openblas not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
        get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
        customize GnuFCompiler
        customize IntelVisualFCompiler
        customize AbsoftFCompiler
        customize CompaqVisualFCompiler
        customize IntelItaniumVisualFCompiler
        customize Gnu95FCompiler
          NOT AVAILABLE

        openblas_clapack_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries openblas,lapack not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
        get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
        customize GnuFCompiler
        customize IntelVisualFCompiler
        customize AbsoftFCompiler
        customize CompaqVisualFCompiler
        customize IntelItaniumVisualFCompiler
        customize Gnu95FCompiler
          NOT AVAILABLE

        atlas_3_10_threads_info:
        Setting PTATLAS=ATLAS
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries tatlas,tatlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries tatlas,tatlas not found in C:\
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\
        <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
          NOT AVAILABLE

        atlas_3_10_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries satlas,satlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries satlas,satlas not found in C:\
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\
        <class 'numpy.distutils.system_info.atlas_3_10_info'>
          NOT AVAILABLE

        atlas_threads_info:
        Setting PTATLAS=ATLAS
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries ptf77blas,ptcblas,atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries ptf77blas,ptcblas,atlas not found in C:\
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\
        <class 'numpy.distutils.system_info.atlas_threads_info'>
          NOT AVAILABLE

        atlas_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries f77blas,cblas,atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\lib
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries f77blas,cblas,atlas not found in C:\
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack_atlas not found in C:\
        <class 'numpy.distutils.system_info.atlas_info'>
          NOT AVAILABLE

        lapack_info:
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        customize MSVCCompiler
          libraries lapack not found in ['C:\\Users\\Advith Krishnan\\OneDrive\\Desktop\\Practice Files\\Programming\\Porjects\\Graph_embedder\\DDGK4G\\embed_env\\lib', 'C:\\']
          NOT AVAILABLE

        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\system_info.py:624: UserWarning:
            Lapack (http://www.netlib.org/lapack/) libraries not found.
            Directories to search for the libraries can be specified in the
            numpy/distutils/site.cfg file (section [lapack]) or by setting
            the LAPACK environment variable.
          self.calc_info()
        lapack_src_info:
          NOT AVAILABLE

        C:\Users\Advith Krishnan\AppData\Local\Temp\pip-install-yaubumyp\numpy_468e07a35c0d4b86a09a59a0549c83b4\numpy\distutils\system_info.py:624: UserWarning:
            Lapack (http://www.netlib.org/lapack/) sources not found.
            Directories to search for the sources can be specified in the
            numpy/distutils/site.cfg file (section [lapack_src]) or by setting
            the LAPACK_SRC environment variable.
          self.calc_info()
          NOT AVAILABLE

        C:\Users\Advith Krishnan\OneDrive\Desktop\Practice Files\Programming\Porjects\Graph_embedder\DDGK4G\embed_env\Lib\site-packages\setuptools\_distutils\dist.py:264: UserWarning: Unknown distribution option: 'define_macros'
          warnings.warn(msg)
        running bdist_wheel
        running build
        running config_cc
        unifing config_cc, config, build_clib, build_ext, build commands --compiler options
        running config_fc
        unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
        running build_src
        build_src
        building py_modules sources
        creating build
        creating build\src.win-amd64-3.1
        creating build\src.win-amd64-3.1\numpy
        creating build\src.win-amd64-3.1\numpy\distutils
        building library "npymath" sources
        No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
        error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
        [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for numpy
        Running setup.py clean for numpy
        error: subprocess-exited-with-error

        python setup.py clean did not run successfully.
        exit code: 1

        [10 lines of output]
        Running from numpy source directory.

        `setup.py clean` is not supported, use one of the following instead:

          - `git clean -xdf` (cleans all files)
          - `git clean -Xdf` (cleans all versioned files, doesn't touch
                              files that aren't checked into the git repo)

        Add `--force` to your command to use it anyway if you must (unsupported).

        [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed cleaning build dir for numpy
      Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

      [notice] A new release of pip is available: 23.2.1 -> 23.3.2
      [notice] To update, run: python.exe -m pip install --upgrade pip
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 23.2.1 -> 23.3.2
[notice] To update, run: python.exe -m pip install --upgrade pip

SciPy/NumPy/Python version and system information
{
  "Compilers": {
    "c": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "cc"
    },
    "cython": {
      "name": "cython",
      "linker": "cython",
      "version": "0.29.36",
      "commands": "cython"
    },
    "c++": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "c++"
    },
    "fortran": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.3.0",
      "commands": "gfortran"
    },
    "pythran": {
      "version": "0.14.0",
      "include directory": "C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pip-build-env-20hlwiiy\\overlay\\Lib\\site-packages/pythran"
    }
  },
  "Machine Information": {
    "host": {
      "cpu": "x86_64",
      "family": "x86_64",
      "endian": "little",
      "system": "windows"
    },
    "build": {
      "cpu": "x86_64",
      "family": "x86_64",
      "endian": "little",
      "system": "windows"
    },
    "cross-compiled": true
  },
  "Build Dependencies": {
    "blas": {
      "name": "openblas",
      "found": true,
      "version": "0.3.21.dev",
      "detection method": "pkgconfig",
      "include directory": "/c/opt/64/include",
      "lib directory": "/c/opt/64/lib",
      "openblas configuration": "USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2",
      "pc file directory": "c:/opt/64/lib/pkgconfig"
    },
    "lapack": {
      "name": "openblas",
      "found": true,
      "version": "0.3.21.dev",
      "detection method": "pkgconfig",
      "include directory": "/c/opt/64/include",
      "lib directory": "/c/opt/64/lib",
      "openblas configuration": "USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS= NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2",
      "pc file directory": "c:/opt/64/lib/pkgconfig"
    },
    "pybind11": {
      "name": "pybind11",
      "version": "2.11.0",
      "detection method": "config-tool",
      "include directory": "unknown"
    }
  },
  "Python Information": {
    "path": "C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-vvhxzz5y\\cp311-win_amd64\\build\\venv\\Scripts\\python.exe",
    "version": "3.11"
  }
}

Kindly help with this @SciPy

There is nowhere near enough information here. First of all from scipy.spatial import distance will not produce the error message displayed. The error trace is output from trying to build scipy, not from the import.
Please give us the output of python -m pip list. Furthermore scipy 1.3.0 is very old, it will not build with modern numpy versions.
Most importantly numpy doesn't appear to be installed, you need to have installed that first.

The following displays the output from !python -m pip list and there is an existing NumPy package (Version: 1.24.3).

Package                      Version

absl-py                      1.0.0
astunparse                   1.6.3
cachetools                   5.3.2
certifi                      2023.11.17
charset-normalizer           3.3.2
decorator                    5.1.1
flatbuffers                  23.5.26
gast                         0.4.0
google-auth                  2.25.2
google-auth-oauthlib         1.0.0
google-pasta                 0.2.0
grpcio                       1.60.0
h5py                         3.10.0
idna                         3.6
jax                          0.4.23
joblib                       1.3.2
keras                        2.13.1
libclang                     16.0.6
Markdown                     3.5.1
MarkupSafe                   2.1.3
ml-dtypes                    0.3.1
networkx                     2.5
numpy                        1.24.3
oauthlib                     3.2.2
opt-einsum                   3.3.0
packaging                    23.2
pip                          23.2.1
protobuf                     4.25.1
pyasn1                       0.5.1
pyasn1-modules               0.3.0
requests                     2.31.0
requests-oauthlib            1.3.1
rsa                          4.9
scikit-learn                 1.3.2
scipy                        1.11.4
setuptools                   65.5.0
six                          1.16.0
sklearn                      0.0
tensorboard                  2.13.0
tensorboard-data-server      0.7.2
tensorflow                   2.13.0
tensorflow-estimator         2.13.0
tensorflow-intel             2.13.0
tensorflow-io-gcs-filesystem 0.31.0
termcolor                    2.4.0
threadpoolctl                3.2.0
tqdm                         4.32.2
typing_extensions            4.5.0
urllib3                      2.1.0
Werkzeug                     3.0.1
wheel                        0.42.0
wrapt                        1.14.1

Also my apologies for not putting the right code in the Reproducible code example section, and yes you are right by saying the error trace is for the building of the package into my virtual environment, as mentioned in the Issue description section.


Disable buggy gemspec Ruby version cop

Please see the commits for more details.

@laurentqro I think you had a better idea with increasing the minimum version, as disabling the target version cop could mean Rubocop suggests things that aren't compatible with the minimum version. I've updated to reflect this - could you take another look?

@benthorner Agreed! Looks good to me 👌


Crash when trying to collect quest rewards

Just started, finished the first quest, click on "receive reward" button, game crashed. Tried reinstalling, did not fix.
Log:
crash-2022-08-20_11.00.27-client.txt

Theres only BetterQuesting mentioned there, looks like its a problem on their side
I think you should report it to them https://github.com/CleanroomMC/BetterQuesting

Pack version

Pack version?

Right, that would be 3.2.0

Looking at the crash, I’m going to echo what DerCommander said, post the log on https://github.com/CleanroomMC/BetterQuesting


improve window with windowex & add taskbar icon

This project is impressive! I've always wanted to develop a fluent Tailscale client for Windows.
In this pull request, I’ve added a tray icon, enabling it to run in the background, making it function more like the official Tailscale client.
Additionally, I’ve replaced the original Window with WindowEx in these codes. WindowEx is a great foundation for WinUI 3 development.

Thank you very much for your contribution!
The tray icon was something I dreaded implementing, turns out it was much simpler than expected.
Looks good to me, thank you again.


[CLOSED] Import cma cases with missing bodies

 Issue by mikejustdoit
Tuesday 10 March 2015 at 13:26 GMT
Originally opened as https://github.com/alphagov/specialist-publisher/pull/443

Generates a body based on the default body copy "Full text of the decision" and links to the document's attachments.
Suggestions for a better approach than CmaImport#document_creator particularly welcome

mikejustdoit included the following code: https://github.com/alphagov/specialist-publisher/pull/443/commits

 Comment by mikejustdoit
Tuesday 10 March 2015 at 13:40 GMT

Oh, here's a small problem. Of the 345 cases that have no body, 34 have no attachments, so end up with just the body copy and no link. Any ideas?

 Comment by evilstreak
Wednesday 11 March 2015 at 16:12 GMT

:+1: Nice work


Fix coverage report in CI

Codecov Report

Merging #1053 (2574aba) into main (d9088eb) will decrease coverage by 5.71%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1053      +/-   ##
==========================================
- Coverage   98.54%   92.83%   -5.72%
==========================================
  Files          49       81      +32
  Lines       10655     7299    -3356
==========================================
- Hits        10500     6776    -3724
- Misses        155      523     +368

Impacted Files
Coverage Δ

test_connectedcomponentscomposite.py

test_identity_sampler.py

test_sampleset.py

test_fixedpolyvariablecomposite.py

test_null_sampler.py

test_exact_solver.py

test_higherorder_polynomial.py

test_serialization_json.py

test_fixedvariablecomposite.py

test_structure_composite.py

... and 120 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9088eb...2574aba. Read the comment docs.


Suggestion: Change inserter default whitelist/blacklist

Mod source
PyAE Beta
Which mod are you having an issue with?

[ ] pyalienlife
[X] pyalternativeenergy
[ ] pycoalprocessing
[ ] pyfusionenergy
[ ] pyhightech
[ ] pyindustry
[ ] pypetroleumhandling
[ ] pypostprocessing
[ ] pyrawores

Operating system

=Windows 10

What kind of issue is this?

[ ] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[ ] Progression
[X] Balance
[ ] Pypostprocessing failure
[ ] Other

What is the problem?
When placing and configuring an inserter for a filtered item it defaults to blacklist this item.
It would streamline the process to set the default to whitelist this item.
 There was a very positive response to this suggestion on the discord.
Steps to reproduce
No response
Additional context
No response
Log file
No response

a whitelist with an empty selection slot picks up nothing, so if you default them to whitelist, a newly placed inserter without the selection will do nothing at all

Ah that sucks. I'll be forever dumping random stuff on my belts then.

See comments above


Async support merge

This is a merge of master to the async-support branch to keep the branch point up to date.
Most of the changes were not appropriate, but I believe these ones should be merged over.

Yep, these are good! LGTM

@meltsufin my git-foo has failed me, as merging branches via a PR has not actually moved the branch point of the real branches.     I think we now need to do a real merge from master to async-support, ignore all the changes and then commit?

I've merged in the changes in this PR to async-support. Are you saying you that not all changes that you intended to include were included here?


./deposit.sh install command fails with permission issue on Linux Ubuntu

System : Linux Ubuntu
Command -
./deposit.sh install
Error -
    cytoolz/dicttoolz.c:17:10: fatal error: Python.h: No such file or directory
     #include "Python.h"
              ^~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-phtudhwx/cytoolz/setup.py'"'"'; __file__='"'"'/tmp/pip-install-phtudhwx/cytoolz/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-6w7rqm5q/install-record.txt --single-version-externally-managed --compile --user --prefix= Check the logs for full command output. ```

if you're using python3.7 this worked for me
sudo apt-get install python3.7-dev

@raj-shekhar1  does it work for you now? Or see other ways to install it: https://github.com/ethereum/eth2.0-deposit-cli/#for-linux-or-macos-users
I'm closing it now. Let me know if it doesn't work for you.


PictoBert Informations

Hi, I am Borghi Simone at University of Modena Italy. I worked as a special education teacher for children with autism. Although that's not my area of research, I like the theme of artificial intelligence applied to language. I am new in the field of Natural Language processing and I find your application very interesting. I would try to test the scripts on Colab, but i find that some links no longer work. Could you provide them to me or give me alternative links? Thanks.

Hi, Borghi. Nice to know you are interested in our work. I noticed this issue some days ago and am looking forward to fixing it. By now, you can download the data from the Code Ocean capsule: https://codeocean.com/capsule/3641889/tree/v1
If you want to download the pre-trained PictoBERT, for example, you can run the following in colab:
!wget https://files.codeocean.com/files/verified/76736ca2-f912-42ae-bfae-010c700abaa8_v1.0/data/pictobert-large-contextual.zip
I will let you know when I fix all the download links.

Thank you very much!!!

Hello !
I am developing an (free and open-source) AAC board application called Pictalk.
I would really love to integrate pictoBERT to Pictalk. Do you plan a release on HuggingFace ? Are you OK with us working on pictoBERT ?
Thanks a lot for your work,
Alex

I read the post on your site. I'm sorry about your brother, but I think
disability is an asset. I worked for a year with Nicolò, a non-verbal child
with autism. This experience changed my life. I am also working on studying
useful applications for people with difficulties.
Simone Borghi
Il giorno sab 18 mar 2023 alle ore 13:01 Jayr Alencar <
@.***> ha scritto:

Hi, Borghi. Nice to know you are interested in our work. I noticed this
issue some days ago and am looking forward to fixing it. By now, you can
download the data from the Code Ocean capsule:
https://codeocean.com/capsule/3641889/tree/v1
If you want to download the pre-trained PictoBERT, for example, you can
run the following in colab:
!wget
https://files.codeocean.com/files/verified/76736ca2-f912-42ae-bfae-010c700abaa8_v1.0/data/pictobert-large-contextual.zip
I will let you know when I fix all the download links.
—
Reply to this email directly, view it on GitHub
https://github.com/jayralencar/pictoBERT/issues/1#issuecomment-1474829218,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYJ634TXNVPR7YFNM5IDSNTW4WP2XANCNFSM6AAAAAAV7KCJFQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hello,
I'm having conversation with a PHD psychologist specialized with autism in spain and collaborating with hospital. My niece have severe autism and I'm totally convinced that AI will help.  I think that make sense to join the effort, and move faster. @Ratatinator97  and @jayralencar what you think to work together and deliver a open source model to help the people ?

Hi everybody. On the model of the excellent PictoBert work, I promoted the
start of an activity at the University of Bologna for an online
application, aided by artificial intelligence. Efforts could be combined.
Il giorno mer 13 dic 2023 alle ore 10:23 mmendozaf @.***>
ha scritto:

Hello,
I'm having conversation with a PHD psychologist specialized with autism in
spain and collaborating with hospital. My niece have severe autism and I'm
totally convinced that AI will help. I think that make sense to join the
effort, and move faster. @Ratatinator97 https://github.com/Ratatinator97
and @jayralencar https://github.com/jayralencar what you think to work
together and deliver a open source model to help the people ?
—
Reply to this email directly, view it on GitHub
https://github.com/jayralencar/pictoBERT/issues/1#issuecomment-1853547915,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYJ634X5LVBBMQDRKLGMNC3YJFXYNAVCNFSM6AAAAAAV7KCJFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJTGU2DOOJRGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi, everyone. It would be great to have an initiative in this field.
Take a look at our recent research: https://www.sciencedirect.com/science/article/abs/pii/S0957417423029196
You can request full-text from ResearchGate: https://www.researchgate.net/publication/375299085_PrAACT_Predictive_Augmentative_and_Alternative_Communication_with_Transformers
We could create a group (discord, slack, or WhatsApp) to discuss next steps. What do you think about it?

Great idea. I am giving you my Unimore contact @.)
and the contact of Giacomo Frisoni, who officially works on this project (
@.) from Unibo.
Il giorno mer 13 dic 2023 alle ore 12:26 Jayr Alencar Pereira <
@.***> ha scritto:

Hi, everyone. It would be great to have an initiative in this field.
Take a look at our recent research:
https://www.sciencedirect.com/science/article/abs/pii/S0957417423029196
You can request full-text from ResearchGate:
https://www.researchgate.net/publication/375299085_PrAACT_Predictive_Augmentative_and_Alternative_Communication_with_Transformers
We could create a group (discord, slack, or WhatsApp) to discuss next
steps. What do you think about it?
—
Reply to this email directly, view it on GitHub
https://github.com/jayralencar/pictoBERT/issues/1#issuecomment-1853741143,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AYJ634RTYIFHT3XIWHMS333YJGGG5AVCNFSM6AAAAAAV7KCJFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJTG42DCMJUGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

I just create a slack channel to chat.
https://join.slack.com/t/slack-6bi4684/shared_invite/zt-28xdc5kuc-q1rH4Nt7PpFfFbodpgEOUQ
Join as emails are not shown on GitHub


ResilientParser rework

many more recovery paths have been added.  it is now rare to see squiggles to the end of the document.  the parser usually is able to recover and parse remaining fields and values.
use new diagnostics API
initial WIP completions support. completions work if you type '.t', showing completions for 'tags', and 'title' fields.  however they don't work if you type '.' alone. and this is a major TODO which i'm not sure how to address yet.  i've added some FIXME and TODO comments for this and welcome any ideas.

Please don't merge yet. I was preparing a screen shot and noticed a regression. Will fix and post screenshot soon.

Ok fixed. Here's what it looks like:


Fix typo for full opacity

0xff means full-transparency (implying aHasFilling->false). 0x00 means no transparency (full-opacity).

Thank you for catching that!


m_spinup_starting_time: deal with DOS and UNIX style line endings more robustly

This is a follow-up to c8e55a3319dd3a5e00ce664873346773a8b7ae32
When I compiled 0.5.12 on Linux, I noticed that the server runs into an exception in ReadProjPresetData, while the pre-compiled 0.5.12 provided for download did not.
As it turned out, the issue was that the projData string looked like this:
[...]
proj_vortex
_damage_robot;7
[...]

with the inital m character missing...
The reason is https://github.com/overload-development-community/olmod/blob/48c18f519495c267c2aacccbbbf79877370ba4af/GameMod/PresetData.cs#L113
This assumes that a full CRLF \r\n string sequence is present as it skips 2 characters after the value.
However, the default projData content is a raw string in MPModPrivateData.cs. My assumption here is that when you compile this project on Windows, you have set up your git so that line endings will be DOS-Style CRLF on checkout, but the checked in files in the repo will be UNIX-style LF (as they should be) . But when I check out this file on Linux, I get the UNIX-stlye line endings, and the string which is compiled into the binary doesn't contain the CRs in the first place.
Note that this issue could also be present with custom projdata files from users, as they might use both line ending conventions.
This small patch makes the handling more robust so that it can deal with both line ending conventions.

Good find. seems we can't escape the Linux/Windows problem, as this is the same exception that 0.5.11 was running into on every round start that we thought we'd fixed. There has to be a better way to do this than counting characters, although I realize that's also what the current version is doing. We could make this a lot more universal by just replacing the RUtility.ReadField(component, array[0], array[1]); call in ProjectileManager.ReadProjPresetData() with a method that scans for specified lines as they are read in line-by-line by the existing code and filters as necessary. We could also very easily extend this if we needed more fields in the future.

@derhass I have the extensible version ready, can you give #333 a breakage test on Linux and see if I missed anything? I tried a couple of projdata files on it and it seems to hold up.

@derhass I have the extensible version ready, can you give #333 a breakage test on Linux and see if I missed anything? I tried a couple of projdata files on it and it seems to hold up.

@CCraigen Yours is a much better solution than my quick-and-dirty fix here. As far as I can tell, it works as expected.
This PR is not needed any more.


Add a Gitter chat badge to README.md

drepl/drepl now has a Chat Room on Gitter
@wilzbach has just created a chat room. You can visit it here: https://gitter.im/drepl/drepl.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

hmm seems like they have a duplicate PR problem

hmm seems like they have a duplicate PR problem

Just fixed the travis badge.


False positive for rule strict-effect-handlers

Function Unknown can lead to scope loosing in Fork API.eslint[effector/strict-effect-handlers](https://eslint.effector.dev/rules/strict-effect-handlers)
And this is not just IDE bug

Maybe this is related with updates of plugin or eslint
Versions on the left side don't have the issue.

Hello! Thanks for report.
Could you please ensure that TS integration for ESLint is set up correctly? (refer to the issue #131)
I believe in this case ESLint fallen back to JS-checks which could be inaccurate and could not check that the field of object ($$editAdress.setAsDefaultFx) is Effect.

It cannot be reproduced — https://stackblitz.com/edit/effector-vite-react-template-dajnx8?file=tsconfig.json,src%2Fmodel.ts%3AL20,src%2Fanother-model.ts,src%2Fapplication.tsx

It cannot be reproduced — https://stackblitz.com/edit/effector-vite-react-template-dajnx8?file=tsconfig.json,src%2Fmodel.ts%3AL20,src%2Fanother-model.ts,src%2Fapplication.tsx

The culprit of the problem is @typescript-eslint/parser v6+
The change that make issue to appear in my local monorepo setup:
"@typescript-eslint/parser": "^5.59.0",
---->
"@typescript-eslint/parser": "^6.7.3",

I think this is related with this one

It is about #144
Will be fixed 🫡
Let me close this one as duplicate


SocialliteProviders... does not exist

BookStack Version: v0.18.2
PHP Version: 7.0

Expected Behavior
Updated to the latest version from the previous version. I should be able to access user settings and the login screen.
Current Behavior
Instead I get the following (After enabling debug)

Steps to Reproduce

When running php artisan cache:clear I get the following as well:

I found the issue.... I had installed php7.1 as well for another project... This was messing up composer install
Now that I purged php7.1 and re-ran the update commands per documentation, it is working again,