Fix/4676 libressl support

The C preprocessor hack to determine OpenSSL & LibreSSL version numbers set them to 0 on failure now. Also I redirected stderr to stdoutput and redirected that to /dev/null to silence errors on the second commit but I'm not sure if a Unix compatible shell is guaranteed, we can omit that if you like.

Oops, forgot to mention this resolves #4676.

Yeah this is still going to break things. Because of openssl and libressl's breaking API changes, assuming the oldest possible version on failure just won't work. It'll fail to link.
Actually, why are we doing this at all. We should just revert the original PR back to the original well-tested version detection that worked for openssl. We then shell out to the preprocessor only for detecting libressl, and if we can't shell out or the headers don't exist or something similar we just assume openssl.

And in parallel we should bug libressl upstream to do something actually sane, and explain our 9and rust's) issues here.

Yeah this is still going to break things. Because of openssl and libressl's breaking API changes, assuming the oldest possible version on failure just won't work. It'll fail to link.
Actually, why are we doing this at all. We should just revert the original PR back to the original well-tested version detection that worked for openssl.

But that's what the original version detection did in the first place, check for 1.0.2 and 1.1.0 if it can (using pkg-config) and assume it's an older version otherwise. If it fails to link with this solution, it would have failed with that too.

And in parallel we should bug libressl upstream to do something actually sane, and explain our (and rust's) issues here.

I agree but I wouldn't hold my breath on it :)

But that's what the original version detection did in the first place, check for 1.0.2 and 1.1.0 if it can (using pkg-config) and assume it's an older version otherwise. If it fails to link with this solution, it would have failed with that too.

But we no-longer check pkg-config --modversion at all. We previously did and it worked fine, now we don't and it doesn't work fine.

We previously did and it worked fine, now we don't and it doesn't work fine.

I think the current solution is fine and every hickup we had with it was due to trivial implementation errors but I am not against the idea of trying pkg-config --modversion first, headers later. I would give it a go but I am swamped with my bachelor's project at the moment.

(@sdogruyol noticed this)
After this commit, on Arch Linux:
$ make clean std_spec
Using /usr/bin/llvm-config [version=6.0.0]
rm -rf .build
rm -rf ./docs
rm -rf src/llvm/ext/llvm_ext.o
rm -rf src/ext/sigfault.o src/ext/libcrystal.a
g++ -c  -o src/llvm/ext/llvm_ext.o src/llvm/ext/llvm_ext.cc -I/usr/include -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O3 -DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
cc -fPIC    -c -o src/ext/sigfault.o src/ext/sigfault.c
ar -rcs src/ext/libcrystal.a src/ext/sigfault.o
./bin/crystal build  -o .build/std_spec spec/std_spec.cr
_main.o: In function `__crystal_main':
/home/blaxpirit/repos/crystal/src/openssl/lib_ssl.cr:218: undefined reference to `SSL_library_init'
/home/blaxpirit/repos/crystal/src/openssl/lib_ssl.cr:218: undefined reference to `SSL_load_error_strings'
/home/blaxpirit/repos/crystal/src/openssl/lib_ssl.cr:218: undefined reference to `OPENSSL_add_all_algorithms_noconf'
/home/blaxpirit/repos/crystal/src/openssl/lib_ssl.cr:218: undefined reference to `ERR_load_crypto_strings'
_main.o: In function `~procProc(Pointer(Void), Nil)@src/openssl/ssl/hostname_validation.cr:66':
/home/blaxpirit/repos/crystal/src/raise.cr:202: undefined reference to `sk_free'
_main.o: In function `~proc2Proc(Pointer(Void), Nil)@src/openssl/ssl/hostname_validation.cr:66':
/home/blaxpirit/repos/crystal/src/raise.cr:202: undefined reference to `sk_free'
_main.o: In function `~proc3Proc(Pointer(Void), Nil)@src/openssl/ssl/hostname_validation.cr:66':
/home/blaxpirit/repos/crystal/src/raise.cr:202: undefined reference to `sk_free'
_main.o: In function `~proc4Proc(Pointer(Void), Nil)@src/openssl/ssl/hostname_validation.cr:66':
/home/blaxpirit/repos/crystal/src/raise.cr:202: undefined reference to `sk_free'
_main.o: In function `~proc5Proc(Pointer(Void), Nil)@src/openssl/ssl/hostname_validation.cr:66':
/home/blaxpirit/repos/crystal/src/raise.cr:202: undefined reference to `sk_free'
_main.o:/home/blaxpirit/repos/crystal/src/raise.cr:202: more undefined references to `sk_free' follow
O-penS-S-L-5858D-igest.o: In function `new_evp_mt_ctx':
/home/blaxpirit/repos/crystal/src/openssl/digest/digest.cr:23: undefined reference to `EVP_MD_CTX_create'
O-penS-S-L-5858D-igest.o: In function `finalize':
/home/blaxpirit/repos/crystal/src/openssl/digest/digest.cr:38: undefined reference to `EVP_MD_CTX_destroy'
O-penS-S-L-5858D-igest.o: In function `clone':
/home/blaxpirit/repos/crystal/src/openssl/digest/digest.cr:42: undefined reference to `EVP_MD_CTX_create'
/home/blaxpirit/repos/crystal/src/openssl/digest/digest.cr:44: undefined reference to `EVP_MD_CTX_destroy'
O-penS-S-L-5858S-S-L-5858C-ontext.o: In function `default_method':
/home/blaxpirit/repos/crystal/src/openssl/ssl/context.cr:4: undefined reference to `SSLv23_method'
O-penS-S-L-5858S-S-L-5858H-ostnameV-alidation.o: In function `matches_subject_alternative_name':
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:34: undefined reference to `sk_num'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:35: undefined reference to `sk_value'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
/home/blaxpirit/repos/crystal/src/openssl/ssl/hostname_validation.cr:66: undefined reference to `sk_pop_free'
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc "${@}" -o '/home/blaxpirit/repos/crystal/.build/std_spec'  -rdynamic  -lyaml -lreadline `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libssl || printf %s '-lssl -lcrypto'` `command -v pkg-config > /dev/null && pkg-config --libs --silence-errors libcrypto || printf %s ' -lcrypto'` /home/blaxpirit/repos/crystal/src/llvm/ext/llvm_ext.o `/usr/bin/llvm-config --libs --system-libs --ldflags 2> /dev/null` -lstdc++ -lgmp -lz -lxml2 -lpcre -lm -lgc -lpthread /home/blaxpirit/repos/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib`
make: *** [Makefile:112: .build/std_spec] Error 1

$ pacman -Q | grep ^openssl
openssl 1.1.0.h-1
openssl-1.0 1.0.2.o-1

Before this commit the same command passes.

Actually https://github.com/crystal-lang/crystal/pull/5973/files#diff-34d89553fe2e09c3fae0521b9f5f0037L8 broke it

That's a link to this very pull request :p

Gotta love recursion :smile:

My fault again, I found an error in the second commit where I redirect stderr to stdout then redirect both to /dev/null to suppress any errors from cc. This is obviously a silly thing to do that since we need stdout output to determine OPENSSL_VERSION. This can be fixed by reverting that commit but I think @RX14's suggestion to revert to the original solution with this hack as the backup seems saner.


Added PULL_REQUEST_TEMPLATE.md

What does this PR do?
Adds the PULL_REQUEST_TEMPLATE.md file and updates the CONTIRBUTING.md file with info about the pull request template. The goal is to make PRs look like this one.
Where should the reviewer start?
See the PULL_REQUEST_TEMPLATE.md and CONTIRBUTING.md files.
How should this be manually tested?
NA
Any background context you want to provide?
This began as a suggestion from @missbikesalot. Thank you!
Screenshots (if appropriate)
NA
Questions:

Have you connected this PR to the issue it fixes? Yes.
Does the documentation need an update? No (this does it).
Does this add new Python dependencies? No.

@RandyLevensalor and @medberry I made the suggested changes and this PR is ready for review again.


Interface cannot be resolved: BoDi.IObjectContainer

Reqnroll Version
2.2
Which test runner are you using?
NUnit
Test Runner Version Number
2.2
.NET Implementation
.NET Framework 4.6.2 or later
Test Execution Method
ReSharper Test Runner
Content of reqnroll.json configuration file
None
Issue Description
When attempting to start a test in a project newly converted from the legacy SpecFlow framework, this error is generated: Interface cannot be resolved: BoDi.IObjectContainer
The trace seems to provide no reference back to the line of test code that it came from.
Stacktrace:

Reqnroll.BoDi.ObjectContainerException : Interface cannot be resolved: BoDi.IObjectContainer
TearDown : Reqnroll.BoDi.ObjectContainerException : Interface cannot be resolved: BoDi.IObjectContainer
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.<>c__DisplayClass3_0.<ResolvePerContext>b__1()
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.ExecuteWithLock(Object lockObject, Func`1 getter, Func`1 factory, ResolutionList resolutionPath, TimeSpan timeout)
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.ResolvePerContext(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Reqnroll.Infrastructure.TestExecutionEngine.<InvokeHookAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnTestRunStartAsync>d__32.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunner.<OnTestRunStartAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunnerManager.<FireTestRunStartAsync>d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunnerManager.<OnTestRunStartAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Tests_NUnitAssemblyHooks.<AssemblyInitializeAsync>d__0.MoveNext() in obj\Debug\NUnit.AssemblyHooks.cs:line 20
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUp(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformOneTimeSetUp()
--TearDown
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.<>c__DisplayClass3_0.<ResolvePerContext>b__1()
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.ExecuteWithLock(Object lockObject, Func`1 getter, Func`1 factory, ResolutionList resolutionPath, TimeSpan timeout)
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.ResolvePerContext(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Reqnroll.Infrastructure.TestExecutionEngine.<InvokeHookAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnTestRunEndAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunner.<OnTestRunEndAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunnerManager.<FireTestRunEndAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunnerManager.<DisposeAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunnerManager.<OnTestRunEndAsync>d__47.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Tests_NUnitAssemblyHooks.<AssemblyCleanupAsync>d__1.MoveNext() in obj\Debug\NUnit.AssemblyHooks.cs:line 28
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NUnit.Framework.Internal.ValueTaskAwaitAdapter.NonGenericAdapter.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunTearDown(TestExecutionContext context)

Steps to Reproduce
Not known
Link to Repro Project
No response

Could you please retry with 2.2.1?

I retried using 2.2.1, but the problem is unchanged.

Would it be possible for you to post a simplified example that reproduces the problem?

Would it be possible for you to post a simplified example that reproduces the problem?

Possibly. I'll try to do that tomorrow.

I tried creating a highly simplified project, but that project runs tests successfully.
I'll try again, but in the meantime, are there any ways I can attempt to debug the original project?

Sure. You'll need to download the Reqnroll source code (or fork) from github. Build in DEBUG.
Next, remove the nuget package references to Reqnroll from your project.
Then you have two options:

Setup a "local" nuget source that points to {path_to_Reqnroll_source}/GeneratedNuGetPackages and then add a reference to Reqnroll.NUnit.2.2x-local (where 'x' is a digit higher than the current release#)
Add a direct reference to the Reqnroll assemblies via the Add Project Reference dialog (add references to the Reqnroll.dll and Reqnroll.NUnit.ReqnrollPlugin.dll from the /Plugins/Reqnroll.NUnit.ReqnrollPllugin/bin/Debug/netstandard2.0 folder)

We're going to be interested in knowing which Hook is being executed when this error occurs and what the arguments are to the constructor of that Hook.
The exception message being shown above is "interesting" in the head-scratching way because it is saying that the object container cannot resolve itself (which should be impossible).
Does your code modify any interface/object registrations?

@clrudolphi here's a new stacktrace with this built configuration.
And no, I don't think we're modifying any registrations. Are there various ways to do that, or is there one easy to spot method call I can search for?
OneTimeSetUp: Reqnroll.BoDi.ObjectContainerException : Interface cannot be resolved: BoDi.IObjectContainer
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.<>c__DisplayClass3_0.<ResolvePerContext>b__1() in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 148
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.ExecuteWithLock(Object lockObject, Func`1 getter, Func`1 factory, ResolutionList resolutionPath, TimeSpan timeout) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 252
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.ResolvePerContext(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 142
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 215
   at Reqnroll.BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 629
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 547
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, String name) in C:\source\github\Reqnroll\Reqnroll\BoDi\ObjectContainer.cs:line 532
   at Reqnroll.Infrastructure.TestObjectResolver.ResolveBindingInstance(Type bindingType, IObjectContainer container) in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestObjectResolver.cs:line 10
   at Reqnroll.Infrastructure.TestExecutionEngine.ResolveArgument(IObjectContainer container, IBindingParameter parameter) in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 431
   at Reqnroll.Infrastructure.TestExecutionEngine.<>c__DisplayClass54_0.<ResolveArguments>b__0(IBindingParameter p) in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 421
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Reqnroll.Infrastructure.TestExecutionEngine.ResolveArguments(IHookBinding hookBinding, IObjectContainer currentContainer) in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 421
   at Reqnroll.Infrastructure.TestExecutionEngine.<InvokeHookAsync>d__50.MoveNext() in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 354
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext() in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 325
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext() in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 341
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnTestRunStartAsync>d__32.MoveNext() in C:\source\github\Reqnroll\Reqnroll\Infrastructure\TestExecutionEngine.cs:line 122
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Reqnroll.TestRunner.<OnTestRunStartAsync>d__10.MoveNext() in C:\source\github\Reqnroll\Reqnroll\TestRunner.cs:line 27
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Reqnroll.TestRunnerManager.<FireTestRunStartAsync>d__36.MoveNext() in C:\source\github\Reqnroll\Reqnroll\TestRunnerManager.cs:line 167
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Reqnroll.TestRunnerManager.<OnTestRunStartAsync>d__48.MoveNext() in C:\source\github\Reqnroll\Reqnroll\TestRunnerManager.cs:line 365
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Tests_NUnitAssemblyHooks.<AssemblyInitializeAsync>d__0.MoveNext() in C:\source\ADOS\Tools\FSF\Tests\obj\Debug\NUnit.AssemblyHooks.cs:line 20
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUp(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeTestCommand.Execute(TestExecutionContext context)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformOneTimeSetUp()

Have you attempted to debug into this hook method? Does the exception throw before entry into the method body? If you do get in to the method body, is the container parameter null?

Setting a breakpoint in this hook doesn't work. The debugger never starts, I presume because this hook doesn't get hit before the exception?

Do you have any other BeforeTestRun hooks in this solution? Any with a negative Order or where Order also equals 0? would be interested to see what they do as well.

No other BeforeTestRun hooks.

Are you using any plugins to Reqnroll?

No plugins.

Oh... I think we missed the most trivial solution! I have just noticed the namespace of the unresolved interface: BoDi.IObjectContainer... 😎
@ccarmannt I'm pretty sure the problem is that you still have a reference to the BoDi package. We have merged that into the Reqnroll package, and the namespace is now Reqnroll.BoDi instead of BoDi! So make sure you remove the BoDi package and change all namespace usings from BoDi to Reqnroll.BoDi.
Please confirm if it helped. 🤞

Indeed, that fixed the problem. Thank you both! @gasparnagy @clrudolphi


Failure on Webkit Mac App

Tested on Safari, Chrome and Webkit Mac app.
The file on this link: https://reminis-pip-pab.s3.amazonaws.com/DW_ET.jpg
It gives "Invalid JPEG metadata: Invalid segment size." error on Webkit Mac App. It is in function parseMetaData(file, callback, options, data) line 137
It gives a strange file attached below on load. It's 4 bytes jpeg. and canvas.toBlob gives null blob.

There is no problem with Chrome and Safari with the same code.

Hi @ozgurozkan123, sorry for the late reply.
Unfortunately the file you provided is not accessible anymore.
Can you upload it somewhere more permanent so we can have a look?

I have the same problem with all my images =(
Chrome on Windows

@darkvovich
The Invalid JPEG metadata: Invalid segment size. error is logged if the length defined in a JPEG APPn segment exceeds the size of the maxMetaDataSize option or the size of the file.
It seems like the image blob generated from a Canvas for your picture contains a JPEG APP2 segment that is larger than what is defined as maxMetaDataSize for image head replacement (currently 256 bytes).
I'm not sure why the Canvas generated image contains a bigger sized meta data segment for your image (since canvas data does not retain JPEG meta data), but I'll release a fix that raises this to 1024 bytes.

Thx!


SteamVR doesn't work on higher resolution HMD

Description of issue
When using a 2560x1440 display, I was unable to get SteamVR working in extended mode, I was able to get the compositor working but SteamVR still said "not ready", room setup didn't work etc.
When I set my HMD resolution to 1920x1080 and scaled it using the GPU (I believe this is a Nvidia only option) everything worked immediately. It isn't clear if this is a SteamVR or SteamVR-OSVR problem.
Steps to reproduce the problem
Use a high resolution HMD
System configuration
GTX 980, latest drivers, latest SteamVR, latest SteamVR-OSVR

@robsonswiss Would you mind giving the latest release a try? It's designed to fix problems with the display orientation and I think that might help. Even if it doesn't, i'll get us to a common base line and we can continue troubleshooting from there. Thanks!

I got same problem with latest release as well. :((
May be my configs helps you
Vrvana_Totem.zip
tv1.extended.landscape.zip

@bagobor Could you also attach a copy of the vrserver.txt file found in the Steam\Logs directory (wherever you have Steam installed)?

@godbyk all steamvr logs attached.
steamvr_logs.zip

@godbyk Do I need to manually edit SteamVR config options? Do you have any examples for that?

@bagobor The vrserver.txt log that you attached appears to be incomplete. What is there looks correct, though.
Since you're not using an OSVR HDK, you will need to modify the steamvr.vrsettings file. This file should exist in the \Program Files (x86)\Steam\config directory. If it doesn't exist, you can create it. You will need to set the displayName property to match a portion of the name of your HMD's display name as provided by the product name EDID value.  The vrserver.txt log file should show you the detected display names (if your HMD is in extended mode).
Here is a minimal example steamvr.vrsettings file that you can use as a template:
{
    "jsonid" : "vrsettings",
    "steamvr" : {
        "activateMultipleDrivers": true
    },
    "driver_osvr": {
        "verbose": false,
        "displayName": "OSVR",
        "scanoutOrigin": "upper-left"
    }
}

The verbose option enables or disables verbose logging of the SteamVR-OSVR driver. If you set this to true, it will print out a lot of debugging information to the vrserver.txt file.
The scanoutOrigin option is the position of the first pixel drawn to the screen (e.g., the upper-left pixel of the Windows desktop) when you're wearing the HMD. For example, the OSVR HDK 2 has the display installed upside-down, so the scanoutOrigin is set to lower-right for that display because the upper-left pixel of the desktop appears at the lower-right corner of the screen when you're wearing the OSVR HDK. If you leave this setting out, it will default to upper-left.
The displayName option, as mentioned above, is a substring that matches the display you wish to use for SteamVR. It should match the EDID product name of your HMD.
For more SteamVR-OSVR configuration settings, see the steamvr.vrsettings file which shows the default values.

Hi
Just thought I'd mention that this is no longer an issue, it works fine at 2160x1200.


Test PR: Test coverage with bpflsm runner included

Purpose of PR?:
Calculate coverage with bpflsm runner included.
Does this PR introduce a breaking change?
No
If the changes in this PR are manually verified, list down the scenarios covered::
Additional information for reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs
Install codecov in the repo and set the CODECOV_TOKEN in the Repository secrets.
Steps - https://docs.codecov.com/docs/quick-start
KubeArmor already has a codecov account, please ensure if the token has been set correctly.
Checklist:

[ ] Bug fix. Fixes #
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] PR Title follows the convention of  <type>(<scope>): <subject>
[ ] Commit has unit tests
[ ] Commit has integration tests

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 52.28%. Comparing base (a6e16a1) to head (b87e064).
Report is 381 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff            @@
##           main    #1819       +/-   ##
=========================================
+ Coverage      0   52.28%   +52.28%
=========================================
  Files         0       42       +42
  Lines         0    10446    +10446
=========================================
+ Hits          0     5462     +5462
- Misses        0     4149     +4149
- Partials      0      835      +835

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fixed legacyScreenSaver bug from macos Sonoma. Fix is provided by Már…

…ton Braun https://zsmb.co/building-a-macos-screen-saver-in-kotlin/
What was done?

Added ...
Fixed ...
Updated ...
...

Why?
This was mainly done because ...
Screenshots
If there's any relevant screenshot please attach it here.

Would love to get this merged if it will fix this for Sonoma on M1 macs!


Training/Building grammar for a language different than English

Considering that :
"We have provided the cascade of grammars used in the Berkeley Parser for English."
Is there a way to obtain grammars for other languages for which the grammar is already created ?
I've downloaded Berkeley Parser grammars, is there a way to obtain a list of grammars for Puck ?
Thanks

There's only english right now.
On Wed, Mar 25, 2015 at 12:35 PM, kk00ss<EMAIL_ADDRESS>wrote:

Considering that :
"We have provided the cascade of grammars used in the Berkeley Parser for
English."
Is there a way to obtain grammars for other languages for which the
grammar is already created ?
I've downloaded Berkeley Parser grammars, is there a way to obtain a list
of grammars for Puck ?
Thanks
—
Reply to this email directly or view it on GitHub
https://github.com/dlwh/puck/issues/5.

can we use the e.g. German grammar (ger_sm5.gr)  provided by Berkeley Parser for puck? -  after converted to text format?

i think it won't work all that well on unknown/rare words because of the
way I did the lexicon, but otherwise it should. If it basically works, I
can help with getting the lexicon patched in.
On Wed, Aug 24, 2016 at 1:55 AM, JimSw2016<EMAIL_ADDRESS>wrote:

can we use the e.g. German grammar (ger_sm5.gr) provided by Berkeley
Parser for puck? - after converted to text format?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dlwh/puck/issues/5#issuecomment-241999384, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAloSQM1K_PrprRR-qeMi-__9SKsHJqks5qjAcBgaJpZM4D0wXP
.

HI David,
I have compared the extracted text files of
(a) ger_sm5.grammar -> same format as Puck's wsj_2.gr.binary
(b) ger_sm5.lexicon-> same format as Puck's wsj_2.gr.lexicon
(c) ger_sm5.splits-> same format as Puck's wsj_2.gr.hierarchy
(d) ger_sm5.words -> same format as Puck's wsj_2.gr.words
If you have time, do consider creating the missing
num.binary
num.unary
numstates
unary
If this process works, this will create NEW POSSIBILITIES for BerkeleyParser communities through GPU which you pioneered.

Did anything happen here? Does it work for German now?


Add readme and Pipelines dir

Added a pipelines dir with a README.
Added a top-level README

Looks good.


pulumi: 2.2.1 -> 2.6.1

Motivation for this change
Update pulumi to version 2.6.1,
update pulumi plugins to the latest version,
improve update.sh script to work on Darwin and generate data.nix automatically.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

LGTM, I have a similar diff

@veprbl @offlinehacker please review


Update external label for location exca

closes #2405

Deployed and populated locations on bibdata-staging
see location exca


Multi-AWS-accounts connection with Azure Security Center

From the documentation site, there's shown the method of integrating with one AWS accounts. What if there are use cases where I have to integrate with multiple accounts (~100 for example) across AWS Organization under Control Tower?
Do I have to add them one by one? Or can I utilize only one account that aggregates all the Security Hub findings across the organization and use that account as an endpoint for Azure Security Center? If so, will SSMAutomation workable in this case?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

 We are actively investigating and will get back to you soon.

Hi @andylim0221, thanks for the feedback and for taking the time to help us improve the docs.
Currently, Security Center doesn't support onboarding at-scale. But the good news is that this is something we're working on, and it's planned to arrive in the near future.
In the meantime, you can aggregate all findings into a single event hub and connect that.
Howver, SSMAutomation will not work on the entire scope (you'll need to give access on each account)
I'm going to close this issue, but feel free to comment again here or open a new one if you have additional requests.
#please-close


Help, the plugin is not starting issue.

I am currently using a plugin, and although I have correctly configured the content, the plugin does not activate when I run hexo g. Additionally, the aiabstact field is not generated in the Front-Matter of my articles. Since the services of OpenAI are not accessible in my region, I am using a proxy API provided by another project. This proxy API works normally for other projects, so I am not clear why this plugin is not activated.

Okay, thank you very much.


Website doesn't change "macOS and Linux versions available" dynamically

I want to close this project. Take a look at the explanation: #101
If you still need this project, please, fork the repo.


Create Dockerfile

Dockerfile to create monocle docker container based on Alpine.

It fails to build on my Mac Docker version 20.10.21, build baeda1f.
#7 65.77 error: failed to compile `monocle v0.4.0`, intermediate artifacts can be found at `/tmp/cargo-installSSgwAw`
#7 65.77
#7 65.77 Caused by:
#7 65.77   package `clap_lex v0.4.1` cannot be built because it requires rustc 1.64.0 or newer, while the currently active rustc version is 1.60.0

Here is a version with debian slim base images.
# select build image
FROM rust:1.68-slim-bookworm as build

RUN DEBIAN=NONINTERACTIVE apt update; apt install -y pkg-config libssl-dev; rm -rf /var/lib/apt/lists/*

# create a new empty shell project
RUN USER=root cargo new --bin my_project
WORKDIR /my_project

# copy your source tree
COPY ./src ./src
COPY ./Cargo.toml .
COPY ./Cargo.lock .

# build for release
RUN CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse cargo build --release

# our final base
FROM debian:bookworm-slim

# copy the build artifact from the build stage
COPY --from=build /my_project/target/release/monocle /usr/local/bin/monocle

RUN DEBIAN=NONINTERACTIVE apt update; apt install -y libssl-dev ca-certificates; rm -rf /var/lib/apt/lists/*

# have a local cache of AS information
RUN monocle whois 400644

It does not have a default ENTRYPOINT and thus you can run something like
docker run --rm monocle:debian-bookworm monocle parse http://archive.routeviews.org/route-views5/bgpdata/2023.04/UPDATES/updates.20230409.0000.bz2


Export (Mac) not handling relative paths with a ~

Describe the bug
while exporting (r-click, export sp-config), the default path/filename doesnt work, and ~/Desktop/file.xml fails.
To Reproduce
Using VSCode for mac:
right click on a source
Select "Export SP-Config"
in the "Enter the file name" box, its prepopulated with some data.  hit enter
"Error running command..." error
try again, enter ~/Desktop/file.json
"Error running command"
try again, with /usr/admin/Desktop/file.json
Success
Expected behavior
the "Default" filename should provide a valid export, saved... somewhere.
relative paths on mac using the ~ operator should be successful, i.e. ~/Desktop/filename.json
Screenshots

Environment
Provide a screenshot of Help > About.
Logs
[Extension Host] Writing to  identitynowconfig-TENANT-sb.identitynow.com-NetSuite INTQA-2023-03-03T14-27-23.json
Error running command vscode-sailpoint-identitynow.export-node-config.view: A system error occurred (EROFS: read-only file system, open 'identitynowconfig-TENANT-sb.identitynow.com-NetSuite-2023-03-03T14-27-23.json'). This is likely caused by the extension that contributes vscode-sailpoint-identitynow.export-node-config.view.

@yannick-beot-sp  that is entirely fair.
is there any other way that the Default "export" can be a valid path, then?
in my mind, "Export... Enter!"
I'm not sure what type of default save-to location i would even propose, but as we are populating a filename into the default value, my assumption would be fore it to work.
or even to replace the default filename with
(unix) \path\to\directory\filename.xml)
(windows) c:\filepath\filename.xml
or, put another way:
if we cant provide a valid path, at least provide a contextual breadcrumb so the user isnt stumbling to find out Why their export is failing.

^ good point re: workspace.
When i first open VSCode, there is no open workspace.
then i begin interacting with the IdentityNow module, try to export, and

you are correct. with a workspace open, it auto-populates:

I started to work on being able to set the default path proposition in the settings
cf. https://github.com/yannick-beot-sp/vscode-sailpoint-identitynow/blob/63aaef66625924217aadd0d17c2eb0f572d5e62b/package.json#L622
The idea is to extend what I just did for new reports to SP-Config exports.
With this capability, you could hard code a path to export to (within your home dir or not), set the architecture of files and folders you want, have dates or not in the filename, etc.

Releases as part of https://github.com/yannick-beot-sp/vscode-sailpoint-identitynow/releases/tag/releases%2Fv0.0.19


fatal: could not read Password for<EMAIL_ADDRESS>No such device or address

Hello, I cannot get it to work. What I am trying to accomplish is copying the changelog from one repo into another, within an organization. I have ensured that my PAT does not have a space in it, and I have tried both ${{ secrets.GITHUB_TOKEN }}, and my PAT with all permissions granted.
Any ideas?

I misspelled my repo name.

I can confirm that misspelling the branch name will result in the same misleading error message (could not read Password for<EMAIL_ADDRESS>No such device or address).


execution error

I'm unable to run the code even after importing selenium.webdriver & dotenv, and I was also not able to run the "pip install -r requirements.txt" command on cmd. And then it showed me to add environment variables but after adding it still didn't work.

I just posted a revised version


Bugfix with namespaces

For a schema of metadata, namespaces were not created. Because of this, there was a difference between the current schema and the schema from the metadata when you run doctrine:migrations:diff command.

Do I need to add any more tests?


How unique is uniqueGlobalDeviceIdentifier?

I have a database of users which are identified by their udid and I'm now transitioning to your uniqueGlobalDeviceIdentifier. I'm wondering, is it possible that one of these new identifier ends up being the same as an old udid (but belonging to a different user)?
The udid used to be SHA1(SerialNumber + IMEI + WiFiAddress + BluetoothAddress), whereas this new identifier is MD5(MAC address). I'm not too familiar with how SHA1 and MD5 work, but is it a possibility that, even though all the variables are different, the end result ends up being the same, since they are encrypted using different algorithms?

Yes that could be possible.


Where do I place Custom Maps?


slow training of LSTM

Hi, I just started using keras. Awesome work!
I tried to use LSTM with the following code
model = Sequential()
model.add(LSTM(4096, 512, return_sequences=True))
model.add(TimeDistributedDense(512, 4096))
model.add(Activation('time_distributed_softmax'))
model.compile(loss='categorical_crossentropy', optimizer='rmsprop')

And I compared the efficiency with char-rnn, and found that the implementation in keras is about 4 times slower than Karpathy's (with the same batchsize).
Am I doing something wrong?
I've attached the theano profile result
Thanks you!

Training time is heavily dependent on network size and batch size. Is it even the same network (size included) at all?
Also, time_distributed_softmax is deprecated now, use softmax instead.

Yes. I tested with one layer LSTM with the same sequence length, memory cell size, input dimension, output dimension and batch size as well.
Thank you for pointing out the depreacated softmax usage.

From the profile, it seems it spent much time on operation: Elemwise{Composite{(i0 * log((i1 / i2)))}}

And what batch size are you using?

I'm using the batch size of 128.

From the profile, it seems it spent much time on operation: Elemwise{Composite{(i0 * log((i1 / i2)))}}

Thanks for the info. Try setting the inner_activation parameter to theano.tensor.nnet.ultra_fast_sigmoid, and tell me what you get.

Hi, in activations.py I added
def ultra_fast_sigmoid(x):
    return T.nnet.ultra_fast_sigmoid(x)

and rebuilt keras and then call LSTM with LSTM(4096, 512, return_sequences=True, inner_activation='ultra_fast_sigmoid')
However, it failed to compile with the error:
Using gpu device 1: Tesla K20m
Build model...
Traceback (most recent call last):
  File "fc_rnn.py", line 40, in <module>
    model.compile(loss='categorical_crossentropy', optimizer='rmsprop')
  File "build/bdist.linux-x86_64/egg/keras/models.py", line 316, in compile
  File "build/bdist.linux-x86_64/egg/keras/optimizers.py", line 75, in get_updates
  File "build/bdist.linux-x86_64/egg/keras/optimizers.py", line 24, in get_gradients
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 547, in grad
    grad_dict, wrt, cost_name)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1288, in _populate_grad_dict
    rval = [access_grad_cache(elem) for elem in wrt]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1093, in access_term_cache
    input_grads = node.op.grad(inputs, new_output_grads)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/scan_module/scan_op.py", line 1851, in grad
    _dC_dinps_t = compute_gradient(Xt, dC_dXt)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/scan_module/scan_op.py", line 1783, in compute_gradient
    return_disconnected='None')
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 547, in grad
    grad_dict, wrt, cost_name)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1288, in _populate_grad_dict
    rval = [access_grad_cache(elem) for elem in wrt]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 953, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1246, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/gradient.py", line 1093, in access_term_cache
    input_grads = node.op.grad(inputs, new_output_grads)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/tensor/elemwise.py", line 672, in grad
    rval = self._bgrad(inputs, ograds)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/tensor/elemwise.py", line 747, in _bgrad
    scalar_igrads = self.scalar_op.grad(scalar_inputs, scalar_ograds)
  File "/home/archy/.local/lib/python2.7/site-packages/Theano-0.7.0-py2.7.egg/theano/scalar/basic.py", line 890, in grad
    self.__class__.__name__)
theano.gof.utils.MethodNotDefined: ('grad', <class 'theano.tensor.nnet.sigm.UltraFastScalarSigmoid'>, 'UltraFastScalarSigmoid')

I'm new to theano and have no idea how to fix this. And I'm running with THEANO_FLAGS=mode=FAST_RUN,device=gpu1,floatX=float32,nvcc.fastmath=True,allow_gc=False python rnn.py .
Maybe there's something wrong in this function?

Hi, I replaced both sigmoid and tanh activations with linear, and get the profile.

I tried running the same network, but I'm getting completely different profiles (on CPU):
Ops
---
<% time> <sum %> <apply time> <time per call> <type> <#call> <#apply> <Op name>
  44.8%    44.8%      18.078s       9.42e-02s     C      192       12   Dot22
  14.3%    59.1%       5.754s       3.60e-01s     Py      16        1   forall_inplace,cpu,grad_of_scan_fn}
   6.2%    65.3%       2.503s       1.56e-01s     C       16        1   Elemwise{Composite{(i0 * log((i1 / i2)))}}
   4.9%    70.1%       1.960s       1.02e-02s     C      192       12   Elemwise{Composite{(i0 - ((i1 * i2) / sqrt((i3 + i4 + i5))))}}
   4.8%    74.9%       1.937s       1.21e-01s     Py      16        1   forall_inplace,cpu,scan_fn}
   3.7%    78.7%       1.503s       1.88e-02s     C       80        5   Sum{axis=[0, 1], acc_dtype=float64}

I profiled on CPU as well, and the CPU profile also shows that the  Dot22 costs most

i noticed the result here is in a different, larger regime than i've been operating in. so i wanted to share a result that is more in the range that most people have been running char-rnn.
on my machine (nvidia 750m) for the first epoch (228 minibatches) i see:

char-rnn is 0.20s/minibatch
keras is 0.28s/minibatch

this is using the following parameters (based on char-rnn's defaults):

using the nietzsche dataset
128 units per layers
2 layers
2e-3 learning rate
0.95 decay rate for rmsprop
50 token sequence length
50 token step size (no overlap)
50 "sentence" batch size
gradient clip at 5
no dropout (dropout did not seem to have a significant effect for char-rnn or keras)

i haven't done any profiling. this is just based on modifying lstm_text_generation.py to more closely match char-rnn https://github.com/kylemcdonald/keras/blob/char_rnn/examples/char_rnn.py
i also haven't compared the loss curves between keras and char-rnn after a fixed number of iterations. i'll try that next. presumably they will vary a bit based on other internal parameters...

@kylemcdonald , in the original version of char-rnn, it predicts character by character, not predicting the next character of a sequence (in your case 50). Your version of # the last layer has return_sequences=False might be not comparable to the original char-rnn?

50 "sentence" batch size

Playing around with the batch size might allow you to get some speedup on GPU. Batch sizes that are too small mean that a lot of time will be spent moving minibatch data to and from the GPU memory.

Your version of # the last layer has return_sequences=False might be not comparable to the original char-rnn?

So how does char-rnn work exactly? I must say I've never looked at it. In any case I would not expect much computational difference between return_sequences=False and return_sequences=True since in both cases the full sequence is being computed anyway. The computation steps are the exact same.

@ffmpbgrnn regarding return_sequences, this could also be done with True for all layers and then using a non-zero index on the predict() result to get the last index. i don't think it's really changing anything, as @fchollet explains above.
but i think i understand what you're saying about the prediction being different. char-rnn samples one character at a time without resetting the state of the network. the keras lstm example, however, feeds in a maxlen sequence for every step, presumably resetting the state of the net before each prediction? meaning any relationships longer than maxlen will absolutely not be captured. (if fit() is also resetting the state of the network, this is a moot point.)
i tried modifying the prediction process to see if anything happens:
        x = np.zeros((1, len(sentence), len(chars)))
        for t, char in enumerate(sentence):
            x[0, t, char_indices[char]] = 1.
        preds = model.predict(x, verbose=0)[0]
        next_index = sample(preds, diversity)
        next_char = indices_char[next_index]
        sys.stdout.write(next_char)

        for iteration in range(400):
            x = np.zeros((1, 1, len(chars)))
            x[0, 0, next_index] = 1.

            preds = model.predict(x, verbose=0)[0]
            next_index = sample(preds, diversity)
            next_char = indices_char[next_index]
...

but this just spits out complete gibberish. this makes me think the state is reset for every predict().
here is the comparison of the loss curves:

keras drops very low, and char-rnn bottoms out somewhere around epoch 20 (i don't have accuracy numbers for char-rnn).
but this graph doesn't seem to capture what's actually being learned. there's definitely something different that char-rnn is doing compared to the keras lstm example. after 20 epochs, char-rnn prints:
$ th sample.lua cv/lstm_19.00e_2x128_0.00d_50_1.4290l.t7 -temperature 1
using CUDA on GPU 0...
creating an LSTM...
missing seed text, using uniform probability over first character
--------------------------
god are questions has not already capable of the inner favourite confess or wither finders and delights--and HOPEOTOL, and then seeks he is always was even for his keens to have growt in all the science, all, in the soul of--home in so Asuated equalize, more ppodenus, that
existence, and are natured to forth and something of the lake
religious spoten of heaused by merit
of the desire them--our ofteic plumsing expluaring heredatical, relicy also, and more
deceivative
ciriums, "I purpfective excitable, it is note-side any greatred of reason
when consequently

etc.
but keras prints:
----- diversity: 1.0
----- Generating with seed: "o rich in futurity. This kind of music
expresses b"
o rich in futurity. This kind of music
expresses be intlest it of the to surm of the mort eans and no
btill se paa antate
fark, ind woord peans.
W

PoWPo)t -eeds aptel cist of beevo aloxhe coatica abol?ess, and to tame of thes to purf so:t arr-istmentated therrerepe then wath imsold to tweresce is is and in wa kesove to as mort that elustion. Nut whac allest a sukould and Bglol,. -whe worde dears
or the world, and jorbend selt atse ald alags
o

to me, the keras output looks a lot like the char-rnn output before the network is "warmed up" by outputting enough data. compare how many words are "real" between the two samples, and the distance between spaces.
in summary: if there is a way to run fit() and predict() without resetting the state of the network (which is presumably happening behind the scenes, but i can't find where) then the results might be more similar to char-rnn. and regarding this issue, it could explain the relative speed of the sampling compared to char-rnn, but not the speed of fit().
finally: thanks @fchollet for the great work on this library, for me it's at the perfect level of abstraction for the level i'm at right now, and i love the way it's designed with callbacks and lambda functions where necessary :)

@kylemcdonald it's an open issue: https://github.com/fchollet/keras/issues/98#issuecomment-122776799
I expect we'll support it within the next couple weeks. I don't think it would be hard to do, we just need somebody to look into it and, you know, you just do it. I've been busy with other things so far.

aha, great! i read through #98 hoping to be able to help, but it seems that implementing stateful training & prediction requires a deeper understanding of theano than i have right now. so i'll step back and watch from the sidelines :)


Wrong exceptions raised on failing operations

This code:
import pyfakefs.fake_filesystem
fs = pyfakefs.fake_filesystem.FakeFilesystem()
fs.CreateDirectory('/Volumes')
fs.CreateDirectory('/Volumes/ramdisk')
fs.CreateDirectory('/Volumes/ramdisk/test')
os0 = pyfakefs.fake_filesystem.FakeOsModule(fs)
component0 = "alpha"
path0 = "/Volumes/ramdisk/test"
path1 = path0 + "/alpha"
path0 = path1 + "/alpha"
os0.symlink(path0,path1)
os0.makedirs(path0)
Produces this failure:
Traceback (most recent call last):
File "bug1.py", line 13, in
os0.makedirs(path0)
File "/Users/alex/tstl/examples/pyfakefs/pyfakefs/fake_filesystem.py", line 3007, in makedirs
self.filesystem.MakeDirectories(dir_name, mode, exist_ok)
File "/Users/alex/tstl/examples/pyfakefs/pyfakefs/fake_filesystem.py", line 1801, in MakeDirectories
current_dir = current_dir.contents[component]
TypeError: string indices must be integers, not str
To match Python os module behavior, it should produce:
Traceback (most recent call last):
File "bug1os.py", line 10, in
os0.makedirs(path0)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 62] Too many levels of symbolic links: '/Volumes/ramdisk/test/alpha/alpha'
A very similar problem, not involving symbolic links, is to replace
os0.symlink(path0,path1)
with
os0.open(path1, os.O_CREAT | os.O_WRONLY | os.O_TRUNC)
that gives the failure:
Traceback (most recent call last):
File "bug2.py", line 14, in
os0.makedirs(path0)
File "/Users/alex/tstl/examples/pyfakefs/pyfakefs/fake_filesystem.py", line 3007, in makedirs
self.filesystem.MakeDirectories(dir_name, mode, exist_ok)
File "/Users/alex/tstl/examples/pyfakefs/pyfakefs/fake_filesystem.py", line 1803, in MakeDirectories
self.CreateDirectory(dir_name, mode & ~self.umask)
File "/Users/alex/tstl/examples/pyfakefs/pyfakefs/fake_filesystem.py", line 1571, in CreateDirectory
current_dir.AddEntry(new_dir)
AttributeError: 'FakeFile' object has no attribute 'AddEntry'
when to match os it should be:
Traceback (most recent call last):
File "bug2os.py", line 9, in
os0.makedirs(path0)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 20] Not a directory: '/Volumes/ramdisk/test/alpha/alpha'
These were discovered using TSTL, a test generation tool (https://github.com/agroce/tstl).

Shall be fixed, please verify.

@agroce - I'm going to close the fixed issues now, you can always reopen them if you find a related problem. Looks like there will be a lot of new issues soon...


[stable/magento] can't install magento with ingress definition

Describe the bug
Installing magento via helm fails if ingress definition is specified in values.yml or as parameter. Helm returns an error:
Error: render error in "magento/templates/ingress.yaml": template: magento/templates/ingress.yaml:8:24: executing "magento/templates/ingress.yaml" at <{{template "magento....>: template "magento.chart" not defined
Error: UPGRADE FAILED: render error in "magento/templates/ingress.yaml": template: magento/templates/ingress.yaml:8:24: executing "magento/templates/ingress.yaml" at <{{template "magento....>: template "magento.chart" not defined
I suspect magento.chart doesn't exist in the template. Ingress template could be {{ .Chart.Name }} instead.
Version of Helm and Kubernetes:
helm: v2.13.1
kubernetes: v1.11.8-gke.6
Which chart:
stable/magento
What happened:
Ingress was added but it seems to be broken
What you expected to happen:
Magento to successfully install in kubernetes with Ingress resource.
How to reproduce it (as minimally and precisely as possible):
add ingress into values.yaml (below), or run helm install --set ingress.enabled=true .:
ingress:
enabled: true
certManager: true
annotations:
kubernetes.io/ingress.class: nginx
hosts:

name: magento.local
path: /
tls: false
tlsSecret: magento.local-tls

Anything else we need to know:

Thanks for the fix! I just added the proper tags to the PR, let's wait for the bot to merge it


Adding monitoring properties results in the inability to obtain telemetry data

Describe the bug
I want to collect the request path in Mixer, so I added the following attribute in the dimensions named requirementsduration
destination_request_url_path: request.url_path | "unknown"
Add destination_request_url_path to the prometheus named handler
The monitoring data of duration cannot be obtained in grafana. Can not get the above changes can be obtained
The configuration file is as follows
apiVersion: config.istio.io/v1alpha2
kind: metric
metadata:
  clusterName: ""
  creationTimestamp: 2018-12-18T03:24:18Z
  generation: 0
  name: requestduration
  namespace: istio-system
  resourceVersion: "119836302"
  selfLink: /apis/config.istio.io/v1alpha2/namespaces/istio-system/metrics/requestduration
  uid: 6742e379-0274-11e9-bf86-00163e0d9de0
spec:
  dimensions:
    connection_security_policy: conditional((context.reporter.kind | "inbound") ==
      "outbound", "unknown", conditional(connection.mtls | false, "mutual_tls", "none"))
    destination_app: destination.labels["app"] | "unknown"
    destination_principal: destination.principal | "unknown"
    destination_request_url_path: request.url_path | "unknown"
    destination_service: destination.service.host | "unknown"
    destination_service_name: destination.service.name | "unknown"
    destination_service_namespace: destination.service.namespace | "unknown"
    destination_version: destination.labels["version"] | "unknown"
    destination_workload: destination.workload.name | "unknown"
    destination_workload_namespace: destination.workload.namespace | "unknown"
    reporter: conditional((context.reporter.kind | "inbound") == "outbound", "source",
      "destination")
    request_protocol: api.protocol | context.protocol | "unknown"
    response_code: response.code | 200
    source_app: source.labels["app"] | "unknown"
    source_principal: source.principal | "unknown"
    source_version: source.labels["version"] | "unknown"
    source_workload: source.workload.name | "unknown"
    source_workload_namespace: source.workload.namespace | "unknown"
  monitored_resource_type: '"UNSPECIFIED"'

apiVersion: config.istio.io/v1alpha2
kind: prometheus
metadata:
  clusterName: ""
  name: handler
  namespace: istio-system
  selfLink: /apis/config.istio.io/v1alpha2/namespaces/istio-system/prometheuses/handler
spec:
  metrics:
  - instance_name: requestcount.metric.istio-system
    kind: COUNTER
    label_names:
    - reporter
    - source_app
    - source_principal
    - source_workload
    - source_workload_namespace
    - source_version
    - destination_app
    - destination_principal
    - destination_workload
    - destination_workload_namespace
    - destination_version
    - destination_service
    - destination_service_name
    - destination_service_namespace
    - request_protocol
    - response_code
    - connection_security_policy
    name: requests_total
  - buckets:
      explicit_buckets:
        bounds:
        - 0.005
        - 0.01
        - 0.025
        - 0.05
        - 0.1
        - 0.25
        - 0.5
        - 1
        - 2.5
        - 5
        - 10
    instance_name: requestduration.metric.istio-system
    kind: DISTRIBUTION
    label_names:
    - reporter
    - source_app
    - source_principal
    - source_workload
    - source_workload_namespace
    - source_version
    - destination_app
    - destination_principal
    - destination_workload
    - destination_workload_namespace
    - destination_version
    - destination_service
    - destination_service_name
    - destination_service_namespace
    - request_protocol
    - response_code
    - connection_security_policy
    - destination_request_url_path
    name: request_duration_seconds
    ...

Version
k8s
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:08:19Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

istio
1.0.3

request.url_path is not available in 1.0.x release. It will be available in 1.1 release. If you check mixer log, you should be able to see statement like "unknown attributes: request.url_path".

request.url_path is not available in 1.0.x release. It will be available in 1.1 release. If you check mixer log, you should be able to see statement like "unknown attributes: request.url_path".

very Thanks， would you like tell me how to  check mixer log, and, If I want to monitor the request URL, the 1.0.3 version is not a way

With istio-telemetry pod id you can use kubectl logs -n istio-system -c mixer telemetry-pod-id to check mixer's log.
Const string value needs to be quoted with ' '. So instead of "unknown", it should be '"unknown"'.

very nice for you, I use kubectl get  metric requestduration -n istio-system, It all use  XXX | "unknown",
Use constants directly to add '

hi, I use this
destination_service_namespace: destination.service.namespace | "unknown"
or
destination_service_namespace: destination.service.namespace | '"unknown"'

An error occurred
failed to evaluate expression for field 'Dimensions[destination_request_url_path]'; failed to parse expression 'request.path | '"unknown"'': unable to parse expression 'request.path | '"unknown"'': 1:16: illegal rune literal

I see the request.path property on the 1.0 document.

in this document https://istio.io/docs/reference/config/policy-and-telemetry/attribute-vocabulary/

OK , work. use destination_request_url_path: request.url_path | "unknown",
and config Add destination_request_url_path to the prometheus named handler

this issue can be closed, very thanks


Compilation error

I was trying to compile the .LESS files of Bootstrap (from Twitter) but I've got a compilation error on these lines:
@-webkit-keyframes progress-bar-stripes {
  from  { background-position: 0 0; }
  to    { background-position: 40px 0; }
}
@keyframes progress-bar-stripes {
  from  { background-position: 0 0; }
  to    { background-position: 40px 0; }
}

I needed to comment those, so DotLess could compile.
I haven't tried another .LESS compiler, but from what I learned when posting an issue to http://github.com/twitter/bootstrap/issues/992 , maybe this is an issue with DotLess.

what version are you using? Key frame support was added in more recent
versions... (download from our github page)
On Jan 13, 2012 7:59 PM, "Nelson Reis" <
<EMAIL_ADDRESS>wrote:

I was trying to compile the .LESS files of Bootstrap (from Twitter) but
I've got a compilation error on these lines:
@-webkit-keyframes progress-bar-stripes {
 from  { background-position: 0 0; }
 to    { background-position: 40px 0; }
}
@keyframes progress-bar-stripes {
 from  { background-position: 0 0; }
 to    { background-position: 40px 0; }
}

I needed to comment those, so DotLess could compile.
I haven't tried another .LESS compiler, but from what I learned when
posting an issue to http://github.com/twitter/bootstrap/issues/992 ,
maybe this is an issue with DotLess.

Reply to this email directly or view it on GitHub:
https://github.com/dotless/dotless/issues/110

I was using version <IP_ADDRESS>, that was included in the SquishIt NuGet package.
I've seen that the DotLess NuGet package also isn't yet updated.
I went to GitHub, downloaded the <IP_ADDRESS> version and added an assembly binding redirect to the new version in the web.config. It works great. Thanks.

I'm closing this issue.

@tigraine could you update nuget? Or you can give me the details how to..
On Jan 14, 2012 3:51 PM, "Nelson Reis" <
<EMAIL_ADDRESS>wrote:

I'm closing this issue.

Reply to this email directly or view it on GitHub:
https://github.com/dotless/dotless/issues/110#issuecomment-3492978

Actually you can just create an account and upload.
At least that's what I did when I pushed a new ver and someone else
had created the nuget
Am 14.01.2012 um 17:05 schrieb Luke Page
<EMAIL_ADDRESS>
@tigraine could you update nuget? Or you can give me the details how to..
On Jan 14, 2012 3:51 PM, "Nelson Reis" <
<EMAIL_ADDRESS>wrote:

I'm closing this issue.

Reply to this email directly or view it on GitHub:
https://github.com/dotless/dotless/issues/110#issuecomment-3492978

Reply to this email directly or view it on GitHub:
https://github.com/dotless/dotless/issues/110#issuecomment-3493054


TEM2015 Test 02: original NationalIntelligence/CoastalReliefModel was not found during RemapIDsOnServices.py execution

The portal item for this service did exist in the test portal, but RemapIDsOnServices.py could not find the item. I didn't manually correct the id in the service json. I left the item owned by admin and shared the item with everyone. There is at least web map that references this service that has GUIDs in the web map data. On the test portal these GUIDs are remapped so they are correct. The only thing endusers would have to do is manually edit the service json using the ags/admin endpoint with the correct GUID and then delete the item owned by admin.
   ---------------------------------------------------------------------------
    Service: NationalIntelligence//CoastalReliefModel.ImageServer

    - Retrieving information about associated portal items stored in the server JSON...

    - Associated portal items...

       - NationalIntelligence/CoastalReliefModel/ImageServer: original item id = c9e8c329e2e64708896ab1f61279b6a4

    **** WARNING: new item id not found.

    - Updating portal item information stored within service JSON (service will be restarted automatically)...

    **** WARNING: there were no new ids found for this service so there is no need to update the service JSON info.

Publishing TEM2015 portal content to another portal to see if the Coastal Relief model service portal item URL is modified to test machine URL.

Can't reproduce this issue. Pubilshed TEM2015 portal content to another 10.3.1 portal and the server name was replaced during content publishing. Closing issue.


W2D2_Tutorial3 wrong equilibrium variance equation in Part C

In part C of W2D2_Tutorial3 the equilibrium variance of the OU process is calculated as:
Var = sig^2 / (2*(1-lambda))
However, I think that this is not the correct analytical solution of the equilibrium variance. A simple mathematical prove (for equabilirum mean=0) is shown below (the ~ sign means "follows the distribution"):
x0 = x0
x1 = lambda*x0 + N(0, sig^2) ~ N(lambda*x0, sig^2)
x2 = lambda*x1 + N(0, sig^2) ~ lambda*N(lambda*x0, sig^2) + N(0, sig^2) ~ N(lambda^2*x0, (lambda^2+1)*sig^2)
x3 = lambda*x2 + N(0, sig^2) ~ N(lambda^3*x0, (lambda^4+lambda^2+1)*sig^2)
.
.
.
xn ~ N(lambda^n*x0, (lambda^(2*n-2) + ... + lambda^4+lambda^2+1)*sig^2)

Note that:
lambda^(2*n-2) + ... + lambda^4+lambda^2+1 = (1-lambda^(2*n)) / (1-lambda^2)

And since lambda < 1, n -> inf, so lambda^(2*n) -> 0, therefore:
lambda^(2*n-2) + ... + lambda^4+lambda^2+1 = 1 / (1-lambda^2)

Therefore,
xn ~ N(lambda^n*x0, sig^2/(1-lambda^2))

So the equilibrium variance should be sig^2/(1-lambda^2) instead of sig^2/(2*(1-lambda))
And if you use this new analytical solution in exercise 3C, you can find that the fit to diagonal line is better.

Fixed, thanks!


Optimize parameters

Here are the results from the parameter optimization script for all read lengths, run on the multi-context seeds branch.
How the script was run
For reproducibility, I plan to make the script publicly available at https://github.com/NBISweden/strobealign-evaluation.
I ran it on strobealign commit c4a7f61 (from PR #388). This was the command:
for r in 50 75 100 150 200 300 500; do
    ./search.py -c c4a7f61 -x -r $r --mapping-rate-slack 1
done

--mapping-rate-slack 1 means that all parameter combinations are skipped for which the mapping rate of one of the datasets is reduced by more than 1 percentage points.
-x means that accuracy for mapping-only mode is optimized (i.e., running strobealign with -x). However, when the list of parameters has been found, the script reports also results for extension-alignment mode so that one can check how its accuracies change.

Suggested changes
Parameters are given as a tuple $(k, s, l, u)$.

Canonical read length
Before
Suggestion

50
(18, 14, -2, 1)
(18, 14, -2, 1) -- No change

75
(20, 16, -3, 2)
(20, 16, -3, 2) -- No change[^1]

100
(20, 16, -2, 2)
(19, 15, 0, 3) or maybe (18, 14, 1, 3)

150
(20, 16, 1, 7)
(21, 17, 2, 6)

250
(22, 18, 2, 12)
(22, 18, 4, 13)

400
(23, 17, 2, 12)
(24, 18, 6, 13)

I am unsure about read length 75 and 100: With these new settings, $k$ would decrease when going from read length 75 to 100, which is a bit unexpected.
[^1]: Already added as new canonical read length
Detailed results

Read length 50: Weighted SE/PE results - mapping-only
parameters       acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(16, 12, -2, -1) 63.4387 75.2934 +0.5503 +1.0594 96.121  96.112  -0.855  -0.864
(16, 12, -2, 0)  63.2349 74.9216 +0.3466 +0.6877 97.123  97.112  +0.146  +0.136
(16, 12, -2, 1)  63.2007 74.8490 +0.3124 +0.6150 97.252  97.244  +0.275  +0.268
(16, 12, -2, 2)  63.1981 74.8449 +0.3097 +0.6110 97.254  97.246  +0.277  +0.270
(17, 13, -2, 0)  63.1308 74.6860 +0.2425 +0.4520 96.977  96.969  -0.000  -0.007
(17, 13, -2, 1)  63.0462 74.5605 +0.1578 +0.3265 97.151  97.143  +0.174  +0.168
(17, 13, -2, 2)  63.0538 74.5557 +0.1654 +0.3218 97.158  97.150  +0.181  +0.174
(18, 14, -2, 0)  62.9883 74.4344 +0.0999 +0.2004 96.756  96.754  -0.220  -0.222
(18, 14, -2, 1)  62.8884 74.2339 +0.0000 +0.0000 96.977  96.976  +0.000  +0.000  *****
(18, 14, -2, 2)  62.8892 74.2215 +0.0008 -0.0124 96.992  96.992  +0.015  +0.016
(19, 15, -2, 0)  62.7956 74.0796 -0.0927 -0.1543 96.444  96.435  -0.532  -0.541
(19, 15, -2, 1)  62.6517 73.8086 -0.2366 -0.4253 96.693  96.682  -0.284  -0.294
(19, 15, -2, 2)  62.6419 73.7709 -0.2465 -0.4630 96.711  96.700  -0.266  -0.276
(20, 16, -3, -1) 62.5432 73.7019 -0.3451 -0.5320 95.983  95.981  -0.993  -0.995
(20, 16, -3, 0)  62.3455 73.3375 -0.5429 -0.8964 96.256  96.254  -0.720  -0.722
(20, 16, -3, 2)  62.3125 73.2746 -0.5759 -0.9593 96.287  96.287  -0.690  -0.689

Read length 50: Weighted SE/PE results - with extension alignment
parameters       sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(17, 13, -2, 2)  93.2023 97.3829 +0.1056 -0.0185 66.0838 80.8513 +0.2597 +0.0863 97.158  99.540  +0.181  -0.025
(17, 13, -2, 1)  93.1958 97.3838 +0.0991 -0.0175 66.0857 80.8555 +0.2617 +0.0904 97.151  99.538  +0.174  -0.027
(18, 14, -2, 1)  93.0968 97.4014 +0.0000 +0.0000 65.8240 80.7651 +0.0000 +0.0000 96.977  99.565  +0.000  +0.000  *****
(16, 12, -2, 2)  93.2631 97.3571 +0.1664 -0.0443 66.3198 80.8874 +0.4958 +0.1223 97.254  99.523  +0.277  -0.042
(18, 14, -2, 2)  93.1042 97.3955 +0.0074 -0.0058 65.8092 80.7592 -0.0148 -0.0059 96.992  99.569  +0.015  +0.004
(16, 12, -2, 1)  93.2566 97.3568 +0.1598 -0.0446 66.3178 80.8901 +0.4938 +0.1251 97.252  99.523  +0.275  -0.042
(17, 13, -2, 0)  93.0792 97.3695 -0.0176 -0.0319 66.1453 80.9034 +0.3213 +0.1384 96.977  99.489  -0.000  -0.076
(16, 12, -2, 0)  93.1696 97.3440 +0.0728 -0.0574 66.3488 80.9302 +0.5248 +0.1651 97.123  99.487  +0.146  -0.077
(18, 14, -2, 0)  92.9687 97.3935 -0.1281 -0.0079 65.9244 80.8640 +0.1003 +0.0990 96.756  99.503  -0.220  -0.062
(19, 15, -2, 1)  92.8954 97.3900 -0.2013 -0.0113 65.5049 80.6756 -0.3192 -0.0894 96.693  99.582  -0.284  +0.017
(19, 15, -2, 2)  92.9051 97.3837 -0.1917 -0.0177 65.4790 80.6566 -0.3450 -0.1084 96.711  99.588  -0.266  +0.023
(19, 15, -2, 0)  92.7360 97.3945 -0.3607 -0.0069 65.6364 80.8018 -0.1876 +0.0367 96.444  99.518  -0.532  -0.047
(20, 16, -3, 0)  92.5362 97.3404 -0.5606 -0.0610 65.1052 80.5565 -0.7189 -0.2086 96.256  99.557  -0.720  -0.008
(20, 16, -3, 2)  92.5494 97.3297 -0.5474 -0.0716 65.0667 80.5175 -0.7573 -0.2475 96.287  99.565  -0.690  +0.000
(20, 16, -3, -1) 92.3606 97.3623 -0.7362 -0.0390 65.2622 80.7295 -0.5618 -0.0355 95.983  99.482  -0.993  -0.083
(16, 12, -2, -1) 92.6051 97.1949 -0.4917 -0.2065 66.5108 81.0825 +0.6868 +0.3174 96.121  99.136  -0.855  -0.429

Read length 75: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(21, 17, -3, -1)        71.7299 82.5870 +0.0342 +0.0823 98.887  98.888  -0.258  -0.260
(20, 16, -3, 0) 71.6998 82.5136 +0.0042 +0.0089 99.119  99.123  -0.025  -0.025
(20, 16, -3, 1) 71.6971 82.5044 +0.0015 -0.0003 99.144  99.148  -0.000  -0.000
(20, 16, -3, 2) 71.6957 82.5047 +0.0000 -0.0000 99.145  99.149  +0.000  +0.000  *****
(20, 16, -3, 3) 71.6957 82.5047 +0.0000 -0.0000 99.145  99.149  +0.000  +0.000
(18, 14, -2, 1) 71.4264 82.4369 -0.2693 -0.0678 99.043  99.044  -0.102  -0.105

Read length 75: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(20, 16, -3, 2) 95.9650 98.4305 +0.0000 +0.0000 74.9245 86.7311 +0.0000 +0.0000 99.145  99.792  +0.000  -0.000  *****
(20, 16, -3, 3) 95.9650 98.4305 +0.0000 +0.0000 74.9245 86.7311 +0.0000 +0.0000 99.145  99.792  +0.000  -0.000
(20, 16, -3, 1) 95.9690 98.4292 +0.0040 -0.0014 74.9124 86.7262 -0.0121 -0.0049 99.144  99.792  -0.000  +0.000
(20, 16, -3, 0) 95.9358 98.4296 -0.0292 -0.0009 74.9443 86.7224 +0.0198 -0.0087 99.119  99.789  -0.025  -0.003
(21, 17, -3, -1)        95.7962 98.4268 -0.1689 -0.0038 74.9228 86.7618 -0.0016 +0.0307 98.887  99.756  -0.258  -0.036
(18, 14, -2, 1) 95.6803 98.3364 -0.2847 -0.0942 74.9116 86.6750 -0.0129 -0.0561 99.043  99.766  -0.102  -0.026

Read length 100: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(18, 14, 1, 3)  77.4674 86.7038 +0.6795 +0.4186 99.235  99.238  -0.367  -0.363
(17, 13, 1, 3)  77.4442 86.6853 +0.6563 +0.4001 99.266  99.267  -0.335  -0.335
(18, 14, 1, 2)  77.3444 86.6959 +0.5565 +0.4107 99.049  99.050  -0.552  -0.552
(16, 12, 1, 3)  77.3820 86.6520 +0.5941 +0.3668 99.289  99.288  -0.312  -0.314
(17, 13, 1, 2)  77.3103 86.6678 +0.5225 +0.3826 99.077  99.078  -0.524  -0.523
(16, 12, 1, 2)  77.2552 86.6521 +0.4673 +0.3669 99.092  99.090  -0.509  -0.511
(19, 15, 0, 3)  77.0658 86.4753 +0.2779 +0.1900 99.471  99.472  -0.131  -0.129
(18, 14, 0, 3)  77.0269 86.4572 +0.2390 +0.1720 99.484  99.484  -0.118  -0.118
(17, 13, 0, 3)  76.9784 86.4225 +0.1905 +0.1373 99.495  99.497  -0.107  -0.105
(20, 16, -2, 2) 76.7879 86.2852 +0.0000 +0.0000 99.601  99.602  +0.000  +0.000  *****

Read length 100: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(18, 14, 1, 3)  96.9435 98.9435 +0.0754 +0.1069 80.2780 89.7660 +0.2204 +0.1587 99.235  99.773  -0.367  -0.033
(17, 13, 1, 3)  96.9600 98.9249 +0.0919 +0.0883 80.3083 89.7473 +0.2507 +0.1400 99.266  99.770  -0.335  -0.036
(16, 12, 1, 3)  96.9284 98.8981 +0.0604 +0.0615 80.2829 89.7221 +0.2254 +0.1148 99.289  99.770  -0.312  -0.036
(19, 15, 0, 3)  96.9144 98.8975 +0.0464 +0.0608 80.1322 89.6795 +0.0747 +0.0722 99.471  99.800  -0.131  -0.007
(18, 14, 1, 2)  96.7661 98.9275 -0.1019 +0.0909 80.2349 89.7641 +0.1774 +0.1568 99.049  99.744  -0.552  -0.063
(18, 14, 0, 3)  96.9002 98.8752 +0.0322 +0.0385 80.1610 89.6567 +0.1035 +0.0494 99.484  99.796  -0.118  -0.010
(17, 13, 1, 2)  96.7713 98.9019 -0.0968 +0.0653 80.2605 89.7290 +0.2030 +0.1217 99.077  99.743  -0.524  -0.064
(17, 13, 0, 3)  96.8696 98.8480 +0.0015 +0.0114 80.1623 89.6259 +0.1047 +0.0186 99.495  99.796  -0.107  -0.011
(16, 12, 1, 2)  96.7328 98.8816 -0.1352 +0.0449 80.2497 89.7367 +0.1921 +0.1295 99.092  99.739  -0.509  -0.067
(20, 16, -2, 2) 96.8680 98.8366 +0.0000 +0.0000 80.0575 89.6073 +0.0000 +0.0000 99.601  99.806  +0.000  +0.000  *****

Read length 150: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(21, 17, 2, 6)  83.8632 90.4686 +0.1062 +0.1215 99.734  99.734  -0.027  -0.028
(20, 16, 3, 6)  83.8398 90.4549 +0.0828 +0.1079 99.717  99.716  -0.045  -0.046
(20, 16, 2, 6)  83.8274 90.4234 +0.0703 +0.0763 99.740  99.740  -0.022  -0.021
(20, 16, 2, 7)  83.8496 90.4017 +0.0926 +0.0546 99.750  99.750  -0.012  -0.012
(20, 16, 1, 7)  83.7570 90.3470 +0.0000 +0.0000 99.762  99.762  -0.000  +0.000  *****

Read length 150: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(21, 17, 2, 6)  98.0012 99.2737 +0.0169 +0.0303 86.2791 92.3460 +0.0071 +0.0317 99.734  99.781  -0.027  -0.000
(20, 16, 2, 7)  98.0302 99.2660 +0.0459 +0.0226 86.3007 92.3419 +0.0286 +0.0276 99.750  99.781  -0.012  -0.000
(20, 16, 3, 6)  97.9624 99.2791 -0.0219 +0.0357 86.2931 92.3486 +0.0210 +0.0344 99.717  99.780  -0.045  -0.001
(20, 16, 2, 6)  97.9862 99.2654 +0.0019 +0.0220 86.2931 92.3399 +0.0210 +0.0257 99.740  99.781  -0.022  -0.000
(20, 16, 1, 7)  97.9843 99.2434 +0.0000 +0.0000 86.2720 92.3143 +0.0000 +0.0000 99.762  99.781  -0.000  +0.000  *****

Read length 200: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(24, 20, 4, 10) 87.4239 91.8470 +0.0426 +0.0874 99.745  99.744  -0.005  -0.005
(23, 19, 4, 11) 87.4297 91.8294 +0.0484 +0.0697 99.747  99.745  -0.003  -0.003
(22, 18, 4, 13) 87.4156 91.7808 +0.0342 +0.0211 99.748  99.746  -0.002  -0.002
(22, 18, 2, 12) 87.3813 91.7597 +0.0000 +0.0000 99.750  99.748  +0.000  +0.000  *****
(20, 16, 4, 13) 87.3175 91.7053 -0.0638 -0.0543 99.749  99.748  -0.000  -0.000

Read length 200: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(24, 20, 4, 10) 98.4480 99.3766 +0.0196 +0.0414 89.4568 93.1151 +0.0218 +0.0208 99.745  99.750  -0.005  +0.000
(23, 19, 4, 11) 98.4582 99.3671 +0.0298 +0.0319 89.4590 93.1004 +0.0240 +0.0061 99.747  99.750  -0.003  +0.000
(22, 18, 4, 13) 98.4712 99.3537 +0.0428 +0.0185 89.4698 93.1054 +0.0349 +0.0110 99.748  99.750  -0.002  +0.000
(22, 18, 4, 12) 98.4487 99.3562 +0.0203 +0.0211 89.4660 93.1114 +0.0310 +0.0170 99.748  99.750  -0.002  +0.000
(22, 18, 2, 12) 98.4284 99.3352 +0.0000 +0.0000 89.4350 93.0943 +0.0000 +0.0000 99.750  99.750  +0.000  +0.000  *****
(20, 16, 4, 13) 98.4127 99.3271 -0.0157 -0.0081 89.4412 93.0820 +0.0063 -0.0123 99.749  99.750  -0.000  +0.000

Read length 300: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(24, 20, 4, 12) 90.7514 94.5846 +0.1400 +0.0707 99.692  99.691  -0.000  -0.000
(23, 19, 5, 13) 90.7711 94.5733 +0.1598 +0.0593 99.692  99.691  +0.000  -0.000
(23, 19, 5, 12) 90.7296 94.5720 +0.1182 +0.0581 99.692  99.691  +0.000  -0.000
(22, 18, 7, 13) 90.7140 94.5651 +0.1027 +0.0511 99.692  99.691  -0.000  -0.000
(23, 19, 4, 12) 90.7349 94.5507 +0.1236 +0.0368 99.692  99.691  +0.000  +0.000
(23, 19, 3, 12) 90.7049 94.5552 +0.0936 +0.0413 99.692  99.691  +0.000  +0.000
(22, 18, 4, 13) 90.7233 94.5468 +0.1120 +0.0328 99.692  99.691  +0.000  +0.000
(22, 18, 4, 12) 90.6926 94.5512 +0.0813 +0.0373 99.692  99.691  +0.000  +0.000
(22, 18, 5, 12) 90.6904 94.5486 +0.0791 +0.0346 99.692  99.691  -0.000  -0.000
(22, 18, 7, 12) 90.6575 94.5556 +0.0462 +0.0417 99.692  99.691  -0.000  -0.000
(22, 18, 3, 13) 90.7065 94.5422 +0.0951 +0.0282 99.692  99.691  +0.000  +0.000
(22, 18, 4, 11) 90.6315 94.5504 +0.0202 +0.0365 99.692  99.691  -0.000  -0.000
(23, 19, 3, 11) 90.6476 94.5413 +0.0363 +0.0273 99.692  99.691  +0.000  +0.000
(22, 18, 3, 12) 90.6577 94.5379 +0.0464 +0.0240 99.692  99.691  +0.000  +0.000
(21, 17, 6, 13) 90.6749 94.5292 +0.0636 +0.0153 99.692  99.691  +0.000  -0.000
(23, 19, 2, 12) 90.6634 94.5319 +0.0521 +0.0179 99.692  99.691  +0.000  +0.000
(21, 17, 5, 12) 90.6420 94.5230 +0.0306 +0.0091 99.692  99.691  -0.000  -0.000
(22, 18, 2, 12) 90.6113 94.5139 +0.0000 +0.0000 99.692  99.691  +0.000  +0.000  *****

Read length 300: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(24, 20, 4, 12) 98.7207 99.4493 +0.0601 +0.0175 92.4619 95.6001 +0.0303 +0.0337 99.692  99.691  -0.000  +0.000
(23, 19, 5, 13) 98.7216 99.4486 +0.0611 +0.0169 92.4686 95.5897 +0.0371 +0.0233 99.692  99.691  +0.000  -0.000
(23, 19, 5, 12) 98.7049 99.4498 +0.0443 +0.0181 92.4559 95.5806 +0.0244 +0.0142 99.692  99.691  +0.000  +0.000
(22, 18, 7, 13) 98.6930 99.4473 +0.0324 +0.0156 92.4516 95.5846 +0.0201 +0.0182 99.692  99.691  -0.000  +0.000
(23, 19, 4, 12) 98.7033 99.4423 +0.0427 +0.0105 92.4470 95.5845 +0.0155 +0.0181 99.692  99.691  +0.000  -0.000
(23, 19, 4, 11) 98.6870 99.4427 +0.0264 +0.0109 92.4326 95.5731 +0.0010 +0.0068 99.692  99.691  +0.000  +0.000
(22, 18, 4, 13) 98.6930 99.4399 +0.0325 +0.0081 92.4509 95.5854 +0.0193 +0.0191 99.692  99.691  +0.000  +0.000
(22, 18, 7, 12) 98.6703 99.4451 +0.0098 +0.0134 92.4344 95.5754 +0.0028 +0.0090 99.692  99.691  -0.000  +0.000
(22, 18, 5, 12) 98.6808 99.4406 +0.0203 +0.0088 92.4361 95.5781 +0.0046 +0.0117 99.692  99.691  -0.000  +0.000
(23, 19, 3, 12) 98.6826 99.4389 +0.0220 +0.0071 92.4384 95.5810 +0.0069 +0.0146 99.692  99.691  +0.000  -0.000
(22, 18, 3, 13) 98.6911 99.4366 +0.0305 +0.0049 92.4458 95.5765 +0.0143 +0.0101 99.692  99.691  +0.000  +0.000
(22, 18, 4, 12) 98.6791 99.4394 +0.0185 +0.0077 92.4351 95.5790 +0.0035 +0.0127 99.692  99.691  +0.000  +0.000
(22, 18, 3, 12) 98.6772 99.4363 +0.0166 +0.0046 92.4372 95.5819 +0.0056 +0.0155 99.692  99.691  +0.000  +0.000
(23, 19, 3, 11) 98.6741 99.4355 +0.0135 +0.0037 92.4297 95.5672 -0.0019 +0.0008 99.692  99.691  +0.000  +0.000
(21, 17, 6, 13) 98.6643 99.4370 +0.0037 +0.0052 92.4432 95.5756 +0.0117 +0.0092 99.692  99.691  +0.000  +0.000
(22, 18, 4, 11) 98.6591 99.4366 -0.0015 +0.0048 92.4134 95.5737 -0.0182 +0.0073 99.692  99.691  -0.000  +0.000
(23, 19, 2, 12) 98.6695 99.4322 +0.0089 +0.0005 92.4311 95.5719 -0.0004 +0.0055 99.692  99.691  +0.000  -0.000
(21, 17, 5, 12) 98.6512 99.4358 -0.0094 +0.0040 92.4250 95.5838 -0.0065 +0.0174 99.692  99.691  -0.000  +0.000
(22, 18, 2, 12) 98.6606 99.4318 +0.0000 +0.0000 92.4316 95.5664 +0.0000 +0.0000 99.692  99.691  +0.000  +0.000  *****

Read length 500: Weighted SE/PE results - mapping-only
parameters      acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(25, 19, 4, 13) 93.2463 95.3391 +0.1628 +0.0819 99.578  99.574  +0.000  +0.000
(25, 19, 5, 12) 93.2352 95.3397 +0.1518 +0.0825 99.578  99.574  +0.000  +0.000
(24, 18, 6, 13) 93.2172 95.3365 +0.1338 +0.0794 99.578  99.574  -0.000  -0.000
(24, 18, 5, 13) 93.2250 95.3340 +0.1416 +0.0768 99.578  99.574  +0.000  +0.000
(24, 18, 4, 13) 93.2231 95.3282 +0.1397 +0.0711 99.578  99.574  +0.000  +0.000
(25, 19, 3, 13) 93.2292 95.3211 +0.1457 +0.0639 99.578  99.574  +0.000  +0.000
(24, 18, 6, 12) 93.1847 95.3295 +0.1013 +0.0723 99.578  99.574  -0.000  -0.000
(25, 19, 3, 12) 93.1989 95.3234 +0.1155 +0.0662 99.578  99.574  +0.000  +0.000
(24, 18, 5, 12) 93.1781 95.3269 +0.0947 +0.0697 99.578  99.574  +0.000  +0.000
(24, 18, 6, 11) 93.1521 95.3260 +0.0686 +0.0688 99.578  99.574  -0.000  -0.000
(24, 18, 4, 12) 93.1863 95.3170 +0.1028 +0.0598 99.578  99.574  +0.000  +0.000
(24, 18, 5, 11) 93.1428 95.3261 +0.0593 +0.0689 99.578  99.574  +0.000  -0.000
(25, 19, 2, 13) 93.1886 95.3147 +0.1051 +0.0575 99.578  99.574  -0.000  -0.000
(24, 18, 3, 12) 93.1562 95.3096 +0.0728 +0.0525 99.578  99.574  +0.000  -0.000
(24, 18, 4, 11) 93.1424 95.3116 +0.0589 +0.0544 99.578  99.574  +0.000  +0.000
(23, 17, 5, 11) 93.1349 95.3104 +0.0514 +0.0532 99.578  99.574  +0.000  +0.000
(22, 16, 6, 13) 93.1627 95.2991 +0.0793 +0.0419 99.578  99.574  -0.000  -0.000
(23, 17, 4, 12) 93.1606 95.2940 +0.0772 +0.0368 99.578  99.574  +0.000  +0.000
(22, 16, 4, 13) 93.1598 95.2869 +0.0764 +0.0297 99.578  99.574  -0.000  +0.000
(23, 17, 6, 11) 93.1209 95.2940 +0.0375 +0.0368 99.578  99.574  +0.000  +0.000
(23, 17, 2, 12) 93.0834 95.2572 +0.0000 +0.0000 99.578  99.574  +0.000  +0.000  *****

Read length 500: Weighted SE/PE results - with extension alignment
parameters      sacc_se sacc_pe diff_se diff_pe acc_se  acc_pe  diff_se diff_pe mprt_se mprt_pe diff_se diff_pe
(24, 18, 6, 13) 98.9471 99.3628 +0.0480 +0.0202 94.6823 96.0611 +0.0372 +0.0042 99.578  99.574  -0.000  +0.000
(24, 18, 5, 13) 98.9497 99.3618 +0.0505 +0.0192 94.6672 96.0698 +0.0220 +0.0130 99.578  99.574  +0.000  -0.000
(24, 18, 7, 13) 98.9453 99.3615 +0.0462 +0.0189 94.6771 96.0688 +0.0320 +0.0120 99.578  99.574  -0.000  +0.000
(24, 18, 5, 12) 98.9447 99.3609 +0.0455 +0.0183 94.6567 96.0654 +0.0116 +0.0086 99.578  99.574  +0.000  -0.000
(24, 18, 6, 12) 98.9406 99.3615 +0.0414 +0.0189 94.6616 96.0662 +0.0165 +0.0094 99.578  99.574  -0.000  +0.000
(24, 18, 5, 11) 98.9356 99.3618 +0.0365 +0.0192 94.6723 96.0657 +0.0272 +0.0088 99.578  99.574  +0.000  -0.000
(24, 18, 7, 12) 98.9345 99.3608 +0.0354 +0.0182 94.6654 96.0724 +0.0202 +0.0156 99.578  99.574  -0.000  +0.000
(25, 19, 5, 12) 98.9419 99.3588 +0.0427 +0.0162 94.6696 96.0604 +0.0244 +0.0036 99.578  99.574  +0.000  +0.000
(25, 19, 4, 13) 98.9430 99.3580 +0.0439 +0.0154 94.6639 96.0634 +0.0188 +0.0065 99.578  99.574  +0.000  +0.000
(24, 18, 6, 11) 98.9268 99.3601 +0.0277 +0.0175 94.6574 96.0676 +0.0123 +0.0108 99.578  99.574  -0.000  +0.000
(24, 18, 4, 11) 98.9328 99.3583 +0.0337 +0.0157 94.6595 96.0644 +0.0143 +0.0075 99.578  99.574  +0.000  -0.000
(24, 18, 4, 12) 98.9381 99.3566 +0.0389 +0.0140 94.6633 96.0668 +0.0182 +0.0099 99.578  99.574  +0.000  -0.000
(24, 18, 4, 13) 98.9441 99.3547 +0.0449 +0.0121 94.6718 96.0734 +0.0266 +0.0165 99.578  99.574  +0.000  -0.000
(24, 18, 5, 10) 98.9251 99.3591 +0.0260 +0.0165 94.6514 96.0552 +0.0063 -0.0017 99.578  99.574  +0.000  -0.000
(24, 18, 4, 10) 98.9219 99.3577 +0.0228 +0.0151 94.6460 96.0627 +0.0008 +0.0059 99.578  99.574  -0.000  -0.000
(23, 17, 5, 11) 98.9233 99.3559 +0.0241 +0.0133 94.6536 96.0629 +0.0085 +0.0061 99.578  99.574  +0.000  +0.000
(25, 19, 3, 12) 98.9333 99.3529 +0.0341 +0.0103 94.6648 96.0596 +0.0197 +0.0027 99.578  99.574  +0.000  -0.000
(24, 18, 6, 10) 98.9174 99.3567 +0.0183 +0.0141 94.6547 96.0626 +0.0096 +0.0058 99.578  99.574  -0.000  +0.000
(25, 19, 3, 13) 98.9323 99.3515 +0.0332 +0.0089 94.6630 96.0695 +0.0179 +0.0127 99.578  99.574  +0.000  -0.000
(24, 18, 3, 12) 98.9263 99.3506 +0.0272 +0.0080 94.6580 96.0602 +0.0129 +0.0034 99.578  99.574  +0.000  +0.000
(23, 17, 4, 12) 98.9211 99.3519 +0.0220 +0.0093 94.6395 96.0539 -0.0056 -0.0030 99.578  99.574  +0.000  +0.000
(23, 17, 6, 11) 98.9158 99.3531 +0.0166 +0.0104 94.6512 96.0580 +0.0061 +0.0011 99.578  99.574  +0.000  +0.000
(25, 19, 2, 13) 98.9266 99.3499 +0.0275 +0.0073 94.6582 96.0497 +0.0131 -0.0071 99.578  99.574  -0.000  -0.000
(22, 16, 4, 13) 98.9109 99.3416 +0.0117 -0.0010 94.6403 96.0507 -0.0048 -0.0062 99.578  99.574  -0.000  +0.000
(22, 16, 6, 13) 98.9088 99.3420 +0.0096 -0.0006 94.6504 96.0528 +0.0052 -0.0040 99.578  99.574  -0.000  +0.000
(23, 17, 2, 12) 98.8991 99.3426 +0.0000 +0.0000 94.6451 96.0568 +0.0000 +0.0000 99.578  99.574  +0.000  +0.000  *****

I am unsure about read length 75 and 100: With these new settings,  would decrease when going from read length 75 to 100, which is a bit unexpected.

Let's try with (19, 15, 0, 3).
Once @Itolstoganov PR is ready, let’s integrate these combinations and then I'll set off a larger benchmark.


Wrong interpretation of the Shapiro-Wilk test

In the Regression_diagnostics notebook , you are presenting the Shapiro-Wilk test.
The Shapiro-Wilk test's null hypothesis is that the data come from a Gaussian distribution. Therefore, the lower the p-value, the higher the change to reject the Gaussian distribution. The notebook says the opposite:

Do you have a source article to reference? This might be good for those interested to study.

Under "Encapsulation", cell 22 you have the line:
self.fitted_ = np.dot(X,mlr.coef_) + mlr.intercept_
Where I think "mlr" should be replaced by "self":
self.fitted_ = np.dot(X,self.coef_) + self.intercept_


[TASK] Update cosmwasm to 1.0.0 rc1

Overview

[ ] update cosmwasm to latest version

Closed in #52


Do not serialize mappings with custom repr as dict-s

Fixes #1296

@sl0thentr0py can you please review this :pray:

I have fixed some tests and the style, but the CI does not trigger again.

As I see, the only test that's failing is the serialization of QueryDict in Django that has a custom __repr__ definition.
We can have a global list of class exclusions where the Django integration can append QueryDict, WDYT?

@vmarkovtsev yes I think we want to preserve older behavior for classes such as QueryDict here. I'm not sure what other things out there in the wild this will affect.
The other option here is to give you a custom def __sentry_repr__ that if available on any class will automatically short circuit the serializer and use that instead. Would that work for you?

Yes, __sentry_repr__ should work. Let's go that path then :+1:

ok I'll make another PR then

closing in favor of https://github.com/getsentry/sentry-python/pull/1322


docs: Use correct name for ls command

@jaredcbaum, not sure if we need redirects from https://capacitorjs.com/docs/cli/commands/list to https://capacitorjs.com/docs/cli/commands/ls?

@jaredcbaum, not sure if we need redirects from https://capacitorjs.com/docs/cli/commands/list to https://capacitorjs.com/docs/cli/commands/ls?

@jcesarmobile added a redirect to the vercel config


Instance and Group Normalization

Implement Group Normalization (paper) and Instance Normalization (paper)

Working on it

https://nn.labml.ai/normalization/instance_norm/index.html
https://nn.labml.ai/normalization/group_norm/index.html