Newest sklearn needs update of numpy pluginto be supported

Hi Nuitka creators,
Thanks very much first to create such a cool tool!
In the release notes of 0.6.5, it wrote that it added support for new plugin for sklearn.  I search in the code base, and find Nuitka onces had a SklearnPlugin.py but was soon deleted.  Below was what I got when try to use sklearn in my script:
My env:
D:\python\ai.dist>python -m nuitka --version
0.6.5
Python: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]
Executable: C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\python.exe
OS: Windows
Arch: x86_64
numpy version:  1.17.0
sklearn version: 0.21.3
In my python script:
#first 3 lines are work around to avoid some issue
import numpy.random.common
import numpy.random.bounded_integers
import numpy.random.entropy
import numpy as np
#Note , any of line below will cause the same error,  I checked them one by one
from sklearn.svm import OneClassSVM
from sklearn.model_selection import train_test_split
from sklearn.externals import joblib
I build my scirpt with :
python -m nuitka --plugin-enable=numpy --standalone --plugin-enable=pylint-warnings --mingw64 ai.py
and get below wanring:
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\defusedxml\common.py:83' may require use of '--i
nclude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\ipykernel\pickleutil.py:223' may require use of
'--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\ipython_genutils\importstring.py:31' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\ipython_genutils\importstring.py:39' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\jedi\evaluate\compiled\access.py:140' may requir
e use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\jinja2\loaders.py:472' may require use of '--inc
lude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\jinja2\utils.py:142' may require use of '--inclu
de-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\jinja2\utils.py:143' may require use of '--inclu
de-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Local
Programs\Python\Python36\lib\site-packages\joblib_memmapping_reducer.py:403' ma
y require use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Local
Programs\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpic
kle.py:965' may require use of '--include-plugin-directory' or '--include-plugin
-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Local
Programs\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpic
kle.py:1000' may require use of '--include-plugin-directory' or '--include-plugi
n-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Local
Programs\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpic
kle.py:1196' may require use of '--include-plugin-directory' or '--include-plugi
n-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\matplotlib\backend_managers.py:343' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\numpy\core\function_base.py:463' may require use
of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\numpy\lib\utils.py:393' may require use of '--in
clude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\numpy\lib\utils.py:892' may require use of '--in
clude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\numpy\lib\utils.py:950' may require use of '--in
clude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\pygments\formatters_init_.py:39' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\pygments\lexers_init_.py:41' may require use
of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\pygments\styles_init_.py:70' may require use
of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\pyrsistent_checked_types.py:216' may require us
e of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\scipy_lib_util.py:160' may require use of '--i
nclude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\scipy\optimize\optimize.py:3110' may require use
of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Local
Programs\Python\Python36\lib\site-packages\sklearn\utils\testing.py:660' may req
uire use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\tornado\util.py:150' may require use of '--inclu
de-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\tornado\util.py:153' may require use of '--inclu
de-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\traitlets\utils\importstring.py:34' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\traitlets\utils\importstring.py:42' may require
use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\zmq\backend\select.py:28' may require use of '--
include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\zmq\eventloop\minitornado\util.py:59' may requir
e use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\zmq\eventloop\minitornado\util.py:62' may requir
e use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\hongdiao\AppData\Roamin
g\Python\Python36\site-packages\zmq\utils\jsonapi.py:25' may require use of '--i
nclude-plugin-directory' or '--include-plugin-files'.
Nuitka:INFO:Copied 1 file from 'numpy' installation.
Then I get below error when I try to run created exe:
D:\python\ai.dist>ai.exe
Traceback (most recent call last):
File "D:\python\ai.dist\ai.py", line 14, in
File "D:\python\ai.dist\sklearn_init_.py", line 76, in
File "D:\python\ai.dist\sklearn\base.py", line 16, in
File "D:\python\ai.dist\sklearn\utils_init_.py", line 20, in
File "D:\python\ai.dist\sklearn\utils\validation.py", line 21, in
File "D:\python\ai.dist\sklearn\utils\fixes.py", line 18, in
File "D:\python\ai.dist\scipy\sparse\linalg_init_.py", line 113, in
File "D:\python\ai.dist\scipy\sparse\linalg\isolve_init_.py", line 6, in
File "D:\python\ai.dist\scipy\sparse\linalg\isolve\iterative.py", line 10, in

ImportError
If I run my python script directly with "python myScript.py" it can succeed.
Thanks again for your great work.

you must use the numpy plugin with options depending on what your script uses, eg. in your case
... --enable-plugin=numpy=scipy,sklearn

Not including these options cannoot lead to a usable standalone.
To be on the safe side, I recommend using hinted compilation. This is able to detect (hopefully) all the needs of your script and then generates Nuitka compile parameters based on that.

Just upgraded my scikit-learn to your level and trying out my test script.
I am always using hinted compilation, so I saw, that you may also need another plugin: multiprocessing.

This is to confirm, that (at least) a basic scikit-learn script can be successfully compiled and executed in standalone mode with the current scikit-learn, scipy and numpy package versions.

Hi Jorj McKie,
Thansk a lot for your kind help.
I try to use hinted compilation, but it sees to get error at the step "python nuitka-hints.py my.py":
My python is extremly simple :
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import OneClassSVM
from sklearn.model_selection import train_test_split
from sklearn.externals import joblib
print("hello world hahaha111 ")
Below are the output of hinted compilation:
D:\python\ai-sk>python get-hints.py ai-sk.py
C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packages\sklea
rn\externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib
is deprecated in 0.21 and will be removed in 0.23. Please import this functiona
lity directly from joblib, which can be installed with: pip install joblib. If t
his warning is raised when loading pickled models, you may need to re-serialize
those models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
hello world hahaha111
Call cleaning has removed 1058 items.
D:\python\ai-sk>python nuitka-hints.py ai-sk.py
NUITKA is compiling 'ai-sk.py' with these options:
--standalone
--remove-output
--recurse-none
Nuitka:INFO:User plugin 'hinted-mods.py' is being loaded.
Nuitka:INFO:'hinted-mods.py' is adding the following options:
Nuitka:INFO:--enable-plugin=numpy=scipy,sklearn
Nuitka:INFO:--enable-plugin=multiprocessing
Nuitka:INFO:--recurse-to for 640 imported modules.
Nuitka:INFO:
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" load environment.
Nuitka:INFO:Injecting plug-in based post load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" for compiled methods.
Problem with statement at ai-sk.py:1:
-> from sklearn.tree import DecisionTreeClassifier
Nuitka:INFO:Interrupted while working on '<Node 'PYTHON_MAIN_MODULE' with {'file
name': 'ai-sk.py', 'main_added': False, 'mode': 'compiled'}>'.
Traceback (most recent call last):
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka_main_.py", line 184, in
main()
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka_main_.py", line 177, in main
MainControl.main()
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\MainControl.py", line 759, in main
main_module = createNodeTree(filename=filename)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\MainControl.py", line 141, in createNodeTree
Optimization.optimize(main_module.getOutputFilename())
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\Optimization.py", line 542, in optimize
makeOptimizationPass(initial_pass=True)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\Optimization.py", line 456, in makeOptimizationPass
changed = optimizeModule(current_module)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\Optimization.py", line 167, in optimizeModule
changed = optimizeCompiledPythonModule(module)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\Optimization.py", line 100, in optimizeCompiledPythonMod
ule
module.computeModule()
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\ModuleNodes.py", line 514, in computeModule
trace_collection=self.trace_collection
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\StatementNodes.py", line 165, in computeStatementsSequence
new_statement = statement.computeStatementsSequence(trace_collection)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\FrameNodes.py", line 182, in computeStatementsSequence
new_statement = trace_collection.onStatement(statement=statement)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\TraceCollections.py", line 549, in onStatement
new_statement, change_tags, change_desc = statement.computeStatement(self)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\StatementNodes.py", line 226, in computeStatement
trace_collection.onExpression(expression=self.getExpression())
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\optimizations\TraceCollections.py", line 530, in onExpression
r = expression.computeExpressionRaw(trace_collection=self)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\ExpressionBases.py", line 1005, in computeExpressionRaw
return self.computeExpression(trace_collection=trace_collection)
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\ImportNodes.py", line 401, in computeExpression
trace_collection=trace_collection, module_name=imported_module_name
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\ImportNodes.py", line 280, in _attemptRecursion
module_package=module_package,
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\nodes\ImportNodes.py", line 199, in _consider
module_kind=module_kind,
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\importing\Recursion.py", line 181, in decideRecursion
module_filename, module_name, module_package, module_kind
File "C:\Users\hongdiao\AppData\Local\Programs\Python\Python36\lib\site-packag
es\nuitka\plugins\Plugins.py", line 251, in onModuleEncounter
module_filename, module_name, module_package, module_kind
TypeError: onModuleEncounter() takes 4 positional arguments but 5 were given

We are on the right track!
Your error is caused by the wrong Nuitka version: you need 0.6.6rc3.
I am testing your script and found several small changes required in the ImplicitImports.py module and in one of the ninted compilation modules.
Once I am done with these, I will let you know from where to download these together with the correct Nuitka version.

Hi JorMcKie,
It is so great! Thanks again for it.

Hi,
I have finished the required changes.
Please copy again the hinting modules from the Nuitka-Utilities repository. Actually, only script „hinted-mods.py“ has changed though.
From the main Nuitka repository, take branch “jorj” to replace your nuitka installation with it.
Its version is 0.6.6rc4 – please check this. Apart from some other changes (made by Kay in the factory branch) it contains updates to plugins “ImplicitImports.py” and “NumpyPlugin.py”, which were required to support the new scikit-learn package.
Kay:
With this version (containing your changes in factory from earlier today) I also successfully compiled & tested my tensorflow test script – you mentioned, that were currently unable to do this test yourself being underway on a car drive.
Jorj
Von<EMAIL_ADDRESS>Gesendet: Sonntag, 25. August 2019 10:47
An<EMAIL_ADDRESS>Cc: Jorj X<EMAIL_ADDRESS><EMAIL_ADDRESS>Betreff: Re: [Nuitka/Nuitka] sklearn plugin is not supported and get runtime error why try to run created EXE (#501)
Hi JorMcKie,
It is so great! Thanks again for it.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHubhttps://github.com/Nuitka/Nuitka/issues/501?email_source=notifications&email_token=AB7IDIQKISDQFDRBS4XLRGLQGKLQ5A5CNFSM4IPGK4V2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5CU6YQ#issuecomment-524636002, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AB7IDITITBPMWJJNHAV3PPTQGKLQ5ANCNFSM4IPGK4VQ.

You rock @JorjMcKie I am merging this all for develop now, might be interrupted though.

Ok, github develop has it, I will make another pre-release soon, but I already rebased jorj branch, so it's clean situation relative to develop again.

... and one more time without internet on an early Monday morning. So have to bother you all with an e-mailed response.
Kay: Will rebase my local repo then as soon as the internet gods will let me.
@shenzhuangwudi: as Kay said, you can now update your Nuitka from develop branch. Do not forget to also update “hinted-mods.py” if you again want to do hinted compilation.
To compile your script took 19 minutes on my machine.
Please let me know how things are working out for you.

Hi JorjMcKie,
It seems that I am not able to update Nuitka from develop branch :
C:\Users\ZXu14>pip install git+https://github.com/Nuitka/Nuitka.git@develop
Collecting git+https://github.com/Nuitka/Nuitka.git@develop
Cloning https://github.com/Nuitka/Nuitka.git (to revision develop) to c:\users
\zxu14\appdata\local\temp\pip-req-build-botvuvwl
Running command git clone -q https://github.com/Nuitka/Nuitka.git 'C:\Users\ZX
u14\AppData\Local\Temp\pip-req-build-botvuvwl'
Running command git submodule update --init --recursive -q
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython26' fa
iled
Failed to clone 'tests/CPython26'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython27' fa
iled
Failed to clone 'tests/CPython27'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython32' fa
iled
Failed to clone 'tests/CPython32'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython33' fa
iled
Failed to clone 'tests/CPython33'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython34' fa
iled
Failed to clone 'tests/CPython34'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython35' fa
iled
Failed to clone 'tests/CPython35'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython36' fa
iled
Failed to clone 'tests/CPython36'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython37' fa
iled
Failed to clone 'tests/CPython37'. Retry scheduled
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Nuitka/Nuitka-CPython-tests' into submodule pa
th 'C:/Users/ZXu14/AppData/Local/Temp/pip-req-build-botvuvwl/tests/CPython26' fa
iled
Failed to clone 'tests/CPython26' a second time, aborting
ERROR: Command errored out with exit status 1: git submodule update --init --rec
ursive -q Check the logs for full command output.
C:\Users\ZXu14>

Just a quick update,  I still get error after trying 0.6.6rc4.  I will try hinted compilation later.
sklearn version: 0.21.3
numpy version: 1.17.0
My script is very simple:
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import OneClassSVM
from sklearn.model_selection import train_test_split
from sklearn.externals import joblib
print("hello world")
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python -m nuitka --version
0.6.6rc4
Python: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit
(AMD64)]
Executable: C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\python.exe
OS: Windows
Arch: x86_64
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python -m nuitka --plugin-
enable=numpy=scipy,sklearn  --plugin-enable=pylint-warnings --plugin-enable=mult
iprocessing --standalone --mingw64 ai.py
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" load environment.
Nuitka:INFO:Injecting plug-in based post load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" for compiled methods.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\AppData\Local\Pro
grams\Python\Python36\lib\site-packages\joblib_memmapping_reducer.py:403' may r
equire use of '--include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\AppData\Local\Pro
grams\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpickle
.py:965' may require use of '--include-plugin-directory' or '--include-plugin-fi
les'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\AppData\Local\Pro
grams\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpickle
.py:1000' may require use of '--include-plugin-directory' or '--include-plugin-f
iles'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\AppData\Local\Pro
grams\Python\Python36\lib\site-packages\joblib\externals\cloudpickle\cloudpickle
.py:1196' may require use of '--include-plugin-directory' or '--include-plugin-f
iles'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\matplotlib\backend_managers.py:343' may require use of '--
include-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\numpy\core\function_base.py:463' may require use of '--inc
lude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\numpy\lib\utils.py:393' may require use of '--include-plug
in-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\numpy\lib\utils.py:892' may require use of '--include-plug
in-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\numpy\lib\utils.py:950' may require use of '--include-plug
in-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\scipy_lib_util.py:160' may require use of '--include-plu
gin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\Desktop\DrillOps
test_demo\Python_demo\scipy\optimize\optimize.py:3110' may require use of '--inc
lude-plugin-directory' or '--include-plugin-files'.
Nuitka:WARNING:Unresolved 'import' call at 'C:\Users\ZXu14\AppData\Local\Pro
grams\Python\Python36\lib\site-packages\sklearn\utils\testing.py:660' may requir
e use of '--include-plugin-directory' or '--include-plugin-files'.
Scons: Environment CC seems to be a gcc, enable mingw_mode.
Nuitka:INFO:Copied 1 file from 'numpy' installation.
Nuitka:INFO:Copied 30 files from 'scipy' installation.
Nuitka:INFO:Copied folder sklearn/datasets/data
Nuitka:INFO:Copied folder sklearn/datasets/descr
Nuitka:WARNING:Ignoring conflicting DLLs for 'concrt140.dll' and using newest fi
le version.
Nuitka:WARNING:Ignoring conflicting DLLs for 'msvcp140.dll' and using newest fil
e version.
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>cd ai.dist
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist>ai.exe
Traceback (most recent call last):
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\ai.py", li
ne 1, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\sklearn__
init__.py", line 76, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\sklearn\ba
se.py", line 13, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\numpy_in
it_.py", line 150, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\numpy\rand
om_init_.py", line 180, in
File "mtrand.pyx", line 1, in init numpy.random.mtrand
ModuleNotFoundError: No module named 'numpy.random.common'

You should be successful using hinted compilation.
It seems I have forgotten to include an implicit import which is new for numpy 1.17.0.

I have successfully tried both alternatives after that correction.
Interesting observation:
Compile time without using hinted compilation is over 1 hour and the dist folder is also much larger ...

@JorjMcKie caching is going to have to be a hot spot of my work, it only takes an implementation of generic constant unpacking. I thought I had created an issue for it, but I apparently never finished it and now it's lost.
Basically we need a form of marshal that has the ability to unpack constant data from a blob, which not only will create more compact code (many times the constant creations make compilation slow), but also will do away with the need to use indexes into the shared constant blob. That will give the same module.numpy.c for every progam on a given platform and Python minor version at least and the clcache and ccache make sense suddenly.
This means basically we need a marshal.c equivalent of CPython for all Python versions, and one that is ideally better at sharing values with itself. I will gradually get there, as this is a severe improvement of scalability. Creating large lists, many attribute names, etc. takes too much code.
Right now, every program has a module.numpy.c that will be different. And a change in a part of the program affects it. That is bad. The difference between local and shared constants, must go away, as must the variable indexes. Then I think it will be a lot more tolerable.
That will then be a third level of compilation, not only bytecode, and compiled, but splitting compiled into 2 things, where one is inter-module optimized, and one is not.
Not sure when I get to this, but it's on my mind a lot. Obviously a lot of code will have to be touched just to be equally good. And I want this code to be shared between extension modules, see the #102 ideas.
Yours,
Kay

@kayhayen - wow that sounds thrilling indeed. Just to check if I am getting the idea:
Would that mean that "interesting" packages like numpy could be submitted to some precompiler to generate module.numpy.c and put that one on stack? From where every user script compilation just fetches it?

@JorjMcKie  I tried hinted mode just now, still get errors . I download the hinted packages from Nuitka/NUITKA-Utilities  just today.  So I think I have the updated hinted scripts.
My simple script:
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import OneClassSVM
from sklearn.model_selection import train_test_split
print("hello world")
The command line history:
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist>python -m nuitka -
-version
0.6.6rc4
Python: 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit
(AMD64)]
Executable: C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\python.exe
OS: Windows
Arch: x86_64
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python get-hints.py ai.py
hello world
Call cleaning has removed 881 items.
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python nuitka-hints.py ai.
py
NUITKA is compiling 'ai.py' with these options:
--standalone
--remove-output
--recurse-none
Nuitka:INFO:User plugin 'hinted-mods.py' is being loaded.
Nuitka:INFO:'hinted-mods.py' is adding the following options:
Nuitka:INFO:--enable-plugin=numpy=scipy,sklearn
Nuitka:INFO:--enable-plugin=multiprocessing
Nuitka:INFO:--recurse-to for 593 imported modules.
Nuitka:INFO:
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" load environment.
Nuitka:INFO:Injecting plug-in based post load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" for compiled methods.

Nuitka:WARNING:Not recursing to unused '_compat_pickle'.
Nuitka:WARNING:Not recursing to unused '_compression'.
Nuitka:WARNING:Not recursing to unused '_weakrefset'.
Nuitka:WARNING:Not recursing to unused 'argparse'.
Nuitka:WARNING:Not recursing to unused 'base64'.
Nuitka:WARNING:Not recursing to unused 'calendar'.
Nuitka:WARNING:Not recursing to unused 'concurrent.futures.thread'.
Nuitka:WARNING:Not recursing to unused 'ctypes._endian'.
Nuitka:WARNING:Not recursing to unused 'email'.
Nuitka:WARNING:Not recursing to unused 'email._encoded_words'.
Nuitka:WARNING:Not recursing to unused 'email._parseaddr'.
Nuitka:WARNING:Not recursing to unused 'email._policybase'.
Nuitka:WARNING:Not recursing to unused 'email.charset'.
Nuitka:WARNING:Not recursing to unused 'email.encoders'.
Nuitka:WARNING:Not recursing to unused 'email.errors'.
Nuitka:WARNING:Not recursing to unused 'email.feedparser'.
Nuitka:WARNING:Not recursing to unused 'email.iterators'.
Nuitka:WARNING:Not recursing to unused 'email.utils'.
Nuitka:WARNING:Not recursing to unused 'encodings'.
Nuitka:WARNING:Not recursing to unused 'enum'.
Nuitka:WARNING:Not recursing to unused 'fnmatch'.
Nuitka:WARNING:Not recursing to unused 'genericpath'.
Nuitka:WARNING:Not recursing to unused 'gettext'.
Nuitka:WARNING:Not recursing to unused 'hmac'.
Nuitka:WARNING:Not recursing to unused 'http'.
Nuitka:WARNING:Not recursing to unused 'importlib._bootstrap'.
Nuitka:WARNING:Not recursing to unused 'importlib._bootstrap_external'.
Nuitka:WARNING:Not recursing to unused 'ipaddress'.
Nuitka:WARNING:Not recursing to unused 'json.decoder'.
Nuitka:WARNING:Not recursing to unused 'json.encoder'.
Nuitka:WARNING:Not recursing to unused 'multiprocessing.util'.
Nuitka:WARNING:Not recursing to unused 'nturl2path'.
Nuitka:WARNING:Not recursing to unused 'posixpath'.
Nuitka:WARNING:Not recursing to unused 'quopri'.
Nuitka:WARNING:Not recursing to unused 'reprlib'.
Nuitka:WARNING:Not recursing to unused 'runpy'.
Nuitka:WARNING:Not recursing to unused 'selectors'.
Nuitka:WARNING:Not recursing to unused 'ssl'.
Nuitka:WARNING:Not recursing to unused 'token'.
Nuitka:WARNING:Not recursing to unused 'urllib.response'.
Nuitka:WARNING:Not recursing to unused 'uu'.
Scons: Environment CC seems to be a gcc, enable mingw_mode.
Nuitka:INFO:Copied 1 file from 'numpy' installation.
Nuitka:INFO:Copied 30 files from 'scipy' installation.
Nuitka:INFO:Compile time 16 minutes.
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>cd ai.dist
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist>ai.exe
Traceback (most recent call last):
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\ai.py", li
ne 1, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\sklearn__
init__.py", line 76, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\sklearn\ba
se.py", line 13, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\numpy_in
it_.py", line 150, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\numpy\rand
om_init_.py", line 180, in
File "mtrand.pyx", line 1, in init numpy.random.mtrand
ModuleNotFoundError: No module named 'numpy.random.common'

Too bad. Do not really understand why this happens. To be sure, can you please send me the JSON file created for script by get-hints.py?
I just uploaded more implicit import for numpy into the jorj branch from where you can upgrade your numpy installation.
If you want to undertake a more adventurous but also quicker step, then directly replace your ImplicitImports.py in nuitka/plugins/standard with this one:
ImplicitImports.zip

@JorjMcKie  Please see the file below:
ai-36-win32-64.json.txt
I will try your branch later.

@JorjMcKie   I directly replace ImplicitImports.py and then do hinted compilation.  It works!

Ahh - good to hear! Thanks for the information. Did you also experience a much shorter compilation time? It was a factor of more than 3 in my case ...
Von meinem iPhone gesendet

@JorjMcKie,   I also see a factor of 3 -4 times faster when do hinted compilation.
I meet a new problem.  Now the simple "Hello world" is working.  But if I add just 1 line. the created exe fails to run.  My script:
import numpy.random.common
import numpy.random.bounded_integers
import numpy.random.entropy
import numpy as np
from sklearn.tree import DecisionTreeClassifier
import math
import csv
import random
from sklearn.svm import OneClassSVM
from sklearn.model_selection import train_test_split
import time
import os
import sys
from sklearn.externals import joblib
#without below line the created exe works fine
model = joblib.load("myModelPath")
print("hello world 222")
The steps and errors:
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python ai.py
C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn
externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is
deprecated in 0.21 and will be removed in 0.23. Please import this functionalit
y directly from joblib, which can be installed with: pip install joblib. If this
warning is raised when loading pickled models, you may need to re-serialize tho
se models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn
base.py:306: UserWarning: Trying to unpickle estimator DecisionTreeClassifier fr
om version 0.19.0 when using version 0.21.3. This might lead to breaking code or
invalid results. Use at your own risk.
UserWarning)
hello world 222
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python get-hints.py ai.py
C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn
externals\joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is
deprecated in 0.21 and will be removed in 0.23. Please import this functionalit
y directly from joblib, which can be installed with: pip install joblib. If this
warning is raised when loading pickled models, you may need to re-serialize tho
se models with scikit-learn 0.21+.
warnings.warn(msg, category=DeprecationWarning)
C:\Users\ZXu14\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn
base.py:306: UserWarning: Trying to unpickle estimator DecisionTreeClassifier fr
om version 0.19.0 when using version 0.21.3. This might lead to breaking code or
invalid results. Use at your own risk.
UserWarning)
hello world 222
Call cleaning has removed 1054 items.
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>python nuitka-hints.py ai.
py
NUITKA is compiling 'ai.py' with these options:
--standalone
--remove-output
--recurse-none
Nuitka:INFO:User plugin 'hinted-mods.py' is being loaded.
Nuitka:INFO:'hinted-mods.py' is adding the following options:
Nuitka:INFO:--enable-plugin=numpy=scipy,sklearn
Nuitka:INFO:--enable-plugin=multiprocessing
Nuitka:INFO:--recurse-to for 595 imported modules.
Nuitka:INFO:
Nuitka:INFO:Injecting plug-in based pre load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" load environment.
Nuitka:INFO:Injecting plug-in based post load code for module 'multiprocessing':
Nuitka:INFO:    Monkey patching "multiprocessing" for compiled methods.

Nuitka:WARNING:Not recursing to unused '_compat_pickle'.
Nuitka:WARNING:Not recursing to unused '_compression'.
Nuitka:WARNING:Not recursing to unused '_weakrefset'.
Nuitka:WARNING:Not recursing to unused 'argparse'.
Nuitka:WARNING:Not recursing to unused 'base64'.
Nuitka:WARNING:Not recursing to unused 'calendar'.
Nuitka:WARNING:Not recursing to unused 'concurrent.futures.thread'.
Nuitka:WARNING:Not recursing to unused 'ctypes._endian'.
Nuitka:WARNING:Not recursing to unused 'email'.
Nuitka:WARNING:Not recursing to unused 'email._encoded_words'.
Nuitka:WARNING:Not recursing to unused 'email._parseaddr'.
Nuitka:WARNING:Not recursing to unused 'email._policybase'.
Nuitka:WARNING:Not recursing to unused 'email.charset'.
Nuitka:WARNING:Not recursing to unused 'email.encoders'.
Nuitka:WARNING:Not recursing to unused 'email.errors'.
Nuitka:WARNING:Not recursing to unused 'email.feedparser'.
Nuitka:WARNING:Not recursing to unused 'email.iterators'.
Nuitka:WARNING:Not recursing to unused 'email.utils'.
Nuitka:WARNING:Not recursing to unused 'encodings'.
Nuitka:WARNING:Not recursing to unused 'enum'.
Nuitka:WARNING:Not recursing to unused 'fnmatch'.
Nuitka:WARNING:Not recursing to unused 'genericpath'.
Nuitka:WARNING:Not recursing to unused 'gettext'.
Nuitka:WARNING:Not recursing to unused 'hmac'.
Nuitka:WARNING:Not recursing to unused 'http'.
Nuitka:WARNING:Not recursing to unused 'importlib._bootstrap'.
Nuitka:WARNING:Not recursing to unused 'importlib._bootstrap_external'.
Nuitka:WARNING:Not recursing to unused 'ipaddress'.
Nuitka:WARNING:Not recursing to unused 'json.decoder'.
Nuitka:WARNING:Not recursing to unused 'json.encoder'.
Nuitka:WARNING:Not recursing to unused 'multiprocessing.util'.
Nuitka:WARNING:Not recursing to unused 'nturl2path'.
Nuitka:WARNING:Not recursing to unused 'posixpath'.
Nuitka:WARNING:Not recursing to unused 'quopri'.
Nuitka:WARNING:Not recursing to unused 'reprlib'.
Nuitka:WARNING:Not recursing to unused 'selectors'.
Nuitka:WARNING:Not recursing to unused 'ssl'.
Nuitka:WARNING:Not recursing to unused 'token'.
Nuitka:WARNING:Not recursing to unused 'urllib.response'.
Nuitka:WARNING:Not recursing to unused 'uu'.
Scons: Environment CC seems to be a gcc, enable mingw_mode.
Nuitka:INFO:Copied 1 file from 'numpy' installation.
Nuitka:INFO:Copied 30 files from 'scipy' installation.
Nuitka:INFO:Compile time 15 minutes.
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo>cd ai.dist
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist>ai.exe
C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\sklearn\externals
joblib_init_.py:15: DeprecationWarning: sklearn.externals.joblib is deprecate
d in 0.21 and will be removed in 0.23. Please import this functionality directly
from joblib, which can be installed with: pip install joblib. If this warning i
s raised when loading pickled models, you may need to re-serialize those models
with scikit-learn 0.21+.
Traceback (most recent call last):
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\ai.py", li
ne 25, in
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\joblib\num
py_pickle.py", line 598, in load
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\joblib\num
py_pickle.py", line 526, in _unpickle
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\pickle.py"
, line 1050, in load
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\pickle.py"
, line 1338, in load_global
File "C:\Users\ZXu14\Desktop\DrillOps\test_demo\Python_demo\ai.dist\pickle.py"
, line 1388, in find_class
ModuleNotFoundError: No module named 'sklearn.externals.joblib.numpy_pickle'
ai-36-win32-64.json.txt

Ok, ok, so we need a couple more implicit imports.
To not let it be an endless story, I will probably come up with many more implicit import statements and simply include everything that might be referenced in a sklearn script.
Please wait a bit to receive another version of ImplcitImports.py ..

Here is another version asannounced ... good luck!
ImplicitImports.zip

@ JorjMcKie ... WoW.. My whole application works without any problem.  Thanks a lot :)

Released as part of 0.6.6, thanks a lot @JorjMcKie this one was tough :)


Discussion about phosphor design and performance

Over on the ipython repo we have been having a discussion about the technical details and performance of phosphor. This is an attempt to migrate that discussion to the phosphor repo. Here is the original discussion:
https://github.com/ipython/ipython/pull/7997
Let's continue things here...

@sccolbert
A couple of responses from the previous issue:

Unfortunately, it's really slow in Firefox in the presence of nesting (even with the fix mentioned in the bug report): https://bugzilla.mozilla.org/show_bug.cgi?id=946167

Then the issue is a bug in Firefox. Assuming Firefox solves that issue, maybe that gives some hope to replicate the phosphor layout using CSS and some Javascript. What do you think?

I'm not sure how you are interacting with the page, so I can't replicate on my end, but here is what I get for both pages when resizing the main splitter a few times (Phosphor is on top)

I'm not sure what is the difference. I'm just resizing the "panels". Could it be a difference between Linux and Windows?

I know you are being tongue-in-cheek with this point, but I actually think this is an important issue. I've spent years building front-end frameworks for large organizations and the types of layouts and interactions that Phosphor optimizes form the foundation of nearly all production business applications. Any shortcomings in their implementation will be felt everywhere.

That's an interesting aspect. If you're going to maintain the code base, I think that can be really good, but in many organizations, those systems will be maintained by several administrators/developers. If the next guy in charge knows nothing about phosphor, he is likely to be very reluctant to inherit that code base. That's a big problem.

continuing from https://github.com/ipython/ipython/issues/7784#issuecomment-88574882
https://github.com/ipython/ipython/issues/7784#issuecomment-88574882

Like i said, breaks on travis. Could have been related to the DDoS of github over the weekend.

I'm more than happy to fix that. Supporting Travis wasn't on my roadmap until now :)

So if we want to set, say, a particular color that has some impact in phophor and whatever, we'll have to duplicate that variable logic in multiple environments

I see. Phosphor's Stylus/CSS is purposefully minimal and only includes what is needed for Phosphor to function. i.e. there aren't really any options or default styles to configure. The user of Phosphor is free (and expected) to provide their own CSS in whatever means they see fit.

Very superficial indeed. Mainly just the type package manager rubbed me the wrong way. I've worked a touch with it in the past on webcola, and ran into several tooling issues if you are outside the visual studio mothership, but that was a year ago.

It's come a long way over the last year+. Versions 1.3 and 1.4 brought heaps of nice improvements, and the NodeJS tooling continues to get better. My dev environment is NodeJS and SublimeText on a Windows box. Visual Studio is too heavy for my liking.

We're here to help, if this is what is chosen! But it does make it hard to evaluate: the generated doc is... thorough... but doesn't help me get an intuitive feel for what I should be doing. As we think about try.jupyter.org we could certainly consider using that as a sample case.

Absolutely. I'm 100% for adding user-land docs, but want to polish off the foundations first. This should be completed over the next couple weeks and will free up my time to focus on these other parts.


When will the latest version get published tp PyPy?

Now :)
https://pypi.org/project/rpi-backlight/2.5.0/


web: fix for 'include all children, as appropriate, in combined unrea…

Sometimes child/grandchild doesn't exist, we needed to account for that with optional chaining.

Closed in favor of https://github.com/tloncorp/tlon-apps/pull/3647


Add hmcts net emails

A new email domain has been created and is in use by the HMCTS sustaining team.
This was needed so that we could add their users.

👍


DBZ-5610 MySQL Connector capture TRUNCATE command as message in table topic

https://issues.redhat.com/browse/DBZ-5610
The truncate command sample data of table topic as below:
key: null
value:
 {     "before": null,     "after": null,     "source": {         "version": "2.0.0-SNAPSHOT",         "connector": "mysql",         "name": "test_avro",         "ts_ms":<PHONE_NUMBER>000,         "snapshot": {             "string": "false"         },         "db": "workdb",         "sequence": null,         "table": {             "string": "dbz5236"         },         "server_id": 112233,         "gtid": null,         "file": "mysql-bin.000035",         "pos": 125109,         "row": 0,         "thread": null,         "query": null     },     "op": "t",     "ts_ms": {         "long":<PHONE_NUMBER>970     },     "transaction": null }

I'll update the document later.

@harveyyue Thanks a lot, this will go into 2.1

@harveyyue Thanks a lot, this will go into 2.1

Get😊

@harveyyue Applied, thanks!


Update README.md to reflect project status

It is civil to warn others that the project is no longer maintained.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Add contrast support

Closes #2

@Kikobeats I think the hover state on contrast cards could be better, currently:
Here it looks ok:

Here, not so good:

Maybe we could just do a subtle change to the background color (darkening/lightening depending on the color)?

I was thinking in that and, instead of use backgorund-color for hover, I use opacitythat is more predictable and less code. What do you think?


Question on how a notify 0x2902 descriptor is set up in the server code

Hi
I am trying to work out how Notify works in the NimBLE-Arudino code but just can't find the right bit.
What I think I know:

the client needs to write to the 0x2902 descriptor to set notifies
the server needs to keep a list of clients that wrote to the descriptor and send a notify message when a write occurs
And I think I've found the both parts in the code and the list of subscriptions.

What I can't work out is where the code creates the 0x2902 descriptor or adds to the subscription list when it is written to.
I can find where it blocks you from creating another one!
I was thinking it would either be in NimBLEService in the start function, or in NimBLECharacteristic in the the creation of a characteristic. And that a 0x2902 descriptor would be created and a onWrite callback created for it to add or remove the client from the subscription list.
Either that or it is buried in the NimBLE code - there are references in ble_gatts.c in nimble/host/src but not sure that is where I should be looking?
Can you point me to the right files / functions to work out how this is done?
Thank you!

This is all taken care of by the library. No need to do anything with the 2902 descriptors here.

Yes - but how, and where in the code?
I want to understand how it does it.
Thanks

I think I have found where the 0x2902 is created - not in NimBLE-Arduino but in the nimble code itself. In \src\nimble\nimble\host\src\ble_gatts.c there is the function ble_gatts_register_chr and in there it creates the CCCD if one is needed (calling ble_gatts_register_clt_cfg_dsc)

Yes, that is where it is created.

So I've traced this through the code, and worked out how the CCCD updates are managed in nimble and NimBLE-Arduino.
It seems that both the nimble library and NimBLE-Arduino cache which clients / centrals are connected and subscribed for notifications - why is this done twice, does NimBLE-Arduino do something additional to nimble?
I made the changes below to NimBLECharacteristic.cpp and it works for notifications - I know that it will break some things because of the code commented out but I just wanted to test that nimble cached the subscribe and could notify.
In summary, I commented out some lines in handleGapEvent (between /* PAUL  and PAUL END */ and swap of notify to just call  ble_gatts_chr_updated(m_handle);
Line 260: handleGapEvent:
int NimBLECharacteristic::handleGapEvent(uint16_t conn_handle, uint16_t attr_handle,
                             struct ble_gatt_access_ctxt *ctxt,
                             void *arg)
{
    const ble_uuid_t *uuid;
    int rc;
    struct ble_gap_conn_desc desc;
    NimBLECharacteristic* pCharacteristic = (NimBLECharacteristic*)arg;

    NIMBLE_LOGD(LOG_TAG, "Characteristic %s %s event", pCharacteristic->getUUID().toString().c_str(),
                                    ctxt->op == BLE_GATT_ACCESS_OP_READ_CHR ? "Read" : "Write");

    uuid = ctxt->chr->uuid;
    if(ble_uuid_cmp(uuid, &pCharacteristic->getUUID().getNative()->u) == 0){
        switch(ctxt->op) {
            case BLE_GATT_ACCESS_OP_READ_CHR: {
                rc = ble_gap_conn_find(conn_handle, &desc);

/* PAUL
                assert(rc == 0);

                 // If the packet header is only 8 bytes this is a follow up of a long read
                 // so we don't want to call the onRead() callback again.
                if(ctxt->om->om_pkthdr_len > 8 ||
                   pCharacteristic->m_value.size() <= (ble_att_mtu(desc.conn_handle) - 3)) {
                    pCharacteristic->m_pCallbacks->onRead(pCharacteristic);
                    pCharacteristic->m_pCallbacks->onRead(pCharacteristic, &desc);
                }

PAUL END */
                ble_npl_hw_enter_critical();
                rc = os_mbuf_append(ctxt->om, pCharacteristic->m_value.data(), pCharacteristic->m_value.size());
                ble_npl_hw_exit_critical(0);
                return rc == 0 ? 0 : BLE_ATT_ERR_INSUFFICIENT_RES;
            }

Line 442: notify
void NimBLECharacteristic::notify(const uint8_t* value, size_t length, bool is_notification) {
    NIMBLE_LOGD(LOG_TAG, ">> notify: length: %d", length);

    void ble_gatts_chr_updated(uint16_t chr_val_handle);
    ble_gatts_chr_updated(m_handle);

    NIMBLE_LOGD(LOG_TAG, "@@@  notify: handle: %d", m_handle);
    NIMBLE_LOGD(LOG_TAG, "<< notify");
} // Notify

Your code works, but I don't understand what you are trying to achieve here?

why is this done twice, does NimBLE-Arduino do something additional to nimble?

This was done for legacy reasons, in the past the notification function would notify all connected clients even though they were not subscribed. This may be a point for optimization now though, thanks.

The code was just to show that nimble also notifies only after a subscribe event. Just a test.
And that was because it looked like both nimble and NimBLE-Arduino cacehed those subscriptions and I wondered why - but if nimble functionality has changed underneath NimBLE-Arduino that makes sense.
Thank you - it's clear to me now.


Attic and large files

I've recently found out about attic and it seems really interesting, but I'm having trouble backing up large files (ie. dvd images). Is it possible to do something about it - skip deduplication, decrease compression or something that would speed things up?
I've tried creating and running attic on a large directory with many small files and it works great, but now I'm creating a new repository of a bunch of home folders and from the output of lsof I can see it takes several hours to backup large files.
Update: I'm using encryption, transferring over ssh with arcfour and compression enabled over a 10mbit line. It seems that the network is the bottleneck.

mentioning ssh-hpn, this would be a nice fit: https://github.com/jborg/attic/issues/99

please close.


Property behaviors

Proposal to add "property behaviors" for factoring out common property implementation patterns.

@jckarter, can you squash this?
@rjmccall would make an excellent review manager for this.

@DougGregor squashed.


⚠ Bofrost.lu has degraded performance

In c05c7d2, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1013 ms

Resolved: Bofrost.lu performance has improved in 2a49f6c after 27 minutes.


Merge dynamic dimension values

Merges the values from multiple composed DynamicMaps addressing https://github.com/ioam/holoviews/issues/920.

Looks like my comment in the issue regarding a KeyError was confused. Anyway, merging dimensions with merge_dimensions makes sense - my only question, is there anywhere else we should be using it?

my only question, is there anywhere else we should be using it?

I think this uses it everywhere it matters, i.e. when merging HoloMaps/DynamicMaps in an Overlay and when computing the dimensions during plotting.

Thanks for adding the doctest!
Happy to merge once the tests pass.

Some syntax error due to the test being multi-line.

Ready to merge when it passes.

Tests have passed. Merging.


Changed environment 'reset' interfaces to match the gymnasium interface.

The "reset" function for each environment has been changed to match the gymnasium interface. This should address issues with external libraries that expect the full interface, such as stablebaselines3 and prevent such issues in future.

this is really helpful!

thanks for your codes.


Workers are using a ton of RAM

dpul1 is now running 3 background workers, and using up ~15GB of RAM — why are they using so much?

We should update Ansible to have 3 workers instead of 5.

Ansible PR: https://github.com/pulibrary/princeton_ansible/pull/836


question: issues/request related to docs

There are couple of issues/request related to docs, isn't thejameskyle/babel-handbook the answer to those?
Should those issues be closed with reference to thejameskyle/babel-handbook
cc/ @thejameskyle

They could be - it's fine to leave here


Game breaking ACE Bug (API issue)

Le'ts say the player initially pulls: 4 and Ace, here we say: Since 4 =< 10, Ace's value is 11, so hand value is 15
Player pulls a 7 when hitting: Now 4+7 = 11 !< 10, so Ace shuld be valued at 1, buts it's been appended as 11 (Error in Perform hit, and Dealer Play functions in db_utility.py) fix these bugs and possibly abstract this tricky hand value countign as a fn in game utility file?

The key issue with this, is the Ace's value being set it's high form (11) even if the deck actually has, over 10 since at time of "Ace compensation", the deck value is still low as Pass 0 and 1 haven't been ran yet...
def GAME_UTIL_calculate_hand(hands, holder: int, pulled_card):

    ACTOR_HAND_VAL = 0
    PULLED_ACE_FOUND = False

    #* ----- Pass 0, Add Pulled card's value immediate, and do a pre-liminary ace check here!
    if pulled_card is not None:
        pulled_card_value = pulled_card["card_value"]

        #! >>>> Preliminary ACE Check
        if pulled_card_value is None:
            PULLED_ACE_FOUND = True
        else:
            ACTOR_HAND_VAL += pulled_card_value

    #* ----- Pass 1, Append all None, Aces onto value
    for card in hands:
        card_value = card["card_value"]
        if card_value is not None and card["holder"] == holder:
            ACTOR_HAND_VAL += card_value

    #* ----- Pass 2, After total non-ace deck computed, see how Ace's value can fit
    for card in hands:
        card_value = card["card_value"]
        if card_value is None and card["holder"] == holder: #Ace Case

            # Max value the Player's hand can be is 10, for me to give them 11 as Ace
            if ACTOR_HAND_VAL <= 10:
                ACTOR_HAND_VAL += 11
            else:
                ACTOR_HAND_VAL += 1

    #* ----- Pass 3 > If an Ace was pulled, we purposefully add all Valued cards, and add the Ace Accordingly
    if PULLED_ACE_FOUND == True:
        if ACTOR_HAND_VAL <= 10:
            ACTOR_HAND_VAL += 11
        else:
            ACTOR_HAND_VAL += 1

    return ACTOR_HAND_VAL

Here the first 2 passes add the card-valued cards, and for passes 2 and 3 we're compensating the aces either already in the hand, or arbitrarily pulled, this function makes sense within the Game Action fn's that use it whilst also doing an action like pulling a card off the deck in that same 1 fn


Fix Fumbles not removing successes and automate health levels on rolls

Primary changes are in the title, a number of smaller changes were also made to formatting in several places to bring things in line with the default linter standards.

There are many situations where you don't want to apply wound penalties. Not sure its a good idea to force it instead of having an option which is by default on.
And it doesn't handle higher than 7 health levels well if someone has taken those merits.
Also, if you roll a result that goes into the negative, it still types out success instead of failure, seems to only say failure when you get 0.

Thanks for the feedback @TheAziraphale  should have everything addressed in an acceptable fashion.

Hi! First off, just so you don't get the wrong idea. I'm a random, so I can't approve the branch :). But I can give some feedback!
I did test it and the rolling works fine now and I can have higher health levels, but no matter if I have "apply wounds" checked or not, it doesn't seem to apply it. This seems to be the case no matter if I have the normal 7 health levels or a custom amount.

Yeah, I figured that out shortly after checking Github when I got an email about a comment on the PR.
It's working locally for me, I'd recommend slapping some console.logs in the mortal-actor-sheet.js and roll-dice.js, should be able to get a good idea of what exactly is going on for you and probably be able to fix the issue.
healthModifier (useWound, wound) is a good place for it in roll-dice.js
callback: async (html) => { is probably a good place to look in mortal-actor-sheet.js

I've merged @TheAziraphale into the main branch of my personal repository, the specific changes for this request are available on roll-changes.


[tw] Localize Runtime for Traditional Chinese

Localize Runtime in Traditional Chinese
https://glossary.cncf.io/runtime/

I would like to work on this term


Support For Blackmagic Camera Tally Through Arduino And Blackmagic 3G-SDI Arduino Shield

What feature would you like to see added to TallyArbiter?
I'm looking for a way to use TallyArbiter with Arduino And Blackmagic 3G-SDI Arduino Shield to trigger the tally lights on black magic cameras such as the (Blackmagic Studio Camera 4K Pro), I need this because I'm using TriCaster and I'd like the built in tally lights to work on the cameras with TriCaster.
I'm guessing the Arduino would just have to act as a listener and listen to the TallyArbiter server running on the network.
I have all the pieces of software and libraries that I would like to use I just don't know how to program them all together to make them work with TallyArbiter.
Libraries and programs:
Arduino Ethernet Shield - Ethernet.h & EthernetUdp ??
Blackmagic 3G-SDI Arduino Shield - BMDSDIControl.h
TallyArbiter - pieces of the TallyArbiter-M5StickCListener ??
I just don't know how to put all these together or if there's an easier way?
Thanks,
Zack

@josephdadams and I were looking at building this listener. Our biggest problem is that we don't have the hardware to develop with.
One thing I would say is that I would use the raspberry pi hat before the arduino one. The raspberry pi will give you native Ethernet.
Ill have a look at everything tomorrow and see what I can do though.

Let me know if I can help test anything with the hardware, Black magic Arduino code is pretty simple just high or low for light on or off and true or false four red or green. Of course that's probably not the hard part,
I was having a hard time figuring out how to integrate TallyArbiter I'm guessing it with WebSocketsClient.h,
,SocketIOclient.h and Arduino_JSON.h but I don't know whole lot about "coding". I was going down a rat hole of trying to use OSC messages but didn't think that would be the best idea.
Unfortunately I don't think Black Magic supports raspberry pi or has any hardware for it?
Anything you guys can do is greatly appreciated!!
Thanks

You're right! I thought they had one.
Do you have a picture of your setup that you could send me for reference of your setup?

![IMG_2160](https://user-images.githubusercontent.com/57646814/137610746-3ff3cbeb-8e64-4dc4-be69-9a6461264137.jpg
https://user-images.githubusercontent.com/57646814/137610814-82a94ce6-a150-491f-a8a4-a6c1dc27c3ca.mov
Arduino Uno Rev3
Blackmagic Studio Camera 4K Pro
Blackmagic 3G-SDI Arduino Shield
Ethernet Shield W5100
Let me know if you need anything else!!

I actually prototyped the code for this over a year ago. Never had the hardware to test on. I may not have kept it but I’ll dig around and see if I can find it. It will need refreshing anyway in light of our 3.0 architecture. Personally rather than implement a listener client per se I would create a program on the Arduino that subscribed to all tally information, similar to how we implemented VMix clients. @JTF4 we could do this over simple TCP rather than websockets.
@zackdickinson16 a remote computer we can access that is connected to the Arduino and shield, along with a webcam pointed at the monitor with the tally light would be extremely helpful in testing.

@JTF4 if you want to use REST api/UDP/etc let me known and I'll implement this in backend.
BTW, can you take a look at #225 since I don't have M5Stackc or M5Atom to test the updated firmware?

@MatteoGheza yep! Do you think we could build a rest based api that allows for device reassigns from the main server? If we could do that, then I think we would have a great protocol for devices that don't play as well with socket.io

@MatteoGheza yep! Do you think we could build a rest based api that allows for device reassigns from the main server? If we could do that, then I think we would have a great protocol for devices that don't play as well with socket.io

I'm working on a new REST API.

How would we push new tally data to a listener client with REST? Does the Device do polling or are we using something like SSE?

If I had to guess it's in error in one of the included libraries and not your direct code. I would start by looking at the actual code contents of the Black Magic library and seeing if it includes that.

I'm pretty sure it's the SocketIoClient.h Library?, but I'm not very good at this stuff so I wouldn't have a clue.


Disable the tests that involve NIO.2 WatchService on old JRE

Motivation:
At least on macOS 13.4 (and probably other OSes), WatchService seems not to notify the file changes quickly enough. As a result, we see the following tests fail:

FileWatcherRegistryTest.runnableWithExceptionContinuesRun
PropertiesEndpointGroupTest.endpointChangePropagatesToListeners
PropertiesEndpointGroupTest.propertiesFileRestart
PropertiesEndpointGroupTest.propertiesFileUpdatesCorrectly

Modifications:

Migrated FileWatcherRegistryTest and PropertiesEndpointGroupTest from JUnit 4 to 5
Added @EnabledForJreRange annotation on the problematic tests

Result:

Build is more stable on the environment where WatchService is not very responsive.
Fixes #3474 and #3075

I bumped the minimum requirements from 11 to 17 for even more build stability. Let me know if you think we should test this on 11.


Fix: Make BigQuery extractors support Relational Metadata store

Summary of Changes

Changes made according to the issues: #1070 & #1071
Additional changes made not mentioned in the issue:

Skip usage counts for wildcard tables: When there is a query like select * from project:dataset:tableprefix*, the 'referenced tables' in the logging response contains the tableId as tableprefix* resulting in a FK constraint violation between table_usage and table_metadata tables.

Tests

Added unit tests for the new configs in bigquery_usage_extractor and bigquery_watermark_extractor.
Documentation

N/A
CheckList
Make sure you have checked all steps below to ensure a timely review.

[x] PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"

In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

[x] PR includes a summary of changes.
[x] PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
[x] In case of new functionality, my PR adds documentation that describes how to use it.

All the public functions and the classes in the PR contain docstrings that explain what it does

LGTM, we'll see if Amundsen community has any concern.


VZ: Support Kernel Image

Similar to QEMU's kernel image support.
When using VZLinuxBootLoader (https://developer.apple.com/documentation/virtualization/vzlinuxbootloader), the Virtualization.framework does not provide EFI to the VM. As a result, the Linux kernel does not recognize a device available as rtc0, which means the correct time cannot be obtained during the kernel boot. Consequently, the VM's time is not corrected until timesync.service starts via cloud-init, which can cause issues like apt-get failures. If, for any reason, timesync.service fails to correct the time, apt-get in the provisioning script might fail, causing limactl to perceive the VM boot as failed.
I'm hesitant to add tests for this feature in CI because I'm unsure of the reasons why timesync.service might fail to correct the time.
I believe inject-cmdline-to-template.sh will be helpful when testing this feature.

When using this change to boot a VM with a kernel image, lima-guestagent will continuously throw errors because /dev/rtc is not present in the VM.
$ limactl shell docker sudo systemctl status lima-guestagent
● lima-guestagent.service - lima-guestagent
     Loaded: loaded (/etc/systemd/system/lima-guestagent.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-08-29 08:52:54 JST; 10min ago
   Main PID: 1670 (lima-guestagent)
      Tasks: 7 (limit: 19101)
     Memory: 7.1M (peak: 8.3M)
        CPU: 2.010s
     CGroup: /system.slice/lima-guestagent.service
             └─1670 /usr/local/bin/lima-guestagent daemon --vsock-port 2222

Aug 29 09:02:14 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:14+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:24 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:24+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:34 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:34+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:44 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:44+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:02:54 lima-docker lima-guestagent[1670]: time="2024-08-29T09:02:54+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:04 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:04+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:14 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:14+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:24 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:24+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:34 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:34+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"
Aug 29 09:03:44 lima-docker lima-guestagent[1670]: time="2024-08-29T09:03:44+09:00" level=warning msg="fixSystemTimeSkew: lookup error: open /dev/rtc: no such file or directory"

@norio-nomura Do you have a sample yaml for running with kernel ??
Can you please share it.

Also could you squash the commits for pkg/vz/*.go?

@norio-nomura Do you have a sample yaml for running with kernel ?? Can you please share it.

You can inject kernel related info into ubuntu.yaml by following:
hack/inject-cmdline-to-template.sh templates/ubuntu.yaml console=hvc0

e.g.
$ hack/inject-cmdline-to-template.sh templates/ubuntu.yaml console=hvc0
kernel_location=https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-vmlinuz-generic
kernel_digest=sha256:69e66230d6acc503082f098fd072a672a804774007209110b353d25bb7007669
cmdline=root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyAMA0 console=hvc0
initrd_location=https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-initrd-generic
initrd_digest=sha256:92acf9f2976b7a8b3b5e9e3f22211d17a6fdb3f2aed981467f91cd6dbfa16ae5
$ limactl start templates/ubuntu.yaml --tty=false
INFO[0000] Terminal is not available, proceeding without opening an editor
INFO[0000] Starting the instance "ubuntu" with VM driver "vz"
WARN[0000] vmType vz: ignoring images[1]: [Kernel Initrd]
INFO[0000] Attempting to download the image              arch=aarch64 digest="sha256:5ecac6447be66a164626744a87a27fd4e6c6606dc683e0a233870af63df4276a" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/ubuntu-24.04-server-cloudimg-arm64.img"
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/346ee1ff9e381b78ba08e2a29445960b5cd31c51f896fc346b82e26e345a5b9a/data"
INFO[0000] Attempting to download the kernel             arch=aarch64 digest="sha256:69e66230d6acc503082f098fd072a672a804774007209110b353d25bb7007669" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-vmlinuz-generic"
INFO[0000] decompressing .04-server-cloudimg-arm64-vmlinuz-generic with gzip
INFO[0000] Decompressing data
17.39 MiB / 17.39 MiB [------------------------------------------] 100.00% ? p/s
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/98c212aa1b30c23b56134728fa6adf3096bea13b93cf1744bf88d35d488be2e7/data"
INFO[0000] Attempting to download the initrd             arch=aarch64 digest="sha256:92acf9f2976b7a8b3b5e9e3f22211d17a6fdb3f2aed981467f91cd6dbfa16ae5" location="https://cloud-images.ubuntu.com/releases/24.04/release-20240821/unpacked/ubuntu-24.04-server-cloudimg-arm64-initrd-generic"
INFO[0000] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/c972a5d4d42628ec1dd6610a5398674a7f85a3bfa916d614064809722e0f4840/data"
INFO[0000] Converting "/Users/norio/.lima/ubuntu/basedisk" (qcow2) to a raw disk "/Users/norio/.lima/ubuntu/diffdisk"
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 213.27 MiB/s
INFO[0017] Expanding to 100GiB
INFO[0017] Attempting to download the nerdctl archive    arch=aarch64 digest="sha256:77c747f09853ee3d229d77e8de0dd3c85622537d82be57433dc1fca4493bab95" location="https://github.com/containerd/nerdctl/releases/download/v1.7.6/nerdctl-full-1.7.6-linux-arm64.tar.gz"
INFO[0017] Using cache "/Users/norio/Library/Caches/lima/download/by-url-sha256/21cc8dfa548ea8a678135bd6984c9feb9f8a01901d10b11bb491f6f4e7537158/data"
WARN[0018] [hostagent] GRPC port forwarding is experimental
INFO[0018] [hostagent] hostagent socket created at /Users/norio/.lima/ubuntu/ha.sock
INFO[0018] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/norio/.lima/ubuntu/serial*.log")
INFO[0018] [hostagent] new connection from  to
INFO[0018] SSH Local Port: 59900
INFO[0018] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0018] [hostagent] [VZ] - vm state change: running
INFO[0028] [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
INFO[0028] [hostagent] The essential requirement 1 of 2 is satisfied
INFO[0028] [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
INFO[0028] [hostagent] The essential requirement 2 of 2 is satisfied
INFO[0028] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0028] [hostagent] Guest agent is running
INFO[0028] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0028] [hostagent] Not forwarding TCP <IP_ADDRESS>:53
INFO[0028] [hostagent] Not forwarding TCP <IP_ADDRESS>:53
INFO[0028] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0028] [hostagent] Not forwarding TCP [::]:22
INFO[0028] [hostagent] Not forwarding UDP <IP_ADDRESS>:53
INFO[0028] [hostagent] Not forwarding UDP <IP_ADDRESS>:53
INFO[0028] [hostagent] Not forwarding UDP <IP_ADDRESS>:68
INFO[0037] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0037] [hostagent] Waiting for the guest agent to be running
INFO[0037] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0037] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0038] READY. Run `limactl shell ubuntu` to open the shell.

Thanks! 🙏🏻


IE11 tests usually fail by timing out after running all tests

Description
The IE11 tests on the main branch usually fail by timing out at the very end, even if all of the tests themselves pass. (The main branch commit history page shows most recent commits as having failed.) This makes it easier to accidentally merge IE11-specific bugs because the pass/fail indication displayed on the PR page doesn't currently align with what we consider passing. We need to figure out what's causing these timeouts so that the action will report the correct result.
Acceptance criteria
The IE11 tests on the main branch should correctly indicate their pass/fail state when reported to the PR's checks (and they should pass).

Closing as dupe of https://github.com/lit/lit/issues/1550


Contact category requests

Support contact category requests. Reference: https://docs.microsoft.com/en-us/graph/api/resources/outlookcategory?view=graph-rest-1.0

Done


Selecting timeline from the editor crashes Godot

The problem
Describe the bug
Trying to access timelines will cause Godot to crash.
To Reproduce
Steps to reproduce the behavior:

Create a new timeline
Run scene or click away to a different tab
Try to view timeline again
Godot Crashes

Expected behavior
I should be able to view and edit timeline if clicking on it and Godot should keep running
Screenshots
If applicable, add screenshots to help explain your problem.
System (please complete the following information):

OS: Windows
Godot Version: 3.2.1
Dialogic Version: 1.2.5

Solutions
Workaround
Reverting to previous version
Possible fixes

This is a realy strange bug. Could you try it out with the 1.3 WIP branch? Make sure to use a test project or make a backup of your project!

Can't reproduce 🤔

I will close this, as I cannot reproduce it either. If you can provide a way to still reproduce this bug, feel free to reopen it.


proposal: log/slog: Add "slog.KindErr" and "slog.Err" Attrs Helper Function

Abstract
This proposal suggests adding a dedicated error type, slog.KindErr, and a helper function, slog.Err, to the "log/slog" package. These additions aim to improve structured error logging by providing a clear and efficient mechanism to distinguish and include errors in log entries.
Proposal

slog.KindErr
add a new type, slog.KindErr, for representing errors in structured log entries. This type will enhance the distinction of errors from other attributes.

const (
	KindAny Kind = iota
	KindBool
	KindDuration
	KindErr
	KindFloat64
	KindInt64
	KindString
	KindTime
	KindUint64
	KindGroup
	KindLogValuer
)

slog.Err Helper Function
Add a helper function, slog.Err, for simplifying the process of logging errors as structured attributes. This function will accept a message and an error, creating a slog.KindErr attribute in the log entry.

// Err returns an Attr for an error
func Err(key string, v error) Attr {
	return Attr{key, ErrValue(v)}
}

Usage
Developers can use slog.Err to conveniently log errors:
slog.Warning("An error occurred in processing",
	slog.Err("validation error", vErr),
	slog.Err("computation error", cErr),
        slog.Bool("retry", true),
)

This approach ensures that error attributes are clearly distinguished from other key-value pairs.

CC @jba

Is this just syntax sugar for slog.String("validation error", vErr.Error()) ?

Is this just syntax sugar for slog.String("validation error", vErr.Error()) ?

No, it's not just syntax sugar. It introduces a new Value Kind slog.KindErr which is distinguishable in handlers between errors and strings. This distinction can be valuable in log processing and analysis.

I think it's syntactic sugar for slog.Any(key, err).
You can write the Err function easily yourself. I'm not sure it's worth adding to slog. Usually people name their error variables err or similar, so it's pretty clear to the reader that you're constructing an Attr that holds an error.
Handlers can distinguish errors with a type assertion. Adding a kind doesn't really do much; the Kinds are there to distinguish the various ways that Value represents certain Go values, but a Value of KindError would look exactly like a value of KindAny.

While it's possible to manually distinguish value types through type assertions, this approach can lead to a degree of ambiguity and inconsistency in log entries. The slog.Err function, on the other hand, is introduced to enhance the clarity and consistency of error logging.
It's worth noting that if we follow the logic that every value type can be distinguished by assertion, one could argue that we don't need slog.Kind and slog.Value types at all. However, the purpose of slog.Kind is to provide a systematic way to categorize and work with different types of values in a structured log. Each slog.Kind serves as a clear marker for a specific type of data, making it easier to understand and manipulate log entries consistently.
The introduction of slog.Err aims to provide a convenient and intuitive mechanism for handling errors, much like other helper functions within the package streamline the process of working with different data types. It contributes to making the "log/slog" package more user-friendly and facilitates a standardized approach to structured error logging, which can be particularly beneficial in larger codebases and collaborative projects.

The slog.Err function, on the other hand, is introduced to enhance the clarity and consistency of error logging.

How does it make it more clear? I argued that seeing the err in slog.Any(key, err) is clear enough. Also, I pointed out that you can define Err in your own code.
How does it make it more consistent? If you'd proposed a canonical key for errors, like we have for message, level and time, then I'd see how that would help consistency. How does adding KindError help? I think your answer is

Each slog.Kind serves as a clear marker for a specific type of data, making it easier to understand and manipulate log entries consistently

but that's not actually the purpose of the different Kinds, and even if it were I don't quite see the argument for consistency.

It's worth noting that if we follow the logic that every value type can be distinguished by assertion, one could argue that we don't need slog.Kind and slog.Value types at all.

That is absolutely true. Value and its kinds are there only for performance. Otherwise we would have used any.

That is absolutely true. Value and its kinds are there only for performance. Otherwise we would have used any.

OK, I understood, Thank you for your feedback.

I would prefer to isolate err as well to have transformation in slog abstraction rather than in the application code.
However, currently I end up with having helper to serve data in OTEL format for otlp exporters:
func Error(val error) slog.Attr {
	stack := make([]byte, 4096)
	n := runtime.Stack(stack, false)

        return slog.Group("error",
	  slog.String("exception.message", val.Error()),
          slog.String("exception.stacktrace", fmt.Sprintf("%s", stack[:n])),
      )
}

ref: https://opentelemetry.io/docs/specs/semconv/exceptions/exceptions-logs/#attributes

That helper looks fine to me. Even if we implemented this proposal's suggestion, it wouldn't include a stack trace, so you'd need to write your function anyway.


[BUG] GetCredentialStatus with invalid user index shows run time failure error

Reproduction steps
Test Case ID:
DRLK-2.9 will be updated to include this check
Describe Issue:
Found during negative test case scenario testing
TC-DRLK-2.9 execute steps till step 9 - all is good.
then execute the below step with invalid user index 0
./chip-tool doorlock get-credential-status '{ "credentialType" : 1 , "credentialIndex" : 1 }'  0 1 --timedInteractionTimeoutMs 1000
there is no response we see a run command failure
Same outcome for index 11( exceeding the number of NumberOfTotalUsers
Same happens with GetUser Command
Expected INVALID_COMMAND
Spec Reference:

Bug prevalence
Always
GitHub hash of the SDK that was being used
NA
Platform
other
Platform Version(s)
No response
Anything else?
DRLK-2.9-negativecase.txt

The command should be GetUser not GetCredentialStatus


useExpanded expanded rows are by index rather than by unique row identifier

Overview
When using useExpanded on a table whose data can change, which rows are expanded is recorded by row index rather than by some unique identifier for the row. This means if a row is expanded but then the data changes and a new row is added before it, the same index is still expanded but this is a different row in terms of data.
Sandbox example
https://codesandbox.io/s/silly-silence-dyf64
Steps To Reproduce

Go to the above sandbox
Expand any of the initial 5 rows
Check the checkbox, which adds 5 more rows at the start
See that a different row (same index as before, but different data) is now expanded, and the row you initially expanded is not
Expand one of rows 6 to 10, note some data in it
Uncheck the checkbox, which removes those first 5 rows again
See that the now you had expanded while still present is no longer expanded

Expected behavior
The row by data that I expanded is still expanded after adding/removing data.
Suggested implementation
Of course by default identifying rows by index is the only thing which makes sense. But I would have thought most sets of data have some kind of unique identifier available, and using one wouldn't be very alien to developers used to React, since we use them all the time for the key prop. Adding a way to hint to react-table which field of each entry in data can be used as a unique identifier, and then using this for things like the store of which rows are expanded, ought to solve the problem.

Use the getRowId table option to use a custom unique ID for the rows.

This works; thanks. I found this hard to find in the docs; it's in the useTable docs. It could do with being mentioned in the useExpanded docs.

I would gladly accept a PR to add it to the useExpanded docs. Thanks!


Could you please release whole codes. Current codes are difficult to reproduce this wonderful research.

Thanks for your attention. Please wait us fro processing some deadlines.


Allow custom TLS cert checks

This PR adds an extension point, TLSCertificateChecker, for custom TLS certificate checking (both client and server side).
The hook provided allows an extension to replace the default X509ExtendedTrustManager's checkClientTrusted and checkServerTrusted methods (the default trust manager is provided as a parameter, so the standard checks can still be used by an extension if desired).
This PR also adds an option to the simple-client-sslcontext extension to disable the standard hostname validation check.

TeamCity failures seem unrelated


Update hana-cloud-mission-extend-09.md

Corresponding changes based on https://github.com/sap-tutorials/Tutorials/issues/18204

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please make the updates to the tags as Michael recommended previously.

Have a different pull request with the corresponding changes based on https://github.com/sap-tutorials/Tutorials/issues/18204 and changes to tags


Unable to find a way to provide icons for tabs on Android Auto

Browsable nonplayable items that are directly under the root item are shown as Tabs. However, I haven't found any way to provide icons for these tabs, it doesn't seem like media3 exposes any function that helps do so. Here are my tab mediaitems (you can notice the attempts to include icons for those tab items by passing extras, but they didn't work):

val subroot_TracklistItem = MediaItem.Builder()
        .setMediaId(nodeTRACKLIST)
        .setMediaMetadata(
            MediaMetadata.Builder()
                .setIsBrowsable(true)
                .setIsPlayable(false)
                .setMediaType(MediaMetadata.MEDIA_TYPE_FOLDER_PLAYLISTS)
                .setTitle("Tracklist")
                .setArtworkUri(Uri.parse(icon4tracklist))
                .setExtras(Bundle().apply {
                    putString("MediaMetadataCompat.METADATA_KEY_DISPLAY_ICON_URI", icon4tracklist)
                    putString("MediaMetadataCompat.METADATA_KEY_ART_URI", icon4tracklist)
                    putString("MediaMetadataCompat.METADATA_KEY_ALBUM_ART_URI", icon4tracklist)
                })
                .build()
        )
        .build()

    val subroot_PlaylistItem = MediaItem.Builder()
        .setMediaId(nodePLAYLIST)
        .setMediaMetadata(
            MediaMetadata.Builder()
                .setIsBrowsable(true)
                .setIsPlayable(false)
                .setMediaType(MediaMetadata.MEDIA_TYPE_FOLDER_PLAYLISTS)
                .setTitle("Playlist")
                .setArtworkUri(Uri.parse(icon4playlist))
                .setExtras(Bundle().apply {
                    putString("MediaMetadataCompat.METADATA_KEY_DISPLAY_ICON_URI", icon4playlist)
                    putString("MediaMetadataCompat.METADATA_KEY_ART_URI", iCONplaylist)
                    putString("MediaMetadataCompat.METADATA_KEY_ALBUM_ART_URI", icon4playlist)
                })
                .build()
        )
        .build()

we also faces this issue some time ago. as far as I recall, this is a limitation of Android Auto itself since it expects already loaded bitmaps here set via METADATA_KEY_ART. the only URL's that would work for tabs are the ones mapped to local assets: android.resource://${context.packageName}/$drawableResId – so I suggest to start playing with some local vector drawable
however, I might be wrong and I'd be glad to also hear the devs input

@NikSatyr
That's what I thought too, I provided a direct URI for the drawables. The two strings icon4playlist and icon4tracklist are drawable res URIs. No avail.
    val icon4tracklist = "android.resource://mpappc/drawable/ic_tracklist" //ancient variation
    val icon4playlist = "android.resource://mpappc/R.drawable.ic_playlist"

@yuroyami this does not seem to be correct a correct way of providing the local drawables' urls as the last part should be an integer drawable id, e.g. android.resource://com.your.package/2031932281
you may wanna use this helper method:
fun getLocalDrawablePath(context: Context, @DrawableRes drawableResId: Int): String {
        return "android.resource://${context.packageName}/$drawableResId"
}

@NikSatyr, Silly me should have thought about using the actual integer representation of the drawables, thank you so much for your input. I confirm that the following builder extension method works for setting tab icon on android auto:
        subroot_TracklistItem = MediaItem.Builder()
            .setMediaId(nodeTRACKLIST)
            .setMediaMetadata(
                MediaMetadata.Builder()
                    .setIsBrowsable(true)
                    .setIsPlayable(false)
                    .setMediaType(MediaMetadata.MEDIA_TYPE_FOLDER_PLAYLISTS)
                    .setTitle("Tracklist")
                    .setArtworkUri(Uri.parse(icontracklist)) <---
                    .build()
            )
            .build()


[boost-build:x86-windows] build failure

Host Environment

OS: [Windows]
Compiler: revision

To Reproduce
Steps to reproduce the behavior:
./vcpkg install boost
Failure logs
Downloading https://raw.githubusercontent.com/boostorg/boost/boost-1.70.0/boostcpp.jam... Failed. Status: 7;"Couldn't connect to server"
Additional context
The url "" is invalid.

update cmake  3.15.+
https://gitlab.kitware.com/cmake/cmake/merge_requests/3763

update cmake 3.15.+
https://gitlab.kitware.com/cmake/cmake/merge_requests/3763

thanks!! it works.


Fix ephemeralsuperseder metadata

fixes posting DELETEs when an ephemeral policy is set

Maybe we should do the check in my new function though? Should it only act on the right message types?

yes i can add it there too

merging over flakes


Upgrade dependencies

[ ] fs2 0.10.0-M2
[ ] scodec-stream 1.1.0-M2
[ ] refined 0.8.2
[ ] scalatest 3.0.2
[ ] http4s 0.15.13a
[ ] akka-stream 2.5.2
[ ] streamz-converter ??? (0.8 is not compatible with fs2 0.10, see https://github.com/krasserm/streamz/issues/38)

Work in progress on branch https://github.com/durban/seals/tree/wip-issue55.


Showcase that parallel steps do work in new upgrade test framework

This is just to showcase that parallel steps in new upgrade test framework do work.

Of course, it produces a race... I didn't bother to write it with channels, as this is just to showcase that operations in single steps can by running in parallel. That should be enough, as other types of parallelism seams unwanted.
/cc @mgencur

/cc @chizhg

Still required?

Yeah. Maybe, I'll add this test to the framework, so the support would be explicit, and fixing the race in the same time.
/remove-lifecycle stale

What's the goal of this PR? Is this supposed to land at some point? What's missing?


Any reason you are locking httpparty < 0.14.0?

spec.add_runtime_dependency "httparty", ">= 0.11.0", "< 0.14.0"

I think I run into an issue with this. It looks like rails 5.0.1 is using json 2.0.2 but httparty 0.13.7 has json ~> 1.8

Hello @delikat! Thanks for your work on this gem. Can we hear from you on this issue? I work on a mid-sized app and the strict version requirement here left no way for me to be able to resolve the version conflict, so I had to end up forking this repo. I am not sure how helpful it is, but I have submitted a PR here: https://github.com/optimizely/ruby-sdk/pull/25

Hi everyone. Apologies for the delayed response. I've gone ahead and approved @gaganawhad's PR (#25), which changes the httparty dependency to ~> 0.11. Once #25 is landed, I'll get a new release out right away. Thanks for your contributions!

Thanks @delikat 🎉

Fixed in v1.0.2 (#27). Thank you for your contributions and patience!


Document mentions configuration

This PR add documentation for the mentions key in repo configurations. Until this is merged a live version of these changes can be see here.

Thanks!


Issue with "Freshworks Freshservice" in @site/docs/reference/integrations/freshworks-freshservice.md

Dear team,
I represent a systems integrator, our customer is using Cortex XSOAR and tried to integrate it with Freshservice ITSM. We have configured the integration from the XSOAR side following the description on the Cortex web-page. Primary purpose is that Freshservice should mirror tickets from XSOAR and back. Unfortunately this integration doesn't work as expected - tickets, present in XSOAR don't appear in Freshservice.
We have engaged Freshservice team, but they said this integration was not officially supported from their side, and they couldn't help, so they proposed to go to the Cortex team to assess this integration. Would you be able to help?
The customer is using Cortex SXOAR under the Palo Alto subscription ‘International Bank of Azerbaijan Republic JSC 57699’

Page: Freshworks Freshservice
Source: https://github.com/demisto/content/blob/master//Packs/FreshworksFreshservice/Integrations/FreshworksFreshservice/README.md

Screenshots

Environment

OS: [e.g. Windows]
Browser: [e.g. chrome, safari, firefox..]
Browser Version:

Suggested fix

@evgenyva hey sorry for the late response here. Please ask the customer to open a ticket with XSOAR support directly. This repository is just for xsoar documentation.

Hi, Judah.
OK, we've done  it already. Thanks.
Best Regards,
Evgeny Vasilyev
ITSM/ITAM Business Development Manager, CIS&APAC
@.***
[signatureImage]

From: Judah Schwartz @.>
Sent: Sunday, March 3, 2024 6:06:17 PM
To: demisto/content-docs @.>
Cc: Evgeny Vasilyev @.>; Mention @.>
Subject: Re: [demisto/content-docs] Issue with "Freshworks Freshservice" in @site/docs/reference/integrations/freshworks-freshservice.md (Issue #1502)
You don't often get email from @.*** Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
@evgenyvahttps://github.com/evgenyva hey sorry for the late response here. Please ask the customer to open a ticket with XSOAR support directly. This repository is just for xsoar documentation.
—
Reply to this email directly, view it on GitHubhttps://github.com/demisto/content-docs/issues/1502#issuecomment-1975175268, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BGJE627QS67QOIDBXCSLBDLYWMU5TAVCNFSM6AAAAABDQVVOCSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZVGE3TKMRWHA.
You are receiving this because you were mentioned.Message ID: @.***>


Type objectContaining and peers as if they returned their described types

Summary
I have code like this:
type MyExpectationConfig = {foo: string, bar: Array<string>}
const config: MyExpectationConfig = {foo: expect.stringContaining(/foo/), bar: expect.arrayContaining('bar')}

which currently fails typechecking on the premise that expect.stringContaining has a void return type.
Test plan
I made this change locally; flow passed as expected.

@cpojer should we have the file here at all? Seems like it's a somewhat dated copy from the flow-typed repo

@SimenB it was added for the sake of flow typing integration tests.

We should at least update to the 22 one

(I am working on the CLA, corporate back-and-forth)

I probably won't have the bandwidth to help upgrade to 22, but I could change the filename if you want? This version is working fine for us, and we're on 22.

@rattrayalex-stripe have you copied the file from this repo? I'd recommend doing flow install jest@22 instead (or 21 if you're using that), and getting https://github.com/flowtype/flow-typed/blob/master/definitions/npm/jest_v22.x.x/flow_v0.39.x-/jest_v22.x.x.js

Ah, I see – I should be submitting a PR to that repo, and you can pull in from there – is that correct?
Apologies, I'm not familiar with flow-typed.

Correct! :) I'm gonna close this, but it wold be great if you could ping us (or send a PR) once it's landed upstream

Sweet!
https://github.com/flowtype/flow-typed/pull/1941


How to get only 1 sentence of answer?

When I input a question (single sentence), I have received a respond (with multi sentences) from my custom model. How to get an answer with only 1 sentence?
Thank you for helping!

Hello, @hoanganhtuanfsoft!
For a give context CakeChat model always outputs one answer, that indeed can include several sentences. For example:
Hey, how are you? → Hi! I'm doing great. What about you?

Is your question about splitting the answer into several grammatical sentences? Like returning only "Hi!" instead of "Hi! I'm doing great. What about you?"
Hey, how are you? → Hi!

If so, this postprocessing can be done using nltk package. Please see the following answer on stackoverflow: https://stackoverflow.com/a/37605851/1712419
You'll need to add the corresponding code after this line: https://github.com/lukalabs/cakechat/blob/master/cakechat/api/response.py#L46

Oh. I see. I thought I can set the number of output sentences.

If so, this postprocessing can be done using nltk package. Please see the following answer on stackoverflow: https://stackoverflow.com/a/37605851/1712419

I know it. But I want to model can inference only 1 sentence without other processing. :D
I built on 61 conversations with 50 epoch. I met bad result :(
I can't get more data because of specific data.