Microsoft Azure CI (Kudu): The Win32 internal error "The handle is invalid" 0x6 occurred while setting character attributes for the console output buffer

What You Are Seeing?
Annoying non-fatal messages

What is Expected?
Console.WriteLine() without colour because the sandbox environment doesn't support coloured console. See https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox :)
What version of Cake are you using?
https://github.com/reactiveui/website
What environment are you running on?  Windows? Linux? Mac?
Microsoft Azure
Are you running on a CI Server?  If so, which one?
Kudu (Azure Websites SCM Deployment)

How Did You Get This To Happen? (Steps to Reproduce)
You can manually invoke ./build.ps1 via web pased powershell repo for a fast debugging REPL experience.
https://github.com/projectkudu/kudu/wiki/Kudu-console
The .deployment file is how you override Kudu to run build scripts.
https://github.com/reactiveui/website/blob/master/.deployment
See https://github.com/projectkudu/kudu/wiki/Customizing-deployments
Output  Logs
Command: powershell -NoProfile -NoLogo -ExecutionPolicy Unrestricted -Command "& "$pwd\build.ps1" 2>&1 | echo"
Write-Host : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:84 char:1
+ Write-Host "Preparing to run build script..."
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Host], HostExcep
   tion
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteHostCommand

Write-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:137 char:5
+     Write-Verbose -Message "Trying to find nuget.exe in PATH..."
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx
   ception
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteVerboseCommand

Write-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:141 char:9
+         Write-Verbose -Message "Found in PATH at
$($NUGET_EXE_IN_PATH.FullName). ...
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx
   ception
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteVerboseCommand

Write-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:172 char:5
+     Write-Verbose -Message "Restoring tools from NuGet..."
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx
   ception
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteVerboseCommand

Write-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:182 char:5
+     Write-Verbose -Message ($NuGetOutput | out-string)
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx
   ception
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteVerboseCommand

Write-Host : The Win32 internal error "The handle is invalid" 0x6 occurred
while setting character attributes for the console output buffer. Contact
Microsoft Customer Support Services.
At D:\home\site\repository\build.ps1:192 char:1
+ Write-Host "Running build script..."
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceUnavailable: (:) [Write-Host], HostExcep
   tion
    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com
   mands.WriteHostCommand

Analyzing build script...
Processing build script...
Installing tools...
Installing addins...
Compiling build script...

Cake.exe : An error occurred when executing task 'Clean'.
========================================
Clean
========================================
Executing task: Clean
Deleting directory D:/home/site/repository/src/reactiveui.net/wwwroot/dashboard
At line:1 char:1
+ & "D:\home\site\repository\tools\Cake\Cake.exe" "build.cake"
-target="Default" - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~
    + CategoryInfo          : NotSpecified: (An error occurr...g task 'Clean'.
   :String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Error: Access to the path 'pack-1027f7a413d10b8d44e4dea91768a8fed383d0d8.idx'
is denied.
Write-Host : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:84 char:1\r\n+ Write-Host "Preparing to run build script..."\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Host], HostExcep \r\n   tion\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteHostCommand\r\n \r\nWrite-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:137 char:5\r\n+     Write-Verbose -Message "Trying to find nuget.exe in PATH..."\r\n+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx \r\n   ception\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteVerboseCommand\r\n \r\nWrite-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:141 char:9\r\n+         Write-Verbose -Message "Found in PATH at \r\n$($NUGET_EXE_IN_PATH.FullName). ...\r\n+         \r\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx \r\n   ception\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteVerboseCommand\r\n \r\nWrite-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:172 char:5\r\n+     Write-Verbose -Message "Restoring tools from NuGet..."\r\n+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx \r\n   ception\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteVerboseCommand\r\n \r\nWrite-Verbose : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:182 char:5\r\n+     Write-Verbose -Message ($NuGetOutput | out-string)\r\n+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Verbose], HostEx \r\n   ception\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteVerboseCommand\r\n \r\nWrite-Host : The Win32 internal error "The handle is invalid" 0x6 occurred \r\nwhile setting character attributes for the console output buffer. Contact \r\nMicrosoft Customer Support Services.\r\nAt D:\home\site\repository\build.ps1:192 char:1\r\n+ Write-Host "Running build script..."\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : ResourceUnavailable: (:) [Write-Host], HostExcep \r\n   tion\r\n    + FullyQualifiedErrorId : SetConsoleTextAttribute,Microsoft.PowerShell.Com \r\n   mands.WriteHostCommand\r\n \r\nCake.exe : An error occurred when executing task 'Clean'.\r\nAt line:1 char:1\r\n+ & "D:\home\site\repository\tools\Cake\Cake.exe" "build.cake" \r\n-target="Default" - ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n~~~\r\n    + CategoryInfo          : NotSpecified: (An error occurr...g task 'Clean'. \r\n   :String) [], RemoteException\r\n    + FullyQualifiedErrorId : NativeCommandError\r\n \r\nError: Access to the path 'pack-1027f7a413d10b8d44e4dea91768a8fed383d0d8.idx' \r\nis denied.\r\nD:\Program Files (x86)\SiteExtensions\Kudu\62.60603.2875\bin\Scripts\starter.cmd powershell -NoProfile -NoLogo -ExecutionPolicy Unrestricted -Command "& "$pwd\build.ps1" 2>&1 | echo"

Have your .deployment file point to a deploy.cmd  that contains
PowerShell -NoProfile -ExecutionPolicy Bypass -Command ".\build.ps1 --Target=Kudu-Deploy|Out-String -Stream; exit $LastExitCode;"

And it'll should work.

Obviously target can differ, but I usually have a kudu one even if it points to "default" task.

@ghuntley can you confirm piping to Out-String sorts your issue?

I'm going ahead and closing this issue, please reopen if you're still having issues.


issue #6098

[*] Do only one thing
[*] Non breaking API changes
[*] Tested

What did this pull request do?
It solves the issue Postgres Max Identifier length is 63 bytes

NamingStrategyConfig struct is added in schema.NamingStrategy, for supporting Max Identifier Lengths in different databases
Identifier Length support is added for Mysql(64),MS Sql Server ( 128),Sqlite (it has unlimited but default 64 is used), Postgres (63)
User Case Description

All test cases are updated. All tests have passed

gorm

Then how we can know which length  to assign ?

gorm

Then how we can know which length to assign ? @a631807682 Need you help. Can you elaborate please.
thanks

We can use Apply to change some config in every drivers.
https://github.com/go-gorm/mysql/blob/master/mysql.go#L85
https://github.com/go-gorm/gorm/blob/master/gorm.go#L143


Error running set-PAWOUDelegation.ps1

Hello,
Im getting this error running Set-PAWOUDelegation.ps1
Set-ACL : Access is denied
At C:\PAWContent 02052018\Set-PAWOUDelegation.ps1:195 char:1

Set-ACL -ACLObject $configContainerACL -Path ("AD:"+($configCN));

  + CategoryInfo          : PermissionDenied: (CN=Configuration,DC=corp,DC=com:String) [Set-Acl], UnauthorizedAccess
 Exception
  + FullyQualifiedErrorId : ADProvider:SetSecurityDescriptor:AccessDenied,Microsoft.PowerShell.Commands.SetAclComman
 d

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fd5878a8-a83e-e75d-ddae-59edc04e7138
Version Independent ID: 0f0cbbe4-20da-3abe-1a7c-effc608d8cca
Content: Privileged Access Workstations
Content Source: WindowsServerDocs/identity/securing-privileged-access/privileged-access-workstations.md
Product: windows-server-threshold
GitHub Login: @coreyp-at-msft
Microsoft Alias: coreyp

Hi, did you resolve this issue? I am having the same trouble


How can i add new line to sent text?

Hey,
I have long shouting message that i need to send every 1 hours. It has alot of new lines. How can I send ,a message with new lines?
for example this text message:
"
Hey,
I am using discord auto message
look there is new line
"

Maybe add text file input, and make it get the text until EOF. I am very bad at programming. but i understand the logic behind it.

When you want a newline, type \n into your message

exactly what i did without telling me, when used this the message was sent like this:
"
Hey, \nI am using discord auto message \n look there is new line
"

pull request #13


The memory the osim env takes increases linearly with the number of steps it took, even if the env gets resetted several times in the middle.

The memory the python process takes increases linearly with the number of steps taken by GaitEnv, is it supposed to be like that?
s = 0
while s<50000:
    o = env.reset()
    d = False
    while not d:
        o, r, d, i = env.step(env.action_space.sample())
        s += 1

This small script takes more than 500MB in my computer, python 2.7 and osx
And while training, the env takes more than 20GB memory for some 1Million steps, am I doing something wrong? There are cases where my training script stopped because of memory errors after training for a day on my little 8GB computer.

I have the same problem.

@kidzik I'd be happy to look through the code with you to identify memory leaks.

Thanks I will look into that in the next few days

@syllogismos @ViktorM I have troubles with reproducing the error. Can you tell me more about your environment (os, python version)?
Can you please try this script https://github.com/stanfordnmbl/osim-rl/blob/master/tests/test.memory.py with guppy? Thanks!

OS - Ubuntu 16.04, python 2.7, your standard conda environment. I'll be able to try script not only in 2-3 days. Usual scenario for the error is just to start training of the Gait env. In between 400K and 500K iterations it gets killed. Rarely after 500K. It takes approximately more than 16 hours of training.

I have the same. It takes ~1h for 1000 steps, without any learning.
Is there a way to speed-up the environement's computations ? (like using GPU)

I suppose you mean 1000 episodes? Or just 1000 steps?
The environment is indeed fairly difficult and time-consuming, but there are some tricks possible: you can run simulations on multiple cores and use an algorithm that can leverage parallel runs, or you can run short episodes at first.
This difficulty is a part of the problem -- finding an algorithm which can learn quickly in such a complex system is a goal on its own.

Yes, parallel computing is working, but I only have 4 cores. Let's move on EPFL servers...

I wrote a script which starts a RunEnv in a seperate process and communicate with it. I recreate that process every 50 episodes to limit it's memory consumption. Before that, the training process consumed ~10GB memory and crashed hard.

Memory leak still exists. I'm aware that one possible workaround is killing and restarting the leaking process, but maybe let's fix the leak :)
My gut feeling is that it happens during Python interaction with OpenSim's C++ code. I saw similar things happening with cap'n'proto Python API, you had to call some function in Python in particular way to prevent memleak. My guess here will be that osim-rl calls some function in OpenSim, e.g. next_frame() or something and OpenSim allocates some memory to return the result. And then it expects that some other function is called, that is never called. Or, there is simply a leak during next_frame() execution, totally inside OpenSim C++ code, but that seem to me less likely, they would probably spot it before or maybe even they auto test the code with valgrind or similar in CI.
Or maybe the Python wrappers generated with SWIG are problematic...

Good point, thanks for the insight. It seems to be dependent on the environment. Can you give us the details about your environment and a script for reproducing the error?

Linux x86_64 4.8.0-53. Conda env as in osim-rl README, with Python 3, tensorflow and psutil:
conda create -n py3 -c kidzik opensim git tensorflow psutil python=3.6
pip install git+https://github.com/stanfordnmbl/osim-rl.git

With Python 2 it's also leaking.
Minimal working example, memleak.py:
import os

import psutil
from osim.env.run import RunEnv
from datetime import datetime

def time():
    return datetime.now().strftime('%H:%M:%S')

def memory_used():
    process = psutil.Process(os.getpid())
    return process.memory_info().rss  # https://pythonhosted.org/psutil/#psutil.Process.memory_info

env = RunEnv(visualize=False)
env.reset()
step = 0
episode = 0
while True:
    observation, reward, done, info = env.step(env.action_space.sample())
    step += 1
    if done:
        episode += 1
        env.reset()
        print("%s Episode %s, steps %s, memory %s" % (time(), episode, step, memory_used()))
        step = 0

Output:
(py3) adam@adam-ThinkPad-T520 ~/Desktop/running/stuff $ python memleak.py
Updating Model file from 30000 to latest format...
Loaded model gait9dof18musc_Thelen_BigSpheres.osim from file /home/adam/miniconda3/envs/py3/lib/python3.5/site-packages/osim/env/../models/gait9dof18musc.osim
20:56:22 Episode 1, steps 120, memory 133820416
20:56:31 Episode 2, steps 117, memory 134230016
20:56:40 Episode 3, steps 120, memory 135041024
20:56:49 Episode 4, steps 118, memory 135041024
20:56:57 Episode 5, steps 116, memory 136663040
20:57:05 Episode 6, steps 117, memory 136663040
20:57:14 Episode 7, steps 117, memory 136663040
20:57:23 Episode 8, steps 122, memory 136663040
20:57:32 Episode 9, steps 117, memory 139636736
20:57:40 Episode 10, steps 118, memory 139636736
20:57:48 Episode 11, steps 119, memory 139636736
20:57:57 Episode 12, steps 117, memory 139636736
20:58:06 Episode 13, steps 118, memory 139636736
20:58:14 Episode 14, steps 119, memory 139636736
20:58:23 Episode 15, steps 120, memory 139636736
20:58:32 Episode 16, steps 119, memory 139636736
20:58:40 Episode 17, steps 117, memory 139636736
20:58:49 Episode 18, steps 118, memory 145854464
20:58:57 Episode 19, steps 116, memory 145854464
20:59:06 Episode 20, steps 117, memory 145854464
20:59:14 Episode 21, steps 119, memory 145854464
20:59:23 Episode 22, steps 118, memory 145854464
20:59:31 Episode 23, steps 119, memory 145854464
20:59:40 Episode 24, steps 119, memory 145854464
20:59:48 Episode 25, steps 116, memory 145854464
20:59:57 Episode 26, steps 117, memory 145854464
21:00:05 Episode 27, steps 119, memory 145854464
21:00:14 Episode 28, steps 118, memory 145854464
21:00:23 Episode 29, steps 119, memory 145854464
21:00:32 Episode 30, steps 118, memory 145854464
21:00:40 Episode 31, steps 118, memory 145854464
21:00:49 Episode 32, steps 118, memory 145854464
21:00:58 Episode 33, steps 117, memory 145854464
21:01:06 Episode 34, steps 117, memory 145854464
21:01:14 Episode 35, steps 119, memory 158560256
21:01:23 Episode 36, steps 117, memory 158560256
21:01:32 Episode 37, steps 118, memory 158560256
21:01:40 Episode 38, steps 117, memory 158560256
21:01:50 Episode 39, steps 121, memory 158560256
21:01:58 Episode 40, steps 118, memory 158560256
21:02:07 Episode 41, steps 117, memory 158560256
21:02:16 Episode 42, steps 117, memory 158560256
21:02:27 Episode 43, steps 119, memory 158560256
21:02:37 Episode 44, steps 117, memory 158560256
21:02:46 Episode 45, steps 118, memory 158560256
21:02:55 Episode 46, steps 118, memory 158560256
21:03:03 Episode 47, steps 118, memory 158560256
21:03:13 Episode 48, steps 122, memory 158560256
21:03:22 Episode 49, steps 117, memory 158560256
21:03:31 Episode 50, steps 117, memory 158560256
(...)

Memory usage always grew.
I noticed that if env.step(env.action_space.sample()) line is replaced with done = True, then the used memory doesn't increase.
So, calls to env.reset() are fine, calls to env.step() are causing the leak.

If this line is commented out, then the memory doesn't increase:
manager.integrate(self.osim_model.state)

I just searched for "leak" in opensim-core repo... one search hit:
{
    double* xValues = new double[2];
    xValues[0] = -1.0;
    xValues[1] = 1.0;
    return xValues; // possible memory leak
}

Don't know if it's related to our problem, but this one looks like a memleak, every new should have a corresponding delete (well, or use RAII)... It's in const double* XYFunctionInterface::getXValues() const.
Another:
            /* NOTE:  If we want to be ambitious here we can also determine
             *  when to free the hash node's "value" (i.e. cpval, or defn).
             *  This seemed tricky however, and I decided that a small
             *  memory leak was better than a possible invalid free().

Thank you very very much! I will check it on my machine and discuss it with the OpenSim team.

@kidzik Any news?

I really have troubles reproducing the error. Your code on my machine outputs
(opensim-rl) lukasz@lukasz-MOBILIZE-NMBL:~/workspace/osim-rl/tests$ python test.leak.py
Updating Model file from 30000 to latest format...
Loaded model gait9dof18musc_Thelen_BigSpheres.osim from file /home/lukasz/anaconda2/envs/opensim-rl/lib/python2.7/site-packages/osim/env/../models/gait9dof18musc.osim
17:33:15 Episode 1, steps 120, memory 128196608
17:33:20 Episode 2, steps 117, memory 128196608
17:33:24 Episode 3, steps 121, memory 128196608
17:33:29 Episode 4, steps 118, memory 128196608
17:33:33 Episode 5, steps 116, memory 128196608
17:33:37 Episode 6, steps 116, memory 128196608
17:33:42 Episode 7, steps 117, memory 128196608
17:33:47 Episode 8, steps 120, memory 128196608
17:33:51 Episode 9, steps 118, memory 128196608
17:33:56 Episode 10, steps 119, memory 128196608
17:34:00 Episode 11, steps 119, memory 128196608
17:34:05 Episode 12, steps 118, memory 128196608
17:34:10 Episode 13, steps 118, memory 128196608
17:34:14 Episode 14, steps 119, memory 128196608
17:34:19 Episode 15, steps 118, memory 128196608
17:34:23 Episode 16, steps 119, memory 128196608
17:34:28 Episode 17, steps 116, memory 128196608
17:34:32 Episode 18, steps 117, memory 128196608
17:34:37 Episode 19, steps 117, memory 128196608
17:34:41 Episode 20, steps 118, memory 128196608
17:34:46 Episode 21, steps 120, memory 128196608
17:34:50 Episode 22, steps 118, memory 128196608
17:34:55 Episode 23, steps 119, memory 128196608
17:34:59 Episode 24, steps 118, memory 128196608
17:35:04 Episode 25, steps 116, memory 128196608
17:35:08 Episode 26, steps 117, memory 128196608
17:35:13 Episode 27, steps 121, memory 128196608
17:35:17 Episode 28, steps 118, memory 128196608
17:35:22 Episode 29, steps 119, memory 128196608
17:35:26 Episode 30, steps 119, memory 128196608
17:35:31 Episode 31, steps 119, memory 128196608
17:35:36 Episode 32, steps 118, memory 128196608
17:35:40 Episode 33, steps 117, memory 128196608
17:35:44 Episode 34, steps 117, memory 128196608
17:35:49 Episode 35, steps 119, memory 128196608
17:35:54 Episode 36, steps 119, memory 128196608
17:35:58 Episode 37, steps 118, memory 128196608
17:36:03 Episode 38, steps 118, memory 128196608
17:36:07 Episode 39, steps 118, memory 128196608
17:36:12 Episode 40, steps 120, memory 128196608
17:36:16 Episode 41, steps 117, memory 128196608
17:36:21 Episode 42, steps 116, memory 128196608
17:36:25 Episode 43, steps 118, memory 128196608
17:36:30 Episode 44, steps 117, memory 128196608
17:36:34 Episode 45, steps 118, memory 128196608
17:36:39 Episode 46, steps 119, memory 128196608
17:36:43 Episode 47, steps 117, memory 128196608
17:36:48 Episode 48, steps 119, memory 128196608
17:36:52 Episode 49, steps 118, memory 128196608
17:36:57 Episode 50, steps 118, memory 128196608
...

and it's still running stable at least after 100 episodes.
Here is my environment
(opensim-rl) lukasz@lukasz-MOBILIZE-NMBL:~$ conda list
# packages in environment at /home/lukasz/anaconda2/envs/opensim-rl:
#
certifi                   2017.4.17                 <pip>
chardet                   3.0.4                     <pip>
freeglut                  3.0.0                         4    kidzik
funcsigs                  1.0.2                    py27_0
gym                       0.9.2                     <pip>
h5py                      2.7.0               np113py27_0
hdf5                      1.8.17                        1
idna                      2.5                       <pip>
keras                     2.0.2                    py27_0
keras-rl                  0.3.0                     <pip>
libgcc                    5.2.0                         0
libgfortran               3.0.0                         1
libgpuarray               0.6.4                         0
libprotobuf               3.2.0                         0
mako                      1.0.6                    py27_0
markupsafe                0.23                     py27_2
mkl                       2017.0.1                      0
mkl-service               1.1.2                    py27_3
mock                      2.0.0                    py27_0
nose                      1.3.7                    py27_1
numpy                     1.13.0                   py27_0
numpy                     1.13.1                    <pip>
openblas                  0.2.19                        0    kidzik
opensim                   4.0.0                   py27_12    kidzik
openssl                   1.0.2l                        0
osim-rl                   1.4.1                     <pip>
pbr                       1.10.0                   py27_0
pip                       9.0.1                    py27_1
protobuf                  3.2.0                    py27_0
psutil                    5.2.2                     <pip>
pyglet                    1.2.4                     <pip>
pygpu                     0.6.4                    py27_1
python                    2.7.13                        0
pyyaml                    3.12                     py27_0
readline                  6.2                           2
requests                  2.18.2                    <pip>
scipy                     0.19.0              np113py27_0
setuptools                27.2.0                   py27_0
six                       1.10.0                   py27_0
six                       1.10.0                    <pip>
sqlite                    3.13.0                        0
tabulate                  0.7.7                     <pip>
tensorflow                1.1.0                    py27_0    conda-forge
theano                    0.9.0                    py27_0
tk                        8.5.18                        0
urllib3                   1.22                      <pip>
werkzeug                  0.12.2                   py27_0
wheel                     0.29.0                   py27_0
yaml                      0.1.6                         0
zlib                      1.2.8                         3

Ubuntu 16.04 and my kernel is 4.4.0-72-generic.
It clearly not a proof that things are ok -- it just shows that it'll be harder to fix... I will try to reproduce it on AWS.

That sheds some light.
So, with Python 2 and 3, kernel 4.8.0-53, there is memory leak.
But, with Python 2 and kernel 4.4.0-72, there isn't.
No idea if kernel version is relevant, there could be some other difference between these envs that matters.
If you are not experiencing leaks, then the leaks in C++ code of OpenSim apparently aren't on the execution path.
Also, I think we may exclude SWIG as culprit, because opensim from conda repo that we both used was built with the same version of SWIG.
So the open question is, what's the source of memleak. Current hypotheses: kernel version, something else.

I checked on AWS, it just takes 5 minutes with prepared AMI (Python 2, kernel 4.4.0), following these instructions, plus conda install psutil and copying memleak.py.
No memory leak.

On one server with kernel 4.9.30 there is memory leak with Python 3.6, but not with Python 2.7.
Maybe new OpenSim bindings for Python 3.6 are leaking.
BTW in the conda channel kidzik there is older version of opensim, for Python3 it will be nice to put there newer version, because it has this helpful patch. Otherwise one have to manually tweak setup.py file.

It's easy to reproduce memory leak with Python 3.6 on AWS with your AMI.
Follow these instructions, then:
conda create -n py3 -c kidzik opensim git tensorflow psutil python=3.6
pip install git+https://github.com/stanfordnmbl/osim-rl.git

Copy and run memleak.py.

One update on my side in Python 2.7:
10:55:07 Episode 14451, steps 116, memory 128729088
so there might be a leak here as well but rather marginal.
Indeed, in Python 3.6 I get the leak. This would indicate that the actual problem might be in OpenSim itself and not in this package. For the challenge, we can't guarantee the full support for Python 3.6 since it's very recent in OpenSim. But I will further investigate it with the team. Thanks for all the updates!

I compiled and installed Python 2.7 OpenSim bindings following this. With SWIG 3.0.10 and g++ 6.3.0.
Now it's leaking.
With opensim from kidzik channel there's no leak.
With what command did you compile opensim that is in kidzik channel (for Python 2)? What g++ and SWIG version?

Now it's leaking.

This supports the idea that the memory leak was introduced in commits to opensim-core between when the python 2.7 and the python 3 opensim packages were created (based on discussion with @kidzik).

Good idea, might be.
@kidzik Do you know on what commit opensim-core was when built and stored in channel kidzik (for Python 2)?

Sure, it's compiled with this conda receipe https://github.com/opensim-org/conda-opensim/blob/master/opensim/meta.yaml, which means it's this commit in opensim-core
399c8d57a779dd5dde2916192f8b92bfc959e269. I don't know which GCC was used though.

Sure, it's compiled with this conda receipe https://github.com/opensim-org/conda-opensim/blob/master/opensim/meta.yaml, which means it's this commit in opensim-core
399c8d57a779dd5dde2916192f8b92bfc959e269. I don't know which GCC was used though.

Great, with https://github.com/opensim-org/opensim-core/commit/399c8d57a779dd5dde2916192f8b92bfc959e269 there's no leak with Python 2.
To find which commit introduced it, git bisect is ideal:
(py2) astelma@chuck:~/opensim-core$ git bisect start
(py2) astelma@chuck:~/opensim-core$ git bisect bad
(py2) astelma@chuck:~/opensim-core$ git bisect good 399c8d57a779dd5dde2916192f8b92bfc959e269
Bisecting: 306 revisions left to test after this (roughly 8 steps)
[3c12ca81228b93c8198aa44d09803bce5e8aa021] Merge branch 'master' into tgcs2017

There are 306 commits to check, however with binary search one needs 8 steps or less.

Good point, thanks, Adam! With @chrisdembia we were thinking that for the challenge we may just stick to https://github.com/opensim-org/opensim-core/commit/399c8d57a779dd5dde2916192f8b92bfc959e269 and cherry pick python3 commits -- this should help us avoid the memory leaks and make the python3 and python2 environments more comparable.

Now the new fixed builds for python 3.6 are available on all platforms, so this error shouldn't happen any more.

This should be fixed in new versions of opensim-core.
https://github.com/opensim-org/opensim-core/pull/2885


:man_farmer:  rviz_rendering test regressions for GLSL 4.5

New test regression on the linux ci jobs:
projectroot.point_cloud_test_target
projectroot.point_cloud_renderable_test_target
rviz_rendering.PointCloudTestFixture.setRenderMode_regenerates_renderables_with_different_size_when_geometry_support_changes
rviz_rendering.PointCloudTestFixture.addPoints_adds_vertices_with_correct_geometry_when_called
rviz_rendering.PointCloudRenderableTestFixture.renderable_contains_a_correctly_filled_buffer

Reference: https://ci.ros2.org/view/nightly/job/nightly_linux_debug/2223/
https://ci.ros2.org/view/nightly/job/nightly_linux-aarch64_debug/1895/
The package failing is rviz_rendering and the failing tests are:
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/test/rviz_rendering/objects/point_cloud_test.cpp#L267
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/test/rviz_rendering/objects/point_cloud_test.cpp#L298
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/test/rviz_rendering/objects/point_cloud_renderable_test.cpp#L91
The underlying problem is that the opengl version used is 4.5(GLSL 4.5)  instead of Opengl 3.1 (GLSL 1.4), so probably these failing tests will have to be adjusted or the opengl version has to be fixed. All the tests seem to be covering the cases of GLSL>=120 and GLSL>=150, for the case of GLSL>=4.5 the tests seem to expect a similar behavior from the older version.
Tagging package maintainers to get feedback. @jacobperron @mjeronimo

Is it just the tests that need to be updated or is there also a change needed related to materials? Here:
https://github.com/ros2/rviz/blob/e4a0d4a9f1c42ea9dd93c971b5e36ef356a4110d/rviz_rendering/src/rviz_rendering/render_system.cpp#L380-L388
I'm not that familiar with the rendering code, but I guess we should confirm that point clouds are still being rendered with the newer opengl version (in addition to fixing tests).

@jacobperron I am not that familiar with the rendering code, do you know someone we could tag to take a look?

or is there also a change needed related to materials? Here:

I didn't see any errors popping out from that part of the code, so I think that only the tests have to be updated.

:man_farmer: I took some time digging into this as it's affecting various jobs on the buildfarm. I think there problem is not related to a change in the number of vertices available in different OpenGL versions, I think it's related to the <>
i.e. These type of comparisons:
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/test/rviz_rendering/objects/point_cloud_test.cpp#L261-#L266
Are not a direct relationship to the opengl version but to the rendering mode being used:
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/src/rviz_rendering/objects/point_cloud.cpp#L141-L158
However we can't access this variable:
https://github.com/ros2/rviz/blob/f5f7c6a548b573959363307e9ad3ffab34acd93d/rviz_rendering/include/rviz_rendering/objects/point_cloud.hpp#L321
As it's private to the point_coud object. Does it make sense to create a getter for this class variable? that way we could change the check for opengl version to check for the rendering mode used.

@Blast545 Thanks for digging into it! Sorry, I haven't had the cycles.
I think adding a getter for that member flag would be fine.


Allowing Procs on password_archiving_count and deny_old_passwords

Hello! I've been using my own fork of this gem in a company project where I need to support Proc calls on config.password_archiving_count and config.deny_old_passwords.
The usage is something like:
config.password_archiving_count = Proc.new { SomeModel.instance.old_password_archiving_count }
config.deny_old_passwords = Proc.new { SomeModel.instance.old_password_archiving_requirement }

We use this on the project to have those configurations set from an admin backoffice. The changes needed to support this behavior are pretty small and, dare I say, harmful. Is this a behavior you'd like to support? I'd be happy to open a PR if that's the case.
If there is another way to achieve this, please let me know so that I can stop using a forked version of this gem 😅

dare I say, harmful

I hope this is a typo 😓 I don't want to add something harmful to the project.
If that's the case, I'd be happy to review a PR.

My bad, yes it's a typo 😅 Here's the PR: https://github.com/devise-security/devise-security/pull/455
Thanks so much for your attention 🙏🏻


Some fixes for our favorite Tango component

Changelog says it all.

@ZhaofengWu, this is the error message we were talking about.


Migrate build system away from setup.cfg

With the project layout that we use, we no longer need setup.cfg. We can simply remove all setuptools-related options and it should work as long as we have a recent enough version of setuptools. See also https://github.com/SciCatProject/scitacean/pull/161
We can wait before implementing this here until the next release of Scitacean to test it.
The only thing that is holding up back from removing setup.cfg altogether after this is flake8. I doubt this is going to change any time soon unless we move away from flake8 like Scitacean.

Is wheels build enough as a test?
I triggered the build manually in the actions and it looks working correctly...?

It probably is. I also tried it locally and it produced an installable wheel containing all expected files.
But I tend to be careful when it comes to changing our release setup. I'd rather debug one deployment than half a dozen.

See also https://github.com/scipp/scippnexus/pull/167#discussion_r1351835959


Update MemeFactory.java

Adding a fix to put "and" instead of a comma for a list of names

Thank you for this pull request! Please check this document for how the Jenkins project handles pull requests.


enable parallel testing

PTAL. I've created a directory context handler commander which enables running tinygo test commands in parallel with others. This could be further streamlined adding a "repoReady" channel to which cloneOrUpdateRepo sends repos that are ready to test.

This is pretty cool for dropping the total testing time down to something reasonable.  With tinygo using all the processors to parallelize the build itself, it's not clear we want to have many different tinygo instances running.  I wonder if we could have a single tinygo build process running, but then parallelize the actual running of the tests?

Sound advice all around, I'll get to fixing it.
About your last comment: How would you go about doing that? How would you use the tinygo command to accomplish that? tinygo test ./... in the corpusDir would test ALL sub-packages... at least without some preprocess or target directory arguments, right?

I've added a global lock for commander type, this will allow the instantiation of multiple commanders, each should be able to run commands and whatnot without interference between commanders. Let's Go crazy.

So I'm definitely concerned about the mixture of os.Chdir commands (which change the directory for the entire process) and concurrency, and having commands potentially running in the wrong place.  Maybe we need to standardize on "never call os.Chdir, only set cmd.Dir for the process about to be run" in order to keep things happy once we move to concurrent execution?

You read my mind- alright I've removed all os.Chdir usaged!

Maybe the way to structure this is to have a single compiler goroutine (which will allow tinygo to use all the cores for compilation) and build the test files, then a separate pool of "workers" which actually run the tests.  So, the main process passes repositories/subdirs to the compiler goroutine which, when it's finished compiling the test, passes it off to the test runner goroutine worker pool.

@dgryski From what I've seen tinygo test -c uses all cores for parsing which is usually the first 10%, the rest of the compilation tasks appear to be single threaded, consuming 100% of one of my CPUs at a time. Is the proposed solution still worth it given this?

Hard to say.  Why don't we go ahead with the current plan and we can see if we need to tweak the concurrency levels of the different parts later.

I'll follow your lead on this, so shall I add a compiler goroutine and testrunner pool? Didn't quite catch if this was "current plan"

No, I meant to go ahead with your current plan, and it it doesn't work, we can fix it in another PR.

Alright, I'll fix a few more things and push em in a few moments, and then it should be ready to merge

PTAL


Port over the core concepts from old docs into Overview section

@leflambeur changes made as requested, plus cleanup spell check.


Add preset for shop=spices

This adds a preset for shop=spices.

@scaidermern Thanks, looks good! I tweaked the name to "Spice Shop" since this sounds marginally more common in American English (iD's default language).

@scaidermern I tweaked the name to "Spice Shop" since this sounds marginally more common in American English (iD's default language).

Thanks! I'm not a native speaker and wasn't quite sure whether to use shop or store in this context.


Add Qiskit textbook

Can I get an opputunity to add this super-useful "Qiskit" textbook to the communnity!

You can directly submit a PR for the same. 👍🏻

Hello Sir, Can you assign this issue to me?

Already done @devmahfuj  bro :)


2x2 px tracker, are they trying to game the (0|1)px tracker detection?

Will email the whole email. Looks like a tracking pixel, quacks like a tracking pixel, but is actually a 2x2 grid of tracking pixels?
<img src=3D"https://link.mypetwarehouse.com.au/mo/2vhXSRX3LE_872477014_2385=
754_337069_3077279.gif" height=3D"2" width=3D"2" border=3D"0" alt=3D""></bo=
dy>

Looks like Emarsys (SAP). First time seeing this. Yikes, will need to make some adjustments.

Added a URL rule for this one but a bit reluctant to expand the generic rule to 2x2px unless there are others out there. Will have to keep an eye out for more 2x2px trackers.


Now works with dash!

Deleted bashism so it now works with dash.
I use Debian too so this fixes issue 7.
Tested to work with various combinations of bash, dash, wget, and curl.

There is another line that you might want to change IMHO:
https://github.com/proprietary/chromium-widevine/blob/fcaae76eeca9d5d900bd1a5b9ed2911555a6bf61/use-standalone-widevine.sh#L10
should not enforce /bin/sh, else I fear the shebang in the file executed is ignored, and the shebang selected shell is ignore (choosing another shell is a legit thing to do even if you fixed the script to a particular shell). This was what was happening locally, but I'm not an script expert.


Azure.Cosmos.Operation => Azure-Cosmos-Operation-Request-Diagnostics

/azp run

/azp run

/azp run

Do we need to update Aspire as well?

Do we need to update Aspire as well?

No, Azure.Cosmos.Operation is still the name of the Activity Source and Azure-Cosmos-Operation-Request-Diagnostics of the Event Source. I just changed a comment in this PR.

Do we need to update Aspire as well?

No, Azure.Cosmos.Operation is still the name of the Activity Source and Azure-Cosmos-Operation-Request-Diagnostics for the Event Source. I just changed a comment in this PR.


Update dockerfile for new repo

System Info
No response
Information

[ ] Docker
[ ] The CLI directly

Tasks

[ ] An officially supported command
[ ] My own modifications

Reproduction
No response
Expected behavior
No response

Completed with https://github.com/Gage-Technologies/embedding-server/commit/377051f07640a52595f13bdb37e7acab62dd6c9f


Frontend Project "URL Shortener" Paragraph Alignment Enhancement

Describe the Feature you want to end
Left-aligned paragraph.
Expected behavior
Centered aligned paragraph.
Screenshots (optional)
Current Alignment

Expected Updated Alignment

Additional context (optional )
N/A

@MohitGupta121


Open links in default browser, not embedded webview

We should change tapping on links to open them in the default browser, not the webview. (description edited by @WesleyAC, original below)

In many iOS apps these days, tapping links will open them by default in your preferred browser (recently added as option in iOS 14).
Zulip on iOS doesn’t appear to offer this, and only has an embedded webview, which adds extra steps, since each link needs to be shared over to the actual browser.
Please add some route to open links in the user’s chosen default browser, preferably by just tapping the link.

I am working in this

I am working on this !

Thanks @jryans for the report!
It looks like we actually switched from opening in Safari to an embedded Safari view, a few years ago, in #1721. There's no record of what the motivation for that was, though, unfortunately.
I'm not personally a real frequent iOS user, so I don't have a strong view myself on the best UX here. From a quick web search, I found this article fairly persuasive that an embedded Safari view might have been best a few years ago but opening in a full browser is the best choice now:
https://pspdfkit.com/blog/2019/open-links-in-safari-not-safari-view-controller/
So I agree, we should make this change.

I think we should do this for both iOS and Android, see #4562 for the reasons there. I'm going to edit this issue to be for both, rather than iOS specific.

For future reference: We ended up making this change by default on iOS, but leaving the in-app browser the default on Android, and offering a setting in either case. Rationale in chat discussion.


Import unpaywall since workflow start time

This is a POC. One issue we have in the kubernetes environment is, some of the importers (unpaywall, and open access button) take days to complete. If the job fails due to a node eviction, maintenance, etc, the importer has to start all over the next week. Utilizing Argo Workflow retry strategies, combined with import_since we can more confidently get through all the publications.
I set days_ago to 7, since we run this import weekly. this could be a rake argument that we set in the workflow. The workflow, upon fatal error, will retry with a back-off strategy. I hijacked the existing rake tasks, but I could see a scenario where we would want a days_since importer, and an all. This was just whipped up to get feedback on this idea.

Interesting concept.  I'm assuming there would be some way to pass the exact days_ago (or may even days and hours) based off how long the failed pod ran?

I think we can only pass an argument at runtime, so the thought is 'grab everything that hasn't been seen in the past 7 days and process it' If the job takes more than 7 days to run there is a fault in this logic, but given it takes less than 7 days to run I think it should work out. (I will admit date math always trips me up)
That said, an operator could override that value and force it to look further back, or closer.

If we 'grab everything that hasn't been seen in the past 7 days and process it' when the job retries after failing, then we wouldn't grab anything if last week's import ran to completion.  If we only ran the import again for things older than the time the job started, then we could get all the ones that didn't run.  You could potentially also do something like: grab everything with a last checked date beyond last Monday (if the import runs on Monday).  Seems like ruby should be able to do something like that.

I'm able to pass a 'workflow started at' time to the job, and I'm currently testing using that datetime as a marker, so something like
Publication.where(unpaywall_last_checked_at: nil..time_started_at)

I've tested this on my djb44 branch, and It appears to be working. If the ARGO_SUBMITTED_AT time is empty, it used Time.now which, I think could catch them all.

I renamed the spec to use import_before, and added two new specs to account for passing the date: parameter.


Thanks!

Hi, just a quick message to say thank you for this.  I've been waiting for something I can run on my pi as the android bridge is just frustrating to have a 2nd device.
This was so easy to use and I now have all my remotes created in node red.
Thanks again

Thanks too

You're welcome!


Moodown affecting the dragging bottoms' functioning when "customising this page"-ing moodle.Nottingham: My home

The module bars in the Module Overview block cannot be dragged (and hence reordered) when Moodown is enabled. Also the icon of the dragging bottoms which indicates the place where the mouse should be dragging have also been changed from the cross to some grey rectangular shape when Moodown is enabled.

Not sure if this problem is caused by Moodown. Anyway, now that the acceleration feature has been moved, Moodown should not interfere with the page content any more.


How can I use it in Node.js

I found the code already pulled out (and nicely in typescript) here in another comment by @pwnoy.
It looks like Highlightjs supports node out of the box along with plugins,
https://highlightjs.readthedocs.io/en/latest/plugin-api.html.  Could we reconfigure this package to be added via a highlightjs plugin for both node and the browser? Or would it be better to create a fork?


Could not find the following Boost libraries: boost_python3

libboost is installed here by default
/usr/lib/x86_64-linux-gnu/libboost_python-py3x.so
But cmake tries to find libboost_python-py3.so and fails.
This can be solved by making a symlink manually like so
cd /usr/lib/x86_64-linux-gnu/
sudo ln -s libboost_python-py3x.so libboost_python3.so
(replace x with your python version)
So please Link to the correct Boost.Python variant if the system Boost is used
Originally solved here

Works too

@gaoyangyiqiao , i dont have a mac, but it looks like you can install boost using brew.
Then i think you can look for the lib in brew's dir.
This might help?

@devxpy Thanks a lot! By brew install boost-python3 and then
cd /usr/local/lib/
sudo ln -s libboost_python36.dylib libboost_python3.dylib

I solved this problem. while I met another problem now,
Undefined symbols for architecture x86_64:
  "_PyInt_FromLong", referenced from:
      boost::python::tuple boost::python::make_tuple<boost::numpy::ndarray, int>(boost::numpy::ndarray const&, int const&) in segment_py.cpp.o
  "_PyObject_IsInstance", referenced from:
      boost::python::detail::caller_arity<4u>::impl<boost::python::tuple (*)(boost::numpy::ndarray const&, float, float, int), boost::python::default_call_policies, boost::mpl::vector5<boost::python::tuple, boost::numpy::ndarray const&, float, float, int> >::operator()(_object*, _object*) in segment_py.cpp.o
  "_PyObject_IsTrue", referenced from:
      check_image_format(boost::numpy::ndarray const&) in segment_py.cpp.o
  "_PyTuple_New", referenced from:
      boost::python::tuple boost::python::make_tuple<boost::numpy::ndarray, int>(boost::numpy::ndarray const&, int const&) in segment_py.cpp.o
  "_PyTuple_Type", referenced from:
      boost::python::detail::converter_target_type<boost::python::to_python_value<boost::python::tuple const&> >::get_pytype() in segment_py.cpp.o
  "__Py_NoneStruct", referenced from:
      __GLOBAL__sub_I_segment_py.cpp in segment_py.cpp.o
  "boost::numpy::initialize(bool)", referenced from:
      init_module_segment() in segment_py.cpp.o
  "boost::numpy::empty(int, long const*, boost::numpy::dtype const&)", referenced from:
      segment(boost::numpy::ndarray const&, float, float, int) in segment_py.cpp.o
      segment_label(boost::numpy::ndarray const&, float, float, int) in segment_py.cpp.o
  "boost::numpy::dtype boost::numpy::detail::get_int_dtype<32, false>()", referenced from:
      segment_label(boost::numpy::ndarray const&, float, float, int) in segment_py.cpp.o
  "boost::numpy::dtype boost::numpy::detail::get_int_dtype<8, true>()", referenced from:
      check_image_format(boost::numpy::ndarray const&) in segment_py.cpp.o
  "boost::python::detail::init_module(char const*, void (*)())", referenced from:
      _initsegment in segment_py.cpp.o
  "boost::python::converter::object_manager_traits<boost::numpy::ndarray>::get_pytype()", referenced from:
      boost::python::detail::caller_arity<4u>::impl<boost::python::tuple (*)(boost::numpy::ndarray const&, float, float, int), boost::python::default_call_policies, boost::mpl::vector5<boost::python::tuple, boost::numpy::ndarray const&, float, float, int> >::operator()(_object*, _object*) in segment_py.cpp.o
  "boost::numpy::ndarray::get_dtype() const", referenced from:
      segment(boost::numpy::ndarray const&, float, float, int) in segment_py.cpp.o
      check_image_format(boost::numpy::ndarray const&) in segment_py.cpp.o
  "boost::numpy::ndarray::get_flags() const", referenced from:
      check_image_format(boost::numpy::ndarray const&) in segment_py.cpp.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [segment.dylib] Error 1
make[1]: *** [CMakeFiles/segment.dir/all] Error 2
make: *** [all] Error 2

Bad luck. Thanks anyway.

You should find the file in brew's folder, not the system local lib?

I'm having a similar linking error using boost-python3 installed using brew:
Undefined symbols for architecture x86_64:
  "boost::python::detail::init_module(char const*, void (*)())", referenced from:
      _initlibbgs in bgslibrary_module.cpp.o
ld: symbol(s) not found for architecture x86_64

@devxpy, using brew install boost-python3 installs the library at /usr/local/Cellar/boost-python3/1.67.0 and creates a symbolic link at /usr/local/lib
libboost_python36-mt.dylib -> ../Cellar/boost-python3/1.67.0/lib/libboost_python36-mt.dylib

I also created the symbolic link following @gaoyangyiqiao comments
cd /usr/local/lib/
sudo ln -s libboost_python36.dylib libboost_python3.dylib

I fixed this problem by properly finding the Python interpreter according to BGS_PYTHON_VERSION variable in CMakelists.txt. See PR #127 for more details.

was bumping up with this error for boot_python2 - this
cd /usr/lib/x86_64-linux-gnu/
sudo ln -s libboost_python-py27.so libboost_python2.so
fixed things for me too. thanks

Your system probably has a old boost library that used a naming convention not compatible with current releases:
You can try renaming them like this:
cd /usr/lib/x86_64-linux-gnu/
sudo ln -s libboost_python-py35.so libboost_python3.so
sudo ln -s libboost_python-py35.a libboost_python3.a


Empty Cell Bug Fix

In the linalg module, I previously defined empty cells as the following:
empty_cells = np.where((mesh[VOLUME][t+1] == 0)))[0][0:self.nreal_count]
This does not correctly parse only real cells if

There are non-empty cells within the real cells
There are empty cells within non-real cells

I added the following, which fixes the issue.
empty_cells = np.where((mesh[VOLUME][t+1] == 0) & (np.arange(len(transport_model.mesh['volume'][t+1])) < self.nreal_count))[0][0:self.nreal_count]
I don't understand why the changes you made altered the representation of volume in the non-real cells, but if you feel OK to proceed this is a good bug fix to include either way!

Riverine simulation is running... looks good so far. I also do not understand why the non-real cell volumes changed. Both HEC-RAS simulations used version 6.4.1. I will open another issue if these riverine simulation results do not make sense. Thank you for the fix.


Rewrite shader and specialization handling in pipelines

Changelog:
### Breaking changes
Changes to shaders and pipelines:
- `EntryPoint` now owns instead of borrows a reference to the `ShaderModule`.
- Instead of an entry point and specialization constants, pipeline construction now takes a `PipelineShaderStageCreateInfo` structure.
- Specialization constants are now provided with a `HashMap` containing `SpecializationConstant` enum values. The `SpecializationConstants` trait is removed, and `vulkano_shaders` no longer generates structs for specialization constants.
- `GraphicsPipelineBuilder` now has a `stages` method that takes an iterator with all shader stages at once, instead of separate methods for each shader type.
- `GraphicsPipelineBuilder` now requires you to provide `input_assembly_state`, `rasterization_state`, `multisample_state` and `color_blend_state` instead of them having default values. You can still call `default()` to generate default values for each of them.

This makes specialization constants a bit more flexible: it's now possible to provide only some of the constants, and leave the rest at their default values. You no longer need to mess with the SpecializationConstants trait if you're not using Vulkano-shaders, you can just provide the constants directly by id without much issue. GraphicsPipelineBuilder also loses most of its type and lifetime parameters, which can be helpful for future changes.
The tessellation and dynamic-local-size examples show off the differences well.

Is it ok now?


Updating readme to support >=<EMAIL_ADDRESS>

Readme updates
Starting from<EMAIL_ADDRESS>createAtomicStyles and  createAtomsFn are deprecated now they renamed to use defineProperties and createSprinkles.
Reference:
https://github.com/seek-oss/vanilla-extract/releases/tag/%40vanilla-extract%2Fsprinkles%401.3.0

Thanks @maitzeth ❤


Add example code to Summaries

I was initially confused about how to use summaries. The only example code I found in the docs was on the page for list nodes, but that uses Render "summary", which is for views, not an article summary. I thought a little example here might clarify the issue for future users.

Merged, thanks.

Great addition, @philgs!  Your example certainly helped me.  Many thanks!


Quickfix: Update text color on FileRow

This sets the text color on the FileRow component to the default text color

This change is


cmake: allow color output from GCC/Clang

This enables color output from GCC and Clang if the environment
variable ZEPHYR_BUILD_COLOR_OUTPUT is set. This makes it a bit
easier to identify compilation errors.
Note that this is enabled manually rather than as default simply
to avoid complications with build systems or various tools which
may not like ANSI color tags.
Signed-off-by: Daniel Leung<EMAIL_ADDRESS>

@nashif what were the reason you closed #25557 ?

probably a mistake?

I can close this one if you decide to merge #25557.


better exception JavaDoc

@chanseokoh fix #279

Make sure to keep @loosebazooka and core library PRs.

Codecov Report

Merging #420 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master    #420   +/-   ##
======================================
  Coverage    65.5%   65.5%
======================================
  Files          64      64
  Lines        1757    1757
  Branches      270     270
======================================
  Hits         1151    1151
  Misses        492     492
  Partials      114     114

Impacted Files
Coverage Δ

...ppengine/cloudsdk/CloudSdkAppEngineDevServer1.java
86.99% <ø> (ø)
:arrow_up:

.../appengine/cloudsdk/CloudSdkAppEngineVersions.java
90.38% <ø> (ø)
:arrow_up:

.../appengine/cloudsdk/CloudSdkAppEngineServices.java
82.6% <100%> (ø)
:arrow_up:

...appengine/cloudsdk/CloudSdkAppEngineGenConfig.java
59.09% <100%> (ø)
:arrow_up:

...ools/appengine/cloudsdk/CloudSdkAppEngineLogs.java
90% <100%> (ø)
:arrow_up:

...appengine/cloudsdk/CloudSdkAppEngineInstances.java
92% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c9227f...ffa0575. Read the comment docs.


Fix test error not showing when expanded

This PR fixes #153993

/assign @connor4312

Thanks for the detailed issue and PR!


Retry subclasses of retryable exceptions

@barisbalic this is how gocardless/activejob-retry works, and I think it's quite nice. Means you can just retry SystemCallError instead of Errno::ETIMEOUT and all its brothers and sisters. I can't get rspec to run at all locally - do we have CI on this repo?

Can't touch dis. Bowm, bowm bowm bowm, bowm bowm.

oops sorry guys, inadvertently clicked on "delete"


Upgrade Ruby to version 2.1.5 as 1.9.3 is deprecated

Closing until we can fix the test failures.

Thanks! I meant to do this last night but forgot all about it.

@evilstreak @issyl0 @alext
I've gotten the build to green by:

upgrading sass and sass-rails to get asset compilation working
introducing @alext's invalid_utt8_rejector gem to catch malformed UTF8 paths, meaning we shouldn't need the failing UTF8 tests anymore

Since Ruby 2.2.1 is on the servers now should we go to that?

Since Ruby 2.2.1 is on the servers now should we go to that?

Actually, 2.2.2 would be a better target (rolled out today).

@bradleywright @alext @evilstreak Rails 3.2.X has some unresolvable warnings with ruby 2.2.2, so even though the tests pass, I am reluctant to upgrade ruby all the way to 2.2.2 until we've migrated this app to Rails 4.

:+1: Sounds sensible to me


Remove 'Recipes' start screen if multiple recipes are already there.

The recipes start screen (showing the "Recipes make sure your devices..." notification) should only be shown when there are no recipes yet on the remote. Else it is a useless 'next' i have to press...

Thanks for the report @Webunity
As this issue is not related to the SDK I will close this.


Deep hash structure are not expanded properly

When retrieving a deep hash structure, the datastore tries to spread the data using dot notation to allow for the data to be searchable. A structure containing an array should use "bracket notation" and be expanded to an array as expected.
const object = {
  key: [
    'value1',
    'value2'
  ];

const expandedObject = NucleusDatastore.expandDotNotationObject(object);

chai.expect(expandedObject.key[0]).to.equal('value1');

Fixed in 0.5.6;


fail to train when reaching roi_align

Thanks for your error report and we appreciate it a lot.
Checklist

I have searched related issues but cannot get the expected help.
The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.
when I train the cascade_mask_rcnn_r101_fpn_1x on my own dataset(COCO format). It reports an error.
THCudaCheck FAIL file=mmdet/ops/roi_align/src/roi_align_kernel.cu line=140 error=48 : unrecognized error code
Traceback (most recent call last):
File "tools/train.py", line 142, in
main()
File "tools/train.py", line 138, in main
meta=meta)
File "/content/drive/My Drive/mmdetection/mmdet/apis/train.py", line 111, in train_detector
meta=meta)
File "/content/drive/My Drive/mmdetection/mmdet/apis/train.py", line 231, in non_dist_train
runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/runner.py", line 371, in run
epoch_runner(data_loaders[i], **kwargs)
File "/usr/local/lib/python3.6/dist-packages/mmcv/runner/runner.py", line 275, in train
self.model, data_batch, train_mode=True, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/apis/train.py", line 75, in batch_processor
losses = model(**data)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
return self.module(*inputs[0], **kwargs[0])
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/core/fp16/decorators.py", line 49, in new_func
return old_func(*args, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/models/detectors/base.py", line 147, in forward
return self.forward_train(img, img_meta, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/models/detectors/cascade_rcnn.py", line 246, in forward_train
rois)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/core/fp16/decorators.py", line 127, in new_func
return old_func(*args, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/models/roi_extractors/single_level.py", line 105, in forward
roi_feats_t = self.roi_layers[i](feats[i], rois)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/content/drive/My Drive/mmdetection/mmdet/ops/roi_align/roi_align.py", line 83, in forward
self.sample_num)
File "/content/drive/My Drive/mmdetection/mmdet/ops/roi_align/roi_align.py", line 28, in forward
sample_num, output)
RuntimeError: cuda runtime error (48) : unrecognized error code at mmdet/ops/roi_align/src/roi_align_kernel.cu:140
Reproduction

What command or script did you run?

python tools/train.py configs/cascade_mask_rcnn_r101_fpn_1x.py

Did you make any modifications on the code or config? Did you understand what you have modified?
I modified:
#1:initial lr to 0.0025
#2:dataset to MyDataset
#3dataset path

What dataset did you use?
My own dataset with COCO format and I set dummy "segmentation" in annotations as a rect.
Environment

Please run python mmdet/utils/collect_env.py to collect necessary environment infomation and paste it here.

sys.platform: linux
Python: 3.6.9 (default, Nov  7 2019, 10:44:02) [GCC 8.3.0]
CUDA available: True
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 10.0, V10.0.130
GPU 0: Tesla T4
GCC: gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
PyTorch: 1.4.0
PyTorch compiling details: PyTorch built with:

GCC 7.3
Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v0.21.1 (Git Hash 7d2fd500bc78936d1d648ca713b901012f470dbc)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CUDA Runtime 10.1
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
CuDNN 7.6.3
Magma 2.5.1
Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

TorchVision: 0.5.0
OpenCV: 4.1.2
MMCV: 0.3.2
MMDetection: 1.1.0+58a3f02
MMDetection Compiler: GCC 7.4
MMDetection CUDA Compiler: 10.0

You may add addition that may be helpful for locating the problem, such as

How you installed PyTorch [e.g., pip, conda, source]
pip
Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.
in the above

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

I mean you can debug during training to see if the inputs are invalid when the training crashes.


[WIP] Update docker

[x] openmpi upgrade to be able to call mpirun outside the container
[x] clang+openmp upgrade
[x] clang-format upgrade
[x] system package manager install of clang-3.9
[x] boost upgrade
[x] trilinos and kokkos updates
[ ] update documentation

Related to #302

We will need to change the tag in jenkins matrix when we merge this.

Bruno please update description when this is ready for review.  There are a lot of changes happening in this PR and it would be easy to overlook some of them.

[ ] openmpi upgrade to be able to call mpirun outside the container
[ ] clang+openmp upgrade
[ ] clang-format upgrade
[ ] system package manager install of clang-3.9
[ ] boost upgrade
[ ] trilinos and kokkos updates

BTW, do we need to update our .clang-format to use the "ReflownComments" you mentioned in #302?
I looked at the logs on Travis and they essentially are changes in indentations/line breaks.

It's enabled by default Option ReflowComments added and enabled by default

Codecov Report

Merging #307 into master will increase coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #307      +/-   ##
==========================================
+ Coverage   95.69%   95.73%   +0.03%
==========================================
  Files          61       61
  Lines        3530     3538       +8
==========================================
+ Hits         3378     3387       +9
+ Misses        152      151       -1

Impacted Files
Coverage Δ

packages/Interface/src/DTK_PolyhedronList.hpp
100% <ø> (ø)
:arrow_up:

...earch/test/tstDetailsDistributedSearchTreeImpl.cpp
96.74% <ø> (ø)
:arrow_up:

packages/Interface/src/DTK_CellList.hpp
100% <ø> (ø)
:arrow_up:

packages/Search/test/tstLinearBVH.cpp
100% <100%> (+0.24%)
:arrow_up:

packages/Search/src/DTK_LinearBVH_def.hpp
100% <100%> (ø)
:arrow_up:

...c/details/DTK_DetailsDistributedSearchTreeImpl.hpp
98.86% <100%> (+0.01%)
:arrow_up:

...ackages/Search/test/tstDetailsTreeConstruction.cpp
98.78% <100%> (ø)
:arrow_up:

packages/Search/src/DTK_LinearBVH_decl.hpp
100% <100%> (ø)
:arrow_up:

packages/Search/test/tstDistributedSearchTree.cpp
98.55% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45051c5...d5c100c. Read the comment docs.

This is ready for review/merge.  The images have already been tagged and pushed to Docker Hub.

@aprokop I have no opinion on the format.  Some changes look good to me, some not so good.  The format configuration is quite arbitrary, you are welcome to do try some changes in the .clang-format configuration file if you like.


Get Financials Endpoint Does Not Handle Blank Values

Summary (include Python version)

a = Stock('AADR')
a.get_financials()
Traceback (most recent call last):
File "", line 1, in
File "/Users/ericxiao/.local/share/virtualenvs/bloom-Eg4XHhZg/lib/python3.6/site-packages/iexfinance/stock.py", line 32, in _format_wrapper
response = func(self, *args, **kwargs)
File "/Users/ericxiao/.local/share/virtualenvs/bloom-Eg4XHhZg/lib/python3.6/site-packages/iexfinance/stock.py", line 391, in get_financials
for symbol in list(data)}
File "/Users/ericxiao/.local/share/virtualenvs/bloom-Eg4XHhZg/lib/python3.6/site-packages/iexfinance/stock.py", line 391, in
for symbol in list(data)}
KeyError: 'financials'

Date/time of issue
Expected behavior
It should handle empty state for finances more gracefully and return an empty dictionary
Actual behavior
On bulk entry for get_financials(), it breaks and shows the KeyError exception

Hello @exiao,
this issue has been fixed with commit https://github.com/addisonlynch/iexfinance/commit/4912720b16a6cf8076576df78e9c55f977e12c6e on branch Dev.
You can checkout that branch or wait until a new release comes out

@exiao the master branch has the same commits and should fix the problem as well. Install via pip install git+https://github.com/addisonlynch/iexfinance. Will be pushed to Pypi in the next release.

Awesome, thank you :)


Only download annotations.

Is your feature request related to a problem? Please describe.
I think downloading just the annotations is essential since most people already have the data.
Describe the solution you'd like
currently exporting in yolo format downloads the uploaded images which is unnecessary. I just want to download the segmentation coordinates.
Thank you.

got same issue

Check this endpoint
https://api.labelstud.io/#operation/api_projects_export_read
And use it with download_all_resources=false


multiProducers and multiConsumers

multiProducers and multiConsumers  can work  simultaneously  safely ?

Not yet. The DefaultWriter needs to be synchronized right now using a mutex (which is bad). It does support multiple readers.


AddRidToRuntimeJson doesn't work for BSD or musl based distros

source-build has a task that adds the current RID to the runtime.json file in Microsoft.NETCore.Platforms.
However, this task assumes that the current RID is based on the linux base RID, which is for libc based Linux distros.  The linux RID is not applicable for BSD or musl based distros.
https://github.com/dotnet/source-build/blob/aea4b384cfb5077422f8ee0dd967251ed1d4c6a7/tools-local/tasks/Microsoft.DotNet.SourceBuild.Tasks/AddRidToRuntimeJson.cs#L56-L57
In order for source-build to support Unix distros that are not libc-based Linux, we need to fix this task to add the correct RID graph in those scenarios.
cc @dotnet/source-build-contrib

AddRidToRuntimeJson was removed with https://github.com/dotnet/installer/pull/17760.  Closing as no longer applicable.


Use of auth_uri and token_uri from auth configuration object

Question: Are the auth_uri and token_uri values from the auth configuration object downloaded from the developer console used by the library?
Example:
{
    "web": {
        "client_id": "XXXXX.apps.googleusercontent.com",
        "project_id": "dummy-id",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://accounts.google.com/o/oauth2/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_secret": "YYYYYY",
        "redirect_uris": [
        ],
        "javascript_origins": [
        ]
    }
}

I looked through the code and searched here and did not find any obvious usage of these.
We have a use case where we need to be able to direct some requests to a different host (think something like Runscope or an internal proxy) by providing the new URIs via the configuration object.

@bshaffer Do you happen to know if the auth_uri and token_uri values from the auth configuration object downloaded from the developer console used by the library?

It looks like they are hardcoded: https://github.com/google/google-api-php-client/blob/a68d080658241cd059cd3434c55193630197f47f/src/Google/Client.php#L947


Fix FavouriteItemsTest that fails frequently on GHA

Checklist

[x] the [contributor license agreement][] is signed
[x] commit message follows [commit guidelines][]

Description of change
testAutoLoggedIn, which is part of FavouriteItemsTest, has been failing quite frequently on GHA. Since the error is  NoSuchElementException and the test is really simple, the first attempt to fix this issue is adding a explicit wait.

I will re-run several times and see if this test fails again.

Well it's all green currently - other than our ol' mate Travis - so I reckon let's merge and see how it goes. It should definitely be an improvement.


Fix install-tools target

It needs GO111MODULE env var and the recently added addlicense tool.
/cc @ccaraman

Codecov Report

Merging #195 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #195   +/-   ##
=======================================
  Coverage   72.74%   72.74%
=======================================
  Files         105      105
  Lines        6094     6094
=======================================
  Hits         4433     4433
  Misses       1435     1435
  Partials      226      226

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 091e19d...b5e855f. Read the comment docs.

Please rebase.


PM2 support

Supporting vertical scalability (https://github.com/gamestdio/colyseus/issues/43) introduced the usage of the cluster module, and manual message passing between workers.
PM2 doesn't support message passing like this, though.

vote up :+1:


chore: rerouting "get started page" and "dashboard" through a redirect

https://trello.com/c/32XaLpuE/57-react-routing-and-redirect-start-on-the-informational-page-login-to-the-dashboard

App now redirects to SaverLife informational page
App will redirect to the informational page whenever somebody is not logged in
App will skip over the informational page when you are logged in.

 React-Routing and Redirect: start on the informational page login to the dashboard


why sub and mul operation of f32x4 is slower than scalar version?

//cost 13ms when data.length is 4000w
fn sum_scalar(data: &[f32]) -> f32 {
    let mut sum = 0.0;
    let pos = vec![2000.0, 2000.0, 2000.0];
    let dir = vec![0.8, 0.6, 0.0];
    for i in 0..data.len() / 4 {
        let x = data[i * 4 + 0];
        let y = data[i * 4 + 1];
        let z = data[i * 4 + 2];
        sum += (x - pos[0]) * dir[0] + (y - pos[1]) * dir[1] + (z - pos[2]) * dir[2];
    }
    sum
}

//cost 16ms when data.length is 4000w
fn sum_simd(data: &[f32]) -> f32 {
    let pos = f32x4::load_or_default(&vec![2000.0, 2000.0, 2000.0]);
    let dir = f32x4::load_or_default(&vec![0.8, 0.6, 0.0]);

    let mut sum = 0.0;
    for i in 0..data.len() / 4 {
        let values = f32x4::from_array([
            data[i * 4 + 0],
            data[i * 4 + 1],
            data[i * 4 + 2],
            data[i * 4 + 3],
        ]);
        sum += ((values - pos) * dir).reduce_sum();
    }

    sum
}

cpu: Intel(R) Core(TM) i5-14400F   2.50 GHz
rustc 1.81.0-nightly (c1b336cb6 2024-06-21)

SIMD instruction sets are optimized for operating on independent lanes. This makes the reduce_sum operation relatively expensive. You can get better performance by using a f32x4 for the sum and then only doing a single horizontal reduction at the end.
Note that the compiler is not able to do this optimization for you, since float operations are not associative.
On a nightly rust version you could also look into the fadd_algebraic / fsub_algebraic / fmul_agebraic intrinsics, which allow the compiler to reorder and auto-vectorize scalar floating point operations.

@jhorstmann Thank you so much. After replace the -* of sum_scalar to fsub and fmul, the runtime of sum_scalar decreased by about 20%.
But things start to be weird when I change the code to
fn sum_scalar(data: &[f32], res: &mut [f32]) {
    let pos = vec![2000.0, 2000.0, 2000.0];
    let dir = vec![0.8, 0.6, 0.0];
    for i in 0..data.len() / 4 {
        let x = data[i * 4 + 0];
        let y = data[i * 4 + 1];
        let z = data[i * 4 + 2];
        res[i] = (x - pos[0]) * dir[0] + (y - pos[1]) * dir[1] + (z - pos[2]) * dir[2];
    }
}

unsafe fn sum_fast(data: &[f32], res: &mut [f32]) {
    let pos = vec![2000.0, 2000.0, 2000.0];
    let dir = vec![0.8, 0.6, 0.0];
    for i in 0..data.len() / 4 {
        let x = data[i * 4 + 0];
        let y = data[i * 4 + 1];
        let z = data[i * 4 + 2];
        *res.get_unchecked_mut(i) = fmul_fast(fsub_fast(x, pos[0]), dir[0])
                + fmul_fast(fsub_fast(y, pos[1]), dir[1])
                + fmul_fast(fsub_fast(z, pos[2]), dir[2])
    }
}

the runtime of sum_scalar and sum_fast become almost equal. I am a beginner to rust, so the results are werid to me.

@Jad2wizard very good question! I see two possible reasons for these having the same performance:

The compiler is not able to remove the bounds checks when accessing data. This means it still processes a single res value per loop iteration, and the overhead of the bounds checks masks any performance difference between the intrinsics and normal arithmetic.

The other issue is related to the data layout. For each res value, the code needs to process 3 different indices of data values. If the compiler wants to vectorize this to produce 4 results at a time for example, each of the 4 lanes of x, y and z would need to be loaded from a different, non consecutive address. Modern cpus have instructions for that, but they are much slower than loading data that is consecutive in memory. Fixing this would require to change your data structures to have different slices for x, y and z.

Without the data layout changes, this is how I would write the function to avoid the bounds checks:
pub fn sum_fast_no_bound_checks(data: &[f32], res: &mut [f32]) {
    let pos = vec![2000.0, 2000.0, 2000.0];
    let dir = vec![0.8, 0.6, 0.0];
    for (r, chunk) in res.iter_mut().zip(data.chunks_exact(4)) {
        let x = chunk[0];
        let y = chunk[1];
        let z = chunk[2];
        *r = fmul_algebraic(fsub_algebraic(x, pos[0]), dir[0])
                + fmul_algebraic(fsub_algebraic(y, pos[1]), dir[1])
                + fmul_algebraic(fsub_algebraic(z, pos[2]), dir[2])
    }
}

With chunks_exact the compiler knows that the indices are in bounds, and zip ensures that the res corresponding to each chunk is also in bounds.

Thank you @jhorstmann, I will try change the data layout as you mentioned


Add a Style::with_writer() method, or an RAII guard equivalent

I have an interesting use case:

I'd like to print out a long string representing stdout from a process
I'd like to apply a color to the entire string
I'd like to strip ANSI escape codes within the string

This doesn't seem possible with owo-colors, but it seems like it would be possible with a little bit of infrastructure:
impl Style {
    fn with_writer<F, T, E, W>(writer: W, f: F) -> Result<T, WithWriterError<E>>
        where F: FnOnce(W) -> Result<T, E>, W: std::io::Write,
}

// this is optional but more generic than forcing F to return std::io::Write
enum WithWriterError<E> {
    Internal(std::io::Error),
    Callback(E),
}

impl WithWriterError<std::io::Error> {
    // flattens the error in case f returns std::io::Error, which is the common case
    pub fn flatten(self) -> std::io::Error {
        ...
    }
}

This would make it possible to write:
style.with_writer(&mut writer, |writer| {
    let mut no_color = strip_ansi_escapes::Writer(writer);
    no_color.write_all(stdout);
}).map_err(|err| err.flatten())?;

The callback pattern can also makes it more robust to panics, assuming an RAII guard is used. (Or actually, it might be a better design to just make an RAII guard part of the public API!)
What do you think?

This would be very useful to me too.
An alternative way to achieve the same thing (although I am not sure about the exception safety of such a solution)  would be to have a writer wrapper in the spirit of (not tested, but it compiles)
struct StyledWrite<'a, T: Write> {
	style: Style,
	inner: &'a mut T,
}

impl<'a, T: Write> StyledWrite<'a, T> {
	fn new(inner: &'a mut T, style: Style) -> Self {
		// write the style prefix sequence into inner here
		Self { inner, style }
	}
}

impl<T: Write> Drop for StyledWrite<'_, T> {
	fn drop(&mut self) {
		// write the style suffix sequence into inner here
	}
}

impl<T: Write> Write for StyledWrite<'_, T> {
	fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
		self.inner.write(buf)
	}

	fn flush(&mut self) -> std::io::Result<()> {
		self.inner.flush()
	}
}

trait WriteExt<T: Write> {
	fn styled(&mut self, style: Style) -> StyledWrite<'_, T>;

	fn red(&mut self) -> StyledWrite<'_, T>;
}

impl<T> WriteExt<T> for T
where
	T: Write,
{
	fn styled(&mut self, style: Style) -> StyledWrite<'_, T> {
		StyledWrite::new(self, style)
	}

	fn red(&mut self) -> StyledWrite<'_, T> {
		self.styled(Style::new().red())
	}
}

which would enable an even nicer API for styling the whole writer for a limited time:
write!(myWrite.red(), "everything is red!");
write!(myWrite.styled(Style::new().bold()), "everything is bold!");

UPDATE: playing with this a little more, it works pretty well, but the writing of a suffix sequence in drop is not infallible, and I am not sure what the proper way of handling errors in drop is. unwrap is IMO pretty unacceptable, because it makes proper error handling impossible, but ignoring the error also does not feel right.

Ah sadly with the RAII pattern, ignoring the error is the only reasonable option in Drop -- . unwrap is definitely much worse because Drop is often called during panics, and double-panicking is a no-no.
BufWriter in Rust's std follows this pattern: https://doc.rust-lang.org/src/std/io/buffered/bufwriter.rs.html#673-680
I'd personally say that dropping causing a color reset is a bit too magical, which suggests to me that the callback pattern would be better.


neural hawkes process

Hi
I'm interested in implementing your work in pytorch and use it in my personal project
I found your repo of your latest work, neural hawkes particle smoothing,
and it seems using C-Lstm which is I guess the main part of neural hawkes process.
So is it really identical?
What part of NHPS is the original neural hawkes process?
Thanks

Hi
I'm interested in implementing your work in pytorch and use it in my personal project

Thanks for your interests!
Sorry for the late reply, but I'd like to answer any questions you have for implementing it!

I found your repo of your latest work, neural hawkes particle smoothing,
and it seems using C-Lstm which is I guess the main part of neural hawkes process.

Yes, you are right!

So is it really identical?

Part of it.
The neural Hawkes particle filtering part is almost identical to the original NHP, because it requires a trained NHP. You can refer to the section-2 and -3.1 of this ICML paper https://arxiv.org/pdf/1905.05570.pdf for details.

What part of NHPS is the original neural hawkes process?

If you want to know which part of that codebase is exactly NHP, maybe you can leave questions there, so I can answer them and also revise my README accordingly?
Or, maybe even more convenient for you, I have a PyTorch version of NHP that is not ready to release yet, but I am okay sharing it with you in person. So maybe you can send me an email if you'd like it?

Thanks

Thank you!

@HMEIatJHU  Can you share me the pytorch version NHP model? I have also send you a mail asking for the same.


Mouse Highlighting: Tooltips don't display for widget actions

Pointed out by @Tyler#5378 in the discord:
Aside from the item stack widget, we don't show a tooltip when some widget has a left click action. The panel switch buttons are the most prominent and easily-accessible example of this, but it'd be nice to show a tooltip for widget-triggered actions in general.
For comparison's sake hovering over one of the panel-change buttons in Konduit looks like this

So this happens because those menu options have no target
https://github.com/runelite/runelite/blob/6239baaa9eeac8924cb07e24ef3ca1453f613076/runelite-client/src/main/java/net/runelite/client/plugins/mousehighlight/MouseHighlightOverlay.java#L72-L75
Not sure what the repercussions of removing that empty check would bring though.

Probably doing something like this:
-if (target.isEmpty()) {
+if (Strings.isNullOrEmpty(option)) {
  return null;
}

and
-tooltipManager.add(new Tooltip(option + " " + target));
+tooltipManager.add(new Tooltip(option + Strings.isNullOrEmpty(target) ? "" : " " + target));

should work ™


Update polymer.d.ts to ver.1.1.2

Update polymer.d.ts to ver.1.1.2 includes es6 class syntax support.
https://github.com/Polymer/polymer/releases/tag/v1.1.2

@laco0416 thanks mate!

Thanks!


Implement clang-tidy

This addresses #63 (see that ticket for context).
This PR builds in:

clang-tidy checks. If clang-tidy detects errors travis will fail
The -fix command is passed so some fixes are automatically done
Other warnings need to be noticed and fixed manually

Other notes:

Uses a custom python script to convert the Makefile output into compile_commands.json
Uses clang-tidy from mason, version controlled in the setup.sh script - refs #58

/cc @GretaCB for review
/cc @mapbox/directions @mapbox/gl-core - thank you for the examples of clang-tidy usage in OSRM and MBGL - I learned from them in order to apply here ✊

🎉  travis properly caught the clang-tidy changes to source files and errored: https://travis-ci.org/mapbox/node-cpp-skel/builds/265773011. Digging in the logs for the make tidy target show why. Going to now create a separate PR that will address the fixes needed.

Fixed landed. Will merge this once travis is green.


 [RPD-T204] Refactor go.mod

Describe the PR
e.g. add cool parser.
Relation issue
e.g. https://github.com/swaggo/swag/pull/118/files
Additional context
Add any other context about the problem here.

Close this PR


Need Revenue query

One would think that a query based on the tag "Revenues" would be all that is required.  Alas, no.  How close is the following to a complete and correct form of the query (for USD-only reports)?
create or replace view sec_dera.fy_revenue_usd_by_lei as
select S.adsh, cik, name, lei, sic, fy, N.ddate, uom, max(value) as revenue
from sec_dera.sub as S join sec_dera.num as N on S.adsh=N.adsh
where (S.form='10-K' or S.form='20-F' or S.form='40-F')
        and S.fy>=DATE('2019-01-01')
        and uom='USD'
        and coreg is NULL
        and qtrs=4
        and (N.tag='Revenue'
             or N.tag='Revenues'
             or N.tag='RevenueFromContractsWithCustomers'
             or N.tag='RevenueFromContractWithCustomerIncludingAssessedTax'
             or N.tag='RevenueFromContractWithCustomerExcludingAssessedTax'
             or N.tag='RevenuesNetOfInterestExpense'
             or N.tag='RegulatedAndUnregulatedOperatingRevenue'
             or N.tag='RegulatedOperatingRevenuePipelines')
group by S.adsh, cik, name, lei, sic, fy, N.ddate, uom

Note that many companies report not only FY data, but often two prior years.  This leads to:
FY2021: 2021, 2020, 2019 data
FY2020: 2020, 2019, 2018 data
FY2019: 2019, 2018, 2017 data
Its nove to have 2017-2021 data, but not nice to have three copies of 2019 data, and two copies each of 2020 and 2018 data.  What is the SQL idiom for thinning this out?
@toki8: please feel free to assign to appropriate resource.

Note that the PRE (presentation) table has tags for income statement (IS), which can help identify which of the revenue tags are actually presented as rolled-up numbers rather than merely constituent elements.


docs: add additional badges to readme

Description
This PR proposes a few more badges for the readme. This shows clearly at first glance, that the project uses pre-commit and ruff. Also there is an indicator of the number of downloads from pypi. What do you think?
The order is not fixed. We can rearrange of course.
Types of Changes

[x] Documentation content changes

Checklist

[x] I have read the contributor guide.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Hmm, I like the ruff one, but I don't like the downloads (I think those kind of stats are misleading). It also looks a bit much. Can you remove the empty lines so that the badges are in one line and wrapping automatically (I hope).

I moved pre-commit and ruff inwards, because those will not change any time soon. But CI, Coverage and Docs status badges might change with each commit. So I put those at the end of the line.
What do you think?

Maybe we should use something like 3.8 - 3.12 instead of listing all versions. If we save some space the shields would not need to wrap anymore. Also I think the docs badge is weird since it concerns a different repo.

Alright, remove the "docs: passing" badge then and only show docs status in docs repo. The badge was already there, I did not add it.
Sure we can hardcode 3.8 - 3.12. I added a dynamic badge recently, so that we never forget to update here. But yes 3.8,9,10,11,12 is quite a long badge.
Will adapt soon.

Add markdown line break after license.

Thanks! We need another badge for the first row, then it would look better. Maybe we find something.

Thanks! We need another badge for the first row, then it would look better. Maybe we find something.

Maybe a badge about the Django version?

Something like:
This badge is usually used for a library, that supports multiple django versions, e.g. 3,4,5... This could show at a quick glance, that django is at 4.2.

Its maybe also cool to show that it is django based.

I came across this one recently:
It shows if there is recent activity.

ne, die finde ich alle ästhetisch schwierig, ich glaube das mit der django version reicht.

Ready.

these admin.register decorators are also in this PR?

these admin.register decorators are also in this PR?

Sorry, no, I need to rebase. That happened because I switched the base branch of thir PR from 2.10 to 2.02. 2.10 contains the admin-register commit. I will fix this.

Fixed.

Can I merge @MyPyDavid?


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/24713

Triggering TagBot for merged registry pull request

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/27146

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/42233

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/69801

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/69893

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/98760

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.


Google Cloud PaLM API integration

Google has just announced the PaLM API https://developers.googleblog.com/2023/03/announcing-palm-api-and-makersuite.html!!

The PaLM API will be released with a new tool called MakerSuite, which allows users to prototype ideas, do prompt engineering, synthetic data generation, and more.

https://github.com/hwchase17/langchain/pull/3575 merged yesterday. Supports PaLM's Text, Chat, and Embeddings APIs.


Addendum to OneToOneInlineForm feature

Addresses an issue found with #2091 where inline forms were not being displayed due to looking for the opposite relationship type on each mapper.

pinging @mrjoes @michaelbukachi

Looks good.