add win_ros script wrappers to make python scripts executable

Python scripts are executable (can be directly called on command line with the help of the shebang line) on Ubuntu, so built packages can be directly used in command line. However, without extra setup on Windows, python scripts cannot be executed directly as on Ubuntu. To work around this issue, we utilize 2 kinds of script wrappers to execute the python scripts.
One way of doing it is creating a batch script that simply adds that python prefix before the python script; the other way of doing it is to wrap the python script inside a win32 wrapper, and execute it using the CreateProcess call (and return error code).
Similar to the change to CMakeLists.txt in this pull request, adding a if(WIN32) + add_windows_helper would be the easiest way to create executable for packages.

looking back at this change, this seems like a hacky way of fixing Python scripts on Windows. @seanyen-msft found the use of setuptools in Python to generate console_scripts, which should help avoid the executable generation problem (Python scripts needs a wrapper to be called on Windows). what's even better is that this would be a maintained method
I will be doing some prototyping to see if this is going to work. @wjwwood and @dirk-thomas, does this sound like something we/ros could use?

We use the console_scripts entry point in all the other python tools that we support and that work on Windows (AFAIK). So it would make sense to use them I think, though I'm not sure the layout of the existing scripts lends themselves to that, so you may need to move code around. But ultimately I think it is probably better than having our own executable for this case.

We use the console_scripts entry point in all the other python tools that we support and that work on Windows (AFAIK). So it would make sense to use them I think, though I'm not sure the layout of the existing scripts lends themselves to that, so you may need to move code around. But ultimately I think it is probably better than having our own executable for this case.

good to know, I'll update on this thread while I try. we have a feeling that having some sort of wrapper might later on become hard to maintain

We use the console_scripts entry point in all the other python tools that we support and that work on Windows (AFAIK).

I did some investigation and prototyping on this and here are a little of conclusion:

changing to use setuptools.setup with entry_points={'console_scripts': [...]} works well for (local) install destination install_isolated (also fairly easy to implement)
haven't had time to dive deeper into catkin build logic to get this working for devel space, there likely needs to be extra steps during build time
console_scripts as entry_points from setuptools would associate the generated .exe file with the local Python interpreter on the local machine. that means, if the build machine on which the Windows package is built has Python interpreter installed at C:\Python2, the .exe would always expect to find Python interpreter wherever it's executed. Up to this point, ROS on Windows porting uses a separate Python installation to work around the "Python already exists" problem. This means the Python interpreter for any (Windows) ROS project would be at a customized local directory, basically making it impossible to work on another machine or even maintain.

with Embedded World approaching, my to-do to unblock us/ROS from this pull request will be to rewrite our own version of cmake/templates/ros_bin.cpp, as a matter of fact, this version of ros_bin.cpp is already different from the original code given it now returns error code of the process (which is important for process monitoring in ROS)
justifications are as follows:

the easiest/most straight-forward approach to make Python scripts executable is to add python at the beginning to execute the Python script with the Python interpreter registered in the ROS environment.
the current problem does not require the .exe win32 wrapper to work independently, it only needs to execute the actual .py file
keeping the .py file also makes it easy for users to see the actual code

some thoughts from working on this:

has ROS ever considering working with virtualenv for isolated Python environment? We had tons of troubles working with (polluted) environments with existing Python installation, venv might be one (easy) way to solve it, imo
console_scripts seems to well for packages like rosinstall_generator, which are installed as Python package. for local ROS packages, I am not sure if it will ever work. maybe it will become feasible once we have an isolated dev environment with virtualenv. any thoughts on this?

has ROS ever considering working with virtualenv for isolated Python environment?

While the instructions don't prescribe the use of virtualenv you are free to use them if you like. I would say ROS is agnostic to it (at least on non Debian platforms since on Debian platforms we do rely on the globally installed Debian packages).

console_scripts seems to well for packages like rosinstall_generator, which are installed as Python package. for local ROS packages, I am not sure if it will ever work. maybe it will become feasible once we have an isolated dev environment with virtualenv. any thoughts on this?

I don't think the presence of a virtual env makes any difference for the console scripts. Especially since the console scripts should be installed globally / in the virtual env but in the install space. One major limitation of console scripts (afaik) is that you can only install them to one location - usually bin - with some effort also into lib/<pkgname>. But you can't install one script into bin and another into lib/<pkgname>.

@kejxu Please comment on the ticket after committing to the branch in order to notify others of the change.

One major limitation of console scripts (afaik) is that you can only install them to one location - usually bin - with some effort also into lib/<pkgname>. But you can't install one script into bin and another into lib/<pkgname>.

Thanks for sharing your expertise! This would make console scripts barely practical, thanks!

Please comment on the ticket after committing to the branch in order to notify others of the change.

Sorry for the confusion, update on this ticket to address all the comments!

I polished the CMake function add_python_executable a bit in 1810908f4e595364d8dee4c2569c6f88ffdfc5e5 (I hope I didn't break anything).
Since the _setup_util.py file will also get an executable wrapper with this PR should the invocation in the setup.bat file be updated to call that executable instead? That would get rid of the custom logic around the Python interpreter in that file.

I polished the CMake function add_python_executable a bit in 1810908 (I hope I didn't break anything).

thanks for helping out! it looks a lot better than what I could come up with my understand of CMake =) and thanks for moving configure_file(WRAPPER_SOURCE) into if(WIN32)! I did a local build and it worked fine; and I will be testing this with our daily build to make sure it does not break anything else
quick question, do we want to change the name mentioned in wrapper.cpp.in?

Since the _setup_util.py file will also get an executable wrapper with this PR should the invocation in the setup.bat file be updated to call that executable instead? That would get rid of the custom logic around the Python interpreter in that file.

update to invoke _setup_util.py instead in setup.bat

I will be testing this with our daily build to make sure it does not break anything else

Thanks - better safe than sorry. Especially since releasing a patch for catkin is rather expensive in terms of resources on our buildfarm :wink:

do we want to change the name mentioned in wrapper.cpp.in?

Since the file is based on their work I would rather keep it as is.

updated setup.bat.in to invoke _SETUP_UTIL_EXE=<EMAIL_ADDRESS>if it exists.
I think it's also safe to skip the %_PYTHON% "%_SETUP_UTIL%" %* step completely, but that seems risky to me without further testing. Given the importance of setup.bat, I think that would be a risk we don't have to take.

better safe than sorry

totally! good news is, this change should not affect any ROS installation on Linux since it's guarded to be Windows-specific.

I think it's also safe to skip the %_PYTHON% "%_SETUP_UTIL%" %* step completely, but that seems risky to me without further testing. Given the importance of setup.bat, I think that would be a risk we don't have to take.

That doesn't sound reasonable to me. If for whatever reason the _setup_util.py.exe file isn't available that means the CMake function proposed in this PR doesn't work as expected. That would be fairly fatal anyway - I don't think having a backup just for the setup util invocation would help much.

Are you confident that the current state covers the necessary use cases on Windows?

Yes, this should cover basic use cases, should be good for the next catkin release. However, I have to be honest and admit that this might not cover all possible scenarios since our bandwidth has not allowed us to test intensively or thoroughly.
If there should be any further (major) improvement/fix/change, I think that could go into a separate release (depends on future development plans). Given our time frame, (imo) it would be good enough if this immediate release of catkin could provide basic (necessary) functionality (enables building and running natively) for (almost) all ROS packages.

Or would you like to have more time for testing?

At this point, we have verified that current state of catkin builds without a problem. Running test suites or real use cases (like turtlesim or turtlebot) could take much longer, so I would prefer to do that after the changes are merged (into ROS and our mainline).
Is there a release plan in vision? As soon as we get catkin changes merged, I will starting cleaning up our ms-iot forks and doing the verification mentioned above. If there is any obvious issue, I will make sure to have the fix out before EOD Friday (3/1)

Sounds good. I am able to do a release tomorrow...

looking into an error found while building desktop_full:
==> Processing catkin package: 'soem'
==> Creating build directory: 'build_isolated\soem'
==> Building with env: 'C:\opt\ros\melodic\x64\env.bat'
==> cmake D:\a\1\a\_ws\src\soem -DCATKIN_DEVEL_PREFIX=D:\a\1\a\_output\devel_isolated\soem -DCMAKE_INSTALL_PREFIX=C:\opt\ros\melodic\x64 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_PREFIX_PATH=C:/opt/rosdeps/x64;C:/opt/ros/melodic/x64 -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_PDB_OUTPUT_DIRECTORY=D:\a\1\a\_symbols -DPYTHON_VERSION=2.7 -DPYTHON_EXECUTABLE=c:\opt\python27amd64\python.exe -DPYTHON_LIBRARIES=c:\opt\python27amd64\libs -DCATKIN_SKIP_TESTING=ON -G NMake Makefiles in 'D:\a\1\a\_output\build_isolated\soem'
-- The C compiler identification is MSVC 19.16.27026.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Using CATKIN_DEVEL_PREFIX: D:/a/1/a/_output/devel_isolated/soem
-- Using CMAKE_PREFIX_PATH: C:/opt/rosdeps/x64;C:/opt/ros/melodic/x64
-- This workspace overlays: C:/opt/ros/melodic/x64
-- Found PythonInterp: C:/opt/python27amd64/python.exe (found suitable version "2.7.15", minimum required is "2.7")
-- Using PYTHON_EXECUTABLE: C:/opt/python27amd64/python.exe
-- Using default Python package layout
-- Found PY_em: C:\opt\python27amd64\lib\site-packages\em.pyc
-- Using empy: C:/opt/python27amd64/lib/site-packages/em.pyc
-- Using CATKIN_SKIP_TESTING: ON (implying CATKIN_ENABLE_TESTING=OFF)
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: D:/a/1/a/_output/build_isolated/soem/test_results
-- Found gtest: gtests will be built
-- Using Python nosetests: C:/opt/python27amd64/Scripts/nosetests-2.7.exe
-- catkin 0.7.14
-- BUILD_SHARED_LIBS is on
OS is win32
LIB_DIR: lib
-- Configuring done
CMake Error: CMake can not determine linker language for target: _setup_util.py_executable_install_python
CMake Error: Cannot determine link language for target "_setup_util.py_executable_install_python".
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    PYTHON_LIBRARIES

-- Build files have been written to: D:/a/1/a/_output/build_isolated/soem
<== Failed to process package 'soem':

Looking at the source of that package I don't see why this is happening. But any package could explicitly not select the C++ compiler by calling e.g. project(<name> LANGUAGE C). And in that case there would be no C++ compiler selected to build the expanded python_win32_wrapper.cpp.in file. Can you please check the content of the CMake variable ENABLED_LANGUAGES in this case (just to figure out why this case fails)?
In general I am not sure if there is a way in CMake to add CXX later if the package explicitly didn't select it. If there this patch could use that to explicitly the C++ compiler on demand when needed to compile the expanded template. If not we might need to reconsider the approach.

You might want to try enable_language().

You might want to try enable_language().

Thanks for sharing your expertise! That's also what I looked into, and I have updated this change accordingly.
To verify the enable_language() change, this build was queued. However, it failed with the following error message:
CMake Error at C:/opt/ros/melodic/x64/share/catkin/cmake/platform/windows.cmake:128 (add_executable):
  add_executable cannot create target "xacro_executable_devel" because
  another target with the same name already exists.  The existing target is
  an executable created in source directory "D:/a/1/a/_ws/src/xacro".  See
  documentation for policy CMP0002 for more details.
Call Stack (most recent call first):
  C:/opt/ros/melodic/x64/share/catkin/cmake/catkin_python_setup.cmake:156 (add_python_executable)
  CMakeLists.txt:15 (catkin_python_setup)

I looked into the source code of xacro and found out that xacro\CMakeLists.txt is calling catkin_python_setup twice. I am not sure if that is intended or not, so have raised https://github.com/ros/xacro/issues/205 and proposed a fix.
Meantime, this issue has been addressed for our ms-iot fork with https://github.com/ms-iot/xacro/commit/0c8f06c3a7734633349cc5fd6abcafb33ca7b389, and locally verified. A new build has been queued. The only 4 packages that were not built in the previous build were:
- xacro
- diff_drive_controller
- gripper_action_controller
- joint_trajectory_controller

xacro has been locally verified, and the other 3 should not have any issue either. pretty sure this will result in a successful build.
what's happening next:

one of our daily builds will exercise --make-args run_tests, that will help me verify if this change would affect any test run (this should not)
after the binaries come out, I will also test a few basic use cases: roscore, rostopic, rosnode, etc. just to verify we cover basic scenarios, this should be finished before EOD 3/1

did some verification =)

this change does not affect running tests. test run didn't have any (new) problem compared to n-1 build. (there seems to be some problem with the number of tests run, I will look into that in a separate change)
tested roscore, rostopic list, rostopic pub, rostopic pub, rosnode list, rosnode ping, all worked
tested roslaunch turtlebot3_bringup turtlebot3_model.launch, this verifies that executable built in packages like (xacro) could be launched. (another pull request for this: https://github.com/ros/ros_comm/pull/1628)

Thank you for this patch and for iterating on it.


starting node takes too long on rpi zero.

This issue is used to describe the work that I'm doing to understand why the pi-zero is so slow to respond.
Looking at a js function of ns-status. This function is very simple for computation perspective (That is no loops or anything alike).
Still in practice it takes between 15 to 20 seconds.
I have stopped all openaps jobs and looked where the time is being spent:
It seems that loading node takes 14 seconds. (found by running time node -e"")
furthermore - running two nodes together takes 28 seconds, running 3 of them takes 43 seconds.
Looking at top, shows that operation is CPU intensive.
So, how many nodes do we run in 20 minutes?
Looking at what was done from 03:00 to 03:20 node was started 50 times, which takes 700 (50 * 14) seconds out of 1200, or 60% of the cpu.
By the way, a rpi 3 runs node in 2.5 seconds, and can probably run 4 of them in parallel on different cores, so it is not affected.
So far for problem high level description.

Possible solutions that I can think of:

Run on something faster. (did not seem feasible from what I have looked for, and probably requires an enormous amount of testing and integration).

Write in another language (again seems like an enormous amount of work).

Decrease the time that it takes to load node. (probably possible, but I did not find how).

Run js using other engine then node.

run node ones and send it the work to be done. This is probably the way node was meant to run.

I'm continuing with #5, but will be happy to hear other people advices about ways to improve.

Something I've meant to experiment with for a couple years now is creating a stateless Node.js demon that listens to a local socket connection, which allows invoking some of the code we've implemented as Javascript using IPC calls from Bash, which would completely eliminate the Node startup time from the time to execute each of these. I'll try to find the time to make a proof of concept for this to see if this would work. Of note, Scott has been pointing out the current code is very much not implemented with the assumption that an instance of the code might be executed multiple times, so if we go this route, we need to be very careful around how to retain the statelessness of the code so multiple invocations of the same instance do not affect the outcomes of the runs.

I have just implemented such a server for the ns-status code.
It finishes its work in less than a second.
Indeed there are global variables there that make the change a little bit harder than it should be.
Is this problem only with determine basal or is it a global one?
I guess that we will still need to decide on things like are we having it single threaded, or do we want to open a thread per request.
How will we do a watchdog for it (restarting it when needed)?
But let's wait for Scott input.

Can you share the code somewhere? I think there's quite a lot of places across the board that need changing if we go with this approach, where the current system just require()s code to create an instance that'll persist data, which need to be checked that they don't use any globals and allow creating new instances for each invoke. Good news is that any of these changes are likely to improve the overall quality of the code. :)

The client code is here:
echo  monitor/clock-zoned.json monitor/iob.json enact/suggested.json enact/enacted.json monitor/battery.json monitor/reservoir.json monitor/status.json monitor/mmtune.json preferences.json  monitor/edison-battery.json  | netcat <IP_ADDRESS> 1337
The server code is attached. Please note that this is only a proof of concept not meant for production code (I only added more global variables, not removed them).
ns-socket-status.txt

(Just out of curiosity, have you tested the startup time of different Node versions? I recall some recent major release implied Node no starts faster, so would be interesting to know the startup speed across Node 8 / 10 / 12)

By the way, the histogram of the different functions that we run is this:
 12 /usr/local/bin/json
  5 /usr/local/bin/ns-status
  2 /usr/local/bin/oref0-calculate-iob
  2 /usr/local/bin/oref0-detect-sensitivity
  2 /usr/local/bin/oref0-determine-basal
  6 /usr/local/bin/oref0-get-ns-entries
  6 /usr/local/bin/oref0-get-profile
  7 /usr/local/bin/oref0-meal
  6 /usr/local/bin/oref0-normalize-temps

so, we can start with the easy ones, and continue from there (for example determine bassal, is called very little, and probably we can do without it).

I have tested with node v10.15.2 (our default).
I'll not be available for the next 24 hours, but I can try to test other versions tomorrow.

(I was actually thinking of using Unix sockets rather than a networking socket, which should be lighter weight than binding to the local IP)

So - @tzachi-dar can you refactor that test so that instead of just changing the one file, do the least changes possible on the ns-status script so it can be invoked in a backward compatible manner, but also through a demon (and have a daemon script that requires that file)? That'd give us a bit better understanding of how complex it'd be to implement the refactor

Coming from #1342, this is exactly the numbers I have found and the problem I stumbled onto. A dry run of node takes 14 seconds, not even thinking about underclocking. This has some unfortunate consequences both on responsiveness and battery time.
The calls to date are the silliest ones I found, but all the oref0-meal and so on are indeed very unnecessarily costly.

Decrease the time that it takes to load node. (probably possible, but I did not find how).

I thought setting NODE_ENV=production would help a lot, but it doesn't. So I did not find any way either.

run node ones and send it the work to be done. This is probably the way node was meant to run.

Looks like a lot of changes, but definitely the way to go imo :+1:

Ok, so I did some more work on this issue.
I have fixed the ns-status file to be ready to be called from a different module.
Changes are minimal. (we obviously need to replace the order of the functions there, but I wanted to make the diff minimal).
Code can be found at: https://github.com/tzachi-dar/oref0/tree/tzachi-node-example
I have also created a demo server using a unix socket to send and receive data. (still at very initial stage).
In order to use it the client can simply run:
echo ns-status aa bb |socat - UNIX-CONNECT:/tmp/unixSocket
So, again changes here are minimal from the client side.
I still need to change the client to read the json files not using require (require will only read the data once).
One more thing to note is that testing of this feature can be very thorough:
We can capture the node calls as they are today, copy the params, and then run against them.
And I would like to emphasize again:
We don't have to run this on all functions.
Some of our functions are very complicated (for example determine-bassel).
But we have many that are simple, and are trivial to be converted.
Converting the trivial ones will allow the pi to work much better as a lot of cpu will be freed.
Any feedback is welcomed.

Sounds like a good approach. I agree we should stick to migrating the things that are easiest to convert and test, and that run the most frequently.

OK, so I'll prepare a pr for
12 /usr/local/bin/json
5 /usr/local/bin/ns-status
and we will see how we continue from here.

Here is a short and optimistic update about the progress:
I have implemented the following modules on the shared node:
oref0-calculate-iob
oref0-meal
oref0-get-profile
oref0-get-ns-entries
Comparing two runs of two nights on the same time, between 02:00 am to 08:59 the old code has completed 66 loops (or a loop every 6.3 minutes).
The new code has completed 81 loops, or a loop every 5.2 minutes.
Reading the raw data, it also seems that with the old code there have been 9 times where a loop was not completed in 10 minutes (biggest time without loop is 17 minutes). In the new code this only happened once (for 10.5 minutes)
Another interesting advantage of the new code is that CPU is not always 100%. This helped to decrease the battery consumption by 30%:
As one can see in the attached pictures, with the new code, 850 mah were needed to power the pi for 6:50 minutes (=> 124 ma).
On the old code, 1376 mah were needed to power the pi for 8:33 hours (=> 161 ma)

Here is the raw data:
grep -e 'Completed oref0-pu|reason' /var/log/openaps/openaps-date.log -i|  grep '11 Mar 2020 0[2-8]' | wc -l
66
root@oapsad5:/home/pi# grep -e 'Completed oref0-pu|reason' /var/log/openaps/openaps-date.log.1 -i|  grep '10 Mar 2020 0[2-8]' | wc -l
81
grep -e 'Completed oref0-pu|reason' /var/log/openaps/openaps-date.log -i|  grep '11 Mar 2020 0[2-8]'
2020-03-11 02:05:17 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:05:17 AM IST
2020-03-11 02:10:39 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:10:39 AM IST
2020-03-11 02:23:34 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:23:34 AM IST
2020-03-11 02:27:25 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:27:25 AM IST
2020-03-11 02:33:31 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:33:31 AM IST
2020-03-11 02:39:49 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:39:49 AM IST
2020-03-11 02:44:00 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:43:59 AM IST
2020-03-11 02:51:33 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:51:33 AM IST
2020-03-11 02:54:35 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:54:35 AM IST
2020-03-11 02:57:44 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 02:57:44 AM IST
2020-03-11 03:04:38 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:04:38 AM IST
2020-03-11 03:11:31 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:11:30 AM IST
2020-03-11 03:16:30 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:16:30 AM IST
2020-03-11 03:23:42 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:23:42 AM IST
2020-03-11 03:34:48 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:34:48 AM IST
2020-03-11 03:45:37 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:45:37 AM IST
2020-03-11 03:49:34 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:49:34 AM IST
2020-03-11 03:53:35 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 03:53:35 AM IST
2020-03-11 04:01:41 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:01:41 AM IST
2020-03-11 04:11:54 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:11:54 AM IST
2020-03-11 04:20:29 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:20:29 AM IST
2020-03-11 04:25:01 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:25:01 AM IST
2020-03-11 04:28:22 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:28:22 AM IST
2020-03-11 04:38:34 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:38:34 AM IST
2020-03-11 04:55:04 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 04:55:04 AM IST
2020-03-11 05:02:56 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:02:56 AM IST
2020-03-11 05:06:31 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:06:30 AM IST
2020-03-11 05:10:39 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:10:39 AM IST
2020-03-11 05:20:00 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:20:00 AM IST
2020-03-11 05:23:33 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:23:33 AM IST
2020-03-11 05:26:57 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:26:57 AM IST
2020-03-11 05:36:26 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:36:26 AM IST
2020-03-11 05:40:47 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:40:47 AM IST
2020-03-11 05:49:59 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:49:59 AM IST
2020-03-11 05:52:50 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:52:50 AM IST
2020-03-11 05:58:52 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 05:58:52 AM IST
2020-03-11 06:02:36 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:02:36 AM IST
2020-03-11 06:06:45 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:06:45 AM IST
2020-03-11 06:21:36 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:21:36 AM IST
2020-03-11 06:25:08 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:25:08 AM IST
2020-03-11 06:29:26 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:29:26 AM IST
2020-03-11 06:32:00 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:32:00 AM IST
2020-03-11 06:39:30 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:39:30 AM IST
2020-03-11 06:42:33 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:42:33 AM IST
2020-03-11 06:50:50 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:50:50 AM IST
2020-03-11 06:56:50 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 06:56:50 AM IST
2020-03-11 07:03:11 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:03:11 AM IST
2020-03-11 07:10:04 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:10:04 AM IST
2020-03-11 07:12:33 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:12:33 AM IST
2020-03-11 07:17:42 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:17:42 AM IST
2020-03-11 07:24:44 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:24:44 AM IST
2020-03-11 07:29:19 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:29:19 AM IST
2020-03-11 07:34:42 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:34:42 AM IST
2020-03-11 07:47:13 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:47:13 AM IST
2020-03-11 07:54:16 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 07:54:16 AM IST
2020-03-11 08:00:50 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:00:50 AM IST
2020-03-11 08:03:45 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:03:45 AM IST
2020-03-11 08:08:54 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:08:54 AM IST
2020-03-11 08:12:31 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:12:31 AM IST
2020-03-11 08:28:09 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:28:09 AM IST
2020-03-11 08:31:27 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:31:27 AM IST
2020-03-11 08:34:57 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:34:57 AM IST
2020-03-11 08:42:50 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:42:50 AM IST
2020-03-11 08:46:45 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:46:45 AM IST
2020-03-11 08:53:37 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:53:37 AM IST
2020-03-11 08:57:52 openaps.pump-loop Completed oref0-pump-loop at Wed 11 Mar 2020 08:57:52 AM IST
root@oapsad5:/home/pi# grep -e 'Completed oref0-pu|reason' /var/log/openaps/openaps-date.log.1 -i|  grep '10 Mar 2020 0[2-8]'
2020-03-10 02:01:04 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:01:04 AM IST
2020-03-10 02:07:30 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:07:30 AM IST
2020-03-10 02:11:28 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:11:28 AM IST
2020-03-10 02:16:34 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:16:34 AM IST
2020-03-10 02:21:55 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:21:55 AM IST
2020-03-10 02:26:26 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:26:26 AM IST
2020-03-10 02:31:32 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:31:32 AM IST
2020-03-10 02:37:18 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:37:18 AM IST
2020-03-10 02:41:14 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:41:14 AM IST
2020-03-10 02:50:03 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:50:03 AM IST
2020-03-10 02:51:04 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:51:04 AM IST
2020-03-10 02:57:06 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 02:57:06 AM IST
2020-03-10 03:01:57 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:01:57 AM IST
2020-03-10 03:10:29 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:10:29 AM IST
2020-03-10 03:12:07 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:12:07 AM IST
2020-03-10 03:17:46 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:17:46 AM IST
2020-03-10 03:22:55 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:22:55 AM IST
2020-03-10 03:27:44 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:27:44 AM IST
2020-03-10 03:32:23 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:32:23 AM IST
2020-03-10 03:36:43 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:36:43 AM IST
2020-03-10 03:43:34 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:43:34 AM IST
2020-03-10 03:48:18 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:48:18 AM IST
2020-03-10 03:51:54 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 03:51:54 AM IST
2020-03-10 04:00:25 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:00:25 AM IST
2020-03-10 04:03:24 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:03:23 AM IST
2020-03-10 04:09:32 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:09:32 AM IST
2020-03-10 04:11:43 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:11:43 AM IST
2020-03-10 04:16:54 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:16:54 AM IST
2020-03-10 04:22:36 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:22:36 AM IST
2020-03-10 04:27:25 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:27:24 AM IST
2020-03-10 04:32:16 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:32:16 AM IST
2020-03-10 04:37:21 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:37:20 AM IST
2020-03-10 04:46:32 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:46:32 AM IST
2020-03-10 04:53:08 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:53:08 AM IST
2020-03-10 04:59:22 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 04:59:22 AM IST
2020-03-10 05:02:58 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:02:58 AM IST
2020-03-10 05:08:39 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:08:39 AM IST
2020-03-10 05:12:34 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:12:34 AM IST
2020-03-10 05:18:19 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:18:19 AM IST
2020-03-10 05:23:41 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:23:41 AM IST
2020-03-10 05:27:27 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:27:26 AM IST
2020-03-10 05:32:49 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:32:49 AM IST
2020-03-10 05:38:42 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:38:42 AM IST
2020-03-10 05:45:44 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:45:44 AM IST
2020-03-10 05:47:58 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:47:58 AM IST
2020-03-10 05:53:32 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:53:32 AM IST
2020-03-10 05:59:24 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 05:59:24 AM IST
2020-03-10 06:02:52 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:02:52 AM IST
2020-03-10 06:07:56 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:07:56 AM IST
2020-03-10 06:12:41 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:12:41 AM IST
2020-03-10 06:17:55 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:17:55 AM IST
2020-03-10 06:22:38 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:22:38 AM IST
2020-03-10 06:28:46 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:28:46 AM IST
2020-03-10 06:35:45 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:35:45 AM IST
2020-03-10 06:38:56 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:38:56 AM IST
2020-03-10 06:42:56 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:42:56 AM IST
2020-03-10 06:48:30 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:48:30 AM IST
2020-03-10 06:56:32 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:56:32 AM IST
2020-03-10 06:57:52 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 06:57:52 AM IST
2020-03-10 07:02:53 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:02:53 AM IST
2020-03-10 07:08:20 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:08:20 AM IST
2020-03-10 07:13:27 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:13:27 AM IST
2020-03-10 07:18:38 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:18:38 AM IST
2020-03-10 07:29:03 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:29:03 AM IST
2020-03-10 07:33:12 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:33:12 AM IST
2020-03-10 07:39:45 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:39:45 AM IST
2020-03-10 07:42:37 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:42:37 AM IST
2020-03-10 07:48:33 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:48:33 AM IST
2020-03-10 07:53:26 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:53:26 AM IST
2020-03-10 07:58:48 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 07:58:47 AM IST
2020-03-10 08:02:20 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:02:20 AM IST
2020-03-10 08:07:14 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:07:14 AM IST
2020-03-10 08:12:46 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:12:46 AM IST
2020-03-10 08:22:50 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:22:50 AM IST
2020-03-10 08:31:18 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:31:18 AM IST
2020-03-10 08:32:56 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:32:56 AM IST
2020-03-10 08:37:48 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:37:48 AM IST
2020-03-10 08:43:31 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:43:31 AM IST
2020-03-10 08:51:13 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:51:13 AM IST
2020-03-10 08:53:48 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:53:48 AM IST
2020-03-10 08:59:35 openaps.pump-loop Completed oref0-pump-loop at Tue 10 Mar 2020 08:59:35 AM IST

Awesome! I believe this would help underclocked 500MHz pis a ton to reliably loop from morning to evening :)

I'm busy with COVID-19 stuff, but happy to merge things once y'all have tested them to your satisfaction.

I hope everything is OK scott.
I think that the first part is to merge https://github.com/openaps/oref0/pull/1361
After that it will be easier to review the next parts.
I'm doing my best to test the next part, but would still wait for a through review by you once you have time.

Yeah, we're doing great, and Washington State finally just today shut down all the schools and is finally doing enough social distancing to probably stop the epidemic from continuing to grow. Still lots to be done to build CoEpi.org and start being able to do automated contact tracing so we can lift the social distancing restrictions in a month or two, but I think we're finally Flattening the Curve enough to avoid becoming another Lombardy.
Commented on #1361: once we're happy enough with the test results I can merge it.

What’s the next step here? 1361 has been merged for months now. Have you done any further testing? Is there something specific that needs reviewed or tested by someone else?

I did all the testing I could do without testing this on a real person.
I'm waiting for someone familiar with the code to review it so I can continue with the testing.

I cannot help with the testing anymore unfortunately

@tzachi-dar which PR needs review?

https://github.com/openaps/oref0/pull/1370

With #1361 and #1370 both merged, is this issue still outstanding? Is there anything that still needs to be tested? Are there any additional modules that need to be converted to use the shared instance of node?

The issue is almost fixed now, but I think that we need to convert also things like determinebasel to get it 100% done.


Status of muted chats is not correctly synchronized

Description:
At least using Whatsapp, it seems that the status of muted chats is not correctly synchronized. In the phone apps you can mute a chat for a period, or forever. It seems nchat does not notice either when it is not running at the time of the chat change.
How to reproduce it:
with nchat closed, unmute a chat in whatsapp, then open nchat. The chat will still be muted.
Environment:

Version: master
OS / distro: Archlinux

Hi @gilcu3 - I did a quick test here and "manually" muting or unmuting a chat in WhatsApp (while nchat is not running) reflects correctly for me once nchat is started.
However, Mute for 8 hours / 1 week with automatic unmute might not work correctly. I'll need to check that.

Hi @d99kris you are partly right, it does work manually muting or unmuting, except in this case:
I am using muted_position_by_timestamp=0, therefore muted chats appear at the end.
At the beginning nchat is closed, the chat is muted. In another client the chat is unmuted. Then nchat is opened, the unmuted chat should go to the front (it has recent messages), but it remains in the back. Surprisingly if I open it, somehow nchat updates its state and the chat goes to the correct position.
The other case where muting is only by a period I think also doesnt work, but maybe it is just the same bug described before.

Hi @gilcu3 - ok thanks, that is useful information 👍

Hi again - finally got a chance to look into this. I believe the issue should be fixed with above commit, but please let me know in case it's not.

FYI - yesterday I found one more muted-related issue w.r.t. cached muted state, a fix was committed in 846bac8.


Provide "Get Initiative" as an attribute

Cards should return a name/value pair that indicates if they have the "Get Initiative" text.

is_quick is available in the Card Set API.

I am aware. This issue was created at the request of the developer for tracking, but was never closed. See
https://github.com/ValveSoftware/ArtifactDeckCode/issues/4#issuecomment-439410555


request timeout optional message inclusion

Like Fault having the original message in the RequestTimeoutExpired message would/could be a valuable addition.

Merged, thanks!


README word error

Description:  // sql-cipher database encyrption (optional)
encyrption should be encryption
thanks

in develop.


Should we put some standard stripper utilities here?

One thing I often do is run a "stripper" script that compacts all my kerboscript code into a form where all the comments and indenting space is removed.
It occurs to me that maybe either putting utility scripts here (they'd have to be OS-dependant to some extent) that do that for others may be nice, or even more ambitious still, we could make a tool that automatically does this with all submitted code and provides a "stripped" folder with the stripped versions.  Note, the original would be preserved as well.  This would be a second copy of the script.
It would be nice if people didn't feel constrained to have to write their scripts compactly just to reduce footprint, and instead could make them well indented and commented, especially as putting scripts here is meant to be a useful teaching tool as well as a library.
(while I'm at it - header comment detection and stripping to auto-gen MD document files could be pretty good too - but that should be a separate issue)

If the purpose is to have scripts that use less memory in-game, wouldn't it make more sense to just use .ksm files? I've not used them much, and am aware that they're apparently bigger in 0.17, but wouldn't it make sense to put any effort into making 'compiled' scripts smaller?

@Dunbaratu It would also be nice to have a "merge scripts I choose into one big file" script. This will reduce the number of copy commands to upload a program onto a vessel. It also could automatically search for the dependencies (by searching an argument of run command in the library folder). Plus compiled version of this big file will take a slightly less space because some string constants will be reused.
@austinjames314 Yes the information about comments and indents isn't stored in .ksm files.

KSM is no longer a large reduction in space unless you use a lot of comments.  But it might be in the future.  I still have some optimizations I could do to the code here and there.
What I am quite interested in, however, is the idea of doing a small program that runs through and strips out doc-comments from the scripts, so you don't need to maintain BOTH a comment in the script describing the arguments, return ,and description of a function, AND repeat that info in the .md file.  It would be nice if we could do a thing similar to javadoc, or doxygen, or C#'s xml comments.

There's a part of me that wants to see if it's possible to fiddle with the compiler in kOS.Safe to make a separate command-line program that will parse the script and make the docs.. but that may be overkill when all it needs to do is read the comment tags and find the name at the start of the next function.

Also: a small script that replaces all white space characters with spaces can considerably reduce the size of .ksm file produced by the compiler by reducing %D section to almost nothing.

Problem: this small script will have to remove comments too so the task is a bit less easy.

@Dunbaratu Does your implementation turn whole file into one long line?

No.  that would be horrible because then any reported errors would be impossible to find - the line number would be wrong.  line breaks are the only whitespace I keep, because I want the line number counts to still be right.

@Dunbaratu I think there should be an optional "extreme compression". Removing line breaks doesn't make working with .ks easier but it reduces the size of .ksm.

@Dunbaratu I think the stripper in the library might improve the code style by it making comments and indents cost nothing.

That's what I was thinking.  I see some people using one-space indents and I think the only reason they're doing that is code size.
I have really never liked this idea of making people pay for comment space.  Yes it's true that the old 60's computers only had a few K, but that space was taken up with low level machine code, not source code with indents and comments, in an object oriented system.

@Dunbaratu

spit out the numerical bytes for the ML, and then WOVE those bit patterns into the rope memory by people following instructions on cards in front of them. (put wire A into slot 53, put wire B into slot 71, etc.)

.ksm sort of does what you said. Am I missing something important?
2. To be explicit: I vote for including stripper into the repo because an argument: "we have a stripper in our lib so add indent or I'll go mad" is much more powerful argument than "there is a stripper hidden somewhere on the internet so you'd better add the indent"


Not working for me on PHPstorm

Hi,
I've been trying to use it without success, even trying to use a fresh phpstorm version, as soon as I click settings I got the following errors:
Tabnine is not able to function properly. Contact<EMAIL_ADDRESS>
com.tabnine.binary.exceptions.BinaryCannotRecoverException
	at com.tabnine.binary.BinaryProcessRequesterProvider.onDead(BinaryProcessRequesterProvider.java:69)
	at com.tabnine.binary.BinaryRequestFacade.executeRequest(BinaryRequestFacade.java:44)
	at com.tabnine.binary.BinaryRequestFacade.executeRequest(BinaryRequestFacade.java:22)
	at com.tabnine.lifecycle.BinaryStateService.updateState(BinaryStateService.java:32)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at com.intellij.util.concurrency.SchedulingWrapper$MyScheduledFutureTask.run(SchedulingWrapper.java:226)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
	at java.base/java.lang.Thread.run(Thread.java:829)

(related to Zendesk ticket #13620)

Can you check the path /Users/{username}/.tabnine/ and tell me the latest version you have there?
Also, would try to uninstall Tabnine?
Try removing the files at:
/Users/{UserName}/Library/Preferences/TabNine
/Users/{UserName}/.tabnine
/Users/{UserName}//Library/Application Support/TabNine
And finally, reinstall Tabnine on PHPStorm.
Would you let me know if that works?

Hi if you still need our help, please ping us here or email<EMAIL_ADDRESS>

Hi if you still need our help, please ping us here or email<EMAIL_ADDRESS>


virt-launcher: Improve libvirtd debug log filters

What this PR does / why we need it:
The current filters already do a decent enough job at filtering out unnecessary noise from the libvirtd debug log, but they can be improved further: the util.object module, in particular, generates a log entry for each lifecycle event     (new, ref, unref, dispose) of each internal object, which is completely uninteresting to the vast majority of users and not particularly useful even when debugging all but the most peculiar libvirt issues.
To give an idea of the impact of this change, I ran a simple experiment: I started a VM, waited 10 seconds, collected the logs and then stopped it, first without this patch and then with it.  The logs went from 9013 lines to just 5513     lines; of the 3500 lines that were filtered out by the change, 3403 can be attributed to util.object.
The wiki page where the original filter settings were taken from has since been superseded by a "knowledge base" article on the libvirt website, so the reference has been updated as well.
Which issue(s) this PR fixes:
None
Special notes for your reviewer:
None
Release note:

NONE

/lgtm
/approve

@andreabolognani we have a debug log test. I think you new settings are removing something from the log which we were checking for.

/lgtm cancel

@rmohr right, that'd be [rfe_id:273][crit:high][vendor:cnv-qe@redhat.com][level:component]VMIlifecycle [rfe_id:273][crit:high][vendor:cnv-qe@redhat.com][level:component]Creating a VirtualMachineInstance [test_id:3197]should log libvirtd debug logs when enabled. I'll look into it.

@rmohr updated so that the test only looks for the useful QEMU_MONITOR_SEND_MSG messages and not for the pointless OBJECT_REF messages, since removing the latter and reducing the noise is the entire point of this PR :)

/lgtm
/retest

/test all


Measure end to end latency

https://github.com/AmbientRun/Ambient/pull/1022 adds debug-local-datagram-latency feature for debugging the latency of sent unreliable messages, it relies on clocks being in sync so it works the best on localhost or via proxy (with both client and server running on the same machine).

Reopening as I'm going to explore adding true end to end latency (from user input to rendered frame).


feat: check commits after the last ci run

Fixes: https://github.com/joyeecheung/node-core-utils/issues/66

Codecov Report

Merging #69 into master will increase coverage by 0.05%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #69      +/-   ##
==========================================
+ Coverage   95.49%   95.54%   +0.05%
==========================================
  Files          13       13
  Lines         444      449       +5
==========================================
+ Hits          424      429       +5
  Misses         20       20

Impacted Files
Coverage Δ

lib/pr_checker.js
95.68% <100%> (+0.19%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b97b984...258f885. Read the comment docs.

@apapirovski thanks for your suggestion , I will fix it later.

@buji1993 Hi, this PR needs a rebase. Also since https://github.com/joyeecheung/node-core-utils/pull/65 has landed, can you add a bullet to each commit so it looks something like:
Commits pushed after the last Full CI run:
- doc: doc changes
- src: some changes

? After that I think we'll be good to merge. Thanks!

@joyeecheung done


Adding dsETH

Summary
Adding dsETH
Supersedes https://github.com/cowprotocol/token-lists/pull/22
Address: 0x341c05c0e9b33c0e38d64de76516b2ce970bb3be
Image:

Verified by Chen over slack


Missing dependency

Bicep version
Bicep CLI version 0.7.4 (5afc312467)
Describe the bug
The scenario is the following. You have a parent resource that you are creating with your bicep template. When the parent resource is created it automatically creates another child resource of that resource without having to define it. For that child resource you need to create child resource (third level) so you define the first child as existing resource so you can put it as parent to the second child resource. When you do that dependsOn is not populated in the second child resource for the parent resource.
To Reproduce
example real world scenario:
resource devTestLabRes 'Microsoft.DevTestLab/labs@2018-09-15' = {
  name: devTestLab.name
  #disable-next-line no-loc-expr-outside-params
  location: resourceGroup().location
  tags: tags
  properties: {
    labStorageType: devTestLab.storageType
    premiumDataDisks: devTestLab.enablePremiumDataDisks ? 'Enabled' : 'Disabled'
    environmentPermission: devTestLab.resourceGroupUserRights
    support: {
      enabled: empty(devTestLab.supportMessage) ? 'Disabled' : 'Enabled'
      markdown: devTestLab.supportMessage
    }
    announcement: {
      title: devTestLab.announcement.title
      markdown: devTestLab.announcement.text
      enabled: empty(devTestLab.announcement.title) ? 'Disabled' : 'Enabled'
      expirationDate: empty(devTestLab.announcement.title) ? null : '${devTestLab.announcement.expirationDate}T${devTestLab.announcement.expirationTime}:00+00:00'
    }
    #disable-next-line BCP037
    vmCreationResourceGroupId: resourceGroup().id
    extendedProperties: {
      RdgTokenSecretName: devTestLab.remoteDesktopConnection.gatewayTokenSecret
      RdpConnectionType: connectionTypeMapping[devTestLab.remoteDesktopConnection.connectionSpeed]
      RdpGateway: empty(devTestLab.remoteDesktopConnection.gatewayHostName) ? '' : devTestLab.remoteDesktopConnection.gatewayHostName
    }
  }
}

resource policySets 'Microsoft.DevTestLab/labs/policysets@2018-09-15' existing = {
  name: 'default'
  parent: devTestLabRes
}

resource allowedVmSizesPolicy 'Microsoft.DevTestLab/labs/policysets/policies@2018-09-15' = {
  name: 'allowedvmsizesinlab'
  parent: policySets
  #disable-next-line no-loc-expr-outside-params
  location: resourceGroup().location
  tags: tags
  properties: {
    description: ''
    evaluatorType: 'AllowedValuesPolicy'
    factData: ''
    factName: 'LabVmSize'
    status: empty(devTestLab.allowedVirtualMachineSizes) ? 'Disabled' : 'Enabled'
    threshold: empty(devTestLab.allowedVirtualMachineSizes) ? '[]' : string(devTestLab.allowedVirtualMachineSizes)
  }
}

form the example resourceMicrosoft.DevTestLab/labs/policysets does not have dependency on devTestLabRes. Of course I can skip defining policySets as existing but than I need to add the name 'default' to the name of allowedVmSizesPolicy. This of course breaks the desired approach of using existing and parent resources.
Additional context
IMPORTANT: Bicep will report that adding dependsOn for the devTestLabRes resource is not needed but at the end will not generate it. So you get a warning that explicit dependency is not needed but at the end it is not generated when converted to ARM.

Right -- it is a dup


Update minitar version

to address https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-10173

right now we're allowing ~> 0.5, which will allow the newer 0.6.1 version of minitar without the CVE to install in most cases (unless someone else pins it lower)
for now, because development is active and lxd-common updates need to work within the chefdk environment, until the chefdk updates its minitar dependency (which will update with berkshelf 7), we need to 'allow' < 0.6

some additional digging revealed that the core chefdk 'binary', chef, uses minitar as well, so there may be a bit more delay before it comes through

chef/chef-dk#1494
just waiting on release


[SPARK-12860][SQL] speed up safe projection for primitive types

The idea is simple, use SpecificMutableRow instead of GenericMutableRow as result row for safe projection.
A simple benchmark shows about 1.5x speed up for primitive types, code: https://gist.github.com/cloud-fan/fa77713ccebf0823b2ab#file-safeprojectionbenchmark-scala

cc @davies

Test build #49542 has started for PR 10790 at commit 6b22c91.

Test build #49542 has finished for PR 10790 at commit 6b22c91.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class BufferedRowIterator
trait CodegenSupport extends SparkPlan
case class InputAdapter(child: SparkPlan) extends LeafNode with CodegenSupport
case class WholeStageCodegen(plan: CodegenSupport, children: Seq[SparkPlan])
     class GeneratedIterator extends org.apache.spark.sql.execution.BufferedRowIterator
case class Project(projectList: Seq[NamedExpression], child: SparkPlan)
case class Filter(condition: Expression, child: SparkPlan) extends UnaryNode with CodegenSupport

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49542/
Test PASSed.

Test build #49551 has started for PR 10790 at commit 560b46b.

Test build #49551 has finished for PR 10790 at commit 560b46b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/49551/
Test PASSed.

merging this into master, thanks!


Handle parallel rate-limited generation requests in pyhooksRetry pausing logic

We have some logic in pyhooks to log pauses when an agent's requests to Vivaria come back with a 429 or a similar retryable status code. However, this logic assumes that multiple parallel generation requests won't be made at the same time by the same agent branch.
Perhaps it's enough to change pyhooks to log pauses across an entire agent process, instead of creating one RetryPauser per request?
Or maybe we need logic that can handle pyhooks being called from multiple separate processes in the same agent?

We're talking about trpc_server_request(...), right?

Is the non-thread-safe thing here the session: aiohttp.ClientSession?
Do I understand our goals correctly:
2.1. If vivaria is down, the agent shouldn't crash. Ideally, only the parts of the agent waiting for vivaria's response will block and all the other parts would keep going (but it's not too bad if the agent will totally freeze if vivaria is down) (?)
2.2. If the LLM API is down in a way we decided is retryable, again we want that part (thread) of the agent to wait for the LLM API to be up again (?)


Submission_2352016_2352226_2352399_2353276_2352941_Group_66

1. Group Member.

Phạm Nam An - 2352016
Đỗ Thành Đạt - 2352226
Nguyễn Thanh Huy - 2352399
Trần Anh Tuấn - 2353276
Phan Minh Phúc - 2352941

2. Best Fit Decreasing with Products Priority (Policy 1).
The Best-Fit Decreasing with Product Prioritization algorithm optimizes product placement in available stocks by prioritizing larger products first. This approach reduces fragmentation and maximizes space utilization. Products are sorted in descending order of their area, and each product is iteratively checked for placement across all available stocks. For each stock, the algorithm calculates the remaining space (waste) after placing the product and updates the position that results in the least waste while satisfying size and orientation constraints. Once the optimal position is found, the product is placed. The process continues until all products are placed or no valid placements remain. This method ensures efficient use of stock space and minimizes leftover waste.
3. Best Fit Decreasing with Stocks Priority (Policy 2).
The Best-Fit Decreasing with Stock Priority algorithm focuses on efficiently placing products into available stocks while prioritizing both product size and stock space usage. The algorithm begins by sorting stocks in ascending order of available space and products in descending order of area to prioritize larger products first. For each stock, it iterates through the sorted products and evaluates two orientations (normal and rotated) to find the position that minimizes leftover space (waste). If a valid position is found where the product fits and waste is minimized, the product is placed, and the stock is updated accordingly. This process repeats until all products are placed or no valid placements remain. By prioritizing larger products and iteratively selecting the best-fit stock, this method ensures optimal space utilization and minimizes fragmentation.
4. Result.
We use the following metrics to evaluate the performance of the algorithm: filled ratio, trim loss, time execution
Best Fit Decreasing with Products Prority Results:

Seed
Filled ratio
Trim loss
Time execution

42
0.27
0.30
12.79

25
0.17
0.32
12.47

19
0.28
0.26
32.79

Best Fit Decreasing with Stocks Prority Results:

Seed
Filled ratio
Trim loss
Time execution

42
0.27
0.30
11.00

25
0.17
0.31
11.47

19
0.28
0.25
42.89

Please revise the PR in Dec 16. Otherwise, your group will get 0 score for code.


How to add parametters to service for export PDF file

I used wkhtmltopdf to export HTML to PDF file
Howerver, I want to export filtered data, not all data, so I have to pass the filter values to the method GetData.
How can I do that?

How about getParams parameter in tool button ?
["getParams" parameter in Common.ReportHelper.createToolButton] will pass to [ Execute method Module/Common/ReportController.cs] as JSON value to JsonConvert.DeserializeObject.
It will map to property of report class that imprement IReport (i.e OrderDetailReport.cs), and use it in GetData method.


Fix peer dep build

This updates the changes to package.json so that build works.

wrong branch, closing

@joshuawilson Your image is available in the registry. Run docker pull registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2670 && docker run -it -p 5000:8080 registry.devshift.net/fabric8-ui/fabric8-planner:SNAPSHOT-PR-2670 and visit http://localhost:5000 to access it.


[Bug] Could not resolve "../../../core/node_modules/@milkdown/exception/src"

 > ./node_modules/@milkdown/utils/lib/prose/atom-helper.js:1:38: error: Could not resolve "../../../core/node_modules/@milkdown/exception/src"
    1 │ import { getAtomFromSchemaFail } from '../../../core/node_modules/@milkdown/exception/src';
      ╵                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Happens on the latest version (4.7.0):
"dependencies": {
  "@milkdown/core": "^4.7.0",
  "@milkdown/preset-commonmark": "^4.7.0",
  "esbuild": "^0.12.16"
}

Repro (with esbuild):

Install core and preset-commonmark
Import both from a file:

import * as _1 from "@milkdown/core";
import * as _2 from "@milkdown/preset-commonmark";

console.log(_1, _2);

Bundle the file with npm run esbuild -- index.js --bundle

Fixed in @4.7.1


Music - youtube player cuts out on long songs/sometimes fails to play items in playlist and needs to be restarted

Not sure if this has to do with the youtube api flakiness or not, but on long-ish songs the bot sometimes cuts off in the middle and immediately starts the next song. No errors in the log, however it does seem suspicious that all of the songBuffer.ContentLength are very close to each other (about 3,140,000, poopybuffer initialized at 4mb?). I'll try building the bot with a larger initialized poopybuffer i guess and see if it fixes the issue. I'm seeing the issue with 5 min long songs.
The other issue is that twice now on long playlists the bot failed to play next song and stopped working, and had to be restarted. http://puu.sh/p9NLz/942190f8aa.png
This one is more likely to be related to youtube api flakiness, however the second error seems like a bug in the bot.

It is a known issue.
About cutting off - what usually fixes it for people is changing discord server location to amsterdam/singpore. We won't track music issues since they aren't going to be fixed until we move to discord.net 1.0 and rewrite music completely.
About songs not being queued (not found in the dictionary), it pretty much means it can't find that song for one of many reasons. Either somehow blocked from streaming, or its blocked in your country.


NPCs starting conversations in dialogue with PC

There seems to be an issue where NPCs who are already engaged in dialogue with the player will frequently start other scripted or random conversations with nearby NPCs despite being already engaged in dialogue. This will then cause only the voice lines of the NPC the player is not currently speaking with to play, so you're only hearing one side of the conversation. I've also had this cause two separate conversations begin playing at once, when the original NPC starts a new conversation with the second or another NPC after you exit dialogue with them.
The worst I've seen it yet is when visiting the PC's parents after taking the Kids Stuff trait. I've had both parents do this if you try talking to them early on in the visit, before they might naturally trigger some of their potential conversations.

Game Version
1.7.23
New Game
Yes
Reproduction Steps
It seems all really need to do is spend time around NPCs that you can open a dialogue window with, who also have scripted dialogue with each other or the potential to have random dialogue interactions. Aside from the character's parents, I've had this trigger in the Lodge between multiple members of Constellation.
In one such instance, Sophia began praising Noel while I was talking to Noel, but I only heard Sophia's side of the conversation, though I believe Noel is supposed to be bashful if I remember the conversation correctly when I heard it trigger normally on my first character. Likewise, after I asked Noel to give me the tour of the Lodge, I did not get one because as soon as I exited the dialogue window with her, she walked over and asked Walter about the state of his marriage. This time I did at least hear the entire conversation from both sides, but the tour did not happen and could not be restarted. I still had ownership of the bed in my assigned quarters however and did not find any other problems with the tower being skipped.

I don’t believe this is an issue with the game anymore. Bethesda has specifically addressed the issue with parent dialogue in a past update as well. Closing as dated.


[SPARK-27049][SQL] Create util class to support handling partition values in file source V2

What changes were proposed in this pull request?
While I am migrating other data sources, I find that we should abstract the logic that:

converting safe InternalRows into UnsafeRows
appending partition values to the end of the result row if existed

This PR proposes to support handling partition values in file source v2 abstraction by adding a util class PartitionReaderWithPartitionValues.
How was this patch tested?
Existing unit tests

close https://github.com/apache/spark/pull/23961 as discussed in https://github.com/apache/spark/pull/23961#pullrequestreview-210526524

Test build #103086 has started for PR 23987 at commit b4c945b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/8549/
Test PASSed.

Test build #103091 has started for PR 23987 at commit 8cfb656.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/8555/
Test PASSed.

Test build #103086 has finished for PR 23987 at commit b4c945b.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class PartitionReaderWithPartitionValues(

Test build #103091 has finished for PR 23987 at commit 8cfb656.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103091/
Test PASSed.

thanks, merging to master!


Extract methods from I18n::JS to new Exporter module, and clarify public interface

Currently the I18n::JS library has a lot of code in the base file lib/i18n/js.rb. This is an anti-pattern and makes the code difficult to work with.
As a first step in refactoring, I've moved much of the internals into a new class I18n::JS::Exporter, and I've exposed only the key methods in I18n::JS. In subsequent PRs I hope to break Exporter into even smaller units.
The big chunks of code from lib/i18n/js.rb and spec/i18n-js.rb have been moved to respective exporter lib/spec files.

Thanks for submitting!
I wanted to do it in the next release anyway :D
But I am not sure if the config should be placed in the Exporter or not
I can see 2 patterns:

Place all config in Configuration class, let people I18n::JS.configure do |config|
Place all config in different related modules, like config_file_path

I can see many gems using pattern 1, but I don't remember seeing pattern 2 elsewhere (except in this PR)
Any thoughts?

Maybe wait for #318 & #20 to be merged first
And then remove the changes for config, it can be a separate thing to handle

We can do both a Configuration class AND this Exporter change. The two are not mutually exclusive.
The configuration should be able to be set via Ruby I18n::JS.configure do |config| AND also loaded from config/i18n-js.yml; as long as there's only one set of config internal variables in-memory we can be flexible about where it is loaded from.

To clarify: I agree that Config does NOT belong in Exporter. I only put the config in the Exporter class to minimize the amount of code change.

Since I am not crystal clear about how to change the config related thing now, making this PR to only extract exporting related methods would make it easier to be merged.
We should discuss config related thing in #298

@PikachuEXE I think it would be easiest to refactor both the config and the exporter in the same PR, or else do the config first.

Then let's wait for me to come up a PR for config extraction first.
Still that needs to wait for #318 & #320 to be merged first.

@PikachuEXE I think we can merge this in advance of the config extraction. The two PRs do separate things--this one concerns where the exporting logic is located (regardless of config). I will rebase this tonight so that it can be merged.

Actual, no rush on this. Lets do config PR first.


Build script fixes

Some more fixes to the build script.

Final fix for platform parsing: Works not only for "14.1" or "14.2" but also for "14.22" now
Capture previous environment and restore after each platform build, to prevent false positive builds when actually vsvars failed. This also solves "Command too long" errors when running the script multiple times from same PS instance (vsvars keeps appending to vars and after few runs it is too much).

Happy merging and happy holidays.

Thank you, you too! :)


python3Packages.mypy-boto3-s3: 1.19.12 -> 1.20.0

Motivation for this change
Update to latest upstream release 1.20.0
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 145136 run on x86_64-linux 1

4 packages built:

python38Packages.dependency-injector
python38Packages.mypy-boto3-s3
python39Packages.dependency-injector
python39Packages.mypy-boto3-s3


large gap in terminal between tab and text

I can't seem to remove the large gap between the tab and terminal text which i was able to do in 1.86 and previous versions

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.87.2
OS Version: Manjaro KDE 23.1.3

Steps to Reproduce:

Simply open the terminal in vscode

I believe this is already fixed, can you see if you can reproduce in the Insiders build?
/label info-needed


DecalProjection

DecalSet now accepts a world space viewProj matrix which allows to reuse the same matrix on multiple objects. Also it allows to pass a perspective projection matrix.
DecalProjection to setup projections in editor. Acts similar to PrefabReference - you can setup projection and then inline it if necessary. Also it applies projection on all drawables in the frustum.

Why there are 3-4 DecalSets for each object even tho there're only 2 decals projected?

Why there are 3-4 DecalSets for each object even tho there're only 2 decals projected?
FYI: this was fixed a long time ago.

If I delete decalprojection, the decalset will stay. This gets weird if I undo/redo stuff multiple times.
Overall, I have concerns about stability of this feature in Editor.
When I have time, I will think what we can do about it.


modelling for models

An example of the issue.
A computational model of porcine heart is not the same thing as a computational model of heart that with parameters derived from experiments on pig.

An example dataset N:dataset:90c6874a-b032-4711-8c1d-b5579c84f28a.

Need the ability to distinguish between publishes e.g. osparc vs pmr.


Stuck with google place autocomplete and multiple markers

Always stuck map with google place autocomplete and multiple markers, have tried a lot of solutions but still have issues.
It works fine for the first time, but once i search any location from google place autocomplete and come back to the map and recreate the new markers it does not move to the new location and shows the half map on the screen.

Hi @princegoyalcorevalue
This issue doesn't seem to describe a bug or a feature request. Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow and if you need help with your code, please see https://www.reddit.com/r/flutterhelp/
If "google place autocomplete" is a third-party package, please file an issue in its dedicated repository.
Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


bug-fix for python3

in python3 .values() returns dict_values object not a list, need to explicitly cast to list

Second commit fixes the README and also adds SYNO.SurveillanceStation.ExternalEvent API support


[Question] GetAnalyticsInstanceId() stubbed out on Desktop platforms

[REQUIRED] Please fill in the following fields:

Pre-built SDK from the website or open-source from this repo: open-source
Firebase C++ SDK version: 10.5.0
Main Firebase Components in concern: Analytics
Other Firebase Components in use: Auth
Platform you are using the C++ SDK on: Mac, Windows
Platform you are targeting: iOS, Android, and Desktop (Windows and Mac)

[REQUIRED] Please describe the question here:
I'm looking into implementing GA4 Analytics in a cross platform C++ application that runs on Mobile, Web and Desktop (Windows and Mac).
Everything's fine on Mobile and Web, but on Desktop there doesn't appear to be any way of using GA4 from a C++ application as far as I can tell.
Without Desktop support for Analytics directly in the this library (I note FR #234), it seems as though the GA4 Measurement Protocol REST API could be used instead. But message posts to this API require an application_instance_id, and from the docs...
https://developers.google.com/analytics/devguides/collection/protocol/ga4/reference?client_type=firebase#payload_post_body
"app_instance_id string
Required. Uniquely identifies a specific installation of a Firebase app. This value needs to be retrieved through the Firebase SDK."
...but the suggested C++ call to GetAnalyticsInstanceId() only has valid implementations on Mobile and is stubbed out on Desktop platforms, so how can REST API calls be issued without this being available?
Is there another way for GA4 be used from a C++ desktop application without this support in the Firebase C++ SDK?

Hi @Will-Cowling-Magnopus,
Yes, unfortunately you've discovered the hard way Firebase C++ SDK does not support Google Analytics on Desktop platforms. The list of supported platforms per product can be found on our Firebase C++ SDK Learn More page.
Google Analytics itself isn't supported on desktop platforms via any SDK, I believe. Sorry!

Also, I recommend upvoting the linked ticket to get more votes for it to be put on a roadmap. Thanks!


Runtime or compile time?

First of all, great works! This library looks like a very nice complement to the tink libraries.
But looking at the readme I see the meta are @async and @await, so are they runtime transformation or not?  Because I think that compile time meta has a convention of starting with @:, e.g @:async)

There's no runtime transformation. They get stripped by the macros so there's no real need for the colon. To be exact the 'async' meta I forgot to strip in this version, but that'll get fixed next. I have thought about supporting both forms, because I've already written @:await several times by accident which is hard to debug until you notice the typo.

I think @async looks like a runtime thing, due to convention. Maybe just support @:async while produce a warning (Context.warning()) when a @async is seen.

Both forms are supported now. I kind of like writing @await instead of @:await :)

Thanks! I prefer @:await though, because it is more consistent with the language =)


Getting Error reading from alluxio://:19998/<path_to_parquet_file> at position xxxxxxx in presto SQL

Alluxio Version:
2.7.1
Describe the bug
Alluxio version: 2.7.1
Presto version : 0.268
Presto JDBC version: 0.268
Presto coordinator: 1, workers: 4
Single Alluxio master
Data Source: data loaded from Azure ABFS to Alluxio via distributedLoad
UFS: Azure ABFS

data updated periodically
metadata sync enabled via alluxio.user.file.metadata.sync.interval=300s but UFS is not updated for one week but still getting the error.

File format: parquet
Tools used for query in presto: DBeaver, presto CLI
All the data files are cached into alluxio and presto is reading from alluxio through hive metadata.
When I am trying run the query with DBeaver, it's giving the following error :
com.facebook.presto.spi.PrestoException: Error reading from alluxio://:19998/<path_to_parquet_file> at position xxxxxxx
To Reproduce
This error is random. It gives an error for some of the files and if we remove it from alluxio using --alluxioOnly option and then if I re-run the query, it does not give any error.
Expected behavior
I expect not to produce this kind of error and it gives the desired results.
Urgency
This is a critical error as I am getting this error almost on a daily basis when it restarts alluxio automatically.
Are you planning to fix it
Not sure how to do it.
Additional context
Important Notes:
The data size for some of the parquet files is large.
This error is a random error when it's trying to read all parquet files stored under Alluxio. It's failing on some of the files randomly on a daily basis

Hi @harsh9898 thanks for reporting the issue. The alluxio URI seems weird to me alluxio://:19998/<path_to_parquet_file>.
Do you mind pasting more related logs in presto or alluxio? Thanks

Hi @harsh9898 thanks for reporting the issue. The alluxio URI seems weird to me alluxio://:19998/<path_to_parquet_file>. Do you mind pasting more related logs in presto or alluxio? Thanks

Sure, here is the correct URI: sorry for the confusion

Alluxio URI ; alluxio://<ip_of_host>:19998/<path_to_parquet_file>

NOTE: Once I removed that file from alluxio storage using --alluxioOnly , error goes away.
I am getting that same error. I got that error again and paste the full error trace here:
com.facebook.presto.spi.PrestoException: Error reading from alluxio://<host_name>:19998/<path_to_parquet_file> at position
at com.facebook.presto.hive.parquet.HdfsParquetDataSource.readInternal(HdfsParquetDataSource.java:66)
at com.facebook.presto.parquet.AbstractParquetDataSource.readFully(AbstractParquetDataSource.java:60)
at com.facebook.presto.parquet.AbstractParquetDataSource.readFully(AbstractParquetDataSource.java:51)
at com.facebook.presto.parquet.reader.ParquetReader.readPrimitive(ParquetReader.java:247)
at com.facebook.presto.parquet.reader.ParquetReader.readColumnChunk(ParquetReader.java:330)
at com.facebook.presto.parquet.reader.ParquetReader.readBlock(ParquetReader.java:313)
at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:193)
at com.facebook.presto.hive.parquet.ParquetPageSource$ParquetBlockLoader.load(ParquetPageSource.java:171)
at com.facebook.presto.common.block.LazyBlock.assureLoaded(LazyBlock.java:298)
at com.facebook.presto.common.block.LazyBlock.getLoadedBlock(LazyBlock.java:289)
at com.facebook.presto.operator.ScanFilterAndProjectOperator$RecordingLazyBlockLoader.load(ScanFilterAndProjectOperator.java:320)
at com.facebook.presto.operator.ScanFilterAndProjectOperator$RecordingLazyBlockLoader.load(ScanFilterAndProjectOperator.java:306)
at com.facebook.presto.common.block.LazyBlock.assureLoaded(LazyBlock.java:298)
at com.facebook.presto.common.block.LazyBlock.getLoadedBlock(LazyBlock.java:289)
at com.facebook.presto.operator.project.InputPageProjection.project(InputPageProjection.java:69)
at com.facebook.presto.operator.project.PageProjectionWithOutputs.project(PageProjectionWithOutputs.java:56)
at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.processBatch(PageProcessor.java:327)
at com.facebook.presto.operator.project.PageProcessor$ProjectSelectedPositions.process(PageProcessor.java:201)
at com.facebook.presto.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:315)
at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:79)
at com.facebook.presto.operator.WorkProcessorUtils$YieldingIterator.computeNext(WorkProcessorUtils.java:65)
at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
at com.facebook.presto.operator.project.MergingPageOutput.getOutput(MergingPageOutput.java:128)
at com.facebook.presto.operator.ScanFilterAndProjectOperator.processPageSource(ScanFilterAndProjectOperator.java:301)
at com.facebook.presto.operator.ScanFilterAndProjectOperator.getOutput(ScanFilterAndProjectOperator.java:245)
at com.facebook.presto.operator.Driver.processInternal(Driver.java:424)
at com.facebook.presto.operator.Driver.lambda$processFor$9(Driver.java:307)
at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:728)
at com.facebook.presto.operator.Driver.processFor(Driver.java:300)
at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1079)
at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:599)
at com.facebook.presto.$gen.Presto_0_268_03318e7____20230201_140638_1.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalStateException: No data is read before EOF
at alluxio.shaded.client.com.google.common.base.Preconditions.checkState(Preconditions.java:508)
at alluxio.client.file.AlluxioFileInStream.positionedReadInternal(AlluxioFileInStream.java:269)
at alluxio.client.file.AlluxioFileInStream.positionedRead(AlluxioFileInStream.java:237)
at alluxio.hadoop.HdfsFileInputStream.read(HdfsFileInputStream.java:154)
at alluxio.hadoop.HdfsFileInputStream.readFully(HdfsFileInputStream.java:171)
at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:107)
at com.facebook.presto.hive.parquet.HdfsParquetDataSource.readInternal(HdfsParquetDataSource.java:58)
... 36 more

@harsh9898 is it happened once or happens regularly? our engineer looks at the issue and find it may not be easy to reproduce, if it happens next time, can we be involved earlier? e.g. before removed that file from alluxio storage using --alluxioOnly. If next time it happens, feel free to ping me @LU QIU (Alluxio) or @Chunxu Tang at Alluxio community slack channel https://slackin.alluxio.io/

@LuQQiu @ChunxuTang
It's happening almost every day for multiple files. It's kind of a regular error for us.
I am afraid, I won't be able to show it as this is happening in production on the client side but I will see what else I can provide apart from the full error log trace. If this happens tomorrow as well then I can provide you a more detailed log for that error. Actually, the query is select count(*) as count from <table_name> . This happens every morning at 9:15 am EST. If you will be available at that time then we can connect and discuss the scenarios. Let's work together to find the root cause of this error.

stats_failed_file.txt
command:
./bin/alluxio fs ls <failed_file>
Output:
-rw-r--r--  8c643fb6-5ab8-42b0-b8e5-1d469dd72665 <admin_user_name><PHONE_NUMBER>       PERSISTED 01-18-2023 02:23:01:000 100% <failed_file>
command:
./bin/alluxio fs copyToLocal <failed_file> <local_path>
Output:
Block<PHONE_NUMBER>422 is expected to be 268435456 bytes, but only 62496768 bytes are available. Please ensure its metadata is consistent between Alluxio and UFS.
command:
hdfs dfs -ls abfs://<data_lake_storage_uri>/<failed_file>
Output:
-rw-r--r--   1 8c643fb6-5ab8-42b0-b8e5-1d469dd72665 <admin_user_name><PHONE_NUMBER> 2023-01-18 02:23 abfs://<data_lake_storage_uri>/<failed_file>

From user:
This is my observance of the multiple errors and behavior is the same as follows:
When one of the blocks of the file is in SSD vs MEM. It produces an error for that particular file and then when we remove that file and run again it runs as it will try to cache again directly from UFS. The observance from my side is that whenever the block of the file is in SSD, it is not able to read the data and produces the error.

May related to https://github.com/Alluxio/alluxio/pull/16597

From user
Yes I think the alluxio cache is the issue. On my side, based on my observations, this is what I did:
Formatted the alluxio using ./bin/alluxio format
Then ran the distributedLoad again
Restarted the alluxio
Once the above steps ran, it seems I am not getting this error.
Hence it proves that alluxio was trying to read some old blocks from the cache which was causing this issue.
But I think this could happen in the future as well when UFS is changing frequently.

Likely to cause because of the https://github.com/Alluxio/alluxio/pull/16597

User is willing to tryout the changes in https://github.com/Alluxio/alluxio/pull/16597 when it's released

From @tcrain
do you know if the files with problems are being written through Alluxio to the UFS? Or if they are written directly to the UFS not using Alluxio? Or are both cases possible?
From @harsh9898
all the files are written directly to UFS. I am not using alluxio to write any data. Alluxio is used just to cache UFS data.

@LuQQiu @tcrain can I know if #16597 solved this problem?


Added one exchange

Added YoBit to exchanges

@TheWildHorse Please merge :)

Also removed CoinMkt

@TheWildHorse Updated some links to SSL. Time to go through these pull request soon?


Support a simple optional cache for RestClient-s

Codecov Report
Patch coverage: 100.00% and project coverage change: -0.66 :warning:

Comparison is base (2a8ca29) 99.95% compared to head (b01ba79) 99.30%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
- Coverage   99.95%   99.30%   -0.66%
==========================================
  Files          25       25
  Lines        6566     6434     -132
==========================================
- Hits         6563     6389     -174
- Misses          3       45      +42

Impacted Files
Coverage Δ

src/runez/http.py
100.00% <100.00%> (ø)

src/runez/logsetup.py
100.00% <100.00%> (ø)

src/runez/system.py
99.57% <100.00%> (-0.35%)
:arrow_down:

src/runez/inspector.py
72.50% <0.00%> (-27.50%)
:arrow_down:

src/runez/__main__.py
91.01% <0.00%> (-8.99%)
:arrow_down:

src/runez/program.py
99.46% <0.00%> (-0.02%)
:arrow_down:

src/runez/ascii.py
100.00% <0.00%> (ø)

src/runez/click.py
100.00% <0.00%> (ø)

... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Added multi language option with file to publish

To the user can change his language at it needs

@rafaelqm thanks for your great work. I am thinking to make it optional. so will try to make it optional, as all user doesn't require their app to be multilingual. so will review it and merge it soon.

Ok, but it isn't only to make multilingual, but in my case, the app will work only in pt-br, so it is better to config all the messages and buttons in one file, instead in all views.

Localization Support added.
To Implement Localization : https://labs.infyom.com/laravelgenerator/docs/6.0/localization


breakpoints, etc don't work v1.0.49, .50

Steps to reproduce
Tell us about your environment:

ndb version: v1.0.49, v1.0.50
Platform / OS version: macos 10.14.5
Node.js version: v10.16.0

What steps will reproduce the problem?
ndb firebase serve
Please include code that reproduces the issue.

throw 'ANY THING YOU WANT'
add a debugger statement anywhere

What is the expected result?
when 'pause on exceptions' is enabled (the stop sign button) or debugger statements are reached, pause and show the stack, exception, etc as-it-does-in v1.0.48
What happens instead?
The same thing that happens when not using a debugger--for handled or unhandled exceptions, etc.... seems broken to me. In case it's unclear: nothing happens and the above should happen.
solution

downgrade with npm i -g<EMAIL_ADDRESS>resume work

To run this command, do I need to install globally firebase package?

<EMAIL_ADDRESS>contains a fix, please check it out and thanks for your report.

@ak239 unfortunately this does not appear to be resolved. My application and environment is unchanged and when running the same commands using the newly released v1.0.51, with the original reported issue above, the application no longer runs. I've tried multiple times to the same effect (and have not restarted my machine). However, as before, downgrading to v1.0.48 resolves the problem. (If these changes in .49 through .51 introduce much longer startup times, or if there's the need for restarts after updating, I can take another look. Otherwise this does not appear to resolve the issue.)

Could you share how can I setup the project to reproduce your bug? It sounds like I should create some kind of hello world app with firebase and run firebase serve from the root.

@jimmont could you help with detailed reproduction steps?

I have a firebase project which I run using firebase serve in the project directory where the .firebaserc and firebase.json files are setup and default subfolder for the functions. When I run it using ndb firebase serve after version 1.0.48 in all versions I've tried over the past months ndb no longer breaks on uncaught errors or on breakpoints. I would expect that setting up any generic firebase project, then running it in the way described here should show the problem. In my case the newest v1.1.4 fails to show the running node processes, fails to stop on uncaught exceptions, fails to pause on breakpoints. From my perspective as a user ndb after 1.0.48 no longer works. Has anyone in your group setup a generic firebase project and tried to, for example, setup a post handler where the code tries setting a property on an undefined property, or throws an error, or set breakpoints? I would imagine this would make an excellent first step in reproducing what I am describing as recurring every time I update ndb since v1.0.48.


Add support for .jinja, .jinja2, .j2 file types

Many IDEs use these filetypes to enable syntax highlighting for Jinja template files. The official docs don't specify a required extension, but mention .jinja as an example of a file type.
Adding support for these file types would improve the development experience using Jinja2 and staticjinja.

I think this should be part of the much larger and much more broad jinja project. I'd be surprised if IDE plugins don't already exist, I'm assuming you searched around? I would search that repo to see if someone has already made an issue for this as well.

The upstream jinja project has a section in their documentation that explains that there is no official extension.
I searched this project, and didn't find anything relating to this.
Would it be that difficult to expand the search from .html to the other file types?

The docs say This will recursively search ./templates for templates (any file whose name does not start with . or _) and build them to . So .html files should be matched and should work. Is something not working for you? Is that the "search" that you are talking about? Or are you talking about IDE syntax highlighting?

I'm going to close this as inactive, but please re-open or ping me if you have follow up!

Apologies for the delay @NickCrews .
When building, .html templates are built and output as .html, however .jinja templates are built and output as .jinja
Could default functionality or a configuration option be added to build certain filetypes to .html?

Hi, sorry, but that isn't something that I want to support built in. Either just start with .html files, or you can use a custom rendering rule as described in https://staticjinja.readthedocs.io/en/stable/user/advanced.html#rendering-rules. Something like (I haven't tested this so you will probably need to adjust):
import os
from pathlib import Path

from staticjinja import Site

def render_jinja(site, template, **kwargs):
    # i.e. posts/post1.j2 -> build/posts/post1.html
    out = site.outpath / Path(template.name).with_suffix(".html")
    os.makedirs(out.parent, exist_ok=True)
    template.stream(**kwargs).dump(str(out), encoding="utf-8")

site = Site.make_site(
    searchpath="src",
    outpath="build",
    rules=[(r".*\.j2", render_jinja)],
)

site.render()


Add gardener-e2e-kind presubmit and periodic

What this PR does / why we need it:
Transforms postsubmit post-gardener-e2e-kind on local-extension into presubmit pull-gardener-e2e-kind (not required for merge currently) and ci-gardener-e2e-kind (on master every 4 hours).
Test run on master is at: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind/1472942458718392320
/kind enhancement

/lgtm


feature: terraformls 0.17

bumping terraformls to 0.17.0

@dbgeek btw, in the mean time, 0.17.1 was released. Mind updated this PR, or making a new one?

Going to merge this for now - we can open up a PR for a new bump later.


tag ecryptfs role

@benrudolph something like this

ah just saw this

does this then add a tag to run all the tasks in main.yml? if so, this is the better solution

@benrudolph yeah this is like shorthand for adding tags: ecryptfs to every task in the role


:seedling: Update integration tests

Summary

~Add a pre-flight test run in the release workflow that sets the bundle version for the integration tests (and could be used in other tests as well)~
Use the ginkgo CLI to run tests (I find the output a bit cleaner since our tests are written with Ginkgo)
Refactors the addon integration tests, making it a bit easier to maintain and also add an addon in the future, also adding a deployment readiness check.

Related issue(s)
Fixes #

/cc @mikeshng

/cc @qiujian16

LGTM

/lgtm