Upgrade/v1.79

I started to upgrade imgui from v1.65 to v1.79

Removed implemented functions that are not part of ImGui anymore (DisplayVisibleMin, DisplayVisibleMax)
Raname ConfigResizeWindowsFromEdges to ConfigWindowsResizeFromEdges
Marked obsolete functions as such
Cross checked both API and added references to unimplemented functions from the new version

I also reordered function and added comments such that it resembles imgui.h as possible in order to facilitate upgrades from here

Updated all enums

It is not finished yet, there is a lot of new unimplemented functions, but it should pass tests and be usable with the new ImGui code. I quickly tested the imgui.show_test_window(), and new features seem to work fine. I am willing to put some time into mapping functions to complete the upgrade. Feedback would be great :)

OH MY GOD. That's huge! And great!
How did you manage to do all of that? Last time I was attempting the update I have realized how many flags there are and how often they change I had to create some custom tooling to automate that stuff.

I'm totally loaded with work at the moment but I quit my current job next week so finally will have more time to do all the reviews, merges and releases. Anyway, don't stress about missing functions. We never had 100% feature parity (imgui has HUGE API). Just tell me when you're ready and we merge it. Later we can do incremental updates.

Coverage increased (+2.8%) to 52.393% when pulling 2df3d4ba98c3300ccff017012e36a5643c31dbe7 on KinoxKlark:upgrade/v1.79 into 41d780356ba4f1becca570b55588fecf386303ba on swistakm:master.

OH MY GOD. That's huge! And great!
How did you manage to do all of that? Last time I was attempting the update I have realized how many flags there are and how often they change I had to create some custom tooling to automate that stuff.

I saw your scripts but was not really sure of how to use them properly. I initially wanted to code my own and even started to list all features that such a script should have to compare versions and propose updates, but I figured out that I needed to go manually through the entire process once before being able to automatize it. Maybe I should regroup my notes and take the time to understand your scripts to plan the development of a generic tool that we could use after each update of ImGui. A semi-automated process that checks changes between ImGui versions and proposes the modifications to do would be of great help for future upgrades.

@KinoxKlark BTW sorry for cancelling your builds on travis. I'm working on a new Appveyor pipeline for macOS builds because the old one on travis became unmanageable. Appveyor isn't perfect but simply works better and faster. Maybe will migrate the manylinux1 build to Appveyor too. When I get it done I'll ask you to rebase on master. OK?

@swistakm Ok no problems ;)

Another note: I've noticed that after I've merged the #167 your builds may fail after the rebase. I think that this patch should be sufficient:
diff --git a/ansifeed-cpp/AnsiTextColored.cpp b/ansifeed-cpp/AnsiTextColored.cpp
index f34486d..4022a75 100644
--- a/ansifeed-cpp/AnsiTextColored.cpp
+++ b/ansifeed-cpp/AnsiTextColored.cpp
@@ -116,8 +116,8 @@ namespace ImGui
                                                          // so this is merely to handle direct calls.

         // Align to be pixel perfect
-        pos.x = (float)(int)pos.x + font->DisplayOffset.x;
-        pos.y = (float)(int)pos.y + font->DisplayOffset.y;
+        pos.x = (float)(int)pos.x;
+        pos.y = (float)(int)pos.y;
         float x = pos.x;
         float y = pos.y;
         if (y > clip_rect.w)
@@ -447,7 +447,7 @@ namespace ImGui
         if (text_end == NULL)
             text_end = text + strlen(text);  // FIXME-OPT

-        const ImVec2 text_pos(window->DC.CursorPos.x, window->DC.CursorPos.y + window->DC.CurrentLineTextBaseOffset);
+        const ImVec2 text_pos(window->DC.CursorPos.x, window->DC.CursorPos.y + window->DC.CurrLineTextBaseOffset);
         const float wrap_pos_x = window->DC.TextWrapPos;
         const bool wrap_enabled = wrap_pos_x >= 0.0f;
         if (text_end - text > 2000 && !wrap_enabled) {

You should be able to simply git apply it.

Hopefully this draft pull request will make it in this year still. At least that's how this pull request looks like. I heavily make use of pyimgui in Python 3.9 and it would be awesome to be finally able to use the new docking layout system and tabbing and more. So many thanks for this already :+1:

Hopefully this draft pull request will make it in this year still. At least that's how this pull request looks like. I heavily make use of pyimgui in Python 3.9 in Debian and it would be awesome to be finally able to use the new docking layout system and tabbing and more. So many thanks for this already 👍

Thanks, that is motivating. Unfortunately, I can't promise anything for this year. Exams are coming, and I will soon have less time. I try to do as much as I can for now.
Concerning docking and viewport: As I understand it, it is still a working branch in the original imgui project. It is safe to use, but I am wondering if we should add it to master. Maybe a good idea would be to also create a docking branch on pyimgui that we would keep up to date. I don't know how it would work with pip.

I have reviewed the list of releases from 1.65 to 1.79 in order to apply all changes, especially breaking ones. Easily mapped functions have been mapped to.
Here a more detailed list of changes for this pull request :

Checkout Imgui v1.79 branch
Update of the imported API in cimgui.pxd
Update of all enums and flags and addition to doc (documentation not finished, just imported)
Mapping of all enums and flags
Mapping of functions:

SetNextWindowSize()
SetScrollHereX()
SetScrollHereY()
SliderAngle()
GetDragDropPayload()
ShowAboutWindow()
BeginTabBar() <- Tab features fully usable :D
EndTabBar()
TabItemButton()
SetTabItemClosed()
ImDrawData::DisplayPos
ImDrawData::DisplaySize
ImDrawData::FramebufferScale
IsItemEdited()
IsItemActivated()
IsItemDeactivated()
IsItemDeactivatedAfterEdit()
IsItemToggledOpen()
ImGuiStyle::LogSliderDeadzone
ImGuiStyle::TabRounding
ImGuiStyle::TabBorderSize
ImGuiStyle::TabMinWidthForCloseButton
ImGuiStyle::SelectableTextAlign
ImGuiStyle::AntiAliasedLinesUseTex
ImGuiStyle::CircleSegmentMaxError
GetBackgroundDrawList()
InputTextWithHint()
SetNextItemWidth()
SetNextItemOpen()
GetTreeNodeToLabelSpacing()
ImGuiStyle::ColorButtonPosition
ImGuiStyle::WindowMenuButtonPosition
ImDrawList::AddNgon()
ImDrawList::AddNgonFilled()
AddInputCharacterUTF16()
OpenPopupOnItemClick()
BeginPopupContextVoid()
IsPopupOpen()
GetGlyphRangesThai()
GetGlyphRangesVietnamese()
ImGuiIO::MetricsRenderIndices
ImGuiIO::MetricsRenderWindows
ImGuiIO::MetricsActiveAllocations

Functions that have changed of prototype:

SliderXXX() now use SliderFlags and no power parameters. power is kept as it is still available marked as OBSOLETED in imgui API.
DragXXX() now use SliderFlags and no power parameters. power is kept as it is still available marked as OBSOLETED in imgui API.
DragXXX()
AddInputCharacter() change signature from unsigned short c to unsigned int c
AddRectXXX() change flags default value to use enum instead of direct number
AddCicle() argument num_segment now defautls to 0 meaning auto-tesselation
AddCicleFilled() argument num_segment now defautls to 0 meaning auto-tesselation
IsKeyPressed() first argument should not default anymore
IsItemClicked() Arguments type change from int to ImGuiMouseButton
BeginPopupContextWindow()
InvisibleButton() Now take a ImGuiBttonFlags in input

As far as I know, it is now fully upgraded to 1.79. Missing mappings were already missing in the old version. New features were added. The list of features to map in cimgui.pxd should be up to date with imgui.h such that it is now easy to see which features miss. I documented new mapping as I implemented them, but it is not complete nor perfect.
Tests should be done to check that there are no mistakes. I quickly tested each new feature but not all possible cases. Documentation should be updated too. I saw mixed references to pyimgui and imgui API in docstrings, but I did not fix it each time. Documentation does not seams to be normalized. I think a massive pass on the documentation should be done. I don't know if all of this should be done in this branch or if we start by merging it to master and plan afterward?
Concerning the next upgrades: v1.79 is the most recent release of ImGui but not up to date with the master branch. It would be nice to have a clear idea of which versions to map in which branch of this project? Should we have an imgui/v1.79 branch with a clean mapping and keep the master branch up mapped to ImGui's master? Should we have a docking branch? I think a mirroring of ImGui's most important branching could be great. It would then be possible to map features as they are appended and changed in the ImGui repo. This way, we could be fast to upgrade when ImGui updates. What do you think?

Wow

That is awesome!
Regarding the branching. My initial idea was to simply merge your changes to master and release the changes as 2.0.0. Supporting all past upstream ImGui versions isn't worth the effort in my opinion.
I don't know what to do about docking branch yet. It is synced with master systematically so we could do the same and release it also on a branch synced with our master. But I'm not sure what to do with PyPI releases. That would take another package name for sure. If we want to do this we would have also to invest in faster build pipelines, because at current situation it takes too much of time to get all the build artifacts. That could also create some confusion if user installs both imgui and e.g. imgui-docking from PyPI.
What we can do for sure is to setup separate branch experimentally for docking feature and see how it goes. Maybe users interested in that particular feature can handle installing from sources until docking becomes official part of upstream release. Still, I'm afraid scaling up to more extra features on branches won't be sustainable.
I'm also not sure what to do with upstream master.

About the master branch or the stable release branch. In my humble opinion, we should stick to the stable release for now, since the master is per say the alpha, ever changing.
If going with the master, I assume that this means, you have to correlate every little change every now and then (worst case, every day), since the master branch changes quite often, in contrast to the stable one. Furthermore, if you don't want every little change, how can you know, which change to adapt to be sure the package is suddenly not broken somewhere (regressions)?
On the other hand, the stable release shouldn't change until the next stable release, I think. Since the pyimgui version is an everyday version for us to work with regularly, it would be reassuring to have it based on something recent but stable. As @swistakm also mentioned, the resources for recompiling everything regularly is limited.
Thus, I would already be very happy and be quite satisfied to having pyimgui be at the same version as their stable though.

Yeah, you make both excellent points.
I changed the config of this pull request from 'draft' to 'ready for review.' The upgrade is not entirely finished because all features are not fully mapped, but it was never the case as I understand it. Maybe we should wait a little more until releasing it as the 2.0.0 to have time to test it and complete it a little more. It would be nice to have a clean version for the release. The reason why I set it ready to review is to "officialize" a little bit more this work as a real work in progress, and maybe you want to merge it to a working branch such that other people start to work on it too?
As far as I know, the upgrade to 1.79 is done since it compiles fine with the imgui-cpp 1.79 branch, and it contains numbers of the new features (e.g. TabBar). I am currently working on some clipboard stuff, new functions to map that break the existing callback system. But I have the impression that I could endlessly work on implementing and fixing features. Maybe it would be time to separate the work features by features?
Concerning branches, my proposal is the following: Indeed, matching every feature branch of imgui is a bit too much and would be too much work and effort. @Johndeep makes good points concerning imgui's master branch. Limiting our work to official releases is probably the best choice. I was afraid that the effort necessary to upgrade after each release would be too big and take too much time to do. But if it is done rigorously after each update with a standardized process (even with semi-automated scripts), I think that it is feasible (more feasible than being up to date with every little change of the master branch indeed). Concerning the docking branch it seems to be a feature that has great interest. Part of the reason why I started this upgrade is that I, myself, wanted to use docking features and found that for implementing the docking branch a first step was to upgrade to the latest release. Thus a pyimgui/docking experimental branch would be great. Interested people could simply install from source for now.
What do you think about the next concerning this upgrade? Should we plan features implementation, testing, and documentation redaction? Should this branch be integrating to pyimgui for community work, or should I keep my current fork working on it?

It seems that there is a need for mapping some of the imgui_internal.h functionality like PushItemFlag() as in #178. Should we provide this? I don't think it has its place in pyimgui.core but maybe in a pyimgui.internal module? What do you think?

It seems that there is a need for mapping some of the imgui_internal.h functionality like PushItemFlag() as in #178. Should we provide this? I don't think it has its place in pyimgui.core but maybe in a pyimgui.internal module? What do you think?

👍 from me for both exposing some internals and doing that in separate module. I just wonder if that would be easy to do as a separate pyx module? I remember that I wanted to modularize the library at the very beginning but had some technical troubles with that. I've eventually put everything in core and just prefixed stuff that should be publicly exposed in different submodule (see extras). If you know how to do that properly, you're more than welcome to do that :)

👍 from me for both exposing some internals and doing that in separate module. I just wonder if that would be easy to do as a separate pyx module? I remember that I wanted to modularize the library at the very beginning but had some technical troubles with that. I've eventually put everything in core and just prefixed stuff that should be publicly exposed in different submodule (see extras). If you know how to do that properly, you're more than welcome to do that :)

Indeed it wasn't easy, but I think I managed to do it. Maybe you should verify that it works as intended and that I didn't forget anything. Adding the internal submodule wasn't really difficult, just adapting the setup.py and finding a way to use common code between core and internal like the error handling. The main difficulty was that both modules share different imgui context. The solution I found was to propagate the change in context trough submodules using a UpdateImGuiContext() calling create_context(), destroy_context() and get_current_context(). I don't think that there is any other global state that every submodule should be aware of, but I may be wrong. If you think that there is a better way to do it, let me know ^^
To test it I mapped PushItemFlag() and PopItemFlag() from internal and tested the disabled features asked in  #178:
disabled = False

# ...

imgui.begin("Form")
clicked, disabled = imgui.checkbox("Disable", disabled)
if disabled:
    imgui.internal.push_item_flag(imgui.internal.ITEM_DISABLED, True)
    imgui.push_style_var(imgui.STYLE_ALPHA, imgui.get_style().alpha * 0.5)

imgui.text('This is a test')
if imgui.button('Click me!'):
    print('Clicked')

if disabled:
    imgui.internal.pop_item_flag()
    imgui.pop_style_var()

imgui.end()

Getting this error at startup on occasion:
  File "imgui/core.pyx", line 2431, in imgui.core.new_frame
imgui.core.ImGuiError: ImGui assertion error ((g.IO.DeltaTime > 0.0f || g.FrameCount == 0) && "Need a positive DeltaTime!") at imgui-cpp/imgui.cpp:6867

This was happening in https://github.com/swistakm/pyimgui/pull/162 as well, I haven't tested 1.3.0 but I'm pretty sure it never happened in 1.2

Getting this error at startup on occasion:
  File "imgui/core.pyx", line 2431, in imgui.core.new_frame
imgui.core.ImGuiError: ImGui assertion error ((g.IO.DeltaTime > 0.0f || g.FrameCount == 0) && "Need a positive DeltaTime!") at imgui-cpp/imgui.cpp:6867

This was happening in #162 as well, I haven't tested 1.3.0 but I'm pretty sure it never happened in 1.2

Thanks, I'll look at it!
Unfortunately, I won't have a lot of time in the next few days since my exam session started at university. I'll come back to this dev mid January :)

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

Thanks for the feedback! Concerning the difficulty using input_text_multiline, can you be a little more precise about the problem you encounter? Is it the buffer_length input parameter that does not feel natural? If so, maybe we should set it to a default value, but I am not sure which one yet. I think it is quite usage dependent.

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

Thanks for the feedback! Concerning the difficulty using input_text_multiline, can you be a little more precise about the problem you encounter? Is it the buffer_length input parameter that does not feel natural? If so, maybe we should set it to a default value, but I am not sure which one yet. I think it is quite usage dependent.

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

Thanks for the feedback! Concerning the difficulty using input_text_multiline, can you be a little more precise about the problem you encounter?

You usually need some way to "commit" an input line. I can add a button to do so, but I prefer pressing the Enter key. But you can't achieve this behavior easily with input_text_multiline, because the default behavior is to return True if "changed." Setting the INPUT_TEXT_ENTER_RETURNS_TRUE flag works for input_text but has the opposite meaning for input_text_multiline -- Enter will open a new line and Ctrl-Enter commit returns True. input_text_multiline needs to be used with IsItemDeactivatedAfterEdit if you want to let Enter key to commit the change.

IsItemDeactivatedAfterEdit is great, otherwise input_text_multiline is very hard to use.

Thanks for the feedback! Concerning the difficulty using input_text_multiline, can you be a little more precise about the problem you encounter?

You usually need some way to "commit" an input line. I can add a button to do so, but I prefer pressing the Enter key. But you can't achieve this behavior easily with input_text_multiline, because the default behavior is to return True if "changed." Setting the INPUT_TEXT_ENTER_RETURNS_TRUE flag works for input_text but has the opposite meaning for input_text_multiline -- Enter will open a new line and Ctrl-Enter commit returns True. input_text_multiline needs to be used with IsItemDeactivatedAfterEdit if you want to let Enter key to commit the change.

Oh, nvm. I didn't notice that imgui.INPUT_TEXT_CTRL_ENTER_FOR_NEW_LINE is a different flag.

Oh, nvm. I didn't notice that imgui.INPUT_TEXT_CTRL_ENTER_FOR_NEW_LINE is a different flag.

Indeed imgui.INPUT_TEXT_CTRL_ENTER_FOR_NEW_LINE should inverse the default comportment and permits unfocus with Enter and create a new line with Ctrl+Enter, which sound like what you want to do. This is a clear case where more documentation is needed, maybe with example cases such as this one for the different flags.

Indeed imgui.INPUT_TEXT_CTRL_ENTER_FOR_NEW_LINE should inverse the default comportment and permits unfocus with Enter and create a new line with Ctrl+Enter, which sound like what you want to do. This is a clear case where more documentation is needed, maybe with example cases such as this one for the different flags.

Finally, my exam session is done, and I will have more time to invest in this project.
Big news, imgui release 1.80 is here, what should we do about that? For now, I continue with 1.79 but should we directly update to 1.80 for the 2.0.0 release? I quickly checked the changelog: the main new thing is the merge of Table in master branch, a feature that has been requested for pyimgui here #196. There are also various renames and minor breaking changes that should be dealt with but nothing to complicate it seems.

Finally, my exam session is done, and I will have more time to invest in this project.
Big news, imgui release 1.80 is here, what should we do about that? For now, I continue with 1.79 but should we directly update to 1.80 for the 2.0.0 release? I quickly checked the changelog: the main new thing is the merge of Table in master branch, a feature that has been requested for pyimgui here #196. There are also various renames and minor breaking changes that should be dealt with but nothing to complicate it seems.

In my humble opinion, if the jump to version 1.80 does not add too much work, it would be advisable to go with 1.80, also because it is now the current stable and there should be many bug fixes and of course the table feature.
Unfortunately, there is still no docking feature though. But if we would at least can raise the minimum version, the feature gap between pyimgui and imgui official can be kept to a minimum.
You mentioned that there are "also various renames and minor breaking changes". Does this refer to the pyimgui v1.3 or the jump from 1.79 to 1.80 from imgui official?

In my humble opinion, if the jump to version 1.80 does not add too much work, it would be advisable to go with 1.80, also because it is now the current stable and there should be many bug fixes and of course the table feature.
Unfortunately, there is still no docking feature though. But if we would at least can raise the minimum version, the feature gap between pyimgui and imgui official can be kept to a minimum.
You mentioned that there are "also various renames and minor breaking changes". Does this refer to the pyimgui v1.3 or the jump from 1.79 to 1.80 from imgui official?

You mentioned that there are "also various renames and minor breaking changes". Does this refer to the pyimgui v1.3 or the jump from 1.79 to 1.80 from imgui official?

I am referring to the jump from 1.79 to 1.80 of imgui. As in each release, they have a section of their changelog dedicated to breaking changes from the previous release to the new one. I didn't get through it in detail, but it is usually quick fixes like function renaming and deletion of old obsolete stuff.
From pyimgui 1.3.0 to 2.0.0, they will be breaking changes as a lot of stuff has been moved or renamed during this massive upgrade, without mentioning the drop of support for old python versions. I think the idea, for now, is to get as close as possible to the official imgui and for future release be more carful about breaking changes and how we want to deal with them.

You mentioned that there are "also various renames and minor breaking changes". Does this refer to the pyimgui v1.3 or the jump from 1.79 to 1.80 from imgui official?

I am referring to the jump from 1.79 to 1.80 of imgui. As in each release, they have a section of their changelog dedicated to breaking changes from the previous release to the new one. I didn't get through it in detail, but it is usually quick fixes like function renaming and deletion of old obsolete stuff.
From pyimgui 1.3.0 to 2.0.0, they will be breaking changes as a lot of stuff has been moved or renamed during this massive upgrade, without mentioning the drop of support for old python versions. I think the idea, for now, is to get as close as possible to the official imgui and for future release be more carful about breaking changes and how we want to deal with them.

Getting this error at startup on occasion:
  File "imgui/core.pyx", line 2431, in imgui.core.new_frame
imgui.core.ImGuiError: ImGui assertion error ((g.IO.DeltaTime > 0.0f || g.FrameCount == 0) && "Need a positive DeltaTime!") at imgui-cpp/imgui.cpp:6867

@TTimo: I think it should be ok now. Negative io.delta_time has been made illegal in imgui 1.68. To fix it, I had to change integration codes, checking for a negative delta time and fixing delta_time to a small positive value when necessary (I think the problem generally arises at startup when delta_time == 0). Depending on the backend, I had to add a process_inputs() function that has to be called every frame. It was already like that for gfwl and pysdl2; I added it to pygame, pyglet, and coco2d. Example code has been updated. Let me know if there are any problems.

Getting this error at startup on occasion:
  File "imgui/core.pyx", line 2431, in imgui.core.new_frame
imgui.core.ImGuiError: ImGui assertion error ((g.IO.DeltaTime > 0.0f || g.FrameCount == 0) && "Need a positive DeltaTime!") at imgui-cpp/imgui.cpp:6867

@TTimo: I think it should be ok now. Negative io.delta_time has been made illegal in imgui 1.68. To fix it, I had to change integration codes, checking for a negative delta time and fixing delta_time to a small positive value when necessary (I think the problem generally arises at startup when delta_time == 0). Depending on the backend, I had to add a process_inputs() function that has to be called every frame. It was already like that for gfwl and pysdl2; I added it to pygame, pyglet, and coco2d. Example code has been updated. Let me know if there are any problems.

@KinoxKlark sounds good! I've never seen the assert fire with DeltaTime being negative, only it being 0

@KinoxKlark sounds good! I've never seen the assert fire with DeltaTime being negative, only it being 0

Could you please add the NavInputs array so I can implement gamepad navigation?

Could you please add the NavInputs array so I can implement gamepad navigation?

Could you please add the NavInputs array so I can implement gamepad navigation?

@tpwrules, IO.NavInputs should now be mapped as io.nav_inputs. ConfigFlags and BackendFlags were already mapped as config_flags and backend_falgs. Please, let me know if there are any issues. I didn't fully test it. I didn't either modify the backends integration code, so if you feel like sharing your implementation of gamepad navigation, it would be great!

Could you please add the NavInputs array so I can implement gamepad navigation?

@tpwrules, IO.NavInputs should now be mapped as io.nav_inputs. ConfigFlags and BackendFlags were already mapped as config_flags and backend_falgs. Please, let me know if there are any issues. I didn't fully test it. I didn't either modify the backends integration code, so if you feel like sharing your implementation of gamepad navigation, it would be great!

@KinoxKlark You mis-spelled the NAV_INPUT_TEWAK_FAST constant :) Other than that, it works perfect. Thank you!
My joystick code is nothing special. Just a loop each frame that reads the relevant buttons using pygame's joystick API and updates nav_inputs appropriately.

@KinoxKlark You mis-spelled the NAV_INPUT_TEWAK_FAST constant :) Other than that, it works perfect. Thank you!
My joystick code is nothing special. Just a loop each frame that reads the relevant buttons using pygame's joystick API and updates nav_inputs appropriately.

I found a way to implements callbacks as requested in #200.
Python callable and user data are bundled together in a _callback_user_info object, which is used as user_data in imgui API. A generic callback C function is used as the function pointer, and this function is in charge of calling the desired python callable and passing it correct user data. I don't know if this is the best way to do it, but it seems to work fine for InputTextCallback and SizeCallback.

I found a way to implements callbacks as requested in #200.
Python callable and user data are bundled together in a _callback_user_info object, which is used as user_data in imgui API. A generic callback C function is used as the function pointer, and this function is in charge of calling the desired python callable and passing it correct user data. I don't know if this is the best way to do it, but it seems to work fine for InputTextCallback and SizeCallback.

Finally, my exam session is done, and I will have more time to invest in this project.

Great to hear that. I hope all the exams went well.

Big news, imgui release 1.80 is here, what should we do about that? For now, I continue with 1.79 but should we directly update to 1.80 for the 2.0.0 release?

I agree with @Johndeep. I think that if the amount of changes isn't that big and you are able to cover them then it is really worth update straight to 1.80. Feel free to make that stretch if you have time for that :) Especially if we can fit all those minor breaking changes directly to the 2.0.0 release.

Python callable and user data are bundled together in a _callback_user_info object, which is used as user_data in imgui API. A generic callback C function is used as the function pointer, and this function is in charge of calling the desired python callable and passing it correct user data. I don't know if this is the best way to do it, but it seems to work fine for InputTextCallback and SizeCallback.

Yeah, those unfortunate callbacks... I will have to take a closer look at the code because despite it looks easy it can easily blow up. Because due to how python references on C-level work and how ImGui runs those callbacks it is extremely easy to cause segmentation fault.
We've already discussed how to properly handle callbacks but never implemented the code for them. You can take a look on #93 where we identified possible problems that may occur and the ways to mitigate them. In the meantime I will try to checkout your code and run some experiments on it.

Finally, my exam session is done, and I will have more time to invest in this project.

Great to hear that. I hope all the exams went well.

Big news, imgui release 1.80 is here, what should we do about that? For now, I continue with 1.79 but should we directly update to 1.80 for the 2.0.0 release?

I agree with @Johndeep. I think that if the amount of changes isn't that big and you are able to cover them then it is really worth update straight to 1.80. Feel free to make that stretch if you have time for that :) Especially if we can fit all those minor breaking changes directly to the 2.0.0 release.

Python callable and user data are bundled together in a _callback_user_info object, which is used as user_data in imgui API. A generic callback C function is used as the function pointer, and this function is in charge of calling the desired python callable and passing it correct user data. I don't know if this is the best way to do it, but it seems to work fine for InputTextCallback and SizeCallback.

Yeah, those unfortunate callbacks... I will have to take a closer look at the code because despite it looks easy it can easily blow up. Because due to how python references on C-level work and how ImGui runs those callbacks it is extremely easy to cause segmentation fault.
We've already discussed how to properly handle callbacks but never implemented the code for them. You can take a look on #93 where we identified possible problems that may occur and the ways to mitigate them. In the meantime I will try to checkout your code and run some experiments on it.

Great to hear that. I hope all the exams went well.

Thanks, indeed it went well.

I agree with @Johndeep. I think that if the amount of changes isn't that big and you are able to cover them then it is really worth update straight to 1.80. Feel free to make that stretch if you have time for that :) Especially if we can fit all those minor breaking changes directly to the 2.0.0 release.

I agree too, so I will stretch this upgrade to 1.80 directly.

Yeah, those unfortunate callbacks... I will have to take a closer look at the code because despite it looks easy it can easily blow up. Because due to how python references on C-level work and how ImGui runs those callbacks it is extremely easy to cause segmentation fault.
We've already discussed how to properly handle callbacks but never implemented the code for them. You can take a look on #93 where we identified possible problems that may occur and the ways to mitigate them. In the meantime I will try to checkout your code and run some experiments on it.

Thanks for the sharing. With a quick look, I see that I have forgotten to consider lambdas. I will take a more in-depth look at this discussion and run more experiments. Indeed it quickly became more difficult than expected; having some feedback and crosschecking on my implementation would be great!

Great to hear that. I hope all the exams went well.

Thanks, indeed it went well.

I agree with @Johndeep. I think that if the amount of changes isn't that big and you are able to cover them then it is really worth update straight to 1.80. Feel free to make that stretch if you have time for that :) Especially if we can fit all those minor breaking changes directly to the 2.0.0 release.

I agree too, so I will stretch this upgrade to 1.80 directly.

Yeah, those unfortunate callbacks... I will have to take a closer look at the code because despite it looks easy it can easily blow up. Because due to how python references on C-level work and how ImGui runs those callbacks it is extremely easy to cause segmentation fault.
We've already discussed how to properly handle callbacks but never implemented the code for them. You can take a look on #93 where we identified possible problems that may occur and the ways to mitigate them. In the meantime I will try to checkout your code and run some experiments on it.

Thanks for the sharing. With a quick look, I see that I have forgotten to consider lambdas. I will take a more in-depth look at this discussion and run more experiments. Indeed it quickly became more difficult than expected; having some feedback and crosschecking on my implementation would be great!

I added various tests for these callbacks in tests/test_crazy_callbacks.py. I tried to include as many cases as I could think of; feel free to complete it. I probably forgot many cases.
For now, I tested:

input_text callbacks are correctly called none, one or multiple time depending on events
If multiple call to set_next_window_size_constraints() before begin() only the most recent one is kept and memory should not leak (to triple check)
Callbacks set with set_next_window_size_constraints() are only valid for the next begin()
Function as a callback
Class as a callback
Class static method as a callback
Class method as a callback
Lamba directly in argument as a callback
Lambda generated by external function as a callback
Closure as a callback
Decorated function as a callback

I hope it will help.

I added various tests for these callbacks in tests/test_crazy_callbacks.py. I tried to include as many cases as I could think of; feel free to complete it. I probably forgot many cases.
For now, I tested:

input_text callbacks are correctly called none, one or multiple time depending on events
If multiple call to set_next_window_size_constraints() before begin() only the most recent one is kept and memory should not leak (to triple check)
Callbacks set with set_next_window_size_constraints() are only valid for the next begin()
Function as a callback
Class as a callback
Class static method as a callback
Class method as a callback
Lamba directly in argument as a callback
Lambda generated by external function as a callback
Closure as a callback
Decorated function as a callback

I hope it will help.

Update from 1.79 to 1.80 has been done!
Sorry for the branch delete/restore; I wanted to rename the branch to upgrade/v1.80 without success ^^

I have updated the whole API to be in sync with imgui 1.80
New flags have been implemented
Various small changes (renaming, etc)
I mapped the new Table API and ran some tests. It seems to work fine, but more tests should be done. I didn't do the documentation yet.

Let me know if you see any problems. I will continue testing and making sure everything is okay.
Concerning the Table API, for sorting one has to call TableGetSortSpecs() which returns an ImGuiTableSortSpecs. This object contains a pointer to ImGuiTableColumnSortSpecs that shall be used as an array to the selected columns to sort. I implemented it such that this given field can be used as a python array or an iterable. The syntax is as close as possible as the original Imgui syntax, i.e. table_sort_specs.specs[idx] and table_sort_specs.specs_count, but I think it is a bit verbose and directly using table_sort_specs as an array would be simpler. Like: table_sort_specs[idx] and table_sort_specs.count. What do you think?

Update from 1.79 to 1.80 has been done!
Sorry for the branch delete/restore; I wanted to rename the branch to upgrade/v1.80 without success ^^

I have updated the whole API to be in sync with imgui 1.80
New flags have been implemented
Various small changes (renaming, etc)
I mapped the new Table API and ran some tests. It seems to work fine, but more tests should be done. I didn't do the documentation yet.

Let me know if you see any problems. I will continue testing and making sure everything is okay.
Concerning the Table API, for sorting one has to call TableGetSortSpecs() which returns an ImGuiTableSortSpecs. This object contains a pointer to ImGuiTableColumnSortSpecs that shall be used as an array to the selected columns to sort. I implemented it such that this given field can be used as a python array or an iterable. The syntax is as close as possible as the original Imgui syntax, i.e. table_sort_specs.specs[idx] and table_sort_specs.specs_count, but I think it is a bit verbose and directly using table_sort_specs as an array would be simpler. Like: table_sort_specs[idx] and table_sort_specs.count. What do you think?

A new release of Imgui is here. I stretched the upgrade to v1.81. This implies:

ListBoxHeader() is now obsolete and BeginListBox() should be used. Thus I added begin_list_box() and keept list_box_header() as a redirection and documented that this function is now obsolete.
ListBoxFooter() is new obsolete and EndListBox() should be used. Thus I added end_list_box() and keept list_box_footer() as a redirection and documented that this function is now obsolete.
Added new flags ImGuiViewportFlags_ mapped as VIEWPORT_FLAGS_NONE, VIEWPORT_FLAGS_IS_PLATFORM_WINDOW, VIEWPORT_FLAGS_IS_PLATFORM_MONITOR, and VIEWPORT_FLAGS_OWNED_BY_APP
Mapped new struct ImGuiViewport
Mapped new function ImGuiViewport* GetMainViewport() as get_main_viewport()

This release implies some changes concerning imgui_freetype but I am not sure if it concerns us or not?

A new release of Imgui is here. I stretched the upgrade to v1.81. This implies:

ListBoxHeader() is now obsolete and BeginListBox() should be used. Thus I added begin_list_box() and keept list_box_header() as a redirection and documented that this function is now obsolete.
ListBoxFooter() is new obsolete and EndListBox() should be used. Thus I added end_list_box() and keept list_box_footer() as a redirection and documented that this function is now obsolete.
Added new flags ImGuiViewportFlags_ mapped as VIEWPORT_FLAGS_NONE, VIEWPORT_FLAGS_IS_PLATFORM_WINDOW, VIEWPORT_FLAGS_IS_PLATFORM_MONITOR, and VIEWPORT_FLAGS_OWNED_BY_APP
Mapped new struct ImGuiViewport
Mapped new function ImGuiViewport* GetMainViewport() as get_main_viewport()

This release implies some changes concerning imgui_freetype but I am not sure if it concerns us or not?

There is a goal to deprecate the concept of a main viewport in the docking branch:
https://github.com/ocornut/imgui/issues/1542#issuecomment-571121789
So it might be better to not even add this to the api?

There is a goal to deprecate the concept of a main viewport in the docking branch:
ocornut/imgui#1542 (comment)
So it might be better to not even add this to the api?

I am not sure to agree, as I understand it GetMainViewport() has been added to the master branch in order to homogenize master and docking branches. Indeed the plan seems to transition to a "No main viewport" in the future, but I don't think this invalidates the Viewport struct which I suppose will just be used differently. Furthermore, we have no idea in how long this change will come and one of the ideas after completing this upgrade was to map in another branch the much desired docking functionalities. Shouldn't we ensure that both API works similarly in that regards if this has been the choice made in official Dear Imgui?

There is a goal to deprecate the concept of a main viewport in the docking branch:
ocornut/imgui#1542 (comment)
So it might be better to not even add this to the api?

I am not sure to agree, as I understand it GetMainViewport() has been added to the master branch in order to homogenize master and docking branches. Indeed the plan seems to transition to a "No main viewport" in the future, but I don't think this invalidates the Viewport struct which I suppose will just be used differently. Furthermore, we have no idea in how long this change will come and one of the ideas after completing this upgrade was to map in another branch the much desired docking functionalities. Shouldn't we ensure that both API works similarly in that regards if this has been the choice made in official Dear Imgui?

In "no main viewport" mode the bounds returned by GetMainViewport() will likely refer to a monitor
So as per linked pyimgui's thread, the linked comment is sort of invalidated by this new direction
Hence adding this function to master
KinoxKlark's answer is right there
The definition of ImGuiViewport evolved:

enum ImGuiViewportFlags_
{
    ImGuiViewportFlags_IsPlatformWindow         = 1 << 0,   // Represent a Platform Window
    ImGuiViewportFlags_IsPlatformMonitor        = 1 << 1,   // Represent a Platform Monitor (unused yet)
    [....]
};

// - Currently represents the Platform Window created by the application which is hosting our Dear ImGui windows.
// - In 'docking' branch with multi-viewport enabled, we extend this concept to have multiple active viewports.
// - In the future we will extend this concept further to also represent Platform Monitor and support a "no main platform window" operation mode.
// - About Main Area vs Work Area:
//   - Main Area = entire viewport.
//   - Work Area = entire viewport minus sections used by main menu bars (for platform windows), or by task bar (for platform monitor).
//   - Windows are generally trying to stay within the Work Area of their host viewport.
struct ImGuiViewport
{

Hey guys (and especially @KinoxKlark) I'm really sorry for stalling with imgui but I still under a lot of load from other commitments (book, job, side-gig) and couldn't find even time to respond to latest issues/questions/PRs. I definitely didn't give up on the project but can't give it any attention this month. Maybe April will be more lucky to me.
Anyway, I know that this will be reoccurring problem and I will be ever-busy always. If anyone want's to step in and take at least a part of the responsibility for the project stewardship I will be more than happy to welcome official maintainers with repo write access and proverbial "keys to the castle". The best if we could have trio passionate contributors that would be able to organize themselves, set their direction, decide on things and further drive the project.

Hey @swistakm, don't worry, I myself have been a bit occupied recently and couldn't give this PR all the attention that it needs. I am hoping to stretch to the latest release (dear imgui 1.82) next week.
I am willing to help and take some responsibilities if it can help this project moving forward. But I am not sure if I have all the requested skills to maintain all of it by myself. I am not that familiar with the entire building pipeline and didn't touch much of the documentation either (else than the addition of docstring to mapped functions). As you suggested, it would be great to form a small team of contributors to organize and help each other.
Concerning this push request, as said two paragraphs earlier, I plan to stretch to the latest imgui release. Probably next week, I am a bit loaded with work this week. After that, I can always touch a bit of code here and there but I have the feeling that this push request is getting out of hand in terms of code size. It is also going in various directions (upgrade + callbacks + internal module + etc) and I think all these things should be focused on one at a time.
Thus here is what I would suggest: The branch, as it is, is clearly not ready to be pushed in production and a quality pass is needed. Unfortunately having all these changes in one pull request seems not to be very friendly to community modification. Maybe we could merge this PR in a specific 'dev' branch or 'upgrade' branch. By doing this, I think it would be clearer what is the development state of this upgrade and that we are preparing 2.0.0, and contributions could be managed via push request directly. For example, people keep mapping functions that have already been mapped here as in #197 or #218, which is comprehensible since it is difficult, even for myself, to see what has been mapped or not (the list is long ^^).
The mapping of functions is pretty straightforward, I tested the majority of them, especially tricky ones, as I mapped them. And as I understand it, some testing is done by using documentation code. Thus I feel this part is pretty solid, maybe going through all of them at some point to be sure.
Parts that most concerns me are:

Change in callbacks mechanism: I have tested it as much as possible and I don't see any obvious memory leaks. But we are never free of a tricky python combination of callables that I am not aware of. This part calls for more systematic testing and code review
Change in build config: I changed a little bit the build config at some point to separate and map the 'internal' submodule. Reviewing and testing this part would be great
Various fixes in integration code should be reviewed too.
And as always: better documentation.

Thus I would suggest a merging of this PR in a dev-specific branch for organizing and attacking these reviews and quality pass separately. If other contributors want to push request stuff for the next major release they would be able to do it in a centralized branch.
What do you think?

I would suggest to freeze it, and not keep following every Dear ImGui upgrade,
otherwise this PR will be a moving target.

I've been using this PR (v1.81) over the upstream for a while now. Looking at the latest comments it seems lots of things happening, including the upstream maintainer-ship being on the table. I do not have any input on how to tackle this, but would like to see some sort of merging of this effort into either upstream or this repo in order not to lose this huge amount of work.
I agree with @learn-more that if the goal posts with merging this keep moving, it might become endless journey. I feel that v1.80 was a huge jump on ImGui side and now with v1.82 things might have been cleaned up just enough to allow merging this PR.
FWIW, I'm working on having ImPlot python binding (actually have it in alpha as I write this) and it heavily relies on this PR. Not sure how to actually release it: a) separate repo or b) ask you guys if you thing it could be included here as part of pyimgui. Sorry for for being off topic.
Thanks!

I would suggest to freeze it, and not keep following every Dear ImGui upgrade,
otherwise this PR will be a moving target.

I agree with you. This PR should be kept stable and be finalized. The keep up with ImGui releases should be done in parallel. I think it is important to have a branch that is kept up to date with official releases, from my experience in this upgrade, going from one release to the other is a manageable amount of work but skipping several releases as it was at first (from v1.65 to v1.79) is a huge investment.
I am not sure how we should proceed from now, maybe we should freeze it in v1.82 and ensure this is stable enough and merge with other PR to release a 2.0.0 version of pyimgui as it was originally suggested by @swistakm. This for now entirely depends on @swistakm's spare time and, as I understand it, it is unfortunately not for soon.

FWIW, I'm working on having ImPlot python binding (actually have it in alpha as I write this) and it heavily relies on this PR. Not sure how to actually release it: a) separate repo or b) ask you guys if you thing it could be included here as part of pyimgui. Sorry for for being off topic.

Yeah! Sounds awesome. I am not sure what to say concerning its release. In my opinion, 'PyImGui' should be kept as simple as the official 'DearImGui' and allow external extensions. The question sounds similar to the one of the docking branch in the official repo and how to deal with it.

Hello! I have great interest in this PR, notably for what @hinxx is working on (I'd love to have implot bindings). Do we have any updates on what's next? Are you all just waiting for @swistakm to give access to the repository to be able to merge that PR upstream, or does the PR still require some work?

Hello, glad to hear about your interest in this PR ! Unfortunately, I have been loaded with work for the end of the semester at my university, I had to do a pause to focus on other projects and I won't be able to continue this until next month. As for now, this PR is functioning but requires more testing to be used as a release. Many minor functionalities can be added (e.g. mapping of functions), but pyimgui have never been fully mapped thus what has already be done could be enough for now. As you mentioned @hinxx was working on an external binding of implot, to this respect we discussed how to share the cpp imgui lib to external modules here: https://github.com/KinoxKlark/pyimgui/pull/2 . I think this is temporarily in pause too but definitely a cool project ^^
For the next, we are waiting on @swistakm to be able to move forward, although there is no clear plan of what to do. There is also a new release of imgui (v1.83) with new functionalities to map but I don't know if we should do it here or start a new PR.

I'm pretty much in the same situation with regards to lack of time in the coming weeks.
I still do have interested in going forward with pyimplot and have something that could be tested and evaluated right now but needs a bit of coordination to make us all comfortable in using it.
I see myself devoting more time to this after summer vacation is over.

fwiw we're using the branch in a small internal app, and haven't had any problems with it

Hello, I will finally have more time to invest in this project!
I am not really sure about how we should continue here. In the absence of @swistakm I don't think there is much we can do concerning the official release. For now, I think the best we can do is to make this pr as robust as possible so that people interested in a stable branch with more up-to-date imgui functionalities could use it by manual installation.
I will need little time to go on stuff that I have missed the last few months. I would appreciate some insight about the following matters and how we should deal with them:

Should we merge here fixes from other pr ? (e.g. #232)
A new release of dear imgui is here, 1.83, what should we do about that? As it was mentioned earlier, it may be not a good idea to indefinitely chasse dear imgui releases. Maybe we should settle to 1.82 and stabilize pyimgui enough. Should we create another branch (and another pr?) for the implementation of new stuff and upgrade to the latest dear imgui?
@hinxx and I were discussing potential changes on the distribution of the cpp dearimgui with pyimgui such that it would be possible for an external module to share a unique implementation of dearimgui making it possible to create extensions (like implot). I did not wrap my head around it for a long time, where were we on that matter and how should we consider it here (if we should)?
I did not have time yet to investigate it deeply but it seems to be an interest for new functionalities like in #233, or questions about the core functioning of pyimgui like in #234 (from way before I came around here ^^). What should we do about that?

I may have forgotten some important stuff, feel free to add to that list.
My hope is that we would be able to maintain from this pr a good enough version to be used by anyone who needs it. And maybe one-day @swistakm would manage to merge it and drop a stable release quickly.
Thank's in advance, looking forward to your advice!

To me it feels like in order to keep this project alive and healthy we really need an official place to have it. Incremental changes to a PR, or several pending PRs adding all new functionality is not a good base for welcoming new members of a community or maintaining a stable code base.
Obviously what would be optimal is to have @swistakm back to help maintaining the project or to give admin rights to the ones that are the most involved in maintaining that project already. (I guess @KinoxKlark and @hinxx ? Not sure who I'm missing) This way everything could be stabilized properly, merged to master and then we could fix CI issues and publish to PyPI. It would also allow merging further fixes easily, maintain the documentation and so on
In the meantime, I guess what could be best is doing the same in a fork, like maybe yours @KinoxKlark?

I have been working on my own projects with this PR for a while now and I can vouch for it as well. I made UI focused app and other with OpenGL render interactions via SDL and didn't get into any troubles.
Only issue I found so far it is a little keyboard input issue. :)
(The shortcut CTRL+A -select all- in text input fields doesn't work. Other shortcut such as CTRL+Arrows -word jump-, Shift+Key -uppercase- or SHIFT+Arrows -character selection- work fine.

Hey all! I really apologize for not being responsive in this project. I hoped that once I will finish my book I will have some time, but it turns out I will  probably be eternally busy and unable to properly support such projects. The least I can do is to handverthe project to someone (or group of people). I may be able to support project with other resource but probably won't have time for active development anymore.
I know that we had this conversation before and we didn't decide on anything but now I think it is the only solution.
Probably that's simply not a thing that can be easily discussed asynchronously via GitHub issues. Best if we could meet online (e.g. via Zoom) and discuss this live.
@KinoxKlark, @TTimo, @hinxx: I was thinking mostly about three of you but anyone who want's to actively participate and has some spare time is free to join. If you are interested please drop me an email (address is in my Github). If you give me your time availability ( in UTC) I will setup a call where we can finally meet and discuss how to move forward. Best if we could meet over weekend because next week I leave my country for a couple of days.

Might I suggest to transfer the repo to an organization, where multiple people can join in on the fun?
or if you want to keep the repo under your name, make a few people able to add / remove participants on this repo.

@learn-more organization of course

Thanks all for your answers!
Nice to hear from you @swistakm. I'll drop you a mail to organize a meeting, it seems more appropriate indeed. I accept to take more responsibilities if it is needed to move forward. I agree with @learn-more, I think we need to make sure multiple people have access and can manage the project in the case where one of us can't be present.
@Froyok, thank you a lot for your feedback. I'll check on that issue as soon as possible ;)

Hey everyone,
You may have noticed that the project is now an organization one. Moreover, I now have admin right to the repo (and organization) thanks to @swistakm. I opened a discussion about the future and management of the project here: #237. I hope to gather your opinions there. We will be able to move forward with the project and prepare future releases!
There are still details to manage about the succession (like PyPi access for distribution). Those will be dealt with as we go along.
I will take some vacation in the coming week so I won't be always present for the development. I will still try to come regularly for the discussion. Feel free to comments in #237 and gives any insights/advice. Thanks in advance!


Add common number controls to lxp-bridge's homeassistant automation

Aside from AC charge on/off, this is the only command I've had to use on my inverter so far. Implementing it is a lot easier now you've done the heavy lifting in #127  :trollface:
I like to keep a "float" of 50% in the battery to power the EPS in case of a power cut. I had a bit of a time working out how to set it, but got there in the end. Having a nice slider show up automatically in HA is a win.
It has the same problem as the switches regarding initial state - the holding register values aren't published automatically like the input registers are, so this is "unknown" until you set it in HA. Maybe there's a better argument for reading those once at startup and publishing them as retained, though, since they only change with user action?
:thinking: I wonder if the inverter sends a message if a holding register is modified either remotely, or by going and fiddling with the physical controls built in to it. If not, that could lead to the HA view getting out of sync with reality, whether the message was retained or not.
Mandatory screenshots:

And a dump of the MQTT messages being sent:
lxp/cmd/XX12345678/set/discharge_cutoff_soc_limit_pct 50
lxp/XX12345678/hold/105 50
lxp/result/XX12345678/set/discharge_cutoff_soc_limit_pct OK

Once we get the basics of this nailed down, we should look at if there's any others we might as well add at the same time.
Just looking through doc/LXP_REGISTERS.txt I can see a few

AC_CHARGE_SOC_LIMIT
FORCED_CHG_SOC_LIMIT
FORCED_DISCHG_SOC_LIMIT
SOC_LOW_LIMIT_EPS_DISCHG
AC_CHARGE_START_BATTERY_SOC
AC_CHARGE_END_BATTERY_SOC

Though some of these won't have "aliased" MQTT messages so we'll need to use the register numbers for both the setting and the monitoring of values if that makes sense. that is to say, set/discharge_cutoff_soc_limit_pct is identical to set/hold/105.

Lemme see about adding those other controls now...

Just a fun note: one of the tests fails locally for me.
     Running tests/test_coordinator.rs (target/debug/deps/test_coordinator-71d31dd043af36e9)

running 3 tests
test publishes_read_hold_mqtt ... ok
test handles_read_input_all ... FAILED
test complete_path_read_hold_command ... ok

failures:

---- handles_read_input_all stdout ----
thread 'handles_read_input_all' panicked at 'assertion failed: `(left == right)`
  left: `Message(Message { topic: "2222222222/inputs/all", payload: "{\"status\":257,\"v_pv\":77.1,\"v_pv_1\":25.7,\"v_pv_2\":25.7,\"v_pv_3\":25.7,\"v_bat\":25.7,\"soc\":1,\"soh\":1,\"p_pv\":771,\"p_pv_1\":257,\"p_pv_2\":257,\"p_pv_3\":257,\"p_charge\":257,\"p_discharge\":257,\"v_ac_r\":25.7,\"v_ac_s\":25.7,\"v_ac_t\":25.7,\"f_ac\":2.57,\"p_inv\":257,\"p_rec\":257,\"pf\":0.257,\"v_eps_r\":25.7,\"v_eps_s\":25.7,\"v_eps_t\":25.7,\"f_eps\":2.57,\"p_eps\":257,\"s_eps\":257,\"p_to_grid\":257,\"p_to_user\":257,\"e_pv_day\":77.1,\"e_pv_day_1\":25.7,\"e_pv_day_2\":25.7,\"e_pv_day_3\":25.7,\"e_inv_day\":25.7,\"e_rec_day\":25.7,\"e_chg_day\":25.7,\"e_dischg_day\":25.7,\"e_eps_day\":25.7,\"e_to_grid_day\":25.7,\"e_to_user_day\":25.7,\"v_bus_1\":25.7,\"v_bus_2\":25.7,\"e_pv_all\":5052902.699999999,\"e_pv_all_1\":1684300.9,\"e_pv_all_2\":1684300.9,\"e_pv_all_3\":1684300.9,\"e_inv_all\":1684300.9,\"e_rec_all\":1684300.9,\"e_chg_all\":1684300.9,\"e_dischg_all\":1684300.9,\"e_eps_all\":1684300.9,\"e_to_grid_all\":1684300.9,\"e_to_user_all\":1684300.9,\"t_inner\":257,\"t_rad_1\":257,\"t_rad_2\":257,\"t_bat\":257,\"runtime\":16843009,\"max_chg_curr\":2.57,\"max_dischg_curr\":2.57,\"charge_volt_ref\":25.7,\"dischg_cut_volt\":25.7,\"bat_status_0\":257,\"bat_status_1\":257,\"bat_status_2\":257,\"bat_status_3\":257,\"bat_status_4\":257,\"bat_status_5\":257,\"bat_status_6\":257,\"bat_status_7\":257,\"bat_status_8\":257,\"bat_status_9\":257,\"bat_status_inv\":257,\"bat_count\":257,\"bat_capacity\":257,\"bat_current\":2.57,\"bms_event_1\":257,\"bms_event_2\":257,\"max_cell_voltage\":2.57,\"min_cell_voltage\":2.57,\"max_cell_temp\":2.57,\"min_cell_temp\":2.57,\"bms_fw_update_state\":257,\"cycle_count\":257,\"vbat_inv\":25.7,\"time\":1670583795,\"datalog\":\"2222222222\"}" })`,
 right: `Message(Message { topic: "2222222222/inputs/all", payload: "{\"status\":257,\"v_pv\":77.1,\"v_pv_1\":25.7,\"v_pv_2\":25.7,\"v_pv_3\":25.7,\"v_bat\":25.7,\"soc\":1,\"soh\":1,\"p_pv\":771,\"p_pv_1\":257,\"p_pv_2\":257,\"p_pv_3\":257,\"p_charge\":257,\"p_discharge\":257,\"v_ac_r\":25.7,\"v_ac_s\":25.7,\"v_ac_t\":25.7,\"f_ac\":2.57,\"p_inv\":257,\"p_rec\":257,\"pf\":0.257,\"v_eps_r\":25.7,\"v_eps_s\":25.7,\"v_eps_t\":25.7,\"f_eps\":2.57,\"p_eps\":257,\"s_eps\":257,\"p_to_grid\":257,\"p_to_user\":257,\"e_pv_day\":77.1,\"e_pv_day_1\":25.7,\"e_pv_day_2\":25.7,\"e_pv_day_3\":25.7,\"e_inv_day\":25.7,\"e_rec_day\":25.7,\"e_chg_day\":25.7,\"e_dischg_day\":25.7,\"e_eps_day\":25.7,\"e_to_grid_day\":25.7,\"e_to_user_day\":25.7,\"v_bus_1\":25.7,\"v_bus_2\":25.7,\"e_pv_all\":5052902.699999999,\"e_pv_all_1\":1684300.9,\"e_pv_all_2\":1684300.9,\"e_pv_all_3\":1684300.9,\"e_inv_all\":1684300.9,\"e_rec_all\":1684300.9,\"e_chg_all\":1684300.9,\"e_dischg_all\":1684300.9,\"e_eps_all\":1684300.9,\"e_to_grid_all\":1684300.9,\"e_to_user_all\":1684300.9,\"t_inner\":257,\"t_rad_1\":257,\"t_rad_2\":257,\"t_bat\":257,\"runtime\":16843009,\"max_chg_curr\":2.57,\"max_dischg_curr\":2.57,\"charge_volt_ref\":25.7,\"dischg_cut_volt\":25.7,\"bat_status_0\":257,\"bat_status_1\":257,\"bat_status_2\":257,\"bat_status_3\":257,\"bat_status_4\":257,\"bat_status_5\":257,\"bat_status_6\":257,\"bat_status_7\":257,\"bat_status_8\":257,\"bat_status_9\":257,\"bat_status_inv\":257,\"bat_count\":257,\"bat_capacity\":257,\"bat_current\":2.57,\"bms_event_1\":257,\"bms_event_2\":257,\"max_cell_voltage\":2.57,\"min_cell_voltage\":2.57,\"max_cell_temp\":2.57,\"min_cell_temp\":2.57,\"bms_fw_update_state\":257,\"cycle_count\":257,\"vbat_inv\":25.7,\"time\":1646370367,\"datalog\":\"2222222222\"}" })`', tests/test_coordinator.rs:87:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

failures:
    handles_read_input_all

test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass '--test test_coordinator'

The time differs. In left, it's<PHONE_NUMBER>; in right,<PHONE_NUMBER>.
:shrug:

Ah this is because there's a special incantation to run the tests ;)
I usually run them with
RUST_LOG=debug cargo test --features=mocks -- --nocapture

the features=mocks is the important part here and freezes the time to a constant used in the tests.

Rightyo. I couldn't see how to set the feature unconditionally for cargo test runs, so the default invocation Just Works, so I've settled for a commit that marks the tests that need the feature - they get ignored if it's not present. Just let me know if you'd rather not have this in this PR.
A basic Makefile so make runs cargo build and make test runs the special cargo test invocation would be another option... ;)

Rightyo. I couldn't see how to set the feature unconditionally for cargo test runs, so the default invocation Just Works, so I've settled for a commit that marks the tests that need the feature - they get ignored if it's not present. Just let me know if you'd rather not have this in this PR.

Yep, been there myself, couldn't find anything to set a feature for a cargo target. Ignoring some tests is fine, good enough for local runs and CI will get the rest.

 Untested as yet, I really need to get back to work :D. Let me know if you think I've got any of them wrong, we can always prune out ones we're uncertain of right now.

👍 Will have a good look through later but looks good at a glance. So I guess some of the names are a little arbitrary at this point, and they're only used for enabling/disabling in the config. Does it actually make sense to even bother with that config or should we just send all sensors and controls if HA is enabled? I only added it when originally adding some HA sensors in case it was useful, but probably its fine to send HA everything and let people worry about what they put in their own dashboards? I'm not a big HA user so not sure of best etiquette here.

I've just remembered that another control I needed to set was the AC charge times, but that's far too complicated for this PR, I think.

I did try looking up how to do this in a HA control but didn't find a likely looking integration. Agreed its worth splitting out though, be nice to get this in so it doesn't become too all-encompassing.

So I guess some of the names are a little arbitrary at this point, and they're only used for enabling/disabling in the config. Does it actually make sense to even bother with that config or should we just send all sensors and controls if HA is enabled?

Hmm. Home Assistant does allow you to disable / hide controls on their end:

So perhaps it's not very useful to be able to configure it at the lxp-bridge end as well.
I'll change my renaming commit to a deleting commit :thumbsup:

OK @celsworth , this is ready for another look. I've still not had occasion to try it out locally though. Maybe tomorrow :sweat_smile:

I think its all working. I can certainly set the numbers using HA now.
I'm not sure if they're all actually useful, for example my "Charge From AC Lower Limit %" and "Charge From AC Upper Limit %" are both 0, but my AC charging works fine. I think the rest of them seem ok.
I'll give this a day for you to test locally as well then we can probably merge tomorrow.

So I guess some of the names are a little arbitrary at this point, and they're only used for enabling/disabling in the config. Does it actually make sense to even bother with that config or should we just send all sensors and controls if HA is enabled?

Hmm. Home Assistant does allow you to disable / hide controls on their end:

So perhaps it's not very useful to be able to configure it at the lxp-bridge end as well.
I'll change my renaming commit to a deleting commit 👍

Sorry I'm a little slow on the comments today but I've been watching this thread and wondering when you'd get to the "what should be exposed in HASS" question. I was going to suggest that there may be some controls which are best exposed, but with the default sent through as "Disabled" for HA, so that power users can enable them if they chose, but novice users are protected from accidents?
I read on a forum someone who had accidentally turned off the entire grouping of controls on their inverter, and as a result caused themselves many problems particularly as their integration exposed all the inverter controls, not only the ones exposed to a "normal user" through the vendor's app.

@FirstRulez definitely something to keep in mind, thanks for mentioning it :thumbsup:
Do you think it applies to any of the controls here, or just something to keep in mind as we add others?

How do we even do that? I didn't spot anything in the HA documentation that would let us specify?

https://www.home-assistant.io/integrations/switch.mqtt/#enabled_by_default ?

@FirstRulez definitely something to keep in mind, thanks for mentioning it 👍
Do you think it applies to any of the controls here, or just something to keep in mind as we add others?

I think the ones you have looked at are safe; just a consideration for the future. My rule of thumb would be:
If it's available in the mobile app from Lux for a normal user (customer, not installer or distributor, etc.) then it's safe to enable by default in HASS.
But that is only my opinion.


Nixos update failed "numpy.distutils.system_info.NotFoundError: No lapack/blas resources found."

Describe the bug
Error during a system update
unpacking channels...
building Nix...
building the system configuration...
these derivations will be built:
 /nix/store/1wyn8ghsi55q5566gr5lhqjmyss17cds-tor-browser-linux64-9.0.9_en-US.tar.xz.drv
 /nix/store/72xgm313ishj9aqqaxrpywiln2s6g6n3-sublime_text_3_build_3211_x64.tar.bz2.drv
 /nix/store/rvdq6n7g0iix49a7iqf1iwap3ki6qzy6-sublimetext3-bin-3211.drv
 /nix/store/2lr9lxhxbrmjd3abdp2lxw35nh4arm70-sublimetext3-3211.drv
 /nix/store/bhldx72i3j93ba51ymrr9y82x9p3kbw2-source.drv
 /nix/store/3f8jzyffsv4xjil6xkp1g1rc35sf1rf2-xmind-8-update8.drv
 /nix/store/67ggczlhbnpk55d72bznizk7wrqnpjjw-teams_<IP_ADDRESS>53_amd64.deb.drv
 /nix/store/9q2sch7c3bz6z0aljd9hfqkw5dznr3c9-teams-<IP_ADDRESS>53.drv
 /nix/store/fxssfq5la12v6wb3zcw4ydd8nrk68vlf-vscode-1.44.1.drv
 /nix/store/vnfbfkpi66qp9067wlap4i0q36ng2y4v-python3.7-scipy-1.4.1.drv
 /nix/store/lxhlzfrhjp860gn0jr340nc0dyz7hcvc-python3.7-pandas-1.0.3.drv
 /nix/store/giay0vmsm4v3pcx73fg899vghgsvlqj8-python3-3.7.7-env.drv
 /nix/store/lbb2habvqrvknygsyv426k63lminjdzx-tor-browser-bundle-bin-9.0.9.drv
 /nix/store/vch2vi2sq7f8g2c0v8j1lw17ac46jrdg-zoom-us-3.5.385850.0413.drv
 /nix/store/360zsh5crkw6wrkmjwj9wa99b1pdnc7q-user-environment.drv
 /nix/store/y9bswb9jhsfwin284xv12ggsk6dlb58g-etc.drv
 /nix/store/g6f6vza0fb257452176y6g60i4fhvblz-nixos-system-p53-nixos-19.09.2407.b67bc34d4e3.drv
building '/nix/store/fxssfq5la12v6wb3zcw4ydd8nrk68vlf-vscode-1.44.1.drv'...
building '/nix/store/vch2vi2sq7f8g2c0v8j1lw17ac46jrdg-zoom-us-3.5.385850.0413.drv'...
building '/nix/store/vnfbfkpi66qp9067wlap4i0q36ng2y4v-python3.7-scipy-1.4.1.drv'...
building '/nix/store/1wyn8ghsi55q5566gr5lhqjmyss17cds-tor-browser-linux64-9.0.9_en-US.tar.xz.drv'...
building '/nix/store/bhldx72i3j93ba51ymrr9y82x9p3kbw2-source.drv'...
building '/nix/store/67ggczlhbnpk55d72bznizk7wrqnpjjw-teams_<IP_ADDRESS>53_amd64.deb.drv'...
building '/nix/store/72xgm313ishj9aqqaxrpywiln2s6g6n3-sublime_text_3_build_3211_x64.tar.bz2.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing setuptools-check-hook
unpacking sources
unpacking source archive /nix/store/jzrmjnhj56rdg13rqxylp3scxjjihr3v-scipy-1.4.1.tar.gz

trying https://dist.torproject.org/torbrowser/9.0.9/tor-browser-linux64-9.0.9_en-US.tar.xz
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed

trying https://xmind.net/xmind/downloads/xmind-8-update8-linux.zip

trying https://packages.microsoft.com/repos/ms-teams/pool/main/t/teams/teams_<IP_ADDRESS>53_amd64.deb
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
unpacking sources
unpacking source archive /nix/store/54hq9dl1g80igcy19s4vq7x90mjr9dkn-VSCode_1.44.1_linux-x64.tar.gz

trying https://download.sublimetext.com/sublime_text_3_build_3211_x64.tar.bz2
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
source root is scipy-1.4.1
setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file scipy-1.4.1/PKG-INFO
patching sources
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
Running from scipy source directory.
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
 libraries mkl_rt not found in ['/nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib']
 NOT AVAILABLE

openblas_lapack_info:
 libraries openblas not found in ['/nix/store/3gia15f4i46fvxxr6n4n3jlp8di2rxyw-blas-3/lib', '/nix/store/qrfg733cck2znlfp7bagrlqsrjxja9bm-lapack-3/lib']
 NOT AVAILABLE

openblas_clapack_info:
 libraries openblas,lapack not found in ['/nix/store/3gia15f4i46fvxxr6n4n3jlp8di2rxyw-blas-3/lib', '/nix/store/qrfg733cck2znlfp7bagrlqsrjxja9bm-lapack-3/lib']
 NOT AVAILABLE

flame_info:
 libraries flame not found in ['/nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib']
 NOT AVAILABLE

atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
 libraries lapack_atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
 libraries tatlas,tatlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
 NOT AVAILABLE

atlas_3_10_info:
 libraries lapack_atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
 libraries satlas,satlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
<class 'numpy.distutils.system_info.atlas_3_10_info'>
 NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
 libraries lapack_atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
 libraries ptf77blas,ptcblas,atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
<class 'numpy.distutils.system_info.atlas_threads_info'>
 NOT AVAILABLE

atlas_info:
 libraries lapack_atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
 libraries f77blas,cblas,atlas not found in /nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib
<class 'numpy.distutils.system_info.atlas_info'>
 NOT AVAILABLE

accelerate_info:
 NOT AVAILABLE

lapack_info:
 libraries lapack not found in ['/nix/store/lm0w36273v76nnph1518gwbv1g45sl8w-python3-3.7.7/lib']
 NOT AVAILABLE

/nix/store/qb19waw8994w0jm43rsxn43b244i8a0v-python3.7-numpy-1.18.1/lib/python3.7/site-packages/numpy/distutils/system_info.py:1730: UserWarning:
   Lapack (http://www.netlib.org/lapack/) libraries not found.
   Directories to search for the libraries can be specified in the
   numpy/distutils/site.cfg file (section [lapack]) or by setting
   the LAPACK environment variable.
 return getattr(self, '_calc_info_{}'.format(name))()
lapack_src_info:
 NOT AVAILABLE

/nix/store/qb19waw8994w0jm43rsxn43b244i8a0v-python3.7-numpy-1.18.1/lib/python3.7/site-packages/numpy/distutils/system_info.py:1730: UserWarning:
   Lapack (http://www.netlib.org/lapack/) sources not found.
   Directories to search for the sources can be specified in the
   numpy/distutils/site.cfg file (section [lapack_src]) or by setting
   the LAPACK_SRC environment variable.
 return getattr(self, '_calc_info_{}'.format(name))()
 NOT AVAILABLE

Traceback (most recent call last):
 File "nix_run_setup", line 8, in <module>
   exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))
 File "setup.py", line 540, in <module>
   setup_package()
 File "setup.py", line 536, in setup_package
   setup(**metadata)
 File "/nix/store/qb19waw8994w0jm43rsxn43b244i8a0v-python3.7-numpy-1.18.1/lib/python3.7/site-packages/numpy/distutils/core.py", line 137, in setup
   config = configuration()
 File "setup.py", line 435, in configuration
   raise NotFoundError(msg)
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
builder for '/nix/store/vnfbfkpi66qp9067wlap4i0q36ng2y4v-python3.7-scipy-1.4.1.drv' failed with exit code 1
cannot build derivation '/nix/store/giay0vmsm4v3pcx73fg899vghgsvlqj8-python3-3.7.7-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/360zsh5crkw6wrkmjwj9wa99b1pdnc7q-user-environment.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/y9bswb9jhsfwin284xv12ggsk6dlb58g-etc.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/g6f6vza0fb257452176y6g60i4fhvblz-nixos-system-p53-nixos-19.09.2407.b67bc34d4e3.drv': 1 dependencies couldn't be built
error: build of '/nix/store/g6f6vza0fb257452176y6g60i4fhvblz-nixos-system-p53-nixos-19.09.2407.b67bc34d4e3.drv' failed

To Reproduce
Steps to reproduce the behavior:

nix-channel --update && nixos-rebuild switch

Expected behavior
A system update to be completed
Additional context
Usual system update
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
these paths will be fetched (0.05 MiB download, 0.28 MiB unpacked):
  /nix/store/5h35s50503spks8gkxdv3kk4szz8vx9g-bash-interactive-4.4-p23-dev
copying path '/nix/store/5h35s50503spks8gkxdv3kk4szz8vx9g-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.32, NixOS, 19.09.2407.b67bc34d4e3 (Loris)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.3`
 - channels(root): `"home-manager-19.09, nixos-19.09.2426.9237a09d8ed, unstable-20.09pre222138.b3c3a0bd183"`
 - channels(mudrii): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Oh this is duplicated bug as the issue is already in discussion on https://github.com/NixOS/nixpkgs/issues/85637


Add functionality to start psc-ide-server?

Currently I'm using the same (shamelessly stolen) code from pscid which starts the server in an experimental branch of psc-pane. It could be great if we could share the code.

What you found in pscid is in turn a dumbed down version of ide-purescript code. I think we'll want to find the right level of sophistication and flexibility for all our three usecases.

I'm now also using node-which to find the executable, to address 2 issues

Could be .cmd or .exe on windows
Warn if there are multiple psc-ide-server versions on the path (and log versions etc)

I wonder if this would be relevant to others, maybe it's less easy to share as there is an npm dependency but at the least could extract the which bindings (I know pulp is using this too to find psa)

The user base for this library is probably going to stay very small and specialized so I'm not too worried about a bit of setup (node dep).
Sounds awesome I want it! :+1: Do you want to PR it or should I port it over? Where should I look in that case?

I may as well make a PR, I have a few thoughts & can make sure it still works for me... Will try and take a look tonight

Oh I did not know about purescript-ide-purescript-core. It also wraps the JSON output which I think could also be used in this project (and in psc-pane).
What about multi-project support? There was some talk about it somewhere, can't remember, but maybe one option would be to just try a range of ports when starting the server?

@anttih https://github.com/purescript/roadmap/issues/51 <- multiproject discussion

@kRITZCREEK thanks!
Ok, so the dependencies are the other way around: purescript-ide-purescript-core depends on this project. But I'm a little confused, what's the relationship between the two? Both have some similar JSON parsing code, for example.

purescript-ide-purescript-core is code that can be shared between the atom and the vs-code plugin. The reason it has error parsing code, is that it used the json errors from the compiler. With the very recent addition of rebuild for psc-ide we now also get json-errors in that context which is why the parsing functionality is duplicated between the two.
Might be worth unifying the error parsing

Just typed out exactly what @kRITZCREEK said... I'd be up for pushing things which are related to psc-ide and have wider use to here.

Fixed by #11


combining Runes produces inconsistent behavior

Bug Report
This document claims that Runes maintain consistency by retaining information about the sources of updates. However, despite reading the document multiple times, I still struggle to understand what exactly this "consistent behavior" entails. In fact, the more I delve into it, the more inconsistencies I discover when combining different Runes through tuple.
To bring clarity and benefit the entire community, I would greatly appreciate it if the CAPI team could provide a precise and technical explanation of what they mean by "consistent behavior." It is essential to understand the specific details of this behavior.
During my initial reading of the document, I formed an impression that the so-called "consistent behavior" could be described as follows:
When different Runes are combined through tuple and emit values during the same task-queue iteration, the resulting Rune produced by tuple will wait until the end of that task-queue iteration. It will then emit a tuple containing the latest value from each input Rune, consolidating the values produced thus far.
This behavior appears to have practical utility for many use cases. While I don't necessarily consider it a good default behavior, it does make some sense if this is the intended behavior of tuple.
However, upon closer examination, I find that the behavior exhibited in the first example aligns with the behavior I described. Yet, the other two examples deviate from this behavior. This inconsistency adds to the confusion and further complicates understanding.
In conclusion, I strongly believe that a precise and technical explanation of the "consistent behavior" exhibited by Runes, particularly in relation to combining Runes through tuple, would greatly benefit all users of CAPI.
Let's examine the examples provided. The first example aligns with the definition I previously proposed, which is promising:
import { Rune } from "capi"
import { delay } from<EMAIL_ADDRESS>
// Some kind of event source; this might correspond to, say, new blocks from a given chain
const a = timer(1000).map((n) => `a${n}`) // a0, a1, a2, ...

// Two different derived asynchronous queries based off of `a`
const ax = a.map((value) => delay(500).then(() => `${value}.x`)) // a0.x, a1.x, a2.x, ...
const ay = a.map((value) => delay(500).then(() => `${value}.y`)) // a0.y, a1.y, a2.y, ...

// Combine the results from each to process in some way
const az = Rune.tuple([ax, ay])

const start = Date.now()
for await (const value of az.iter()) {
  console.log(Date.now() - start, value)
}

// Every update is based on only one value from `a`; this is good.
/*
  503 [ "a0.x", "a0.y" ]
  1505 [ "a1.x", "a1.y" ]
  2507 [ "a2.x", "a2.y" ]
  3509 [ "a3.x", "a3.y" ]
  4512 [ "a4.x", "a4.y" ]
  ...
*/

However, the second example raises concerns. It appears to either be non-compliant with the defined behavior, or there may be a significant issue with the Rune's scheduler or the implementation of the timer function (although the latter seems more likely):
import { Rune } from "capi"
import { delay } from<EMAIL_ADDRESS>
const a = timer(1000).map((n) => `a${n}`) // a0, a1, a2, ...
const b = timer(1500).map((n) => `b${n}`) // b0, b1, b2, ...

// Combine them with `ls` like before:
const ab = Rune.tuple([a, b])

const start = Date.now()
for await (const value of ab.iter()) {
  console.log(Date.now() - start, value)
}

// This still has the correct behavior, without having to change the combinator:
/*
  2 [ "a0", "b0" ]
  1003 [ "a1", "b0" ]
  1506 [ "a1", "b1" ]
  2005 [ "a2", "b1" ]
  3007 [ "a3", "b1" ] // Why was this tuple emitted? It seems inconsistent with the behavior of the first example
  3007 [ "a3", "b2" ] // because this one should have been queued in the same macrotask :/
  4009 [ "a4", "b2" ]
  4509 [ "a4", "b3" ]
  5010 [ "a5", "b3" ]
  6011 [ "a5", "b4" ] // This value should have not been emitted according to the initial claim
  6012 [ "a6", "b4" ] // Only this one should have been emitted.
  7013 [ "a7", "b4" ]
  7514 [ "a7", "b5" ]
  8021 [ "a8", "b5" ]
  9020 [ "a8", "b6" ] // likewise
  9023 [ "a9", "b6" ] // Only this one should have been emitted.
  // ^ A 3 ms discrepancy after so few emitted values seems to indicate that there is some serious overhead going
  // on with the Runes, which will IMO not help at producing consistent behaviors.
*/

As for the third example, it presents its own set of puzzling aspects:
import { Rune } from "capi"
import { delay } from<EMAIL_ADDRESS>
const a = timer(1000).map((n) => `a${n}`) // a0, a1, a2, ...
const b = timer(1500).map((n) => `b${n}`) // b0, b1, b2, ...
const c = timer(2500).map((n) => `c${n}`) // c0, c1, c2, ...

const ab = Rune.tuple([a, b])
const bc = Rune.tuple([b, c])

// The combinator is the same as always
const allTogetherNow = Rune.tuple([ab, bc])

const start = Date.now()
for await (const value of allTogetherNow.iter()) {
  console.log(Date.now() - start, value)
}

// And everything works correctly; `a` is faster than `b` is faster than `c`,
// and `b`'s value is always consistent:
/*
  2 [ [ "a0", "b0" ], [ "b0", "c0" ] ]
  1004 [ [ "a1", "b0" ], [ "b0", "c0" ] ]
  1504 [ [ "a1", "b1" ], [ "b1", "c0" ] ]
  2005 [ [ "a2", "b1" ], [ "b1", "c0" ] ]
  2502 [ [ "a2", "b1" ], [ "b1", "c1" ] ]
  3005 [ [ "a2", "b2" ], [ "b2", "c1" ] ] // Why is this one here?
  3007 [ [ "a3", "b2" ], [ "b2", "c1" ] ] // Only this one should have appeared... Also why did this one come so late? What's going on?
  4008 [ [ "a4", "b2" ], [ "b2", "c1" ] ]
  4508 [ [ "a4", "b3" ], [ "b3", "c1" ] ]
  5005 [ [ "a4", "b3" ], [ "b3", "c2" ] ] // WHOOPS!
  5009 [ [ "a5", "b3" ], [ "b3", "c2" ] ]
  6009 [ [ "a5", "b4" ], [ "b4", "c2" ] ] // Another emitted tuple that shouldn't be here
  6011 [ [ "a6", "b4" ], [ "b4", "c2" ] ]
  7013 [ [ "a7", "b4" ], [ "b4", "c2" ] ]
  7508 [ [ "a7", "b4" ], [ "b4", "c3" ] ] // another one
  7510 [ [ "a7", "b5" ], [ "b5", "c3" ] ]
  8013 [ [ "a8", "b5" ], [ "b5", "c3" ] ]
  9013 [ [ "a8", "b6" ], [ "b6", "c3" ] ] // and another one
  9014 [ [ "a9", "b6" ], [ "b6", "c3" ] ]
*/

For what it's worth, I'm pretty sure that the "consistent behavior" defined earlier accurately captures the intention of the tuple Rune combinator. However, the lack of a proper scheduler in Runes, combined with a potentially naive implementation of the timer function, may be the root cause of these inconsistencies.
Interestingly, achieving this "consistent behavior" with RxJS is remarkably straightforward. I will delve into the details of this comparison in a separate discussion to provide a comprehensive explanation.

I'm merging all of the conversation into this discussion.

I'm merging all of the conversation into this discussion.

I believe it is crucial to distinguish the broader discussion about Runes from the specific bug mentioned here.
Based on the documentation, it appears that the observed behavior where Runes that were expected to update together are not actually updating together should be considered a bug. I think that this discrepancy between the promised behavior and the actual behavior highlights an issue that needs to be addressed and resolved.

As I cover in https://github.com/paritytech/capi/discussions/1145#discussioncomment-6388571, your description of Rune's intended behavior is not accurate. Rune is working as intended.


HAWKULAR-745 : Update angular-wizard so wz-disabled is supported

Also remove misleading patternfly resolution.

Merged #606


VirtualMachineError: revert: Arbitrage not profitable

Hi!
I'm having the following error:
Transaction sent: 0x1c1d48dec4082b1f46b5a7b88adf3fac9105229389df25cb9f521d36217c27c3
Gas price: 0.0 gwei   Gas limit: 12000000   Nonce: 6
SimpleArbitrage.makeArbitrage confirmed (Arbitrage not profitable)   Block: 14762600   Gas used: 44416 (0.37%)
File "brownie/_cli/run.py", line 50, in main
args[""], method_name=args[""] or "main", _include_frame=True
File "brownie/project/scripts.py", line 103, in run
return_value = f_locals[method_name](*args, **kwargs)
File "./scripts/arbitrage.py", line 37, in main
arbitrage_tx = arbitrage.makeArbitrage({"from": account})
File "brownie/network/contract.py", line 1693, in call
return self.transact(*args)
File "brownie/network/contract.py", line 1577, in transact
allow_revert=tx["allow_revert"],
File "brownie/network/account.py", line 680, in transfer
receipt._raise_if_reverted(exc)
File "brownie/network/transaction.py", line 421, in _raise_if_reverted
source=source, revert_msg=self._revert_msg, dev_revert_msg=self._dev_revert_msg
VirtualMachineError: revert: Arbitrage not profitable
Trace step -1, program counter 689:
File "contracts/SimpleArbitrage.sol", line 38, in SimpleArbitrage.withdraw:
}
    modifier onlyOwner() {
        require(msg.sender == owner, "only owner can call this");
        _;
    }

Terminating local RPC client...
The only thing that I haven't done, it's having WETH in my wallet, could be that?

No the contract reverts with "Arbitrage not profitable" which means that the current difference in price between the 2 exchanges is less than the transaction fee So the smart contract does not excute the arbitrage

Ah! Ok, need to look better at the code so I can put it running in a loop.
Good work you made here, thanks!

@Aymen1001 what percentage of the time will an arbitrage opportunity actually be available? seems like with eth on these two exchanges it will always revert with "Arbitrage not profitable"

Yes ETH/DAI is a known pair to everyone so any arbitrage opportunity will
not last long
I think to find good arbitrage you need to check unknown tokens pairs on
both exchanges or to build a bot that watches the hole market searching for
those opportunities
Le sam. 11 juin 2022 à 20:17, christianduncan @.***> a
écrit :

@Aymen1001 https://github.com/Aymen1001 what percentage of the time
will an arbitrage opportunity actually be available? seems like with eth on
these two exchanges it will always revert with "Arbitrage not profitable"
—
Reply to this email directly, view it on GitHub
https://github.com/Aymen1001/uniswap-sushiswap-arbitrage/issues/3#issuecomment-1152984586,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AT6N7NGSSBJB7TXCOKPWLJLVOTQ4VANCNFSM5VZCZMMA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Document FileReader is nil

If i send document to my bot, in backend document filereader is nil
func(c tele.Context) error {
		document := c.Message().Document
                ....
}

FileURL is empty
I use gopkg.in/tucnak/telebot.v3
Same problem with gopkg.in/tucnak/telebot.v2

Same problem

Hello. FileReader is used only to upload the data files that we have io.Reader from client to server. If you want to download a file, you can use the Download function
https://github.com/tucnak/telebot/blob/3ae6ec85d5509548fa7455b27750f015d6e984d2/bot.go#L1117


[enhancement] [UI]: Add ability to sort/autosort models in ModelUI

Is there an existing issue for this?

[X] I have searched the existing issues

Contact Details
No response
What should this feature add?
Add the ability to sort the Model View and auto-sort once a new model is being installed.
Currently, new installed models are just added to the end of a list without the ability to sort them.
The following sorting options seem to be most useful to me:
°Name
°Type
°Install date (is that possible?)
Maybe some others are useful too. Being able to mix sorting options would be great, but probably a bit complicated.
This would make model management and model search much more convenient.
Greetings
Alternatives
No response
Additional Content
No response

Agree. This is on my wishlist too. I'm mostly concerned about the model pulldowns in the loader node and linear views, where I'd prefer to see the models sorted first by base type (as they are) and then alphabetically by name using case-intensive sorting.


Geolocation Control

Hi
Is it possible to get this control added to the library?
https://github.com/Azure-Samples/AzureMapsCodeSamples/blob/master/AzureMapsCodeSamples/Device Sensors/Geolocation control.html
Looks a simple control with a js script to get the users location and show an icon on the map. Sample is here:
https://azuremapscodesamples.azurewebsites.net/Device Sensors/Geolocation control.html

Looks possible, as long as you include the js script to your application. This would be somehow similar to the overview or to the scale bar control. I'll try to take a look at it this week

That's what I'm guessing - as long as the js is added like the overview control hopefully this can be added as a new control - would be ideal for the app I'm building if could.

@philvessey This has been added on the v1.1.0.
You can instantiate a GeolocationControl class and add it to the map like any other controls on the AzureMap element or by using AddControlsAsync on your instance of Map. It supports the two GeolocationSuccess and GeolocationError events. The GetLastKnownPositionAsync method returns a Feature<Point> with the last known location. You can also modify the options of the control using SetOptionsAsync and dispose it using DisposeAsync. The control will not be usable anymore after it has been disposed and the methods will throw a ControlDisposedException if they are called after disposal. Once disposed, the control is not part anymore of the Controls collection on the Map.
There is also a new IGeolocationService which allows you to check if the browser supports the geolocation by calling IsGeolocationSupportedAsync. You can inject it directly on your razor pages.
As explained before, you need to include the script tag on your application to make it work. You can find it on this Git repository : https://github.com/Azure-Samples/azure-maps-geolocation-control
    <script src="~/azure-maps-geolocation-control.min.js"></script>

You can take a look at the documentation to find a working example https://github.com/arnaudleclerc/AzureMapsControl.Components/tree/develop/docs/controls/geolocation
I hope this fits. I will close this issue, but feel free to reopen it necessary 😊

Looks good from a quick glance over the sample - I like that you have added a browser support check too.


feat: Improve Performance of  delete Command with Concurrency

Fixes: #79
Description:
This PR enhances the performance of the harbor project delete command by introducing concurrency to handle multiple project deletions in parallel. The changes ensure that the command is more efficient, especially when dealing with multiple projects.
Changes:

Concurrency Implementation:
Added goroutines to execute runDeleteProject concurrently for each project specified in the command arguments.

Error Handling:

Implemented an error channel (errChan) to collect errors from each goroutine.
Ensured all errors are logged appropriately after all goroutines have completed.

Benefits:

Performance: By running deletions concurrently, the command execution time is significantly reduced when dealing with multiple projects.
Reliability: Enhanced error handling ensures that all errors are captured and logged, providing better visibility into potential issues.

Testing:

[x] Tested the command without args and with multiple project names to ensure concurrent deletions work as expected.

Example Usage:

# Delete multiple projects concurrently
harbor project delete project1 project2 project3

# Delete a single project by prompting the user
harbor project delete

Before:

The delete commands not have multiple deletion at a time.

After:

The delete commands deletes multiple resources at a time.

Could you add multiple selection in the TUI.

Currently working on TUI multiple selection.
Will squash the commits.

I am planning to add the multiple TUI selection as a base component in separate PR.
which can be used by multiple commands that support multiple selection and their associated action
with that we can mimic the behavior of multiple select and action in TUI.

This PR is ready for review @Vad1mo

ready to merge.