Libretro 'Controller Info / Input Info' Updating

Digging through the input handler. I see where the callbacks happen from a core (such as pcsx-rearmed) and descriptors are sent over which will descript the possible controller info (such as standard, dualshock, negcon, guncon, mouse, (and an empty string for some reason? "")). But these then get stored in to private variables such as _controllerDescriptions  which never get used again, nor there is any way to access it. The same goes for the input mapping with _inputDescriptors and _buttonDescriptions which will describe all the input buttons.
I see there is a variable loaded with ControllersMap     = wrapper.InputHandler.DeviceMap; within the bridge, but this just empty. Never set up.
What I'm trying to figure out here is how to remap buttons and select the device type (dualshock, standard, etc...) for a specific port within the API, but it isn't clear how that is to be done here. Is there something I'm missing with this?

Still trying to actively investigate myself....but just making a note here of a bug I found with recording the Controller Info with this line here: https://github.com/Skurdt/SK.Libretro/blob/cf14f8392a40119542a9e0d66a89d096ec4aa4db/Scripts/Input/InputHandler.cs#L302
As controllerDescription is a stack variable, this added by reference in to the _controllerDescriptions List.... but as soon as the next loop comes, that stack variable is updated with a new value and therefore that first value that was just added to the List is then updated and the previous data is lost.
A fix (or workaound) I did here was to just change it to this
_controllerDescriptions.Add(new retro_controller_description
{
    id = controllerDescription.id,
    desc = controllerDescription.desc,
});

To tell a core what id to use (at least for pcsx), it's on the bridge class using SetControllerPortDevice(port, device).
I had a working UI with names like Lightgun/DualShock/etc... but it looks like the refactoring wasn't finished...
You should be able to access the id to name mapping by making the _controllerDescriptions/_controllersInfo variables public and public properties on the bridge to access them. They should not be empty if the core calls the SetControllerInfo callback.
I'm not sure what you mean by stack variable being updated.
This is a local variable but it's both a class and is created using Marshal.PtrToStructure so it's obviously not on the stack. And even then, when the loop comes around it created a new variable altogether.
I never had any issues creating a new instance of a class and add it to a list, but maybe there's something else going on?

To tell a core what id to use (at least for pcsx), it's on the bridge class using SetControllerPortDevice(port, device).

Yes, I found that, but it isn't quite clear how many ports there are to be from the one of the Core to Frontend callbacks. Also device goes to an enum RETRO_DEVICE, but that id to be used with the device parameter is to come from the info within the retro_controller_description class which is contained within that private variable _controllerInfo within the internal sealed class InputHandler.
For example, attaching the debugging and stepping through what pcsx-reamed sends is below with 7 num_types. If I were to make an interface with a dropdown and items to the const char* (string) within that class to callback SetControllerPortDevice. It would pass that uint to be as that enum RETRO_DEVICE, but all those uint values it gives appear to be way out side of those enum values except for the standard controller which is just 1 (I'm also assuming the null (0x0) it gives to just be a "None" controller).
“standard” – 1
“analog” – 261
“dualshock” – 517
“negcon” – 773
“guncon” – 260
“mouse” – 258
“” – 0 (NULL, NULL)
For a frontend to know what are the possible types of controllers it would need some way to access these descriptor strings and their corresponding id with it. I may need to look into modifying the code myself for how to expose this info.... but it isn't quite clear what the max number of port is to be to from this information.

I had a working UI with names like Lightgun/DualShock/etc... but it looks like the refactoring wasn't finished... You should be able to access the id to name mapping by making the _controllerDescriptions/_controllersInfo variables public and public properties on the bridge to access them. They should not be empty if the core calls the SetControllerInfo callback.

I wasn't referring to _controllerDescriptions/_controllersInfo being empty. Those actually have valid data in them once that bug is fixed. It's just that ControllersMap from the Instances is empty and appears to be unfinished I'm assuming as it appears to be unused. ControllersMap is linked below
https://github.com/Skurdt/SK.Libretro/blob/cf14f8392a40119542a9e0d66a89d096ec4aa4db/Unity/Scripts/Runtime/LibretroInstance.cs#L47

I'm not sure what you mean by stack variable being updated. This is a local variable but it's both a class and is created using Marshal.PtrToStructure so it's obviously not on the stack. And even then, when the loop comes around it creates a new variable altogether. I never had any issues creating a new instance of a class and add it to a list, but maybe there's something else going on?

I may be wrong there... I'm a C/C++ engineer by trade (not really a C# guru here), but a local variable is just something I'd assume to be on a stack. I don't see it creating a new variable altogether but rather reusing the same variable... Anyways... something is already off

Remapping buttons is done on the frontend side, if using the input system, then add/replace the mappings there in Unity. The LibretroUnityFE repo has some starting point on how to do that but that's not fully implemented (it also has the descriptors for input but not in the latest commit...)

I see... so just _inputDescriptors is used for informational purposes only?

It's impossible to know the number of ports, everything is callback based. A core asks every frame the state of all buttons for all ports it wants, this library only act as a bridge to gather everything to send it all back.
Basically, you can join 20 players on the unity side and it will store all the states for all 20 players for 1 frame.
If the core only asks for 2 players (like port 0 and 1 or whatever integer it uses... some core a very weird), then only those 2 sets of data are used by it (for the pcsx_rearmed core, it processes 8 players. Or at least it defines 8 controllers, maybe per game the request varies... No idea to be honest).
The “” – 0 (NULL, NULL) is because they don't send the array length along with it. It's basically how they send a variable length array in C, the receiver advances on the pointer until there's a null entry.
Those uints are not necessarily wrong, the values in the base enum are base input devices, libretro uses a subclass system where one value can shift to whatever the core wants to do.
libretro.h defines these:
#define RETRO_DEVICE_TYPE_SHIFT         8
#define RETRO_DEVICE_MASK               ((1 << RETRO_DEVICE_TYPE_SHIFT) - 1)
#define RETRO_DEVICE_SUBCLASS(base, id) (((id + 1) << RETRO_DEVICE_TYPE_SHIFT) | base)

pcsx_rearmed defines these:
#define RETRO_DEVICE_PSE_STANDARD   RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_JOYPAD,   0)
#define RETRO_DEVICE_PSE_ANALOG     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   0)
#define RETRO_DEVICE_PSE_DUALSHOCK  RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   1)
#define RETRO_DEVICE_PSE_NEGCON     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   2)
#define RETRO_DEVICE_PSE_GUNCON     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_LIGHTGUN, 0)
#define RETRO_DEVICE_PSE_MOUSE      RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_MOUSE,    0)

In any case, the last null value should be ignored and you probably shouldn't send that as the device.
I pushed a commit that uses the device map, though I haven't tried accessing it yet in the UI. Do with that what you can...

It's impossible to know the number of ports, everything is callback based. A core asks every frame the state of all buttons for all ports it wants, this library only act as a bridge to gather everything to send it all back.

I think I just figured this out, by just reading the documentation of retro arch (aka the source code). This can be just a simple counter within that same function as here
https://github.com/libretro/RetroArch/blob/903f540c057d3c29321fa678c81e26be56f65f3c/runloop.c#L2823C14-L2823C14

Basically, you can join 20 players on the unity side and it will store all the states for all 20 players for 1 frame. If the core only asks for 2 players (like port 0 and 1 or whatever integer it uses... some core a very weird), then only those 2 sets of data are used by it (for the pcsx_rearmed core, it processes 8 players. Or at least it defines 8 controllers, maybe per game the request varies... No idea to be honest). Similarly, if you only join 1 player and the cores asks for more, the wrapper will send all 0 for all other ports.
The “” – 0 (NULL, NULL) is because they don't send the array length along with it. It's basically how they send a variable length array in C, the receiver advances on the pointer until there's a null entry.

Yeah, that make sense

Those uints are not necessarily wrong, the values in the base enum are base input devices, libretro uses a subclass system where one value can shift to whatever the core wants to do.
libretro.h defines these:
#define RETRO_DEVICE_TYPE_SHIFT         8
#define RETRO_DEVICE_MASK               ((1 << RETRO_DEVICE_TYPE_SHIFT) - 1)
#define RETRO_DEVICE_SUBCLASS(base, id) (((id + 1) << RETRO_DEVICE_TYPE_SHIFT) | base)

pcsx_rearmed defines these:
#define RETRO_DEVICE_PSE_STANDARD   RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_JOYPAD,   0)
#define RETRO_DEVICE_PSE_ANALOG     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   0)
#define RETRO_DEVICE_PSE_DUALSHOCK  RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   1)
#define RETRO_DEVICE_PSE_NEGCON     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_ANALOG,   2)
#define RETRO_DEVICE_PSE_GUNCON     RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_LIGHTGUN, 0)
#define RETRO_DEVICE_PSE_MOUSE      RETRO_DEVICE_SUBCLASS(RETRO_DEVICE_MOUSE,    0)

In any case, the last null value should be ignored and you probably shouldn't send that as the device.

I see... So I was misunderstanding in how it works. Then it appears in the "NONE" case, it would just be an id of 0.

I pushed a commit that uses the device map, though I haven't tried accessing it yet in the UI. Do with that what you can...

Thanks!! I took a quick look. I'll have to try it out. There is also the _buttonDescriptors that would need to be exposed... but I don't think that's quick as high of priority for what I'm about to get into.
Also, sorry I'm doing PRs here, it's rather complicated how much I've forked the repo since then with getting Android support (Quest 2) on Unity (but given the news which I'm sure you've seen that came out, who knows if it's more interesting getting this library working on something like maybe Godot) (I'll get around to publishing what I've already done soon here).... but another issue I found was with the analog to digital code here.
https://github.com/Skurdt/SK.Libretro/blob/e55306956a3b13a3533d74ad795c1e5328b9b6d4/Unity/Scripts/Runtime/Input/PlayerInputProcessor.cs#L176
Essentially, what was going on was that with an analog stick, a user could hold the stick in the upper left corner and then roll around to the bottom right never returning to the 0,0 position which will enter the cancel function clearing every bit. What would happen is those bits would be stuck on for 'up' and 'left' because the SetBitIf function can only set bits, it will never clear them. I also found that the left and right were swapped. This was just a simple change to the below (it may be an interesting single enhancement to make that analog2digital flag an enum to select between NONE, LEFT_ANALOG, or RIGHT_ANALOG  rather than only having the option for the left)
_joypadButtons.SetBit((uint)RETRO_DEVICE_ID_JOYPAD.UP, AnalogLeftY > 0);
_joypadButtons.SetBit((uint)RETRO_DEVICE_ID_JOYPAD.DOWN, AnalogLeftY < 0);
_joypadButtons.SetBit((uint)RETRO_DEVICE_ID_JOYPAD.LEFT, AnalogLeftX < 0);
_joypadButtons.SetBit((uint)RETRO_DEVICE_ID_JOYPAD.RIGHT, AnalogLeftX > 0);

I see... so just _inputDescriptors is used for informational purposes only?

Most of the stuff stored on the library side that comes from the core are to interpret and use on the fe side (UI, all sorts of different IDs, etc...). You can either send it back (like for input data), show as visual elements to a user or simply ignore.

Yeah.... that's what it appears like to me.

I see... So I was misunderstanding in how it works. Then it appears in the "NONE" case, it would just be an id of 0.

Like I said, I don't think you should set it to none, it's only used to tell the end of the array. In the commit I pushed, I skipped it and it's not added to the collection. The issue is that a core may or may not handle a 0... I prefer not tempting luck in this project, it rarely worked to do so so far :p
Most gaming systems don't care how many controllers are physically plugged in, adding a new player is basically done by pressing a menu that selects multiplayer, press a button on a different device or add a coin in a different coin slot (arcade).
I also said that you can ignore data and not store it/send it back but in a lot of cases, I have to... All of the attempts to implement this all thing, publicly available when I started this, either ignored or didn't implement a lot of the environment callbacks. It made a lot of cores stall or straight up crash (mostly crashes :p).
I don't know about the AnalogToDigital stuff, it was mostly untested, quite niche and the name is kinda wrong... It's here because mame2003_plus (maybe others), when set to digital input (in the core options) was doing some weird mappings and didn't allow the sticks to be processed as buttons for whatever reason.
For the right stick, it didn't make sense to implement the workaround, digital input never(?) include a right stick (aka. a right side DPad).
The input system for unity did change also, and the entire performed/cancelled/started thing didn't behave the same for me depending on the versions... In more recent projects, I use the interface based way instead of registering to these events manually.
I don't really care about the Unity news, I don't make any money with it.
But if I were to switch engine, I wouldn't go godot but rather unreal, adding libretro support in C++ should be less buggy as well (hope...).

 That did allow me to read the descriptors and id for my UI.
https://youtu.be/nDFg0RJCp3w?si=J4tIBiCkfkAUNSx2&t=195


Add Invasive Weed Data [ 4 hours ]

Nick has contact info with the person who has the data. They've been unresponsive as of yet. We have a couple more weeks to prod them for the data.
If we get it with time to cut tiles and include data in the reports, great. If not, then this will have to be scrapped.

The weed data was not comprehensive or useful. Must be scrapped for now :(


Cluster node tries to join itself after upgrade to 1.1.0

VerneMQ Version: 1.1.0
OS: Linux, Debian, Kubernetes-1.6

After upgrading from 1.0.1 to 1.1.0, I'm running into a problem which looks the same as this:
https://github.com/erlio/vmq_server/issues/92
Running the same way using 1.0.1, I don't have this problem. I've tried starting the cluster from scratch on both versions.
This is running on Kubernetes and using the technique of getting the pod ip addresses from here (important part of script pasted below): https://github.com/georgepoenaru/vernemqube/tree/master/vernemq-docker
# Cluster discovery implementation based on https://github.com/thesandlord/kubernetes-pod-ip-finder
if env | grep -q "KUBE_VERNEMQ_DISCOVERY_URL"; then
    response=$(curl ${KUBE_VERNEMQ_DISCOVERY_URL})
    IFS=','
    nodes=($(echo "$response" | tr -d '[]"'))
    length=$(echo ${#nodes[@]})
    for i in "${nodes[@]}"
    do
      if [ "$i" != "null" ] && [ "$i" != "$IP_ADDRESS" ] && (($length > 1)); then
        echo "Start Joining to<EMAIL_ADDRESS>        echo "-eval \"vmq_server_cmd:node_join('VerneMQ@${i}')\"" >> /etc/vernemq/vm.args
      fi
    done
    IFS=''
fi

Partial log:
2017-07-03 15:32:27.695 [info] <0.31.0> Application vmq_server started on node<EMAIL_ADDRESS>2017-07-03 15:32:27.739 [info] <0.3.0>@plumtree_peer_service:attempt_join:50 Sent join request to<EMAIL_ADDRESS>2017-07-03 15:32:27.739 [debug] <0.454.0> Supervisor inet_gethost_native_sup started undefined at pid <0.455.0>
2017-07-03 15:32:27.739 [debug] <0.59.0> Supervisor kernel_safe_sup started inet_gethost_native:start_link() at pid <0.454.0>
2017-07-03 15:32:27.864 [info] <0.327.0>@vmq_cluster_mon:handle_info:126 cluster node<EMAIL_ADDRESS>UP
2017-07-03 15:32:27.869 [info] <0.211.0>@plumtree_peer_service_manager:write_state_to_disk:100 writing state {[{[{actor,<<120,41,107,123,55,162,41,75,44,177,100,19,74,146,213,142,159,164,4,44>>}],1},{[{actor,<<229,191,137,216,143,132,213,217,78,172,87,132,203,46,72,105,194,144,188,61>>}],1}],{dict,2,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[['VerneMQ@<IP_ADDRESS>',{[{actor,<<229,191,137,216,143,132,213,217,78,172,87,132,203,46,72,105,194,144,188,61>>}],1}]],[],[],[],[['VerneMQ@<IP_ADDRESS>',{[{actor,<<120,41,107,123,55,162,41,75,44,177,100,19,74,146,213,142,159,164,4,44>>}],1}]],[],[],[],[],[],[],[]}}},{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}} to disk <<75,2,131,80,0,0,1,122,120,1,203,96,206,97,96,96,96,202,96,2,81,140,25,76,41,12,172,137,201,37,249,69,185,64,174,72,133,102,118,181,249,34,77,111,157,141,41,194,94,147,174,246,205,95,194,162,147,149,8,84,133,77,241,211,253,157,55,250,91,174,222,244,91,19,222,114,90,207,35,243,208,132,61,182,64,197,89,25,156,41,12,44,41,153,201,37,137,76,137,2,64,200,145,24,144,104,144,33,144,133,6,50,24,33,98,96,163,193,110,74,97,16,11,75,45,202,75,245,13,116,48,52,48,208,179,176,212,51,50,49,215,51,50,50,33,205,126,144,61,56,13,53,55,209,51,180,52,211,51,52,180,192,110,40,142,16,128,187,29,225,57,6,98,60,7,215,7,100,0,0,210,156,115,29>>
2017-07-03 15:32:27.892 [debug] <0.326.0> Supervisor vmq_cluster_node_sup started<EMAIL_ADDRESS>at pid <0.463.0>
2017-07-03 15:32:27.938 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:27.951 [debug] <0.466.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:27.951 [warning] <0.466.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:27.952 [debug] <0.346.0> Supervisor vmq_diversity_script_sup started vmq_diversity_script_sup_sup:start_link("/etc/vernemq-cfg/gw_auth_hook.lua") at pid <0.353.0>
2017-07-03 15:32:28.953 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:28.953 [debug] <0.472.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:28.954 [warning] <0.472.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:29.955 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:29.956 [debug] <0.475.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:29.956 [warning] <0.475.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:30.957 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:30.958 [debug] <0.477.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:30.958 [warning] <0.477.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:31.959 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:31.960 [debug] <0.480.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:31.960 [warning] <0.480.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:32.962 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:32.965 [debug] <0.481.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:32.965 [warning] <0.481.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available
2017-07-03 15:32:33.966 [info] <0.463.0>@vmq_cluster_node:connect:207 successfully connected to cluster node<EMAIL_ADDRESS>2017-07-03 15:32:33.967 [debug] <0.484.0>@vmq_cluster_com:process_bytes:135 connect request from unknown cluster node<EMAIL_ADDRESS>2017-07-03 15:32:33.967 [warning] <0.484.0>@vmq_cluster_com:loop:74 terminate due to remote_node_not_available

I can recreate this by starting up 2 disconnected brokers, and then run vmq-admin cluster join discovery-node=VerneMQ@<other broker's IP Address>. Both brokers report the warnings.

@kellycampbell thank you for reporting!
First, not for you, but for later reference and readers: be sure to only call vmq-admin cluster join on one of the nodes, that is enough to build a cluster.
Second, I don't know much about Kubernetes (actually I'm totally scared by Kubernetes... :)
And third, is this with a totally fresh state? (does the problem occur with old or with fresh metadata dirs?)

In the case of a two pod cluster, only the second pod is calling vmq-admin cluster join.
By fresh state, I mean no metadata directories. It's basically starting a new docker container with no persistent volumes. I will try to reproduce using only docker in order to give you a better test case.

I couldn't reproduce with the stock Erlio docker image, so I compared to my customized docker image and found that I had the listeners set to use <IP_ADDRESS>. I replaced those to match the erlio docker image (${IP_ADDRESS} and it fixed this issue.
echo "listener.tcp.default = ${IP_ADDRESS}:1883" >> /etc/vernemq/vernemq.conf
echo "listener.ws.default = ${IP_ADDRESS}:8080" >> /etc/vernemq/vernemq.conf
echo "listener.vmq.clustering = ${IP_ADDRESS}:44053" >> /etc/vernemq/vernemq.conf
echo "listener.http.metrics = ${IP_ADDRESS}:8888" >> /etc/vernemq/vernemq.conf
The reason I had changed these was for kubectl port-forward because it doesn't work with localhost only listeners for some reason. Closing this report out.

@kellycampbell OK!
There were always some troubles with Containers, AWS etc and <IP_ADDRESS>.
But it didn't actually come to my mind here. Thank you so much for this effort!


Geolocator position updates stopped

I am using this package to get location update stream.
Everything is good when app is active.
I am using this pacakge with flutter_background_service to send location update event even app is closed.
  @pragma('vm:entry-point')
    void onStart(ServiceInstance service) async {
      await PreferencesManager.init();
      const LocationSettings locationSettings = LocationSettings(
        accuracy: LocationAccuracy.high,
        distanceFilter: 10,
      );
      StreamSubscription<Position> positionStream = Geolocator.getPositionStream(locationSettings: locationSettings).listen(
          (Position? position) {
            print("Location Tracking");
            LocationApi locationApi = LocationApi();
            locationApi.sendLocation(position?.latitude, position?.longitude);
            print(position == null ? 'Unknown' : '${position.latitude.toString()}, ${position.longitude.toString()}');
      });
      DartPluginRegistrant.ensureInitialized();

      if (service is AndroidServiceInstance) {
        service.on('setAsForeground').listen((event) {
          service.setAsForegroundService();
        });

        service.on('setAsBackground').listen((event) {
          service.setAsBackgroundService();
        });
      }

      service.on('stopService').listen((event) {
        service.stopSelf();
      });
    }

But when I close app, I get this log.
FlutterGeolocator       com.example.flutter_background_location_service                 D  Detaching Geolocator from activity
FlutterGeolocator       com.example.flutter_background_location_service                 D  Flutter engine disconnected. Connected engine count 2
FlutterGeolocator       com.example.flutter_background_location_service                 D  Disposing Geolocator services
FlutterGeolocator       com.example.flutter_background_location_service                 E  Geolocator position updates stopped
FlutterGeolocator       com.example.flutter_background_location_service                 E  There is still another flutter engine connected, not stopping location service

Any solution for this?

I (more or less) have the same issue, although I'm not using flutter_background_service but flutter_foreground_task.. When starting up the app (and checking for the required permissions using the methods provided by the library) I see this:
Attaching Geolocator to activity
Creating service.
Binding to location service.
Geolocator foreground service connected
Initializing Geolocator services
Flutter engine connected. Connected engine count 1

Then, after the new isolate for the foreground service is started in which the app listens for position updates this is shown:
Geolocator foreground service connected
Initializing Geolocator services
Flutter engine connected. Connected engine count 2
Geolocator position updates started

So far, so good. At this point I'm receiving position updates as expected. But then, when I close the app I stop receiving those updates:
Detaching Geolocator from activity
Flutter engine disconnected. Connected engine count 1
Disposing Geolocator services
Geolocator position updates stopped
There is still another flutter engine connected, not stopping location service

By looking at the logs it seems the reference counting works fine, but despite that Geolocator stops sending updates.

In order to help us investigating the issue, could you please post the output of flutter doctor -v and I suspect that you found this issue while testing on Android, is that correct?

Ah, yes, the issue is on Android. Here is the output of flutter doctor -v:
[✓] Flutter (Channel stable, 3.10.5, on macOS 13.4 22F66 darwin-arm64, locale en-GB)
• Flutter version 3.10.5 on channel stable at /Users/ferryspiering/Library/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 796c8ef792 (8 days ago), 2023-06-13 15:51:02 -0700
• Engine revision 45f6e00911
• Dart version 3.0.5
• DevTools version 2.23.1
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
• Android SDK at /Users/ferryspiering/Library/Android/sdk
• Platform android-33, build-tools 31.0.0
• ANDROID_HOME = /Users/ferryspiering/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 14E222b
• CocoaPods version 1.11.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2022.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
[✓] IntelliJ IDEA Community Edition (version 2021.3.2)
• IntelliJ at /Applications/IntelliJ IDEA CE.app
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
[✓] VS Code (version 1.79.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.12.2
[✓] Connected device (2 available)
• macOS (desktop) • macos  • darwin-arm64   • macOS 13.4 22F66 darwin-arm64
• Chrome (web)    • chrome • web-javascript • Google Chrome 114.0.5735.133
[✓] Network resources
• All expected network resources are available.
• No issues found!

same issues :(

Use TaskQueues.
2023-09-26 01:07:16.607 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              D  Geolocator foreground service connected
2023-09-26 01:07:16.607 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              D  Initializing Geolocator services
2023-09-26 01:07:16.607 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              D  Flutter engine connected. Connected engine count 2
2023-09-26 01:07:16.607 12319-12468 FlutterJNI              io.hexasoft.ceert_audit              W  FlutterJNI.loadLibrary called more than once
2023-09-26 01:07:16.608 12319-12359 ResourceExtractor       io.hexasoft.ceert_audit              I  Found extracted resources res_timestamp-22-1695672416033
2023-09-26 01:07:16.608 12319-12358 FlutterJNI              io.hexasoft.ceert_audit              W  FlutterJNI.prefetchDefaultFontManager called more than once
2023-09-26 01:07:16.610 12319-12319 FlutterJNI              io.hexasoft.ceert_audit              W  FlutterJNI.init called more than once
2023-09-26 01:07:18.138 12319-12481 flutter                 io.hexasoft.ceert_audit              I  task: backgroundUploadTask
2023-09-26 01:07:18.139 12319-12481 flutter                 io.hexasoft.ceert_audit              I  inputData: {backgroundTask: one off}
2023-09-26 01:07:18.142 12319-12481 flutter                 io.hexasoft.ceert_audit              I  Scheduled task executed!
2023-09-26 01:07:22.144 12319-12319 FileSaver               io.hexasoft.ceert_audit              D  Detached From Engine
2023-09-26 01:07:22.145 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              D  Flutter engine disconnected. Connected engine count 1
2023-09-26 01:07:22.150 12319-12406 WM-WorkerWrapper        io.hexasoft.ceert_audit              I  Worker result SUCCESS for Work [ id=5a80a0e9-9a16-47db-bf76-ceea1ca95461, tags={ be.tramckrijte.workmanager.BackgroundWorker } ]
2023-09-26 01:07:22.153 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              D  Disposing Geolocator services
2023-09-26 01:07:22.153 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              E  Geolocator position updates stopped
2023-09-26 01:07:22.153 12319-12319 FlutterGeolocator       io.hexasoft.ceert_audit              E  There is still another flutter engine connected, not stopping location service

[✓] Flutter (Channel stable, 3.10.6, on macOS 13.5.2 22G91 darwin-x64, locale en-PK)
• Flutter version 3.10.6 on channel stable at /Users/muhammad/flutter3.10.6
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision f468f3366c (3 months ago), 2023-07-12 15:19:05 -0700
• Engine revision cdbeda788a
• Dart version 3.0.6
• DevTools version 2.23.1

This happened with me as well when i switched off the internet

Also running into this now. Seems to be a regression that was introduced at some point. I originally made the PR to do the reference counting and will have a look at this and fix it again. It might be that someone tried to address this issue https://github.com/Baseflow/flutter-geolocator/issues/1395 so we might need a new flag to do reference counting on Android. In my opinion though people should be closing the stream manually when their application is stopped or moved to the background. Any thoughts on how to approach this @mvanbeusekom ?

Manually closing the stream when needed sounds like a great option.

Also running into this now. Seems to be a regression that was introduced at some point. I originally made the PR to do the reference counting and will have a look at this and fix it again. It might be that someone tried to address this issue #1395 so we might need a new flag to do reference counting on Android. In my opinion though people should be closing the stream manually when their application is stopped or moved to the background. Any thoughts on how to approach this @mvanbeusekom ?

@Wackymax Did you already find some time to have a look at this?

Not yet noOn 10 Feb 2024, at 17:21, Ferry @.***> wrote:
Also running into this now. Seems to be a regression that was introduced at some point. I originally made the PR to do the reference counting and will have a look at this and fix it again. It might be that someone tried to address this issue #1395 so we might need a new flag to do reference counting on Android. In my opinion though people should be closing the stream manually when their application is stopped or moved to the background. Any thoughts on how to approach this @mvanbeusekom ?
@Wackymax Did you already find some time to have a look at this?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

I have same issue with this.

Hi @Wackymax,
Unfortunately I don't have a lot of experience with running tasks in the background. However I have been reading up on it and I am wondering if it would not make more sense to give more control to the application developer and let the developer explicitly start / stop the foreground service when the app detects that it is moved in or out of the background? The application can then start a new (separate) stream that specifically focusses on handling location updates the occur in the background.
I have the feeling that (especially on Android) this would remove some of the complexity around the reference counting and sharing the same stream.
How do you feel about this?
DISCLAIMER: as mentioned I don't have experience with Android foreground / background services, so I might be missing some really important aspects here.

That is definitely possible. In fact that is what I am doing in my app. Starting the foreground service means in my mind that the intent of the developer is to start a background service for Geolocator, making the reference counting necessary. If the developer just wants the location stream to stay alive while the app activity is alive then don’t pass in the foreground notification config. That will then not start a background service that does reference counting. The only point for passing the foreground config is to try and keep the activity alive a bit longer by raising it’s priority.
If the app developers do want to have a background service that runs even when the app activity is killed then they should use a background service flutter package and start the geolocator in that package.
This entire thread is confusing me a little because it seems like the expectation is that the location service should stop when the activity is destroyed, but that is already the default behaviour when there is no foreground notification passed in. Unless I am misunderstanding something.

On 20 Mar 2024, at 11:37, Maurits van Beusekom @.***> wrote:
Hi @Wackymax https://github.com/Wackymax,
Unfortunately I don't have a lot of experience with running tasks in the background. However I have been reading up on it and I am wondering if it would not make more sense to give more control to the application developer and let the developer explicitly start / stop the foreground service when the app detects that it is moved in or out of the background? The application can then start a new (separate) stream that specifically focusses on handling location updates the occur in the background.
I have the feeling that (especially on Android) this would remove some of the complexity around the reference counting and sharing the same stream.
How do you feel about this?
DISCLAIMER: as mentioned I don't have experience with Android foreground / background services, so I might be missing some really important aspects here.
—
Reply to this email directly, view it on GitHub https://github.com/Baseflow/flutter-geolocator/issues/1252#issuecomment-2009130611, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYF27MYNT42RT3BQ2ATPX3YZFKG5AVCNFSM6AAAAAAXWCAKC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBZGEZTANRRGE.
You are receiving this because you were mentioned.

@Wackymax In my situation it is like this. I want to have a “service” that keeps on running even when the Android activity is killed. This service should get the data from the GPS, and get some data from a BLE sensor. For this I’m using the following packages:
flutter_background_service: ^5.0.5
geolocator: ^10.1.0
flutter_blue_plus: 1.31.12
With flutter_background_service I start a new isolate and do everything I need from there. So within that isolate I’m using geolocator and flutter_blue_plus. That works just fine. However whenever I kill the app/activity I see this:
D/SessionLifecycleClient(13736): Sending lifecycle 2 to service
D/SessionLifecycleService(13736): Activity backgrounding at 629943097
D/FlutterGeolocator(13736): Detaching Geolocator from activity
D/FBP-Android: [FBP] onDetachedFromActivity
D/FlutterGeolocator(13736): Flutter engine disconnected. Connected engine count 1
D/FlutterGeolocator(13736): Disposing Geolocator services
E/FlutterGeolocator(13736): Geolocator position updates stopped
E/FlutterGeolocator(13736): There is still another flutter engine connected, not stopping location service
The BLE stuff keeps on running, but the location updates stop. Probably because geolocator is tied to the activity, and the BLE stuff isn’t.
Note that I can’t use the foreground notification of geolocator as I already have such a notification from flutter_background_service.
I’m more than happy to help out with this issue, but I’d like some guidance from you on how to approach this. I’m sure I’m not the only one who would like to use geolocator in the way that I do.

If that's the case I'll take a look. I am using Geolocator in the same manner. On 21 Mar 2024, at 13:19, Fesp @.***> wrote:
@Wackymax In my situation it is like this. I want to have a “service” that keeps on running even when the Android activity is killed. This service should get the data from the GPS, and get some data from a BLE sensor. For this I’m using the following packages:
flutter_background_service: ^5.0.5
geolocator: ^10.1.0
flutter_blue_plus: 1.31.12
With flutter_background_service I start a new isolate and do everything I need from there. So within that isolate I’m using geolocator and flutter_blue_plus. That works just fine. However whenever I kill the app/activity I see this:
D/SessionLifecycleClient(13736): Sending lifecycle 2 to service
D/SessionLifecycleService(13736): Activity backgrounding at 629943097
D/FlutterGeolocator(13736): Detaching Geolocator from activity
D/FBP-Android: [FBP] onDetachedFromActivity
D/FlutterGeolocator(13736): Flutter engine disconnected. Connected engine count 1
D/FlutterGeolocator(13736): Disposing Geolocator services
E/FlutterGeolocator(13736): Geolocator position updates stopped
E/FlutterGeolocator(13736): There is still another flutter engine connected, not stopping location service
The BLE stuff keeps on running, but the location updates stop. Probably because geolocator is tied to the activity, and the BLE stuff isn’t.
Note that I can’t use the foreground notification of geolocator as I already have such a notification from flutter_background_service.
I’m more than happy to help out with this issue, but I’d like some guidance from you on how to approach this. I’m sure I’m not the only one who would like to use geolocator in the way that I do.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

@Wackymax, I have spend some more time looking into how to keep the location services running in the background and I am starting to understand the concept of Foreground services in Android.
I also think I understand the concept you implemented regarding the flutter engine reference counting, as this probably comes into play when using the flutter_background_service plugin (or something similar).
Am I correct to assume that the current working procedure would be to:

Start the Geolocator.getPositionStream when the app is active (increases the connectedEngines to 1).
Register a callback with the flutter_background_service plugin to listen to the Geolocator.getPositionStream (increasing the connectedEngines to 2).
Force closing the application causes the onDetachedFromEngine to be called, decreasing the connectedEngine back to 1)
The flutter_background_service keeps an Dart isolate running listening to the position stream even when the application is "force" closed.

@Wackymax, I have spend some more time looking into how to keep the location services running in the background and I am starting to understand the concept of Foreground services in Android.
I also think I understand the concept you implemented regarding the flutter engine reference counting, as this probably comes into play when using the flutter_background_service plugin (or something similar).
Am I correct to assume that the current working procedure would be to:

Start the Geolocator.getPositionStream when the app is active (increases the connectedEngines to 1).
Register a callback with the flutter_background_service plugin to listen to the Geolocator.getPositionStream (increasing the connectedEngines to 2).
Force closing the application causes the onDetachedFromEngine to be called, decreasing the connectedEngine back to 1)
The flutter_background_service keeps an Dart isolate running listening to the position stream even when the application is "force" closed.

The way I understand the current implementation is that the Android Foreground service (GeolocatorLocationService) will keep the native part active. However to keep receiving location updates on the Dart side something extra is needed, like the flutter_background_service or a custom implementation keeping an isolate running and listening to the location stream.

The way I have it in my app that seems to work correctly is to only do Step 2. You are correct in that the native part will stay active but you need something to handle the dart side correctly.
One thing I have noticed is that my connected engine count is 4 when the activity is closed so I have a lot more happening in my app that is maybe keeping the location updates alive.
I am a bit busy on my project now to reach some deadlines but I can help to review a PR if someone creates one? I think the correctly solution would probably be to keep track of the engine id's for the engines that need location updates and check that before deciding to stop location updates. It will also help to solve https://github.com/Baseflow/flutter-geolocator/issues/1395

I am facing this as well. The way I could reproduce this is, keep my app offline for a while and then try building in debug mode again(while internet is OFF).
Though i don't see why geolocator plugin should be remotely related to internet availability.

Hello~Any further updates about this? Me aswell face the situdation that when app is losing network data for a while, entire "position updates" will stop. @saurabhkumar8112

We don't use anything related to the netword for position data so I'm not sure why that might be happening. Maybe it's something related to Android and power saving switching off the location updates after a while?

when i change battery optimization to unrestricted the location in the background works just fine.
as I understand so far, location in the background has to be dependent only by "location always permission" so I am not so sure why unrestricted my app battery helps solve this


Implement FromStr for Cwt

It would be nice to be able to do something like this:
let cert_data = "HC1:NCFOXNEG2NBJ5*H:QO-.OMBN+XQ99N*6RFS5YUCH%BM*4ODMT0NSRHAL9.4I92P*AVAN9I6T5XH4PIQJAZGA2:UG%U:PI/E2$4JY/KB1TFTJ:0EPLNJ58G/1W-26ALD-I2$VFVVE.80Z0 /KY.SKZC*0K5AFP7T/MV*MNY$N.R6 7P45AHJSP$I/XK$M8TH1PZB*L8/G9YPDN*I4OIMEDTJCJKDLEDL9CZTAKBI/8D:8DKTDL+S/15A+2XEN QT QTHC31M3+E3+T4D-4HRVUMNMD3323623423.LJX/KQ968X2+36/-KKTC 509UE1YH/T1NTICZUI 16PPT1M:YUKQU7/EAFQ+JU2+PFQ51C5EWAC1ASBE/V9.Q5F$PYBEO.0:E5 96KA7N95ZTM L7HHP4F5G+P%YQ+GONPPCHPMR73SOM352Q4FIR L7 SP5.PDSOSNQKIR%*O* OUKRTSOL0PNLE.$FW2I9R7P3LEERQ2Q$CS8-QL4W.X0WB0/89-M7O9F:4AV0OGXP7MEKC53WRXY41A1/:R/J9URTB%LKXAD-OAZ0GA5%UCI/I910G-8H3";
let cwt: dgc::Cwt = cert_data.parse().unwrap();

This will be possible by implementing the FromStr trait on Cwt.

This one will make @allevo proud! 😇

Any idea how to implement it? just convert it to &[u8] and convert into Cwt ? or the string is a completely different implementation?

If the parse is made by ciborium, just use a ciborium::de::from_reader into it, from_reader takes anything that impl trait Read, and Read is implemented for &[u8], a String becomes &[u8] with the method as_bytes

My idea was to just be an alternative to call this function directly: https://github.com/rust-italia/dgc/blob/main/src/parse.rs#L173

So literally something like this:
impl FromStr for Cwt {
    type Err = ParseError;

    fn from_str(s: &str) -> Result<Self, Self::Err> {
        // remove prefix
        let data = remove_prefix(data)?;

        // base45 decode
        let decoded = decode_base45(data)?;

        // decompress the data
        let decompressed = decompress(decoded)?;

        // parse cose payload
        let cwt = parse_cwt_payload(decompressed)?;

        Ok(cwt)
    }
}

I only have one doubt in terms of ergonomics:

decode returns the payload (DgcContainer)
from_str returns the cwt object (Cwt)

I am not really sure if it will be more ergonomic and consistent to always return a Cwt...
One of the advantages of having a CWT is that you have access to more data (headers, signature, etc). But at the same time you don't care about this data unless you want to verify the signature (which is the reason why decode returns directly the paylod, it's a use case that assumes you don't care about the signature, otherwise you'd use validate()
Thoughts?

I tried to reverse the logic: implement the parse into Cwt and DgcContainer validate invokes it. So in this way, the decode* public APIs are implemented in this way:

pub fn decode_cwt(data: &str) -> Result<Cwt, CwtParseError> {
    data.parse()
}

pub fn decode(data: &str) -> Result<DgcCertContainer, CwtParseError> {
    let cwt = decode_cwt(data)?;
    Ok(cwt.payload)
}

As you can see, i changed the errortype reversing the inclusion (CwtParseError has a variant called ParseError). Is it make sense? Which differences are between the 2 errors?

@allevo, i like the idea of decode_cwt and decode. I think it will save us from a bit of code duplication.
With that being said, I am starting to think that maybe we shouldn't leak the CWT. At the end of the day is a little bit of an implementation detail. What really matter is not the encoding format (CWT) but the DgcCertContainer.
What about implementing FromStr only for DgcCertContainer (and not exposing decode_cwt publicly?
I know this is a bit philosophical, but it's probably better to keep the api surface small and simple...
Regarding the 2 errors. I think i was implementing CWT as a standalone lib initially, that's maybe why i came up with 2 different error sets (one for CWT decoding and one for the whole parsing). Let's definitely revisit this decision if it does not make too much sense to you...

Removing Cwt from the public interface would make the problem of having to return the tuple in the result - because one might want to read the data even if it's not valid - rather difficult to solve.
To hide the implementation details there are other options, like we could make Cwt have all its fields private and make it impl Deref<Target=DgcCertContainer>. But even then, I don't see why someone shouldn't be able to access the key id, the algorithm, and the signature.
Regarding the FromStr proposal I don't quite like it because rustdoc hides doc comments on trait method implementations. So one can't know for sure what data the parse method parses, and implementing it for DgcCertContainer would be even more confusing. It could be written in the documentation but from the method itself it's difficult to do understand it.
To give an example, in the docs will be written "DgcCertContainer::parse Parses a string s to return a value of this type." and someone may understand that it parases the JSON object of the greenpass, someone else might think that it parses the CBOR of the HCERT, another one may think that it parses the raw COSE data, etc.
What I agree on is merging the ParseError types in one.

Some things I want to add that I also referenced in the other issue:
I think that those functions (decode_cwt, and validate) should be methods of the Cwt struct and that the Cwt struct should be renamed to a more specific name as we are not parsing a generic CWT but one defined by the European Commission that contains an HCERT and that is signed.
validate might even be a method of the TrustList but in my mind, one should be able to write
Cwt::decode(data)?.verify(trust_list) == SignatureValidity::Valid

You raise some very good points, @Rimpampa!
I am sold on the fact that it might make sense to expose the CWT (even though right now some important fields are private, maybe we should revisit that).
On the FromStr in am not highly opinionated, so I am more than ok not adding this feature and closing this issue if we don't see a strong benefit to it! It's definitely not a priority (it does not add any new feature  and it does not solve any bug).
Regarding API ergonomics I think the validation is the one that seems to be raising more skepticism, so there are probably many opportunities to revisit that.
I like your suggestions to make validate a method of Cwt (which gives us one more reason to keep Cwt publicly accessible), but again, I would like to consider validation in a more comprehensive way as being discussed in #21.
If we can figure out a decent API that will allow us in the future to do "all types" of validation easily that would be awesome.
Should we close this issue then and continue the API conversation in #21?

I forgot to mention that i did some renaming in #29 and also added a diagram that should help with understanding how the data is laid out across the various containers/structs.


New Module for Product Hub -WumMerger

WumMerger
This will update the changes for jaggery app files from component dep.
customizedList.json can be used to input the updated and added files separately.
This module will merge(override if exists) those files in the WUM updated vanilla pack
https://jira.wso2telco.com/jira/browse/EXTGW-487

Closing the pull request due to building failure. Working on the issue.


Update all deps.

No-Issue

Looks relevant because it landed in DRF 3.15.0
https://github.com/encode/django-rest-framework/commit/4abfa28e0879e2df45937ac8c7a9ffa161561955

I think the correct way to deal with the %(user)s thing is to make it %%(user)s:
In [1]: 'foo %(bar)s' % {'bar': "aa"}
Out[1]: 'foo aa'

In [2]: 'foo %%(bar)s' % {'bar': "aa"}
Out[2]: 'foo %(bar)s'

^ The problem is that this is test input mocking user input. The solution is that you don't ever format a string from a user. Before this DRF upgrade they didn't format the string passed into ValidationError, thus the problem. Due to the DRF change, DAB code can't safely pass in a string with user data like it is doing now.


Add a license

Would be great if this lib had some license attached - would suggest MIT or Apache

Done

Thanks so much!

can I use it in commercial or play store app ?


Regarding Abstractive Summaries

As mentioned in the paper, during fine-tuning, you masked some tokens in the summaries and then you predicted those tokens. But during inference (only test-data is given), you don't have the summaries. So, how did the prediction occur during inference time? I mean, how you gave the inputs for inference and how did the decoding work?

The decoding procedure is similar to the vanilla seq2seq decoder. In each time step, we generate one token, and then feed it into the model for the next time step’s prediction. Both greedy search and beam search can be employed for decoding.


Exception: Error: Invalid box type: None

Loaded ground truth annotations for 81 samples.
Filtering tracks
Traceback (most recent call last):
File "/home/qw/Tangme10/SparseEnd2End/script/train.py", line 140, in
main()
File "/home/qw/Tangme10/SparseEnd2End/script/train.py", line 127, in main
train_api(
File "/home/qw/Tangme10/SparseEnd2End/tool/trainer/train_sdk.py", line 119, in train_api
runner.run(data_loaders, cfg["workflow"])
File "/home/qw/Tangme10/SparseEnd2End/tool/runner/iter_based_runner.py", line 109, in run
iter_runner(iter_loaders[i], **kwargs)
File "/home/qw/Tangme10/SparseEnd2End/tool/runner/iter_based_runner.py", line 42, in train
self.call_hook("after_train_iter")
File "/home/qw/Tangme10/SparseEnd2End/tool/runner/base_runner.py", line 249, in call_hook
getattr(hook, fn_name)(self)
File "/home/qw/Tangme10/SparseEnd2End/tool/hook/evaluation.py", line 267, in after_train_iter
self._do_evaluate(runner)
File "/home/qw/Tangme10/SparseEnd2End/tool/hook/evaluation.py", line 278, in _do_evaluate
key_score = self.evaluate(runner, results)
File "/home/qw/Tangme10/SparseEnd2End/tool/hook/evaluation.py", line 365, in evaluate
eval_res = self.dataloader.dataset.evaluate(
File "/home/qw/Tangme10/SparseEnd2End/dataset/nuscenes_dataset.py", line 328, in evaluate
ret_dict = self._evaluate_single(
File "/home/qw/Tangme10/SparseEnd2End/dataset/nuscenes_dataset.py", line 498, in _evaluate_single
nusc_eval = TrackingEval(
File "/home/qw/anaconda3/envs/sparsee2e/lib/python3.8/site-packages/nuscenes/eval/tracking/evaluate.py", line 97, in init
pred_boxes = filter_eval_boxes(nusc, pred_boxes, self.cfg.class_range, verbose=verbose)
File "/home/qw/anaconda3/envs/sparsee2e/lib/python3.8/site-packages/nuscenes/eval/common/loaders.py", line 219, in filter_eval_boxes
class_field = _get_box_class_field(eval_boxes)
File "/home/qw/anaconda3/envs/sparsee2e/lib/python3.8/site-packages/nuscenes/eval/common/loaders.py", line 283, in _get_box_class_field
raise Exception('Error: Invalid box type: %s' % box)
Exception: Error: Invalid box type: None
After deleting the logger, the model training shows the above error, can you help me with this, thanks a lot!

It is very similar to the previous question. I suggest adding the --no-validate argument. For example, you can run the following command:clear && python script/train.py dataset/config/[your_config.py] --no-validate

hi Dr. Thomas，Thank you very much for your reply
Using the -no-validate command will allow the training to run without hindrance. Last week, I directly deleted it in the code for the logger problem.it is also useful
However, the problem of Invalid box type: None still appears in the test. I found that the v1.0-mini dataset uses dump_nusc_ann_json.py to generate the required .json file, which contains NaN values. I replaced it with ''0.0'', and the problem of Invalid box type: None was solved. But when I use trainval to do it, the same thing happens. At the same time, when using the trainval dataset to generate anchors by K-means, there will be a warning of no gt boxes. I don't know if this will have an impact. Recently I am going to convert the resnet code to SNN, but I don't know if I need to modify the loss function
Not sure if there is anything that I am not doing correctly. Thank you for your great work and effort in maintaining this!
Your work is very great and very useful to me, thank you for your reply, and I am looking forward to your update of the code！

Some solutions for others：

File "script/test.py", line 88, in main
print(checkpoint["meta"]["CLASSES"])
KeyError: 'CLASSES'
add calsses to ckpt
import torch
checkpoint_path = "/home/qw/Tangme10/SparseEnd2End/e2e_worklog/sparse4d_temporal_r50_1x1_bs1_256x704_mini/iter_66.pth"
checkpoint = torch.load(checkpoint_path, map_location='cpu')

if 'meta' not in checkpoint:
checkpoint['meta'] = {}
classes = ['car', 'truck','construction_vehicle','bus', 'trailer', 'barrier','motorcycle','bicycle','pedestrian', 'traffic_cone']
checkpoint['meta']['CLASSES'] = classes
torch.save(checkpoint, checkpoint_path)
print("Checkpoint updated successfully with CLASSES.")
2.data_time problem
i just chaged the code  from textlog
if "time" in log_dict.keys():
self.time_sec_tot += log_dict["time"] * self.interval
time_sec_avg = self.time_sec_tot / (runner.iter - self.start_iter + 1)
eta_sec = time_sec_avg * (runner.max_iters - runner.iter - 1)
eta_str = str(datetime.timedelta(seconds=int(eta_sec)))
log_str += f"eta: {eta_str}, "
# log_str += (
#     f'time: {log_dict["time"]:.3f}, '
#     f'data_time: {log_dict["data_time"]:.3f}, '
# )
# statistic memory
log_str += (
f'time: {log_dict.get("time", 0):.3f}, '
f'data_time: {log_dict.get("data_time", 0):.3f}, ')
# if torch.cuda.is_available():
#     log_str += f'memory: {log_dict["memory"]}, '

Q1: When using the trainval dataset to generate anchors by K-means, there will be a warning of no gt boxes. I don't know if this will have an impact?
A1: I have added a continue statement in dump_nusc_ann_json.py to exit the loop. It means there's no valid gtbox in this frame. When we cluster the boxes using K-Means, we can ignore this waring.
Q2:  I found that the v1.0-mini dataset uses dump_nusc_ann_json.py to generate the required .json file, which contains NaN values. I replaced it with ''0.0'', and the problem of Invalid box type: None was solved？
A2: I think there is no need to deal with this issue additionally. Because I've added this processing logic to the dataset script dataset/nuscenes_dataset.py：
if self._with_velocity:
    gt_velocity = np.array(info["gt_velocity"]).reshape(-1, 2)[mask]
    nan_mask = np.isnan(gt_velocity[:, 0])  # 判断标注文件中的速度是否都有效
    gt_velocity[nan_mask] = [0.0, 0.0]  # 标注中速度无效设置为0
    gt_bboxes_3d = np.concatenate([gt_bboxes_3d, gt_velocity], axis=-1)

Q3: However, the problem of Invalid box type: None still appears in the test？
A3: I think if the training link is ok with no problem, you can directly start the NuScenes full data training(you can use config:dataset/config/sparse4d_temporal_r50_1x4_bs22_256x704.py) instead of NuScenes MINI. In my personal experience, when the training converges, the test doesn't have the problem of invalid box type: None.
Q4: Recently I am going to convert the resnet code to SNN, but I don't know if I need to modify the loss function？
A4: In my personal experience, you don't need to modify the loss function, you just need to train the whole model: SNN + Sparsehead start with small learning rate. If you pretriain the backbone:SNN in nuImages or ImageNet, it is easier to converge and get good trainging results.

I apologize for not getting back to you until now.thank you very much for your reply, it has been very helpful.
I have only recently started learning about SNN and am exploring this area, I didn't really understand the last piece of advice you gave and wasn't sure how to go about implementing the two methods you mentioned. Yesterday I modified the /modules/backbone/resnet.py
`class SpikingNeuron(nn.Module):
def init(self, threshold=1.0, decay=0.9):
super(SpikingNeuron, self).init()
self.threshold = threshold
self.decay = decay
self.membrane_potential = None
def forward(self, x):
    if self.membrane_potential is None or self.membrane_potential.shape != x.shape:
        self.membrane_potential = torch.zeros_like(x)
    self.membrane_potential = self.decay * self.membrane_potential + x
    out = (self.membrane_potential >= self.threshold).float()
    self.membrane_potential = self.membrane_potential * (1 - out)
    return out`

`    def forward(self, x):
"""Forward function."""
    def _inner_forward(x):
        identity = x

        out = self.conv1(x)
        out = self.norm1(out)
        out = self.spike_fn(out)

        out = self.conv2(out)
        out = self.norm2(out)
        out = self.spike_fn(out)

        out = self.conv3(out)
        out = self.norm3(out)

        if self.downsample is not None:
            identity = self.downsample(x)

        # Ensure the dimensions match for addition, including channels
        if out.shape != identity.shape:
            # Adjust channels if necessary
            if out.shape[1] != identity.shape[1]:
                identity = torch.nn.Conv2d(identity.shape[1], out.shape[1], kernel_size=1, stride=1, bias=False)(identity)

            # Adjust spatial dimensions (H, W)
            identity = nn.functional.interpolate(identity, size=out.shape[2:], mode='nearest')

        out += identity

        return out

    if self.with_cp and x.requires_grad:
        out = cp.checkpoint(_inner_forward, x)
    else:
        out = _inner_forward(x)

    out = self.spike_fn(out)

    return out`

After I made this modification there were some problems :
RuntimeError: The size of tensor a (16896) must match the size of tensor b (264) at non-singleton dimension 0
so I had to modify the loss function
`    def loss(self, depth_preds, gt_depths):
loss = 0.0
for pred, gt in zip(depth_preds, gt_depths):
# 调整预测和真实深度的形状，以匹配相应的维度
if pred.shape != gt.shape:
min_shape = [min(p, g) for p, g in zip(pred.shape, gt.shape)]
pred = pred[:, :min_shape[1], :min_shape[2], :min_shape[3]]
gt = gt[:, :min_shape[1], :min_shape[2], :min_shape[3]]
        pred = pred.contiguous().flatten()
        gt = gt.contiguous().flatten()

        min_length = min(pred.size(0), gt.size(0))
        pred = pred[:min_length]
        gt = gt[:min_length]
        fg_mask = torch.logical_and(gt > 0.0, torch.logical_not(torch.isnan(pred)))
        gt = gt[fg_mask]
        pred = pred[fg_mask]
        pred = torch.clamp(pred, 0.0, self.max_depth)
        with autocast(enabled=False):
            error = torch.abs(pred - gt).sum()
            _loss = error / max(1.0, len(gt) * len(depth_preds)) * self.loss_weight
        loss = loss + _loss
    return loss
`

I've found that  memory clock grows until it explodes.
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 68.00 MiB (GPU 0; 23.68 GiB total capacity; 19.18 GiB already allocated; 63.75 MiB free; 21.49 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF（I think it must be my own fault.）
resnet+core_block.zip
Thank you for your answers Looking forward to your updates and replies!

Hi, I’m sorry for the late reply.
I haven't used SNN as an image encoder, but I found that the feature shapes extracted from the backbone have changed. From SNN, we obtain the following feature shapes:
torch.Size([6, 256, 64, 176])
torch.Size([6, 512, 4, 11])
torch.Size([6, 1024, 1, 1])
torch.Size([6, 2048, 1, 1])

We hope to extract features at different scales from the original images. Therefore, we use ResNet to downsample and achieve the following feature shapes:
xx= torch.Size([6, 256, 64, 176])
xx= torch.Size([6, 512, 32, 88])
xx= torch.Size([6, 1024, 16, 44])
xx= torch.Size([6, 2048, 8, 22])

I hope this information is useful for you.

非常感谢您的回复，对于SNN的修改我已经完成。但是在训练时由于我只有一张24G的3090，所以只能在batchsize=1的时候训练，感觉很容易训飞，我尝试将学习率降低到1e-6，，您觉得大概什么学习率较为合适，这个会不会太小了

batchsize大点尽可能把3090的显存吃满, 学习率1e-6可以的. 如果训练崩了, 还可以再降低一个数量级.
学习率低点只会影响训练收敛速度和收敛效果, 学习率高了可能直接导致训练发散.
当你找到一个可以让训练收敛的学习率后,可以把它慢慢大, 加速收敛速度.

非常感谢您的帮助，基本上解决了我目前的问题，感谢您的无私奉献，谢谢！
期待您后续的更新！


Endpoint /1/tokens/{YourToken}/webhooks only returns "unauthorized token"

I can register a webhook at the same endpoint but trying to issue a GET  results in "unauthorized token".

@rwilson0 Would you mind asking your question over here: https://community.developer.atlassian.com/c/trello

 ok


[Run] URL plugin is false triggering with \string

ℹ Computer information

PowerToys version:
PowerToy Utility:
Running PowerToys as Admin:
Windows build number: [run "winver"]

📝 Provide detailed reproduction steps (if any)

…
…
…

✔ Expected result
What is the expected result of the above steps?
❌ Actual result
What is the actual result of the above steps?
📷 Screenshots
Are there any useful screenshots? WinKey+Shift+S and then just paste them directly into the form
Version: <IP_ADDRESS>
OS Version: Microsoft Windows NT 10.0.19041.0
IntPtr Length: 8
x64: True
Date: 3.12.2020 16:24:23
Exception:
System.ComponentModel.Win32Exception (2): The system cannot find the file specified.
at System.Diagnostics.Process.StartWithShellExecuteEx(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
at Microsoft.Plugin.Uri.Main.<>c__DisplayClass23_0.b__0(ActionContext action)
at PowerLauncher.ViewModel.MainViewModel.<>c__DisplayClass23_0.b__0()
at System.Windows.Threading.Dispatcher.Invoke(Action callback, DispatcherPriority priority, CancellationToken cancellationToken, TimeSpan timeout)
at System.Windows.Threading.Dispatcher.Invoke(Action callback)
at PowerLauncher.ViewModel.MainViewModel.OpenResultsEvent(Object index, Boolean isMouseClick)
at System.Windows.Input.CommandManager.TranslateInput(IInputElement targetElement, InputEventArgs inputEventArgs)
at System.Windows.UIElement.OnKeyDownThunk(Object sender, KeyEventArgs e)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
at System.Windows.Interop.HwndKeyboardInputProvider.ProcessKeyAction(MSG& msg, Boolean& handled)
at System.Windows.Interop.HwndSource.CriticalTranslateAccelerator(MSG& msg, ModifierKeys modifiers)
at System.Windows.Interop.HwndSource.OnPreprocessMessage(Object param)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(Object source, Delegate callback, Object args, Int32 numArgs, Delegate catchHandler)

@matejchalupka can you provide more details on what you tried to execute here?  It looks like it is trying to execute a program but can't find it.

I tried to execute new search in Google chrome tabs.with syntax \google.com\search...

Od: Clint Rutkas<EMAIL_ADDRESS>Odesláno: čtvrtek 3. prosince 2020 21:21
Komu: microsoft/PowerToys
Kopie: matejchalupka; Mention
Předmět: Re: [microsoft/PowerToys] PT run problem. (#8364)
@matejchalupkahttps://github.com/matejchalupka can you provide more details on what you tried to execute here? It looks like it is trying to execute a program but can't find it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/PowerToys/issues/8364#issuecomment-738285112, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQ5WVWDDDKM37PSHRPXAUPTSS7XNZANCNFSM4UMEELBQ.

\ would be a file. // is URL
it does seem like it reacts correctly however. which can cause the confusion and seems like the URL plugin is false registering.

Can I pick up this for 0.31?

By all means, no one has claimed dibs

\ is a network file path.

@crutkas with https://github.com/microsoft/PowerToys/pull/9538 in 0.33 PT Run isn't crashing anymore with invalid URI.
Also https://github.com/microsoft/PowerToys/issues/8891 and https://github.com/microsoft/PowerToys/issues/9013 should be fixed.

@wshaver this is actually like type an URL or a path into run (Win+R).
There is already #9270 opened for a better UNC handling in PT Run.

@davidegiacometti i have a file server and it works correctly.  To me, the oddness is seeing file://nas/ open in browser is an oddity when i type in \nas

@crutkas do you mean \nas?

\\nas

The oddity to me is URL plugin handling files when we have Folder plugin.

i think that is the bug here, a UNC path is a folder and should be ignored by the URL plugin

@crutkas @davidegiacometti
https://github.com/microsoft/PowerToys/issues/9270 is tracking this.
I'm OK with the folder plugin handling \\, and we have to document it so users will know which plugin does what.

Closing this in favor of https://github.com/microsoft/PowerToys/issues/9270
The crash was fixed by https://github.com/microsoft/PowerToys/pull/9538


cmake: by default install in install folder

Closing this PR, since https://github.com/rdkcentral/ThunderTools/pull/99 is merged now.


Eliminate or replace wait_ms() instances

Eliminate or otherwise replace wait_ms() instances with Thread::wait()
wait_ms() does a busy wait which is wasteful when dealing with multiple threads. When using an NRF52_DK target calling wait_ms() from multiple threads causes application hang. Replacing these occurrences with Thread::wait() solves this issue.

@saedelman - SD driver is updated. Can you please verify if with NRF52_DK?

@deepikabhavnani - confirmed.


Training problem

When I run ./run_qagnn__csqa.sh to train CommonsenseQA, I got the error below. How do you solve this?

Hi, I think you need to download the required data by running ./download_preprocessed_data.sh. Please refer to the Section 1. Download Data in our README. Thanks.

Hi, I think you need to download the required data by running ./download_preprocessed_data.sh. Please refer to the Section 1. Download Data in our README for more details.

Hi, thanks for replying. I actually did download the data, I think the ent_emb_paths variable is not defined in the code.


bug Show defined variables as it is if it is not defined in the data object

@PreemyJoshy123 Please provide a full reproduction with code + template! A simple image of the output log has no use to reproduce the problem.

@mathe42  The below is the create report function and the template i am using to replace variables.Here i am not assigning value to variable data .But after creating report in need to retain that ${data} as it is . How it is possible .
const buffer1 = await createReport({
template: content,
cmdDelimiter: ['${', '}'],
data: {
firstname:'jhon',
employeename:'jhon smith'
},
});

That is not possible... You could use
const data = {...}
data.data = data

createReport{{..., data})

we can't asume that ${data} will be always that it will change according to each user . i m using this for creating letters for the employees in my project . If we mention any variable that is not defined in data object we need to show that variable as it is . How it is possible ??

You could do ${ myVar ?? 'fallback'}
Not sure what you want to accive can you give some examples?

What about:
${ myVar ?? '${myVar}'} in template?

@mathe42  Can you show me an example for error handler to catch errors in template commands .

@PreemyJoshy123
https://github.com/guigrpa/docx-templates#custom-error-handler
Closing this issue because it is not a bug.


Are trackers properly loaded/checked?

A VAST specialist testing my platform reported that he only saw this tracker sent:

Whereas he expected to see something like this:

If you are interested in looking into this issue I can provide a test url.

Yes, is possible send us the test url. Will take me a bit of time to answer because I'm not working anymore @MailOnline


Plasma: New install has wrong color scheme

Describe the bug
The default color scheme for Plasma desktop after installation should be Breeze. The color scheme on first boot is actually Oxygen. To make matters worse, System Settings will report that the scheme is Breeze, even though it is obviously Oxygen.
To Reproduce
Steps to reproduce the behavior:

Install NixOS with services.xserver.desktopManagers.plasma5.enable = true.
Open System Settings > Colors.
Observe that Breeze is selected, but the actual colors are those of the Oxygen scheme (dark grey application background, blue window titlebars).
Select Oxygen and Apply. Observe that there is no change.
Select Breeze and Apply. Observe that the Breeze color scheme is now applied.

Expected behavior
The default color scheme at first boot is Breeze.
Screenshots
Oxygen colors:

Breeze colors:

Notify maintainers
@ttuegel
Metadata
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: plasma5
# a list of nixos modules affected by the problem
module: services.xserver.desktopManager.plasma5

Keep this issue open.


Array index on a 2D matrix not correct

While trying out the examples in the excellent manual, I see a discrepancy between what the manual reports and code that I try in Visual Studio 2022.
// create arrays
Array<Int64, 1> array_1d = jluna::safe_eval("return Int64[1, 2, 3, 4, 5]");
Array<Int64, 2> array_2d = jluna::safe_eval("return reshape(Int64[i for i in 1:9], 3, 3)");
// print
Base["println"]("array_1d: ", array_1d);
Base["println"]("array_2d: ", array_2d);
// access elements at index (0-based)
Int64 one_d_at_3 = array_1d[2];
Int64 two_d_at_2_2 = array_2d[1, 1];
// print
std::cout << one_d_at_3 << std::endl;
std::cout << two_d_at_2_2 << std::endl;
This generates
[JULIA][LOG] initialization successful (1 thread(s)).
array_1d: [1, 2, 3, 4, 5]
array_2d: [1 4 7; 2 5 8; 3 6 9]
3
2
For array_2d the element first element of the second row is returned instead of the second, which is 5 as shown  in the manual

Fixed by https://github.com/Clemapfel/jluna/commit/d0d32627a01c5b7ca7ad4eeb7f8497289cb0eff4, this fix will go live in version 1.1


NHD must be reversed

Seems that it has to be reversed.

@rultor release, tag is 0.10.5

@rultor release, tag is 0.10.5

@yegor256 OK, I will release it now. Please check the progress here

@rultor release, tag is 0.10.5

@yegor256 Done! FYI, the full log is here (took me 15min)

Oops! There is no order for job gh:yegor256/jpeek#47.

Oops! Job gh:yegor256/jpeek#47 is not in scope.


Adding profile page servlet (UsersServlet), added to web.xml, created…

… .jsp, and created test for servlet although it has no tests currently

This is really basic because I'm not entirely sure what I am doing, but will build on this


Adding a bql to json endpoint

Coverage increased (+0.01%) to 99.529% when pulling a2dee682100d070c0b2c7a33f5c254b08ba2661a on hbxie:master into 86f5a17ca9a5b1541d04b643c36baab6b352f311 on bullet-db:master.


How to use deepface by GPU

What's the way to use gpu for deepface?
The module of tensorflow-gpu is deprecated now and merged into tensorflow.

deepface does not do anything with gpu. if you use gpu with tf, then it will be used in deepface, too.
i was using tensorflow-gpu. if it is deprecated, then it can be used by default? i recommend you to ask this question in tf repo.

deepface does not do anything with gpu. if you use gpu with tf, then it will be used in deepface, too.
i was using tensorflow-gpu. if it is deprecated, then it can be used by default? i recommend you to ask this question in tf repo.

Thanks, this is the offical tips for tensorflow gpu: https://www.tensorflow.org/install/gpu

@glovenone were you able to use the deepface library on gpu?


Removed float from tags

Description
Tags use a float and a clear, but are already inline-flex so it's unnecessary and has caused some issues.
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.

@GarthDB thanks! Please leave package-lock.json out of future PRs unless they change dependencies, it made merging this a little tough.


Exceeds Dex Limit of 64k in its current state.

I just cloned the repo and tried to build
./gradlew clean assemble

The build fails with the following error:
Dex: The number of method references in a .dex file cannot exceed 64K.

Is this a known issue? I can try and submit a pull request to fix this soon. I just wanted to check before a pull request.

✌ merged PR. Thanks for this


ZQuantum <-> Qiskit circuit conversions for standard gates

Implements conversion between the circuits. Includes a workaround I've came up with to overcome the circular imports between _gates and _builtin_gates modules.
Roadmap:

✅ zquantum built-in gates <-> Qiskit built-in gates - added in this PR
❌ zquantum built-in gates <-> Qiskit custom gates - not needed at the moment, all our gates are implemented in Qiskit
❌ zquantum custom gates <-> Qiskit built-in gates - not supported. As discussed internally, if a user stumbles upon this we should guide them to create a custom gate instead.
🔁 zquantum custom gates <-> Qiskit custom gates - not ready yet, will do in another PR. I've already spent so much time on this task I don't wanna risk digging to deep here without merging to dev.

@alexjuda Thanks, looks good to me 👍


Segment Tree

Submission Title
Segment Tree
Submission Type
Data Structure
Submission Description
Finding the range sum queries and range xor queries using Segment Tree
Contributors
C++

Submission Title
Segment Tree
Submission Type
Data Structure
Submission Description
Finding the range sum queries and range xor queries using Segment Tree
Contributors
C++

assign me this task

Hi, I could work on the segment tree for C++, Java or Python. Could you assign me this task, please?

Please assign me this task

thanks @eveyone :v:


Add support for unlimited map size

Discussed in https://github.com/microsoft/ebpf-for-windows/discussions/3323

Originally posted by saxena-anurag March  2, 2024
There are some production cases possible where it is hard to predict the maximum map size that will be needed beforehand. HVCI requirements (hence native modules) also makes harder to change the map size at runtime.
One possible solution for this is to support unlimited map size. This support can be added for all maps except array map types, and LRU hash. This should not cause any verification or safety issues, at the same time allowing unlimited size for hash maps.
Proposal

User passes 0 map size to indicate unlimited entries.
If map size is unlimited, ebpfcore chooses a larger bucket count than the current default.
Any possible verifier changes to allow 0 map size n the verification step.

Proposal: Instead keep using non-0 map size, but ebpfcore ignores the max size. Most likely the same is done by Linux.


I'M NOT RESPONSABLE.... BLAH BLAH BLAH....
This part of the repo contains a list of alive services to be exploited for cryptojacking purposes, some pdf with techniques and studies,
and also a sample repo of mine explaining how to take advantage of services like coinhive and make your miner stealth.
You cand also find the example here:
https://github.com/JonnyBanana/Coin-Hive_Hidden-Miner.github.io

https://github.com/SkyBulk/Mining_Stuff/blob/main/Cryptojacking/README.md

THIS IS ONLY FOR TESTING POURPOSE!
I'M NOT RESPONSABLE.... BLAH BLAH BLAH....
This part of the repo contains a list of alive services to be exploited for cryptojacking purposes, some pdf with techniques and studies,
and also a sample repo of mine explaining how to take advantage of services like coinhive and make your miner stealth.
You cand also find the example here:
https://github.com/JonnyBanana/Coin-Hive_Hidden-Miner.github.io
An updated list of working services to be exploited for cryptojacking purposes

WEBMINEPOOL
https://webminepool.com/

CRYPTOLOOT
https://crypto-loot.org/

/* TEAM /
Your title: Nicolò Saporito, Sappoh Sama Workshop, WebChain Solutions
Site:jonnybanana.github.io, www.webchain.info
Mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Youtube: http://www.youtube.com/c/HowToHackItalia,  http://www.youtube.com/c/SappohSamaWorkshop
Twitter: https://twitter.com/jonnybanana84
Github:  https://github.com/JonnyBanana
Facebook: https://www.facebook.com/SapperSapoSaps
Ebay Shop: https://www.ebay.it/str/sapposamaworkshop
Work PhotoAlbum: https://howtohack.imgur.com/
My PGP:
<EMAIL_ADDRESS>8c8f7dfc02d993b2418786e567040a7d27ccdff7
<EMAIL_ADDRESS>cc2572b2aa752fbfe48f93c7a6063b8be8964e09
Mailvelope
FA6594C6C31FAAE54D9BD87EAF05B231635B9F59
Location: Bologna, Italy.
[...]

/ SITE'S E ALGHORITMS*/ GDRICGUUK7AUGVOCOQDSB7RS67FKNHYRNVLYMFAULNYC2TUWLFGXFSBJ


torchrun:command not f

hi , torchrun:commad not found
Can you help me take a look at this issue? thankyou

maybe you can try this command
CUDA_VISIBLE_DEVICES=4,5,6,7  python -m torch.distributed.launch --nproc_per_node 4 --master_port 12356 main.py pretrain_aimclr --config ./config/ntu60/pretext/pretext_aimclr_xview_joint.yaml

If training with a single card, remember to modify the learning rate to 0.1 as the batchsize is reduced to a quarter.

Hello, please give me some advice. If I use a single GPU for training, should I execute the following command?
CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.launch --nproc_per_node=1 main.py pretrain_actclr --config ./config/ntu60/pretext/pretext_actclr_xview_joint.yaml
And, should my torch be 1.4.0? Thank you for your help.
1374149633
@.***

------------------ 原始邮件 ------------------
发件人:                                                                                "LanglandsLin/ActCLR"                                                                                @.>;
发送时间: 2024年2月27日(星期二) 下午3:34
@.>;
<EMAIL_ADDRESS>主题: Re: [LanglandsLin/ActCLR] torchrun:command not f (Issue #9)
If training with a single card, remember to modify the learning rate to 0.1 as the batchsize is reduced to a quarter.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

yes, and I think the version of torch should be all fine and I use 1.8 for training.

If I am using a single GPU, what command should I execute? thanks for your help.

I think this will work
CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.launch --nproc_per_node=1 main.py pretrain_actclr --config ./config/ntu60/pretext/pretext_actclr_xview_joint.yaml

you can delete this code:
https://github.com/LanglandsLin/ActCLR/blob/23c62f5eff793bc5157da2ea6a22b6b4b7e5cd93/processor/processor.py#L175C2-L179C59

you can delete this code: https://github.com/LanglandsLin/ActCLR/blob/23c62f5eff793bc5157da2ea6a22b6b4b7e5cd93/processor/processor.py#L175C2-L179C59

Thankyou very much


Can’t copy image to another SD card as openhabian uses whole SD card by default (8GB or 16GB SD card)

SD card comes usually with different sizes of a few bytes, so using an SD copy program don’t allow you to copy the image to another SD card. It might better to not take the always the full available bytes per SD card. An better option might be to take a bit less as available like 7.8 GB or 15.8 GB) … sure a few bytes are lost but you won’t run into issues using an SD image copy program.

Hey @palatia, that's a good idea! I've however no saying in that.
Could you create a new (nearly identical) ticket at https://github.com/debian-pi/raspbian-ua-netinst ? That's the base project we are using for openHABian and your idea needs to be implemented there.

sure, no problem. i haved added it here -
https://github.com/debian-pi/raspbian-ua-netinst/issues/439
Thanks for the link to the downsizer script. Looks easier as the steps i were using.

Additionally I just found https://github.com/Drewsif/PiShrink


Enum error setting operation mode for DHW with VRC700

Before submitting a new issue

[x] I redownloaded the latest release and restarted Home Assistant
[ ] I've enabled debug logs and attached the relevant logs to this ticket

Problem description
When I try to change the dhw operation mode to ‘DAY’ using water_heater.set_operation_mode I receive this message: 'DAY' is not a valid DHWOperationMode
Logs

Logger: homeassistant.helpers.script.websocket_api_script
Source: helpers/script.py:485
First occurred: 11:08:38 (1 occurrences)
Last logged: 11:08:38

websocket_api script: Error executing script. Unexpected error for call_service at pos 1: 'DAY' is not a valid DHWOperationMode
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 485, in _async_step
    await getattr(self, handler)()
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 723, in _async_call_service_step
    response_data = await self._async_run_long_action(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/script.py", line 685, in _async_run_long_action
    return long_task.result()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2279, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2316, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 892, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 962, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/config/custom_components/mypyllant/water_heater.py", line 208, in async_set_operation_mode
    DHWOperationMode(enum_value),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 744, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 1158, in __new__
    raise ve_exc
ValueError: 'DAY' is not a valid DHWOperationMode

Same here with latest version v0.7.3.
ecoCOMPACT VSC 206/4-5 90
VR 921 (Firmware Version 0357.40.33)
VRC 700
Same problem with switching to "AUTO":
Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:240
Integration: Home Assistant WebSocket API ([documentation](https://www.home-assistant.io/integrations/websocket_api), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22))
First occurred: 12:07:59 (1 occurrences)
Last logged: 12:07:59

[140303839772992] 'AUTO' is not a valid DHWOperationMode
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 240, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2279, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2316, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 892, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 962, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/config/custom_components/mypyllant/water_heater.py", line 208, in async_set_operation_mode
    DHWOperationMode(enum_value),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 744, in __call__
    return cls.__new__(cls, value)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/enum.py", line 1158, in __new__
    raise ve_exc
ValueError: 'AUTO' is not a valid DHWOperationMode

and different problem with "MANUAL":
Logger: homeassistant.components.websocket_api.http.connection
Source: components/websocket_api/commands.py:240
Integration: Home Assistant WebSocket API ([documentation](https://www.home-assistant.io/integrations/websocket_api), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+websocket_api%22))
First occurred: 12:10:16 (1 occurrences)
Last logged: 12:10:16

[140303839772992] 400, message='Bad Request, response was: {"title":"Bad Request","status":400,"detail":"400 BAD_REQUEST \\"Failed to read HTTP message\\""}', url=URL('https://api.vaillant-group.com/service-connected-control/vrc700/v1/systems/52108911-9b9b-474e-bcad-6815b5d9f2a9/domestic-hot-water/255/operation-mode')
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 240, in handle_call_service
    response = await hass.services.async_call(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2279, in async_call
    response_data = await coro
                    ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/core.py", line 2316, in _execute_service
    return await target(service_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 892, in entity_service_call
    single_response = await _handle_entity_call(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 962, in _handle_entity_call
    result = await task
             ^^^^^^^^^^
  File "/config/custom_components/mypyllant/water_heater.py", line 206, in async_set_operation_mode
    await self.coordinator.api.set_domestic_hot_water_operation_mode(
  File "/usr/local/lib/python3.12/site-packages/myPyllant/api.py", line 837, in set_domestic_hot_water_operation_mode
    return await self.aiohttp_session.patch(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 691, in _request
    await raise_for_status(resp)
  File "/usr/local/lib/python3.12/site-packages/myPyllant/http_client.py", line 49, in on_raise_for_status
    raise e
  File "/usr/local/lib/python3.12/site-packages/myPyllant/http_client.py", line 46, in on_raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1060, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Bad Request, response was: {"title":"Bad Request","status":400,"detail":"400 BAD_REQUEST \\"Failed to read HTTP message\\""}', url=URL('https://api.vaillant-group.com/service-connected-control/vrc700/v1/systems/52108911-9b9b-474e-bcad-6815b5d9f2a9/domestic-hot-water/255/operation-mode')

Obviously there’s no MANUAL mode in the official app for the VRC700, only DAY, AUTO and OFF.

Wanted to write the same thing just now.
You can switch to "OFF", but thats it, there is no way back to AUTO, or TAG/DAY. You have to use the App to revert back.

Please try https://github.com/signalkraft/mypyllant-component/releases/tag/v0.7.4b0

Please try https://github.com/signalkraft/mypyllant-component/releases/tag/v0.7.4b0

Thank you very much
I can confirm that this is working as expected. Changing Auto/Day/Off in Home Assistant changed the setting without any error and shows it in the MyVaillant app and vice-versa
But one more thing => "Manual" Mode is still present in Home Assistant and does not exist. If I press it, it also gives an Error message
[548261367232] 400, message='Bad Request, response was: {"title":"Bad Request","status":400,"detail":"400 BAD_REQUEST \"Failed to read HTTP message\""}', url=URL('https://api.vaillant-group.com/service-connected-control/vrc700/v1/systems/2f049a64-215f-45a4-843b-82f8126669f3/domestic-hot-water/255/operation-mode')
Which is obvious, because this Mode does not exist :-)

I've made another change to the modes and presets in https://github.com/signalkraft/mypyllant-component/issues/113#issuecomment-1958162371 - let me know if that improve things, or breaks things for you.

so... i have the VR920 as gateway
seems to work fine, but are there issues i have missed?
for water heating i understand the modes, Day, Auto and off
but what does Cylinder boost do?

@signalkraft
Regarding the Hot Water implemtation .... Hopefully the last questions from me :-)

Is it possible to remove Manual mode from the hot water implementation because as written by me or @rkeicher this mode just does not exist.

Also Cylinder Boost does not work - Error message: 'CYLINDER_BOOST' is not a valid DHWOperationModeVRC700

But this "should" work because in the MyVaillant App there is an option "Warmwasser-Boost" ... I just guess this is meant to be Cylinder Boost?

Please give https://github.com/signalkraft/mypyllant-component/releases/tag/v0.7.4b4 a shot, both issues should be fixed.

Ok, my bad. Try https://github.com/signalkraft/mypyllant-component/releases/tag/v0.7.4b5 please

and what does Cylinder Boost do that auto do not?

Ok, my bad. Try https://github.com/signalkraft/mypyllant-component/releases/tag/v0.7.4b5 please

It´s working now.
Only speaking for myself, but that should be it for hot water :-)

and what does Cylinder Boost do that auto do not?

I couldn't find an exact description but it is a "one time heating of the water with maximum power". Maybe it goes above the normal temperature and heats the water even hotter ... that´s all I could find.

Thanks for testing!


Wrong coordinates when embedded on scrollable page

I would like to embed excalidraw on a page that scrolls, while changing .excalidraw position from fixed to e.g. relative. Result:

codesandbox
is this supposed to work or am I missing something?

Haven't checked, but I think we'd have to update container offsets on body/window scroll event. Not sure if a good idea since it wouldn't catch cases where the excalidraw component is mounted in a non-body scrollable container. @ad1992 thoughts?
Be it as it may, you will be able to handle this yourself soon (when we release @excalidraw/excalidraw) by setting the offsets manually.

Haven't checked, but I think we'd have to update container offsets on body/window scroll event. Not sure if a good idea since it wouldn't catch cases where the excalidraw component is mounted in a non-body scrollable container. @ad1992 thoughts?

Haven't looked into this yet. But this is a valid use case so will look into this if it can be supported out of the box.

@vojtaholik you can try out @excalidraw/excalidraw.
closing this since this library is deprecated so we will fix the issues in the base repo.


Publish schemas to content store

As we want to remove Finder API from the stack we needed to find some other way of publishing Finders to the Content Store. Placing them into Specialist Publisher seemed like the most natural approach so this PR takes the steps required for this to happen. The following is contained within this PR:

Move the schemas and metadata from Finder API to a /finders folder and remove the old schemas
Change the file paths in the Wiring
Treat old style single select values as multi select
Add presenters for Finder Content Items
Add a publishing_api_finder_publisher class
Add a rake task to publish the Finders
Explain new finders and presenters folders in README

Amended based on @evilstreak's comments. We now use the file modified time as the timestamp for the content item.


Broken version: .dmg artifact built from CI and attached to beta26 release fails at launch

For now I believe this is a regression caused by monorepo update (#49), but exact cause is not clear.

The error appears when launching app from macOS disk image as built by CI and attached to release.
The error does not appear in development mode.
The error does not appear if the app/disk image is compiled and built locally

I spawned a new release branch from pre-monorepo commit, and the JS error doesn’t happen there. We might work in it for now.


ISPN-6907: Repl/Dist cache with Transactions doesn't respect shared store semantics

…ared store semantics
https://issues.jboss.org/browse/ISPN-6907
Once this has been accepted I'll issue a PR for this on the 8.2.x branch

Looks good to me.  @ryanemerson is there is a test we can tweak quickly to confirm this behavior?  If not I can integrate tomorrow anyways.

Looks good, pulling.

Integrated into master, thanks @ryanemerson !


Store events metadata

what is the goal of the origin field? Can't it be included in the metadata json?
The idea behind the origin field is to discriminate a bit between the different metadata.
Events from starknet will have different metadata than events from the API or from another chain.
But we cannot know which type of metadata it will be just based on the aggregate type itself. So this extra field will allow us to programmatically discriminate between this diversity of metadata schema.
It can be included in the json, but I think it's better to have it outside in order to know the target before deserialization.
It's also more high-level/domain-related than block_number or transaction_hash. Like timestamp it is common to all events, regardless of their nature/origin. For those reasons I think it's better to have it flattened.
I think the concept of event metadata should only exist at the infrastructure layer.
StorableEvent is defined in the domain. It contains everything to be stored with an event.
If we want to take metadata out of StorableEvent we change the Store trait
	fn append(&self, aggregate_id: &A::Id, events: Vec<StorableEvent<A>>) -> Result<(), Error>;
can become
	fn append(&self, aggregate_id: &A::Id, events: Vec<(StorableEvent<A>, json::Value)>) -> Result<(), Error>;
But this trait is still defined in the domain, so it changes nothing regarding your concern.


split \n doesn't work

split(variables('Test'), '\n') doesn't work. I clicked 'Peek Code' and saw it converted single slash to double slash ( "value": "@split(variables('Test'), '\n')"). Is that the reason why it doesn't work?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 03aa9e88-7fa0-56ab-f794-58f0c37b6d7a
Version Independent ID: 25c5ef5c-6ba4-f5ff-f94b-8f2818ec2feb
Content: Reference guide for functions in expressions - Azure Logic Apps
Content Source: articles/logic-apps/workflow-definition-language-functions-reference.md
Service: logic-apps
GitHub Login: @ecfan
Microsoft Alias: estfan

"value": "@split(variables('Test'), '\n')"

Test =
"test1
test2"

@linxlims Thank you for your feedback! We will review and update as appropriate.

@linxlims I've tried a couple of scenarios and I can confirm that changing the double slash to a single slash does the trick.
I believe escaping the slash is the expected behavior though it might not seem so in this case. Using the code view is the way to go for this scenario.

@linxlims Just following up here... Hope my previous comment clears things up.
Since there is on doc update required here, we will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.

@PramodValavala-MSFT Thanks for your response. I tried to modify it in the code view but it says it's a readonly editor... Any idea on this?