State max characters 255

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.57.0
Python release (python3 --version):
3.4
Component/platform:
sensor or input_text
Description of problem:
Before 0.57.0 i had a sensor which i populated with lyrics. This no longer works and throws an error that the max length of a state is 255 characters. Same applies for the new input_text component.
Expected:
Would like to be able to set the state to more than 255 characters, which was possible before 0.57.0
Problem-relevant configuration.yaml entries and steps to reproduce:

Traceback (if applicable):
2017-11-05 16:14:43 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.4/site-packages/aiohttp/web_protocol.py", line 422, in start
    resp = yield from self._request_handler(request)
  File "/srv/homeassistant/lib/python3.4/site-packages/aiohttp/web.py", line 306, in _handle
    resp = yield from handler(request)
  File "/usr/lib/python3.4/asyncio/coroutines.py", line 143, in coro
    res = yield from res
  File "/usr/lib/python3.4/asyncio/coroutines.py", line 143, in coro
    res = yield from res
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/http/ban.py", line 58, in ban_middleware_handler
    return (yield from handler(request))
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/http/__init__.py", line 423, in handle
    result = yield from result
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/components/api.py", line 225, in post
    hass.states.async_set(entity_id, new_state, attributes, force_update)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/core.py", line 752, in async_set
    state = State(entity_id, new_state, attributes, last_changed)
  File "/srv/homeassistant/lib/python3.4/site-packages/homeassistant/core.py", line 538, in __init__
    "State max length is 255 characters.").format(entity_id))
homeassistant.exceptions.InvalidStateError: Invalid state encountered for entity id: sensor.lyrics. State max length is 255 characters.

Additional info:

0.57 had a breaking change:
https://github.com/home-assistant/home-assistant/pull/9696
https://home-assistant.io/blog/2017/11/04/release-57/#breaking-changes
You can use state attributes for longer strings.

I'm unclear how to fix this issue.   My sensor takes the Last message (TTS) from MQTT topic and makes it available as a sensor value.    This value is routinely over 255 characters.  I'm can't find in the documentation how to create an attribute for the MQTT to get around the 255 character limitation.  Right now, it just errors out.
sensor:
  - platform: mqtt
    state_topic: "polly/lastmsg"
    name: "Last Message"

Any guidance would be appreciated.

I think you would need a custom component to couple mqtt subscription and setting a state with attributes.

Thanks for the response.   I'm probably an edge case so I'll just deal with the occasional error in my logs.  It would be nice though to be able to easily set the attribute entries for custom sensors via YAML.

still not clear how to get this :(


PactV4 - Same provider name for BDCT & Message based interactions

Scenario:
I have microservice which acts a provider (BDCT) with the pacticipant name "microservice-api". This same microservice will also acts as provider for message based interactions (2 event types - ServiceCreate & ServiceDelete). I planned to use the same pacticipant name "microservice-api" for message based interaction & BDCT. But unable to do so.
Issue:
When I use the same pacticipant name "microservice-api", the message based interactions are automatically getting verified based on the OAS document published as part of BDCT. But I was expecting that the message based interactions shouldnt be verified against OAS contract and wait until I run the Provider verifier tests for "microservice-api" events.
Workaround:
As per the discussion with the community support, we need this feature request for this item. For now, we can follow the work around of using 2 different provider names (one for BDCT & one for message based interactions).  https://pact-foundation.slack.com/archives/CLS16AVEE/p1689695352765749?thread_ts=1688139376.146759&cid=CLS16AVEE

Thanks for raising this @jegadeesan-ponnusamy.


echo $? doesnt give me a '0' in windows.

Okay sorry i know there's another closed thread mentioning something similar... but even when i use the 1.12.1 version... i still get the error of non 0 when i key in "$?"
#2212
#2211
(https://github.com/scikit-build/ninja-python-distributions/issues/157#issue-1437180712)
what other information should i look at to troubleshoot this?
C:\Users\kairi\gitclones>cd ninja

C:\Users\kairi\gitclones\ninja>dir
 Volume in drive C has no label.

 Directory of C:\Users\kairi\gitclones\ninja

18/08/2024  13:29    <DIR>          .
18/08/2024  13:29    <DIR>          ..
18/08/2024  13:29             1,080 .clang-format
18/08/2024  13:29               643 .clang-tidy
18/08/2024  13:29               163 .editorconfig
18/08/2024  13:29    <DIR>          .github
18/08/2024  13:29               670 .gitignore
18/08/2024  13:29               891 appveyor.yml
18/08/2024  13:29            10,365 CMakeLists.txt
18/08/2024  13:29            28,653 configure.py
18/08/2024  13:29             1,480 CONTRIBUTING.md
18/08/2024  13:29            11,560 COPYING
18/08/2024  13:29    <DIR>          doc
18/08/2024  13:29    <DIR>          misc
18/08/2024  13:29             2,677 README.md
18/08/2024  13:29             1,588 RELEASING.md
18/08/2024  13:29    <DIR>          src
18/08/2024  13:29    <DIR>          windows
              11 File(s)         59,770 bytes
               7 Dir(s)  41,561,018,368 bytes free

C:\Users\kairi\gitclones\ninja>git checkout release
branch 'release' set up to track 'origin/release'.
Switched to a new branch 'release'

C:\Users\kairi\gitclones\ninja>"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" x64
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.11.0
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\Users\kairi\gitclones\ninja>py configure.py --bootstrap
bootstrapping ninja...
warning: A compatible version of re2c (>= 0.15.3) was not found; changes to src/*.in.cc will not affect your build.
depfile_parser.cc
lexer.cc
build.cc
build_log.cc
clean.cc
clparser.cc
debug_flags.cc
deps_log.cc
disk_interface.cc
dyndep.cc
dyndep_parser.cc
edit_distance.cc
eval_env.cc
graph.cc
graphviz.cc
json.cc
line_printer.cc
manifest_parser.cc
metrics.cc
missing_deps.cc
parser.cc
state.cc
status.cc
string_piece_util.cc
util.cc
version.cc
subprocess-win32.cc
includes_normalize-win32.cc
msvc_helper-win32.cc
msvc_helper_main-win32.cc
minidump-win32.cc
getopt.c
ninja.cc
Generating code
Finished generating code
wrote build.ninja.
bootstrap complete.  rebuilding...
[35/35] LINK ninja.exe
Generating code
Finished generating code

C:\Users\kairi\gitclones\ninja>ninja --help
usage: ninja [options] [targets...]

if targets are unspecified, builds the 'default' target (see manual).

options:
  --version      print ninja version ("1.12.1")
  -v, --verbose  show all command lines while building
  --quiet        don't show progress status, just command output

  -C DIR   change to DIR before doing anything else
  -f FILE  specify input build file [default=build.ninja]

  -j N     run N jobs in parallel (0 means infinity) [default=8 on this system]
  -k N     keep going until N jobs fail (0 means infinity) [default=1]
  -l N     do not start new jobs if the load average is greater than N
  -n       dry run (don't run commands but act like they succeeded)

  -d MODE  enable debugging (use '-d list' to list modes)
  -t TOOL  run a subtool (use '-t list' to list subtools)
    terminates toplevel options; further flags are passed to the tool
  -w FLAG  adjust warnings (use '-w list' to list warnings)

C:\Users\kairi\gitclones\ninja>ninja --version
1.12.1

C:\Users\kairi\gitclones\ninja>echo $?
$?

C:\Users\kairi\gitclones\ninja>

The answer is simple, google is your friend...
When developing on Windows, you have to always keep in mind that it is posixly-stupid
$? is a POSIX standardized environment variable for exit status that does not exist in Windows, unless you were to create an alias for it on your own (but Windows might limit the type of characters that can be in a variable).
The first clue that you're not actually accessing a variable is that the content after echo is exactly the same as what is printed.
The variables in the command prompt are wrapped in % instead of starting with $
most of the IEEE documentation for $? is here, almost nothing in the following document applies to Windows:
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html
You have to access a variable called %errorlevel% instead:
https://stackoverflow.com/questions/334879/how-do-i-get-the-application-exit-code-from-a-windows-command-line

2 other things to keep in mind...
exit status is always a number, not just zero or "non-zero", so $? as a result is obviously invalid.
the exit status only applies to what is EXACTLY preceding the print of the exit status, not just the last "major" command or "only if there is a failure". If your example were to actually print the exit status when you intended to, it would only be testing the result of ninja --version and nothing else.
example:
$ something
-bash: something: command not found
$ echo $?
127

$ something; true
-bash: something: command not found
$ echo $?
0

$ true; something
-bash: something: command not found
$ echo $?
127

$ ^C
$ echo $?
130

$ something
-bash: something: command not found
$ ^C
$ echo $?
130

$ true
$ ^C
$ echo $?
130


Support for Yeelight Candela Light (BLE, Bluetooth Low Energy)

Good evening!
As for now, I have only one device that is not connected to HomeKit via this or any other plugin, and that is Yeelight Candela.
This lamp can be controlled via Bluetooth using the Yeelight app, and I know that Homebridge supports Bluetooth devices, since I did successfully connect Hygrothermograph via BLE using Homebridge Mi Hygrothermograph plugin.
But for this lamp, no supported plugin has been found for Homebridge, however, the plugin for Homey Pro exists.
I wonder if someone possibly could implement support for this device in the scope of Homebridge MIOT plugin.
It would be awesome. I'm sure that there're lots of people who bought this lamp and would like to use it via HomeKit

I know that some bluetooth devices also support the miot protocol, so in theory it should be possible. On the other hand i have absolutely no experience with bluetooth so it would require a lot of investigation for me, and i would also need a miot capable bluetooth device. The point is, i am not sure if investing time in that makes sense, since most of the bluetooth devices are battery based where frequent polling would just kill the battery so special handling would be needed...

It would be amazing! I’m waiting for yeelight candela control since 2018

@merdok I understand your concern, in any case, it's up to you to decide.
But I must say that these devices can be connected to Xiaomi Bluetooth Gateway, for obvious reasons, to poll climate data, so the battery life should not be an issue. I may be wrong here, but isn't the BLE protocol made specifically to save power and thus replaces polling with the broadcast?
Candela light, for example, in my case is always connected via USB, since I use it as a daily night lamp or ambiance lamp when watching movies, and compared to the BLE energy consumption, the LED consumption is significantly higher, so even with frequent polling, the difference should be negligible.
And you always may place a warning, so users will know that connecting such devices will reduce their longevity a bit, so that will be a choice to make.

Let see, maybe at some point i will have a look it...

Thanks so much!

I have a couple of lemesh.light.wy0c02 BLE Mesh Light running on the Xiaomi IoT Module-MHCB07P. These are very popular in china and no supported plugin has been found for Homebridge. Hope to see it available.

Well this is currently not my priority due to my goal to improve and stabilize the current wifi devices as much as possible so it might take a while till i can have a look at that (and because of the lack of such bluetooth devices).
I mean if someone has knowledge about that and wants to help implementing such functionality, then it would be great if they could implement the protocol class for managing the connection to the bluetooth devices which would definitely help here.

There are increasing number of Xiaomi smart devices using Bluetooth. I have 4 units of Xiaomi Smart Power Trip 2 (5 outlet), model  qmi.plug.psv3.  They connects to Xiaomi Gateway 3 with Bluetooth Mesh. Although Xiaomi Gateway 3 is dubbed as Homekit compatible but devices connected to Bluetooth Mesh do not appear on HomeKit.  I also have two units of Xiaomi wall switch ZNKG02HL work only with Bluetooth Mesh.
It would be really wonderful if this plugin could be expanded to manage the connection to the Bluetooth devices.

Below is the Json file of qmi.plug.psv3:
{"type":"urn:miot-spec-v2:device:outlet:0000A002:qmi-psv3:1:0000C816","description":"Single Control Outlet","services":[{"iid":1,"type":"urn:miot-spec-v2:service:device-information:00007801:qmi-psv3:1:0000C816","description":"","properties":[{"iid":1,"type":"urn:miot-spec-v2:property:manufacturer:00000001:qmi-psv3:1:0000C816","description":"Device Manufacturer","format":"string","access":["read"]},{"iid":2,"type":"urn:miot-spec-v2:property:model:00000002:qmi-psv3:1:0000C816","description":"Device Model","format":"string","access":["read"]},{"iid":3,"type":"urn:miot-spec-v2:property:serial-number:00000003:qmi-psv3:1:0000C816","description":"Device Serial Number","format":"string","access":["read"]},{"iid":4,"type":"urn:miot-spec-v2:property:firmware-revision:00000005:qmi-psv3:1:0000C816","description":"Current Firmware Version","format":"string","access":["read"]}]},{"iid":2,"type":"urn:miot-spec-v2:service:switch:0000780C:qmi-psv3:1:0000C816","description":"Switch","properties":[{"iid":1,"type":"urn:miot-spec-v2:property:on:00000006:qmi-psv3:1:0000C816","description":"Switch Status","format":"bool","access":["read","write","notify"]},{"iid":2,"type":"urn:miot-spec-v2:property:mode:00000008:qmi-psv3:1","description":"Mode","format":"int8","access":["read","write","notify"],"value-range":[0,1,1]},{"iid":3,"type":"urn:miot-spec-v2:property:temperature:00000020:qmi-psv3:1","description":"Temperature","format":"float","access":["read","notify"],"unit":"celsius","value-range":[-30,125,0.1]},{"iid":4,"type":"urn:miot-spec-v2:property:working-time:00000079:qmi-psv3:1","description":"Working Time","format":"uint32","access":["read","notify"],"unit":"minutes","value-range":[0,1440,1]}],"actions":[{"iid":1,"type":"urn:miot-spec-v2:action:toggle:00002811:qmi-psv3:1","description":"Toggle","in":[],"out":[]}]},{"iid":3,"type":"urn:miot-spec-v2:service:power-consumption:0000780E:qmi-psv3:1","description":"Power Consumption","properties":[{"iid":1,"type":"urn:miot-spec-v2:property:power-consumption:0000002F:qmi-psv3:1","description":"Power Consumption","format":"float","access":["read","notify"],"value-range":[0,20000000,0.01]},{"iid":2,"type":"urn:miot-spec-v2:property:electric-power:00000066:qmi-psv3:1","description":"Electric Power","format":"float","access":["read","notify"],"unit":"watt","value-range":[0,10000,0.0001]},{"iid":3,"type":"urn:miot-spec-v2:property:voltage:00000031:qmi-psv3:1","description":"Voltage","format":"uint32","access":["read","notify"],"value-range":[0,500000,1]},{"iid":4,"type":"urn:miot-spec-v2:property:electric-current:00000030:qmi-psv3:1","description":"Electric Current","format":"uint32","access":["read","notify"],"value-range":[0,100000,1]}]},{"iid":4,"type":"urn:qmi-spec:service:relay-count-down:00007801:qmi-psv3:1","description":"relay-count-down","properties":[{"iid":1,"type":"urn:qmi-spec:property:count-down-time:00000001:qmi-psv3:1","description":"count-down-time","format":"int32","access":["read","notify","write"],"unit":"seconds","value-range":[1,86400,1]},{"iid":2,"type":"urn:qmi-spec:property:remain-time:00000002:qmi-psv3:1","description":"remain-time","format":"int32","access":["read","notify"],"unit":"seconds","value-range":[0,86400,1]},{"iid":3,"type":"urn:qmi-spec:property:enable-count-down:00000003:qmi-psv3:1","description":"enable-count-down","format":"bool","access":["read","notify","write"]}]},{"iid":5,"type":"urn:qmi-spec:service:relay-loop-task:00007802:qmi-psv3:1","description":"relay-loop-task","properties":[{"iid":1,"type":"urn:qmi-spec:property:open-time:00000001:qmi-psv3:1","description":"open-time","format":"int32","access":["read","notify","write"],"unit":"seconds","value-range":[0,86400,1]},{"iid":2,"type":"urn:qmi-spec:property:close-time:00000002:qmi-psv3:1","description":"close-time","format":"int32","access":["read","notify","write"],"unit":"seconds","value-range":[0,86400,1]},{"iid":3,"type":"urn:qmi-spec:property:enable:00000003:qmi-psv3:1","description":"enable","format":"bool","access":["read","notify","write"]}]},{"iid":6,"type":"urn:qmi-spec:service:system:00007803:qmi-psv3:1","description":"system","properties":[{"iid":1,"type":"urn:qmi-spec:property:enable-led:00000001:qmi-psv3:1","description":"enable-led","format":"bool","access":["read","notify","write"]},{"iid":2,"type":"urn:qmi-spec:property:enable-buzzer:00000002:qmi-psv3:1","description":"enable-buzzer","format":"bool","access":["read","notify","write"]},{"iid":3,"type":"urn:qmi-spec:property:status:00000003:qmi-psv3:1","description":"status","format":"uint8","access":[],"value-list":[{"value":0,"description":"Normal"},{"value":1,"description":"Protected-OverCurrent"},{"value":2,"description":"Protected-OverTemperature"},{"value":4,"description":"Alarm-OverCurrent"},{"value":8,"description":"Alarm-OverTemperature"}]},{"iid":4,"type":"urn:qmi-spec:property:keep-relay:00000004:qmi-psv3:1","description":"keep-relay","format":"bool","access":["read","notify","write"]},{"iid":5,"type":"urn:qmi-spec:property:update-power:00000005:qmi-psv3:1","description":"update-power","format":"bool","access":["read","notify","write"]}],"events":[{"iid":1,"type":"urn:qmi-spec:event:alarm:00005001:qmi-psv3:1","description":"alarm","arguments":[3]}]},{"iid":7,"type":"urn:qmi-spec:service:calibration:00007804:qmi-psv3:1","description":"calibration"},{"iid":8,"type":"urn:qmi-spec:service:test-simulation:00007805:qmi-psv3:1","description":"test-simulation","properties":[{"iid":1,"type":"urn:qmi-spec:property:temperature:00000001:qmi-psv3:1","description":"temperature","format":"int8","access":["read","notify","write"],"unit":"celsius","value-range":[-100,100,1]},{"iid":2,"type":"urn:qmi-spec:property:current:00000002:qmi-psv3:1","description":"current","format":"int8","access":["read","notify","write"],"value-range":[0,100,1]}]}]}
~
Hope this helps.

This is the spec which is publicly available... I need actual device communication logs...

I'd add the BLE Alarm clocks to the list, the default app can't even automatically update DST through the hub... archaic stuff, a replacement would be lovely.
That aside, a Bluetooth HCI dump of certain actions would help?

I mean sure, you can provide one.

I found a way to add BLE devices and control it through mi cloud.
use cli print all mi cloud devices list without filter. you can get model name, token and did.but localip is empty.
then generate device js file with miot plugin. and move it to homebridge-miot/lib/modules/***/device/
set device config like this:
{
    "name":  "XXX",
    "ip": "<IP_ADDRESS>",
    "token": "<device token>",
    "deviceId": "<device id>",
    ....
}

and reload homebridge.

Yeah, I guess the recent changes where I removed the dependency for an ip to be required even when a device is set to a micloud only allows to make it work.
Also please note that it is now also possible to control Bluetooth devices which are connected to your bridge.

I found a way to add BLE devices and control it through mi cloud. use cli print all mi cloud devices list without filter. you can get model name, token and did.but localip is empty. then generate device js file with miot plugin(with require mi cloud checked). and move it to homebridge-miot/lib/modules/***/device/
set device config like this:
{
    "name":  "XXX",
    "ip": "<IP_ADDRESS>",
    "token": "<device token>",
    "deviceId": "<device id>",
    "model": "<device model>",
    ....
}

and reload homebridge.

May i know what did you use as the ip?

ip can be anything when using micloud, with a local connection you need to specify your bridge ip .

ip can be anything when using micloud, with a local connection you need to specify your bridge ip.

i got it working with micloud, however i am not able to control the BLE lights on a local connection with the homebridge ip.

Not homebridge ip, but your xiaomi mi HUB ip where all your devices are connected.

So it isn't possible to control it without a xiaomi hub?

No, without a hub not yet. That would require to implement the whole xiaomi Bluetooth miot protocol...

@merdok why don't you consider generate device class js file atomically when first run and cache it.

It is being already done. But for it to work the plugin needs the model information, so for Bluetooth mesh devices you need to manually specify the correct model in the config.json.

I found a way to add BLE devices and control it through mi cloud. use cli print all mi cloud devices list without filter. you can get model name, token and did.but localip is empty. then generate device js file with miot plugin(with require mi cloud checked). and move it to homebridge-miot/lib/modules/***/device/
set device config like this:
{
    "name":  "XXX",
    "ip": "<IP_ADDRESS>",
    "token": "<device token>",
    "deviceId": "<device id>",
    "model": "<device model>",
    ....
}

and reload homebridge.

hello
I followed your steps to add the Xiaomi Door and Windows Sensor 2 (isa.magnet.dw2hl) but I can only see the battery level of the sensor on homebridge, the other properties do not show up. Check the homebrigde log below.
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Initializing device services
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Device services: [
"magnet-sensor",
"battery"
]
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Initializing device properties
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Device properties: [
"magnet-sensor:illumination",
"magnet-sensor:contact-state",
"battery:battery-level"
]
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Initializing device actions
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Device not supported! Using a generic device with limited properties! Consider requesting device support!
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Initializing accessory!
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Accessory successfully initialized!
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Registering 1 accessories!
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Everything looks good! Initiating property polling!
[12/15/2022, 7:42:29 PM] [miot] [Door sensor] Log in to MiCloud!
[12/15/2022, 7:42:30 PM] [miot] [Door sensor] Successfully connected to MiCloud! Setting up miot device from MiCloud connection!
[12/15/2022, 7:42:30 PM] [miot] [Door sensor] Connected to device: isa.magnet.dw2hl
[12/15/2022, 7:42:30 PM] [miot] [Door sensor] Doing initial property fetch.
[12/15/2022, 7:42:30 PM] [miot] [Door sensor] Starting property polling.
can you help me with this?

That is already good news that you got it to work! Unfortunately as  you noticed, contact sensors are not supported yet but that should not be a big issue for me to add them. I will try to add them in the next update. For now you could use the propertyMonitor to get a trigger when the sensor is triggered.

That is already good news that you got it to work! Unfortunately as you noticed, contact sensors are not supported yet but that should not be a big issue for me to add them. I will try to add them in the next update. For now you could use the propertyMonitor to get a trigger when the sensor is triggered.

Thanks for the tip @merdok ! Setting the propertyMonitor I was able o get a trigger when the contact sensor is triggered. It has a 5 to 10 sec delay between when i close/open the door and the property state alters on homebridge. I guess that this delay is due to the reading's being done through Mi Cloud and not locally through the Xiaomi Smart Home Hub. Correct?
It's being interpreted by homebridge as a light sensor, and the states changes for 0 to 1 LUX according to the door is opened or closed.

Glad that you got it working!
The delay is because of the property polling, the state only updates periodically by the specified polling interval.

Thanks again!! I reduced the interval to 1 sec and it's working almost instantaneously!
Is there any way I can change how Homebridge "views" the device, and change it from a light sensor to a door sensor (ex.: change the icon, the units and device type)

@pedromcortereal You have to observe if 1 sec will not cause any issues for you.
The view of the device will be different as soon as i release the next update.

@ahmadnazirafiq you can check in this discussion as somebody else was asking that a couple of days ago: https://github.com/merdok/homebridge-miot/discussions/442

Hi @merdok! Do you think you could help us a little bit regarding how to set the property monitor and how to change the view in Hombridge? It is currently seen as a light sensor (I guess). Thanks!

You are missing the value which you expect. Setting a valueOperator to less or equal does not make sense as you did not specify what value should be taken to compare too.
Also if the Door Sensor is a numeric value then you have to specify what value is the triggered state.

{
                    "name": "Door Sensor",
                    "deviceId": "blt.4.1d07onrhkgg02",
                    "model": "isa.magnet.dw2hl",
                    "pollingInterval": 1,
                    "deepDebugLog": false,
                    "customAccessory": false,
                    "onlyMainService": true,
                    "buzzerControl": false,
                    "ledControl": false,
                    "childLockControl": false,
                    "modeControl": false,
                    "suppressAutoServiceCreation": [
                        "temperature",
                        "relativeHumidity",
                        "illumination"
                    ],
                    "propertyMonitor": [
                        {
                            "property": "magnet-sensor:contact-state",
                            "name": "Door Sensor",
                            "value": "1",
                            "valueOperator": "equal"
                        }
                    ]
                },

Still not working. Tried with value set to 0 as well and still no luck.

The value type is probably not a string, also I see above that the door sensor is a contact sensor so it seems that it is fine, isn’t it?

When I use the interface from the Plugin tab, it writes the value between apostrophes. Also, it doesn’t work because it will always say it’s closed (it appears in the Home app, but it doesn’t do anything).

Yeah the ui unfortunately has always the type set to strong so you need to edit those values manually in the config.
But did you validate that the property actually changes? Like with the miot cli commands?

So, I tried with 0, 1,  "0", "1", false, true, "false" and "true" and none of them works.
Regarding the CLI I tried the following:
root@denis-server:/var/lib/homebridge $ miot cloud device-info "blt.4.1d07onrhkgg02"
 INFO  Getting info for device with id blt.4.1d07onrhkgg02 from country cn...
 SUCCESS  Got device info from MiCloud
{
  "did": "blt.4.1d07onrhkgg02",
  "token": "censored",
  "longitude": "0.0",
  "latitude": "0.0",
  "name": "Door Sensor",
  "pid": "6",
  "localip": "<censored>",
  "mac": "<censored>",
  "ssid": "",
  "bssid": "",
  "parent_id": "",
  "parent_model": "",
  "show_mode": 1,
  "model": "isa.magnet.dw2hl",
  "adminFlag": 1,
  "shareFlag": 0,
  "permitLevel": 16,
  "isOnline": false,
  "desc": "",
  "extra": {
    "isSetPincode": 0,
    "pincodeType": 0,
    "fw_version": "2.1.0_0059",
    "needVerifyCode": 0,
    "isPasswordEncrypt": 0
  },
  "uid":<PHONE_NUMBER>,
  "pd_id": 2443,
  "password": "",
  "p2p_id": "",
  "rssi": 0,
  "family_id": 0,
  "reset_flag": 0
}
root@denis-server:/var/lib/homebridge $ miot test <censored> --token censored
 INFO  Connecting to device at <censored>
Could not connect to device, handshake timeout
 ERROR  Could not connect to the device at <censored>! Make sure the ip and token are correct!

When it is a bool, then value should be either true or false.
miot test is for testing the local connection.
You have to send the command to retrieve the property over mi cloud as it seems that you have that set up.

You mean like this?
root@denis-server:/var/lib/homebridge $ miot cloud get-props 2.1
 INFO  Executing get_properties command over MiCloud country cn with params 2.1 - timeout: 10000
(MiCloud) No result in response from MiCloud! Message: data type not valid
 SUCCESS  Response from MiCloud -> null
root@denis-server:/var/lib/homebridge $ miot cloud get-props 2.2
 INFO  Executing get_properties command over MiCloud country cn with params 2.2 - timeout: 10000
(MiCloud) No result in response from MiCloud! Message: data type not valid
 SUCCESS  Response from MiCloud -> null
root@denis-server:/var/lib/homebridge $ miot cloud get-props 3.1
 INFO  Executing get_properties command over MiCloud country cn with params 3.1 - timeout: 10000
(MiCloud) No result in response from MiCloud! Message: data type not valid
 SUCCESS  Response from MiCloud -> null

No, it has to be a raw parameter string.
Have a look at the robot cleaner docs, there is an example for miot send get_properties but the param string should be similar for the cloud command.

By the way, I grabbed this debug log:
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Preparing property poll!
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Splitting properties into chunks. Number of chunks: 1. Chunk size: 14
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Chunks:  [
 [
  "magnet-sensor:illumination",
  "magnet-sensor:contact-state",
  "battery:battery-level"
 ]
]
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] (MiCloud) Encrypted request https://api.io.mi.com/app/miotspec/prop/get - {"params":[{"did":"blt.4.1d07onrhkgg02","siid":2,"piid":1},{"did":"blt.4.1d07onrhkgg02","siid":2,"piid":2},{"did":"blt.4.1d07onrhkgg02","siid":3,"piid":1}]}
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Successfully updated property chunk! RAW: [{"did":"blt.4.1d07onrhkgg02","iid":"0.2.1","siid":2,"piid":1,"code":-704010000,"exe_time":0},{"did":"blt.4.1d07onrhkgg02","iid":"0.2.2","siid":2,"piid":2,"code":-704010000,"exe_time":0},{"did":"blt.4.1d07onrhkgg02","iid":"0.3.1","siid":3,"piid":1,"code":-704010000,"exe_time":0}]
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Error while parsing response from device for property magnet-sensor:illumination. Response: {"did":"blt.4.1d07onrhkgg02","iid":"0.2.1","siid":2,"piid":1,"code":-704010000,"exe_time":0}
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Error while parsing response from device for property magnet-sensor:contact-state. Response: {"did":"blt.4.1d07onrhkgg02","iid":"0.2.2","siid":2,"piid":2,"code":-704010000,"exe_time":0}
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Error while parsing response from device for property battery:battery-level. Response: {"did":"blt.4.1d07onrhkgg02","iid":"0.3.1","siid":3,"piid":1,"code":-704010000,"exe_time":0}
[7/20/2023, 12:04:05 PM] [miot] [Door Sensor] Device properties updated:
[
  "magnet-sensor:illumination: 0 | (not synced)",
  "magnet-sensor:contact-state: false | (not synced)",
  "battery:battery-level: 0 | (not synced)"
]

Yeah, there is an error during property sync, that would explain why you do not see the sensor update.
I do not know what that error code means, it comes from the server, not sure why… maybe the device cannot be reached?

I found this and from the translation (Bluetooth device cloud error - 704010000 Unknown, the plug-in is powerless.) of the first reply it seems that my battery is dead (which is weird because in the Mi Home app it works).

I also noticed that in the cloud device info from above the sensor is not online.
Instead of a cloud connection you could also try to do a direct connection over the gateway and check if that helps.

Any idea how to do that?

The ip and token needs to be from your gateway, the deviceid and model needs to from the device connected to the gateway which you want to control.

Tried it. It doesn't work.

What does the debug log say in that case? Did you tried to do a test connection using the miot cli?

The ip or token which you provided are most probably incorrect

What makes you say that?

Because the plugin cannot connect to the device, this is usually caused by an incorrect token or ip address.
The token in the cloud can be outdated if you did not sync it.

@kikiwora Also wanted to control my Candela lamps, so wrote this plugin https://github.com/vsternbach/homebridge-yeelight-ble and you're welcome to try it.
Note that unfortunately I couldn't make it stable enough to run purely on nodejs with noble, so adapted some python library for bluetooth manipulation for Yeelight BLE devices that serves as a backend for my homebridge plugin.


firebase ios15 cannot receive offline push

firebase ios15 cannot receive offline push,but android and ios15- works well

cannot receive offline push

Do you mean when we push a notifications in offline mode, it isn't received on iOS, but received on Android ?

android and ios15- works well

Did you mean to say iOS 14 or iOS < 15 ?
Can you provide flutter doctor -v, messaging plugin version, a sample payload you are sending along with any error log you get while doing so ?

Channel beta, 2.2.2

This is very old beta version. Please upgrade to latest beta or master and try again to see if behavior still persists or not.

Sorry, I can't upgrade firebase_ Messaging version, because my fluent SDK needs to be upgraded, which will cause many problems. But now I've figured out that it's not the IOS mobile phone system. It's because some mobile phones can't get tokens. Here are some logs
Failed to fetch default token Error Domain=NSURLErrorDomain Code=-1200 "发生了SSL错误，无法建立与该服务器的安全连接。" UserInfo={NSErrorFailingURLStringKey=https://device-provisioning.googleapis.com/checkin, NSLocalizedRecoverySuggestion=您仍要连接此服务器吗？, _kCFStreamErrorDomainKey=3, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <2E69AB9E-0BDD-4A6F-BE98-0DD8A5CB6D77>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(

Failed to fetch default token Error

Are you using vpn ? If so, it could be happening due to it. Try removing the vpn and use regular network and see if it helps.

I have the same problem


Small fixes to sample.

Got some suggestions for fixes/changes to the sample from an external partner, this PR takes some of these small fixes and removes some dead code.

In general, changes should target the develop branch rather than the main branch. However, we just merged develop into main and we are expecting some more changes before doing an additional merge, so I'll complete this PR.

Hi @judzmura  Thank your taking in these fixes. However I think you forgot the commit [bab2233] (https://github.com/microsoft/Windows-driver-samples/pull/1215/commits/bab2233e85d57a73b355d28d1365dbf5cfbee4ca) from the original pull request. Commit adds missing stuff on the inf file that is referenced by the sample code (most importantly configuration registry keys)


Add URL mixin

Adds a URL mixin. Also removed the relatively useless test

Not sure about this one since it adds another dependency of django-cms.

I think this is an ideal candidate for dependency detection: the internal link can be a PageField if settings.INSTALLED_APPS contains cms, but a URLField if not. I would expect you need to mirror the if statement in the migration and document well.

I think this is an ideal candidate for dependency detection: the internal link can be a PageField if settings.INSTALLED_APPS contains cms, but a URLField if not. I would expect you need to mirror the if statement in the migration and document well.

I've also seen packages have a variant. Requests does it with the security package where if you use requests[security] it pulls down the additional packages. We do something similar and have mixins[cms].


Add favicon

Now that we have the logo, a favicon would be great!

Done with commit 8c141df2113233b5ce5eb103a74d2f3f48e31cac


Possible to simply adjust error schema (eg monkey patch)?

Looking to remove "detail" as a key in error dict for verify, refresh, blacklist and have the error output same as other endpoints. Don't want to rewrite a ton of stuff was hoping there might be a way to monkey patch it.

You can monkeypatch it here: https://github.com/jazzband/djangorestframework-simplejwt/blob/e67944ad8199ae02515dd46a2ba7700890ee1d31/rest_framework_simplejwt/exceptions.py#L13


NATIVE-362: Make sure failures in the FetcherModule do not render the app unusable

Please see the bug description: https://issues.integreat-app.de/browse/NATIVE-362
Can you think of an other reason this happened?


Configuration API for v1.23

/hold
Reference target is not correct.

/hold cancel

These changes look ready to merge. Thanks @tengqm .
/lgtm
Feel free to release the hold.
/hold

Releasing the hold for follow up work.
/hold cancel


Added doas-sudo - inserts doas or sudo before current command

Small function that adds sudo (or doas on OpenBSD), before a command.
Using !! command is not an equivalent, as you can use this function before you made the mistake of forgetting sudo.
The short-cut is in line with other zsh short-cuts, like <esc>h to open a man-page.
The snippet should follow coding guidelines etc, but let me know if not.

My only issue with this implementation is that I personally spam the <ESC> key on occasion. May I suggest a different, more deliberate bind such as <ESC>!?

Sure, they're close enough on the keyboard, and if I really hate it I can override it from my .zshrc. I'll update my pull-request.

Great, squash those commits, reference this as closing #56 and use the commit format "[input] add doas/sudo insert bind". Thanks!

@aplatest Once you squash the commits (rebase with git, then force push) I'll merge this. Thanks again!

Oh My Zsh and Prezto all became full of specific configurations and customizations and got far away from being a simple and fast set of good defaults for Zsh. I'm just moving from Prezto to Zim and I was happy to see how Zim is a simpler subset of Prezto (I even compared some similar files to find that Zim introduced some few good additions and removed lots of more complex code). Shouldn't Zim be kept slim, avoiding request like this that introduce 17 lines of code just to have a custom variation of what could be done with sudo !!... It's cool, of course. But it's also very specific. And is it worth an addition of 17 lines of code?

Since there seems to be some opposition (which has persuaded me), I'm going to hold off on merging this. If there are users who want this functionality, please give the original comment a "thumbs up" so I can get a feel for how many people would find this useful.

Just for reference, this was discussed before in https://github.com/Eriner/zim/issues/16

Since the changes discussed in #88 seems like the way zim is headed, wouldn't this commit be better fit for an external module (or perhaps a openbsd module)?

I think the idea for an openbsd module is the best route to take here.


feat(*): Add `arrayName functionality

As discussed in  #11, added functionality to specify array name of subscriptions.
Also did a bit of clean up. Let me know if I need to put some of these things back.

Wasn't sure if you wanted me to update the version or if you would do that.

Thanks, looks really good. We need to get back the yarn.lock file and you can also bump the version. ( 2.4.0 )

@NetanelBasal I bumped the version. It looked like the yarn.lock file wasn't being used as there's also a package-lock.json. Can we used one or the other?

Closes #15.


Removed zfill in data.py

Padding month and days that are less than 2 digits with a leading zero is now done with the str.format() syntax.

@johnwatterlond looks good! Can you just fix that merge conflict in README first and then I will accept the PR.

I fixed it, merging now.


BFS.py

The graph search algorithm in python

Hey @Ivikassingh bfs and dfs are already implemented for python. You can work on any other algo/ds but be sure to make an issue for it first 😄


Conflicts between pymacaron-async and pyyaml

Hi, users are unable to run pymacaron-async due to dependency conflict with pyyaml package. As shown in the following full dependency graph of pymacaron-async, pymacaron requires pyyaml >=5.1.2，while pymacaron-core requires pyyaml ==5.1.2.
According to pip’s “first found wins” installation strategy, pyyaml 5.2 is the actually installed version. However, pyyaml 5.2 does not satisfy ==5.1.2.
Dependency tree-----------
pymacaron-async - 0.0.56
| +- celery(install version:4.4.0 version range:*)
| +- pycurl(install version:<IP_ADDRESS> version range:*)
| +- pymacaron(install version:1.0.213 version range:*)
| | +- click(install version:7.0 version range:*)
| | +- flask(install version:1.1.1 version range:>=1.0.4)
| | | +- click(install version:7.0 version range:>=5.1)
| | | +- itsdangerous(install version:1.1.0 version range:>=0.24)
| | | +- jinja2(install version:2.10.3 version range:>=2.10.1)
| | | | +- markupsafe(install version:1.1.1 version range:>=0.23)
| | | +- werkzeug(install version:0.16.0 version range:>=0.15)
| | +- flask-compress(install version:1.4.0 version range:*)
| | +- flask-cors(install version:3.0.8 version range:*)
| | | +- flask(install version:1.1.1 version range:>=0.9)
| | | | +- click(install version:7.0 version range:>=5.1)
| | | | +- itsdangerous(install version:1.1.0 version range:>=0.24)
| | | | +- jinja2(install version:2.10.3 version range:>=2.10.1)
| | | | | +- markupsafe(install version:1.1.1 version range:>=0.23)
| | | | +- werkzeug(install version:0.16.0 version range:>=0.15)
| | | +- six(install version:1.13.0 version range:*)
| | +- pyjwt(install version:1.7.1 version range:*)
| | +- pymacaron-core(install version:1.0.164 version range:>=1.0.146)
| | | +- flask(install version:1.1.1 version range:>=1.0.4)
| | | +- bravado-core(install version:5.1.2 version range:==5.13.2)
| | | +- PyYAML(install version:1.0.164 version range:==5.1.2)
| | +- pymacaron-unit(install version:5.1.2 version range:>=1.0.10)
| | +- pytz(install version:2019.3 version range:*)
| | +- pyyaml(install version:5.2 version range:>=5.1.2)
| +- pymacaron-unit(install version:1.0.16 version range:*)
| +- redis(install version:3.3.11 version range:*)

Thanks for your help.
Best,
Neolith

Suggested Solution

Ask your upstream project pymacaron to fix his direct dependency to be pyyaml ==5.1.2. I have checked this revision will not affect your downstream projects now.
Ask your upstream project pymacaron to remove his direct dependency pyyaml, and use pyyaml transitively introduced by pymacaron-core.

@erwan-lemonnier Which solution do you prefer, 1 or 2?
Please let me know your choice. I can submit a PR to solve this issue.


DataReceived never goes off & more

I'm making a simple chatapp with TCP.
Connected was fired perfectly, and I could send my JSON.
I was finished with the backend, and I added some stuff to DataReceived, but it didn't fire even though I know I'm sending json.
 They after a second of doing this stacked up and became from {hello:info} to this {hello:info}{hello:info}{hello:info}
Thanks for maintaining!,
Luke.

 At first glance you may want to use WatsonTcp instead since it provides framing. Will respond in more detail later￼

Hi @lukeacat sorry for the delay.
The reason you may be seeing multiple messages is that SimpleTcp provides no framing layer.  What this means is that your client might be sending a 12 byte message (i.e. {hello:info}) but it's just going into a big memory buffer on the receiver side (server) and the server has no idea how many bytes to read from the socket.  SimpleTcp just has a background loop that continues tries to read as much as it can (up to the amount specified in Settings.StreamBufferSize, in bytes), and when it gets some amount of data back from the socket, it fires DataReceived.
Framing (see https://blog.stephencleary.com/2009/04/message-framing.html) encapsulates your data with a smaller header that indicates how many bytes of data are being sent.  In your case, framing your {hello:info} message could be as simple as 12:{hello:info}, and the server could read one byte at a time until it hits the delimiter (i.e. :), and parses what it has read into an int, and then it knows to read 12 bytes, i.e. {hello:info}.
So, what is likely happening is the TCP layer is receiving data from the server and making it available for read.  When SimpleTcp receives that data, it drains as much as it can (up to, again, Settings.StreamBufferSize) and fires the event.
If you need message-by-message events, you need framing - there's no way around that.  That's why I would recommend looking at WatsonTcp.
Cheers, Joel

This doesn't address the DataReceived issue. That's what I was mostly concerned about.
I already made a working fix for framing on the server side.
Thanks, Luke!

DataReceived will only fire when there is data actually read from the socket.  Is there any way you could provide some sample code that reproduces this?
Also the framing on WatsonTcp is pretty straightforward.  I could walk you through how to implement framing to support it if you've already done it client side.

Hmm, I can put my game on Github (It's in its early stages). I'll report back, since the project should be opensource.

https://github.com/lukeacat/SimpleGame Here's the project!
Thanks for maintaining as always, Joel!

Even more Info:
When using the provided node.js client (test.js) inside the backend folder with a name, everything works how I want it to work.

Another clue: I tried this with Websockets (I recoded the backend and tried websockets)
and the same thing is happening.. WTF?

(talking about websockets)

Mm, I am so confused. I started the server in Administrator mode right? Yeah, I did that, but the c# client didn't work. Ok, well that's fine. Now, I made a quick web browser client, and it WORKED! It doesn't work with normal permissions.. What's going on?

Hi @lukeacat in https://github.com/lukeacat/SimpleGame/blob/master/frontend/Form1.cs can you try using await client.SendAsync?
Am I correct in assuming that you're only using SimpleTcp on the client side?  The only thing I could think of is that more than one message is making it into the receive buffer in between attempts to read from the socket.  That would cause data to be bundled together in the DataReceived event.
Firing the DataReceived event can cause a (very, very minor) delay between socket reads, and if two messages arrive in the socket between these reads, data can be commingled.  Another reason why framing is really important (side note: websockets do provide framing natively).
I'm having a really hard time reproducing the issue even when running on separate nodes, but I'm also using the SimpleTcp client and server (not the JS backend you have in your repo).
Is there a way in your backend to minimize the runtime of socket.on('data', function(data) {?  I imagine that with JS being single threaded, anything introducing a delay in the completion of this event will allow data to queue up in the receive buffer from multiple client sends.  It looks like the code 1) parses JSON to object and then 2) on "connect", broadcasts a message to everyone (which may add to the runtime of the method)
Thoughts?  Cheers, Joel

Hey Joel, I'm closing this in light on my recent discoveries:
 What went wrong?:
Earlier, I got CS0120 on the DataReceived part (theForm was made in a different thread).
This meant that dataRecieved was started.
How'd you fix it?:
I added await to client.sendAsync, and that's all I really did.
I don't know what went wrong last time, but now it works.
Also, to debug I added MessageBox.Show().
Does it work now?:
Well, we still don't know what happened, but everything else seems fine for now.
Answering those questions:
1. Yes, I'm only using STCP on Client Side.
2 & 3: Desync is possible, but it shouldn't drop entire messages. Should it?
4. Minimizing runtime in JS isn't really possible, it's as good as you write as good as you get.
Thanks for your continued help Joel,
luke out!

Hi @lukeacat wow, well, at very least I am really glad it's working for you now.
I can't speak to why messages got dropped, that indicates that they didn't arrive, or, an exception is being encountered.  You can try setting client.Logger to output to a messagebox, console, etc, and see if something shows up there that might indicate if there is an exception.  I'm betting that await was keeping the thread waiting on the completion of SendAsync, and what was likely happening before was that void Connected was exiting prior to the unawaited SendAsync completing.  I'm doubtful that void Connected completing prior to unawaited SendAsync would cause the task to be garbage collected, but I could be wrong (not an expert in forms/WPF and threading).
Cheers Luke and have a good one!


MB-15796: TOO can view review request service item update modal

Jira ticket
Summary
This PR uses the customizable modal @afoon create here to create a preliminary modal that shows the HHG tag as well as the title Review request: service item update because of @afoon changes, the service item details portion is already included even though that was not a part of the AC of the original ticket.
Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers

[ ] HHG Label is displayed in top left corner:
[x] Title of modal is Review request: service item update
[ ] Has unit tests

Setup to Run the Code

Instructions for starting storybook
Instructions for starting the MilMove application
Instructions for running tests

How to test

Access http://officelocal:3000/testharness/list and scroll down to button that says HHGMoveInSITWithAddressChangeRequestOver50Miles then click it.
Copy value of locator field
Access the Office App
Login as a TOO
Navigate to move with value of locator field you copied earlier
Click Move task order tab
Scroll down to Review Request button in approved service items portion and click it

Frontend

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, Edge).
[ ] There are no new console errors in the browser devtools.
[ ] There are no new console errors in the test output.
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.

Screenshots
Review Request Modal

Messages

:book:
:link: MB-15796

  Generated by :no_entry_sign: dangerJS against 38c38837473bfdd852dc7cde0519b3c5530fdae2

The review modal looks good but I am not able to check the edit modal because the functionality isn't in place yet.

@afoon @AnnaGingle maybe I'm misunderstanding the ticket? Because in the implementation notes it says that this ticket should not include the SIT Address Information nor anything about the edit modal.

@StuTruss The A/C is confusing me as well. I'm not sure why this story has less A/Cs than the Edit Address Form which also includes the form and functionality.

I changed the base branch to make it easier to see what changes were made just on this PR. When the Edit Address Modal PR is merge into main, Github will automatically base it back to main again.
Additionally, if ReviewRequestServiceItem work is complete before Edit Modal is ready to merge, we can merge the ReviewRequestModal changes into Edit Address Modal branch. Then changes for both PRs will be merged into main at the same time.
cc: @rogeruiz @StuTruss @donaldthai

I've gotten confused by the two PRs, so I'm going to merge this into @afoon's PR so I'm working from a single branch.


Include mapping statistics

The authors should include the number of reads that mapped back to the assembly at their 1 kb cutoff, as this would help us gauge the inclusivity of the assembly for both the DNA and RNA reads.

Post-QC alignment statistics for all paired-end reads:
RNA

72,049,030 paired-end reads
56,564,095 mapped (78.51%)
52,876,306 properly paired (73.39%)

DNA

131,320,884 paired-end reads
100,539,905 mapped (76.56%)
92,908,932 properly paired (70.75%)

No mappings for post-QC orphan reads: 6M DNA and 1M RNA reads.
If we count these as unmapped, we're slightly underselling our results:
RNA

73,165,986 whatever reads
56,564,095 mapped (77.31%)
52,876,306 properly paired (72.27%)

DNA

137,365,053 whatever reads
100,539,905 mapped (73.19%)
92,908,932 properly paired (67.64%)

Thus, I just started the Bowtie2 mapping of the orphan reads.

Stats for the orphan reads, quickly pasted before I have to leave:
RNA
1116956 + 0 in total (QC-passed reads + QC-failed reads)
871963 + 0 mapped (78.07%:-nan%)
DNA
6044169 + 0 in total (QC-passed reads + QC-failed reads)
4921691 + 0 mapped (81.43%:-nan%)

Adding up the numbers:

RNA
73,165,986 reads
57,436,058 mapped
78.5 %

DNA
137,365,053 reads
105,461,596 mapped
76.8 %


Proposal: Make SLEIGH a standalone library

Hi there! Firstly, thanks so much for developing Ghidra.
To give a bit of background, I'm an engineer at Trail of Bits and am part of their research practice that develops binary lifting tools among other things. We've been interested in building binary analysis and lifting tools on top of SLEIGH, which led us to putting together this CMake project for SLEIGH.
At the moment, projects that use SLEIGH tend to independently vendor the sources under this directory and maintain their own build scripts, wrappers, bindings, etc. Another example of this is the angr project, which uses its own pypcode library, that maintains a set of build scripts and Python bindings.
I think it'd be great for the ecosystem if we could separate SLEIGH into its own official library so it can be used elsewhere, with Ghidra being the main user.
I don't have a firm idea of how this should work, but I wanted to get a feel for whether you're interested in this idea. If so, I'd be happy to partner with the Ghidra team to help make this happen.

Consider this a co-signature and I would be happy to contribute as well. I would love to see libsla usable in a Capstone-like fashion, but have had just enough trouble root-causing some of the parsing issues that come up that I've had to back-burner it for the time being.

I would love to see this happen!
Related, this popped up on my radar recently. Just dropping it here for context.
https://github.com/jstaursky/coronium

just so u guys know, i recently got this in my recommendations , maybe something worth checking out https://github.com/lifting-bits/sleigh

Thanks @UwUeeb!  Since this hasn't been on our roadmap and someone provided an effective and well-maintained solution, I am going to go ahead and close this.

@ryanmkurtz
That repo was created by my colleagues and I, and is what lead me to file this issue to begin with. It's been useful to us but, in my opinion, it doesn't address all of the issues that come with SLEIGH being maintained within Ghidra. If you agree, could you please re-open this?
Firstly, it's a maintenance burden keeping it up to date and is easy to get wrong. We periodically sync with the HEAD of Ghidra and run some testing in CI to ensure that things continue working. But even then, there are ways for things to silently fall out of sync. For example, we recently noticed that our list of headers that we install under the library's include/ were out of date (https://github.com/lifting-bits/sleigh/pull/107). We're now considering adding some script to check for this, but I expect that these sorts of issues will continue to crop up.
Additionally, our version of the SLEIGH library strips out some dependencies (BFD, ZLib, Iberty) to make it more practical to build on different platforms. At the moment, this works fine because these headers are included in files that contain optional implementations like LoadImageBfd, however there's nothing to guarantee that it will remain this way since our library-ification of SLEIGH happens independently to Ghidra development. I'm concerned that if anything happens to make these dependencies harder to separate (perhaps SLEIGH begins including a BFD header elsewhere), this is going to be hard to deal with.

I will reopen it since there could be a future where we have a need to make sleigh into a standalone library, but as I mentioned it is not currently on our roadmap.

@ryanmkurtz Can you clarify whether you (and the team) are open to further discussing, reviewing, and accepting changes to the C++ decompiler/sleigh code to make third-party solutions (such as lifting-bits/sleigh, for which I am a maintainer) easier to maintain and package?
Some improvements I have in mind:

Restructuring the project directory into a more traditional organization (example: Makefile, src/, include/, test/unittests/, test/datatests/)

While this isn't strictly necessary, it's a bit weird for the Makefile to reference the parent directory.

Separating header files from source files (as mentioned in an earlier comment) into include/ghidra/*.hh (for example)

This is useful when the decompiler headers are included in another project to prevent conflicts with header names, i.e. #include <ghidra/type.hh> instead of #include <type.hh>. The actual directory names could be up for discussion.

Add missing header guards to prevent double-inclusion

Open PR that requires some direction/comments to resolve ambiguity https://github.com/NationalSecurityAgency/ghidra/pull/4479

More clean-up of using declarations in header files.

While https://github.com/NationalSecurityAgency/ghidra/pull/4715 is technically closed and solved by https://github.com/NationalSecurityAgency/ghidra/commit/c2522c81c6f7eae1c3ba63bc0995332816f15b24 the using std::<symbol>, still pollutes consumers that include the headers.
Fortunately, I spent some time writing a tool (RemUsing) to automate almost the entire refactor of replacing the symbols from the standard library with their fully-qualified name, i.e. string -> std::string.

As evidence, I used RemUsing to automate this refactoring to generate the patches in https://github.com/lifting-bits/sleigh/pull/167 (happy to discuss this more)

I understand some of these "improvements" are subjective. Still, I'm happy to discuss why and how each is generally the best practice for C++ development and that making the changes will encourage community engagement.
I also understand that reviewing PRs and discussing these things takes time. However, I hope that if the implementation is completed (and reviewed) by the community and the implementation/idea doesn't drastically affect the Ghidra team's internal development, then we can all benefit.

Also, I assume API changes and code refactoring to support more flexible library-like use-cases for the decompiler would be addressed on a case-by-case basis. Like the comment mentioning a capstone-like usage of the disassembler, I had trouble trying to cleanly implement this feature in the trailofbits/mishegos tool. I had to rewrite some of sleigh.cc to remove the disassembler caching mechanism that couldn't be removed easily using C++ language features or sleigh APIs.
Are these sorts of PRs, to improve the flexibility of sleigh/decompiler as a library, open for inclusion in this project?

If there's a better way to organize or track individual requests, I can open separate issues or PRs for more isolated discussion. Please let me know if/how I should do this.
Thank you for your time!

@ekilmer Sorry for the delay, I wanted to bring this up to the team for discussion so I could respond appropriately (I am not actually in charge of anything).  Generally speaking, we are very open to making Ghidra as extensible, shareable, accessible, etc. as possible.  However, doing so requires work like everything else...sometimes a lot of work since the codebase is very old and established.  So, it comes down to fitting it in with the priorities of all the other work we need to do.
We recognize that this ticket has value to the community...it's gaining momentum but not to the point where we are ready to start working on it ourselves.  Having said that, we are open to accepting PR's that would improve this.  Ideally they would be trickled in in manageable chunks that are easy to review, with priority given to the pieces that have the most impact.  These PR's will have to then be prioritized against all of our other work, but we are open to it.  It would help if the PR described the impact it would have so we can prioritize more appropriately.
Some other things:

Have you considered using the full blown Ghidra release in your workflow using say, headless and GhidraScripts?  Is that just too slow performance-wise?
Rather than using the C++ stuff, have you considered just using the SoftwareModeling jar?  (someone on the team suggested this).
Regarding #4715, we didn't know that it was solved insufficiently.  This might be a good place to start with a PR.


Revert "[NoQA] Fix the new contributor PR comment"

Reverts Expensify/App#8082

cc @mountiny

Broken workflow run: https://github.com/Expensify/App/actions/runs/1995172499

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @chiragsalian in version: 1.1.44-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[User Interface: Redesign]: Repository-level Metadata Compatibility badges

Issue Name
Repository-level Metadata Compatibility badges
Issue Description
According to the NIAID team, a repository owner was excited about the Metadata Completeness badge, but lamented that it was not informative enough. This prompted the NIAID team to request improvements to the legend for the metadata completeness badge suggesting the inclusion of missing fields. Since missing fields can be repository-limited, it makes more sense for this sort of information to be provided at the repository level. Towards that end, we will create a repository-level metadata compatibility badge which will help visualize how compatible repositories are with the metadata ingested by the NIAID Data Ecosystem.
Note, this issue is for the design of the badge only. Where the badge will be displayed has yet to be discussed. Potential locations include the sources page, and the 'featured pages' which have yet to be developed.
Dependencies
The data needed for this issue will be discussed in the following GH Ticket:
https://github.com/orgs/NIAID-Data-Ecosystem/projects/6/views/2?pane=issue&itemId=53297771
Issue Discussion
The history of this ticket can be sourced to this issue: https://github.com/orgs/NIAID-Data-Ecosystem/projects/6/views/5?pane=issue&itemId=50899235
Mockup URL
https://nde-dev.biothings.io/compatibility
WCAG Compliance Check

[ ] The ui changes are not expected to significantly affect WCAG compliance (ex- wording changes)
[ ] The ui changes have visual elements which have yet to be tested for WCAG compliance
[x] The ui changes have visual elements which have been tested for WCAG compliance
[ ] The ui changes have passed all checks for WCAG compliance

Related WBS task

[ ] Check here if the ui change is centered on the look and feel (https://github.com/NIAID-Data-Ecosystem/nde-roadmap/issues/52)
[X] Check here if the ui change includes new features (https://github.com/NIAID-Data-Ecosystem/nde-roadmap/issues/53)
[ ] Check here if the ui change was suggested by the NIAID team (https://github.com/NIAID-Data-Ecosystem/nde-roadmap/issues/2)

For internal use only. Assignee, please select the status of this issue

[ ] Not yet started
[ ] In progress
[ ] Blocked
[ ] Will not address

Status Description
No response
UI change status check list

[ ] This ui change has yet to be discussed between NIAID, Scripps, Leidos
[X] This ui change has been discussed/reported between NIAID, Scripps, Leidos
[x] This ui change has been mocked up
[ ] This ui change has been implemented on Dev
[ ] This ui change has been approved for Staging
[ ] This ui change has been implemented on Staging
[ ] This ui change has been approved for Production
[ ] This ui change has been implemented on Production

@rshabman @sudvenk @lisa-mml @hartwickma we have a few potential designs for the repository-level metadata compatibility badge which can be found at: https://nde-dev.biothings.io/compatibility
An initial draft of the documentation for the repository-level metadata compatibility badge can be found at: https://docs.google.com/document/d/1Q1_S3x239slpAhMpOnNTJKLzdc9ik7Up4H6Dty-U7xs/edit?pli=1 . This draft is expected to be updated with images that explain the badge, once a badge type has been selected.
We are changing the status of this issue to Review requested-deliverable and would like to include this in the agenda for the biweekly discussion on 2024.04.16 with OCGR.

Per the discussion on 2024.06.11, the preferred badge is the bar badge; with the following suggested changes:

Rather than using value-based coverage which depends on % coverage of metadata in the repo, and can be harder to interpret with all the different shades of pink/purple, provide a simple indicator of is it available or not
Remove the % coverage from the hover info, since there will not be shades of color to reflect % coverage
The % coverage information is interesting, so leave it in the metadata as we may want to revisit this matter at another time, but remove it from the badge
The order of all metadata fields in the badge should be kept consistent between the repositories to allow users to quickly evaluate if a repository collects certain metadata
The metadata compatibility documentation should be updated to reflect a positional changes from a way for seeing the coverage of metadata in each repository as a potential indicator as to whether the repository allows for metadata (in case a user is looking for a place to deposit their dataset)

@rshabman @sudvenk @lisa-mml @hartwickma we will focus on updating the bar chart mock up to reflect suggestions 1-4. We will update the documentation for the repository-level badge based on suggestion 5.

@rshabman @sudvenk @lisa-mml @hartwickma The bar chart/grid plot hybrid has been updated to reflect the requsted changes and can be viewed here: https://nde-dev.biothings.io/compatibility
If approved for implementation on Staging, these 'badges' would be added to the 'Sources' page

Hi @gtsueng thank you for providing the updated mockups, these look very nice.
One of the requested updates:
Remove the % coverage from the hover info, since there will not be shades of color to reflect % coverage
but this information still appears in hover boxes, where in some cases, a solid box is at 86% (Date, Access Clinical Data) and DDE Fundamental shows all boxes filled but reports 97%.
As discussed, please remove % from hovers and update percentages that appear next to the boxes as a static figure so that they represent a binary report of whether or not that meta data is collected collected/reported by the repositry.
Please also provide a link to where the updated documentation can be found.

@candicecz please update the hover text such that:

If NOT augmented AND:

If percentage>0: "<field name> metadata is collected and available for this source."
If percentage=0: "<field name> metadata was not found for this source"

If augmented AND:

percentage > 0: "<field name> metadata is collected and available for this source.  was also augmented for this source."
percentage = 0: "<field name> metadata was not found for this source, but was augmented for this source."

@hartwickma @rshabman @lisa-mml @sudvenk
The badge has been updated and can be viewed here: https://nde-dev.biothings.io/compatibility
The Ask: We seek permission to add the badge to Staging

Hi @gtsueng, thank you for this update. Unless there is substantial change or concern, please move towards production. NIAID does not need to review in between steps.

@hartwickma @lisa-mml @rshabman @sudvenk
The repository-level metadata compatibility badges are now live on production
I am marking this issue as pending close out and will close it after a week if there are no further changes to be made.


Add Vector reflect methods

Add Vector::reflectX() and Vector::reflectY() methods.
The reflectX() methods reflects the X-coordinate (flips the vector about the Y-axis).
The reflectY() methods reflects the Y-coordinate (flips the vector about the X-axis).
Hopefully the meaning of the names isn't too confusing, and won't cause people to swap them by accident.

Looks like Codacy is undergoing maintenance and is unable to run. Maybe check on it in the morning to see if it can be re-run.


Check result of fread call

The function get_tag_header calls fread(buffer, ID3_HEADER, 1, file) to indicate that it absolutely needs 10 bytes to work on, but it doesn't assign the result of this call. When the file is shorter than 10 bytes, the fread call returns 0 and leaves buffer indeterminate. In these conditions, buffer should not be passed to the function get_tag_header_with_buffer, which would find itself processing indeterminate data.
This pull request makes get_tag_header return NULL when the fread call returns 0.

Thanks for your contribution 👍


this.nzDropdownMenu.descendantMenuItemClick$ is undefined

Reproduction link
Demo
Steps to reproduce
I am trying to make custom filter in my table but i'm unable to open dropdown, it return error of
ERROR TypeError: this.nzDropdownMenu.descendantMenuItemClick$ is undefined

What is expected?
Open each table heading dropfown
What is actually happening?
get error

Environment
Info

ng-zorro-antd
10.1.2

Browser
Firefox

Code

                {{ column.name }}
                <nz-filter-trigger [(nzVisible)]="visible" [nzActive]="searchValue.length > 0" [nzDropdownMenu]="'menu' +  i">

                </nz-filter-trigger>

                <nz-dropdown-menu #menu{{i}}="nzDropdownMenu">

                        Search By {{ column.name }}

                        <input nz-input placeholder="Type here..." />

                        Sort Order

                        <nz-select ngModel="none" class="text-left mr-2" style="width:100%;">
                            <nz-option nzValue="none" nzLabel="None"></nz-option>
                            <nz-option nzValue="asc" nzLabel="Ascending"></nz-option>
                            <nz-option nzValue="desc" nzLabel="Descending"></nz-option>
                        </nz-select>

                        Filter Column

                        <nz-select ngModel="contains" class="text-left mr-2" style="width:100%;">
                            <nz-option nzValue="contains" nzLabel="Contains"></nz-option>
                            <nz-option nzValue="like" nzLabel="Like"></nz-option>
                        </nz-select>

                      <button type="button" class="btn btn-warning w-100 mt-2" (click)="search()"> Search</button>

                </nz-dropdown-menu>

Note If I use [nzDropdownMenu]="menu" and #menu="nzDropdownMenu" (as of documents) it will open all headers dropdown at the same time but since I've added index number to have unique ids it returns this error (mentioned above).
any idea?

I think you're using the same variable（visible）

I think you're using the same variable（visible）

Would you mind share with me what do you think i should do? I'm really stuck at this.

I think you're using the same variable（visible）

Would you mind share with me what do you think i should do? I'm really stuck at this.
I think u can remove [(visible)] in the

I think you're using the same variable（visible）

Would you mind share with me what do you think i should do? I'm really stuck at this.

I think u can remove [(nzVisible)] on the nz-filter-trigger

@to0simple I've already removed that it didn't work

@to0simple I've already removed that it didn't work

https://stackblitz.com/edit/angular-ecxh6j-wenfft?file=src%2Fapp%2Fapp.component.ts

@to0simple

  {{ column.name }}
  <nz-filter-trigger [nzActive]="searchValue.length > 0" [nzDropdownMenu]="'menu' + i">

  </nz-filter-trigger>

  <nz-dropdown-menu #menu{{i}}="nzDropdownMenu">
  ..... rest of it...

Also as your sample <nz-dropdown-menu #menu{{i}}> none of them work 😞  did I miss something from your sample?!

@to0simple

  {{ column.name }}
  <nz-filter-trigger [nzActive]="searchValue.length > 0" [nzDropdownMenu]="'menu' + i">

  </nz-filter-trigger>

  <nz-dropdown-menu #menu{{i}}="nzDropdownMenu">
  ..... rest of it...

Also as your sample <nz-dropdown-menu #menu{{i}}> none of them work 😞 did I miss something from your sample?!

  {{ column.name }}
  <nz-filter-trigger [nzActive]="searchValue.length > 0" [nzDropdownMenu]="menu">

  </nz-filter-trigger>

  <nz-dropdown-menu #menu="nzDropdownMenu">
  ..... rest of it...

u can try this way . remove i

@to0simple Thanks man it worked eventually ❤


Investigate PAR/Hemingway reading level discrepancy

I did a reading level audit of the ODI website. I noticed a couple discrepancies between Hemingway and the PAR widget:

Innovation Community of Practice (Hemingway says 8, PAR says 9)

I wonder if the bullet points that don’t end in periods might be interpreted as run-on sentences.

Contact us (Hemingway says 4, PAR says 8)

Similarly, I wonder if the blocks for Kellie’s contact info and the address block might be culprits here.

Curious! I get reading level 4 on contact us with the email addresses and 3 without. I wonder if Hemingway is doing something behind the scenes. 🤔

Aaron left a note in the readme about Hemingway

We discovered some interesting issues with hemingway during this process and opened bug reports with them to help them stop using a . in an email address or a curly apostrophe as word delimiters.

Have a feeling this is related somehow.

I have notes on the development process for this feature in coda:
https://coda.io/d/Engineering-wiki_dL_3_CR2bVn/Workarounds_sun-Q?searchClick=428dfeab-982a-43c5-b296-aa0bb5bb87f1_L_3_CR2bVn
The 4/17 update details some discoveries Michael and I had when working
with Hemingway. One of them was that Hemingway treats . in an email as a
word delimiter but my local parser doesn't. This is debatable but maybe the
way we are scoring it makes more sense.
Michael's results below point to maybe the email addresses not being the
root cause of the discrepancy on this page because we'd expect Hemingway
treating an email with dots as multiple words to give it a better grade
level. Maybe they changed their parsing behavior on email addresses too.
FYI I rescheduled my legoland trip so I am watching for any support
requests via email or slack.
I hope that coda doc helps you understand what is going on here if you
weren't aware of it already.
Aaron
On Tue, Jun 13, 2023 at 10:21 AM zakiya @.***> wrote:

Putting a note here so I remember.

In local tests, default ari is 8.27.
It drops to 4.25 if I delete all email addresses (not just the href)
Tried adding format: 'skip' to the a selector in readibility.mjs but
no change in score.

I'm not finding obvious documentation at
https://www.npmjs.com/package/readability-scores so I'm going to wait
till @aaronhans https://github.com/aaronhans gets back from vacation
before going too far down the rabbit hole.
—
Reply to this email directly, view it on GitHub
https://github.com/cagov/innovation.ca.gov/issues/187#issuecomment-1589726480,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACWIUGBKKV7JDFTZUOVYZLXLCORVANCNFSM6AAAAAAZBFFXYY
.
You are receiving this because you were mentioned.Message ID:
@.***>

The Coda doc was definitely helpful.
I removed a tags in from the parser and now the Contact Us page scores at 3rd. Grade. That seems consistent with  @m-sullivan7's results and probably more accurate because people don't need to comprehend email addresses.
Let me know what you two think.
PR preview here

Sounds like a good solution, nice job!
On Mon, Jun 19, 2023, 11:36 AM zakiya @.***> wrote:

The Coda doc was definitely helpful.
I removed a tags in from the parser and now the Contact Us page scores at
3rd. Grade. That seems consistent with @m-sullivan7
https://github.com/m-sullivan7's results and probably more accurate
because people don't need to comprehend email addresses.
Let me know what you two think.
PR preview here https://187-reading-level.pr.digital.ca.gov/
—
Reply to this email directly, view it on GitHub
https://github.com/cagov/innovation.ca.gov/issues/187#issuecomment-1597605221,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACWIUDADX57PWUT3M6ARH3XMCL4JANCNFSM6AAAAAAZBFFXYY
.
You are receiving this because you were mentioned.Message ID:
@.***>

One more data point for this discussion: the PAR widget says the homepage is 7th grade reading level (score of 95). When I put the following content into Hemingway, it says it's 6th grade.

We're improving services for all Californians
The  Office of Data and Innovation (ODI) helps departments meet Californians  where they are. We use data, human-centered design and technology to  create services that are easier for everyone to use.
See what we've done
Changing state government through the Data and Innovation Fund
This  fund is a key part of how we're improving services for Californians.  When paired with the innovative approach of ODI's cross-functional  teams, we make a real impact on people's lives.
Learn more about the Fund
Who we work with
California Governor's Office of Emergency Services

California Department of Technology

Department of Cannabis Control

Employment Development Department

California Department of Public Health

A team as diverse as the state we serve
There's  no typical ODI team member. We come from throughout the state and bring  a diversity of experience to our work. What unites us? Our passion to  make a difference for all Californians.
Meet our team
Great people wanted
We're always looking for people who want to do big things!
Join us

(I noticed some opportunities for content improvement that might further help the reading level, but I thought I'd share this in case we wanted to look into the continued discrepancy.)

With help from @m-sullivan7 I did a comparison of the methodology that Aaron was using under the hood, and the Hemingway tool that Michael was using.  What I ultimately found is that both tools are using the same metric (the automated-readability-index) which just uses character, word and sentence counts.  The formula is
GRADE = 4.71 * (characterCount / wordCount) + 0.5 * (wordCount / sentenceCount) - 21.43
This produces a floating point grade value.  Hemingway rounds it down.
What we found is that if you simply cut and paste the text into the Hemingway tool, it tends to artificially inflate the sentence count due to additional linefeeds, and this inflated sentence count can give an artificially better score.
For example, it lowered the score of the homepage from 7.6 --> 5.3.  This was mostly because we measure about 19 sentences, but Hemingway was measuring about 26.  The number produced were:
ODI javascript
841 159 19 (score = 7.6)
Hemingway
832 166 26 (score = 5.3)

So, in short, the way the text is parsed has a lot to do with the final score, and simply pasting text from our site into Hemingway artificially inflates the sentence count which gives a nicer score.  You can achieve a more honest score, by cutting and pasting the text from the site into a programmer's text editor, and then editing out the extra line feeds that break up the sentences into multiple lines, and then pasting that into Hemingway.
Going to go ahead and close this.


Never compare two objects just with === or it will except on a certain moment

I'm totally new to this project and even to angular. But i have to maintain now a large angular-app for my new job - and so i made my first experience with "ui-router".
The first thing i had to implement is a persistence layer for our own history-states management. So i moved along and just implement a simple localStorage approach to save our states array and load it back again when the user makes a hard-reload in the browser or re-opens the app in our PhoneGap build.
So imagine this case - you load the /home of your app, navigate through some states and - you make a hard-relod.
Now, during the reload, this whole states array will be loaded from localStorage and now - all objects in the array will have new memory addresses... Now i get to the point ;-)
Never compare two objects with just "objectA === objectB" - because internally, all that happens is that the JS interpretor will compare the memory addresses of those two objects.
If the objects are basically the same and you expect a "true" as returned value - it will only be true as long as you're working in the same "context/session".
As soon as something happens that will re-instantiate those objects (localStorage read-back is enough to get there ;)) - even if the object is still the same for you as a human, for the computer it's not anymore...
I'm a little bit concerned after a short search - because i find several points in the code of this project where objects are compared in that way.
You would better make your own compare-functionality, where you invent a kind of a UID for every object you want to compare and afterwards, you compare only those UID's of the object.
Or you ask yourself again "what do i really need to compare here? is there already a property in the involved objects which would be sufficient for comparison? Yes? So then use this instead the whole object...
For now, we're fixing the issue by explicitly passing only the string for the state, and not the whole state-object itself - just to workaround the problematic if in state.js:findState
if (state && (isStr || (!isStr && (state === stateOrName || state.self === stateOrName)))) {
  return state;
}

Keep up the good work! All the best!
my two cents..

Angular has a solution for this already: angular.equasls
But as for the code here, even though you "saved" the state, once you reload the page, the state is different. It may have the same properties but it is a different object.
Some of this may be gone anyway in the new 1.0 feature branch.

But as for the code here, even though you "saved" the state, once you reload the page, the state is different. It may have the same properties but it is a different object.

Yeah, that's exactly what i tried to say in my not-so-short issue-description :-)
If there is a solution with angular.equals, why you're not using it? At least we have a workaround so it's ok you closed the issue ;) But theoretically, the problem still exists


Test suite for Transfer API method

This PR introduces the following changes:

Creates test scaffolding + scenarios for the transfer method in the DebtTokenAPI
Refactors the transferFrom test scaffolding so that it is reusable between transfer and transferFrom
Add invariant checking to the transfer method.

🔥


Could not block IP address of brute force attack using modsecurity & core rule set.

We are using apache windows web server. When client hits our web URL we have a log in page (this is apache managed) that authenticates the user.
We need to take action for unsuccessful log in attempts by the user.
We wanted to delay log in process for the brute force attacker after 3 unsuccessful log in attempts.
For this we are experimenting the use of Mod Security module (ModSecurity 2.x) .
I have used the following configuration file in main apache config file (httpd.conf)
modsecurity-minimal.conf
crs-setup.conf
REQUEST-912-DOS-PROTECTION.conf
I have enabled the log and checked configuration statements are being executed. Logging is happening.
However even after 3 unsuccessful attempts the log in page is not delayed.
I have below settings in modsecurity-minimal.conf
SecRule RESPONSE_STATUS "403" "phase:4,id:4001,setvar:ip.block_script=+1,expirevar:ip.block_script=1800,msg:'Attack Detected'"
SecRule ip:block_script "@ge 2" "phase:1, deny, status:406, id:4002, msg:'Repetitive Attack from the same IP Address'"
Please help me how to fix this issue.

Hi there, thanks for writing in, yet I think you are using the wrong tool to fight this.
The DoS protection rules are not the strongest part of CRS and they are not useful for bruteforce attacks unless you know them by heart and tweak them to work.
What you should be looking into is mod_qos that is doing exactly what you want.
I am closing this. If you are convinced this is a CRS malfunction, then please point out exactly where the error is.

Thank you. However I dont think mod_qos will fix our issue.
We are using Apache server on Windows. We do not have our own custom login page for our web server. We are relying on OS controlled(NTLM/Kerberos Authentication)log in page. However log in page do not prevent user log in (delayed log in for ex) after unsuccessful authentication. Hence we thought to use Mod Security to prevent it.
Also we used CRS Corerule sets like below. Still we are getting the issue stated above.
https://github.com/fabiocicerchia/OWASP-CRS/blob/master/experimental_rules/modsecurity_crs_11_brute_force.conf


Rust integration: Adding variables & clients.

Co-authored-by: Dima Dorezyuk<EMAIL_ADDRESS>Signed-off-by: Holger Rapp<EMAIL_ADDRESS>

We updated also https://github.com/EVerest/everest-core/pull/344 to work with this new code.

@hikinggrass @a-w50 Friendly ping? This has been sitting for a week now.

@hikinggrass @a-w50 Friendly ping? This has been sitting for a week now.

I'm still working on the cmake integration, but I hope to have something to push sometime today

@hikinggrass @a-w50 Friendly ping? This has been sitting for a week now.

I'm still working on the cmake integration, but I hope to have something to push sometime today

This (in combination with the changes I just pushed on everest-core) should allow rust modules to be built with cmake without having to manually specify the location of everest-framework or to rely on heuristics. There have been some changes to the config handling in everest-framework recently which I've already rebased against.
The cmake support isn't completely ready yet and still needs some adjusting before merging.
One thing I've noticed with the rust modules in general: it looks like they do not yet publish true to everest/<module_id>/ready once their initialization code has run and triggering this manually doesn't seem to call the on_ready handler yet. That's probably something we should fix before merging this as well.

I did a flyby of the code, I have little time this week, so might not be able to dive deeper - thanks for the work and review so far!

@hikinggrass What is this still waiting on?

@hikinggrass What is this still waiting on?

just a rebase, I'm squashing right now :)

❤ ❤ 🎆 H O O R A Y  🚀 ❤ ❤


1.37.5

For new BSML 1.12.0 and BS 1.37.5

You forgot to update the manifest for BSML/game version.

Thanks


Storefront Analytics

We want to show artists:

How many people visited
What they looked at
?

Okay these are the Cadillac version vs the Chevy ... could start with the chevy

would be great to see the traffic sources
new vs return visitors
by hour, day, week, month, year

Other ideas that would help me as an artist stay motivated and/or might make this the go-to place for the consumer (may be the wrong place to post this):

ratings (likes, hearts, stars, etc.)
Wishlist
gratuity ($ or coffee, chocolate, pizza, beach vacation)
SM sharing (Twitter, FB, IG, TikTok, Reddit, LinkedIn, etc.)
maybe comments
badges (could have some real fun with this)
auctions
opportunity to take special orders
Subscribing to an artist to get an email when they drop a new item
or a patron type monthly support (support our artists for as little as a flare a week and get first opportunity at new art)

Thank you for your feedback @wilogate this is definitely not the wrong place and is definitely the RIGHT place to share feedback/ideas, especially if they keep you motivated in being a starbeam.one artist. 😁
We may be able to do ratings at some point, but to be certain I am not a fan of them when it comes to art.  Too much can be made of bias or groupthink when that is introduced.  There is a certain element of that with social media.  "What should I think of this?  Well, I will look at the ratio to start."  This might be OK for discussion, but for art and how that makes one feel I think this is more 1-to-1 and should be personal without having someone else's judgment muddy their waters.
However, that said, I have my own idea on how to assist with "likability" and it's more effective than likes.  The best Like someone could give, after all, is a purchase.  This is a Super Like in my respects and the ultimate goal.  Of course, with social media, millions of Likes are given and they cost more to process/host/render than they do in giving financial support to those you are providing it for.
I call it ... Fretware!  Have you ever used Hotel sites and you are able see how many people are looking at an offer or the last time someone purchased a reservation from the hotel?  I would like to introduce the same thing for our storefronts.  "10 people are looking at this right now, 2 people are purchasing it" ... etc.  I think this will be a much more effective way at getting purchases as it shows activity and interest.
As for comments, this becomes a moderation challenge, and also impacts bias which is a huge enemy in my world.  A lot of sites open their doors to comments which at first seem like a very obvious feature but once you have a sense of what people have to say you quickly learn you do not want to hear it. :)  Maybe there might be a way to introduce this however that does not cause such a liability.
Gratuity:  if you are thinking tipping, I appreciate the idea, but if you are going to spend money on someone, why not purchase a license for .25 and get something back in return with resale value that you can sell at a later date?
Social Media Sharing: We do have a pretty basic sharing mechanism at the bottom of every storefront but it is not custom tailored to different platforms.  I will get this captured.
OK... as for the others, believe it or not I have/had these captured locally/privately, but I will now get them captured here as this is becoming the project management hub for starbeam.one:
Badges: Captured in #28
Auctions: #31
Special orders: #29
Follow Artist: #33
Subscriptions: #34
Wishlist: #35 need help with this one. :)
Thank you again for your feedback and please feel free to continue the conversation at the issues above to help provide more detail and to flesh them out accordingly. 👍

I never did update w/ Google Analytics.  I want to leverage them for page views but this is not possible:
https://support.google.com/analytics/thread/197663637

Maybe use this for views: https://countapi.xyz/

Want to provide a way to provide custom storefront referral links and see if they have been viewed.


fix(doc): how to get password for the ubuntu user

Mac OS 10.12.3
Vagrant 1.9.4
VirtualBox 5.0.8
After following instructions to set up a Vagrant machine to try rkt (git clone, vagrant up), I could not login in the machine via vagrant ssh since it asked me for a password.
It was a bit painful to find so I added instructions in the documentation on how to get it.

Seems reasonable. Would you mind squashing your commits?


[BUG] Error deploying model, unknown built-in op

I have gone through the example: opensearch-py-ml/examples/demo_deploy_cliptextmodel.html
Model is correctly registered in opensearch cluster but the final command of the example:
ml_client.deploy_model(model_id)
ends with an error Exception: Model deployment failed
Afrer trying to deploy the model using console directly on the cluster i.e.:
POST /_plugins/_ml/models/<model_id>/_deploy
and checking the corresponding task output:
GET /_plugins/_ml/tasks/<task_id>
I see that deployment fails with the following error returned by the task:
{
"model_id": "uDNmgZIBC9ZdJM8aMbns",
"task_type": "DEPLOY_MODEL",
"function_name": "TEXT_EMBEDDING",
"state": "FAILED",
"worker_node": [
"xbE0btjVQUGBAWrVqCmUoQ"
],
"create_time":<PHONE_NUMBER>169,
"last_update_time":<PHONE_NUMBER>632,
"error": """{"xbE0btjVQUGBAWrVqCmUoQ":"\nUnknown builtin op: aten::scaled_dot_product_attention.\nHere are some suggestions: \n\taten::_scaled_dot_product_attention\n\nThe original call is:\n  File "code/torch/transformers/models/clip/modeling_clip.py", line 190\n    key_states = torch.transpose(torch.view(_54, [_49, -1, 8, 64]), 1, 2)\n    value_states = torch.transpose(torch.view(_55, [_48, -1, 8, 64]), 1, 2)\n    attn_output = torch.scaled_dot_product_attention(query_states, key_states, value_states, attn_mask, 0., False, scale=0.125)\n                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE\n    attn_output0 = torch.transpose(attn_output, 1, 2)\n    input = torch.reshape(attn_output0, [_47, _51, _52])\n"}""",
"is_async": true
}

Hey, im getting exactly the same issue. Then i tried to upload ONNX model type and im getting:
{     "model_id": "A51B-JIBVoJr-PiFnVNy",     "task_type": "DEPLOY_MODEL",     "function_name": "TEXT_EMBEDDING",     "state": "FAILED",     "worker_node": [         "lNpYAlQ6T0Wu5ymYW4OQ4g"     ],     "create_time":<PHONE_NUMBER>213,     "last_update_time":<PHONE_NUMBER>643,     "error": "{\"lNpYAlQ6T0Wu5ymYW4OQ4g\":\"Error code - ORT_FAIL - message: Deserialize tensor onnx::MatMul_3915 failed.GetFileLength for /usr/share/opensearch/data/ml_cache/models_cache/models/A51B-JIBVoJr-PiFnVNy/9/intfloat/multilingual-e5-large/onnx__MatMul_3915 failed:Invalid fd was supplied: -1\"}",     "is_async": true }