WaterMeterController Support (YS5008-UC)

Please add support for Water Meter controller YS5008-UC
https://shop.yosmart.com/pages/ys5008-support
This is a brand new product from Yolink so I understand why it is not supported yet but it would be awesome if it could be added. I think this product will be quite popular at it's price point. Way cheaper than Moen Flo or Phyn Plus.

Do you have one to test with?  I would need to see the log messages as the plugin discovered the device and what messages are received.
The plugin already supports the original YoLink water valve (I have a YS4909-UC) and if this new one uses the same basic commands then I would expect open/close to work.  But the additional features that measure the flow will not, and I don't think HomeKit would support them anyway.

Yes I do have one, I actually got this error message when I added it to the Yolink app and then looked in my homebridge instance and saw the errors about it not being supported.  I've attached the relevant section of the log files with the error.
homebridge.log.txt

Thanks for the log.  It is reporting itself as a different device so current plugin will not support.  I should be able to add support but I may need you to do a couple or rounds of testing with me.  I'll let you know when I have something for you to try.
Thanks.

Gotcha, yes this is a completely new product from Yolink combining their water meter and smart valve into one device. I think it will be a popular product from them given it's competitive price point.  It's half the cost of others like Moen flow. I'm happy to do any testing on my end just let me know once you need. I'm getting the device installed by a plumber next week.

I just published a beta version to NPM.  Go to the plugin in Homebridge and select "manage version" from the dot-dot-dot menu.  It is v1.5.8-beta.1
I believe it should correctly report status of the valve.  I am not sure whether it will be able to set the valve (open or close it) as I am guessing that the data to send is the same as what I send for the older style valve.  But that may be a bad assumption... as the format of the status data received from this valve is different (per your log file).  Unfortunately YoLink have not published the API for this device at their developer API document page yet.  If it doesn't work, then I will have to reach out to them for documentation.
New feature in this device is that it appears to report whether water is flowing or not.  HomeKit supports this, so I added support for that.  Again, I don't have one to test (and you won't be able to test that before you have it installed!.
Please turn on verbose logging and report back with what it shows.  I'm particularly interested in seeing what is logged when you control the device from your phone YoLink app... as that will cause messages to be sent to the plugin reporting status.
Thanks.

I installed v1.5.8-beta.1 and the water meter now shows up in HomeKit and does not give a "device not supported" error in homebridge. However, when I try to control the valve it throws an API error. I am attaching the logs for your review. Also, in HomeKit it keeps showing "starting" on the device but that does not change.
homebridge-yolink.log.txt

I'll need to look and see if I can find some documentation on what this device expects.  Also, did you enable verbose logging in the plugin settings?  If not please do that and try restarting again.
The 00201 errors are, unfortunately, fairly common (see my comments on that in the technical notes section of README.  So not clear whether they are relevant.
Thanks.

I had enabled verbose logging for homebridge itself but did not realize I needed to do separately for plugin.  See snippet below of verbose output when trying to close/open the valve.

The log looks odd because it is missing data I would expect to see after the SENDING: and RECEIVING: statements (will be JSON strings).  What version of Homebridge UI are you using?  It may be clipping that data when you only view logs for the YoLink plugin.  They are showing fine for me with the UI v4.56.4.
I need to see the full YoLink log to check that the device is properly registered... assuming the log is properly reporting the JSON data.  Can you download and attach that please.  You should redact any appearance of your userAccessId and secretKey (they may appear twice, once when the config file is logged, and again when they are used. Other identifiers are okay to leave in as they have temporary life... but if you want to be super cautious you can redact the access_token... it has a life of a couple of hours.
Also, next time you test, please do not try and control the valve from HomeKit / Apple Home.  Instead use the OEM YoLink app to control the valve... that should trigger messages on the log which I need to see.  I am looking for something like this.
[7/18/2024, 8:36:15 AM] [YoLink] [verbose] MQTT message: yl-home/d3d51dfc169c4f7aa02b1bf5901a92db/d88b4c01000369a3/report
{
  "event": "Manipulator.setState",
  "time":<PHONE_NUMBER>062,
  "msgid": "1721306175062",
  "data": {
    "state": "open",
    "loraInfo": {
      "netId": "010205",
      "signal": -71,
      "gatewayId": "d88b4c1603008c02",
      "gateways": 1
    }
  },
  "deviceId": "d88b4c01000369a3"
}

But yours will say WaterMeterController instead of Manipulator.

I am also looking for something like this during plugin initialization...
[7/18/2024, 8:27:37 AM] [YoLink] [verbose] [YS4909-UC (d88b4c01000369a3) Water Main Valve] checkDeviceState (refresh after 3600 seconds)
[7/18/2024, 8:27:37 AM] [YoLink] [YS4909-UC (d88b4c01000369a3) Water Main Valve] YoLinkAPI.getDeviceState [lite]
[7/18/2024, 8:27:37 AM] [YoLink] [verbose] YoLinkAPI.getAccessToken
[7/18/2024, 8:27:37 AM] [YoLink] [verbose] SENDING:
{"time":1721305657484,"method":"Manipulator.getState","targetDevice":"d88b4c01000369a3","token":"C346EFC1AEEADD1D9C69CE30815DBD30"}
[7/18/2024, 8:27:39 AM] [YoLink] [verbose] RECEIVED:
{"code":"000000","time":1721305659209,"msgid":1721305659209,"method":"Manipulator.getState","desc":"Success","data":{"state":"open","battery":4,"powerSupply":"battery","delay":{"ch":1,"off":0},"openRemind":0,"version":"0911","time":"2024-06-18T00:27:38.000Z","tz":-4,"loraInfo":{"netId":"010205","signal":-71,"gatewayId":"d88b4c1603008c02","gateways":1}}}
[7/18/2024, 8:27:39 AM] [YoLink] [verbose] [YS4909-UC (d88b4c01000369a3) Water Main Valve] Battery level: 100%
[7/18/2024, 8:27:39 AM] [YoLink] [YS4909-UC (d88b4c01000369a3) Water Main Valve] At 7/18/2024, 8:27:39 AM: Device state updated: Valve (both): open, Battery: 4 [lite]
[7/18/2024, 8:27:39 AM] [YoLink] Device initialized: YS4909-UC (d88b4c01000369a3) Water Main Valve
[7/18/2024, 8:27:39 AM] [YoLink] [verbose] [YS4909-UC (d88b4c01000369a3) Water Main Valve] Set data refresh timer to run every 3598 seconds

Thanks.

I just pushed a new beta.  Please try it and capture logs.
Thanks.

Attached is an updated log. With your latest beta I can actually control the valve now so that's progress! I think there may still be some issues with reporting data to homekit though. For instance I see the words "waiting" and "starting" a lot in the homekit UI for the meter. I have not captured logs when controlling the valve from the yolink app but I can certainly do that if you still need me to.
homebridge-yolink.log.txt

when I initially change the state of the valve in homekit it updates the status, for instance when I open the valve it says "running" as the status in homekit. However if a leave it for a while and come back it just says "starting" or "waiting". If I close the valve it says "off" in homekit. I have not left it closed to see if I come back if it still says that but I might try and see what happens. It is worth nothing this is not just a smart valve but also a water meter so hopefully get can get that to report as well. Of course that can't really be tested until it is installed into my water line (hopefully next week).

Good, we are making progress.  The log file is very helpful, I can now see all the data fields and I do see at least one change I'll need to make.
I have the original manipulator valve, and I also see "starting" and "waiting" when I control the valve.  I am not sure what causes those but it may have something to do with the InUse characteristic.  This is supposed to indicate that fluid is flowing through the valve.  For the original YoLink valve it did not report whether water was flowing, so I just assume that if the valve is open then it is InUse.   However, your valve does include a waterFlowing value, and I now use that to set the InUse characteristic property... which for now is always false.  But once it is installed, that waterFlowing value should change to true (or false) and hopefully that will make the "starting" and "waiting" states change.

As for anything other than open/close, our options are limited.  The HomeKit accessory documentation does not include water meters (or power meters).  Some 3rd party apps have added support for device types not in the official spec.  Examples are smart plugs which can measure power consumed by the connected device, but it is hard to implement (and I have not done it for the YoLink smart plugs).  Water meters are likely in the same situation.
It might be possible to add a leak detector accessory, that is in the HomeKit spec, and I do support the YoLink leak detectors.  This water meter is reporting a number of possible alarm conditions, one of which is leak.  So that may be possible.
I'm also interested in the temperature field.  Right now it is presumably reporting ambient temperature... but I'm curious if once you have it installed it will report the water temperature.

I also see quite a lot of YoLink API error code: 020104 Device is busy, try again later. errors in the log.  These will likely go away once you settle down with it installed.  See comments in my readme.

However, your valve does include a waterFlowing value, and I now use that to set the InUse characteristic property... which for now is always false. But once it is installed, that waterFlowing value should change to true (or false) and hopefully that will make the "starting" and "waiting" states change.

However, this may not be desired behavior... if you open a valve that controls your whole house, then water may not flow immediately... another faucet would need to be opened.  We will have to test this once you have it installed and may have to revert back to how I have it for the older valve.

@dkerr64 The temperature field will always show the ambient temperature as it measures that for the anti-freezing function and alerting built in to the device. I am also aiming to buy one soon so I made some investigation on its possibilities.

I just pushed beta 3 which adds support for the temperature sensor.  Note that temperature sensors (that are embedded within another type of device) are not enabled automatically... you have to request it in the configuration either globally for all devices (that provide temperature) or per-device.  See README.
As I do not have one of these devices I cannot test this, so I'd appreciate some feedback and a verbose log.

And beta 4 has leak sensor added.  Again I cannot test this so I'd appreciate feedback and verbose log.

Would you like me to install beta 3, collect logs and then install beta 4 and do the same or can I just go direct to beta 4? I do have some of the leak sensors too I was planning to use in tandem with the valve.

However, your valve does include a waterFlowing value, and I now use that to set the InUse characteristic property... which for now is always false. But once it is installed, that waterFlowing value should change to true (or false) and hopefully that will make the "starting" and "waiting" states change.

However, this may not be desired behavior... if you open a valve that controls your whole house, then water may not flow immediately... another faucet would need to be opened. We will have to test this once you have it installed and may have to revert back to how I have it for the older valve.

I think that's fine actually because then I can easily see if a device (sink, shower etc) is using water in my house.

Would you like me to install beta 3, collect logs and then install beta 4 and do the same or can I just go direct to beta 4? I do have some of the leak sensors too I was planning to use in tandem with the valve.

Just go straight to beta 4.
Thanks

See updated logs after upgraded to beta 4. I removed anything prior to the beta 4 upgrade. In homekit it still says a status of "waiting" when the valve is open. Maybe that will change when water is flowing though it? Also I do have one of Yolink's stand alone water leak detectors but I don't see that in homekit, I guess you were talking about the leak detection feature of the water meter/valve itself.
homebridge-yolink.log.txt

When the valve is closed it does report a status of "off" in homekit but when it's open it just says "waiting".

Also, here is how the meter/valve shows up in the Yolink app.

Oh I just noticed there is a firmware update already. This is a brand new device from Yolink so that does not surprise me. Let me know if you want me to go ahead and update the firmware. That might break some stuff but better to find out now than later.

Oh I just noticed there is a firmware update already. This is a brand new device from Yolink so that does not surprise me. Let me know if you want me to go ahead and update the firmware. That might break some stuff but better to find out now than later.

Yes go ahead and update.  It can take some time because the LoRa is very slow.

When the valve is closed it does report a status of "off" in homekit but when it's open it just says "waiting".

Yes I think this is expected until you have it installed... the waterFlowing status is always false... which is what would be expected when the valve is closed.  When you turn it on I think it is waiting for the waterFlowing state to change to true... which is not happening until you have it installed.

See updated logs after upgraded to beta 4. I removed anything prior to the beta 4 upgrade. In homekit it still says a status of "waiting" when the valve is open. Maybe that will change when water is flowing though it? Also I do have one of Yolink's stand alone water leak detectors but I don't see that in homekit, I guess you were talking about the leak detection feature of the water meter/valve itself.

Your existing leak detectors should appear in Apple Home as "Alarm sensors" which in a room on MacOS version show in the top right.  Click on that and a window opens listing them, like this...

And yes, I have added a leak sensor accessory to the water meter itself.  I do not know how that will appear in Apple Home, if not like above then it may be inside the settings for the valve itself... for example, this dehumidifier has three accessories... the dehumidifier itself, a temperature sensor and a humidistat...

I suspect that the water temperature sensor and leak detector may appear there.

Your most recent log looks good.  I'm assuming you did not enable the temperature sensor as I don't see logs related to that.
I think we're at the point of waiting until this is installed to see how it works in real life.

How does this device determine if there is a leak?  I'm guessing it is something like measuring a really slow flow rate... or an excessively fast flow rate?  Somehow it would be good to test that a leak alert is properly sent to HomeKit.

Looking at the YoLink app screenshots, I see that the temperature is shown.  I decided that for this device we should always create a temperature accessory service HomeKit rather than require a config settings.  This is enabled in beta 5.

Found the my other water leak sensor in the alarms section as well as some other devices from a different brand I thought were not showing up. Turns out I was just not looking in the right place. so thanks for that. As for the leak sensor built into the water meter itself I'm not sure exactly how that detects leaks. I agree I think it makes sense to enable the temperature sensor by default and I am indeed seeing this after updating to the latest beta. I kinda wish it was measuring the water temp but I guess ambient air temp makes more sense to alert if it's getting near freezing mark. I don't know exactly which day yet but I am hoping to get a plumber to install the valve this week. Once that is done I'll send you some updated logs. I really appreciate all the work you're doing on this.

Here's how the device shows up now in Homekit.

Looking good.  I will wait until after you have it installed and we confirm our theory on the water flowing / in-use indicator before I release it into the mainline.
Thanks.

Just had the valve installed today.  Opened it and turned faucets on and off and captured the logs. Unfortunately homekit does not seem to update if water is flowing so I think we are still missing something. Also, not sure if this matters but it appears the temperature is in fact water and not ambient temp. At least it says "water temp" in the Yolink app.
homebridge-yolink.log_07-24-2024.txt

Is the Apple Home app still showing "waiting" when you open the valve?
I have to decide what to do with the HomeKit "InUse" characteristic.  For the old valve I set it to in-use whenever the value was open, but for this new valve I use the value of "waterFlowing" in the data we get back from it.  But if that data is not coming in promptly then we may not be able to rely on it.
Your log does show "waterFlowing" as true sometimes, false other times.  But I don't know what Apple Home does with the InUse setting.
I just pushed a beta 6... no material differences, I tweaked the code that handles MQTT messages, and now check if the valve reports a fault or not.

So I updated to beta-6 and now the status does eventualy update in homekit but it is far from instant. It look about 30 seconds to update in the Yolink app and I was pressing the manual refresh button so I'm not sure how long it would of taken if I didn't do that. I've attached the latest logs as well as a few screenshots. This device is a bit different since it's not just a valve. It's a water meter with a valve. So I do think it makes sense for "water flowing = in use" however maybe that status could be broken out separately? Maybe as a "sensor" and then have the valve status report "open" or "closed"? I will say I don't really how it always says "starting" in homekit when the valve is open but water is not running.
homebridge-yolink.log_07-24-2024_9PM.txt

Here's what it looks like in Yolink app. Also it's worth noting even when status is showing as "running" in homekit when you click into the details view of the valve it still says "waiting".

I just did another test this time not opening the Yolink app but just opening homekit and looking at the status of the valve. I turned on a sink and at the 1:30 mark it was still not showing as "running". Then I opened up the Yolink app which showed a status of "water is currently running" then I switched back to the homekit app and the status had updated there. So it seems the Yolink app triggered somekind of status refresh. There is even a manual update refresh button in the valve.

The water meter/valve runs on batteries by default but does have the option of connecting a power adapter. I wonder if it would update it's status more often if it was AC powered. It seems like even in the Yolink app the water flowing status is not real-time.

So, I can explain what is going on, but I'm not sure if there are any changes we can make.  Let me explain how things work...
We can ask the device for current state, and we receive unsolicited messages from the device which report its state.
When you open up Apple Home it generally requests an update.  In your log it does this only once (timestamp 1:12:49am) where a WaterMeterController.getState request is sent by us to the device... and you can see in the log the request sent and the received status.
The reason there is only one instance of this in the log is because I have deliberately throttled the number of times I ask a device for status... HomeKit can be very talkative and YoLink imposed rate limits on their servers, and the LoRa protocol is slow.  So I severely cut back how often I actually ask the device for status, instead relying on the unsolicited updates and providing cached status when HomeKit asks.  In the log HomeKit is actually asking for status 90 times... usually multiple in the space of under a second!
The unsolicited messages come in as MQTT and they take two forms in your log... a WaterMeterController.Report event and a WaterMeterController.getState event.
Report events are sent regularly by a device to provide updated state, generally they come in regular intervals. There are 8 report events over a couple of hours, all of them report water not flowing.
The getState events appear to be triggered when you use the YoLink app... it is telling all other interested parties (us) that someone (the YoLink app) has requested device status, and provides the status.  There are 4 of these in the log in the space of ~90 seconds.  The first two report water flowing as false, the second two as true.
What I am not seeing in the log is any real time reporting of changes.  Other devices send a StatusChange event, which would come through MQTT as WaterMeterController.StatusChange but there are none in the log.  So the device is not reporting in realtime when water starts/stops flowing.
The workaround to this is to add a config option to the plugin settings to change the "refresh after" value. Do this just for this device, so in the config user interface add the device using its device identifier (yours is d88b4c01000a13fb) then set the refresh time to something like 30 or 60 seconds, that will result in a lot more requests for status actually going through to the device.
I think I will also to reach out to YoLink and ask.  I have seen them update device firmware in the past to add MQTT messages and it feels like that is what is needed here, polling every 30-60 seconds is really not ideal.
In other words... the plugin is working, it is reporting what it receives from the device.
Thanks.

@ssnthedj I emailed YoLink yesterday, no reply yet.  In the meantime I have changed the plugin (beta 7) to report InUse based on whether valve is open or closed... the same as I do for the older valves that do not report it.  I also added a config setting to allow users to switch this to use the waterFlowing Status from the valve if desired.
If YoLink update the firmware to accurately report the status change then we can try changing the default.
I've documented this in the Device Notes of the readme.
If this looks good in your testing then I will remove beta tag and publish it in a release update.

@matrixd2 I see you work on the YoLink home assistant plugin.  I wonder if you could review the discussion in this thread and comment?  Thanks.

@dkerr64 thanks for all the updates and testing you've done. I updated to the latest beta and it is reporting water is "on" when valve is open. I think that's better than the using the "water flowing" data is that is so delayed. I appreciate you reaching out to yolink about that. Even in the Yollink app the status of "water flowing" is quite delayed. I've seen it take several minutes to update. Really not ideal at all. Maybe they can make some changes. I do wonder if it was a battery saving measure for it to update less often. I am going to try and order a power adapter for mine because honestly I don't like relying on batteries. Strangely Yolink does not sell one just lists the voltage so I need to find something compatible.
I did have one thought would it be possible to break out the "water flowing" into a separate sensor in homekit? Right now we have water temp, leak sensor so maybe water flowing could be added there but the main status is still if the value is open/closed?

@dkerr64 Thank you for the work you are doing now.
Water is considered to be flowing only when the water flow exceeds a certain threshold. So there may be a delay in the water flowing state.
When the water flowing status changes, the device will not send data to the server, so your mqtt client will not receive the StatusChange message, You need to refresh by calling the getState API

I did have one thought would it be possible to break out the "water flowing" into a separate sensor in homekit? Right now we have water temp, leak sensor so maybe water flowing could be added there but the main status is still if the value is open/closed?

Leak and Temperature are easy to break out... as HomeKit has specific services for those.  But the water flowing (or InUse) state is built into the HomeKit Valve service and there is no separate service available for this.    I'm not sure that this buys us anything as unless/until YoLink provide more timely reporting its status.  Unless the value can be relied upon, I think it is misleading to report it at all.

@dkerr64 Thank you for the work you are doing now.
Water is considered to be flowing only when the water flow exceeds a certain threshold. So there may be a delay in the water flowing state.
When the water flowing status changes, the device will not send data to the server, so your mqtt client will not receive the StatusChange message, You need to refresh by calling the getState API

@matrixd2 Thank you for confirming what we have observed.  This is unfortunate, hopefully YoLink will update the firmware to provide a StatusChange MQTT message.  Regularly sending getState requests is really not the right way to do this.  With rate limits on YoLink cloud servers and the generally low capacity and slow speed of the LoRa network polling should be a last resort.  That said, I can do it and have documented it as a work-around for now.
I did send an email to R&D Team (address posted at bottom of the API page) brining this to their attention, but no reply yet.
Also, can you confirm how a leak or freeze is notified?  Will I receive a MQTT for those, if so what will the event be?  I have seen Alert events for other devices, is that what the WaterMeterController will send too... that is what I have assumed and coded to for now.
Thanks.

@miztahsparklez opened #100 for Yolink YS5006-UC which I closed as duplicate of this issue.  Looking at the JSON status provided for it, there are two notable differences... unlike the YS5008-UC this device does not report waterFlowing or temperature values.
Current plugin beta should work for the YS5006-UC, but I will have to put in some special handling to not add a temperature accessory to HomeKit.
The JSON also shows "battery": 4, "powerSupply": "PowerLine", so we now know what is reported when the valve has permanent power connection... but I don't think there is anything to pass back to HomeKit for this.

@ssnthedj and @miztahsparklez please try beta.9 that I just published.  I simplified the code a bit and only add temperature sensor if it exists (to properly support the YS5006-UC).
Thanks

Cool, I was able to see a "Running" status with the YS5006.
I was able to dig up some old documentation: https://forum.iobroker.net/assets/uploads/files/1634848447889-apple-spezifikation-homekit.pdf
It seems that the required characteristics are : Active, In Use, and Valve Type.
I'd assume Active is the on/off of the valve.  In Use is if water is flowing (does yolink return a boolean value? I guess you solved this by the "waterflowing" value above.) , and valve type would be genericValve (although, I wonder if irrigation or one of the other valve types would allow for different options).
I know for my LG washer/dryer via homebridge plugin can also show time left, which is likely using Set Duration or Remaining Duration.  However, I wonder if any of these could show how long the water has been running.  Could we use any of the overrun values or possibly show meter information?  I am betting the "waiting..." we are seeing is this time value.

@miztahsparklez Yes you will see "running" status because I set the InUse characteristic based on the valve being open or closed... we're assuming that water is running when it is open.  The YS-5006 does NOT send a separate waterFlowing status (based on the data you provided in #100 ) whereas the TS-5008 does.  But as you can see in the above discussion we have determined that we cannot rely on the waterFlowing status to be accurately reported.  I'm documenting it in the readme here.
I am familiar with the documentation you linked, I used it extensively when creating this plugin.  The duration information is for irrigation valve types, and we do not use that type in this plugin.  Also, I do not know what the overrunXxx status values are meant to report.

Just a note that YoLink have now published some API documentation for this device..  So when I get a chance I will review it to see if any changes need to be made to this plugin.
Otherwise, if folks monitoring this thread are happy... I'll switch from beta and release a mainline update.

Sounds good!  I think if you communicate with the expectation of what people will see, I think it's a worthwhile improvement from what it was before and gets the basic point of on/off and in use.
The overrun values correspond to "maximum continuous water usage", "maximum continuous water usage duration", and likely the leak detection schedule.  Usage is measured in volume.  Duration is a time limit.  Schedule is usage over time.  Any of these can be set to trigger the valve to turn off within the Yolink app.
My thought was if you used the water usage duration, there would hopefully be some sort of countdown to 0, or maybe a count up for how long the water has been on (and maybe this is your indicator for determining in use if non zero value).

Support released in version 1.6.0

@dkerr64 thanks for all your heard work on this! I've been out of town and thus have not been able to test beta 10 yet with my device. I will be back Sunday afternoon and should be able to update and test that evening.


Add user registration to react

Depends on #24 for Django to have registration support.
Add a user form in react so that someone can register.  Have it post to the appropriate registration URLS with the appropriate data.

Closed by !25


Permit conversion into and creation from parts.

Copy of https://github.com/matthunz/futures-codec/pull/54 by @romanb.

This PR generalises Framed::release() et al to Framed::into_parts(), providing corresponding Framed::from_parts. This is pretty much analogous to tokio_util::codec::FramedParts. There are mainly two use-cases:
1. Resuing framed buffers with a new codec ([tokio-rs/tokio#717](https://github.com/tokio-rs/tokio/issues/717)).

2. Protocols that use a particular `Framed` only e.g. during a handshake phase after which they want to drop down to direct use of the underlying I/O stream. Since the `Framed` may have read and buffered data beyond the handshake frames, it is necessary to drain the remaining read buffer first when doing this. While one can already obtain the read buffer from `Framed::read_buffer()`, using it to capture the buffer before calling `Framed::into_inner()` requires to unnecessarily clone the `BytesMut`.

I also tweaked the documentation a bit to make it clearer that the into_inner() methods may discard buffered data. Since this PR removes / renames Framed::release() et al to into_parts, the API changes are not backward-compatible.

@romanb do you consent to your work being used in asynchronous-codec?

@romanb do you consent to your work being used in asynchronous-codec?

Sure!

@romanb do you consent to your work being used in asynchronous-codec?

Sure!


Week 3 Individual

[x] Video 2 minutes

https://www.loom.com/share/3ceee80a5711422485707b6853e4d0f6

[x] Commits on cross over partner

https://github.com/Paul-D6/RepoTri3/commits?author=rohitd3

Graded by Nathan Shih
10/10
Rohit has everything shown nice and clearly in his video, his commits are thorough and the video was easy to follow along with as well as see each requirement.

Grading for Akhil by Nathan Shih
10/10
Akhil's video showed proof of all tasks being done and even had time stamps provided for each section shown. The video is smooth and easy to follow.


API Key

Now need an API Key ?

@kerosev yes since HaveIBeenPwned is now using API v3 it requires an API key which has to be purchased from HIBP website, more information is in the readme


[Feature Request] Mantra of Flames & Possibility to state the color of sockets

If you have mantra of flames equipped, it would be nice if in configuration you could have the options to say how many buffs are, so you can have an idea of your dsps if lets say you play with a support.
It would also be nice if you could choose the colors of the sockets for items like scaeva, prismatic eclipse or the new dialla's item.
Thanks again for everything Openal <3

Duplicate of #648

#589 is now resolved; #648 remains but this duplicates it anyway so I will close this.


Modularize libraries so they can be required individually.

This is a first attempt at making it possible to require individual Flow libraries. This fixes #42 and https://github.com/HipByte/motion-game/issues/52. There's still a little more work to be done, but as a first attempt is works pretty well. I was able to try this out in a test app and it works. I think having this ability is really important for Flow. Let me know what you think.

Here's the test app that I was referring to: https://github.com/andrewhavens/motion-game-with-flow-example

I have rebased to remove conflicts. Please let me know if you'd like to see anything change in this PR to make it ready to merge.

@hboon and others, do you have any thoughts on this PR? Anything you want to see changed before it's merged?

On my todo list.


Import errors with vite project

Describe the bug
vite projects fail to consume js-big-decimal.
To Reproduce
Repro repo is here: https://github.com/NiloCK/vite-bigdecimal-import-reproduction
See the readme.
Expected behavior
In the repo, yarn dev should serve the sample app page, with the counter integer replaced by a bigDecimal.
Additional context
The vite site seems to be mistakenly importing content from the dist/node folder, instead of dist/web
There is some issue with:

vite's bundler
js-big-decimal's packaging / distribution pipeline
some combination of the two

Response from the related issue filed vitejs/vite: https://github.com/vitejs/vite/issues/13031#issuecomment-1527404705
Quoting in full:

The bundled file is from dist/node/..., where it should be from dist/web/....

js-big-decimal doesn't declare exports/browser field and therefore Vite resolves the package with main field that points to dist/node/....
You can specify the actual file instead.
import "js-big-decimal/dist/web/js-big-decimal.js"

Although, It doesn't work because js-big-decimal doesn't support ESM.
Closing as it's working as intended.

@NiloCK  Please validate and close this.


feat: Implement ser/de for the remaining all

The comments on this pull request have been addressed.

The comments on this pull request have been addressed.


config options are cached

Probably related to https://github.com/LinusBorg/vue-cli-plugin-test-attrs/issues/3
Changing the attrs option in the config appears to be cached. Changing it has no effect until I reinstall node modules.

Good catch.
Can be fixed by including the options from vue.config.js here:
https://github.com/LinusBorg/vue-cli-plugin-test-attrs/blob/develop/index.js#L34


[FLINK-25111][table-api][table-planner] Add config option to determine CAST behaviour

What is the purpose of the change
Add a new config option: table.exec.sink.legacy-cast-behaviour, so that users can
choose between the legacy behaviour of CAST or the new one, including improvements and fixes.
Brief change log

Introduce a new config option
Add a new method to the CastRule contexts, so retrieve the value of this config option
Change Row->String rule to be able to use the , delimiter for legacy behaviour vs ,  for the new one.

Verifying this change
Added possibility to CastTestRules so that tests can run with legacy behaviour enabled & disabled.
Added tests for casting ROW -> STRING, to verify the legacy vs the new behaviour
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

Haven't produced the doc html, until after any comments (on this PR) regarding the doc part of the config option are concluded.
I wouldn't spend to much time on the docs now, since I've added a TODO, and when the major change with TRY_CAST, we can mention this "breaking" change in the docs of the new config option.

Thx @twalthr Please check again, apologies, I've already squashed the fixup commits to the 2 commits of the PR.


Fix linux packaging

Add .pyx and .hpp to smt distribution in order to be able to compile extensions when running pip install smt and bump version number to 0.3.2 to release on Pypi.

Coverage remained the same at 77.792% when pulling 281b265cc4d5b5939483ba6327e974d3712165d2 on relf:fix-linux-packaging into 5c8d6195ef19328f1a48c6787c780162f84ec2ad on SMTorg:master.

:white_check_mark: Build SMT 0.3.523 completed (commit https://github.com/SMTorg/SMT/commit/0213723159 by @relf)


#54: ClinicalCase package

Provides support for dealing with clinical data represented in the SDTM format
Sources: https://github.com/datagrok-ai/public/tree/master/packages/ClinicalCase

This issue has been mirrored in Jira: https://reddata.atlassian.net/browse/GROK-14742

This issue has been mirrored in Jira: https://reddata.atlassian.net/browse/GROK-16005


Website documentation is broken

http://superscriptjs.com/documentation/scripting

The documentation has moved to the wiki in this repository.

Ok. You should remove the broken one in the website.

@silentrob Can you redirect from the superscriptjs.com site to this wiki for documentation? This way we avoid future confusion and duplication.

Fixed and deployed.


[d3d9] Add a software cursor emulation config option

Fixes #3020. The game tries to create a 64 x 64 cursor, which wasn't supported prior to this PR, but is otherwise happy with our existing code path catering for hardware cursors. It even happily switches cursor icons in game on hover over certain objects.
Since this isn't exactly a software cursor implementation, rather a workaround, I've created a config option for it.
As far as I know this is the only d3d9 game that actually depended on software cursor support and this PR is enough to get it working properly. I can't guarantee other games will work just as well, since I haven't done a lot of validations and exotic bitmap sizes may cause issues. But we can improve the logic a bit if that ever happens to be a problem.
I welcome any testing, but this shouldn't affect the previous hardware cursor logic (unless I messed up badly somewhere, which doesn't seem to be the case based on my tests at least :sweat_smile:).
Would also appreciate very critical pairs of eyes on my memory allocations and use of pointers, since it's not something I get to do every day (and frankly I'm getting too old for it). Or I'm open to better alternatives, if any exist.

I happened to make a very close pull to yours recently: #3828 #3827

Indeed, saw those. Together with @Blisto91's regular mentions about implementing a software cursor (which, ironically, I still haven't), they reminded me there's a game that actually cares about these things. To be fair, my main objective here is to fix Dungeon Siege 2, so it's far less broad in scope.

I vote for this pull that it would be better to support variable cursor size rather than hard-code as 32x32. I did some test on Windows 11 and Xfce + X11 both of them being able to display a 64x64 cursor.

I'm on MATE + X11 and it also works fine.

And FYI, from what I read from MS document, cursor size API should have a fallback capability scaling the size back to 32x32 when the underlying platform can not support different size. I didn't write such code.

A good point, but that's still not being done. The hardware cursor path always assumes a 32 x 32 cursor regardless of bitmap size and does no scaling, which has never been a problem really, so it's probably good enough. Getting Dungeon Siege 2 to display a cursor was actually a lot less work initially, however the cursor was getting clipped to the top left 32 x 32, which I also had to  address.

Confirmed on Linux and Windows in Dungeon Siege II from GoG

Confirmed on Linux and Windows in Dungeon Siege II from GoG

Thanks for testing and confirming, especially on Windows!

I've added one more... well, I'd like to say "elegant fix", but we only have "cursed hack" on the menu today... in order to make the cursor emulation work properly with Castle Strike, a d3d8 game that also tries to create a 64 x 64 hardware cursor, but then also tries to control it with win32 calls (don't ask). Retested everything with Dungeon Siege II, which doesn't seem to mind any of the changes.
I'll PR the config option for Castle Strike against d8vk, since it's a d3d8 game.

There's a bit of a hitch on Windows, since the cursor won't show initially, rather only after alt+tabbing out and back in, but that bit is probably due to some events we're not handling. It works fine in Wine/Proton at least.
There's some minor differences when compared to WineD3D's software cursors (namely the cursor stay on screen during loading screens), but a somewhat janky cursor is better than no cursor. It also seems to hide/show just fine in-game, so that's good.
I've run out of ideas how to make this any better without actually implementing a software cursor, which would be a major pain I'm not sure anyone is willing to pursue, so I'll leave it as is.

I don't quite get why this is a config option. Is there any downside to always enabling/exposing this?

I don't quite get why this is a config option. Is there any downside to always enabling/exposing this?

There's two reasons for that mostly:

Hardware cursors are only ever supposed to be 32 x 32 pixels, especially in windowed mode. I've done some tests out of curiosity and it looks like WineD3D validates this, so we're going out of spec to make some games happy without actually implementing software cursors. This path is essentially a workaround for the absence 64 x 64 pixels icon support and mouse input message handling in the case of hardware cursors. Only 2 games out there that we know of would need it to work properly.
This bit is a rather cursed hack done exclusively for Castle Strike, which relies on some cursed form of win32 message handling to show the cursor on its main menu (then it show/hides it properly with d3d8 calls during gameplay). I would not hijack the window cursor pointer for every well behaving game out there.

Honestly, why don't we just implement software cursor?

Honestly, why don't we just implement software cursor?

... go ahead :). This PR is mostly a stopgap until we get there and this path should be removed once we do. That being said, the window message handling bit sounds like a potential major pain to get right (and a bit beyond what I'd venture to implement).


User can set goals in Easy FDC

As a program evaluator I want to be able to set my own goals in Easy FDC for each indicator I'm going to report (instead of taking WMFR goals :p).
This probably needs a bit of refactoring in fdc.py and in the webapp @JeanFred

Not sure of this: the goals are only used in the output Wikitext This looks like a lot of interface clutter for no substantial advantage.

Quite true, not sure the value it would bring.


Quickstart wizard

The idea here is to have a step by step, walk through style wizard that asks questions and leads them to the correct documentation.  This is the first pass at it.
The Library I used for this (and made some changes) to also supports multiple languages, so if we decide to go down this route, and we have all the steps locked in, we can add additional strings for different regions.
If people think this would be helpful to new people discovering Owncast, we can move forward with it.  If you find it unnecessary, we can totally drop the idea.  As an aside, I was thinking of us doing something similar for helping people troubleshoot to replace our Troubleshooting document that nobody reads.
To test it, check out this branch and visit /wizard.

Closes https://github.com/owncast/owncast/issues/1338

I think this is a great idea and I like the initial progress. Everything seems to go where it should and the disclaimers are good. The only comment I have at this point is that it needs back/start-over buttons.

Any web people here familiar with the browser history APIs and can add a back button?

I figured it out.  The browser back button now works and I added a "Restart" button.

Fixed light mode.

I tweaked some CSS to reduce the spacing between items to fit a bit more on the screen.  I'm not sure if this looks "good" or not, but it's something.

I didn't like having to press the "Next" button so I changed it to auto-navigate once you selected a choice. I think it's better.

@gingervitis If you get a chance could you look at the appearance of this and make any changes you'd like to see?

I just updated the Restart button a bit to fix to font and to not make it so weirdly large.

Merging!  But if anybody finds this to be less helpful, slower, or more confusing compared to the current list of documents we have now then we should get rid of it.


feat(time format): support gov.uk create time format

whois the "gov.uk" create time is "before Jan-2006".
It need to be support.

Hello @hodge1997
Thanks for your work.

If you think this repository is helpful, please share it with friends, thanks.


use pytorch2onnx to convert libra_rcnn is error

Describe the bug
I use pytorch2onnx.py to convert libra_rcnn to onnx,but it didn't work.
Reproduction
python tools/pytorch2onnx.py configs/libra_rcnn/voc_2_libra_faster_rcnn_r50_fpn_1x.py work_dirs/libra_faster_rcnn_r50_fpn_1x/latest.pth --out libra.onnx

Environment
cuda 10.0
python3.6.9
pytorch1.4
onnx 1.6
onnxruntime 1.2.0
Error traceback
  File "tools/pytorch2onnx.py", line 125, in <module>
    main()
  File "tools/pytorch2onnx.py", line 117, in main
    onnx_model = export_onnx_model(model, (input_data, ), args.passes)
  File "tools/pytorch2onnx.py", line 44, in export_onnx_model
    operator_export_type=OperatorExportTypes.ONNX_ATEN_FALLBACK,
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/__init__.py", line 148, in export
    strip_doc_string, dynamic_axes, keep_initializers_as_inputs)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 66, in export
    dynamic_axes=dynamic_axes, keep_initializers_as_inputs=keep_initializers_as_inputs)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 416, in _export
    fixed_batch_size=fixed_batch_size)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 296, in _model_to_graph
    fixed_batch_size=fixed_batch_size, params_dict=params_dict)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 135, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/__init__.py", line 179, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/utils.py", line 657, in _run_symbolic_function
    return op_fn(g, *inputs, **attrs)
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in wrapper
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 128, in <listcomp>
    args = [_parse_arg(arg, arg_desc) for arg, arg_desc in zip(args, arg_descriptors)]
  File "/home/juzheng/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/onnx/symbolic_helper.py", line 81, in _parse_arg
    "', since it's not constant, please try to make "

RuntimeError: Failed to export an ONNX attribute 'onnx::Gather', since it's not constant, please try to make things (e.g., kernel size) static if possible

pytorch2onnx is still experimental and not all models are supported now.


Muth Distribution

Muth Distribution added to stdlib
source: http://www.math.wm.edu/~leemis/chart/UDR/UDR.html

Can you please fix this PR so that it has no conflicts?


Assume database object names are lowercase

Fixes #59

I had to take a look and run some tests, but it look like Athena only supports having lowercase table and column names. So this seems a reasonable assumption to have.
Source: https://docs.aws.amazon.com/athena/latest/ug/tables-databases-columns-names.html


One test fixture doesn't produce MS2 scans

fragscan_dataset_spectra
produced empty MS2 scans.
Chemical objects don't have any children. Problem is this method:
https://github.com/sdrogers/vimms/blob/be48bd5c9f4db9f63cc7bd7f0b42bba7d1e339db/vimms/Chemicals.py#L378-L389
Should it be returning an empty list on line 383?
At the moment, I've removed it from all tests.

Should it be returning an empty list on line 383?

It shouldn't. It's an old unfinished code that Vinny and I forgot to complete.

What does it need for completion?

We need to test the block of codes below that return to see if they actually work.
I remember we were in a hurry doing something else at that time, so we just returned an empty kids (line 383) as a hack to make this run -- instead of returning the kids that are properly initialised (line 384-389).

What should they do? I know they should make a spectrum, but in what way? Pull an intact spectrum from the dataset?

I think we wanted to sample ms2 spectrum from the dataset and adjust the relative intensities of ms2 peaks in that spectrum, so it's proportional to the parent ms1 peak?
Add comments below, based on looking at the codes:
 def _get_children_spectra(self, parent):
     # this should hopefully sample an intact ms2 spectrum from the dataset?
     spectra = self.peak_sampler.get_ms2_spectra()[0]
     kids = []

    # turn ms2 spectrum intensities into relative intensities
     intensity_props = self._get_msn_proportions(None, None, spectra.intensities)

    # randomly sample one `parent_mass_prop`
    # this is the proportion of all fragment intensities in a spectra over the parent intensity
    # returns number between 0 and 1
     parent_mass_prop = self.peak_sampler.get_parent_intensity_proportion()

    # create ms2 peaks having the right intensities??
     for i in range(len(spectra.mzs)):
         kid = MSN(spectra.mzs[i], spectra.ms_level, intensity_props[i], parent_mass_prop, None, parent)
         kids.append(kid)
     return kids

ok, so it:

grabs a spectrum
normalises the spectrum (so max is 1: https://github.com/sdrogers/vimms/blob/ec4d0da06a03b6cf6e77409381739e5ac398afcc/vimms/Chemicals.py#L442-L444)
Get an intensity proportion from the peak sampler...how much of the parent intensity becomes fragments
create the MS2 peaks with the correct intensities

It looks good to me, I'll try uncommenting and see what happens 👍

it would be good if we have another file to test the chemical generation process? Maybe can create one test_chemicals.py next to test_controllers.py.

yes, please do so - separate issue though


Xilinx vivado log4j issue

I am using CFU + Vivado 2020.1 which is using the log4j-1.2.15.jar.
Is it possible to upgrade Xilinx tools to the latest version and use it with CFU?

Hi @akku-t , we don't know of any problems using CFU Playground with the most recent release of Vivado, but we haven't tested it.
If you want, I can assist you installing and using Symbiflow tools instead of Vivado (if you're using a XC7 family device).


Can not compile in Ubuntu16.04

==> rel (generate)
ERROR: Unable to generate spec: read file info /usr/lib/erlang/man/man1/x86_64-linux-gnu-gcov-tool.1.gz failed
ERROR: Unexpected error: rebar_abort
ERROR: generate failed while processing /home/dcy/app/emqttd/rel: rebar_abort
Makefile:36: recipe for target 'rel' failed
make: *** [rel] Error 1

@dcy, It seems rebar cannot read the man files:
/usr/lib/erlang/man/man1/gcov-tool.1.gz
/usr/lib/erlang/man/man1/x86_64-linux-gnu-gcov-tool.1.gz

@dcy I think this can be solved with a workaround by deleting a symlink and rerunning the make command rm /usr/lib/erlang/man. Don't know if this breaks something else down the line.


Opting in to ALGO throws an index error

Calling client.prepare_asset_optin_transactions(0) throws
txns = client.prepare_asset_optin_transactions(asset.id, ADDRESS)
  File "D:\Users\User\AppData\Local\Programs\Python\Python38\lib\site-packages\tinyman\v1\client.py", line 49, in prepare_asset_optin_transactions
    txn_group = prepare_asset_optin_transactions(
  File "D:\Users\User\AppData\Local\Programs\Python\Python38\lib\site-packages\tinyman\v1\optin.py", line 20, in prepare_asset_optin_transactions
    txn = AssetOptInTxn(
  File "D:\Users\User\AppData\Local\Programs\Python\Python38\lib\site-packages\algosdk\future\transaction.py", line 1436, in __init__
    super().__init__(
  File "D:\Users\User\AppData\Local\Programs\Python\Python38\lib\site-packages\algosdk\future\transaction.py", line 1354, in __init__
    self.index = self.creatable_index(index, required=True)
  File "D:\Users\User\AppData\Local\Programs\Python\Python38\lib\site-packages\algosdk\future\transaction.py", line 301, in creatable_index
    raise IndexError("Required an index")
IndexError: Required an index

As I understand it this should never actually be used. I've added a check in #31 to address this

Thank you for raising this issue. I agree with you; it shouldn't be used for ALGO (0 asset id for Tinyman SDK).


[Feature] Improve and fix Prometheus & Grafana integrations

Why are these changes needed?
The old document (observability.md) of Prometheus and Grafana has some issues that make users hard to follow.

Inconsistency between install.sh and observability.md

ServiceMonitor example in the doc and ServiceMonitor instance created by the script.
PodMonitor example in the doc and PodMonitor instance created by the script.

Inconsistency between observability.md and KubeRay operator kuberay/#230.

The document uses port 9001 as a metrics endpoint, but KubeRay operator uses 8080 as a default metrics endpoint.

Some missing pieces:

Both examples of ServiceMonitor and PodMonitor in observability.md lack some labels to match serviceMonitorSelector and podMonitorSelector.
How to access Prometheus UI?
How to access Grafana UI?
How to import Ray Dashboard into Grafana?

Related issue number
Closes #871
Checks

[ ] I've made sure the tests are passing.
Testing Strategy

[ ] Unit tests
[x] Manual tests
[ ] This PR is not tested :(

Prometheus UI: Prometheus can detect both ServiceMonitor (ray-head-monitor) and PodMonitor (ray-worker-monitor).

Grafana UI: Load Ray Dashboard into Grafana via config/grafana/dashboard_default.json.

cc @Yicheng-Lu-llll would you mind reviewing this PR? Thanks!

Gentle ping @scarlet25151. Would you mind taking a look at this PR? Thank you!


STORM-160 Allow ShellBolt to set env vars (particularly PATH)

Support to allow ShellBolt to set environment variables
Example :
WordCountTopology.java
builder.setBolt("split", (IRichBolt) new SplitSentence().setEnv(envMap) ,  8).shuffleGrouping("spout");
where envMap is a map of environment variables to be exported.
Map<String, String> envMap = new HashMap<>();

envMap.put("PATH","/usr/local"); // append to PATH variable

envMap.put("HOME", "/usr/local"); // override HOME variable
NOTE: PATH variable is appended by default while others are overridden.

Looks good to me. Check why the build fails

As someone who has been following this issue for a long time, thanks for putting this together!  It would be even better if there were a way to specify these files via the Clojure DSL too.

The build failure seems to be only with JDK 7 during the  storm.kafka.TridentKafkaTest
[main] ERROR kafka.producer.async.DefaultEventHandler - Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: test
It passes with JDK 8.

Regarding test failures, there're random failures on Kafka tests with Travis CI.
So if failure occurs only Kafka tests, you can ignore them for now.
LGTM on changeset.
Btw, seems like we can apply this to ShellSpout, too. What do you think?

Thanks HeartSaVioR. Yes we can apply this to ShellSpout as well. I will do the changes shortly.

The changes to ShellSpout have been committed now.

LGTM.
Btw, there're many unrelated commits so I can hardly see actual changeset easily.
How about creating new pull request or rebase which points to current master + actual changeset?

Thanks HeartSavior. I have cleaned up the commit history now.

Overall looks good to me.  Just a few very minor comments.

Thanks Derek for the comments. I have now pushed in the changes.

@bourneagain
Travis CI complains that your PR has compile failure cause of missing imports. Could you fix it?
One minor note, please post / add commits after building / testing from your local machine. Thanks!

Thanks @HeartSaVioR . I have fixed the imports issue and tested on local machine to see it working fine. I shall keep a note of this in future. Thanks.

Thanks @HeartSaVioR . I apologize for the mistake. I have now removed the local debug I had put on during testing.

+1, LGTM.

@HeartSaVioR  Thanks. Can this be merged into the master now?

@bourneagain Yes, let's merge. :)

I merged it into master branch at ASF git repos, but seems like Github mirror is out of sync.
I'll resolve issue when mirror keeps up ASF repo.

Thank you @HeartSaVioR


custom the objective function for LTR

Hi there,
I noticed that there is a sample for custom objective:
https://github.com/Microsoft/LightGBM/blob/master/R-package/demo/multiclass_custom_objective.R
However,  how shall we define a custom objective when the objective includes some other information other than preds and dtrain? For example, in the lambdarank case, we may have the query_id information as well.

I figure it out.


Cannot stringify nested input arguments

This code:
execute(mutation$.upsertIssue(
	{
		by: {
			year: 2022,
			issue: 19,
		},
		create: {
			title: "A",
			subtitle: "B",
		},
		update: {
			title: "A",
			subtitle: "B",
		},
	},
	issueUpsertResult$.ok.errorMessage,
))

Results in the following query
mutation {
        upsertIssue(
                by: ,
                "year": 2022,
                "issue": 19,
                create: ,
                "title": "A",
                "subtitle": "B",
                update: ,
                "title": "A",
                "subtitle": "B"
        ) {
                ok
                errorMessage
        }
}

Expected mutation:
mutation {
        upsertIssue(
                by: { "year": 2022, "issue": 19 },
                create: { "title": "A", "subtitle": "B" },
                update: { "title": "A", "subtitle": "B" },
        ) {
                ok
                errorMessage
        }
}

I tested, cannot reproduce.

I reproduced.
This is a bug, although it's not recommended style: hard code style.
execute(mutation$.doSomething({...blalba...}, fetcher));
You should use the more recommended code: parameterized style
execute(         mutation$.doSomething(fetcher),          { variables: {...blalba...} } );
Parameterized style can run successful, so you can continue your job. In fact you should always use parameterized style.
Even if I fix this bug in the future to make hardcoded style can work, please don't use the hardcoded style.

3.0.17 is released.
You can use hardcode style code to validate it again. If it works, please close this issue.
Remember, even if this issue has been resolved, please don't use hardcode style in your real projects.

This works very well! Thanks a lot for the quick response!


[Event Request] Page 9305 Sales Order List - Action Posting - OnAfterPostingSetSelectionFilter

Please add the following Event/-s OnAfterPostingSetSelectionFilter to Action Posting in Page 9305 Sales Order List
[Event Request] PAG9305 - Sales Order List - Action Posting
                    trigger OnAction()
                    var
                        SalesHeader: Record "Sales Header";
                        SalesBatchPostMgt: Codeunit "Sales Batch Post Mgt.";
                        BatchProcessingMgt: Codeunit "Batch Processing Mgt.";
                    begin
                        CurrPage.SetSelectionFilter(SalesHeader);
                        // NEW
                         OnAfterPostingSetSelectionFilter(SalesHeader,Rec)
                        // NEW
                        if SalesHeader.Count > 1 then begin
                            BatchProcessingMgt.SetParametersForPageID(Page::"Sales Order List");

                            SalesBatchPostMgt.SetBatchProcessor(BatchProcessingMgt);
                            SalesBatchPostMgt.RunWithUI(SalesHeader, Count, ReadyToPostQst);
                        end else
                            PostDocument(CODEUNIT::"Sales-Post (Yes/No)");
                    end;

    [IntegrationEvent(false, false)]
    local procedure OnAfterPostingSetSelectionFilter(var SalesHeaderToPost: Record "Sales Header"; CurrPageSalesHeader : Record "Sales Header")
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 13199.


Add linting and build step sanity check to Travis CI.

npm run lint should pass.
npm run build should pass.

https://github.com/aframevr/aframe-core/pull/770


Poor testing data recon

Hi.
I used the code to train on my cortical surface data. after 6 days of training, the training data recon is good, but it's not able to reconstruct the test data at all.
The loss is 0.0012, sdf loss: 0.00098 and vae loss: 1^-5.
Can you give any suggestions? Is it more about hyper-parameter tuning? My dataset have 600 surfaces.
@gene-chou
this is the test recon

and this is the training data recon.

I think this data seems quite difficult to reconstruct. It's possible that the conditioning point cloud doesn't have enough information to reconstruct smooth surfaces. Additionally, see #25 regarding data.

Hey thanks for your response @gene-chou
I am using Pointcloud with 30K points. When visualizing it, they are quite clear with enough density. Since the model is able to reconstruct the training data well, I am assuming its a generalization issue.
by any chance did you face anything like this during your trainings?

Yes, generalization required either more data or more hyperparameter tuning. Later I had a chance to train on Objaverse and I was able to get good results without any tuning.

Hey
Thanks!
I'll see if I can increase my dataset :)


minCount on Task in applicationmetadata.json not working

Description of the bug
The app in the making follows this process:

The user, in this case DAGL+REGNA+UTINN, fills out the form, signs it, and then another user, REVI, signs. This completes the process.
At each confirmation stage, there is choice between rejecting and signing the document.

If the user presses the sign button, then a signature is added.
If they press reject, then no signature is added.

In applicationmetadata.json both of the processes are defined with a minimum count of 1 for signature:
{
     "id": "signature1",
      "allowedContentTypes": [
        "application/json"
      ],
      "taskId": "Task_2",
      "maxCount": 1,
      "minCount": 1,
      "enablePdfCreation": false,
      "enableFileScan": false,
      "validationErrorOnPendingFileScan": false,
      "enabledFileAnalysers": [],
      "enabledFileValidators": []
    }
{
     "id": "signature2",
      "allowedContentTypes": [
        "application/json"
      ],
      "taskId": "Task_3",
      "maxCount": 1,
      "minCount": 1,
      "enablePdfCreation": false,
      "enableFileScan": false,
      "validationErrorOnPendingFileScan": false,
      "enabledFileAnalysers": [],
      "enabledFileValidators": []
    }

However, it is still possible for signature2 to occur before signature1 or even complete the form without any signatures.
Steps To Reproduce
This can be reproduced with the existing signing-test app.
However, this app is created with minCount: 0, so this first needs to be changed to 1 for both signatures.

Fill out the form and press the submit button
Choose reject with the role of either DAGL or REGN
Then log in again with the role of REVI, open the form and press sign.

I followed these steps while testing locally and got a completed form with only one signature

Additional Information
Tests have only been done from local testing, not tt02.

This turned out to be a local testing issue. When testing on test environment the form was not added to the inbox if the form-filler pressed reject instead of sign.

It seems it is not a local testing issue after all. After testing my own app in tt02, I noticed the form is still submitting even though there is a lack of signatures. I have edited the issue to reflect this change

fixed with v4


Normalize does not set  display block

What should happen?
normalize should set the <main> tag to display: block,
like it has done at version 6.4.4-rc1 with normalize.scss
What happens instead?
Nothing is set for the <main> tag. IE is known for not implementing/support
the main tag, so at least display: block have to be set to make it behave
like a div.
Possible Solution

Consider using another vendor normalize
Introduce your own normalization with:
main { display: block; }

Test Case and/or Steps to Reproduce (for bugs)
Use a <main> tag in IE with foundation and add for instance a background color.
Place some content inside the main and the background color will not be shown.

https://github.com/necolas/normalize.css/blob/master/normalize.css#L32
https://github.com/csstools/normalize.css/blob/master/normalize.css#L50
Can you provide a codepen which reproduces this?

https://github.com/necolas/normalize.css/blob/master/normalize.css#L32
https://github.com/csstools/normalize.css/blob/master/normalize.css#L50
It seems this was fixed in normalize 8.0.1, see https://github.com/necolas/normalize.css/blame/8.0.1/normalize.css#L32 and https://github.com/necolas/normalize.css/commit/df07c00a92c683e591b68bcc45075ff7253b7482
I propose that we switch to csstools/normalize.css which is the right normalize imo.

You are right, you are using normalize 8.0.0 at the moment:
https://github.com/zurb/foundation-sites/blob/develop/scss/vendor/normalize.scss#L2
So updating should fix it too

Well, in the long run, updating is not the solution. We have to migrate to csstools/normalize.css. See https://github.com/csstools/normalize.css/issues/3#issuecomment-421401639 and https://github.com/necolas/normalize.css/issues/664

more followed and maintained

Would you quantify that? I maintained the necolas version while he walked away from the project. I’ll admit that it’s hard to follow updates to that version since I’m blocked from the project.

@jonathantneal I corrected my comment.

normalize.css is a project by Jonathan Neal, co-created with Nicolas Gallagher.

Please stop your BS. The reason you're blocked is because you gaslight me.

@jonathantneal @jonathantneal
Hello Guys, I'm happy to see you two there, but at first I have to make this very clear:
You will not start another fight.

I’m really sorry to read that, Nicolas. I would love, love to be on good terms again. I understand if you’re not interested at this time, and I do hope it changes. Would you consider unblocking me from the project if I agree not to comment on any issues until you say it’s okay? It really sucks to not be un-included like that.
As for this thread; y’all pick the libraries that serves your needs best.
I think the issue of body { margin: 0 } is a kind of tabs vs spaces thing. I tend to think normalize should stick to fixes, while many other intelligent people think certain opinions are practical.
I would have personally expected people to “die on a hill” for adding * { box-sizing: border-box } before preserving body { margin: 0 } but the later has obviously had a major influence on the project. I didn’t expect that.
@ncoden, totally understood. Please delete this message if you feel it’s aggravating.

@necolas, I just saw your updated comment. Any chance we can talk? My email is<EMAIL_ADDRESS>I’d love a voice call, just incase our typed words come across the wrong way to each other.

If I may continue with the purpose of this issue and the problem that Foundation is facing.
We care more about our developer's best interests than the struct following of the "browser normalization" approach. But on the other side, we want to rely on a watched and well-maintained project.
@necolas What is the maintenance status of necolas/normalize.css? Is there someone that currently ensure a technological watch on the project? As you can see, many improvments were provided in the csstools' version this year. Would you consider adding them to your version?
@jonathantneal Developers generally don't like separations in communities and projects, and most often it doens't end very well. Do you think that creating a second normalize.css project is the best solution? Would there be others options that would be in the best interest of the whole community (for example at Foundation we generate several .css so the user can pick the one that fit the best its needs) ?

@ncoden, I will think more about that. At the moment, I do not like having 2 separate projects. I would prefer the “opinionated” and “fixes only” versions both live in the same repo. I can add the opinionated version to csstools if you think that would be helpful.
I want to be sensitive to others who think the projects should be separate with separate names. For instance, there is @sindresorhus’ excellent and well-known version https://github.com/sindresorhus/modern-normalize/. How much of a name change works?
In the meantime, could you take any fixes from mine and make them PRs on necolas’ version if they are not already? That should make it easier to use his version and end any immediate concerns.

I want to be sensitive to others who think the projects should be separate with separate names.

In the meantime, could you take any fixes from mine and make them PRs on necolas’ version if they are not already? That should make it easier to use his version and end any immediate concerns.

IMHO these 2 solutions in the same time would be the best. Using an other name to avoid confusions, and importing each project advantages into the other. I hope that this conflict will be resolved one day, but until then we would have two distinguishable projects, one with opinionated rules and the other without.

Would you mind helping me with the name? Naming things is hard. Right now it’s @csstools/normalize.css on npm and the README.md just calls it normalize.css, since they have the same initial git history.
I’ve immediately updated the README.md to call it csstools-normalize (following the sindresorhus convention) and I’ve added the opinionated body style. I can add the others after referencing the current necolas release.
Anything else I can do to help?

@DanielRuf What do you think?

@jonathantneal The best you could do is a csstools/normalize version has a distinguishable name and a stable version that include an "opinionated" option (with all the rules from the @necolas' version, not something made in a rush).
Once this is done, released and stable, we could switch to csstools/normalize. But I'd like to get the opinon of @DanielRuf on this. @necolas I'd love to have your opinon too.

He’s asked not to be mentioned in this thread. If you could edit your comment, I will delete this one.

Please stop mentioning me in this thread

@jonathantneal There is something I did not understood: What is the difference between csstools-normalize and sanitize.css ?

csstools’ normalize is just browser fixes, while sanitize is normalize plus opinionated changes based on developer preferences — things that will never ship as the default — things like box-sizing: border-box.

@jonathantneal What is the difference with necolas' normalize then?

That normalize includes 2 opinionated styles — font styles on inputs and margin removal on the body.


Support cognito admin user password auth flow

Applies the user credentials pattern from the ADMIN_NO_SRP_AUTH flow
to the ADMIN_USER_PASSWORD_AUTH auth flow for Cognito admin_initiate_auth
requests.
2nd attempt, hopefully the build works this time.

Coverage decreased (-0.3%) to 94.313% when pulling f560307982ec66081f21579ec3f50b2e143cd929 on robjwilkins:support_cognito_ADMIN_USER_PASSWORD_AUTH_flow_v2 into c9c30b82867294030833cef292db167955bc8240 on spulec:master.

This fixes #2663

thanks @bblommers
Do you know the process for getting the Docker image on docker hub updated (https://hub.docker.com/r/motoserver/moto)? I guess this isn't part of the CICD pipeline.

No, @robjwilkins, it's not part of the pipeline. It typically only  happens with a full release.
There is an outstanding issue for this: #3203


Add support for Tempest API

The API looks pretty simple.
Test URL:  https://swd.weatherflow.com/swd/rest/observations?api_key=a8f5dbda-af0a-4b57-99b9-f10baa88f27b&device_id=280616&bucket=b&time_start=1690221094&time_end=1690228535
Field documentation: https://apidocs.tempestwx.com/reference/observation-record-format

Done


Need example for sharding/replica set

before posting here I've googled, but not found.
Can some one provide example for morphia with replica set or shardinga and which annotations need to use for replica set?
Thanks in advance

There is no difference in your app in those cases. Morphia has nothing to do there.


Using "ES6 Import" instead of require lead to TypeError after compilation

Hello,
I tried to use
import Twig from 'twig';

instead of
const Twig = require('twig').twig;

And the build passes successfully but when I'm loading the app in the browser, I got this error:
TypeError: he() is not a function (he being the Twig import)
I may missed something but I was wondering if twigjs is ES6-compatible ?

No, it does not currently support ES modules. This is a duplicate of #622.


Idea: Connection to matplotlib over BytesIO

Note: This is only an idea, and should not have high priority.
Related to #363
It would be really cool to have manim smoothly interact with matplotlib.
Therefore one idea:
Do you think it is possible, to save figures created in matplotlib to a byte stream, which is then read by ImageMobject?
Then there is no reading and writing to the disk in between.
One ide based on the idea of https://mellowd.dev/python/using-io-bytesio/, I tried the following:
from manim import *
import matplotlib.pyplot as plt
import io

class ConnectingMatplotlib(Scene):
    def construct(self):
        t = np.arange(0.0, 2.0, 0.01)
        s = 1 + np.sin(2 * np.pi * t)
        fig, ax = plt.subplots()
        ax.plot(t, s)
        ax.set(xlabel='time (s)', ylabel='voltage (mV)',
               title='About as simple as it gets')
        ax.grid()

        b = io.BytesIO()
        plt.savefig(b, format='png')
        img = ImageMobject(b.read())  ## error: IndexError: tuple index out of range
        self.add(img)

But reading the byte stream is not possible just like that.
Do you think this is something, that can be easily solved?

For me, running the code from https://mellowd.dev/python/using-io-bytesio/ always leads to an empty image. I think that before accessing the byte stream, something like b.seek(0) has to be called.
Anyhow, matplotlib also allows to get the rendered figure in a format suitable for ImageMobject by calling buffer_rgba() of the corresponding canvas:
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

from manim import Scene, ImageMobject

class ConnectingMatplotlib(Scene):
    def construct(self):
        t = np.arange(0.0, 2.0, 0.01)
        s = 1 + np.sin(2 * np.pi * t)

        fig, ax = plt.subplots()
        ax.plot(t, s)

        ax.set(xlabel='time (s)', ylabel='voltage (mV)',
               title='About as simple as it gets')
        ax.grid()

        fig.canvas.draw()
        img = ImageMobject(fig.canvas.buffer_rgba())

        self.add(img)
        self.wait()

@behackl : Wow,thank you so much! I'm speechless and happy!
With saving and loading from disk (my method):
--- 35.34524869918823 seconds ---
With your method:
--- 6.816277742385864 seconds ---
And here is a script:
from manim import *
import matplotlib.pyplot as plt

def my_function(amplitude, x):
    return amplitude * np.sin(x)

def get_image_plot(amplitude, x):
    fig, ax = plt.subplots()
    ax.plot(x, my_function(amplitude, x))
    plt.ylim(-1, 1)
    ax.set(xlabel='distance (x)', ylabel='amplitude',
           title='My animated plot')
    fig.canvas.draw()
    img = ImageMobject(fig.canvas.buffer_rgba()).scale(4.5)
    plt.close(fig)
    return img

class ConnectingMatplotlib(Scene):
    def construct(self):
        x_values = np.linspace(0, 30, 400)
        amp1 = 0.5
        amp2 = 1
        tr_amplitude = ValueTracker(amp1)
        image = get_image_plot(amp1, x_values)
        self.add(image)

        def update_image(mob):
            new_mob = get_image_plot(tr_amplitude.get_value(), x_values)
            mob.become(new_mob)

        image.add_updater(update_image)

        self.play(tr_amplitude.set_value, amp2, run_time=3)

import time
from pathlib import Path

if __name__ == "__main__":
    script = f"{Path(__file__).resolve()}"
    start_time = time.time()

    os.system( f"manim  -l --custom_folders  --disable_caching  -p -c 'BLACK" + script)
    print("--- %s seconds ---" % (time.time() - start_time))

oh snap this is awesome! Perhaps in the future we may have to define a MatplotlibScene if this feature becomes popular.

I am also super excited about what this matplotlib connection could become!!
I mean, with this feature, you could take more or less every existing figure, and easily configure them with rate functions, set the timings, and use Valuetrackers for certain parameters. You can also set the figures with functions like next_to, and change with e.g. fade in and fade out, which is a huge advantage about the way matplotlib figures are animated at the moment.
As matplotlib is a very big library, maybe we could even make an own package out of it?
Something like maniplot? :D

manimplotlib


https version of site not marked as anomoly despite same fingerpint

http://explorer-beta.ooni.io/search?until=2019-08-01&input=https%3A%2F%2Fwww.gaystarnews.com%2F&probe_cc=MY&since=2017-06-01
https://www.gaystarnews.com for Malaysia is marked as anamoly

https://explorer-beta.ooni.io/measurement/20190721T105616Z_AS4818_BTmbtJBZKG8XxS5jFnvULtBun1EafR62hooaLWbD1zYKSVicpJ?input=https://www.gaystarnews.com/

http://www.gaystarnews.com for Malaysia is marked as confirmed

https://explorer-beta.ooni.io/measurement/20190619T024122Z_AS4788_u8hzWglxR8DTfzIsgR34QQM0JsEXiPKKZTbO715obVRQJKTDJh?input=http://www.gaystarnews.com/

Both of these results have results that should match the fingerprint for a block page in Malaysia:
"<IP_ADDRESS>"

This is because we currently only look at the HTTP response body for detecting the fingerprint and not the DNS queries.
The HTTPS request is failing, because the cert is not valid and hence we don't get the blockpage.
We should start using DNS anomalies to compute these flags too in the future.


[WIP] Ref stores raw pointer

THIS IS UNSOUND!!! See https://github.com/google/zerocopy/issues/368.


How do i create a custom skill compendium?

It's the title, on the page it says i should be able to create a skill compendium so i can use five point fudge with my compendium, but i can't find a way to do it.
Once i created my game (set in something of a cthulhu-like universe, i can let you publish it into the system if you like)

Sorry, i found a way a little after posting it here. Keep up the good work! This system is the best system!


Persisting blob in store

@zhanziyang I'm using a vuex plugin that persists the session state, but the blob output doesn't persist after page refresh. It becomes an empty object. (it's the only one with this behavior). Do you know why?

I don't understand. Technically everything inside vuex store get destroyed after page refresh, right?

Right, but I'm using this plugin: vuex-persistedstate. I'll try to do in another way here. Thanks


Using append to compose multiple independent SQL queries

If I have 2 different queries like:
const seedOne = SQL`
  INSERT INTO table (col1, col2) VALUES (1, 2);
`;

const seedTwo = SQL`
  INSERT INTO table (col1, col2) VALUES (3, 4);
`;

Is there a way to compose seedOne and seedTwo together so I can run a single query?
const seedCombined = seedOne + seedTwo

const seedCombined = seedOne.append(seedTwo)

?

That's it? Awesome.


Add syntax highlighting to mdx code blocks

Guide: https://mdxjs.com/guides/syntax-highlighting
Use this library: https://github.com/FormidableLabs/prism-react-renderer

#23


FetchError: request to http://<IP_ADDRESS>:8899/ failed, reason: socket hang up

I get error FetchError: request to http://<IP_ADDRESS>:8899/ failed, reason: socket hang up during start anchor test
How I can investigate it? If i remove CPI in my program tests pass

My validator.log:
https://gist.github.com/beautyfree/a37d0637d5fdaf3983ae7a65276759c6


Test driver Integration

Description:
This PR is regarding changes for integrating ion-js with ion-test-driver.
Completetd Tasks:

Make test-driver separate from ion-js

remove compareEquivs from IonEventStream
remove use of IonCliError and IonComparisonReport from IonEventstream.
remove use of equivs method fromiontests
Placed comparisonResult in src for using it in compare methods.
modularize the test-driver code.

test-driver package settings

create package.json for test-driver.
create tsconfig for test-driver.
add grunt task to build test-driver.
add gitignore files for test-driver.

TO DO:

remove unnecessary import from iontests.

Completed Tasks:

added scripts in package.json to run and build test-driver from root directory ion-js. (0dc2a74)
added code to write event stream with available events before throwing error. (31544b3)
check null for event streams in all (basic, equivs and non-equivs) comparisons. (b02596e)
added parseFieldName with support of symbolTokens. (616b677)
changes to give correct event index when writing comparison report and Cli Errors. (8eb0f62)


bloop: 1.0.0 -> 1.1.0

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Oh, I didn't realize it's unfree. Built locally with sandbox on, looks good.


opencv  error during training process

I1220 07:38:26.009254 19032 solver.cpp:362] Iteration 3860, Testing net (#0)
I1220 07:38:26.009270 19032 net.cpp:723] Ignoring source layer train_data
I1220 07:38:26.009274 19032 net.cpp:723] Ignoring source layer train_label
I1220 07:38:26.009274 19032 net.cpp:723] Ignoring source layer train_transform
I1220 07:40:26.499217 19032 solver.cpp:429]     Test net output #0: loss_bbox = 1.63662 (* 2 = 3.27324 loss)
I1220 07:40:26.499321 19032 solver.cpp:429]     Test net output #1: loss_coverage = 10.7671 (* 1 = 10.7671 loss)
I1220 07:40:26.499325 19032 solver.cpp:429]     Test net output #2: mAP = 0
I1220 07:40:26.499328 19032 solver.cpp:429]     Test net output #3: precision = 0
I1220 07:40:26.499331 19032 solver.cpp:429]     Test net output #4: recall = 0
I1220 07:41:21.482302 19032 solver.cpp:242] Iteration 3888 (0.226819 iter/s, 211.623s/48 iter), loss = 13.1519
I1220 07:41:21.482405 19032 solver.cpp:261]     Train net output #0: loss_bbox = 1.5995 (* 2 = 3.199 loss)
I1220 07:41:21.482411 19032 solver.cpp:261]     Train net output #1: loss_coverage = 13.5516 (* 1 = 13.5516 loss)
I1220 07:41:21.482417 19032 sgd_solver.cpp:106] Iteration 3888, lr = 7.54746e-06
I1220 07:42:52.471341 19032 solver.cpp:242] Iteration 3936 (0.527531 iter/s, 90.99s/48 iter), loss = 24.961
I1220 07:42:52.471431 19032 solver.cpp:261]     Train net output #0: loss_bbox = 0.981416 (* 2 = 1.96283 loss)
I1220 07:42:52.471437 19032 solver.cpp:261]     Train net output #1: loss_coverage = 0.995832 (* 1 = 0.995832 loss)
I1220 07:42:52.471444 19032 sgd_solver.cpp:106] Iteration 3936, lr = 7.50547e-06
I1220 07:44:23.474804 19032 solver.cpp:242] Iteration 3984 (0.52745 iter/s, 91.0039s/48 iter), loss = 6.86588
I1220 07:44:23.474898 19032 solver.cpp:261]     Train net output #0: loss_bbox = 1.11368 (* 2 = 2.22736 loss)
I1220 07:44:23.474903 19032 solver.cpp:261]     Train net output #1: loss_coverage = 3.99163 (* 1 = 3.99163 loss)
I1220 07:44:23.474910 19032 sgd_solver.cpp:106] Iteration 3984, lr = 7.46417e-06
I1220 07:45:54.492636 19032 solver.cpp:242] Iteration 4032 (0.527369 iter/s, 91.0179s/48 iter), loss = 18.8171
I1220 07:45:54.492748 19032 solver.cpp:261]     Train net output #0: loss_bbox = 1.66934 (* 2 = 3.33869 loss)
I1220 07:45:54.492753 19032 solver.cpp:261]     Train net output #1: loss_coverage = 9.52689 (* 1 = 9.52689 loss)
I1220 07:45:54.492760 19032 sgd_solver.cpp:106] Iteration 4032, lr = 7.42355e-06
I1220 07:47:25.475150 19032 solver.cpp:242] Iteration 4080 (0.527576 iter/s, 90.9822s/48 iter), loss = 14.7126
I1220 07:47:25.475260 19032 solver.cpp:261]     Train net output #0: loss_bbox = 4.48436 (* 2 = 8.96873 loss)
I1220 07:47:25.475265 19032 solver.cpp:261]     Train net output #1: loss_coverage = 11.6653 (* 1 = 11.6653 loss)
I1220 07:47:25.475271 19032 sgd_solver.cpp:106] Iteration 4080, lr = 7.38358e-06
I1220 07:48:56.459316 19032 solver.cpp:242] Iteration 4128 (0.527568 iter/s, 90.9836s/48 iter), loss = 22.178
I1220 07:48:56.459406 19032 solver.cpp:261]     Train net output #0: loss_bbox = 0.91269 (* 2 = 1.82538 loss)
I1220 07:48:56.459411 19032 solver.cpp:261]     Train net output #1: loss_coverage = 6.5104 (* 1 = 6.5104 loss)
I1220 07:48:56.459419 19032 sgd_solver.cpp:106] Iteration 4128, lr = 7.34426e-06
I1220 07:50:27.478549 19032 solver.cpp:242] Iteration 4176 (0.527366 iter/s, 91.0184s/48 iter), loss = 10.1156
I1220 07:50:27.478693 19032 solver.cpp:261]     Train net output #0: loss_bbox = 6.9837 (* 2 = 13.9674 loss)
I1220 07:50:27.478699 19032 solver.cpp:261]     Train net output #1: loss_coverage = 15.8192 (* 1 = 15.8192 loss)
I1220 07:50:27.478705 19032 sgd_solver.cpp:106] Iteration 4176, lr = 7.30555e-06
I1220 07:51:58.494304 19032 solver.cpp:242] Iteration 4224 (0.527388 iter/s, 91.0147s/48 iter), loss = 13.4028
I1220 07:51:58.494428 19032 solver.cpp:261]     Train net output #0: loss_bbox = 2.07173 (* 2 = 4.14347 loss)
I1220 07:51:58.494433 19032 solver.cpp:261]     Train net output #1: loss_coverage = 6.74864 (* 1 = 6.74864 loss)
I1220 07:51:58.494441 19032 sgd_solver.cpp:106] Iteration 4224, lr = 7.26745e-06
I1220 07:52:38.301617 19032 solver.cpp:362] Iteration 4246, Testing net (#0)
I1220 07:52:38.301697 19032 net.cpp:723] Ignoring source layer train_data
I1220 07:52:38.301700 19032 net.cpp:723] Ignoring source layer train_label
I1220 07:52:38.301702 19032 net.cpp:723] Ignoring source layer train_transform
OpenCV Error: Null pointer (create() called for the missing output array) in create, file /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp, line 2492
OpenCV Error: Null pointer (create() called for the missing output array) in create, file /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp, line 2492
Traceback (most recent call last):
File "/home/young/SoftWare/caffe/python/caffe/layers/detectnet/clustering.py", line 133, in forward
bbox = cluster(self, data0, bottom[1].data)
File "/home/young/SoftWare/caffe/python/caffe/layers/detectnet/clustering.py", line 227, in cluster
boxes_cur_image = vote_boxes(propose_boxes, propose_cvgs, mask, self)
File "/home/young/SoftWare/caffe/python/caffe/layers/detectnet/clustering.py", line 193, in vote_boxes
self.gridbox_rect_eps)
cv2.error: /tmp/binarydeb/ros-kinetic-opencv3-3.2.0/modules/core/src/matrix.cpp:2492: error: (-27) create() called for the missing output array in function create
And I got this error in terminal " Train Caffe Model task failed with error code 1
".
I had tried three kind of detection mission. After I trained for a while, it gave me this error. Is anyone getting this error?
Thanks.

I solved this by changing the opencv version to libopencv4tegra and libopencv4tegra-dev libopencv4tegra-repo libopencv4tegra-python.
So I guess it's the opencv environment causes the fault.
After this change my training goes well, and the mAP value goes right after several iteration.
Just in case, some one else would fall into this trouble.


Improve check_rebuild_pkgs

Signed-off-by: tu1h<EMAIL_ADDRESS>
What type of PR is this?

/kind feature
What this PR does / why we need it:

Fixed a fatal where the build failed when the last tag did not generate the appropriate os-package. See pipeline
Fixed current packages.yml unchanged and with no file generation problem

See building pipeline
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

/lgtm

/lgtm

/approve


cli: add a client cli to interact with zot server

We could just follow the single binary model.

@rchincha should we close this ?

I think this can be closed. I tagged this in a PR but seems like I missed it in an edit later.


We need to set ALICE_WEBID before running the tests

If we don't set ALICE_WEBID then storage related tests will fail.
Related Issue: https://github.com/nodeSolidServer/node-solid-server/issues/1736

Actually, this is now diverging from how the run-against-nss script does it. Is that a conscious choice or an oversight?
https://github.com/solid-contrib/solid-crud-tests/blob/main/run-against-nss.sh#L7-L8


feat: Support more of the options that are available when calling flutter run

Right now I can't run tests with the following which is blocking me in some cases:

--web-port
--dart-define-from-file

Would we be able to get them added as options in the fluttium.yaml as well as when running tests?

Without --dart-define-from-file, using fluttium will be a bit cumbersome..

Yes that would be quite useful indeed! Shouldn't be hard to add either as both the YAML config and the CLI are pretty straight forward in how they add parameters if anyone wants to try.
I am not sure when I'll get to it myself as I am trying to refocus on #293


instance system.adapter.nuki-extended.0 terminated with code 6

Describe the bug
Seit kurzem geht der nuki adpater auf Rot ohne mein dazutun.
Wenn ich den Adapter dann wieder per Hand starte, wird wieder alles grün und er läuft ganz normal ohne Fehler.
Herzliche Grüße
Fabio
The nuki adpater recently went red without my doing anything. Then these error messages come up, which I only got after the update to v2.6.1.
If I start the adapter again by hand, everything turns green again and it runs normally without any errors.
Screenshots & Logfiles
host.nuc
2022-06-13 06:09:39.073	error	instance system.adapter.nuki-extended.0 terminated with code 6 (UNCAUGHT_EXCEPTION)
host.nuc
2022-06-13 06:09:39.073	error	Caught by controller[1]: at processImmediate (internal/timers.js:464:21)
host.nuc
2022-06-13 06:09:39.073	error	Caught by controller[1]: at Immediate.Async.drainQueues [as _onImmediate] (/opt/iobroker/node_modules/bluebird/js/release/async.js:15:14)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at Async._drainQueues (/opt/iobroker/node_modules/bluebird/js/release/async.js:102:5)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at _drainQueue (/opt/iobroker/node_modules/bluebird/js/release/async.js:86:9)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at _drainQueueStep (/opt/iobroker/node_modules/bluebird/js/release/async.js:93:12)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at Promise._settlePromises (/opt/iobroker/node_modules/bluebird/js/release/promise.js:725:18)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at Promise._settlePromise0 (/opt/iobroker/node_modules/bluebird/js/release/promise.js:649:10)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at Promise._settlePromise (/opt/iobroker/node_modules/bluebird/js/release/promise.js:604:18)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at Promise._settlePromiseFromHandler (/opt/iobroker/node_modules/bluebird/js/release/promise.js:547:31)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at tryCatcher (/opt/iobroker/node_modules/bluebird/js/release/util.js:16:23)
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at /opt/iobroker/node_modules/iobroker.nuki-extended/nuki-extended.js:602:13
host.nuc
2022-06-13 06:09:39.072	error	Caught by controller[1]: at setAction (/opt/iobroker/node_modules/iobroker.nuki-extended/nuki-extended.js:618:14)
host.nuc
2022-06-13 06:09:39.071	error	Caught by controller[1]: TypeError: nukiWebApi.setAction is not a function
host.nuc
2022-06-13 06:09:39.070	error	Caught by controller[0]: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
nuki-extended.0
2022-06-13 06:09:38.537	warn	Terminated (UNCAUGHT_EXCEPTION): Without reason
nuki-extended.0
2022-06-13 06:09:38.536	info	terminating
nuki-extended.0
2022-06-13 06:09:38.536	info	Adapter stopped und unloaded.
nuki-extended.0
2022-06-13 06:09:38.531	error	nukiWebApi.setAction is not a function
nuki-extended.0
2022-06-13 06:09:38.531	error	TypeError: nukiWebApi.setAction is not a function at setAction (/opt/iobroker/node_modules/iobroker.nuki-extended/nuki-extended.js:618:14) at /opt/iobroker/node_modules/iobroker.nuki-extended/nuki-extended.js:602:13 at tryCatcher (/opt/iobroker/node_modules/bluebird/js/release/util.js:16:23) at Promise._settlePromiseFromHandler (/opt/iobroker/node_modules/bluebird/js/release/promise.js:547:31) at Promise._settlePromise (/opt/iobroker/node_modules/bluebird/js/release/promise.js:604:18) at Promise._settlePromise0 (/opt/iobroker/node_modules/bluebird/js/release/promise.js:649:10) at Promise._settlePromises (/opt/iobroker/node_modules/bluebird/js/release/promise.js:725:18) at _drainQueueStep (/opt/iobroker/node_modules/bluebird/js/release/async.js:93:12) at _drainQueue (/opt/iobroker/node_modules/bluebird/js/release/async.js:86:9) at Async._drainQueues (/opt/iobroker/node_modules/bluebird/js/release/async.js:102:5) at Immediate.Async.drainQueues [as _onImmediate] (/opt/iobroker/node_modules/bluebird/js/release/async.js:15:14) at processImmediate (internal/timers.js:464:21)
nuki-extended.0
2022-06-13 06:09:38.530	error	unhandled promise rejection: nukiWebApi.setAction is not a function
nuki-extended.0
2022-06-13 06:09:38.530	error	Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch().
Versions:
nuki-extended.0V v2.6.1
Node.js v14.19.3
NPM 6.14.17
4.0.23

fixed in 2.6.3

Vielen Dank. ;-)


Pex on MySQL

Hi, everytime I want to change my pex to sql
(Backend: file --> sql) my pex breaks down. Am I doing something wrong or is it just a glitch?
Thanks for helping

Hello?

Thanks for the reply! Great ;-;

if you dont have enough patience to wait for a response (so far less than a day after posting) i dont have enough patience to respond.