Implement zigbee2mqtt protocol

1 Add option to disable lamps in Home Assistant wizard
2 Zigbee2mqtt driver is implemented. MQTT must be enabled and active in the HyperHDR->Advanced->Network section.

Hi,
i use now this build https://github.com/awawa-dev/HyperHDR/actions/runs/12522954359
Linux-bookworm-arm-32bit-armv6l-installer.

Setup mqtt under expert networkservice

2024-12-28T11:20:09.201Z [MQTT] (mqtt.cpp:109) Closing MQTT
2024-12-28T11:20:09.201Z [MQTT] (mqtt.cpp:51) Starting the MQTT connection. Address: <IP_ADDRESS>:1884. Protocol: NO SSL. Authentication: YES, Ignore errors: YES
2024-12-28T11:20:09.201Z [MQTT] (mqtt.cpp:56) MQTT topic: HyperHDR/JsonAPI, MQTT response: HyperHDR/JsonAPI/response
2024-12-28T11:20:09.208Z [MQTT] (mqtt.cpp:127) Connected

it stays connected

Normally after connecting, i get a new device in Fhem. But nothing.

Is HyperHdr sending periodically serverinfos over mqtt for state?

Because of something is wrong for me, i can not add a zigbee2mqtt device @ controllertyp.

You must run zigbee2mqtt wizard in the LED hardware. I will add a tutorial later in the wiki.

The problem is, that HyperHdr says, it is connected. But my broker is quiet.
Does HyperHdr send periodically status infos? I ask, because i need infos where to dig and find the problem.

No, it search for devices only when you run the wizard. Could you check HyperHDR logs when the app says it havent found zigbee devices? Are you able to view the communication in the mosquito? The z2m has the default topic zigbee2mqtt?

Here you can see, that the topic is zigbee2mqtt. At the picture, i'm conncted with MqttExplorer to my fhem broker.
It is some Fhem Modul, not mosquitto

When i run the wizard and the error appears, the log say:
2024-12-28T12:33:04.080Z [WEBSOCKET] (HyperAPI.cpp:1097) message: [{"command":"leddevice","ledDeviceType":"zigbee2mqtt","subcommand":"discover","tan":704}]
2024-12-28T12:33:05.581Z [LEDDEVICE_ZIGBEE2MQTT] Could not find any Zigbee2mqtt devices. Check HyperHDR MQTT network / MQTT broker / Zigbee2mqtt configuration
2024-12-28T12:33:05.581Z [LEDDEVICE_ZIGBEE2MQTT] (DriverNetZigbee2mqtt.cpp:279) devicesDiscovered: [{"devices":[],"ledDeviceType":"zigbee2mqtt"}]
2024-12-28T12:33:05.581Z [WEBSOCKET] (HyperAPI.cpp:1121) response: [{"devices":[],"ledDeviceType":"zigbee2mqtt"}]
2024-12-28T12:33:17.489Z [WEBSOCKET] (HyperAPI.cpp:946) log streaming activated for client ::ffff:<IP_ADDRESS>

For discovery topic zigbee2mqtt/bridge/devices is used. Could you send it manually and save the output (json)?

Mh, the topic gets only updated when i restart zigbee2mqtt. Otherwise it is quiet. Maybe i must update z2m or something
at my fhembroker is blocking it.
Topic:
zigbee2mqtt/bridge/devices
Value:
[{"date_code":"20181025-69","definition":{"description":"Tint LED bulb GU10/E14/E27 350/470/806 lumen, dimmable, color, opal white","exposes":[{"features":[{"access":7,"description":"On/off state of this light","label":"State","name":"state","property":"state","type":"binary","value_off":"OFF","value_on":"ON","value_toggle":"TOGGLE"},{"access":7,"description":"Brightness of this light","label":"Brightness","name":"brightness","property":"brightness","type":"numeric","value_max":254,"value_min":0},{"access":7,"description":"Color temperature of this light","label":"Color temp","name":"color_temp","presets":[{"description":"Coolest temperature supported","name":"coolest","value":153},{"description":"Cool temperature (250 mireds / 4000 Kelvin)","name":"cool","value":250},{"description":"Neutral temperature (370 mireds / 2700 Kelvin)","name":"neutral","value":370},{"description":"Warm temperature (454 mireds / 2200 Kelvin)","name":"warm","value":454},{"description":"Warmest temperature supported","name":"warmest","value":556}],"property":"color_temp","type":"numeric","unit":"mired","value_max":556,"value_min":153},{"access":7,"description":"Color temperature after cold power on of this light","label":"Color temp startup","name":"color_temp_startup","presets":[{"description":"Coolest temperature supported","name":"coolest","value":153},{"description":"Cool temperature (250 mireds / 4000 Kelvin)","name":"cool","value":250},{"description":"Neutral temperature (370 mireds / 2700 Kelvin)","name":"neutral","value":370},{"description":"Warm temperature (454 mireds / 2200 Kelvin)","name":"warm","value":454},{"description":"Warmest temperature supported","name":"warmest","value":556},{"description":"Restore previous color_temp on cold power on","name":"previous","value":65535}],"property":"color_temp_startup","type":"numeric","unit":"mired","value_max":556,"value_min":153},{"access":7,"description":"Color of this light in the CIE 1931 color space (x/y)","features":[{"access":7,"label":"X","name":"x","property":"x","type":"numeric"},{"access":7,"label":"Y","name":"y","property":"y","type":"numeric"}],"label":"Color (X/Y)","name":"color_xy","property":"color","type":"composite"},{"access":7,"description":"Color of this light expressed as hue/saturation","features":[{"access":7,"label":"Hue","name":"hue","property":"hue","type":"numeric"},{"access":7,"label":"Saturation","name":"saturation","property":"saturation","type":"numeric"}],"label":"Color (HS)","name":"color_hs","property":"color","type":"composite"}],"type":"light"},{"access":2,"description":"Triggers an effect on the light (e.g. make light blink for a few seconds)","label":"Effect","name":"effect","property":"effect","type":"enum","values":["blink","breathe","okay","channel_change","finish_effect","stop_effect","colorloop","stop_colorloop"]},{"access":7,"category":"config","description":"Controls the behavior when the device is powered on after power loss","label":"Power-on behavior","name":"power_on_behavior","property":"power_on_behavior","type":"enum","values":["off","on","toggle","previous"]},{"access":1,"category":"diagnostic","description":"Link quality (signal strength)","label":"Linkquality","name":"linkquality","property":"linkquality","type":"numeric","unit":"lqi","value_max":255,"value_min":0}],"model":"404000/404005/404012/404019","options":[{"access":2,"description":"Controls the transition time (in seconds) of on/off, brightness, color temperature (if applicable) and color (if applicable) changes. Defaults to 0 (no transition).","label":"Transition","name":"transition","property":"transition","type":"numeric","value_min":0},{"access":2,"description":"When enabled colors will be synced, e.g. if the light supports both color x/y and color temperature a conversion from color x/y to color temperature will be done when setting the x/y color (default true).","label":"Color sync","name":"color_sync","property":"color_sync","type":"binary","value_off":false,"value_on":true},{"access":2,"description":"State actions will also be published as 'action' when true (default false).","label":"State action","name":"state_action","property":"state_action","type":"binary","value_off":false,"value_on":true}],"supports_ota":false,"vendor":"Müller Licht"},"disabled":false,"endpoints":{"1":{"bindings":[{"cluster":"genLevelCtrl","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}}],"clusters":{"input":["genBasic","genIdentify","genGroups","genScenes","genOnOff","genLevelCtrl","lightingColorCtrl","haDiagnostic","touchlink","65244"],"output":["genTime","genOta"]},"configured_reportings":[{"attribute":"currentLevel","cluster":"genLevelCtrl","maximum_report_interval":3600,"minimum_report_interval":5,"reportable_change":1},{"attribute":"onOff","cluster":"genOnOff","maximum_report_interval":3600,"minimum_report_interval":0,"reportable_change":0}],"scenes":[]},"242":{"bindings":[],"clusters":{"input":[],"output":["greenPower"]},"configured_reportings":[],"scenes":[]}},"friendly_name":"Lampe02","ieee_address":"0x000d6ffffe21ee30","interview_completed":true,"interviewing":false,"manufacturer":"MLI","model_id":"ZBT-ExtendedColor","network_address":13042,"power_source":"Mains (single phase)","software_build_id":"2.2","supported":true,"type":"Router"},{"date_code":"20190308","definition":{"description":"TRADFRI motion sensor","exposes":[{"access":1,"description":"Indicates whether the device detected occupancy","label":"Occupancy","name":"occupancy","property":"occupancy","type":"binary","value_off":false,"value_on":true},{"access":1,"category":"diagnostic","description":"Indicates whether the device detected bright light (works only in night mode)","label":"Illuminance above threshold","name":"illuminance_above_threshold","property":"illuminance_above_threshold","type":"binary","value_off":false,"value_on":true},{"access":1,"category":"diagnostic","label":"Requested brightness level","name":"requested_brightness_level","property":"requested_brightness_level","type":"numeric","value_max":254,"value_min":76},{"access":1,"category":"diagnostic","label":"Requested brightness percent","name":"requested_brightness_percent","property":"requested_brightness_percent","type":"numeric","value_max":100,"value_min":30},{"access":2,"category":"config","description":"Initiate device identification. This device is asleep by default.You may need to wake it up first before sending the identify command.","label":"Identify","name":"identify","property":"identify","type":"enum","values":["identify"]},{"access":5,"category":"diagnostic","description":"Remaining battery in %","label":"Battery","name":"battery","property":"battery","type":"numeric","unit":"%","value_max":100,"value_min":0},{"access":1,"category":"diagnostic","description":"Link quality (signal strength)","label":"Linkquality","name":"linkquality","property":"linkquality","type":"numeric","unit":"lqi","value_max":255,"value_min":0}],"model":"E1525/E1745","options":[{"access":2,"description":"Sets the duration of the identification procedure in seconds (i.e., how long the device would flash).The value ranges from 1 to 30 seconds (default: 3).","label":"Identify timeout","name":"identify_timeout","property":"identify_timeout","type":"numeric","value_max":30,"value_min":1},{"access":2,"description":"Time in seconds after which occupancy is cleared after detecting it (default 90 seconds).","label":"Occupancy timeout","name":"occupancy_timeout","property":"occupancy_timeout","type":"numeric","value_min":0},{"access":2,"description":"Set to false to also send messages when illuminance is above threshold in night mode (default true).","label":"Illuminance below threshold check","name":"illuminance_below_threshold_check","property":"illuminance_below_threshold_check","type":"binary","value_off":false,"value_on":true}],"supports_ota":true,"vendor":"IKEA"},"disabled":false,"endpoints":{"1":{"bindings":[{"cluster":"genPollCtrl","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genPowerCfg","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}}],"clusters":{"input":["genBasic","genPowerCfg","genIdentify","genAlarms","genPollCtrl","touchlink","manuSpecificIkeaUnknown"],"output":["genIdentify","genGroups","genOnOff","genLevelCtrl","genOta","touchlink"]},"configured_reportings":[{"attribute":"batteryPercentageRemaining","cluster":"genPowerCfg","maximum_report_interval":65000,"minimum_report_interval":3600,"reportable_change":10}],"scenes":[]}},"friendly_name":"TRADFRI_Motion","ieee_address":"0x680ae2fffea40441","interview_completed":true,"interviewing":false,"manufacturer":"IKEA of Sweden","model_id":"TRADFRI motion sensor","network_address":55544,"power_source":"Battery","software_build_id":"2.0.022","supported":true,"type":"EndDevice"},{"date_code":"20181025-69","definition":{"description":"Tint LED bulb GU10/E14/E27 350/470/806 lumen, dimmable, color, opal white","exposes":[{"features":[{"access":7,"description":"On/off state of this light","label":"State","name":"state","property":"state","type":"binary","value_off":"OFF","value_on":"ON","value_toggle":"TOGGLE"},{"access":7,"description":"Brightness of this light","label":"Brightness","name":"brightness","property":"brightness","type":"numeric","value_max":254,"value_min":0},{"access":7,"description":"Color temperature of this light","label":"Color temp","name":"color_temp","presets":[{"description":"Coolest temperature supported","name":"coolest","value":153},{"description":"Cool temperature (250 mireds / 4000 Kelvin)","name":"cool","value":250},{"description":"Neutral temperature (370 mireds / 2700 Kelvin)","name":"neutral","value":370},{"description":"Warm temperature (454 mireds / 2200 Kelvin)","name":"warm","value":454},{"description":"Warmest temperature supported","name":"warmest","value":556}],"property":"color_temp","type":"numeric","unit":"mired","value_max":556,"value_min":153},{"access":7,"description":"Color temperature after cold power on of this light","label":"Color temp startup","name":"color_temp_startup","presets":[{"description":"Coolest temperature supported","name":"coolest","value":153},{"description":"Cool temperature (250 mireds / 4000 Kelvin)","name":"cool","value":250},{"description":"Neutral temperature (370 mireds / 2700 Kelvin)","name":"neutral","value":370},{"description":"Warm temperature (454 mireds / 2200 Kelvin)","name":"warm","value":454},{"description":"Warmest temperature supported","name":"warmest","value":556},{"description":"Restore previous color_temp on cold power on","name":"previous","value":65535}],"property":"color_temp_startup","type":"numeric","unit":"mired","value_max":556,"value_min":153},{"access":7,"description":"Color of this light in the CIE 1931 color space (x/y)","features":[{"access":7,"label":"X","name":"x","property":"x","type":"numeric"},{"access":7,"label":"Y","name":"y","property":"y","type":"numeric"}],"label":"Color (X/Y)","name":"color_xy","property":"color","type":"composite"},{"access":7,"description":"Color of this light expressed as hue/saturation","features":[{"access":7,"label":"Hue","name":"hue","property":"hue","type":"numeric"},{"access":7,"label":"Saturation","name":"saturation","property":"saturation","type":"numeric"}],"label":"Color (HS)","name":"color_hs","property":"color","type":"composite"}],"type":"light"},{"access":2,"description":"Triggers an effect on the light (e.g. make light blink for a few seconds)","label":"Effect","name":"effect","property":"effect","type":"enum","values":["blink","breathe","okay","channel_change","finish_effect","stop_effect","colorloop","stop_colorloop"]},{"access":7,"category":"config","description":"Controls the behavior when the device is powered on after power loss","label":"Power-on behavior","name":"power_on_behavior","property":"power_on_behavior","type":"enum","values":["off","on","toggle","previous"]},{"access":1,"category":"diagnostic","description":"Link quality (signal strength)","label":"Linkquality","name":"linkquality","property":"linkquality","type":"numeric","unit":"lqi","value_max":255,"value_min":0}],"model":"404000/404005/404012/404019","options":[{"access":2,"description":"Controls the transition time (in seconds) of on/off, brightness, color temperature (if applicable) and color (if applicable) changes. Defaults to 0 (no transition).","label":"Transition","name":"transition","property":"transition","type":"numeric","value_min":0},{"access":2,"description":"When enabled colors will be synced, e.g. if the light supports both color x/y and color temperature a conversion from color x/y to color temperature will be done when setting the x/y color (default true).","label":"Color sync","name":"color_sync","property":"color_sync","type":"binary","value_off":false,"value_on":true},{"access":2,"description":"State actions will also be published as 'action' when true (default false).","label":"State action","name":"state_action","property":"state_action","type":"binary","value_off":false,"value_on":true}],"supports_ota":false,"vendor":"Müller Licht"},"disabled":false,"endpoints":{"1":{"bindings":[{"cluster":"genLevelCtrl","target":{"endpoint":1,"ieee_address":"0x680ae2fffea61e8f","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x680ae2fffea61e8f","type":"endpoint"}},{"cluster":"genLevelCtrl","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}}],"clusters":{"input":["genBasic","genIdentify","genGroups","genScenes","genOnOff","genLevelCtrl","lightingColorCtrl","haDiagnostic","touchlink","65244"],"output":["genTime","genOta"]},"configured_reportings":[{"attribute":"currentLevel","cluster":"genLevelCtrl","maximum_report_interval":3600,"minimum_report_interval":5,"reportable_change":1},{"attribute":"onOff","cluster":"genOnOff","maximum_report_interval":3600,"minimum_report_interval":0,"reportable_change":0}],"scenes":[]},"242":{"bindings":[],"clusters":{"input":[],"output":["greenPower"]},"configured_reportings":[],"scenes":[]}},"friendly_name":"Lampe01","ieee_address":"0xd0cf5efffe27933a","interview_completed":true,"interviewing":false,"manufacturer":"MLI","model_id":"ZBT-ExtendedColor","network_address":9102,"power_source":"Mains (single phase)","software_build_id":"2.2","supported":true,"type":"Router"},{"date_code":"20190401","definition":{"description":"TRADFRI remote control","exposes":[{"access":2,"category":"config","description":"Initiate device identification. This device is asleep by default.You may need to wake it up first before sending the identify command.","label":"Identify","name":"identify","property":"identify","type":"enum","values":["identify"]},{"access":5,"category":"diagnostic","description":"Remaining battery in %","label":"Battery","name":"battery","property":"battery","type":"numeric","unit":"%","value_max":100,"value_min":0},{"access":1,"category":"diagnostic","description":"Triggered action (e.g. a button click)","label":"Action","name":"action","property":"action","type":"enum","values":["toggle","brightness_up_click","brightness_down_click","brightness_up_hold","brightness_up_release","brightness_down_hold","brightness_down_release","toggle_hold","arrow_left_click","arrow_left_hold","arrow_left_release","arrow_right_click","arrow_right_hold","arrow_right_release"]},{"access":1,"category":"diagnostic","description":"Link quality (signal strength)","label":"Linkquality","name":"linkquality","property":"linkquality","type":"numeric","unit":"lqi","value_max":255,"value_min":0}],"model":"E1524/E1810","options":[{"access":2,"description":"Sets the duration of the identification procedure in seconds (i.e., how long the device would flash).The value ranges from 1 to 30 seconds (default: 3).","label":"Identify timeout","name":"identify_timeout","property":"identify_timeout","type":"numeric","value_max":30,"value_min":1},{"access":2,"description":"Set to false to disable the legacy integration (highly recommended), will change structure of the published payload (default true).","label":"Legacy","name":"legacy","property":"legacy","type":"binary","value_off":false,"value_on":true}],"supports_ota":true,"vendor":"IKEA"},"disabled":false,"endpoints":{"1":{"bindings":[{"cluster":"genPollCtrl","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genScenes","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genPowerCfg","target":{"endpoint":1,"ieee_address":"0x8c65a3fffe548700","type":"endpoint"}},{"cluster":"genLevelCtrl","target":{"endpoint":1,"ieee_address":"0xd0cf5efffe27933a","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0xd0cf5efffe27933a","type":"endpoint"}},{"cluster":"genLevelCtrl","target":{"endpoint":1,"ieee_address":"0x000d6ffffe21ee30","type":"endpoint"}},{"cluster":"genOnOff","target":{"endpoint":1,"ieee_address":"0x000d6ffffe21ee30","type":"endpoint"}}],"clusters":{"input":["genBasic","genPowerCfg","genIdentify","genPollCtrl","touchlink","manuSpecificAmazonWWAH","manuSpecificIkeaUnknown"],"output":["genIdentify","genGroups","genScenes","genOnOff","genLevelCtrl","genOta","touchlink"]},"configured_reportings":[{"attribute":"batteryPercentageRemaining","cluster":"genPowerCfg","maximum_report_interval":65000,"minimum_report_interval":3600,"reportable_change":10}],"scenes":[]}},"friendly_name":"Tradfri01","ieee_address":"0x680ae2fffea61e8f","interview_completed":true,"interviewing":false,"manufacturer":"IKEA of Sweden","model_id":"TRADFRI remote control","network_address":29511,"power_source":"Battery","software_build_id":"24.4.5","supported":true,"type":"EndDevice"},{"disabled":false,"endpoints":{"1":{"bindings":[],"clusters":{"input":["genBasic","genIdentify","genOnOff","genTime","genOta","26","lightingColorCtrl"],"output":["genBasic","genIdentify","genGroups","genScenes","genOnOff","genLevelCtrl","genPollCtrl","lightingColorCtrl","msIlluminanceMeasurement","msTemperatureMeasurement","msRelativeHumidity","msOccupancySensing","ssIasZone","haMeterIdentification","haApplianceStatistics","haElectricalMeasurement","seMetering","touchlink","manuSpecificLegrandDevices","manuSpecificSamsungAccelerometer"]},"configured_reportings":[],"scenes":[]},"242":{"bindings":[],"clusters":{"input":[],"output":["greenPower"]},"configured_reportings":[],"scenes":[]}},"friendly_name":"Coordinator","ieee_address":"0x8c65a3fffe548700","interview_completed":true,"interviewing":false,"network_address":0,"supported":true,"type":"Coordinator"},{"date_code":"20190401","definition":{"description":"TRADFRI remote control","exposes":[{"access":2,"category":"config","description":"Initiate device identification. This device is asleep by default.You may need to wake it up first before sending the identify command.","label":"Identify","name":"identify","property":"identify","type":"enum","values":["identify"]},{"access":5,"category":"diagnostic","description":"Remaining battery in %","label":"Battery","name":"battery","property":"battery","type":"numeric","unit":"%","value_max":100,"value_min":0},{"access":1,"category":"diagnostic","description":"Triggered action (e.g. a button click)","label":"Action","name":"action","property":"action","type":"enum","values":["toggle","brightness_up_click","brightness_down_click","brightness_up_hold","brightness_up_release","brightness_down_hold","brightness_down_release","toggle_hold","arrow_left_click","arrow_left_hold","arrow_left_release","arrow_right_click","arrow_right_hold","arrow_right_release"]},{"access":1,"category":"diagnostic","description":"Link quality (signal strength)","label":"Linkqualit

This ZigBee2mqtt topic is retain type: every time you subscribe it you should receive a response. Unless for some reason mqtt broker doesn't support it.

Ah, i was not fast enough for answer. I get it. After reading the z2m docu again, i checked that it must be the retrain flag.
I noticed that the flag at my build in broker was off. Now i can test zigbee.

I think qmqtt library can check mqtt broker capabilities. I'll try to test it and warn the user in logs if the retain messages are not supported.


Update init_ddp_connection logging info message and update function name

Proposed refactoring or deprecation
init_ddp_connection is not only used by DDP, but also used by FSDP and other plugins inherit DDP. The log info and rank_zero_info saying ""initializing ddp" or "DDP processes", which could be confusing for user
Motivation
Debuggability and readability
Pitch
Update info message to be generic message
Update function name to initialize_distributed_connction as it's used by fsdp and sdp
Additional context

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, finetuning and solving problems with deep learning

Bolts: Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Lightning Transformers: Flexible interface for high performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

Good idea, let's do this


key backup: Display the restore button only if required

@manuroe commented on Wed Feb 06 2019
The restore button is currently always displayed even after a successful recover. This is a disturbing UX.
To fix it we need to:

[x] update the spec so that matrix clients can detect there is a difference between backed up keys and locally stored keys
[x] implement this spec change on server
[ ] implement it on clients

@manuroe commented on Wed Feb 06 2019
Spec proposal: https://github.com/uhoreg/matrix-doc/pull/2.

I Think we can close this due to #2980


Potentially found the solution to your Registrant Key issues, it's a Javascript problem!

Hi there, so I was playing with the API and I noticed something that, judging by comments you left in the repo, you have also noticed.
When using the test routes that display the JSON response, any call I tried to make using the registrant key given from the response to a registrant create call would fail with a 404 as if that registrant did not exist, which was very confusing since the API had just given me a response with the apparently incorrect key!
It was then that I noticed that, like the registrant keys in your comments, my registrant keys were also ending in a bunch of zeros.
I then looked at the raw response data and to my surprise, the registrant key that was in the raw response was NOT the same key that my browser was displaying!
It seems that the GoToWebinar API actually breaks a JSON standard by encoding this Registrant Key as an INT; the number is simply too big! Firefox was interperating the INT and simply rounding up the last few digits for whatever reason.
In further testing, it seems like the package works just fine in PHP land, the correct key is present in the response object. All that was broken was the way that Firefox chose to display the key, which led me to input the incorrect key when making subsequent calls.
This may be old news to you but I thought I would mention this here for anyone else that has had the same problem, as I suspect you may have judging by the comments left in the registrant routes file of the repo.
I would recommend perhaps mentioning this wierd little bug in the README as I'm sure I won't be the last person to spend way too long bashing their head against this issue!

@sethjamesofficial  thanks so much for this! Very interesting. I will take a look at it.


publish 1.2.1

https://github.com/ssbc/ssb-config/pull/4#issuecomment-73585994
i dont think this got published yet

published

this can be closed?


Update package version

@label-u/components 1.0.0-alpha.20 (2023-02-20)
Bug Fixes

components: dont inject duplicate style into header (616a170)
components: tagTool undefined to object (51c03c6)

:tada: This PR is included in version 1.0.0-alpha.20 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.21 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.21 :tada:
The release is available on npm package (@alpha dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.1 :tada:
The release is available on npm package (@alpha dist-tag)
Your semantic-release bot :package::rocket:


debian 9 + empty polybar

Hello. debian 9. Collected on the manual, but there is an empty panel. Config is standard. I thought that the standard panel is empty, but on the arch, it is installed with all the elements. Please tell me what I'm doing wrong.
screenshot - https://ibb.co/goQFAm

@dripfeeder Have you fixed this? What was the solution? I have the same issue now.

My problem was solved after commenting this line:
; font-0 = fixed:pixelsize=10;1

And enabling siji font:
sudo mv /etc/fonts/conf.d/70-no-bitmaps.conf /etc/fonts/conf.d/70-no-bitmaps.conf.old
fc-cache -fv

Stolen solution from reddit


[html] HTML colors angle brackets wrongly in dark theme

Angel brackets should be blue not orange in this screenshot.

fix delivered

I guess you should assign it to me for verification :hamster:


App's on_stop method gets called twice

If one closes the running app using the OS native cross button, everything goes as expected. However, if one decides to stop the application using the stop method, the on_stop will be triggered twice. Here's a quick sample to demonstrate the issue:
from kivy.app import App
from kivy.uix.button import Button

class TestApp(App):
    def on_stop(self):
        print('Stopping!')

    def build(self):
        b = Button(on_release = app.stop)
        return b

app = TestApp()
app.run()

Issue seen on Kivy 1.9.2-dev0 for Linux

I think this is actually to do with how touch events work. Sometimes the user will release the button "between frames" and the touch will get dispatched twice. The standard widgets routinely check if they've handled a given touch object already by storing it in a Property and comparing.

@LogicalDash I highly doubt that's the case. Try running the script a few times, the issue persists. I suspect the issue to be in the following.
These are the last lines of code of the app.py run method:
        runTouchApp()
        self.stop()

These are the last lines of code of the base.py runTouchApp method:
try:
        if EventLoop.window is None:
            _run_mainloop()
        else:
            EventLoop.window.mainloop()
    finally:
stopTouchApp()

And finally, those are the last lines of code of the app.py stop method:
    def stop(self, *largs):
        '''Stop the application.
        If you use this method, the whole application will stop by issuing
        a call to :func:`~kivy.base.stopTouchApp`.
        '''
        self.dispatch('on_stop')
        stopTouchApp()

        # Clear the window children
        for child in self._app_window.children:
self._app_window.remove_widget(child)

It seems odd to me that the stopTouchApp gets called twice. Perhaps, instead of explicitly calling stop at the end of runTouchApp, we could dispatch the on_stop method and be done with it?

Duplicate of https://github.com/kivy/kivy/issues/2397.


Is this project dead?

@dunyakirkali It has not been "maintained" for a while but the current version should work fine.
Are you experiencing any issue?

Not really.
I wanted to give Router a try on Swift 4 and saw that there was a branch for it which was last updated 10 months ago so I was curious.


stream error: stream ID 1; INTERNAL_ERROR; received from peer"

Hi i have done the changes in config.toml for NSE url but its still taking the old one and throwing error.| time=2023-10-04T09:20:50.639Z level=ERROR source=/home/runner/work/mii-lama/mii-lama/cmd/main.go:51 msg="failed to init nse manager" component=mii-lama error="failed to login to NSE API: failed to send HTTP request: Post "https://lama.nseindia.com/api/V1/auth/login\": stream error: stream ID 1; INTERNAL_ERROR; received from peer"
mii-lama exited with code 1
below url is passed in config.toml
[lama.nse]
url = "https://lama.nseindia.com" # Endpoint for NSE LAMA API Gateway

Dupe: https://github.com/zerodha/mii-lama/issues/13#issuecomment-1704640811


Build failure on Mageia Linux x86-64 version 6 with llvm-3.6.2-1.mga6.src.rpm

Hi all,
I am getting this build failure with commit 718c37398270b1a9fafa85a7ba2af286f4d53a5f on branch master on Mageia 6 x86-64 with llvm-3.6.2-1.mga6.src.rpm:
shlomif@telaviv1:~/Download/unpack/prog/pony/ponyc$ make config=release
trace.c
cycle.c
gc.c
actormap.c
delta.c
objectmap.c
actor.c
messageq.c
stack.c
fun.c
hash.c
list.c
event.c
asio.c
epoll.c
threads.c
options.c
start.c
scheduler.c
cpu.c
mpmcq.c
pool.c
heap.c
pagemap.c
alloc.c
stdfd.c
time.c
ssl.c
directory.c
paths.c
lsda.c
stat.c
posix_except.c
socket.c
Linking libponyrt
postfix.c
operator.c
literal.c
call.c
reference.c
ffi.c
control.c
match.c
array.c
gentype.c
genlib.c
genmatch.c
genoperator.c
genexpr.c
gendesc.c
gencontrol.c
genfun.c
host.cc
gencall.c
gentrace.c
genident.c
genreference.c
genprim.c
genheader.c
genobj.c
genopt.cc
genexe.c
genbox.c
codegen.c
genname.c
dwarf.c
symbols.cc
package.c
program.c
use.c
platformfuns.c
paths.c
assemble.c
lookup.c
alias.c
matchtype.c
reify.c
viewpoint.c
subtype.c
cap.c
traits.c
syntax.c
sugar.c
expr.c
pass.c
scope.c
docgen.c
names.c
finalisers.c
flatten.c
import.c
printbuf.c
bnfprint.c
lexer.c
parserapi.c
ast.c
builder.c
parser.c
id.c
token.c
stringtab.c
symtab.c
frame.c
error.c
source.c
paint.c
reach.c
Linking libponyc
trace.c
cycle.c
gc.c
actormap.c
delta.c
objectmap.c
actor.c
messageq.c
stack.c
fun.c
hash.c
list.c
event.c
asio.c
epoll.c
threads.c
options.c
start.c
scheduler.c
cpu.c
mpmcq.c
pool.c
heap.c
pagemap.c
alloc.c
stdfd.c
time.c
ssl.c
directory.c
paths.c
lsda.c
stat.c
posix_except.c
socket.c
Linking libponyrt-pic
gtest_main.cc
gtest-all.cc
Linking libgtest
main.c
Linking ponyc
build/release/libponyc.a(genopt.o):(.rodata._ZTI11HeapToStack[_ZTI11HeapToStack]+0x10): undefined reference to `typeinfo for llvm::FunctionPass'
collect2: error: ld returned 1 exit status
Makefile:383: recipe for target 'build/release/ponyc' failed
make: *** [build/release/ponyc] Error 1
shlomif@telaviv1:~/Download/unpack/prog/pony/ponyc$

Hi,
it appears that you have compiled LLVM without RTTI. Can you confirm that LLVM is compiled with LLVM_ENABLE_RTTI ?

@shlomif - as a hint, if you're using the autotools-based system, you should be able to do this with REQUIRES_RTTI=1 in your env when the call to make is executed.  For example, env REQUIRES_RTTI=1 make -j 16.

Thanks! I'll try.

OK, I was able to build ponyc fine using an  RTTI-enabled version of LLVM. Can you fix ponyc's build system to detect and throw an error about it?

Cool. I'll open another issue for potentially 'detecting' the build flags used for LLVM.


Fast forward rules

The current forward rule implementation is hilariously inefficient. We're currently working on something much better.

Exciting!


[Linux] Pip module not found issue added to readme

@asc249 can you mention the solution in the issue?
Also In Installing python3 step. Add a check for python3 version command as well as pip version check. There we can mention installing pip command


handling deployvm API

Signed-off-by: Abhishek Kumar<EMAIL_ADDRESS>Description

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Screenshots (if appropriate):
How Has This Been Tested?

@DaanHoogland please see

Thanks @shwstppr, looking good. Let me know once its ready so we incorporate it into the big PR


[WIP]Adjust threadstest to support Nonstop

The nonstop cooperative threading model requires some adjustments to the new threadstest tests:

The use of OSSL_sleep in the rwtorture and rcutorture tests
encounters an error when specifying times of 1000ms or larger, limit
that on nonstop to 999ms instead

The use of rwlocks in the tests is bad.  Because Nonstop appears to
not consider pthread_rwlock_rdlock to be a yield point, deadlocks can
occur when a reader thread acquires the cpu, as it will never yield.
For Nonstop use write locks in the tests exclusively to create thread
rescheduling events

For RCU, the read side by design never has to block, creating a
simmilar problem to (2).  Insert an explicit yeild for Nonstop
to force thread scheduling

Checklist

[x] tests are added or updated

Can you remove the change involving reducing the time size when this is finalized? #23926 fixes this.

Removed

I'm a little (very) confused. I tried merging your changes with 3.3.0-alpha1 and have all kinds of undefined externals. I might have to reset the repository and try again.
More usefully, on the other hand, with #23926 and master (at 9abcf11696 actually passes). #23926 also seems to fix #23927 and #23928. I'm now wondering whether the usleep(1000000) with that invalid value has somehow messed up the thread library - considering how long I have worked in PUT, this surprises the tar out of me. Although, calling sleep() followed by usleep() forces invoking the scheduler twice, which can have significant side-effects. @nhorman What do you think?

I'm on my phone so this is just a guess, but could the addition of write.lock in the read side make the usleep error moot (i.e we still need to fix it, but the regular schedule exchanges make the failed sleep irrelevant

I'm on my phone so this is just a guess, but could the addition of write.lock in the read side make the usleep error moot (i.e we still need to fix it, but the regular schedule exchanges make the failed sleep irrelevant

I'm not sure. With the OSSL_sleep fix I put in, the code seems to be much happier (all tests pass in master), even without your fix. #23926 should be applied to all 3.x series releases at the earliest opportunity. I am suspecting that usleep(1000000) somehow caused problem in the NonStop PUT library - it is documented as "don't do this" but no reason given.

@nhorman FYI: Your fix did work also, including the other two failed tests. At this point, it is a toss-up as to whether to include this or use master as is with the OSSL_sleep fix.

I'm in favor of minimal changes, so if you're OSSL_sleep fix fixes the issues you're encountering, I'll just close this.  Please confirm and I will do so.


Get-AzMetric and Get-AzMetricDefinition needs more detail in regards to output

The Get-AzMetricDefinition returns a variety of outputs, depending on the resource type.  For a VM, the list is:

Percentage CPU
Network In
Network Out
Disk Read Bytes
Disk Write Bytes
Disk Read Operations/Sec
Disk Write Operations/Sec
CPU Credits Remaining
CPU Credits Consumed
Per Disk Read Bytes/sec
Per Disk Write Bytes/sec
Per Disk Read Operations/Sec
Per Disk Write Operations/Sec
Per Disk QD
OS Per Disk Read Bytes/sec
OS Per Disk Write Bytes/sec
OS Per Disk Read Operations/Sec
OS Per Disk Write Operations/Sec
OS Per Disk QD
Data Disk Read Bytes/sec
Data Disk Write Bytes/sec
Data Disk Read Operations/Sec
Data Disk Write Operations/Sec
Data Disk Queue Depth
OS Disk Read Bytes/sec
OS Disk Write Bytes/sec
OS Disk Read Operations/Sec
OS Disk Write Operations/Sec
OS Disk Queue Depth
Inbound Flows
Outbound Flows
Inbound Flows Maximum Creation Rate
Outbound Flows Maximum Creation Rate
Premium Data Disk Cache Read Hit
Premium Data Disk Cache Read Miss
Premium OS Disk Cache Read Hit
Premium OS Disk Cache Read Miss
Network In Total
Network Out Total

Unfortunately, the documentation does not provide any references (links) to documentation regarding the specific metrics for each resource type. The reader is left to guess what the differences are between some of the less obvious metrics, such as 'Disk Read Bytes/Sec' vs. 'Per Disk Read Bytes/Sec' vs. 'Data Disk Read Bytes/Sec' and so on.  A link to a page where all of the various metrics/counters are described would be most helpful.

Reference documentation is handled by the engineering team. Please file an issue with engineering.


Slurm scripts for downstream tasks

Hi, thank you for your awesome work!
Could you please release the exact script to reproduce downstream results? i.e. Could you please fill out this part in run_genomics_benchmark.sh and run_nucleotide_transformer.sh, for both Caduceus-Ph and Caduceus-Ps?
Expected args:

CONFIG_PATH
PRETRAINED_PATH
DISPLAY_NAME
MODEL
MODEL_NAME
CONJOIN_TRAIN_DECODER
CONJOIN_TEST
TASK
LR
BATCH_SIZE
RC_AUG

Thank you very much in advance!

The exact values of these parameters are set in the wrapper_ scripts for each of the downstream tasks:

https://github.com/kuleshov-group/caduceus/blob/main/slurm_scripts/wrapper_run_genomics.sh
https://github.com/kuleshov-group/caduceus/blob/main/slurm_scripts/wrapper_run_nucleotide_transformer.sh

The appendix in the paper has additional details about hyperparameters such as BATCH_SIZE and LR for each subtask within a benchmark

Thank you so much!!


filter_rank_genes_groups condition should be OR instead of AND

here is the code for marker filter
I think the 3 condition need to be OR instead of AND
gene_names = gene_names[
(fraction_in_cluster_matrix > min_in_group_fraction) &
(fraction_out_cluster_matrix < max_out_group_fraction) &
(fold_change_matrix > min_fold_change)
]

...

...

Versions:

...

It makes sense to use AND logic, because the function keeps genes that satisfy all three conditions.

Fraction of cells inside the cluster expressing the gene must be greater than min_in_group_fraction
Fractions of cells outside the cluster expressing the gene must be less than max_out_group_fraction
Fold change must be greater than min_fold_change

But there are remaining issues (calculation of fold change and using the absolute value of the fold change) in this function that needs to be updated #863

Thanks for clarification. I had thought it is for filtering out genes.
On Mon, May 18, 2020 at 2:21 AM Rachel Ng<EMAIL_ADDRESS>wrote:

It makes sense to use AND logic, because the function keeps genes that
satisfy all three conditions.

Fraction of cells inside the cluster expressing the gene must be
greater than min_in_group_fraction
Fractions of cells outside the cluster expressing the gene must be
less than max_out_group_fraction
Fold change must be greater than min_fold_change

But there are remaining issues (calculation of fold change and using the
absolute value of the fold change) in this function that needs to be
updated #863 https://github.com/theislab/scanpy/issues/863
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/theislab/scanpy/issues/1213#issuecomment-629970781,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADG2PVZVFSYU3ST4ESJFS33RSDHVXANCNFSM4NAA5V2A
.

After https://github.com/theislab/scanpy/pull/1156 I will update the function.

After #1156 I will update the function.

Wait, does it use OR logic now?? Doesn't AND logic make more sense???


Kubernetes is now in Docker for Mac and Windows Stable

File: docker-for-mac/kubernetes.md
Please update error message that says it is only in edge

I dont have a mac, and if documentation is incorrect, I could just be changing it.  I did adjust the Windows Getting started and it appears the documentation is updated in most places but the Getting started page.  If I had a mac and could install the release to prove it ... I'd update it.  I would suggest that @ManoMarks you actually edit and submit the code in a PR.

I was hoping that a Docker Tech Writer would pick this up but didn't. My current job doesn't let me contribute to projects without approval.

awkward ... so your personal GitHub account on personal time is not allowed in the context of contribution?  Wow - if its google doing this as an underpinned portion of employment ... I am shocked.  I did scope some other spots but the automatons and parameters should be injecting the correct versions on a "rebuild".

Honestly I could and it probably wouldn't be an issue but technically I should get approval and it's a lot of hassle. I filed this issue when I worked at Docker and talked to the techwriting team at the time, that was the right way to go and they haven't fixed it yet.

Closing issue as it refers to content that has since been removed/reorganised/archived


full_access_as_app missing

Hi David,
I've used the Merge-Mailbox script without issues when using API Delegated Permissions EWS.AccessAsUser.All.
This is fine for processing individual mailboxes, however i need to loop and process several mailboxes, this means it will auth on each mailbox, i understand that from another issue post you replied on means I'll have to switch to app auth.
When i've tried to do this it appears that i can't find 'full_access_as_app' in the list of the available API's, and when i edit the manifest file, it doesn't recognise it as an app (just displays the ID reference).
Does this mean the full_access_as_app has been deprecated and this script can no longer be ran using app auth?
This is the error i get when running the following
.\Merge-MailboxFolder.ps1 -SourceMailbox<EMAIL_ADDRESS>-SourceArchive -ProcessSubfolders -CreateTargetFolder -Office365 -OAuth -OAuthTenantId $tenantId -OAuthClientId $clientId -OAuthSecretKey $secretKey -Impersonate
26/01/2024 10:56:57 [ErrorReported] ERROR (ThrottledFolderBind): Exception calling "Bind" with "3" argument(s): "The request failed. The remote server returned an error: (403) Forbidden."
Failed to open source message store<EMAIL_ADDRESS>Please check that you have impersonation permissions
(I've given my account ApplicationImpersonation via an Exchange Online role, but not 100% sure if that error is just linked to full_access_as_app not having API perms?)
Any help would be appreciated, thank you.

I just followed the guide (https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/how-to-authenticate-an-ews-application-by-using-oauth#configure-for-app-only-authentication) to update the manifest and it worked fine for me.  I'm not aware of any deprecation, that shouldn't happen until EWS itself is deprecated in 2026.

Thanks for the quick response David.
That's strange, I've just tried again and it worked straight away!...
Either way, I've just ran the script with app-auth on a couple of test mailboxes it worked flawlessly.. Thank you so much for a great script 👍


Removing allocations for repeated use of object_and_preserve

When using CheapThreads.jl, I get allocations when new References are created in object_and_preserve. This makes sense, but I'd really like to do away with this is possible as my aim is to use CheapThreads.jl within DifferentialEquations.jl and I don't want the allocs to build up. In my case the arg that is being referenced is a struct, but you also get allocations for other !isbits types, e.g.:
julia> f1(arg) = @time StrideArraysCore.object_and_preserve(arg)
julia> f1(1)
  0.000000 seconds
(1, nothing)

julia> f1("string")
  0.000000 seconds (1 allocation: 16 bytes)
(StrideArraysCore.Reference{String}("string"), StrideArraysCore.Reference{String}("string"))

julia> struct TestStruct
       a::Vector{Int64}
       end

julia> ts = TestStruct([1])
TestStruct([1])

julia> f1(ts)
  0.000000 seconds (1 allocation: 16 bytes)
(StrideArraysCore.Reference{TestStruct}(TestStruct([1])), StrideArraysCore.Reference{TestStruct}(TestStruct([1])))

I can see two possible solutions:
1. Allow users to create their own References
This just needs object_and_preserve to be extended like
@inline object_and_preserve(r::Reference) = (r, r)

When added, it works like so:
julia> ref = StrideArraysCore.Reference("string")
StrideArraysCore.Reference{String}("string")

julia> f1(ref)
  0.000000 seconds
(StrideArraysCore.Reference{String}("string"), StrideArraysCore.Reference{String}("string"))

julia> ref = StrideArraysCore.Reference(ts)
StrideArraysCore.Reference{TestStruct}(TestStruct([1]))

julia> f1(ref)
  0.000000 seconds
(StrideArraysCore.Reference{TestStruct}(TestStruct([1])), StrideArraysCore.Reference{TestStruct}(TestStruct([1])))

This seems to work fine with batch from CheapThreads.jl.
2. Encourage users to extend object_and_preserve, store! and load for their structs
For example:
@inline function ThreadingUtilities.load(p::Ptr{UInt}, ::Type{TestMutableStruct}, i)
    i, ref = ThreadingUtilities.load(p, Ptr{TestMutableStruct}, i)
    i, Base.unsafe_pointer_to_objref(ref)::TestMutableStruct
end
@inline function ThreadingUtilities.store!(p::Ptr{UInt}, r::TestMutableStruct, i)
    ThreadingUtilities.store!(p + i, reinterpret(UInt, pointer_from_objref(r)))
    i + sizeof(UInt)
end
@inline StrideArraysCore.object_and_preserve(r::TestMutableStruct) = (r, r)

This seems to be fine with CheapThreads, but I don't know if there are implications of doing this.
julia> arg_test(args, start, stop) = nothing

julia> mutable struct TestMutableStruct
       a::String
       end

julia> tms = TestMutableStruct("a")
TestMutableStruct("a")

julia>  u = zeros(100)
...
julia> @time batch(arg_test, (10, 8), u, tms)
  0.000003 seconds

The above extensions only work for mutable structs because of pointer_from_objref.
I'm interested to know what people's thoughts are on the best way forward for this!

Why not 1 & 2? I.e., let users decide what works best for their use case.

Base.unsafe_pointer_to_objref(ref)::TestMutableStruct

This or something like it seems like a good solution.
julia> x = Ref(("hi", "world", 9))
Base.RefValue{Tuple{String, String, Int64}}(("hi", "world", 9))

julia> p = Base.unsafe_convert(Ptr{typeof(x[])}, x)
Ptr{Tuple{String, String, Int64}} @0x00007f2a74688db0

julia> tsptr(x::Ptr{T}) where {T} = (Base.unsafe_pointer_to_objref(x)::Base.RefValue{T})[]
tsptr (generic function with 1 method)

julia> tsptr(p)
("hi", "world", 9)

julia> @benchmark tsptr($p)
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     1.765 ns (0.00% GC)
  median time:      1.772 ns (0.00% GC)
  mean time:        1.798 ns (0.00% GC)
  maximum time:     12.976 ns (0.00% GC)
  --------------
  samples:          10000
  evals/sample:     1000

I don't know if there is a way to avoid the type instability all together by using something other than Any:
unsafe_pointer_to_objref(x::Ptr) = ccall(:jl_value_ptr, Any, (Ptr{Cvoid},), x)

e.g.
tsptr2(x::Ptr{T}) where {T} = ccall(:jl_value_ptr, Ref{Base.RefValue{T}}, (Ptr{Cvoid},), x)[]
tsptr2(p)

yields
julia> tsptr2(p)
("hi", "world", 9)

julia> @code_warntype tsptr2(p)
MethodInstance for tsptr2(::Ptr{Tuple{String, String, Int64}})
  from tsptr2(x::Ptr{T}) where T in Main at REPL[13]:1
Static Parameters
  T = Tuple{String, String, Int64}
Arguments
  #self#::Core.Const(tsptr2)
  x::Ptr{Tuple{String, String, Int64}}
Body::Tuple{String, String, Int64}
1 ─ %1 = Core.apply_type(Main.Ptr, Main.Cvoid)::Core.Const(Ptr{Nothing})
│   %2 = Base.cconvert(%1, x)::Ptr{Tuple{String, String, Int64}}
│   %3 = Core.apply_type(Main.Ptr, Main.Cvoid)::Core.Const(Ptr{Nothing})
│   %4 = Base.unsafe_convert(%3, %2)::Ptr{Nothing}
│   %5 = $(Expr(:foreigncall, :(:jl_value_ptr), Ref{Base.RefValue{T}}, svec(Ptr{Nothing}), 0, :(:ccall), :(%4), :(%2)))::Base.RefValue{Tuple{String, String, Int64}}
│   %6 = Base.getindex(%5)::Tuple{String, String, Int64}
└──      return %6

It loads a Base.RefValue{T}, which is why we pass Ref{Base.RefValue{T}} as the return type.
In your example, it'd be Ref{TestMutableStruct}.

Sorry for the delay in replying. Yeah enabling both sounds good.
I'm intrigued to see how much of a difference removing the type instability makes to performance - will give it a go.

I'm intrigued to see how much of a difference removing the type instability makes to performance - will give it a go.

I saw basically no difference (you can't get much faster than 1.8 ns), but would do it on principle anyway. Would be easier on the branch predictor.

Yep also saw no difference for structs as you expected.
For mutable structs, tsptr doesn't work:
julia> mutable struct TestMutableStruct
       a::String
       end

julia> tms = TestMutableStruct("a")
TestMutableStruct("a")

julia> ref = Ref(tms)
Base.RefValue{TestMutableStruct}(TestMutableStruct("a"))

julia> p = Base.unsafe_convert(Ptr{typeof(ref[])}, ref)
Ptr{TestMutableStruct} @0x000000012a49d140

julia> Base.unsafe_pointer_to_objref(p)
TestMutableStruct("a")

julia> tsptr(p)
ERROR: TypeError: in typeassert, expected Base.RefValue{TestMutableStruct}, got a value of type TestMutableStruct
Stacktrace:
 [1] tsptr(::Ptr{TestMutableStruct}) at ./REPL[21]:1
 [2] top-level scope at REPL[22]:1

Is this expected? Seems that :jl_value_ptr returns the mutable struct in this case.

julia> tsptr2(x::Ptr{T}) where {T} = ccall(:jl_value_ptr, Ref{TestMutableStruct}, (Ptr{Cvoid},), x)
tsptr2 (generic function with 1 method)

julia> tsptr2(p)
TestMutableStruct("a")


Update WASM module to 0.7.1

Is your feature request related to a problem? Please describe.
New WASM Module version is include with the latest MobiFlight Connector Release
Describe the solution you'd like
New WASM Module version is include with the latest MobiFlight Connector Release
Additional context
Version 0.7.1

Good idea


Fix remaining Scala warnings (#970)

This pull request is a last step of #974 which fixes #970.

@jbaiera Oh, I've missed your last comment. Updated this request to remove -Ywarn-value-discard scalac option.


[Client] Fix send chunking message failed when ordering key is set.

Master Issue: #13688
Motivation
The root cause is the same as #13688. Chunking messages share the same metadata. And the type of ordering key is bytebuf, which can not be serialized twice.
Modifications
Reset the field before sending.
Verifying this change

[ ] Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

See testChunkingWithOrderingKey

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

Documentation
Check the box below and label this PR (if you have committer privilege).
Need to update docs?

[x] no-need-doc
bug fix

@gaoran10 @MarvinCai @codelipenghui  This is similar to the fix with schemaVersion #12720. PTAL

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

@RobertIndie Please also help review this PR.

From my perspective, it's so terrible to have all these issues just because all chunks share the same MessageMetadata. I'd like just deep copying this object instead of sharing the object and setting some fields carefully.
I also realized this potential issue when I implement the chunking message feature for C++ client, so I do the deep copy for the initial implementation.
Though for the sake of overhead of bytes copy, this solution looks good. But does it really affect? So I won't approve but neither will I request changes.

I'd like just deep copying this object instead of sharing the object and setting some fields carefully.

Yes, it's better to do it this way in the first implementation of chunking.
But before this fix, I have checked all the other bytes field. This is the last one.
And since there is no direct way to deep copy of this protobuffer object.
So I think it's OK to do it by resetting field for now.


Ability to load release details from somewhere

Hi is it possible to add an ability to load data about opensearch versions with api?
Not from the running cluster but from https://opensearch.org or any other host.
like /api/opensearch/versions - returning list of all versions
then
/api/opensearch/version/2.3/plugins - returning list of supported plugins
/api/opensearch/version/2.3/nodes/types - returning list of supported node types
and so on
Thank you in advance!

What's your use-case @boriskezikov? Are you looking for this information in an API format? Examples of what these would return?

@dblock My use case is an ability to parse actual data about different versions of opensearch automatically.
I need this information and  this and list of other details which i need to fetch automatically specified by version.

Cluster information/stats is https://opensearch.org/docs/1.2/opensearch/rest-api/cluster-stats/, plugins are https://opensearch.org/docs/latest/api-reference/cat/cat-plugins/ and nodes are https://opensearch.org/docs/latest/api-reference/cat/cat-nodes/. Is that what you're looking for?


feat: Add info cargo subcommand

close https://github.com/rust-lang/cargo/issues/14081
close https://github.com/rust-lang/cargo/issues/948
fcp https://github.com/rust-lang/cargo/pull/14141#issuecomment-2243683924
This PR added a new info cargo subcommand.
Background
This adds a new subcommand to Cargo, cargo info. This subcommand would allow users to get information about a crate from the command line, without having to go to the web.
The main motivation for this is to make it easier to get information about a crate from the command line. Currently, the way to get information about a crate is to go to the web and look it up on crates.io or find the crate's source code and look at the Cargo.toml file. This is not very convenient, especially not all information is displayed on the crates.io page.
This command also has been requested by the community for a long time. You can find more discussion about this in cargo#948.
Another motivation is to make the workflow of finding and evaluating crates more efficient. In the current workflow, users can search for crates using cargo search, but then they have to go to the web to get more information about the crate. This is not very efficient, especially if the user is just trying to get a quick overview of the crate. This would allow users to quickly get information about a crate without having to leave the terminal.
Example usage:
./target/debug/cargo info clap --verbose
  Credential cargo:token get crates-io
clap #argument #cli #arg #parser #parse
A simple to use, efficient, and full-featured Command Line Argument Parser
version: 4.5.8 (latest 4.5.9)
license: MIT OR Apache-2.0
rust-version: 1.74
documentation: https://docs.rs/clap/4.5.8
repository: https://github.com/clap-rs/clap
crates.io: https://crates.io/crates/clap/4.5.8
features:
 +default         = [std, color, help, usage, error-context, suggestions]
  color           = [clap_builder/color]
  error-context   = [clap_builder/error-context]
  help            = [clap_builder/help]
  std             = [clap_builder/std]
  suggestions     = [clap_builder/suggestions]
  usage           = [clap_builder/usage]
  cargo           = [clap_builder/cargo]
  debug           = [clap_builder/debug, clap_derive?/debug]
  deprecated      = [clap_builder/deprecated, clap_derive?/deprecated]
  derive          = [dep:clap_derive]
  env             = [clap_builder/env]
  string          = [clap_builder/string]
  unicode         = [clap_builder/unicode]
  unstable-doc    = [clap_builder/unstable-doc, derive]
  unstable-styles = [clap_builder/unstable-styles]
  unstable-v5     = [clap_builder/unstable-v5, clap_derive?/unstable-v5, deprecated]
  wrap_help       = [clap_builder/wrap_help]
dependencies:
<EMAIL_ADDRESS><EMAIL_ADDRESS>owners:
  kbknapp (Kevin K.)
  github:rust-cli:maintainers (Maintainers)
  github:clap-rs:admins (Admins)
note: to see how you depend on clap, run `cargo tree --invert --package<EMAIL_ADDRESS>

note: this is showing the --verbose output to show every thing the user can possibly see.  Normal operation does not include

dependencies

Detailed design

Content
Explanation
Why

clap
Name
The basic information.

#argument #cli #arg #parser #parse
Keywords (clickable)
It's more like a category, which you can use to search for relevant alternatives.

A simple to use, efficient, and full-featured Command Line Argument Parser
Description
The basic information.

version: 4.5.8 (latest 4.5.9)
Version
The basic information.

license: MIT OR Apache-2.0
License
When choosing a crate, it is crucial to consider the license.

rust-version: 1.74
MSRV
When choosing a crate, it is crucial to make sure it can work with your MSRV.

documentation: https://docs.rs/clap/4.5.8
Documentation Link
Use these links can find more docs and information.

repository: https://github.com/clap-rs/clap
Repo Link
Use these links can find more docs and information.

crates.io: https://crates.io/crates/clap/4.5.8
crates.io Link
Use these links can find more docs and information.

features:
Default Features And Other Features
It helps for enabling features.

dependencies:
All dependencies
It indicates what it depends on.

owners:
Owners
It indicates who maintains the crate.

note: to see how you depend on clap, run cargo tree --invert --package<EMAIL_ADDRESS>A note for cargo tree command
It will prompt the user that the package is depended on under the workspace, and the dependencies can be viewed using the cargo tree command.

Rendering features

For features enabled by users, a + prefix and colored output are now used for better visibility.
For features enabled automatically, colored output is used to distinguish them.
For disabled features, non-colored output is used to clearly indicate their status.

Rendering deps
Only show dependencies in verbose mode.

For dependencies required by the package, a + prefix and colored output are now used for better visibility.
For dependencies that are optional and activated, colored output is used to distinguish them.
For dependencies that are optional and not activated, non-colored output is used to clearly indicate their status.

Some important notes
Downloading the crate from any Cargo compatible registry
The cargo info command will download the crate from any Cargo compatible registry. It will then extract the information from the Cargo.toml file and display it in the terminal.
If the crate is already in the local cache, it will not download the crate again. It will get the information from the local cache.
Pick the correct version from the workspace
When executed in a workspace directory, the cargo info command chooses the version that the workspace is currently using.
If there's a lock file available, the version from this file will be used. In the absence of a lock file, the command attempts to select a version that is compatible with the Minimum Supported Rust Version (MSRV). And the lock file will be generated automatically.
The following hierarchy is used to determine the MSRV:

First, the MSRV of the parent directory package is checked, if it exists.
If the parent directory package does not specify an MSRV, the minimal MSRV of the workspace is checked.
If neither the workspace nor the parent directory package specify an MSRV, the version of the current Rust compiler (rustc --version) is used.

Prior art
NPM
npm has a similar command called npm info.
For example:
$ npm info lodash

<EMAIL_ADDRESS>| MIT | deps: none | versions: 114
Lodash modular utilities.
https://lodash.com/

keywords: modules, stdlib, util

dist
.tarball: https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz
.shasum: 679591c564c3bffaae8454cf0b3df370c3d6911c
.integrity: sha512-v2kDEe57lecTulaDIuNTPy3Ry4gLGJ6Z1O3vE1krgXZNrsQ+LFTGHVxVjcXPs17LhbZVGedAJv8XZ1tvj5FvSg==
.unpackedSize: 1.4 MB

maintainers:
- mathias<EMAIL_ADDRESS>- jdalton<EMAIL_ADDRESS>- bnjmnt4n<EMAIL_ADDRESS>
dist-tags:
latest: 4.17.21

published over a year ago by bnjmnt4n<EMAIL_ADDRESS>
Poetry
Poetry has a similar command called poetry show.
For example:
$ poetry show pendulum

name        : pendulum
version     : 1.4.2
description : Python datetimes made easy

dependencies
 - python-dateutil >=2.6.1
 - tzlocal >=1.4
 - pytzdata >=2017.2.2

required by
 - calendar >=1.4.0

insta-stable
As @weihanglo mentioned in https://github.com/rust-lang/cargo/pull/14141#issuecomment-2189223552, commands that shadow third-party commands tend to be insta-stabilized to avoid an intermediate period where users can't access the third-party command (built-ins get priority) nor the built-in command (requires nightly)
For the cargo-info command, there are two commands that this would shadow

cargo-information: hasn't been around too long and only has 4k downloads
cargo-info : been around longer and has 63k downloads

We might be able to get away with having this unstable but starting from the assumption of insta-stabilization.

I remembered last time when we imported new commands we insta-stablized them (cargo-add and cargo-remove) because built-in commands shadow external ones, making them unusable.
If the Cargo team are happy with making this a built-in command, I suppose we could do the same thing.

Overall, seems ready to go for adding in tests, completions, and docs.

@hi-rustin I modified the PR description

Added a note on the example usage about --verbose
Made some clarifications to the stabilization plan

@rfcbot fcp merge
This insta-stabilizes a cargo info <pkgname> command, as suggested in #948 and with parallels in other ecosystems.  The command is a UI-focused command (few compatibility constraints) with a focus on what the caller-facing side of a library.

You might also want to add cargo-info to src/doc/man/cargo.md. See #14339.

Added. Thank you!

@epage @weihanglo I guess this PR is ready to merge. Could you please help merge it when you have time? If there is anything I need to update, please let me know.
Thanks for your review!  💚 💙 💜 💛 ❤
@epage Thank you for guiding me in this work, I really appreciate it. Thank you for all your time.

I tried giving this a spin, and was very surprised to find that it was requesting my token for crates.io. Is there a reason it must do that? That seems like it will be a bad user experience if someone needs to enter a password or use a biometric or hardware key in order to use this command.

I tried giving this a spin, and was very surprised to find that it was requesting my token for crates.io. Is there a reason it must do that? That seems like it will be a bad user experience if someone needs to enter a password or use a biometric or hardware key in order to use this command.

I will take a look. It shouldn't require it. I believe I fixed it in my previous local test.

I tested it locally:
❯ ./target/debug/cargo logout
      Logout token for `crates-io` has been removed from local storage
note: This does not revoke the token on the registry server.
    If you need to revoke the token, visit <https://crates.io/me> and follow the instructions there.

cargo on  rustin-patch-info is 📦 v0.83.0 via 🐍 v3.12.4 via 🦀 v1.82.0-nightly
❯ ./target/debug/cargo info serde
serde #serde #serialization #no_std
A generic serialization/deserialization framework
version: 1.0.204
license: MIT OR Apache-2.0
rust-version: 1.31
documentation: https://docs.rs/serde
homepage: https://serde.rs
repository: https://github.com/serde-rs/serde
crates.io: https://crates.io/crates/serde/1.0.204
features:
 +default      = [std]
  std          = []
  alloc        = []
  derive       = [serde_derive]
  rc           = []
  serde_derive = [dep:serde_derive]
  unstable     = []
note: to see how you depend on serde, run `cargo tree --invert --package<EMAIL_ADDRESS>
It seems it doesn't require it.

I also logged out and couldn't reproduce.
As we develop reproduction steps, we can iterate from there.
@bors r+

:pushpin: Commit ba0721501484b2cfd9a2820a37be158083c38a7d has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit ba0721501484b2cfd9a2820a37be158083c38a7d with merge f3fee6d2501567faf0a7d5fc29e4bce9425cdaa5...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing f3fee6d2501567faf0a7d5fc29e4bce9425cdaa5 to master...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing f3fee6d2501567faf0a7d5fc29e4bce9425cdaa5 to master...

:eyes: Test was successful, but fast-forwarding failed: 422 Changes must be made through a pull request.

Opened https://github.com/rust-lang/cargo/issues/14409 regarding the authentication problem.


Incorrect filenames with --print-json and -x

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
When using extract audio with print-json (I am invoking yt-dlp from another program), both the filename and _filename fields contain the filename of the file before audio extraction, with no way to get the actual resulting file.
This is a problem because I use these fields to read the resultant file off of the disk, yet the extension is often incorrect.
I suggest either changing one of these fields to take into account the final filename after extraction, or add a new field for this purpose.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
Output is too many characters for a Github issue body (presumably due to JSON).

Check this gist: https://gist.github.com/yellowsink/1ba2ae5ec6dc0ce86c412e3777ebbef6

The final filename can be obtained only after the download+postprocess
❯ yt-dlp test:youtube -O after_move:filepath -vx --no-sim
[debug] Command-line config: ['--ignore-config', 'test:youtube', '-O', 'after_move:filepath', '-vx', '--no-sim']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b644025b] (source)
[debug] Lazy loading extractors is disabled
[debug] Plugins: ['SamplePluginIE', 'SamplePluginPP']
[debug] Git HEAD: 9f14daf22
[debug] Python 3.11.0 (CPython AMD64 64bit) - Windows-10-10.0.22000-SP0 (OpenSSL 1.1.1q  5 Jul 2022)
[debug] exe versions: ffmpeg N-107787-gc469c3c3b1-20220814 (fdk,setts), ffprobe N-107787-gc469c3c3b1-20220814, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1725 extractors
[debug] [TestURL] Extracting URL: test:youtube
[TestURL] Test URL: https://www.youtube.com/watch?v=BaW_jenozKc&t=1s&end=9
[debug] [youtube] Extracting URL: https://www.youtube.com/watch?v=BaW_jenozKc&t=1s&end=9
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[info] BaW_jenozKc: Downloading 1 format(s): 251
[debug] Invoking http downloader on "https://rr3---sn-gwpa-jjwe.googlevideo.com/videoplayback?expire=1668755760&ei=0Nx2Y6niJdW2rtoPgIekoAU&ip=<IP_ADDRESS>&id=o-ABFLaP209_RiQQFEkfUyGhZJEG97s9LTDns050ZbSZVA&itag=251&source=youtube&requiressl=yes&mh=Rf&mm=31%2C29&mn=sn-gwpa-jjwe%2Csn-gwpa-h55e7&ms=au%2Crdu&mv=m&mvi=3&pl=20&initcwndbps=218750&spc=SFxXNtaZdBUpFrJxFjKw9cC-NmTA_t0&vprv=1&svpuc=1&mime=audio%2Fwebm&gir=yes&clen=142292&dur=9.841&lmt=1506055557712479&mt=1668733735&fvip=1&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAP70gxktzy_ZDQFuTTxbNVfBviWsC_NilNBNwZLBGSx_AiBVamyCG6G_LNBQjdc0YA92ir0OhV_rxlIk3atn4HWfoQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgSTIuMLtS6eN55YuwdKrv7iWGtyqo8EzCh90XEaoB1CgCIQDZcugrF00KqNLZ_kDBkoxl6ZFngLJUrrDH_5mOtj6d_A%3D%3D"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].webm
[debug] ffmpeg command line: ffprobe -show_streams "file:youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].webm"
[ExtractAudio] Destination: youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].opus
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].webm" -vn -acodec copy -movflags "+faststart" "file:youtube-dl test video ＂'⧸⧹ä↭𝕐
[BaW_jenozKc].opus"
Deleting original file youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].webm (pass -k to keep)
D:\Programs\Source\yt-dlp\yt-dlp\youtube-dl test video ＂'⧸⧹ä↭𝕐 [BaW_jenozKc].opus

Replace filepath with %()j to get the full json

-O after_move:filepath is exactly what I needed, thank you very much :)


I can't run the demo, it provides a error, Object(...) is not a function

I've updated the demo and tested it on my end using a fresh clone. Can you confirm it works on your end as well?

when I run this demo, it still provides an error .
H264NALDecoder.worker.js:5 Uncaught TypeError: Object(...) is not a function at eval (H264NALDecoder.worker.js:5)
the version of tinyh264 is 0.0.3, but the package version at npm  is 0.0.1
does it work on your end?

Good catch. Version 0.0.3 (with the fix) should be available now on npm.

import the package, function init is undefined, I think init may be call by promise.
when I console.log init, it shows undefined, I think the code of "exports {init}" doesn't work.
can you run the demo on your machine?

when I use the lastest demo , it runs well .  but I can't find the differences between 0.0.1 and 0.0.3。 could you tell me  your changes of code

There's a file in 0.0.1 with name foo.worker.js This is picked up by webpack and treated as stand-alone webworker (which is not the intention). Inside the demo project is the actual bar.worker.js that imports foo.worker.js and the result is a worker that spawns a worker... the result is a mess that gives this error.

Thank you @Zubnix for your lastest answer.
But I use another file of h264, that decoder fails, it provide "TinyH264Decoder.ERROR = 3 or
TinyH264Decoder.PARAM_SET_ERROR = 4". if I should discard some frame?

I'm not sure what the problem is. It is perhaps important to know that TinyH264 only handles restricted-baseline and baseline decoding. Anything more will probably not work. You also have to feed it exact h264 Access Units (AU), which is basically a complete frame and consists of 1 or more H264 Network Abstraction Layer Units (NAL(U)s). If you give it the whole file at once, it won't really know how to read the next AU, you'll need an H264 (stream) parser for that.


ionic resources generated icon has error

From @heqiang421 on May 7, 2017 15:43
What version of the CLI are you using? Output of ionic --version
2.2.3
Short description of the problem:
type ionic resources
i found icon is not right

The original picture is ok. i used default-pic

What behavior are you expecting?
icon is right
package.json
 "dependencies": {
    "@angular/common": "4.1.0",
    "@angular/compiler": "4.1.0",
    "@angular/compiler-cli": "4.1.0",
    "@angular/core": "4.1.0",
    "@angular/forms": "4.1.0",
    "@angular/http": "4.1.0",
    "@angular/platform-browser": "4.1.0",
    "@angular/platform-browser-dynamic": "4.1.0",
    "@ionic-native/core": "3.6.1",
    "@ionic-native/in-app-browser": "3.6.1",
    "@ionic-native/splash-screen": "3.6.1",
    "@ionic-native/status-bar": "3.6.1",
    "@ionic/storage": "2.0.1",
    "d3": "^4.5.0",
    "echarts": "^3.4.0",
    "ionic-angular": "3.1.1",
    "ionicons": "~3.0.0",
    "rxjs": "5.1.1",
    "sw-toolbox": "3.4.0",
    "zone.js": "^0.8.5"
  },
  "devDependencies": {
    "@ionic/app-scripts": "1.3.7",
    "typescript": "~2.2.1"
  },

Copied from original issue: driftyco/ionic#11544

https://github.com/driftyco/ionic-cli/issues/2226

How to solve this problem


NFS: iterate over all endpoint addresses

When the provisioner is deployed as a daemonset, its service has one address for each pod deployed. The IP selection shouldn't assume just one address.
Example of endpoints object on OpenShift when the nfs-provisioner is deployed as a daemonset to two nodes:
apiVersion: v1
items:
- apiVersion: v1
  kind: Endpoints
  metadata:
    name: nfs-provisioner
    namespace: ns
  subsets:
  - addresses:
    - ip: <IP_ADDRESS>
      nodeName: f25-ose2.vm
      targetRef:
        kind: Pod
        name: nfs-provisioner-4-sp7ch
        namespace: ns
    - ip: <IP_ADDRESS>
      nodeName: f25-ose2.vm
      targetRef:
        kind: Pod
        name: nfs-provisioner-4-xbxhq
        namespace: ns
    ports:
    - name: 2049-tcp
      port: 2049
      protocol: TCP
    - name: 20048-tcp
      port: 20048
      protocol: TCP
    - name: 111-tcp
      port: 111
      protocol: TCP
    - name: 111-udp
      port: 111
      protocol: UDP

Maybe I am missing something and there's a good reason for ignoring multi-address endpoints. If so, feel free to close this with some brief explanation.

/cc @erikbsap

Thanks for making this PR, miminar. I'm interested in this as well.
Usecase: I have a cluster with 10 nodes, each of them has about 50 GB to spare. So I'd like to provide the nfs-provisioner on each of these nodes, but put all behind one service. All of these nodes are bare metal in a data center not under my control, so just repprovisioning the 50 GB to the other hosts would not be easy, maybe impossible if they are not separate disks.

One service can't have multiple nfs-provisioner endpoints because if somebody tries to mount using the service IP, kubernetes will round-robin the request and it will probably go to the wrong instance.
The docs talk about using nodePort as a workaround, the downside being that now you're taking up & exposing the node's entire nfs port. I don't recall any other workaround.

Thank you @wongma7 for the explanation. I've changed the PR to reword the error reporting to make the cause more clear.

/lgtm
/area/nfs
thank you


Docs: doc.deno.land link in "First steps" page is broken

The link to the deno docs on the "First steps" page is broken.
It leads to this URL: https://doc.deno.land/https/github.com/denoland/deno/releases/latest/download/lib.deno.d.ts

You are using a very outdated version of the manual (v1.5.1 to be specific).  The current (and recent) versions of the manual are working as expected.
Was there a link somewhere that led you to the very outdated version of the manual?

How odd. I have no idea how I have landed on such an outdated version. But I must have followed a link there. I set up deno yesterday, but didn't get around to tinker with it. And now I don't remember how I got to the docs.
I suppose there's not much to do here unless I remember how I arrived at the outdated docs.


Paper Discussion 9b: IoTPOT: Analyzing the Rise of IoT Compromises

Please add your feedback and reviews below.

Reviewer: Sean McBride
Review Type: Comprehension Review
Problem Being Solved:
There have been anecdotal reports that the number of telnet-based attacks against IOT systems have increased. What is the nature of these attacks?
Main Contributions:

Validated the anecdotal observation that there has been a large increase in Telnet-based attacks on IOT devices.
Designed a novel Honeypot (IOTPOT) with a frontend that simulates the Telnet protocol and a backend (IOTBOX) and uses qemu to execute malicious commands in ephemeral sandboxes and then store the request-response interactions in a database for later memoization via the front-end.
Ran the honeypot for a bit over a month to capture 76,605 malware download attempts and then performed post-op analysis of the attacks, demonstrating the flow of telnet-attacks across several families.

Questions:

ZORRO removed binaries from Unix locations in path to make it harder for other botnets to steal the node. Wouldn't this potentially break the device itself, making the end-user aware something is happening?
What is the advantage of having a slow progressive attack (ZORRO) rather than performing the entire exploit in one go? Is this for security reasons or just for engineering / distributed system reasons (our pool of MIPS penetrators is saturated...)?
What is it about the telnet protocol that makes is especially vulnerable? On the flip side, why is this exposed publicly on a router?

Reviewer: Akinori Kahata
Review type: Comprehensive

The problem being solved.

Recently, the number of Internet of things devices is increasing, and many of these devices are vulnerable to cyberattacks. Especially, many of these are used by Distributed Denial of Service (DDoS) attack as part of a botnet. To prevent those IoT devices from becoming a part of a botnet, to clarify the attack pattern of adversaries is essential.

The main contributions.

Firstly, by researching attacks from adversaries to darknet of Japan, the researchers showed the number of scanning to IP addresses from a device of a botnet. Next, they made IoT Honeypot (IoTPOT), and using by IoTPOT gathered information of cyberattack, concretely intrusion, infection and monetization. As a result, they revealed 17 malware binaries and attack types of each malware.

Questions.

Why they chose attacks that use telnet for their first research? I want to know comparing data with the number of using other ports.
I think the goal of security research is to improve the cybersecurity of society. What is the recommendation for developers and consumers to improve cybersecurity from the result of this research?

Reviewer: Alvaro Albero
Review Type: Analytical
Problem domain
The number of attacks to IoT devices keeps increasing and many of these attacks are Telnet attacks. The authors want to understand more on how these attacks are performed and what are they targeting.
Main contributions
The authors perform an analysis and confirm the increasing number of Telnet attacks. First, they create a honeypot to receive this kind of attacks by simulating different IoT devices. When an attack is performed, they analyze it and capture the malware samples. Second, they propose a Box that is able to run the attacker’s commands and the captured malware for further understanding. Finally, they have organized the attacks in different families based on the actions performed, the architectures they are targeting and their goals.
Questions

Are the devices counted in section 2 malicious? Have they been compromised or just example user devices?
Does the IoTBox only gets used when the attacker is unknown?  I understand that the commands are sent to the IoT box and then the system learns from them and creates a profile. Once there is a profile, the IoTBox is never needed again for that type of attack?
How are they logging all the actions that an attacker performs? From using a dictionary, to deleting binaries and downloading files to performing DoS attacks.

Critiques

I really liked this paper, explains an interesting concept to gather information about malware attacks to IoT devices that had not been used before, as previous work was mainly done with low interaction honey pots. They provide a lot of detail an information in most of the sections explaining how they did a previous research. They explain the design and implementation of the honey pot and the exact commands that the attacks tried to perform, something that helped me understand better how security attacks work. They also provide good information about IoTBox and the results.
The only part that I would have liked more detail is when they talk about downloading manually the collected malware samples. I guess they found these IP addresses based on the IP addresses that the infected devices try to download files from, but I do not think they explicitly say it. They could have also given more detail about this “malware repositories”, where are they hosted, how you can access them, what type of malware you can download, etc.

Reviewer: Rick Sear
Review Type: Comprehension
Problem being solved
IoT devices often have weak or no passwords, making them vulnerable to very simple cyberattacks. This paper introduces a honeypot to draw in attackers and gain insight into their techniques, as well as provide some analysis of what they found.
Important contributions
The paper introduces IoTPOT, which acts like IoT devices running on various hardware architectures. Attackers enter the IoTBOX environment; their interactions are monitored by a Frontend Responder, Profiler, and Downloader to gather information about the attack and attacker.
Questions

This paper was published in 2015. What's the landscape like today? Do attackers still go for these easy Telnet attacks? How much have the infection patterns (3.4.2) changed?
What is C&C communication? All I can find on Google is listings for companies.
Is there any danger that, now that this research is published, attackers will be able to figure out sophisticated ways of avoiding honeypots? From a networking perspective, how hard is it to tell the difference between honeypot virtualized hardware and an actual IoT device?

Reviewer: Eric Wendt
Review Type: Critical
Problem being solved
IoTPOT describes itself as a honey-pot for gathering security analytics for IoT devices and attacks. The tools that IoTPOT provides are extended features from what already exists. Analyzing attacks threats to IoT systems is important for the sheer amount of them exist.
Main contributions

The first contribution this paper provides is a diagram of how their system works. This isn't super detailed, and more details would have been helpful, but for the sake of simplicity, the diagrams are effective.
Another contribution is how their programs run. They essentially emulate CPU hardware in virtual IoT devices. These devices then deliberately open themselves up for attack. The system will then monitor the attacker's behaviour at every step.

Questions

how are attacks identified, what technology do they use?
Is the system purely for identifying attacks, or are there prevention methods built in as well?
Why do some of the attacks log off after successfully getting creds? It's like there's a separate attack after the fact.

Critiques

please more detail. I got no sense of how this system actually identifies and handles attacks, just simply that they do.
Explain Telnet better?

Reviewer: Sam Frey
Review Type: Critical
Problem:
IoT devices often lack sufficient security to protect themselves against basic intrusion attempts. Many IoT devices continue to use weak, default login credentials, leaving themselves accessible to attackers. This makes them an easy target, and their always-on nature makes them the perfect candidates for a malicious botnet.
Important Contributions:
The authors created IoTPOT, a honeypot, to attract malicious attackers and capture malware samples. With these samples, the authors can gain a better understanding of attacks against IoT devices, and thus, a better idea of how to protect them. With IoTPOT, the authors were able to identify 4 distinct families of malware attacking IoT devices.
Questions:

Why is the distribution of devices in section 2 so skewed towards DVRs? Is there something about DVRs that makes them "ideal candidates" for a telnet-based attack?
Is current anti-virus so bad that only 2 out of the 43 collected samples of malware were recognized? That seems like security is falling way behind. Why did it take this long for someone to recognize this?
Were attackers aware that their DoS attacks were being blocked by the Access Controller, or did the IoTPOT have a way to make it appear to the attacker that everything was working the way they expected?

Critique:
While the paper is well written and easy to understand, it lacks substance. The authors did a great job of explaining what they did, but not so much how they did it. I do recognize that this may be an intentional decision though. As Rick (@searri) brought up, publishing this paper makes it available to attackers as well. Too much detail on implementation could render IoTPOT useless.

Reviewer: Reese Jones
Review Type: Critical
Problem Bring Solved:
Even though there have been advancements in security for General Purpose Computers and other "normal" devices, IoT Hardware consistently relies on security techniques which leaves them open to compromise and attack. The systems are often so insecure they are run off of basic default/no password protection and are therefore vulnerable to some of the most basic intrusion attacks available.
Main Contributions:
The paper discusses IoTPOT, a honeypot created by the authors which is meant to lure in any potential attackers. On top of that IoTPOT is meant to help used those lured in attackers to grab  information about attacks in order to better understand the security threats facing IoT devices.
Questions

What exactly is the technology behind identifying attacks? I felt as though that was not discussed enough.

Is Telnet as much an attack vector today as it was when the paper was published? A quick google search did not yield an IoT and Telnet article dated after 2016.

What about Telnet made it the -in my reading- most significant attack vector in the whole paper? As in why is it such a risk.

Critiques

Through reading this my main critique lies with the fact the paper exists at all, doesn't having this information out there in any capacity completely negate the purpose of developing and designing the IoTPOT system?

Just as mentioned in my questions, if they're going into so much detail re: the IoTPOT system already I would have liked to see more about how attacks are monitored and cataloged.

Reviewer: Rachell Kim
Review Type: Comprehension
Problem Being Solved:
There has been an increase in DDoS attacks on Telnet-based IoT devices since 2014. This issue prompted the need to analyze and study how IoT devices are being compromised by different types of malware.
Main Contribution:
The authors propose a honeypot called IoTPOT to simulate IoT devices and capture details of Telnet-based attacks. Moreover, they also propose and utilize a backend virtual environment called IoTBOX to execute captured malware and perform analysis. From the results, the authors have been able to identify different families of malware and certain characteristics of their behaviors.
Questions:

What does it mean to categorize device type by HTTP title? And what “available manuals” are they referring to for categorization when there exists no telling keyword?
I’m a bit confused at how 70k attackers were able to find the IoTPOT over a 39 day period, but this might be rooted in my misunderstanding of what a honeypot is. Is it emulating device information over a network to attract attackers? Or is it an open port? How do they find the honeypot to log in?

Reviewer: Alex Jacobson
Review Type: Critical
Summary of problem being solved:
The threats against IOT devices are increasing. The authors design honeypot and sandbox in order to attract these attacks to analyze them.
Main contributions:
The authors are kind enough to summarize their own contributions. They detect a large increase in the number Telnet based attacks and the involvement of IOT devices. They propose a Honeypot to analyze those attacks, and they also propose IOT-box to run the malware on different cpu architectures in order to further analyze the attacks.
Questions:
What is Telnet? Seems like some communication protocol, but I do not know what exactly it is.
What is a darknet? I believe the illegal online drug marketplaces are hosted on the dark web. Are darknet and dark web equivalent?
What is 23/TCP? Is it a port? Not entirely clear on that.
Critiques:
The paper uses a lot of vocabulary that I am not familiar with. Perhaps i’m just unprepared to read this paper, but a bit of background would have been nice. For example, I do not know exactly what a honeypot is, as it is used in this paper. I eventually inferred that it is liked honey too bees, trying to attract the attacks in order to analyze them, but overall more background would have been nice.
The authors use a device to attract attacks and analyze them. They learned that there are at least four DDoS malware families targeting IOT devices. While this is nice to know, what is the significance? What can we do with that knowledge in order to prevent attacks? Would have been good to discuss that.


[zh] sync /workload-resources/job-v1.md

Sync with en text:
content/zh-cn/docs/reference/kubernetes-api/workload-resources/job-v1.md

See preview please.

/lgtm

/approve


add ability to use component props in connectMultireducer

Aloha!
It would be good to have the same api for * connectMultireducer* as react-redux connect has.
https://github.com/rackt/react-redux/blob/master/docs/api.md#connectmapstatetoprops-mapdispatchtoprops-mergeprops-options
In my usecase I want to use ownProps of component to connect to some kind of key-value structure in state where key - is a property of component connected using multireducer.
For such usecase fix is pretty simple. But to support the same api as original react-redux connect - it might take a little bit more time.
What's your opinion about that feature?
Cheers.

@jsdmc FYI I created a fork of this project that has the same interface as react-redux connect, as well as some other changes. It is here: https://github.com/adailey14/multireducer

@adailey14 Let's add some tests, and help @erikras to make this lib better)

Created pull request https://github.com/erikras/multireducer/pull/82

@jsdmc ok sounds good.
What do you think of some of the other semantics I introduced in my fork? If you like them I can work them into a PR for this lib and make them backwards compatible (so the current API would still work):
Instead of:
connectMultireducer(mapStateToProps(preSlicedState), actionCreatorHash),
It provides:
connectMultireducer(mapStateToProps(topLevelState, multireducerKey), mapDispatchToProps(dispatch, multireducerKey))
And it provides multireducerBindActionCreators(actionCreators, multireducerKey, dispatch) to use within mapDispatchToProps.
This patches the dispatch function to catch any 'thunk' action creators so that actions they dispatch will have the multireducerKey attached. This allows 'thunk' middleware to function as expected. A limitation here is you cannot dispatch actions to any other part of the reducer tree from within these thunks, since the multireducerKey would be attached to those actions. Potentially we could pass in 2 dispatch functions in such cases, a 'global' dispatch and a 'multireducer' dispatch to allow dispatching both types of actions.
I also switched the way multireducerKey is attached. Instead of wrapping the actionCreator so it returns a wrapper object with a key and the action, instead I append the key directly to the action's type attribute. This allows the action to be passed through middleware and have the structure middleware is expecting. The appending string to type part was also useful for me for debugging in redux devtools, but maybe was a bad idea for the flexibility of the library. Maybe it should be added as a property to the action. This does limit you to using objects as actions though, rather than simpler data structures.

@adailey14 I played around with your fork. All looks good, but it doesn't have the same api as origin connect. Your version of connectMultireducer doesn't allow to access to ownProps of component.
Supporting thunk actions is a big plus. I'd like to use this functionality also!
@erikras What's your opinion about redux-thunk support?

@erikras Any chance to merge @adailey14 's effort?


Allow Parallel Usage of Different Brokers

Closes #50

Contains things that need to be dealt with in future issues. Due to lack of time, we will merge this anyway. Related Issues will be linked as they are created.


Incorrect Promise Date Displayed in Product Summary Card

Current behavior
The promise date shown in the Product summary card corresponds to the created date of the purchase order, rather than the actual promise date.
Expected behavior
The promise date displayed in the Product summary card should accurately reflect the corresponding purchase order's promise date.
Steps to reproduce the issue

Go to the Catalog product details page using the provided link: Catalog product details page.
Observe the promise date information presented in the product summary card.
Go to the demo OMS by following this link: View Purchase order page.
Observe the promise date mentioned on the Order details page and compare it with the Order date.

Can you handle fixing this bug by yourself?

[ ] YES
[ ] NO

Environment details

Browser:
OS:
Code Version:

Additional information
Catalog Product Detail page:
View Purchase Order page:

This is fixed with https://github.com/hotwax/preorder/pull/153
Please verify

Checked, yes its fixed.


MSSQL nvarchar  - missing column in output file

Hello,
I was doing bit of poc getting tables from MS SQL server to file system. The table in MS SQL has columns with nvarchar data type and after extracting the data into parquet file, those nvarchar columns are missing. Any thoughts to deal with this?

Hello @AmpGana ,
could you please post the output written to standard error? Also add the -vvv for more verbose logging. What version did you use.
Earlier versions of odbc2parquet emitted a warning for variadic sized columns if no upper bound has been specified. Newer versions should fail with an error though.

As no NVarchar columns are missing in my tests, and there is no further feedback, this is sadly not actionable. I will close the issue. If there is an issue with the most recent version of odbc2parquet and some information about how to reproduce it, feel free to reopen the issue.
Best, Markus


Metrics via dagger

Introduces Dagger to provide dependencies.

Migrates an existing MetricsSystem module up into the metrics gradle module.
Makes BesuCommand accessible from Dagger components, to enable configuration reuse.
Uses CachedMerkleTrieLoader as an example of how to get and use a MetricSystem.

@jflo I attempted a second pass after reading up on Dagger, but I think the recent merge was dodgy and has polluted the PR files so it's hard to review now.
Are you back to working on it or is it still ready for review?

still getting my head around dagger also. This change doesn't get us out of passing the metricsSystem around eg to transactionPool - it just makes setup easier?

Correct. Once transactionPool is itself constructed by Dagger, then we no longer need to pass it or the BesuContext around ourselves.

Do you have any suggestions of what would be a good one to get started with to familiarise with dagger?

The ideal candidate would be something used rarely, but deeeeeply within the overall object graph.