semp-tcp.py integer error for port number

Hi Niels,
I know you made this semp-tcp version for my request and couldnt test it yourself.
I recieved the gateways and am testing it now.
The Meter is working via ModbusTCP, i have run the script with semp.rtu.py fine for the last couple of days.
I get this error with the script:
pi@raspberrypi4:~/eastronsolaredgetcp $ python3 semp-tcp.py
2020-12-14 14:43:59 INFO: Created <Thread(t_update_2, initial)>: meter1 sdm630 SDM630(<IP_ADDRESS>:8899, connectionType.TCP: timeout=1, retries=3, unit=0x1)
2020-12-14 14:44:00 INFO: Starting <Thread(t_update_2, started<PHONE_NUMBER>)>
Traceback (most recent call last):
File "semp-tcp.py", line 251, in
confparser["server"].get("port", fallback=default_config["server"]["port"])
File "/home/pi/.local/lib/python3.7/site-packages/pymodbus/server/sync.py", line 616, in StartTcpServer
server = ModbusTcpServer(context, framer, identity, address, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/pymodbus/server/sync.py", line 339, in init
self.handler)
File "/usr/lib/python3.7/socketserver.py", line 452, in init
self.server_bind()
File "/usr/lib/python3.7/socketserver.py", line 466, in server_bind
self.socket.bind(self.server_address)
TypeError: an integer is required (got type str)

Oh, BTW this is my config
[server]
address = <IP_ADDRESS>
port = 8898
meters = meter1
log_level = INFO
[meter1]
dst_address = 2
src_address = 1
type = sdm630
host = <IP_ADDRESS>
port = 8899

Great to hear you're trying it out. The TCP port is now explicitly cast to integer. Give it another try.

python3 semp-tcp.py
2020-12-14 15:37:18 INFO: Created <Thread(t_update_2, initial)>: meter1 sdm630 SDM630(<IP_ADDRESS>:8899, connectionType.TCP: timeout=1, retries=3, unit=0x1)
2020-12-14 15:37:19 INFO: Starting <Thread(t_update_2, started<PHONE_NUMBER>)>
Traceback (most recent call last):
File "semp-tcp.py", line 251, in
confparser["server"].getint("port", fallback=default_config["server"]["port"])
File "/home/pi/.local/lib/python3.7/site-packages/pymodbus/server/sync.py", line 616, in StartTcpServer
server = ModbusTcpServer(context, framer, identity, address, **kwargs)
File "/home/pi/.local/lib/python3.7/site-packages/pymodbus/server/sync.py", line 339, in init
self.handler)
File "/usr/lib/python3.7/socketserver.py", line 452, in init
self.server_bind()
File "/usr/lib/python3.7/socketserver.py", line 466, in server_bind
self.socket.bind(self.server_address)
OSError: [Errno 99] Cannot assign requested address

Is <IP_ADDRESS> the IP of your Pi, and is it assigned to an interface? Can you ping the address?

<IP_ADDRESS> is ip of the gateway connected to the solaredge.

In that case you need to use the IP adress of the Pi, not the gateway. In turn you will need to tell the gateway to be a TCP client and provide it with the IP of the Pi.

So my config is now:
address = <IP_ADDRESS>
port = 5502
i should tell my gateway to:
server: <IP_ADDRESS>
port: 5502

Correct.
Any clues?

I see this coming buy:
2020-12-14 16:35:55 DEBUG: Started thread to serve client at ('<IP_ADDRESS>', 7506)
2020-12-14 16:35:55 DEBUG: Client Connected [<IP_ADDRESS>:7506]
With, what it looks like, a changing port all the time?

I checked on the inverter itself. It says com error, missing meter.
So it doesn't seem to work? What could be wrong?

Could you provide a screenshot of the gateway setup screen, specifically where you set the server IP, port, and maybe some other options? As far as I can see you are receiving requests from the inverter, Factory Response[ReadInputRegistersResponse: 4] for example.

i did once again checked if the connection between PI-->Gateway-->inverter is working with switchting back to non-solaredge-logger and using the solaredge-modbus script.
The connection is working, so the gateway working as SERVER works okay!

Turns out the modbus messages you're seeing are caused by the meter updating values in semp-tcp.py, not the inverter sending requests. I will need to look into the documentation of your gateway a bit more to see what settings are required.

Hi Niels,
Im using this unit: EE11
http://www.hi-flying.com/elfin-ee10-elfin-ee11

The documentation is frustratingly incoherent. Would you mind setting the gateway back to tcp client, with all the settings back to how you had them, and then put as many screenshots of the system, serial port, and socket settings pages. Perhaps it's as simple as a single option set incorrectly.

Hi Niels,
These are the settings for the serial port. Not much more to be set actually...

Could you set the baud rate to 9600?
Also, what options are available in the Protocol dropdown?

Like i write before, the serial to tcp server part works as I can retrieve inverter data with your Solaredge-modbus scripts.

Our messages crossed each other.
These are the protocol options.
Maybe it should be None

No, if I understand the documentation correctly the Modbus protocol setting enables Modbus RTU to TCP translation.
I understand the Inverter acting as server works (gateway is client to the inverter modbus rtu server). But in this case the inverter must act as client, and it may be more particular regarding serial settings.

Would it be possible to write a simple script to see if the inverter is sending/requesting data only and no info of the meter?
This way we can debug the connection to the meter?

Would it be possible to write a simple script to see if the inverter is sending/requesting data only and no info of the meter?
This way we can debug the connection to the meter?

Sure. Have you tried a baud rate of 9600 already?

Yes i tried. No difference as in I don't see any data appearing in SolarEdge.
I put my laptop away and copying data from the tool on my iPhone is very messy....

Try the following script. It creates a simple Modbus TCP server, and has logging set to debug.
#!/usr/bin/env python3

import logging
import sys

from pymodbus.server.sync import StartTcpServer
from pymodbus.constants import Endian
from pymodbus.device import ModbusDeviceIdentification
from pymodbus.transaction import ModbusRtuFramer
from pymodbus.datastore import ModbusSlaveContext
from pymodbus.datastore import ModbusServerContext
from pymodbus.payload import BinaryPayloadBuilder

log_handler = logging.StreamHandler(sys.stdout)
log_handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s: %(message)s", datefmt="%Y-%m-%d %H:%M:%S"))

logger = logging.getLogger()
logger.addHandler(log_handler)
logger.setLevel(logging.DEBUG)

slave = ModbusSlaveContext()
block_1601 = BinaryPayloadBuilder(byteorder=Endian.Big, wordorder=Endian.Little)
block_1601.add_32bit_int(1234) # config passcode
slave.setValues(3, 1600, block_1601.to_registers())

identity = ModbusDeviceIdentification()
server_ctx = ModbusServerContext(slaves={"2": slave}, single=False)

StartTcpServer(
    server_ctx,
    identity=identity,
    address=("<IP_ADDRESS>", 5502)
)

How fast should i start to see data coming in Putty?
pi@raspberrypi4:~/eastronsolaredgetcp $ python3 test.py
2020-12-15 13:47:32 DEBUG: [1234, 0]
2020-12-15 13:47:32 DEBUG: setValues[3] 1601:2
2020-12-15 13:47:34 DEBUG: Started thread to serve client at ('<IP_ADDRESS>', 3561)
2020-12-15 13:47:34 DEBUG: Client Connected [<IP_ADDRESS>:3561]
In the gateway at communication settings i can also set the local port. If i set it to 5502 this port is also shown in putty.

Hi, so the output now is:
pi@raspberrypi4:~/eastronsolaredgetcp $ python3 test.py
2020-12-15 13:47:32 DEBUG: [1234, 0]
2020-12-15 13:47:32 DEBUG: setValues[3] 1601:2
2020-12-15 13:47:34 DEBUG: Started thread to serve client at ('<IP_ADDRESS>', 3                                                                                561)
2020-12-15 13:47:34 DEBUG: Client Connected [<IP_ADDRESS>:3561]

i changed at Serial port settings Protocol to NONE and this immediatly happened.
I also saw data the data counter started to increase on the TCP-client part of the status page
2020-12-15 14:26:31 DEBUG: Frame check failed, ignoring!!
2020-12-15 14:26:33 DEBUG: Handling data: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:33 DEBUG: Processing: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:33 DEBUG: Frame check failed, ignoring!!
2020-12-15 14:26:35 DEBUG: Handling data: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:35 DEBUG: Processing: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:35 DEBUG: Frame check failed, ignoring!!
2020-12-15 14:26:37 DEBUG: Handling data: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:37 DEBUG: Processing: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
2020-12-15 14:26:37 DEBUG: Frame check failed, ignoring!!

2020-12-15 13:47:32 DEBUG: [1234, 0]
2020-12-15 13:47:32 DEBUG: setValues[3] 1601:2

This is a part of the script, one of the registers is being filled with dummy data.

2020-12-15 13:47:34 DEBUG: Started thread to serve client at ('<IP_ADDRESS>', 3 561)
2020-12-15 13:47:34 DEBUG: Client Connected [<IP_ADDRESS>:3561]

This is your gateway connecting with the Pi. Immediately after this message you should be seeing a large number of requests every second. They look like this:

2020-12-14 17:00:28 DEBUG: Getting Frame - 0x3 0x3 0xf2 0x0 0x6
2020-12-14 17:00:28 DEBUG: Factory Request[ReadHoldingRegistersRequest: 3]
2020-12-14 17:00:28 DEBUG: Frame advanced, resetting header!!
2020-12-14 17:00:28 DEBUG: validate: fc-[3] address-1011: count-6
2020-12-14 17:00:28 DEBUG: getValues fc-[3] address-1011: count-6
2020-12-14 17:00:28 DEBUG: send: [ReadRegisterResponse (6)]- b'02030c999a439000000000000000008deb'

You're not, so either the gateway is not forwarding messages from your inverter, or the inverter is not sending the messages. Both could be a configuration issue in the gateway or inverter.
Remove the gateway and connect the Pi your inverter directly using the RS485 to USB adapter and run semp-rtu.py. This way you can rule out the inverter being the problem.

Immediately the proxy is running again.

Immediately the proxy is running again.

Ok, great. Then we can ignore the inverter.
To rule out a networking issue, are you blocking traffic with a firewall on your Pi? If so, be sure to turn it off.
I am also curious what the test script shows when you choose frame as protocol.

Do you have a clue what we need to set here then:

I didn't install a firewall myself on the pin so I don't think so

Run sudo iptables -L to check.

Do you have a clue what we need to set here then, I can switch tag to off too.

No clue, might only be useful if the data is known beforehand. Try turning it off?

pi@raspberrypi4:~$ sudo iptables -L����������������������������������������������������������Chain INPUT (policy ACCEPT)������������������������������������������������������������������target     prot opt source               destination         �����������������������������������������������������������������������������������������������������������������������������Chain FORWARD (policy ACCEPT)����������������������������������������������������������������target     prot opt source               destination         �����������������������������������������������������������������������������������������������������������������������������Chain OUTPUT (policy ACCEPT)�����������������������������������������������������������������target     prot opt source               destination

Dont pay attention to the black question marks...... those are spaces

Firewall is fine, not blocking anything. I guess it's just a matter of finding the correct configuration on your gateway.
You should be able to see things are working right away using the test script. It may also be useful to use the acompanying desktop utility to check whether the gateway is actually receiving messages. Remember that the inverter might need different serial settings depending on whether it is set to non-solaredge logger (inverter as server) or modbus multi-devices (inverter as client). In my experience it will use baud 9600 when it is a modbus client.

its now showing this:
2020-12-16 08:50:21 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:21 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:21 DEBUG: Frame check failed, ignoring!!
2020-12-16 08:50:23 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:23 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:23 DEBUG: Frame check failed, ignoring!!
2020-12-16 08:50:25 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:25 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:25 DEBUG: Frame check failed, ignoring!!

its now showing this:
2020-12-16 08:50:21 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:21 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:21 DEBUG: Frame check failed, ignoring!!
2020-12-16 08:50:23 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:23 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:23 DEBUG: Frame check failed, ignoring!!
2020-12-16 08:50:25 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:25 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-16 08:50:25 DEBUG: Frame check failed, ignoring!!

Excellent, this is progress. I see similar messages. Have you tried the original semp-tcp.py?

Hi Niels,
i checked it, following is coming by.
I checked the solaredge app, and i have seen that there where values above the house icon and the grid was showing 0
0,39 kw production, 0.39 kw load(house) 0.0kw export.
 This means the inverter messages are corrupt or invalid and not being processed. The gateway is forwarding something, but not quite correctly. Are there any options set that add tags or something similar to the messages?
You're seeing 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab, where 0x2 is the address, 0x3 is read holding registers, and so on. So it looks like a valid message, but there is stuff added to the end which doesn't make sense.
You will need to switch back to the test script and fiddle with the gateway options until you stop getting the error and requests more like this:
2020-12-14 17:00:28 DEBUG: Getting Frame - 0x3 0x3 0xf2 0x0 0x6
2020-12-14 17:00:28 DEBUG: Factory Request[ReadHoldingRegistersRequest: 3]
2020-12-14 17:00:28 DEBUG: Frame advanced, resetting header!!

So seeing data with 0x2 0x3 is at least good, now we need to fine tune like you said.....
Mm wonder what to check. I hope I have some time tonight!

I tried many different settings in the serial port part of the gateway but the only thing putting the data stream back to 0x0 data is the baud rate. changing data bits and stopbits are still giving 0x02 data etc.

The serial settings of the gateway should be baud 9600, 8 data bits, 1 stop bit, no parity. These are the settings of the device we want to emulate, the wattnode meter, which is what the solaredge inverter expects and will use to connect.
The documentation of this rs485-wifi gateway is rubbish -- there literally isn't a complete or correct english sentence in any of the documents. Not much I can recommend other than that you experiment with the various protocol and buffer settings.
The test script will show you when you've found the right combination.

would this make any sense? found it somewhere in a github post
from pymodbus.transaction import ModbusRtuFramer
client = ModbusClient('<IP_ADDRESS>', port=8899, framer=ModbusRtuFramer)

Give it a try, but as far as I know the framer property only works on serial clients and servers. You can try adding it to the StartTcpServer function in the test script.

Hi Niels,
So i didnt had a lot of time the last days but i want to inform you about the following:
If the RS485-Ethernet dongle is setup as server i can run the script to retrieve data from the inverter which in my opinion means the inverter is working and connected correctly. The issue is, as i can figure it out, the part when the inverter needs to be a client.
Is it possible to run the script below as "server" and then request the information again with the dongle as client?
python3 example-tcp-inverteronly.py <IP_ADDRESS> 8899
Inverter(<IP_ADDRESS>:8899, connectionType.TCP: timeout=1, retries=3, unit=0x1):
Registers:
Manufacturer: SolarEdge
Model: SE3500
Type: Single Phase Inverter
Version: 0003.2221
Serial: 7316DD53
Status: Sleeping
Temperature: 20.77°C
Current: 0.00A
Voltage: 230.20V
Frequency: 50.00Hz
Power: 0.00W
Power (Apparent): 0.00VA
Power (Reactive): 0.00VAr
Power Factor: 100.00%
Total Energy: 7815914Wh
DC Current: 0.00A
DC Voltage: 0.00V
DC Power: 0.00W
With script:
#!/usr/bin/env python3
import argparse
import json
import time
import solaredge_modbus
if name == "main":
argparser = argparse.ArgumentParser()
argparser.add_argument("host", type=str, help="ModbusTCP address")
argparser.add_argument("port", type=int, help="ModbusTCP port")
argparser.add_argument("--timeout", type=int, default=1, help="Connection timeout")
argparser.add_argument("--unit", type=int, default=1, help="Modbus unit")
argparser.add_argument("--json", action="store_true", default=False, help="Output as JSON")
args = argparser.parse_args()
inverter = solaredge_modbus.Inverter(
    host=args.host,
    port=args.port,
    timeout=args.timeout,
    unit=args.unit
)

values = {}
values = inverter.read_all()
meters = inverter.meters()
batteries = inverter.batteries()
values["meters"] = {}
values["batteries"] = {}

for meter, params in meters.items():
    meter_values = params.read_all()
    values["meters"][meter] = meter_values

if args.json:
    print(json.dumps(values, indent=4))
else:
    print(f"{inverter}:")
    print("\nRegisters:")

    print(f"\tManufacturer: {values['c_manufacturer']}")
    print(f"\tModel: {values['c_model']}")
    print(f"\tType: {solaredge_modbus.C_SUNSPEC_DID_MAP[str(values['c_sunspec_did'])]}")
    print(f"\tVersion: {values['c_version']}")
    print(f"\tSerial: {values['c_serialnumber']}")
    print(f"\tStatus: {solaredge_modbus.INVERTER_STATUS_MAP[values['status']]}")
    print(f"\tTemperature: {(values['temperature'] * (10 ** values['temperature_scale'])):.2f}{inverter.registers['temperature'][6]}")

    print(f"\tCurrent: {(values['current'] * (10 ** values['temperature_scale'])):.2f}{inverter.registers['current'][6]}")

    if values['c_sunspec_did'] is solaredge_modbus.sunspecDID.THREE_PHASE_INVERTER:
        print(f"\tPhase 1 Current: {(values['p1_current'] * (10 ** values['current_scale'])):.2f}{inverter.registers['p1_current'][6]}")
        print(f"\tPhase 2 Current: {(values['p2_current'] * (10 ** values['current_scale'])):.2f}{inverter.registers['p2_current'][6]}")
        print(f"\tPhase 3 Current: {(values['p3_current'] * (10 ** values['current_scale'])):.2f}{inverter.registers['p3_current'][6]}")
        print(f"\tPhase 1 voltage: {(values['p1_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p1_voltage'][6]}")
        print(f"\tPhase 2 voltage: {(values['p2_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p2_voltage'][6]}")
        print(f"\tPhase 3 voltage: {(values['p3_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p3_voltage'][6]}")
        print(f"\tPhase 1-N voltage: {(values['p1n_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p1n_voltage'][6]}")
        print(f"\tPhase 2-N voltage: {(values['p2n_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p2n_voltage'][6]}")
        print(f"\tPhase 3-N voltage: {(values['p3n_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p3n_voltage'][6]}")
    else:
        print(f"\tVoltage: {(values['p1_voltage'] * (10 ** values['voltage_scale'])):.2f}{inverter.registers['p1_voltage'][6]}")

    print(f"\tFrequency: {(values['frequency'] * (10 ** values['frequency_scale'])):.2f}{inverter.registers['frequency'][6]}")
    print(f"\tPower: {(values['power_ac'] * (10 ** values['power_ac_scale'])):.2f}{inverter.registers['power_ac'][6]}")
    print(f"\tPower (Apparent): {(values['power_apparent'] * (10 ** values['power_apparent_scale'])):.2f}{inverter.registers['power_apparent'][6]}")
    print(f"\tPower (Reactive): {(values['power_reactive'] * (10 ** values['power_reactive_scale'])):.2f}{inverter.registers['power_reactive'][6]}")
    print(f"\tPower Factor: {(values['power_factor'] * (10 ** values['power_factor_scale'])):.2f}{inverter.registers['power_factor'][6]}")
    print(f"\tTotal Energy: {(values['energy_total'] * (10 ** values['energy_total_scale']))}{inverter.registers['energy_total'][6]}")

    print(f"\tDC Current: {(values['current_dc'] * (10 ** values['current_dc_scale'])):.2f}{inverter.registers['current_dc'][6]}")
    print(f"\tDC Voltage: {(values['voltage_dc'] * (10 ** values['voltage_dc_scale'])):.2f}{inverter.registers['voltage_dc'][6]}")
    print(f"\tDC Power: {(values['power_dc'] * (10 ** values['power_dc_scale'])):.2f}{inverter.registers['power_dc'][6]}")

Is it possible to run the script below as "server" and then request the information again with the dongle as client?

I'm not quite sure what you're suggesting. When the inverter RS485 port is configured for "Non-Solaredge Logger" it acts as a server, which means a client can get the values like those in the script you posted. When you set the inverter RS485 port to "Modbus multi-device" the inverter will try to connect to other modbus devices as a client. If you want the inverter to get information from a meter it needs to be in "Modbus multi-device" configuration, and will always be a client.

Okay that's clear, in multi device mode it's a client not a server.
Mmm than I really need to find out WHY it's not working. I need to see if I can make some kind of test setup which will reply what I send to it.
Maybe any tips on that?

Okay that's clear, in multi device mode it's a client not a server.
Mmm than I really need to find out WHY it's not working. I need to see if I can make some kind of test setup which will reply what I send to it.
Maybe any tips on that?

Not much I can do to help. Either I need one of those units to experiment with or someone else needs to share their experiences with a different type of gateway.

Would it have samenhing to do with the following?
1.9.1. UART Free-Frame
The converter support UART Free-Frame function, which will check the intervals between any two bytes when receiving UART data. If the interval time exceed the set value, it will register it as the end of one frame, or it will receive data utill the internal buffer is full, (Default: 512, the largest 1400 bytes), then transfer to Socket Channel.
The default UART Free-Frame interval time is 50ms, it will be packaged into another frame if received UART data interval time is greater than 50ms. The user can also set this interval time to minimum 10ms through Cli command and webpage.
If interval time is set to 10ms and customer MCU can’t send next byte within 10ms, the serial data will be break into two frame.

Would it have samenhing to do with the following?

Maybe? Is it on? Have you tried turning it off?

Buffer and gap size is always on. Any idea what those should be? Maybe that causes the unwanted data?

You could try setting the buffer size in smaller or larger halves (e.g., 256, 128, or 1024, 2048), and the gap time in 10s of ms (e.g., 40, 30, 60, 70). Run the test script and see if you get better results.

Will play with that and otherwise I will connect my other SDM unit to the gateway and take the usb-rs485 dongle of that for the inverter. That worked like a charm 😛

Maybe a bit much to ask but could you hook your rs485 converter that you use for the SDM to the meter to check which settings you need on yours?

Maybe a bit much to ask but could you hook your rs485 converter that you use for the SDM to the meter to check which settings you need on yours?

It's not readily accessible. I may get another tgw715 for the inverter to try it out, but it's not on my to-do for a while.

I understand, for you it's working okay and I just want it to connect to tcp but that's just for ease of mind that if the rpi reboots I don't have to check which USB port is what again because the usb converter doesn't have any unique info to do persistent names to it.
Anyhow, can you once more tell me how the data should look like when using the test script? Then I can try and start to play around more with those buffersize settings etc.

Anyhow, can you once more tell me how the data should look like when using the test script? Then I can try and start to play around more with those buffersize settings etc.

2020-12-14 17:00:28 DEBUG: Getting Frame - 0x3 0x3 0xf2 0x0 0x6
2020-12-14 17:00:28 DEBUG: Factory Request[ReadHoldingRegistersRequest: 3]
2020-12-14 17:00:28 DEBUG: Frame advanced, resetting header!!
2020-12-14 17:00:28 DEBUG: validate: fc-[3] address-1011: count-6
2020-12-14 17:00:28 DEBUG: getValues fc-[3] address-1011: count-6
2020-12-14 17:00:28 DEBUG: send: [ReadRegisterResponse (6)]- b'02030c999a439000000000000000008deb'

Well I tried a lot with the settings but it didn't work as hoped..
I dont think I will spend more money on another rs485 gateway and hook it up to usb Instead.
Thanks for the help!

It was worth a shot. Hopefully someone else with another gateway will come along and help out. Otherwise, when I get another gateway I can do more experimenting on my own. Thanks for trying!

Quick update.
I ordered a second tGW-715 recently to try this myself, and I have it running sucessfully.

Inverter is configured with RS485-1 set to Modbus (Multi-Device)
tGW-715 has its serial port set to 9600,8,N,1
tGW-715 is configured to talk Modbus RTU, and connect as TCP client to another PC
The PC is running semp-tcp.py
The PC has a SDM120 (Modbus address 2) configured in semp-tcp.conf, and talks to it over Modbus TCP

My semp-tcp.conf:
[server]
address = <IP_ADDRESS>
port = 1502
log_level = DEBUG
meters = sdm120

[sdm120]
src_address=1
dst_address=2
type=sdm120
host=<IP_ADDRESS>
port=502
ct_current=50

After setting everything up I see the following requests from the inverter:
2020-12-28 16:04:03 DEBUG: Handling data: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x3 0xf2 0x0 0x6
2020-12-28 16:04:03 DEBUG: Processing: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x3 0xf2 0x0 0x6
2020-12-28 16:04:03 DEBUG: Factory Request[ReadHoldingRegistersRequest: 3]
2020-12-28 16:04:03 DEBUG: validate: fc-[3] address-1011: count-6
2020-12-28 16:04:03 DEBUG: getValues fc-[3] address-1011: count-6
2020-12-28 16:04:03 DEBUG: send: [ReadRegisterResponse (6)]- b'02030000000f02030c6666438e0000000000000000'

When running the test.py script, you will see this:
2020-12-28 16:12:23 DEBUG: Handling data: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x6 0x40 0x0 0x17
2020-12-28 16:12:23 DEBUG: Processing: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x6 0x40 0x0 0x17
2020-12-28 16:12:23 DEBUG: Not a valid unit id - 2, ignoring!!

To clear things up a bit: in this configuration your PC running semp-tcp.py is a Modbus slave with regard to the inverter, and TCP server to the tGW-715. The tGW-715 Modbus gateway is transparent in the Modbus context but will connect as TCP client to your PC. The inverter is a Modbus master, and will talk Modbus RTU to the tGW-715. This is distinctly different from the Sunspec (Non-SE Logger) mode, where the inverter is a Modbus slave.

inverter (rtu master) <--> ethernet module (rtu slave? and tcp client?) <--> semp.tcp.py (tcp server)
Your are running transparent transmission mode so no real Modbus RTU-TCP conversion is running on the gateway right?
Is that the same as your are describing?
How do the modbus settings and tcp client settings look like in your device?

Almost. The gateway is not a Modbus master or slave, it is not involved other than translating Modbus RTU to TCP and vice versa. The gateway is however a TCP client, connecting to the TCP server provided by semp-tcp.py, which is a Modbus slave.
Some settings from the gateway:
9600,8N1
No flow control
Modbus RTU
Slave timeout 300ms
Character timeout 4 bytes
No tagging or other complex functionality is enabled.

so it wraps the modbus RTU to TCP but isnt running a ModbusRTU to ModbusTCP conversion?
A: MODBUS over TCP means a MODBUS RTU packet wrapped in a TCP packet.
B: MODBUS TCP means a MODBUS TCP packet wrapped in a TCP packet.
A and B are not compatible. They transfer like data but have different structures.

so it wraps the modbus RTU to TCP but isnt running a ModbusRTU to ModbusTCP conversion?

The gateway translates between Modbus RTU frames and Modbus TCP packets. The packets arriving at my Modbus TCP server do so without any RTU checksums, and are therefore Modbus TCP, not Modbus RTU over TCP.

So it's a real translator from
Modbus RTU to ModbusTCP. Good to know. Now we(or i) can try and see if we can make the server read modbusrtu over tcp..
Because I think my gateway(again not your problem) is not working as a real modbusrtu to ModbusTCP translator.
Although when it's running as server for the non-Solaredge logger protocolits working fine, that's what I don't understand.

Because I think my gateway(again not your problem) is not working as a real modbusrtu to ModbusTCP translator.
Although when it's running as server for the non-Solaredge logger protocolits working fine, that's what I don't understand.

I believe it is, otherwise you wouldn't be able to use solaredge_modbus to talk to your inverter via the EW-11.
All of your screenshots are from the web interface. Have you tried using the application they recommend in their documentation, IOTService? Download it here.
Download the Elfin user guide, and have a look at page 30. There you see how the EW-11 needs to be configured:
System
Should be set to your preferences. Not sure what other options Network Mode offers, but that shouldn't matter.
UART
Baudrate: 9600
Data bits: 8
Stop bits: 1
Parity: None
Flow control: can't see the options, but needs to be off, none, or similar
Buffer size: unsure, but leave as is
LANN and WiFi
Set to your preference.
SOCKET
Delete any existing socket, and start with a new one, name it whatever you want.
Protocol: TCP-CLIENT
Server addr: the IP of the machine running semp-tcp.py
Server port: the port configured in semp-tcp.conf
Local port: not important, 8899 is fine
Keep alive: 60 is fine
Time out: unsure, 0 might be fine
Rout: uart
Buffer size: same as uart buffer size
Security: disable
Connect mode: always
Heartbeat: disable
Register mode: unsure what the other options are, try link?
Confirm the settings. You should then see a Device Status page which shows the connection status and RX/TX details.
Try this while running semp-tcp.py, not the test script, and let me know whether you see messages like this:
2020-12-28 16:04:03 DEBUG: Handling data: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x3 0xf2 0x0 0x6
2020-12-28 16:04:03 DEBUG: Processing: 0x2 0x3 0x0 0x0 0x0 0x6 0x2 0x3 0x3 0xf2 0x0 0x6
2020-12-28 16:04:03 DEBUG: Factory Request[ReadHoldingRegistersRequest: 3]
2020-12-28 16:04:03 DEBUG: validate: fc-[3] address-1011: count-6
2020-12-28 16:04:03 DEBUG: getValues fc-[3] address-1011: count-6
2020-12-28 16:04:03 DEBUG: send: [ReadRegisterResponse (6)]- b'02030000000f02030c6666438e0000000000000000

tried again playing around with the settings and getting this now: (BTW i cant get this data into the embedded field you are posting. Code doesnt seem to work for that!?)
2020-12-29 20:44:54 DEBUG: Running transaction 14
2020-12-29 20:44:54 DEBUG: SEND: 0x0 0xe 0x0 0x0 0x0 0x6 0x1 0x4 0x0 0x50 0x0 0x1c
2020-12-29 20:44:54 DEBUG: New Transaction state 'SENDING'
2020-12-29 20:44:54 DEBUG: Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
2020-12-29 20:44:54 DEBUG: Handling data: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-29 20:44:54 DEBUG: Processing: 0x2 0x3 0x6 0x40 0x0 0x17 0x4 0xab
2020-12-29 20:44:54 DEBUG: Frame check failed, ignoring!!
2020-12-29 20:44:54 DEBUG: Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
2020-12-29 20:44:54 DEBUG: RECV: 0x0 0xe 0x0 0x0 0x0 0x3b 0x1 0x4 0x38 0x44 0x51 0x28 0xbb 0x45 0x83 0x86                                                                0xeb 0x44 0xb6 0x72 0x24 0x45 0x73 0x93 0xc2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0                                                                0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x44 0xb6 0x9e 0x3d 0x45 0x73 0xd8 0x2b 0x40 0x4a 0x1c 0xd3 0x41 0x1                                                               c 0xbf 0x47
2020-12-29 20:44:54 DEBUG: Processing: 0x0 0xe 0x0 0x0 0x0 0x3b 0x1 0x4 0x38 0x44 0x51 0x28 0xbb 0x45 0x8                                                               3 0x86 0xeb 0x44 0xb6 0x72 0x24 0x45 0x73 0x93 0xc2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0                                                               x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x44 0xb6 0x9e 0x3d 0x45 0x73 0xd8 0x2b 0x40 0x4a 0x1c 0xd3 0x                                                               41 0x1c 0xbf 0x47
2020-12-29 20:44:54 DEBUG: Factory Response[ReadInputRegistersResponse: 4]
2020-12-29 20:44:54 DEBUG: Adding transaction 14
2020-12-29 20:44:54 DEBUG: Getting transaction 14
2020-12-29 20:44:54 DEBUG: Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
2020-12-29 20:44:54 DEBUG: [17489, 10427, 17795, 34539, 17590, 29220, 17779, 37826, 0, 0, 0, 0, 0, 0, 0,                                                                0, 0, 0, 0, 0, 17590, 40509, 17779, 55339, 16458, 7379, 16668, 48967]
2020-12-29 20:44:54 DEBUG: [b'DQ', b'(\xbb']
2020-12-29 20:44:54 DEBUG: [b'E\x83', b'\x86\xeb']
2020-12-29 20:44:54 DEBUG: [b'D\xb6', b'r$']
2020-12-29 20:44:54 DEBUG: [b'Es', b'\x93\xc2']
2020-12-29 20:44:54 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:44:54 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:44:54 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:44:54 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:44:54 DEBUG: [b'D\xb6', b'\x9e=']
2020-12-29 20:44:54 DEBUG: [b'Es', b'\xd8+']
2020-12-29 20:44:54 DEBUG: [b'@J', b'\x1c\xd3']
2020-12-29 20:44:54 DEBUG: [b'A\x1c', b'\xbfG']
2020-12-29 20:44:54 DEBUG: Current transaction state - TRANSACTION_COMPLETE

i increased the buffer size to 1024 in the socket and uart settings:
2020-12-29 20:57:50 DEBUG: Running transaction 6
2020-12-29 20:57:50 DEBUG: SEND: 0x0 0x6 0x0 0x0 0x0 0x6 0x1 0x4 0x0 0x50 0x0 0x1c
2020-12-29 20:57:50 DEBUG: New Transaction state 'SENDING'
2020-12-29 20:57:50 DEBUG: Changing transaction state from 'SENDING' to 'WAITING FOR REPLY'
2020-12-29 20:57:50 DEBUG: Changing transaction state from 'WAITING FOR REPLY' to 'PROCESSING REPLY'
2020-12-29 20:57:50 DEBUG: RECV: 0x0 0x6 0x0 0x0 0x0 0x3b 0x1 0x4 0x38 0x44 0x51 0x49 0x9e 0x45 0x83 0x99 0x2c 0x44 0xd4 0x10 0x47 0x45 0x73 0x93 0xc2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x44 0xd4 0x40 0x50 0x45 0x73 0xd8 0x2b 0x40 0x7f 0xc6 0x6 0x41 0x1c 0xbf 0x47
2020-12-29 20:57:50 DEBUG: Processing: 0x0 0x6 0x0 0x0 0x0 0x3b 0x1 0x4 0x38 0x44 0x51 0x49 0x9e 0x45 0x83 0x99 0x2c 0x44 0xd4 0x10 0x47 0x45 0x73 0x93 0xc2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x44 0xd4 0x40 0x50 0x45 0x73 0xd8 0x2b 0x40 0x7f 0xc6 0x6 0x41 0x1c 0xbf 0x47
2020-12-29 20:57:50 DEBUG: Factory Response[ReadInputRegistersResponse: 4]
2020-12-29 20:57:50 DEBUG: Adding transaction 6
2020-12-29 20:57:50 DEBUG: Getting transaction 6
2020-12-29 20:57:50 DEBUG: Changing transaction state from 'PROCESSING REPLY' to 'TRANSACTION_COMPLETE'
2020-12-29 20:57:50 DEBUG: [17489, 18846, 17795, 39212, 17620, 4167, 17779, 37826, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 17620, 16464, 17779, 55339, 16511, 50694, 16668, 48967]
2020-12-29 20:57:50 DEBUG: [b'DQ', b'I\x9e']
2020-12-29 20:57:50 DEBUG: [b'E\x83', b'\x99,']
2020-12-29 20:57:50 DEBUG: [b'D\xd4', b'\x10G']
2020-12-29 20:57:50 DEBUG: [b'Es', b'\x93\xc2']
2020-12-29 20:57:50 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:57:50 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:57:50 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:57:50 DEBUG: [b'\x00\x00', b'\x00\x00']
2020-12-29 20:57:50 DEBUG: [b'D\xd4', b'@P']
2020-12-29 20:57:50 DEBUG: [b'Es', b'\xd8+']
2020-12-29 20:57:50 DEBUG: [b'@\x7f', b'\xc6\x06']
2020-12-29 20:57:50 DEBUG: [b'A\x1c', b'\xbfG']
2020-12-29 20:57:50 DEBUG: Current transaction state - TRANSACTION_COMPLETE

i got it running now!!!
in the import section of the TCP script already you had:
from pymodbus.transaction import ModbusRtuFramer
I added in the StartTCPServer part: framer=ModbusRtuFramer, see below
    StartTcpServer(
        server_ctx,
        identity=identity,
        framer=ModbusRtuFramer,
        address=(
            confparser["server"].get("address", fallback=default_config["server"]["address"]),
            confparser["server"].getint("port", fallback=default_config["server"]["port"])
        )

i got it running now!!!
in the import section of the TCP script already you had:
from pymodbus.transaction import ModbusRtuFramer
I added in the StartTCPServer part: framer=ModbusRtuFramer, see below

Great news! I will add the functionality as an option in the configuration file.

Yeah! Thanks for the help, sorry that you had to buy another not so cheap unit to confirm your scripting!
Btw: in semp-tcp.conf instead of Port you have still baud in the server part.
Here are the final settings I have configured in the Elfin EE-11. You could put that info in the readme to!

Thanks, I've made the changes.
I see you've set protocol to None. Does setting it to Modbus cause the frame errors to return?

Hi, just checked.
I changed the setting to modbus and the data to the Solaredge stopped.

Ok, weird. Closing this issue now that your situation is working.

Hello gents, (hallo nmakel),
Based on this posting I was able to connect my SE-5000H via EW11A to my home automation (OpenHab) running sem-tcp.py. It took me a long while and setting Protocol to 'None' instead of ModBus was the trick (weird IMNHO).
Need to figure out if my data from MQTT (taken from dsmr binding and converted) is correct.
Thus far while it's dark outside and no solar energy is generated.
{
"import_energy_active": 617.000,
"export_energy_active": 0,
"energy_active": 617.000,
"l1_power_active": 361.000,
"l2_power_active": 233.000,
"l3_power_active": 24.000,
"power_active": 618.000,
"l1n_voltage": 230,
"l2n_voltage": 231.7,
"l3n_voltage": 231.1,
"voltage_ln": 231.1,
"l1_current": 1,
"l2_current": 1,
"l3_current": 0,
"frequency": 50
}

I saw the L1, L2, L3 in the SetApp and arrow/values in SolarEdge app shortly and disappearing/showing now.
Maybe I need to publish faster then every 30 seconds into MQTT.
Thanks both for spending the time!!
Ferry

Can you share your MQTT info? How did you get this working?

Hi Maikel-K,
Retrieving the smartmeter values via P1 port on OpenHab with DSMR binding.
Converting and calculating values before publishing on MQTT in JSON format.
{
"import_energy_active": 0.453,
"export_energy_active": 0,
"energy_active": 0.453,
"l1_power_active": 208.000,
"l2_power_active": 229.000,
"l3_power_active": 16.000,
"power_active": 0.45300000,
"l1n_voltage": 233.2,
"l2n_voltage": 233.9,
"l3n_voltage": 234.7,
"voltage_ln": 234.7,
"l1_current": 1,
"l2_current": 1,
"l3_current": 0,
"frequency": 50
}

Not sure yet if those values are correct because my consumption (Verbruik) of 3.34 MHh was very high. Had power_active in order of 453 and divided now by 1000.
Still testing to find out what the actual input values should be (W or kWh).


Can't get the rojo serve to work

Is there any .rbxl file I could use?

D:

This repo is meant to be a learning utility for anyone interested in rbxts development. Although I agree a place file would make the game run without as much tinkering, this specific repository isn't meant to be 100% complete. It is something I'm open to in the future though once I am more experienced and can write more production-ready work.


Newrelic.properties was not found! Mapping file for variant [devRelease] not uploaded.

I migrated my gradle from Gradle 7.3 to 8.0.
I encountered this problem with new relic.

Execution failed for task ':app:minifyReleaseWithR8'.

I found this Git Issue that fixed my problem. I added a proguard-rules.pro file in my app folder and also indicated it on my app level build.gradle
proguard-rules.pro
-dontwarn com.newrelic.agent.android.ndk.AgentNDK$Builder
-dontwarn com.newrelic.agent.android.ndk.AgentNDK
-dontwarn com.newrelic.agent.android.ndk.AgentNDKListener

-keep class org.apache.http.** { *; }
-dontwarn org.apache.http.**

-keep class com.squareup.okhttp.* { *; }
-dontwarn com.squareup.okhttp.**
-dontwarn okio.**

-keep class okhttp3.** { *; }
-dontwarn okhttp3.**
-dontwarn okio.**

-keep class retrofit2.** { *; }
-dontwarn retrofit2.**

After I applied this fix I still encounter the newrelic.properties error.

@ItsNotPat did you add newrelic.properties file for your project?

hello @ndesai-newrelic , do you have an example for newrelic.properties file?
I can not find it in your example app on github. Thanks a lot

please follow this: https://docs.newrelic.com/docs/mobile-monitoring/new-relic-mobile-android/install-configure/android-agent-crash-reporting/#mapping-file

I did not add newrelic.properties to my flutter project. Ill try to follow the instructions above.


Don't run the pending blueprints specs

Marking tests pending can be useful if iterating rapidly, but leaving
this in master can lead to some confusion over what is failing because
the output from the reporter looks almost identical to something that
failed. Marking these tests using xit instead will still leave them
pending but won't run the code contained in them:
bundle exec rspec --format progress /home/tim/src/manageiq/spec/requests/api/blueprints_spec.rb

Randomized with seed 39352
..*.........*........

Pending: (Failures listed here are expected and do not affect your suite's status)

  1) Blueprints API POST /api/blueprints can publish multiple blueprints
     # Temporarily skipped with xit
     # ./spec/requests/api/blueprints_spec.rb:202

  2) Blueprints API POST /api/blueprints/:id publishes a single blueprint
     # Temporarily skipped with xit
     # ./spec/requests/api/blueprints_spec.rb:249

Finished in 5.12 seconds (files took 7.5 seconds to load)
21 examples, 0 failures, 2 pending

@miq-bot add-label api, test
@miq-bot assign @abellotti

This LGTM!!


Add optional secret to webhook form

As per Pullrequest https://github.com/TryGhost/Ghost/pull/13980
there needs to be a possibility to add the secret to a webhook.
This PR adds another text input field to the webhook form.

I added a note to the Ghost PR to explain why this hadn't been merged (which I really thought I had done a long time ago) - the Ghost PR needs updating to only output the header if a secret is set, so that secret stays optional.
Maybe we can fixup both PRs and get them in, as this would be nice to have in.

Hi @sainthkh , I'm really sorry that we didn't get around to reviewing this PR in a reasonable timeframe. At the moment there are some changes needed to the server PR and this one needs updating as it doesn't pass tests.
We're in the process of cleaning up all the PRs on this repo, before moving the admin to be part of the Ghost monorepo. Once this is done, getting this over the line if you're still interested will be much easier, as you'll be able to update your Ghost PR with both sets of changes, and more easily test that it all works together.
If you're still interested in making these changes, please update the PR on the Ghost monorepo once admin is merged in (should be in the next few days). We'll then take a look at the PR right away.


Multi instance refactor start

What?
Refactor the start api to remove the need to pass the instance id as we have it now already
...
Why?
We need this to be able to support multi instances of beams in a single app
...

CC @pusher/mobile

Codecov Report

:exclamation: No coverage uploaded for pull request base (feature-multi-instance@47ea61b). Click here to learn what that means.
The diff coverage is 96.22%.

@@                    Coverage Diff                    @@
##             feature-multi-instance     #127   +/-   ##
=========================================================
  Coverage                          ?   89.28%
=========================================================
  Files                             ?       54
  Lines                             ?     2903
  Branches                          ?        0
=========================================================
  Hits                              ?     2592
  Misses                            ?      311
  Partials                          ?        0

Impacted Files
Coverage Δ

...ns/PushNotifications/PushNotificationsStatic.swift
20.58% <100%> (ø)

Tests/IntegrationTests/ClearAllStateTest.swift
97.29% <100%> (ø)

Tests/IntegrationTests/DeviceInterestsTest.swift
95.55% <100%> (ø)

Tests/IntegrationTests/SetUserIdTest.swift
98.7% <100%> (ø)

Tests/IntegrationTests/ApplicationStartTests.swift
96.77% <100%> (ø)

Tests/Internal/ServerSyncProcessHandlerTests.swift
98.05% <100%> (ø)

Sources/EventTypeHandler.swift
68.18% <100%> (ø)

Sources/PushNotifications.swift
83.33% <100%> (ø)

Tests/IntegrationTests/StopTests.swift
98.24% <100%> (ø)

Sources/Internal/ServerSyncProcessHandler.swift
83.08% <84.61%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47ea61b...0cbe21f. Read the comment docs.


(bug) Categories incorrectly state completion %

This image says category completion is 1%, not 100%

Fixed with 65aaa68


SIGSEGV while running go on ppc64le

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
master
root@localhost:~/go# git rev-parse HEAD
eca45997dfd6cd14a59fbdea2385f6648a0dc786
root@localhost:~/go#

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
ppc64le
What did you do?
export PATH=/usr/local/go/bin/:$PATH
export GOROOT_BOOTSTRAP=go env GOROOT
root@localhost:~/go/bin# go version
go version go1.8.3 linux/ppc64le
cloned the latest code here:
root@localhost:~/go/src# pwd
/root/go/src
run make.bash
root@localhost:~/go/src# ./make.bash
Use the build go binary:
root@localhost:~/go/bin# pwd
/root/go/bin
root@localhost:~/go/bin# ./go version
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xc88320 pc=0x112c8]

runtime stack:
invalid spdelta runtime/cgo(.text).sigfillset 0x112c0 0x112c8 0x0 -1
runtime.throw(0x44825a, 0x2a)
	/root/go/src/runtime/panic.go:606 +0x68
runtime.sigpanic()
	/root/go/src/runtime/signal_unix.go:360 +0x264
invalid spdelta runtime/cgo(.text).sigfillset 0x112c0 0x112c8 0x0 -1

goroutine 1 [running]:
runtime.systemstack_switch()
	/root/go/src/runtime/asm_ppc64x.s:209 +0x10 fp=0xc420036758 sp=0xc420036738 pc=0x64e40
runtime.main()
	/root/go/src/runtime/proc.go:128 +0x60 fp=0xc4200367c0 sp=0xc420036758 pc=0x3cf80
runtime.goexit()
	/root/go/src/runtime/asm_ppc64x.s:1353 +0x4 fp=0xc4200367c0 sp=0xc4200367c0 pc=0x67464
root@localhost:~/go/bin#

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
What did you expect to see?
Instead, I should see a golang version
What did you see instead?
I see a SIGSEGV

/cc @laboger

@ceseo

@mkumatag which revision of Go are you using? Can you try rolling back to 101fbc2 (git checkout 101fbc2) and retrying? It looks like the ppc64le builder has been failing since 8e5ac83 (cmd/go: stop linking cgo objects together with ld -r).

@mundaym I'm using latest one
root@localhost:~/go# git rev-parse HEAD
eca45997dfd6cd14a59fbdea2385f6648a0dc786
root@localhost:~/go#

I rolled back to 101fbc2 and things are fine,
root@localhost:~/go/bin# ./go version
go version devel +101fbc2 Wed Sep 20 20:27:13 2017 +0000 linux/ppc64le
root@localhost:~/go/bin#

@mundaym @ianlancetaylor not really sure what is wrong with the latest version!

Yikes I haven't been watching the build board lately and I see we have been randomly failing in runtime since e7e4a4ffa and this latest failure of SEGV was introduced with @ianlancetaylor 's change 8e5ac83.
segv happens here:
0x112c0 <runtime/cgo(.text).sigfillset>:	std     r2,24(r1)
0x112c4 <runtime/cgo(.text).sigfillset+4>:	addis   r12,r2,100
=> 0x112c8 <runtime/cgo(.text).sigfillset+8>:	ld      r12,192(r12)
0x112cc <runtime/cgo(.text).sigfillset+12>:	mtctr   r12
0x112d0 <runtime/cgo(.text).sigfillset+16>:	bctr
In golang prior to the change to omit the ld -r where it works, this code looks like this:
=> 0x112c0 <runtime/cgo(.text).sigfillset>:	std     r2,24(r1)
0x112c4 <runtime/cgo(.text).sigfillset+4>:	addis   r12,r2,-1
0x112c8 <runtime/cgo(.text).sigfillset+8>:	ld      r12,32368(r12)
0x112cc <runtime/cgo(.text).sigfillset+12>:	mtctr   r12
0x112d0 <runtime/cgo(.text).sigfillset+16>:	bctr
So my theory is that the TOCs from the individual .o files should be getting merged into a single TOC at some point, but if the external linker isn't involved in the final link, how's it going to happen except by the ld -r? My understanding is that the go tool is linked with the internal linker and not the external linker, so in that case the ld -r is still useful for ppc64le.
This brings up a different concern however, why this wasn't caught in the buildbots?

You mean the trybots? They don't run on ppc64x, they only check that the compile runs.

FYI...  this doesn't affect ppc64 because 'go' is built statically there.

@mundaym This breaks the 'go' tool so I was somewhat thinking that it should fail during the build.  But I see now that go_bootstrap is built as static but the final go tool is built as a dynamic binary containing cgo using the internal linker which is the case that is broken by the removal of ld -r.

Any updates? builds are still failing with the latest code. @ianlancetaylor any comments?

@ianlancetaylor I don't know what your thoughts are on this.  This regression makes the go tool unusable on the golang build so we would like to get this fixed.
I can create a fix to back out your change for ppc64le, i.e., continue to use the ld -r on ppc64le but not on other platforms.
I believe the other option is to fix the internal linker to correctly handle merging of the TOCs, but that might take me a while because I don't know the internal linker well enough.

What happens if we simply add sys.PPC64 to this list in cmd/link/internal/ld/config.go?
	if iscgo && SysArch.InFamily(sys.ARM64, sys.MIPS64, sys.MIPS) {
		return true, objabi.GOARCH + " does not support internal cgo"
	}

That seems to me like the right temporary fix unless and until somebody fixes the internal linker.  The downside is that we won't generate fully statically linked binaries for pure Go programs on ppc64le.  Is that acceptable?

Isn't this going to force external linking only when iscgo is true? So pure Go programs on ppc64le should still be truly statically linked with this change.
I just tried it, and that's what happened with a simple hello.go without cgo, which is pure Go AFAIK?
I think this is the correct temporary fix, but this means the 'go' tool is linked with the external linker and not the internal linker.  In both cases it is a dynamic binary. I'm not sure what the downside might be for using a different linker.

Sorry, what I wrote was a little confusing.  I think this will change the behavior of any pure Go program that imports the net or os/user packages.  I think that currently they will use internal linking, and with my suggestion they will use external linking.  You're right that they will be dynamically linked either way.
I think the only significant difference is that with this change a pure Go program (that imports the net or os/user package) will only be buildable if the C linker is installed on the system where the Go program is being built.  Without this change such a program would be buildable even if the C linker was not installed at all.  On GNU/Linux I doubt that is worth worrying about (on Windows, on the other hand, it would be a problem).
Does the change actually fix the problem?

Yes this fixes the problem, sorry I wasn't explicit on that.
Today those programs depend on using 'ld -r' during the link, so doesn't that mean they need to have the C linker installed?

You're right that clearly invoking ld -r means that you have the C linker.  I must be misremembering something about this.

This is related to #15409 is it not?
From my POV I'd be fine with deleting all the code related to internal linking support for cgo but that might not be terribly friendly.

Yes, it seems likely that fixing this would fix #15409.
Internal linking is quite important on Windows but I agree that it is less interesting on GNU/Linux systems.  Still, in general, it is a goal that people can install a Go distribution and, as long as they don't use cgo, use it with no C compiler or linker installed.  With the current standard library that means that the internal linker must support at least the code generated for the net and os/user packages.

The recent breakage in the ppc64x builds and the discussion about cross builds made me curious, so I did a cross build of golang for ppc64le and found that the go binary created is statically (and therefore internally) linked. I moved the cross built golang over to a ppc64le machine and the tests all passed. So I think that means an installation of a Go distribution for ppc64le from a cross build on an x86 could be installed with no C compiler or linker.
At this point I don't understand why the go binary is built differently when doing a cross. It seems that it should be built the same way whether cross or natively built. It should still have multiple TOCs in both cases?

By default when cross-compiling we assume that we don't have a C cross-compiler available, so we build without cgo support.  You should be able to get the same result cross and native, assuming you have a C cross-compiler, if you set CGO_ENABLED=1 in the environment.

@cherrymui @jeremyfaller Do you have a guess on how much work it would be to get internal linking to work with cgo on ppc64le? Maybe some relocation types are missing and then this TOC merging would have to be done.

I'm not familiar with the ABI to tell. I would guess as long as we actually understand the ABI, it won't take much work. Maybe keep track the multiple TOCs, and group them together (?) according to the ABI?

Actually I'm sure if the title truly explains the problem here. It's been a while so I'm looking back through to try and understand what's really wrong. I know the problem happened because there were two relocatable files to be linked together and they both had references which needed plt generation. They previously worked because 'ld -r' was done for all those files prior to the final link. When the 'ld-r' was removed this quit working and the fix was to disable internal linking with cgo on ppc64le.
Is this the ABI you mean? https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi-1.9.html

I tried to enable internal linking with cgo on master in cmd/link/internal/ld/config.go and got this error during the build:
 ./make.bash
Building Go cmd/dist using /home/boger/golang/go-linux-ppc64le-bootstrap. (devel +91f997b Wed Jan 6 06:08:34 2016 +0000 linux/ppc64le)
Building Go toolchain1 using /home/boger/golang/go-linux-ppc64le-bootstrap.

Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for linux/ppc64le.
# cmd/trace
.plt: initialize bounds (16 < 24)
panic: unexpected empty container symbol

goroutine 1 [running]:
cmd/link/internal/loader.(*Loader).AddInteriorSym(0xc000912000, 0x22a5e, 0x22a68)
	/home/boger/golang/link/go/src/cmd/link/internal/loader/loader.go:1678 +0x2d0
cmd/link/internal/ld.(*dodataState).allocateDataSections(0xc000012d80, 0xc000276000)
	/home/boger/golang/link/go/src/cmd/link/internal/ld/data.go:1661 +0x3874
cmd/link/internal/ld.(*Link).dodata(0xc000276000, 0xc0034b6000, 0x23a38, 0x23a38)
	/home/boger/golang/link/go/src/cmd/link/internal/ld/data.go:1503 +0xa20
cmd/link/internal/ld.Main(0x4386a0, 0x10, 0x20, 0x1, 0x1, 0x41, 0x0, 0x0, 0x2a58dc, 0x10, ...)
	/home/boger/golang/link/go/src/cmd/link/internal/ld/main.go:301 +0x1078
main.main()
	/home/boger/golang/link/go/src/cmd/link/main.go:68 +0x228

I added print statements before the panic and the empty container was the .got.

@cherrymui I think I found out why the "initialize bounds" error message is happening in .plt.
In cmd/link/internal/ppc64/asm.go function addpltsym, when an entry is added to the plt using plt.Grow, that doesn't update the size. If I add a call to plt.SetSize to increase the plt size then I can get past this error.
By the way, is there a tool like readelf to see what relocations are in the runtime/cgo(.text) and runtime/cgo(.toc) sections? It looks like the problem is that each input file has a runtime/cgo with (.text) and (.toc) but then at the final link, the relocations assume a single TOC. So some of the r2 addresses and relocated offsets are wrong because of this.

If I add a call to plt.SetSize to increase the plt size then I can get past this error.

That makes sense.

is there a tool like readelf to see what relocations are in the runtime/cgo(.text) and runtime/cgo(.toc) sections?

Sorry I'm not sure I understand what you mean. Why readelf itself doesn't do?

I was able to get readelf to work, I was trying it on the wrong type of file.
I'm trying to understand how the SELFGOT data should be processed because that's the problem I'm trying to resolve next.
In cmd/link/internal/loadelf/ldelf.go Load function is trying to load files and puts entries into the SELFGOT data. I added some prints here and see a few lines like this (the last number on the line is the localSymVersion).
### Reading file: /home/boger/golang/link/go/pkg/linux_ppc64le/runtime/cgo.a(_x004.o) ###
### Setting SELFGOT section: .toc for runtime/cgo(.toc) 250 ###
### Reading file: /home/boger/golang/link/go/pkg/linux_ppc64le/runtime/cgo.a(_x005.o) ###
### Setting SELFGOT section: .toc for runtime/cgo(.toc) 251 ###
### Reading file: /home/boger/golang/link/go/pkg/linux_ppc64le/runtime/cgo.a(_x009.o) ###
### Setting SELFGOT section: .toc for runtime/cgo(.toc) 255 ###

And then in cmd/link/internal/ld/data.go function allocateDataSections it tries to process the SELFGOT entries and there are a few problems with that code:

The .got is an entry in the SELFGOT but that has size 0 which causes a failure in the code below it. I added code to skip entries for the .got.
There is an expectation that runtime/cgo(.toc) is a symbol to be used as an interior symbol but that doesn't exist anywhere. I get an error saying that the interior symbol doesn't exist in the symtab. All I could find are places where the TOC symbols are created. Should these runtime/cgo(.toc) symbols be created? If not what should the interior symbol be for these.
There is an array called DotTOC in ctxt that has been filled in with .TOC. symbols for each symbol version but I don't see where or how those are to be used.
Once the program is created there is only one TOC symbol and all the relocations are based on a single TOC. But the code to load r2 and within the callstubs (plus maybe others?) are assuming the runtime/cgo(.toc) for the "version" it was generated. Maybe this is where the DotTOC values should be used but I don't see how they should be.

I don't know if the code in allocateDataSections for processing the SELFGOT for PPC64 is currently used for AIX or if it ever executed at all.


feat: strategic-merge patch or alike for merging arrays in multi-part state file and bases

Currently this:
releases:
- name: foo
---
releases:
- name: bar

results in:
releases:
- name: bar

That's because helmfile doesn't concat arrays.
The fix isn't that straightforward. Helmfile uses mergo under the hood. mergo has the option called WithAppend that merge arrays within a map or struct by concatenating. But just turning on doesn't work - resulting array items to be duplicated by double-rendering as reported in #615.
An alternative way would be to use strategic-merge patch to merge arrays according to the context. For example releases should be merged by name, or the combination of name and selectors.

I assume no update on this currently.  I was looking for a good way to keep helmfiles in subdirectories but then have a top-level helm which combined them all.  For test I was trying just some basic releases as you listed above, and then something like:
{{ $myfile1 := (readFile "myapp1/helmfile.yaml" | fromYaml) }}
{{ $myfile2 := (readFile "myapp2/helmfile.yaml"  | fromYaml ) }}
{{ deepCopy $myfile1 | merge $myfile2 | toYaml }}

But I get the same result as you mention above.

@bradenwright Hey! Thanks for reminding me about this one.
Would it be enough if Helmfile supports merging releases by name?
Do you need anything other than that in practice?

Essentially I'm trying to use depends: instead of helmfiles:, so I need 1 file that contains my dependencies.  But I have too many charts to keep them all in 1 top level file.
  It would be a bonus if I could still run the helmfile in myapp1, myapp2, etc.  but really mainly I plan on running helmfile from the top level helmfile, that way I can deploy all the apps with 1 command or just install a single app (with its dependencies).
So I think merging releases would be the main thing I'm missing, merging repos would be nice as well but not as important.  I'd also be fine not merging so to speak but just appending, for releases b/c I won't name them the same, for repos merging would be nice so I don't have to create duplicates.
For other sections of the helmfile I could always just make common.yaml and render it or something along those lines.  Only places I intended to use merge are for repos and releases
*** Side note ***
I'm open to other approaches, and I'm trying something with merging values, and then leveraging those below (or trying to render a template).  But I'm not great at golang so struggling trying to get it working.
helmfile.yaml
{{ $myfile1 := readFile "myapp1/helmfile-release.yaml" | fromYaml }}
{{ $myfile2 := readFile "myapp2/helmfile-release.yaml" | fromYaml }}
{{ $myvals := merge $myfile1 $myfile2 -}}

{{ range $k, $v := $myvals.repositories -}}
repositories:
- name: {{ $k }}
  url: {{ $v.url }}
{{- end }}

{{ range $k, $v := $myvals.releases -}}
releases:
- name: {{ $k }}
  chart: {{ $v.chart }}
  namespace: {{ $v.namespace }}
  version:  {{ $v.version }}
{{- end }}

cat myapp1/helmfile-release.yaml
repositories:
  bitnami:
    url: https://charts.bitnami.com/bitnami

releases:
  myapp1:
    chart: bitnami/wordpress
    namespace: default
    version:  9.3.6

cat myapp2/helmfile-release.yaml
repositories:
  bitnami:
    url: https://charts.bitnami.com/bitnami

releases:
  myapp2:
    chart: bitnami/wordpress
    namespace: default
    version:  9.3.6

Thanks for replying so quick :)

I should mention that I have been able to get this working repositories render multiple times, so apply takes slightly longer, but it works.
cat helmfile.yaml:
{{- $filePaths := (exec "bash" (list "-c" "echo -n **/helmfile-release.yaml") | splitList " ") }}

helmDefaults:
  kubeContext: {{ .Environment.Name }}

environments:
  {{ .Environment.Name }}:
    values:
    - ../environments/default.yaml.gotmpl
    - ../environments/{{ .Environment.Name }}.yaml.gotmpl

repositories:
{{ range $filename := $filePaths }}
{{ $file := readFile $filename | fromYaml }}
{{ if (hasKey $file "repositories") }}
{{ range $k, $v := $file.repositories -}}
- name: {{ $k }}
  url: {{ $v.url }}
{{- end }}
{{- end }}
{{- end }}

releases:
{{ range $filename := $filePaths }}
{{ $file := readFile $filename | fromYaml }}
{{ range $k, $v := $file.releases -}}
- name: {{ $k }}
  chart: {{ $v.chart }}
  namespace: {{ $v.namespace }}
  version:  {{ $v.version }}
  condition: {{ $k }}.enabled
  labels:
    name: {{ $k }}
  values:
  - {{ $k }}/values.yaml.gotmpl
  needs:
  {{- range $v.needs }}
    - {{ . }}
  {{- end }}
  secrets:
  {{- range $v.secrets }}
    - {{ . }}
  {{- end }}
{{- end }}
{{- end }}

cat traefik/helmfile-release.yaml
repositories:
  traefik:
    url: https://containous.github.io/traefik-helm-chart

releases:
  traefik:
    chart: ../charts/traefik
    namespace: wi-system
    version:  8.1.4
    needs:
      - kube-system/namespaces
    secrets:
      - ../secrets/{{ .Environment.Name }}/traefik.yaml
      - ../secrets/{{ .Environment.Name }}/shared-api-wi-tronix-local.yaml

cat namespaces/helmfile-release.yaml
releases:
  namespaces:
    chart: ../charts/namespaces
    namespace: kube-system
    version:  0.1.0
    needs: []
    secrets: []

I would really appreciate a feature to merge releases blocks as well. I basically have a large number of releases I would like to logically separate in different files, but that I would like to apply all at once. My current solution uses helmfiles:, but then I run into issues with the environment specification and sharing that between the files.


feat(energy): use SI macro for watt-hour units

Although watt-hours are technically non-SI units, they use the same set of prefixes.

@jonahsnider Would you mind reviewing this when you have a moment?
I'd also like to change the best array to something like ['GWh', 'MWh', 'KWh', 'Wh']. I think joules should rather require an explicit conversion because most quantities of energy are expressed in terms of watts.

Thanks for the contribution! I'm unfortunately very busy this week, so I might not be able to take a look at the changes until early next week. I'll see if I have time tonight though.

:tada: This PR is included in version 4.14.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

Thanks!


OnMount ref in Script setup ref undefined when OnMount ref in setup are

Version
3.2.20
Reproduction link
codesandbox.io
Steps to reproduce
Error one => https://codesandbox.io/s/unruffled-snow-3u1ol?file=/src/components/Slider.vue
Working one => https://codesandbox.io/s/cold-brook-8cr41?file=/src/components/Slider.vue
What is expected?
Both should work in the same way
What is actually happening?
The ref in OnMount in script setup is undefined

if you change @vue/compiler-sfc to version "3.2.20" in package.json it works. the codesandbox seems to use an older version.
It also works in the playground


Allow opting out of Floating UI on

Describe the feature in detail (code, mocks, or screenshots encouraged)
Hi, hope you're doing well! ☀In the createSelect builder we can pass positioning: null to opt out of floating UI but I couldn't find the equivalent in the select bit docs. This is needed so we can handle the presentation and positioning of options ourselves e.g, using CSS grid or a bottom sheet/dialog. Also Thanks! for #285
What type of pull request would this be?
Enhancement
Provide relevant links or additional information.
https://melt-ui.com/docs/builders/select#api-reference
https://m3.material.io/components/bottom-sheets/overview

This is something we have on the roadmap!


spot: init at 0.1.14

Motivation for this change
Closes: https://github.com/NixOS/nixpkgs/issues/114784
Unfortunately does not support Spotify Connect at the moment.
cc @colemickens do you want me to add you as a maintainer?
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 122599 run on x86_64-linux 1

1 package built:

spot

It builds and I could use my spotify account :+1:

Thank you!


Add a top-row header showing the currently selected page

The draw_page function is a bit of a hack, it has to write directly to the controlling terminal. This has been tested successfully on both serial outputs and virtualized physical outputs. To avoid weirdness, escape sequences are used to write the string directly to the required rows/columns, overwriting the existing header. Since the headers are always the same length, this just changes the highlighted page. A line return is NOT added; fzf does the right thing and starts drawing correctly on the next row, at column 0. fzf has to be told that the screen is 1 row shorter than it actually is, since it'll by default run in full screen mode.
Also add a special character ^ to help key text, indicating that the line should be centered on the screen.

This solution is unworkable with the linux console. Abandoning for now!


Rename wofkflow_object to workflow_object  in models managers

There is a spelling mistake in managers. kindly rename the file. Rename wofkflow_object to workflow_object  in models managers

Closing this issue with this Pull Request


Multitenant OIDC support added to the documentation

Fixes #5723

Hi @Geetha-Savithriamma , may I please ask you for a review? :-)

@ljamen @Geetha-Savithriamma Is everything OK now? :-)


Proposed names/meanings for #9677 discussion

Feel free to add comments here, or in the issue #9677

@vladsud - This diff illustrates the changes @scottn12 and I are proposing around container connection API naming.  Biggest difference from previous discussions is that we'd like to leave connected untouched, since it already carries a weight of the existing meaning throughout.
I have a few nuanced questions from reading through the code / existing comments which I'll put as comments in here to discuss.  Thanks!

@vladsud - I recall you talking about some different abstractions that Word is using, like it doesn't have a websocket connection or something?  Can you remind me of what that is so we can check the proposed naming/concepts against that paradigm?


fix c.func_name in ObjectJSON.callable_to_dict for python 3

The property func_name was substituted by __name__ in python 3.

see: https://docs.python.org/3/whatsnew/3.0.html#operators-and-special-methods

This pull requests implements a fix to be able to use the code in python 2 and python 3.

Sorry it's already handled in #830


[SE-0066] Adopt new function type syntax

Enclose single-argument functions in parentheses to silence newly-added (https://github.com/apple/swift/commit/3d2b5bcc5350e1dea2ed8a0a95cd12ff5c760f24) warnings.

@swift-ci please test

From the CI log, it appears that I missed a few. Once my local toolchain finishes building I'll update the rest.

Thanks @briancroom !


求助！无法获取 Gemini API，直接卡在第一步了。

How is Gemini Pro Chat deployed?
Node
Describe the bug
You do not have permission to view this page.

Console Logs
No response
Participation

[X] I am willing to submit a pull request for this issue.

换IP

换到英/美IP试试


natural language processing with RNN and CNN

Paper: Recursive deep models for semantic compositionality over a sentiment treebank
Scenario: sentiment analysis
Given a phrase of any length and type
Predict the parse tree and the sentiment of each node
Implementation: RNN (recursive - not recurrent - neural net)

word representation: d-dimentional vector
vocabulary: d x |V| matrix; V is the vocabulary, randomly initialized and trained as a parameter
phrase: every phrase is represented by a parse tree with words as leaves
merge: the process to create a parent node and adopt 2 child nodes
merge score: the mergability
tree score: the sum of all merge scores in the construction of the tree

The overall implementation is too complicated and the descriptions for both training and prediction are not clear.
Highlights

recursive neural net does not show up in action very often and its characteristics are not clear either

Paper: Recurrent Neural Networks for Language Understanding
Scenario: language modeling
Given a word sequence
Predict the next word continuously
Scenario: language understanding
Given a phrase
Predict the semantic label of the phrase
Implementation: RNN
The RNN reads a phrase one word at a time. Every word is represented as a 1 hot vector. The RNN continuously predict labels for every word as it reads the phrase. Every word should have only one label.
Highlights

one hot representation for a word (very raw)
a single recurrent layer
the recurrent layer can remember the history
labels are assumed mutually exclusive

Questions

is it better to have multiple small recurrent layers or a single big recurrent layers


JVM vs native protobuf serialization of Element big integers.

Its not obvious if the bytes for HACL and BigInteger are compatible.
Need to test and make sure they are interoperable.
Since python and any other language must read/write this, maybe we have to go back to hex or base64 encoding?

added passing tests VerifyJvmRecordTest, VerifyNativeRecordTest


The -light version should be inline-block by default also

Connects to https://github.com/vaadin/components-team-tasks/issues/280

This change is

 but CI failing, shouldn't be related with this patch

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks broke.

Comments from Reviewable

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks broke.

Comments from Reviewable


exclude commented lines when doing a search against all files/folders in the explorer

I know it is possible to craft a regex search but that is a cumbersome experience every time you need to search something.
If there as a global setting, or a setting from within the search box, that could be used to hide lines that are commented out, that would make the user experience a lot better IMHO.

duplicate of #55832


Check clock skew

The locking depends on the machines in question having the same concept of the current time.
We should check this in some way before maintaining the lock.

Fixed with 81adaa82d096abb498a363aa13f5954975171c9c


Jitsi | Call will disconnect after 5 minutes

Hello Team,
I have deployed jitsi as part of the dependency to fully deploy workadventure.
the workadventure is now properly working on our side but when we joined in a bridge call through jitsi we got this pop-up.

I would just like to know how can I proceed on this?
Note that my colleague signed up for the free version of jitsi as a service for 25 users only. I would like to know if it is possible to override the values.yaml to enable the jitsi as a service using api key of the jitsi as a service?
This is my whole code to deploy jitsi via helm
resource "helm_release" "jitsi" {
  name       = "jitsi"
  repository = "https://jitsi-contrib.github.io/jitsi-helm/"
  chart      = "jitsi-meet"
  namespace  = kubernetes_namespace.jitsi.metadata[0].name
  version    = "1.4.1"

  values = [
    yamlencode({
      web = {
        ingress = {
          enabled  = true
          pathType = "Prefix"
          annotations = {
            "kubernetes.io/ingress.class"    = "nginx"
            "cert-manager.io/cluster-issuer" = "letsencrypt"
          }
          hosts = [
            {
              host  = "jitsi.${var.domain}"
              paths = ["/"]
            }
          ]
          tls = [
            {
              secretName = "jitsi-tls"
              hosts      = ["jitsi.${var.domain}"]
            }
          ]
        }

        service = {
          type = "ClusterIP"
        }

        config = {
          disableThirdPartyRequests : true
          enable_features_based_on_tokens : true
          enable_jitsi_meet_api : true
          resolution : 720
          disableSimulcast : false
          constraints = {
            video = {
              height = {
                ideal = 480
                max   = 720
              }
            }
          }

          publicURL : "https://jitsi.${var.domain}"

          deploymentInfo : {
            shard : "jitsi.${var.domain}",
            region : "self-hosted",
            userRegion : "self-hosted"
          }

          privateKey : module.onepassword_items.items["jaas_private_key"].sections["root"]["private_key"].value
          appId : module.onepassword_items.items["jaas_private_key"].sections["root"]["api_key"].value
        }
      }

      jvb = {
        useHostPort = true
        useNodeIP   = true
        service = {
          type = "ClusterIP"
        }
      }

      prosody = {
        service = {
          type = "ClusterIP"
        }
        config = {
          xmpp = {
            domain = "jitsi.${var.domain}"
          }
        }
      }

      jicofo = {
        service = {
          type = "ClusterIP"
        }
      }
    })
  ]
}

Questions:

how can we pass the app_id and private_key correctly into our code so that we will be able to use the free version of the jitsi as service for 25 users.

Hope you can help me on this @spijet

https://github.com/jitsi-contrib/jitsi-helm/discussions/143


WFLY-5165 Quickstart Arquillian tests log warning deprecated protocol…

… warning

that would be question for @jamezp

Hopefully soon. I'm working on some batch issues at the moment, then hopefully by the end of this week I can review these PR's and get a release out.


[BUG] Deterministic WINTER stackid from raw images

Describe the bug
Determine winter stackid from raw images, to make it very deterministic, by adding a CustomImageBatchModifier processor.

Maybe we should just replace CustomImageBatchModifier with CustomImageBatchModifier, and have the functions do the for-loop over images? I can't see a downside to that, honestly.


Feature request: Easy way to share scenes

My brother isn't a programmer, I gave him an exact copy of my project and told him to edit the scene, fixing some UI stuff. When he's done there's no way for me to copy the .tsn files back into the project, I have to copy the entire project.
my scripts are stored in /scripts, my scenes are stored in /scenes, he has an exact copy, just has to edit the scenes and give them back to me.I know and use git, he's not a programmer, he doesn't know it and won't tell him to learn it.
In fact next month I'm hiring a graphic designer to help me with the intro scene, she might need to work in godot, because it's so easy for me to teach her, as long as she's sticking to the GUI, that way I won't waste my time adding whatever she does in godot, she can add it herself.  I don't want to teach her git too.

When he's done there's no way for me to copy the .tsn files back into the project, I have to copy the entire project.

Why would that be? Do you get some error on broken dependencies or something? It should work perfectly fine as long as your project's copy has all the resources used on scene, in the same paths.

I'm not sure, he gives me the .tsn back, I rerun the game, I see the old game, before he did his edits. Am I missing something?

If you have the scene open, you need to reload it when you replace the file (the easiest way is to use Scene -> Revert Scene). Godot doesn't do this automatically.

Oh thanks I'll try your answer soon, and reopen this issue if it did not work, I'll close it for now to reduce the number of open issues by one :) Thanks so much!


What is this project's relationship to DefinitelyTyped?

I just found out that typings for lodash exist in both this project and on DefinitelyTyped. This brings up a few questions:

Is there any relationship between the two, or are they disjoint efforts?
If they are separate projects, why is that the case?
If they are separate projects, what are the strengths and weaknesses of using one over the other?

I'm asking here because DT has thousands of issues, and I suspect this one will get buried.

Both are distinct projects, but typings is now deprecated: the goal is to have a single system (@types backed by DefinitelyTyped)
Typings was created to address some issues of Definitely Typed, mainly that it offered only global definitions (causing conflicts on duplication), there were issues on transitive dependencies and all the definitions were hosted in single repo so it was painful to contribute.
Typings made type management sane. DefinitelyTyped went through a transition and is now mostly OK from a consumer point of view (you can simply install types with @types) but it's still pretty hard to contribute to it. Also, typings tricked the compiler to accept multiple type definitions by smartly creating bundles, on the other hand @types is understood by the compiler using Node's resolution.

If you want a future-proof solution, use @types (types from Definitely Typed). It improved during the previous year and is the officially supported solution.

@demurgos Thanks for that response. I'm closing this issue because you answered my question, but I do have one follow-up question.
Since DT is what is officially supported, would it ever make sense to contribute to this project? Should it be deprecated to avoid confusion?

The repositories of the types organization are still maintained to not break the workflow of people relying on typings, and because of the long-awaited Issue 4.
To be precise, I recommended to you to use @types, not DefinitelyTyped directly: instead of downloading types from Github, use the ones published on npm. There is a level of indirection: you are not tied to DefinitelyTyped or the types Github organization. Even if only DefinitelyTyped is used at the moment, there is some effort to use the individual repositories of the types Github organization instead (at least partially). This would allow individual issue trackers, PRs, CI, etc.
If your projects needs type-definitions for a library and you don't want to wait for others to fix your problem, I'd recommend the following workflow:

Write type definitions in your own project (I usually use a directory called custom-typings and set it as a type root). Once it works in your project, share it:
Open an issue / PR on the repo of the library: ask them to host their own type definitions. This is the best case scenario and some libraries are happy to do it. (It allows to simply npm install my-lib and have both the types and implementation)
Otherwise, focus on DefinitelyTyped to get your change on @types as soon as possible. You can open a PR on the corresponding types repo, but until the issue 4 is solved, you won't have any direct benefit out of it.


adding some dumped resources

These files are dumped by UnityAssetStudio. contains both recipies and icons.

Nice.
Dunno if we can add icons themselves to the repo (or if we need them), I'm worried about potential copyright stuff. What I'd be interested in more is the "Signal" variant of the DSPIcon enum, this corresponds to "signals" tab for icons.
Also it's interesting that the "primary" language for tech, item etc. names is Chinese. This will make generating enum names more complicated :/


Removed pos_label from CohenKappa

Forgot to remove pos_label from CohenKappa last time. Also removed labels from both HammingLoss and HammingLossMulti (it's deprecated, doesn't do anything and will be removed in sklearn 0.23)

Thanks a lot!


Remove the show four corners button?

I toggled the four corners button the other day by accident and thought it was a bug for a few seconds when I couldn’t see the circles on the corners. Does anyone really want these off? I don’t find them intrusive, just informative.

I second this. We could just remove the button but leave the feature in the code if we make the toolbar configurable in the future.


HttpClient from HttpClientFactory deadlocks in [WebMethod] called from Angular

My setup

ASP.NET WebForms using .NET framework 4.6.1
Angular 1.6.6
IHttpClientFactory implementation from Microsoft.Extensions.Http both version 2.1.1 & 2.2.0-preview3-35497

When making a request from Angular to a server-side method that in turns makes an HTTP request using an HttpClient obtained from the IHttpClientFactory implementation, the application deadlocks.
It's worth mentioning that if instantiating an HttpClient manually, the method does not deadlock.
In order to use IHttpClientFactory outside a netcoreapp2.0 project, I've used the following code:
IServiceCollection services = new ServiceCollection();
services.AddHttpClient();
ServiceProvider serviceProvider = services.BuildServiceProvider();
IHttpClientFactory _factory = serviceProvider.GetService<IHttpClientFactory>();
To Reproduce
Steps to reproduce the behavior:
Call the Load method from an Angular controller:
[WebMethod]
   public static async Task> Load()
  {
var data = await GetDataAsync().ConfigureAwait(false);
}
private static async Task<HttpResponseMessage> GetDataAsync()
{
HttpClient client = _factory.CreateClient();
// client = new HttpClient();
return await client.GetAsync("http://www.google.com").ConfigureAwait(false);
}
Expected behavior
The method GetDataAsync() returns.
Actual Behavior
The method  GetDataAsync hangs, most probably deadlocking.
Note that if uncommenting the line client = new HttpClient(); the method does not hang anymore.

This looks like another repo of #999 - which was fixed recently. If you need a workaround for this, you could solve it by removing our logging message handlers from the services after calling AddHttpClient. That will remove our logging support, which is the root cause of the bug.


Section on evaluation frameworks

Would it be useful to have a "frameworks" section? This would include links to evaluation frameworks like lm-eval from EleutherAI adn crfm-helm from Stanford.

thanks for the suggestion, definitely! if you want to do a PR for this, that'd be grand!

@jas-ho Will submit one!

amazing, thanks!
On Mon, May 20, 2024 at 10:21 AM aditya thomas @.***>
wrote:

@jas-ho https://github.com/jas-ho Will submit one!
—
Reply to this email directly, view it on GitHub
https://github.com/apartresearch/evaluations-starter/issues/1#issuecomment-2119926951,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTRY4KGAV5T554DFCCFPPLZDGW7BAVCNFSM6AAAAABH6NKVPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZHEZDMOJVGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Jason Hoelscher-Obermaier
Co-Director, Apart Research https://apartresearch.com/
Book a meeting https://zcal.co/jas-ho/apart-20 with me
+4368120163911
𝕏 https://twitter.com/JasonObermaier | in
https://www.linkedin.com/in/jas-ho | gh https://github.com/jas-ho


Fix button font (was falling back to Arial)

Spiritual followup to https://github.com/Brightspace/d2l-my-courses-ui/pull/300, just missed a font-family.

Imma be #2edgy5me and just merge this, since it's just a font change


Scan Low Latency Setting

Hi,
Is it possible to add support for SCAN_MODE_LOW_LATENCY  for Android 5.x and above ?
https://developer.android.com/reference/android/bluetooth/le/ScanSettings.html#SCAN_MODE_LOW_LATENCY

Yes it is possible :) Adding it as feature request.

Thanks.  Since we are on the roll,  please include the "MATCH" series.   :)

please include the "MATCH" series.

What? I don't get it.

What I mean is since you are going to look at adding support for SCAN_MODE_LOW_LATENCY , it would be wonderful to add the MATCH_* scan setting.  Thank you.

Hmm I had a quick look at MATCH_ stuff. It seems, that it is just for filtering. We abstracted the filtering.
You can use StartScanningForDevicesAsync with a filter callback.
adapter.StartScanningForDevicesAsync(device => device.Name.Contains("foo"));

https://github.com/xabre/xamarin-bluetooth-le/blob/master/Source/Plugin.BLE.Abstractions/Extensions/AdapterExtenstion.cs#L45
Or does it provide more?

Hi,
I came across addition information Android BlE.  It might be of some help.  Thank you.
https://devzone.nordicsemi.com/attachment/7dc38a5edf15efb8c328b13544a4ac94

We need to investigate if there are similar options on iOS.

There aren't similar controls on iOS.

Implemented in 1.2.2.
adapter.ScanMode = ScanMode.LowLatency;

What I mean is since you are going to look at adding support for SCAN_MODE_LOW_LATENCY , it would be wonderful to add the MATCH_MODE_STICKY, MATCH_NUM_FEW_ADVERTISEMENT series scan setting. Thank you.

The match mode has been added in #607. Therefore I'm finally closing this issue.


Where do I config SMTP account name and password?

This is a duplicate of #295, but see this section in the docs.  Transport is a configuration for sending email (e.g. SMTP configuration username and password).  For full reference see https://nodemailer.com/transports/.


granite-code:20b-instruct-q8_0  error loading model vocabulary: unknown pre-tokenizer type: 'refact'

What is the issue?
llama_model_loader: - type  f32:  419 tensors
llama_model_loader: - type q8_0:  209 tensors
llama_model_load: error loading model: error loading model vocabulary: unknown pre-tokenizer type: 'refact'
llama_load_model_from_file: exception loading model
libc++abi: terminating due to uncaught exception of type std::runtime_error: error loading model vocabulary: unknown pre-tokenizer type: 'refact'
time=2024-05-25T03:28:13.829-04:00 level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: abort trap error:error loading model vocabulary: unknown pre-tokenizer type: 'refact'"
[GIN] 2024/05/25 - 03:28:13 | 500 |   446.67725ms |     | POST     "/v1/chat/completions"
OS
macOS
GPU
Apple
CPU
Apple
Ollama version
0.1.38

getting the same error with granite-code:20b-instruct-q3_K_M model in docker desktop running on 4090 GPU

Error: llama runner process has terminated: signal: abort trap error:error loading model vocabulary: unknown pre-tokenizer type: 'refact'
macOS 14.5
ollama version is 0.1.38

Thanks for the report. As noted above this model (and other granite family models) will work in ollama v0.1.39+ now.


[Help wanted] Writers for upcoming issues in June and July

Taking time off
Hey everyone 😄
I will be "off the grid" coming up and I'm looking for volunteers for the next few issues. If no one is available, that's fine, we'll just take a break for those weeks.
Upcoming issues:

[x] 74: June 15, @benasher44
[x] 75: June 22 @gregheo
[x] 76: June 29 @garricn
[x] 77: July 6 @RomanVolkov

Responsibilities:
I'll be getting online periodically to help with anything that comes up. As usual, we'll also keep a GitHub issue open for each Swift Weekly issue to gather notes to help the current writer.
For anyone in the OSS community new to contributing, see our contributing guide. By now, everything is automated (emails, tweets, RSS). All you need to do is write some markdown, and then push to GitHub! 😄

cc @gregheo @BasThomas @modocache — let me know if you guys are interested! If not, that's ok too. 😄

If interested, please comment here and I'll update the main issue! 👍

I'd be happy to do #74!

@benasher44 Just added you to the authors.yml and GH org. https://github.com/SwiftWeekly/swiftweekly.github.io/blob/master/_data/authors.yml#L21-L24

And here I was thinking you just got "on the grid". 🤔
I am afraid I wont be able to write any full issues - graduation and all that. Like Jesse though, I will of course be hanging around, so let me know if there's any questions and I am happy to help out. 😃

@BasThomas - Ha, yeah 😅 After this, I'll be back and won't be leaving for awhile.

I'll take 75. If there's a lack of volunteers, I can publish 75 on the 29th so there's an issue every two weeks.

Thanks @gregheo ! 👍  Sounds good to me.
If we don't have anymore volunteers, I'll update the schedule in the main issue.

I can do 76

I would like to get 77.

Thanks @garricn and @RomanVolkov ! If you have any questions, let me know.
@BasThomas Should also be able to help answer questions. 😄

Looking forward to it 🤗 Thank you for the opportunity!

Awesome, thanks all! Looking forward to these 😃

Cool👍🏻

Friendly ping, @benasher44 :)

Hi! Did I forget something? I've been working on 74 locally and will push my WIP branch this morning!

Nice!

What I have for 74 so far is up! #268

To be sure, when should the "next" issue be created and what time period should it cover? For example, issue 74 will be released this Thursday, correct? So should Issue 75 be created on Thursday after Issue 74 is released and should it cover the time period of this Thursday through the following Wednesday?

@garricn that sounds about right. You could create the draft earlier in _drafts with no ill effects, but makes sense to run the script on Thursday so there's only one issue "in flight" at a time.

yep! that's right @garricn ! 👍
thanks @gregheo ! ❤


fix: absolute paths should not use the guest cwd

This is a follow-up to #30.
Removing the vim.loop.fs_realpath check helps with relative paths that exist in the guest, but it breaks absolute paths.
For example, in a neovim terminal
cd /tmp/
vim /tmp/a/file.txt

Would concatenate the path to /tmp//tmp/a/file.txt
I'm guessing checking for the real path was supposed to handle that scenario, it just conflicted with relative paths that matched the host cwd.
This commit checks each file to see if it's absolute and avoid prepending the guest cwd.

Tested, fix works :+1:


Proposal: delete auto-exported file that are renamed

Currently if I rename a notebook, the exported file with old name is not deleted.

Hi @cccntu, thank you for using nbautoexport and for opening the issue.
Have you taken a look at the experimental clean command? You can check out the documentation page for it to see more about how to use it. We developed the clean command for this exact use case. The clean command works by checking files that exist against files that are expected to exist based on present notebooks. However, this is something you'll have to do manually, and it won't work automatically.
As for developing some kind of automatic functionality:

I don't know of a way to know when a notebook as been renamed. If you're renaming the notebook through Jupyter and Jupyter has some kind of event and hook for it, we can potentially do something with that, but I don't know about any functionality like that in Jupyter. However, this still won't work if you're renaming the notebook using a shell or if you rename it through your OS file browser.
We did consider having the aforementioned clean command be able to run automatically after exporting, but we decided that having files deleted without user input needs a really high bar of reliability, and that clean's heuristic isn't good enough.

Let us know what you think!

Hi @cccntu, thank you for using nbautoexport and for opening the issue.
Have you taken a look at the experimental clean command? You can check out the documentation page for it to see more about how to use it. We developed the clean command for this exact use case. The clean command works by checking files that exist against files that are expected to exist based on present notebooks. However, this is something you'll have to do manually, and it won't work automatically.
As for developing some kind of automatic functionality:

I don't know of a way to know when a notebook as been renamed. If you're renaming the notebook through Jupyter and Jupyter has some kind of event and hook for it, we can potentially do something with that, but I don't know about any functionality like that in Jupyter. However, this still won't work if you're renaming the notebook using a shell or if you rename it through your OS file browser.
We did consider having the aforementioned clean command be able to run automatically after exporting, but we decided that having files deleted without user input needs a really high bar of reliability, and that clean's heuristic isn't good enough.

Let us know what you think!

Hi @jayqi , thank you for the thorough explanation.

I just checked out the doc and tried the clean command, and I think it's good. Also, I think the design consideration is very reasonable.
I also tried the export command, but found they always exports every files. Maybe we can check the modified time and only export the necessary files? (like in Make) And user can add this command in Makefile without generating all the files each time. Maybe call this command sync. I can create another issue if you prefer.

Hi @jayqi , thank you for the thorough explanation.

I just checked out the doc and tried the clean command, and I think it's good. Also, I think the design consideration is very reasonable.
I also tried the export command, but found they always exports every files. Maybe we can check the modified time and only export the necessary files? (like in Make) And user can add this command in Makefile without generating all the files each time. Maybe call this command sync. I can create another issue if you prefer.

Awesome. I am going to close this issue for now since it looks like the clean command is good enough. We can later if there is a lot of demand for an automatic solution.
I've opened #67 for your suggestion about export.

Awesome. I am going to close this issue for now since it looks like the clean command is good enough. We can later if there is a lot of demand for an automatic solution.
I've opened #67 for your suggestion about export.


Stability testing with ADE repo

task for code text change testing with ADE repo.
Current goal is to reach less than 1%

328 failing tests so far (4396 passing)

266 with current develop

174 with tree-sitter 0044 (3.82% failed)

After making this change https://github.com/BalticAmadeus/AblFormatter/issues/267 the number of errors was reduced to 91

82 with tree-sitter 0045