Robyn server hangs while serving rasa model

Bug Description
Sample code
from robyn import Robyn
from rasa.core.agent import Agent
import uuid6

app = Robyn(__file__)
agent = Agent.load("./models/20230818-091151.tar.gz")

@app.get("/")
async def h(request):
    return "Hello, world!"

@app.post("/chat")
async def chat(request):
    print("chat")
    return await agent.handle_text("Hi", sender_id=uuid6.uuid7().hex)

app.start(port=8080)

when i run the above server stuck processing the request even tried via post man as you can see it stuck in the logs as slots values and does not process
---logs--
chat
DEBUG:rasa.core.lock_store:Issuing ticket for conversation '018a0a6363af764ba677c15c824fbdb8'.
DEBUG:rasa.core.lock_store:Acquiring lock for conversation '018a0a6363af764ba677c15c824fbdb8'.
DEBUG:rasa.core.lock_store:Acquired lock for conversation '018a0a6363af764ba677c15c824fbdb8'.
DEBUG:rasa.core.tracker_store:Could not find tracker for conversation ID '018a0a6363af764ba677c15c824fbdb8'.
DEBUG:rasa.core.processor:Starting a new session for conversation ID '018a0a6363af764ba677c15c824fbdb8'.
DEBUG:rasa.core.processor:Policy prediction ended with events '[]'.
DEBUG:rasa.core.processor:Action 'action_session_start' ended with events '[<rasa.shared.core.events.SessionStarted object at 0x7ffa23ff6be0>, ActionExecuted(action: action_listen, policy: None, confidence: None)]'.
DEBUG:rasa.core.processor:Current slot values:
doc_url: None
document: None
video: None
audio: None
image: None
kairon_action_response: None
bot: 64de24d9fad53e0694a04261
session_started_metadata: None
Steps to Reproduce
No response
Your operating system
Linux
Your Python version (python --version)
3.8
Your Robyn version
latest
Additional Info
No response

@sfahad1414 , can you please share the model too?

model is available in the repo https://github.com/sfahad1414/RasaRobyn.git

Hey @sfahad1414 👋
The rasa model being used requires a Python version <3.8, which is no longer supported by the Python community.

The issue will go away if you will use an updated model. Do let me know if you have any more questions 😄

even training model on python 3.8.17 now models loads but hang on inference
2023-08-20 21:48:33 [debug    ] processor.actions.policy_prediction prediction_events=[]
2023-08-20 21:48:33 [debug    ] processor.actions.log          action_name=action_session_start rasa_events=[<rasa.shared.core.events.SessionStarted object at 0x7f2f33288d90>, ActionExecuted(action: action_listen, policy: None, confidence: None)]
2023-08-20 21:48:33 [debug    ] processor.slots.log            slot_values=     doc_url: None
document: None
video: None
audio: None
image: None
kairon_action_response: None
bot: 64de24d9fad53e0694a04261
session_started_metadata: None

Hi @sansyrox 👋,
did you tried the new model

Hey @sfahad1414 👋
I didn't enough time in the past two days. I will get back to you by Friday 😄

Hey @sfahad1414 👋
I tried running you repo but I am getting the following issue
INFO:robyn.logger:SERVER IS RUNNING IN VERBOSE/DEBUG MODE. Set --log-level to WARN to run in production mode.
INFO:rasa.core.processor:Loading model ./models/20230818-091151.tar.gz...
Traceback (most recent call last):
  File "app.py", line 6, in <module>
    agent = Agent.load("./models/20230818-091151.tar.gz")
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 345, in load
    agent.load_model(model_path=model_path, fingerprint=fingerprint)
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 352, in load_model
    self.processor = MessageProcessor(
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/core/processor.py", line 101, in __init__
    self.model_filename, self.model_metadata, self.graph_runner = self._load_model(
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/core/processor.py", line 138, in _load_model
    metadata, runner = loader.load_predict_graph_runner(
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/engine/loader.py", line 26, in load_predict_graph_runner
    model_storage, model_metadata = model_storage_class.from_model_archive(
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/engine/storage/local_model_storage.py", line 86, in from_model_archive
    cls._extract_archive_to_directory(
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/engine/storage/local_model_storage.py", line 128, in _extract_archive_to_directory
    LocalModelStorage._assert_not_rasa2_archive(temporary_directory)
  File "/Users/sanskar/test_dumpster/RasaRobyn/venv/lib/python3.8/site-packages/rasa/engine/storage/local_model_storage.py", line 137, in _assert_not_rasa2_archive
    raise UnsupportedModelVersionError(
rasa.exceptions.UnsupportedModelVersionError: The model version is trained using Rasa Open Source 2.8.15 and is not compatible with your current installation which supports models build with Rasa Open Source 3.5.0 or higher. This means that you either need to retrain your model or revert back to the Rasa version that trained the model to ensure that the versions match up again.

Is your requirements.txt file in sync with the required dependencies?

@sansyrox
use this model https://github.com/sfahad1414/RasaRobyn/blob/main/models/20230820-074946-asphalt-ocean.tar.gz it is train on rasa 3 version on python 3.10

@sansyrox did you tried the new model and requirements.txt

Hey @sfahad1414 👋
I did but I am unable to replicate the bug. Can you please create an isolated environment so that I can reproduce this bug?

@sansyrox added the Dockerfile https://github.com/sfahad1414/RasaRobyn/blob/main/Dockerfile

@sansyrox did you tried this

Hey @sfahad1414 👋
Your repo still errors out.
Here is the error I get
31   Stored in directory: /root/.cache/pip/wheels/85/52/e7/dfa571c9f9b879e3facaa1584f52be04c4c3d1e14054ef40ab
#9 76.31   Building wheel for python-crfsuite (setup.py): started
#9 76.47   Building wheel for python-crfsuite (setup.py): finished with status 'error'
#9 76.47   error: subprocess-exited-with-error
#9 76.47
#9 76.47   × python setup.py bdist_wheel did not run successfully.
#9 76.47   │ exit code: 1
#9 76.47   ╰─> [24 lines of output]
#9 76.47       running bdist_wheel
#9 76.47       running build
#9 76.47       running build_py
#9 76.47       creating build
#9 76.47       creating build/lib.linux-aarch64-cpython-310
#9 76.47       creating build/lib.linux-aarch64-cpython-310/pycrfsuite
#9 76.47       copying pycrfsuite/_logparser.py -> build/lib.linux-aarch64-cpython-310/pycrfsuite
#9 76.47       copying pycrfsuite/__init__.py -> build/lib.linux-aarch64-cpython-310/pycrfsuite
#9 76.47       copying pycrfsuite/_dumpparser.py -> build/lib.linux-aarch64-cpython-310/pycrfsuite
#9 76.47       running build_ext
#9 76.47       building 'pycrfsuite._pycrfsuite' extension
#9 76.47       creating build/temp.linux-aarch64-cpython-310
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/lib
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/lib/cqdb
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/lib/cqdb/src
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/lib/crf
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/lib/crf/src
#9 76.47       creating build/temp.linux-aarch64-cpython-310/crfsuite/swig
#9 76.47       creating build/temp.linux-aarch64-cpython-310/liblbfgs
#9 76.47       creating build/temp.linux-aarch64-cpython-310/liblbfgs/lib
#9 76.47       creating build/temp.linux-aarch64-cpython-310/pycrfsuite
#9 76.47       gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Icrfsuite/include/ -Icrfsuite/lib/cqdb/include -Iliblbfgs/include -Ipycrfsuite -I/usr/local/include/python3.10 -c -std=c99 crfsuite/lib/cqdb/src/cqdb.c -o build/temp.linux-aarch64-cpython-310/crfsuite/lib/cqdb/src/cqdb.o
#9 76.47       error: command 'gcc' failed: No such file or directory
#9 76.47       [end of output]
#9 76.47
#9 76.47   note: This error originates from a subprocess, and is likely not a problem with pip.
#9 76.47   ERROR: Failed building wheel for python-crfsuite
#9 76.47   Running setup.py clean for python-crfsuite
#9 76.60   Building wheel for fire (setup.py): started
#9 76.78   Building wheel for fire (setup.py): finished with status 'done'
#9 76.78   Created wheel for fire: filename=fire-0.5.0-py2.py3-none-any.whl size=116934 sha256=71691e5aaffa38fe218a3cd7cf9b9126fbe0a92f85577171a10fb3e34160bf77
#9 76.78   Stored in directory: /root/.cache/pip/wheels/90/d4/f7/9404e5db0116bd4d43e5666eaa3e70ab53723e1e3ea40c9a95
#9 76.78   Building wheel for future (setup.py): started
#9 77.11   Building wheel for future (setup.py): finished with status 'done'
#9 77.12   Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492024 sha256=caba1310c711b9fc168b5c361a7990cc57ed3f3399545f7325d65a61532ff494
#9 77.12   Stored in directory: /root/.cache/pip/wheels/5e/a9/47/f118e66afd12240e4662752cc22cefae5d97275623aa8ef57d
#9 77.12 Successfully built jieba mattermostwrapper randomname docopt jax fire future
#9 77.12 Failed to build python-crfsuite
#9 77.12 ERROR: Could not build wheels for python-crfsuite, which is required to install pyproject.toml-based projects
------
executor failed running [/bin/sh -c python -m pip install -r requirements.txt]: exit code: 1

still open, will work on a single process, but need to fix the multiprocess issue


TypeError: Cannot read properties of undefined (reading '0')

The plugin makes my homebridge crashing frequently:
[04/11/2023, 13:51:59] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:111:36)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:20
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:27)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

Konfiguration:
Homebridge v1.7.0
homebridge-fibaro-home-center v1.2.20
Node.js v18.16.0
Raspbian GNU/Linux Bullseye (11)
Fresh Install on Raspberry Pi 3+
Home Center 2 v4.630
Logfile excerpt:
[04/11/2023, 13:56:18] [FibaroHC] Updating value for device:  131  parameter: Temperature Display Units
[04/11/2023, 13:56:18] [FibaroHC] Updating value for device:  131  parameter: Current Temperature
[04/11/2023, 13:56:18] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:111:36)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:20
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:27)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
[04/11/2023, 13:56:18] Got SIGTERM, shutting down Homebridge...
[04/11/2023, 13:56:18] [iRobot] Disconnecting From Roomba: Roomba
[04/11/2023, 13:56:18] [iRobot] Roomba Roomba connection closed
[04/11/2023, 13:56:23] [HB Supervisor] Homebridge Process Ended. Code: 143, Signal: null
[04/11/2023, 13:56:28] [HB Supervisor] Restarting Homebridge...

@Krillle can you try latest version? v 1.4.0

@mkz212  v1.4.0 is installed. I will monitor and report. Thank you for taking care.

@mkz212  unfortunately same issue with v1.4.0.
It always happens after Device 131, which is a heating zone.
[10/11/2023, 19:26:54] [FibaroHC] Temperatur 137.0.1 [139]: 25.5 C
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:59] [FibaroHC] Wohnbereich [131]: 23.0 C
[10/11/2023, 19:27:59] [FibaroHC] Wohnbereich [131]: 0 %
[10/11/2023, 19:27:59] [FibaroHC] Temperatur 129.0.1 [131]: 23.5 C
[10/11/2023, 19:27:59] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:110:36)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:20
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:27)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
[10/11/2023, 19:27:59] Got SIGTERM, shutting down Homebridge...

I will try to check it out. If you can, provide more data - for which device is this happening? Or spontaneously or during starting / turning off the device? How often? Maybe also put here API  response for this device? (You will get the API by entering in browser:  IP/api/device where IP is your Home Center IP. (text, no screenshot)

It seems to happen spontaneously, but not always always on device 131, which is a heating zone.
http://HC2.local/api/devices/131 reports:
{
  "id": 131,
  "name": "Temperatur 129.0.1",
  "roomID": 4,
  "type": "com.fibaro.temperatureSensor",
  "baseType": "com.fibaro.multilevelSensor",
  "enabled": true,
  "visible": true,
  "isPlugin": false,
  "parentId": 129,
  "remoteGatewayId": 0,
  "interfaces": [
    "zwave"
  ],
  "properties": {
    "parameters": [
      {
        "id": 1,
        "lastReportedValue": 1,
        "lastSetValue": 1,
        "size": 2,
        "value": 1
      },
      {
        "id": 2,
        "lastReportedValue": 1,
        "lastSetValue": 1,
        "size": 2,
        "value": 1
      },
      {
        "id": 3,
        "lastReportedValue": 0,
        "lastSetValue": 0,
        "size": 2,
        "value": 0
      },
      {
        "id": 4,
        "lastReportedValue": 3,
        "lastSetValue": 3,
        "size": 2,
        "value": 3
      },
      {
        "id": 5,
        "lastReportedValue": 50,
        "lastSetValue": 50,
        "size": 2,
        "value": 50
      },
      {
        "id": 6,
        "lastReportedValue": 400,
        "lastSetValue": 400,
        "size": 2,
        "value": 400
      },
      {
        "id": 7,
        "lastReportedValue": 50,
        "lastSetValue": 50,
        "size": 2,
        "value": 50
      },
      {
        "id": 8,
        "lastReportedValue": 400,
        "lastSetValue": 400,
        "size": 2,
        "value": 400
      },
      {
        "id": 9,
        "lastReportedValue": 0,
        "lastSetValue": 0,
        "size": 2,
        "value": 0
      },
      {
        "id": 10,
        "lastReportedValue": 240,
        "lastSetValue": 240,
        "size": 2,
        "value": 240
      },
      {
        "id": 11,
        "lastReportedValue": 240,
        "lastSetValue": 240,
        "size": 2,
        "value": 240
      },
      {
        "id": 12,
        "lastReportedValue": 2,
        "lastSetValue": 2,
        "size": 2,
        "value": 2
      },
      {
        "id": 13,
        "lastReportedValue": 210,
        "lastSetValue": 210,
        "size": 2,
        "value": 210
      }
    ],
    "pollingTimeSec": 0,
    "zwaveCompany": "Heatit",
    "zwaveInfo": "6,3,83",
    "zwaveVersion": "1.5",
    "categories": "[\"climate\"]",
    "configured": true,
    "dead": "false",
    "deadReason": "",
    "deviceControlType": "0",
    "deviceIcon": "1004",
    "emailNotificationID": "0",
    "emailNotificationType": "0",
    "endPointId": "0",
    "log": "",
    "logTemp": "",
    "manufacturer": "",
    "markAsDead": "true",
    "model": "",
    "nodeId": "32",
    "offset": "0.00",
    "parametersTemplate": "720",
    "productInfo": "1,155,0,1,0,1,1,5",
    "pushNotificationID": "0",
    "pushNotificationType": "0",
    "remoteGatewayId": "0",
    "saveLogs": "true",
    "serialNumber": "h'00000001",
    "showFireAlarm": "true",
    "showFreezeAlarm": "false",
    "smsNotificationID": "0",
    "smsNotificationType": "0",
    "unit": "C",
    "useTemplate": "true",
    "userDescription": "",
    "value": "23.10"
  },
  "actions": {
    "reconfigure": 0
  },
  "created":<PHONE_NUMBER>,
  "modified":<PHONE_NUMBER>,
  "sortOrder": 174
}

I don't think it's this device (id 131) that causes the problem. It's rather one of: Climate Zone or Heating Zone? Do you have something like that? If so, can you paste the API response for them?

Also, paste the current error here (from version 1.4.0) please.

The problem seems to be related to Heating Zones indeed.
The log file reports [FibaroHC] Wohnbereich [131]: 23.0 C. "Wohnbereich" is a Heating Zone
The actual device 131 is the temperature sensor of a heating thermostat in a room in that heating zone.
We habe an edge case: Other than the rest of the heating zones, this zone has only one thermostat.
This is the error with v1.4.0:
[10/11/2023, 19:26:54] [FibaroHC] Temperatur 137.0.1 [139]: 25.5 C
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:06] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:07] [FibaroHC] power is not a number.
[10/11/2023, 19:27:59] [FibaroHC] Wohnbereich [131]: 23.0 C
[10/11/2023, 19:27:59] [FibaroHC] Wohnbereich [131]: 0 %
[10/11/2023, 19:27:59] [FibaroHC] Temperatur 129.0.1 [131]: 23.5 C
[10/11/2023, 19:27:59] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:110:36)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:20
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:27)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
[10/11/2023, 19:27:59] Got SIGTERM, shutting down Homebridge...

@Krillle please try next update (not version 1.4.0 but version above ). If this still does not fix the problem, write.

@mkz212 still same issue with v1.4.1:
[12/11/2023, 11:43:24] [FibaroHC] Wohnbereich [131]: 23.0 C
[12/11/2023, 11:43:24] [FibaroHC] Wohnbereich [131]: 0
[12/11/2023, 11:43:24] [FibaroHC] Temperatur 129.0.1 [131]: 24.4 C
[12/11/2023, 11:43:25] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:110:34)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:18
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:25)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
[12/11/2023, 11:43:25] Got SIGTERM, shutting down Homebridge...

Device 131 API response:
{
  "id": 131,
  "name": "Temperatur 129.0.1",
  "roomID": 4,
  "type": "com.fibaro.temperatureSensor",
  "baseType": "com.fibaro.multilevelSensor",
  "enabled": true,
  "visible": true,
  "isPlugin": false,
  "parentId": 129,
  "remoteGatewayId": 0,
  "interfaces": [
    "zwave"
  ],
  "properties": {
    "parameters": [
      {
        "id": 1,
        "lastReportedValue": 1,
        "lastSetValue": 1,
        "size": 2,
        "value": 1
      },
      {
        "id": 2,
        "lastReportedValue": 1,
        "lastSetValue": 1,
        "size": 2,
        "value": 1
      },
      {
        "id": 3,
        "lastReportedValue": 0,
        "lastSetValue": 0,
        "size": 2,
        "value": 0
      },
      {
        "id": 4,
        "lastReportedValue": 3,
        "lastSetValue": 3,
        "size": 2,
        "value": 3
      },
      {
        "id": 5,
        "lastReportedValue": 50,
        "lastSetValue": 50,
        "size": 2,
        "value": 50
      },
      {
        "id": 6,
        "lastReportedValue": 400,
        "lastSetValue": 400,
        "size": 2,
        "value": 400
      },
      {
        "id": 7,
        "lastReportedValue": 50,
        "lastSetValue": 50,
        "size": 2,
        "value": 50
      },
      {
        "id": 8,
        "lastReportedValue": 400,
        "lastSetValue": 400,
        "size": 2,
        "value": 400
      },
      {
        "id": 9,
        "lastReportedValue": 0,
        "lastSetValue": 0,
        "size": 2,
        "value": 0
      },
      {
        "id": 10,
        "lastReportedValue": 240,
        "lastSetValue": 240,
        "size": 2,
        "value": 240
      },
      {
        "id": 11,
        "lastReportedValue": 240,
        "lastSetValue": 240,
        "size": 2,
        "value": 240
      },
      {
        "id": 12,
        "lastReportedValue": 2,
        "lastSetValue": 2,
        "size": 2,
        "value": 2
      },
      {
        "id": 13,
        "lastReportedValue": 210,
        "lastSetValue": 210,
        "size": 2,
        "value": 210
      }
    ],
    "pollingTimeSec": 0,
    "zwaveCompany": "Heatit",
    "zwaveInfo": "6,3,83",
    "zwaveVersion": "1.5",
    "categories": "[\"climate\"]",
    "configured": true,
    "dead": "false",
    "deadReason": "",
    "deviceControlType": "0",
    "deviceIcon": "1004",
    "emailNotificationID": "0",
    "emailNotificationType": "0",
    "endPointId": "0",
    "log": "",
    "logTemp": "",
    "manufacturer": "",
    "markAsDead": "true",
    "model": "",
    "nodeId": "32",
    "offset": "0.00",
    "parametersTemplate": "720",
    "productInfo": "1,155,0,1,0,1,1,5",
    "pushNotificationID": "0",
    "pushNotificationType": "0",
    "remoteGatewayId": "0",
    "saveLogs": "true",
    "serialNumber": "h'00000001",
    "showFireAlarm": "true",
    "showFreezeAlarm": "false",
    "smsNotificationID": "0",
    "smsNotificationType": "0",
    "unit": "C",
    "useTemplate": "true",
    "userDescription": "",
    "value": "24.50"
  },
  "actions": {
    "reconfigure": 0
  },
  "created":<PHONE_NUMBER>,
  "modified":<PHONE_NUMBER>,
  "sortOrder": 174
}

Full log:
[12/11/2023, 11:43:35] [HB Supervisor] Starting Homebridge with extra flags: -I -P /var/lib/homebridge/node_modules --strict-plugin-resolution
[12/11/2023, 11:43:35] [HB Supervisor] Started Homebridge v1.7.0 with PID: 26851
[12/11/2023, 11:43:37] Loaded config.json with 2 accessories and 4 platforms.
[12/11/2023, 11:43:37] Loaded 85 cached accessories from cachedAccessories.
[12/11/2023, 11:43:38] ---
[12/11/2023, 11:43:38] Loaded plugin<EMAIL_ADDRESS>[12/11/2023, 11:43:38] Registering accessory 'homebridge-dummy.DummySwitch'
[12/11/2023, 11:43:38] ---
[12/11/2023, 11:43:39] Loaded plugin<EMAIL_ADDRESS>[12/11/2023, 11:43:39] Registering platform 'homebridge-dyson-pure-cool.DysonPureCoolPlatform'
[12/11/2023, 11:43:39] ---
[12/11/2023, 11:43:39] Loaded plugin<EMAIL_ADDRESS>[12/11/2023, 11:43:39] Registering platform 'homebridge-fibaro-home-center.FibaroHC'
[12/11/2023, 11:43:39] ---
[12/11/2023, 11:43:40] Loaded plugin<EMAIL_ADDRESS>[12/11/2023, 11:43:40] Registering accessory 'homebridge-http-switch.HTTP-SWITCH'
[12/11/2023, 11:43:40] ---
[12/11/2023, 11:43:41] Loaded plugin<EMAIL_ADDRESS>[12/11/2023, 11:43:41] Registering platform 'homebridge-irobot.iRobotPlatform'
[12/11/2023, 11:43:41] ---
[12/11/2023, 11:43:41] Loading 4 platforms...
[12/11/2023, 11:43:41] [FibaroHC] Initializing FibaroHC platform...
[12/11/2023, 11:43:41] [iRobot] Initializing iRobotPlatform platform...
[12/11/2023, 11:43:41] [DysonPureCoolPlatform] Initializing DysonPureCoolPlatform platform...
[12/11/2023, 11:43:41] Loading 2 accessories...
[12/11/2023, 11:43:41] [Alle weg] Initializing DummySwitch accessory...
[12/11/2023, 11:43:42] [Garagentor] Initializing HTTP-SWITCH accessory...
[12/11/2023, 11:43:42] [Garagentor] Switch successfully configured...
[12/11/2023, 11:43:42] [iRobot] Loading accessory from cache: Roomba
[12/11/2023, 11:43:42] [iRobot] Logging into iRobot...
[12/11/2023, 11:43:45] [iRobot] Configuring roomba: Roomba
[12/11/2023, 11:43:46] [iRobot] Restoring existing accessory from cache: Roomba
[12/11/2023, 11:43:47] [DysonPureCoolPlatform] Device with serial number NN2-EU-KKxxxxA: product type 475
[12/11/2023, 11:43:47] [DysonPureCoolPlatform] Device with serial number VS9-EU-KFxxxxA: product type 438
[12/11/2023, 11:43:47] [DysonPureCoolPlatform] Accessories initialized.
Setup Payload:
X-HM://002xxxxxxxW6T
Enter this code with your HomeKit app on your iOS device to pair with Homebridge:

    ┌────────────┐
    │ xxx-xx-xxx │
    └────────────┘

[12/11/2023, 11:43:48] Homebridge v1.7.0 (HAP v0.11.1) (Homebridge) is running on port 51194.
[12/11/2023, 11:43:48] [iRobot] Succefully connected to roomba Roomba
[12/11/2023, 11:43:49] [FibaroHC] Loading accessories
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:44:01] [FibaroHC] power is not a number.
[12/11/2023, 11:46:23] [FibaroHC] Temperatur 44.0.1 [46]: 24.6 C
[12/11/2023, 11:46:33] [FibaroHC] Temperatur 133.0.1 [135]: 29.2 C
[12/11/2023, 11:47:40] [FibaroHC] Wohnbereich [131]: 23.0 C
[12/11/2023, 11:47:40] [FibaroHC] Wohnbereich [131]: 0
[12/11/2023, 11:47:40] [FibaroHC] Temperatur 129.0.1 [131]: 24.5 C
[12/11/2023, 11:47:40] TypeError: Cannot read properties of undefined (reading '0')
    at GetFunctions.getCurrentTemperature (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/getFunctions.ts:197:85)
    at Poller.manageValue (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:110:34)
    at /var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:49:18
    at Array.map (<anonymous>)
    at Poller.poll (/var/lib/homebridge/node_modules/homebridge-fibaro-home-center/src/pollerupdate.ts:47:25)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
[12/11/2023, 11:47:40] Got SIGTERM, shutting down Homebridge...


Permission issue - "cms | placeholder | Can use Structure mode" not set, users can still access structure board

Summary
I appear to have run into an issue regarding User/Group permissions, specifically the
cms | placeholder | Can use Structure mode permission.
As I understand it, this limits a user from being able to access the structure board, and can only edit in content mode.
When this is not assigned to a user/group the structure board is still accessible to that user. It appears as though the permission is checked as the structure button (top right) that expands the structure board is removed. However, when double clicking to edit a plugin the structure board then expands.

The expanded structure board, with the cms | placeholder | Can use Structure mode not set, note the structure button is not visible.
Expected behaviour
The structure board should not be accessible when a user/group does not have the “cms | placeholder | Can use Structure mode” permission set.
Actual behaviour
The structure board can be accessed regardless of the cms | placeholder | Can use Structure mode permission. It can be accessed by editing a plugin (double click), which will then expand the structure board.
Environment

Python version: 3.6.4
Django version: 1.9.13
django CMS version: 3.5.0

Could you please check if the issue is reproducible on 3.5.1?

Actually, latest develop would be better, it might be this: https://github.com/divio/django-cms/commit/917d768e4ad277bc84e042a5fe9198a565597371


i want to know how to transform the dom file object to the file object which can be submitted to server

maybe ,you do not understand me,
that is ok ,
i will say it in detail ,
for example
<
input type="file" id="file1" />
var fileobj=document.getElementById("file1");
here is the dom file object.
but when pass the file to server ,we use the file parameter which is not the dom object.
but another object which contails file type ,file size ,and file name.
var files=[];
$upload.upload({
url: 'http://www.baidu.com/',
fields: {
'name': $scope.thegood.name
},
file: files
})
now i want to know how to transfrom the dom file object to the needed file object.

You can pass the other properties of the file using fields the same way you are sending the name.
$upload.upload({
url: 'http://www.baidu.com/',
fields: {
'name': $scope.thegood.name,
'type':$scope.thegood.type,
'size':$scope.thegood.size,
},
file: files
})

and then on the server handle that.


Scripts HTTP API

Summary:

Scripts queried through HTTP now instead of JSON. Uses HashMap cache.
rawHex and cborHex both exported.
/info endpoint to get revision and available scripts.

IIRC we can use Capture if we do want to do this in a dynamic way.

Hmm, good point. I'll look into it. We do need to merge this relatively soon, due to the blockage it has on off-chain, but it's relatively easy to migrate, thankfully.

Looks like it was a trivial change. Might as well make it now.


fix: remove aggressive timeout context setting middleware

Description
Remove a bunch of experimental machinery around controlling request context timeouts. This style of control leads to strange double-commit scenarios for the http.Response type and results in incorrect 504 responses where the client is behaving but behind a slow connection.
This changeset addresses https://github.com/SpecterOps/BloodHound/issues/13.
Motivation and Context
There's a long history of debug attempts related to the code being cleaned up here. To save everyone a long read, the gist is that this was an experiment in aggressively trying to preserve UX for multi-user, single-tenant instances that back BloodHound Enterprise.
How Has This Been Tested?
Integration test passes to validate that request handling operates as expected. One negative test harness written in Python that won't be committed.
Types of changes

[ ] Chore (a change that does not modify the application functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] Documentation updates are needed, and have been made accordingly.
[x] I have added and/or updated tests to cover my changes.
[ ] All new and existing tests passed.
[ ] My changes include a database migration.

It looks like this middleware is the only place that used the Prefer header. Currently every endpoint in our OpenAPI documentation has this parameter specified. We should probably remove that to reflect this change in functionality. The question is, should we do it in this PR or a separate one?

The comment about the Prefer header is a good question. I have a sneaking suspicion that it still might be worth supporting. We could pull the Prefer header logic out in its own middleware so that it can set user deadline expectations in the request context for further consumption downstream without each handler having to parse or inspect the Prefer header.
I think a follow up discussion around use of the header and its documentation is worth having with the team. This bug is painful enough, however, that I'm not keen on blocking merge of this change for long.

After further internal team discussion, I'm going to rewrite portions of this PR to include better plumbing for the Prefer header so that users can bypass default transaction time-outs per-request.

This changeset has been updated with follow-up work:

A new type in the BH context package has been added to track user timeout preferences.
Prefer header is no longer tossed out but instead is now parsed and stored in the request context.Context value for future introspection. The context timeout is now correctly preserved and may be relied on for downstream context-aware function calls.
Cypher queries now honor the Prefer header correctly.
Early cancellation of responses has been removed.


IDictionaryEnumerator not disposed

The call to IDictionary.GetEnumerator() seems to have an implicit allocation.
Is there a reason this code doesn't use a foreach loop so that the try ... finally ... Dispose pattern is automatically applied?
https://github.com/PowerShell/PowerShell/blob/fe5f43cdc30ae5e353d3dc6a18747b9d632df811/src/Microsoft.Management.Infrastructure.CimCmdlets/CimNewCimInstance.cs#L284

Does IDictionaryEnumerator implement IDisposable?
https://learn.microsoft.com/en-us/dotnet/api/system.collections.idictionaryenumerator?view=net-7.0
https://learn.microsoft.com/en-us/dotnet/api/system.collections.ienumerator?view=net-7.0

IDictionaryEnumerator does not provide any implementation. A more derived type may implement IDisposable as well as IDictionaryEnumerator.

IDictionaryEnumerator does not provide any implementation. A more derived type may implement IDisposable as well as IDictionaryEnumerator.

That may well be the case but I don't think that is part of the contract. If something returns an interface without IDisposable it is not expected that all code has to check to see if something is disposable or not. The typical pattern for IDisposable is in a "using" where you explicitly create something and have ownership over its scope. If the requirement is now to check to see if it is disposable then that is going to break about half of all C# code in existence, it is a complete change in the contract.
https://learn.microsoft.com/en-us/dotnet/api/system.collections.generic.dictionary-2.getenumerator?view=net-7.0
Does not mention any requirement to check for IDisposable. The main comment is the recommendation to use foreach.

So I fully support the recommendation to use foreach, because it is recommended by the documentation and it should make the code simpler, removing the need to explicitly call MoveNext().


Request to Add Search Functionality in Event Section ⚡

What would you like to share or ask?

I often find myself browsing the Event section of the community website to discover upcoming events, workshops, and conferences. However, as the number of events grows, it becomes increasingly challenging to locate specific events based on criteria such as topic, date, or location. Therefore, I would like to request the addition of a search functionality to improve the user experience and make it easier to find relevant events.

Additional information
Expected Behavior:

I kindly request the implementation of a search functionality in the Event section of the DevinTech community website. This feature should enable community members to search for events based on various criteria, including but not limited to:
Keywords: Users should be able to search for events by entering specific keywords related to the event title, topic, speaker, or any other relevant information.
Date Range: Users should have the ability to search for events within a specific date range, allowing them to find events that match their availability or interest during a particular time period.
Location: It would be beneficial to include a search option to filter events based on location. This would allow users to find events happening in their city, region, or country, facilitating local engagement.
Tags or Categories: Incorporating tags or categories associated with events would enable users to search for events that fall under specific topics of interest, making event discovery more efficient.

Hey @Nishitbaria Great idea

@Susmita-Dey ,please review and merge my pull request


Replace CSV exports with XLSX

Contacts
Diversity report
Reasonable Adjustments report

@warrensearle There's no need to keep the CSV exports, we only had them because we didn't have XSLX.


add pick_signal_channels to baserawio.py

@samuelgarcia
following #728 I've added a function to the baserawio.py, keeping the options for ordering and strict selecting the channels in other functions

Hello @sin-mike! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file neo/rawio/baserawio.py:

Line 484:100: E501 line too long (101 > 99 characters)

Hi @sin-mike.
Sorry for the super super long delay.
I prefer a lot this idea becaise it is generic.
However, I am wondering how this pick_signal_channels could return another reader and not modify inplace the actual reader.
I think the inplace modification can be dangerous.
What do you think ?

Hi @sin-mike.
Sorry for the super super long delay.
I prefer a lot this idea becaise it is generic.
However, I am wondering how this pick_signal_channels could return another reader and not modify inplace the actual reader.
I think the inplace modification can be dangerous.
What do you think ?

Hi Samuel,
I've been using this feature since then in this exact version, and I didn't observe any drawbacks in our processing pipelines. However, I understand your concern. My suggestion then is to make a protected field _header, so that the function can change only the public header, while the _header is kept untouched. And make a self.reset_header function
Mike

Hi @sin-mike
sorry again for this super long delay.
I like a lot the  user_order=False, strict_selection=True but I think it is not possible to include the pick_signal_channels (inplace modification) in the neo master/
I think this will lead to some border effect for some rawios even if if works for you.
Maybe you you remove remove this pick_signal_channels right now so I could merge the user_order=False, strict_selection=True and then you could make another PR that mimic the  pick_signal_channels(..) but instead of inplace modification it could just duplicate the the rawio instance and mofify the header. It would be less dangerous.
What do you think ?

Hi @sin-mike.
Sorry for the discontinuous discussion about this old PR.
We will release neo 0.9.0 in september.
Do you think you could a bit more work to test my last idea or would you prefer to prefer to postpone this PR in september.
My last idea is basically to have something like this:
import neo

rawreader = neo.rawio.MyFormetIO(filename=...)  # this have all channel
# this is a copy of the rawreader but with less channels
subrawreader = neo.rawio.tools.pick_signal_channels(rawreader)

This would avoid an inplace modification which is dangerous in my mind.
What do ou think ?

Hi Mike,
as the relase date is approaching I propose to postpone this to 0.10.
Cheers

Dear Mike.
I have work a bit rawio API here #949
The first change you propose here is not possible (in place channel modification).
As I said having another class doing it would be totally ok.
The second change  strict_selection=True is no more relevant in the rawio refactoring becaise ids have to be unique inside a stream and this is tested always.
About user_order=False I think this is safer to always keep order.
So I close this PR.
Thanks again for you participation in neo.


Can it be combined with Selenium + Python ?

I think this is a very nice library but I don't know if it can be combined with Selenium with Python? Thanks.

Hello! Sorry, I'm not @petrpatek but I hope I can answer your questions just as well :)
The support for this is still unofficial, so it requires a bit of hacking. Consider the following snippet:
import {FingerprintInjector}  from 'fingerprint-injector';
import {FingerprintGenerator} from 'fingerprint-generator';

const injector = new FingerprintInjector();
const generator = new FingerprintGenerator();
const fingerprint = generator.getFingerprint({
    'devices': ['desktop'],
    'operatingSystems': ['windows'],
    // ... your fingerprint options
});

const injectable_code = injector['getInjectableScript'](fingerprint);
// driver.execute_async_script(injectable_code) ...?

See the second line from the bottom - calling injector.getInjectableScript with the fingerprint as an argument returns JS code (as string), which you can then run in your browser (using Selenium, any other browser automation tool, or even copy-paste it to the browser console). With some extra elbow grease, you can even pass this string to your Python Selenium script, if that's the case.  Note that we use the square bracket property accessor here (instead of .), as getInjectableScript is a private property of the FingerprintInjector class.
Last but not least - this handles only part of the injection - make sure to set the browser launch arguments (e.g. screen size, etc.) correctly, so it's in line with the "in-browser" injected features.
In case of any problems, feel free to leave another comment. Thank you for reaching out to us!

Sorry for the one last ping - closing this with wontfix, but the working solution is in the comment above :)

inject a javascript code

Hi, selenium+chrome could inject a javascript code just by few lines code if there is a js inject file!

selenium+chrome support:
If could combine fingerprint's package into a single Js file, could execute it directly in python to complete the injection, which takes about 3 lines of code. All need to do is call a python wrapped method, which automatically interacts with selenium. Instead of programming directly to selenium, here is the code for the python example:
with open(f"fingerprint*.js", encoding='utf-8') as f1...
driver. execute_cdp_cmd (' fingerprint *. Js')


Including license files

Thanks for developing this useful package. I needed a quick way to view parquet files from the terminal, and this package works great.
I am working on a submission to conda-forge. This will create conda binaries for all operating systems, and also auto-update whenever you release a new version to PyPI.
https://github.com/conda-forge/staged-recipes/pull/15926
There are a few license issues I have to address before the recipe is accepted:

The license is MIT but there is no file included that contains the license text, which is a requirement of the MIT license. This can be addressed by adding a license file and then listing it in MANIFEST.in
This package vendors parquet.thrift, which is released under the Apache License, which also requires that it is included in the distributed software

Ideally we could add these here in the source repo. I'm happy to send a PR if you'd like. Otherwise I can address them in the conda recipe. As a previous example, here is a previous PR of mine to include the package license as well as the license of the vendored file: https://github.com/randy3k/lineedit/pull/4

Actually, since you are using poetry, you should be able to specify the license files in the include field of pyproject.toml instead of using MANIFEST.in.

I attempted to address these issues in PR #16. Also, would you like to be added as a maintainer of the conda-forge recipe? It requires minimal effort, essentially merging the occasional PR.


[React] Set changeOrigin: true for proxied Sitecore requests in connected mode

Description / Motivation
When running React in connected mode against development environments, proxied Sitecore requests (visitor identification, media, etc) may result in 504 (Gateway Timeout) errors. Setting changeOrigin: true resolves this issue.
Testing Details

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation update (non-breaking change; modified files are limited to the /docs directory)

This fix also fixes 404 errors. See this topic for more info: https://sitecore.stackexchange.com/questions/31227/sitecore-requests-give-a-404-in-connected-mode


feat: merchant-fulfillment

Codecov Report

Merging #113 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #113   +/-   ##
=======================================
  Coverage   99.65%   99.65%
=======================================
  Files          23       23
  Lines        1171     1171
  Branches       73       73
=======================================
  Hits         1167     1167
  Misses          4        4

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76ccb91...bf7754f. Read the comment docs.

@moltar I fixed the issue with the mock tests, fix api-extractor issues and added a todo comment, can I merge this now?


Error when querying a Lance dataset

Hello,
thank you for your earlier support, I am now testing Lance without dictionary columns.
I however have the following problems:

Querying a Lance dataset with duckDB does not accept the WHERE clause
I'm not able to connect Polars to a Lance dataset

Here is the schema of my dataset:
InvoiceNo: large_string
StockCode: large_string
Description: large_string
Quantity: int64
InvoiceDate: date32[day]
UnitPrice: double
CustomerID: large_string
Country: large_string
Sales: double
Here is how I initialize the datasets:
dataset = pa.dataset.dataset('data_clean.parquet', format='parquet')
lance.write_dataset(dataset, "data_clean.lance")
dataset_lance = lance.dataset("data_clean.lance")

Regarding the WHERE clause:
# Works
duckdb.query("SELECT * FROM dataset_lance LIMIT 10")

# Works
duckdb.query("SELECT * FROM dataset WHERE StockCode = '84406B' LIMIT 10")

# Does not work
duckdb.query("SELECT * FROM dataset_lance WHERE StockCode = '84406B' LIMIT 10")
# error: arrow_scan: get_next failed(): Invalid: C Data interface error: Io error: Internal error: Cannot evaluate binary expression Eq with types LargeUtf8 and Utf8. This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

Now regarding loading to Polars:
import polars as pl

# works
pl.scan_pyarrow_dataset(dataset)

# does not work
pl.scan_pyarrow_dataset(dataset_lance)
# Error: TypeError: self.dataset,self.wrapped cannot be converted to a Python object for pickling

Let me know if you need any more information

This is really helpful thanks!

Filter error: we're using datafusion under the hood for filter predicate evaluation. The error message looks like it's saying that the column and the filter value are LargeUtf8 and Utf8 and datafusion does not know to automatically cast here. We'll file a github issue upstream and see if there's a good way to deal with it from Lance.
polars => we'll look into this
There was a question from #715 "If I understand, you do something similar to Delta Lake for tracking different versions of the file?"  => yes, conceptually they're similar. main difference is that lance manage the versions and metadata within the data directory and it's not specifically optimized for spark.
We're releasing v0.3.17 soon with the list-of-dict support. let us know if it works for you?

hi @gsilvestrin , you are right. I had not restarted the kernel. I confirm that Lance works well with Dictionaries now.
The file is now working well, so I would like to try Lance on some bigger data.
I am trying to reproduce the tests on https://blog.eto.ai/benchmarking-random-access-in-lance-ed690757a826.
Here is my attempt:
https://colab.research.google.com/drive/1iG1MXJV-9hrqm4YcO_MzR0Qt67sMFfEj?usp=sharing
Here is the error I have:
OSError: LanceError(Arrow): Invalid argument error: Invalid UTF8 sequence at string index 0 (0..3): invalid utf-8 sequence of 1 bytes from index 1
Would you know what the reason can be?

FYI: I shared about Lance on LinkedIn and tried to reproduce the random access performance of Lance.
https://www.linkedin.com/posts/lucazanna_data-lance-parquet-activity-7044448500830359552-6az2
It looks good!
Look forward to the improvements mentioned in the blog post regarding predicate pushdown. And working with Polars :)
I don't write in Rust (yet) but if I can contribute to the Lance project with something else, for example testing feel free to let me know

awesome! Testing is super helpful. please keep the feedback coming :)
The fix is in review in #727

The filter error on LargeUTF8 array should be fixed in #733 , which will be included in the next release

thank you @eddyxu! This means that everything is mentioned will be fixed in the next release.
So only thing missing would be loading to Polars, which probably it's more complicated to fix


Sustainfest(?), launching a new landing page and campaign for getting more contributors involved

As an organisation, we would like to get contributors more involved with our proposed projects, so that our community can grow and we can facilitate real-life impact & growth.
The suggestion here is to create a landing-page, detailing issues that will be good for first-time contributors to a given project, along with a guide on how to get involved in said projects.
The issues should be filterable by "language" and "category".
The page should also hold a bit of a "manifesto" or "guide".

Meaning we need 4 things:

A backend/api for delivering the issues
A frontend for showing this information / pulling the issues
A marketing campaign for pushing this out there, both with a strategy for social media and with our partners
A method of counting/tracking contributions via this campaign (perhaps via #hastag in the pr/issues)

As discussed with @Ly0n and @andrew. 🙌

I'm happy to take the lead on pushing that out there, and will be starting to work on the landingpage in the next days. 👍

Small note: Sustainfest might be confusing. Open Sustainable Tech Fest, maybe?

Small note: Sustainfest might be confusing. Open Sustainable Tech Fest, maybe?

Absolutely, several aspects of the naming and campaign still need to be determined. Perhaps there's some way we could track that here too, although it is a bit marketing focused, what do you reckon?

Marketing is how your project becomes known - it's a core part of the process.
One thing I would say - I'm not sure setting up a mechanishm for first time contributors outside of the project is as useful as showcasing the value of the project to contributors through the main access points. What's in it for contributors? That question is more important than surfacing small issues that they may or may not be able to do.

Small note: Sustainfest might be confusing. Open Sustainable Tech Fest, maybe?

We should not name it SustainFest, as this would lead to confusion with SustainOSS. Other option could be OpenSustainFest or OpenSustainabilityFest. We could also remove the Fest completly and call it OpenSustainMatch :partying_face:

One thing I would say - I'm not sure setting up a mechanishm for first time contributors outside of the project is as useful as showcasing the value of the project to contributors through the main access points. What's in it for contributors? That question is more important than surfacing small issues that they may or may not be able to do.

Maybe some background here. What comes a little bit short in the description, is that we would not like to highlight the issues of this projects but from all project listed on OpenSustain.tech. Many people contacted me in the past, looking for open source project in the sustainability sector they can contribute to. What we are planning to do is to match projects with people. @andrew already did most of the backend work: https://ost.ecosyste.ms/issues
This page just showes the open issues from OpenSustain.tech that are labeled with firstgoodissue, hacktoberfest and help wanted. The current database is just to hard for people to browse and they get lost in to many projects. Also most of the projects demand a lot of domain expertise. With this page we can help guide people to projects that are beginnner friendly but still targeting a sustainability goal.
In a first iteration we would like people to be able to filter issues by project domain ( like Photovolatic) and programming languages (Scala).
What @RichardLitt is right about, is the fact that we maybe should not show people the issue directly but the projects that are having this labels.

What's in it for contributors?

We do have some ideas for OpenSustain.tech. One is already in the issues #199.  The other will be that contributors can create a note on the website landing page on a prominent position.


[BUG] find(..., fetch_links=True) does not work on Documents with is_root=True

Describe the bug
*beanie=1.17.0
I have three objects, BookingInfoBase, BookingInfoDoc1 and BookingStatusDoc1
BookingInfoDoc1 class inherits from BookingInfoBase and it is also linked to BookingStatusDoc1.
See the structure in python in the followings:

class BookingStatusDoc1(Document):
    id: str
    is_default: int
    description: Optional[str] = None

class BookingInfoBase(Document):
    """
    # BookingInfoBase
    #       |
    #       |
    # BookingInfoDoc1
    """
    company: str
    id: str

    class Settings:
        is_root = True
        name = 'BookingInfoCollection'

class BookingInfoDoc1(BookingInfoBase):
    record_date: str
    start_date: str
    end_date: str
    status: Optional[int] = None
    status_payload: Optional[Link[BookingStatusDoc1]] = None

I query BookingInfoBase via find with both fetch_links=True and with_children=True.
The find'method returns BookingInfoDoc1 with BookingStatus as an unfetched link object, namely:  <beanie.odm.fields.Link object at xxxxx>.
To Reproduce
search_result = BookingInfoBase.find(fetch_links=True, with_children=True).to_list()

for booking in search_result:
   print(f'booking type: {type(booking)} \n \n data is: {booking})

The returned value are as the followings:
booking type:
     <class 'booking.model.mongo_db.booking.BookingInfoDoc1'>
data is:
     id='smarteroa-5668'
     company='smarteroa'
     record_date='2023-01-28 17:42:34'
     start_date='2023-02-02 19:00:00'
     end_date='2023-02-02 20:30:00'
     status=smarteroa-1
     status_payload=<beanie.odm.fields.Link object at 0x7fcbe90acd60>

Expected behavior
I except the returned value be the following:
booking type:
     <class 'booking.model.mongo_db.booking.BookingInfoDoc1'>
data is:
     id='smarteroa-5668'
     company='smarteroa'
     record_date='2023-01-28 17:42:34'
     start_date='2023-02-02 19:00:00'
     end_date='2023-02-02 20:30:00'
     status=smarteroa-1
     status_payload=
          BookingStatusDoc1(
               id='smarteroa-1'
               is_default=0
               description:"yet to confirm"
          )

Additional context
I following the demotration on the Documentation-Inheritance though the demo did not use is_root and link to its children.
I manage to figure out a work-around by re-iterate search_result and use fetch_all_links to its item but this is cumbersome.
search_result = BookingInfoBase.find(fetch_links=True, with_children=True).to_list()

for booking in search_result:
   await booking.fetch_all_links()
   print(f'booking type: {type(booking)} \n \n data is: {booking})

I wonder if there is any parameter is missed to have the aforementioned inheritance-link case to work elegantly?

Hi, thank you for the issue. I'll reproduce and fix this if this is a bug.

I have checked this, and at this moment, it is impossible to fix as it is not a mistake. For link prefetching, I use aggregation pipelines. To support this case, I have to run different aggregation pipelines for different child classes and merge the result there in the aggregation to make the whole operation atomic. I don't say that it is impossible, but it requires a lot of experimentation.
I have converted it from a bug to a feature request.


(sqlite3.OperationalError) database is locked

After kicking off the following commands:
# bcbio_nextgen.py -w template gatk-variant project1.csv /atlas/garvin_inst_2014/raw_reads/NA12878D_HiSeqX_R1.fastq /atlas/garvin_inst_2014/raw_reads/NA12878D_HiSeqX_R2.fastq

# bcbio_nextgen.py ../config/project1.yaml

I get:
/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/matplotlib/__init__.py:872: UserWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.
  warnings.warn(self.msg_depr % (key, alt_key))
[2015-11-28T06:02Z] Resource requests: bwa, sambamba, samtools; memory: 2.00, 2.00; cores: 16, 16, 16
[2015-11-28T06:02Z] Configuring 1 jobs to run, using 1 cores each with 2.00g of memory reserved for each job
[2015-11-28T06:02Z] Timing: organize samples
[2015-11-28T06:02Z] multiprocessing: organize_samples
[2015-11-28T06:02Z] Using input YAML configuration: /athena/bcbio_test/project1/config/project1.yaml
[2015-11-28T06:02Z] Checking sample YAML configuration: /athena/bcbio_test/project1/config/project1.yaml
Traceback (most recent call last):
  File "/usr/local/bin/bcbio_nextgen.py", line 226, in <module>
    main(**kwargs)
  File "/usr/local/bin/bcbio_nextgen.py", line 43, in main
    run_main(**kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 37, in run_main
    fc_dir, run_info_yaml)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 80, in _run_toplevel
    for xs in pipeline.run(config, run_info_yaml, parallel, dirs, samples):
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py", line 139, in run
    [x[0]["description"] for x in samples]]])
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 28, in run_parallel
    return run_multicore(fn, items, config, parallel=parallel)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py", line 86, in run_multicore
    for data in joblib.Parallel(parallel["num_jobs"])(joblib.delayed(fn)(x) for x in items):
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 657, in __call__
    self.dispatch(function, args, kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 404, in dispatch
    job = ImmediateApply(func, args, kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py", line 142, in __init__
    self.results = func(*args, **kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py", line 50, in wrapper
    return apply(f, *args, **kwargs)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py", line 226, in organize_samples
    return run_info.organize(*args)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 55, in organize
    item = add_reference_resources(item)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/run_info.py", line 139, in add_reference_resources
    data["genome_resources"] = genome.get_resources(data["genome_build"], ref_loc, data)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/genome.py", line 41, in get_resources
    return _ensure_annotations(cleaned, data)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/genome.py", line 50, in _ensure_annotations
    resources["rnaseq"]["gene_bed"] = gtf.gtf_to_bed(transcript_gff, out_dir)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/gtf.py", line 58, in gtf_to_bed
    db = get_gtf_db(gtf)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/rnaseq/gtf.py", line 42, in get_gtf_db
    return gffutils.FeatureDB(db_file)
  File "/usr/local/share/bcbio/anaconda/lib/python2.7/site-packages/gffutils/interface.py", line 130, in __init__
    ''')
sqlite3.OperationalError: database is locked

Any idea what this might be?
-Alex

Alex;
Sorry about the issue. This normally occurs in cases where another process is writing a shared sqlite3 database, or a previous processed died leaving a lock on the file. I'm not sure what genome you're working from, but if you look at the equivalent of:
/path/to/bcbio/genomes/Hsapiens/GRCh37/rnaseq/ref-transcripts.gtf.db

Are there lock files or active processes writing to this (say, from a simultaneous install)? If not, you could try removing the *.db file then re-running the installer (not as root, from the other thread) to re-build it.
Hope this helps fix things for you.

Should I not run the installer as root?

Alex;
No, I'd avoid this. It's not a great idea to run anyone's third party script as root. In bcbio's case, this would also require using root/sudo for every update since the installed files will be owned by root. If you want to avoid any manual chmod steps, run the installer as your regular user with the --sudo flag which will chmod directories to avoid needing to use root in the future. Hope this helps.

Should I not run the installer as root?

I've run the installer as root and as a regular user now and attempted to wipe the rna-transcripts.gtf.db file and rerun the installer all together several times now. This seems successful every time but when I go to run it I keep getting that same error message. I can access this file by manually using sqlite3 but bcbio_nextgen.py gives this error message.
I'd really like to get this to work. Is there anything else you can recommend to get this to work? Maybe I have a configuration off or something... I've followed the automated installer instructions multiple times now and satisfied every error I've encountered but still nothing.
Thanks,
Alex

Alex;
Sorry about the continued problems. Is there anything special about the filesystem that might be causing problems? I know sqlite can have issues on some shared or samba mounted filesystems, so that might explain the issue.
I also pushed a fix to try and work around this. Since the BED file should exist from the data installation process, I avoid trying to get the GTF database unless it's missing. Hopefully that will skip over this step and let you get things running. If you upgrade with:
bcbio_nextgen.py upgrade -u development

it will grab this fix and hopefully get you going. Hope this helps.

Brad,
Good call on the filesystem. I happen to know that any remote sqlite databases will only allow read privileges. If there is writing going on that might explain the issue. I am using an NFS share to house all my data for the run. I suppose that might be recognized as remote  from sqlite's stance.
I updated to the latest development build of bcbio and still no luck sadly.
I will make attempts to put all data on a local disk but honestly this kind of destroys the paradigm of the product in my eyes. Sqlite's shortcomings here are reflecting in the pipeline unfortunately.
Cheers,
Alex

Hi @aaiezza,
That database is never written to, only read from. Can you access the database with sqlite3 if you do it from the node the job is running on?

Could you post up a ls -l of the files in the GRCh37/rnaseq directory?

@roryk I gave that a second that after posting actually. Also I haven't parallelized anything yet; just running on a single machine with 16 cores and 30Gb of memory. I can access the database file using sqlite3 as is.

I've found in forums however that reference multiple processes attempting to connect to the same db file and the goto solution seems to be an increase in connection timeout. I don't see how that is applicable here nor would I know how to add that.

If you download https://gist.githubusercontent.com/roryk/0868086be6490e303d99/raw/8440547af492a42cdf7d92bf69cc07dd1302b627/test_gffutils.py and run it with:
/path/to/the/installed/bcbio/python test_gffutils.py /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/rnaseq/ref-transcripts.gtf

Do you encounter the same error?

New error

Hi Alex,
Sorry you'll have to provide the path to the bcbio-installed python. It looks like it should be in /usr/local/share/bcbio/anaconda/bin/python

I see. Unfortunately I'm not incredibly familiar with python. I do have that anaconda directory but how do I add it?
Thank you by the way for the help!

Hi Alex,
No worries-- if you run this:
/usr/local/share/bcbio/anaconda/bin/python test_gffutils.py /usr/local/share/bcbio/genomes/Hsapiens/GRCh37/rnaseq/ref-transcripts.gtf

it should work.

Ooh I see. I didn't expect that.
Running that now yes I get the same error:

How is /usr/local being mounted? If you copy like ref-transcripts.gtf and ref-transcripts.db to your home directory and rerun the test_gffutils.py script pointing at them, does it work? I am thinking that something weird is going on with the mount somehow that is confusing sqlite. If you connect with sqlite3 and actually try to perform a query, say with:
select * from features limit 1;
does it give you the locked error?

Here's how it is mapped:

This is because I don't have enough room on the local drive:

But you are right on that attempt there:

When I move it to a local directory I am able to access it. This is confusing though and annoying. Does this mean I can't use any NFS shares because sqlite is finicky?

Now we're at something I'm not too good at. We use NFS shares all the time so it should be okay. Here is an example setup of one that works, if any of the options help:
hsphfs4:/srv/export/hsphfs4/share_root on /net/hsphfs4/srv/export/hsphfs4/share_root type nfs (rw,nosuid,nodev,vers=3,intr,sloppy)

I see. I'll mess around with some of those options in the mount and see if something changes.
Thanks so much for your help.

Hi,
what @roryk discovered remembered to @xaxis3 that nolock option should do the job. something like this in the FSTAB configuration.
babbage:/home/Shared    /media/shared   nfs     nolock,...

YES!

You guys are champs!
It's running great. Thank you all! Truly great debugging.

Awesome-- what was the change that worked?

@roryk Literally adding the nolock option to the fstab.

@lpantano saves the day. Also, hello fellow RIT alum. :)

Nice!
I would say @roryk and @xaxis3 (Judit :) no better than a sys admin to ask) saved it.

On Nov 30, 2015, at 21:12, Rory Kirchner<EMAIL_ADDRESS>wrote:
@lpantano https://github.com/lpantano saves the day. Also, hello fellow RIT alum. :)
—
Reply to this email directly or view it on GitHub https://github.com/chapmanb/bcbio-nextgen/issues/1123#issuecomment-160825850.

Beer is on me Judit.

Good times! I'm in. @roryk when did you graduate?

I updated the GRCh37 RNA-seq tar file, so now the BED file is included along with everything else and doesn't have to be generated from the gffutils database.


Remove 'None' from changeset tag when using blank imagery layer

When using the None imagery layer (displays no background imagery and very useful), it gets added the changeset's imagery_used= tag. Would it be possible to remove this from being added to the tag?
Example changeset

I agree, including "None" in there is kind of pointless :)


Remove manual download instructions for Keptn CLI

Section 4 includes instructions to manually download the current release of the CLI as well as instructions for installing it with curl and brew.  To simplify the tutorial, we should remove the manual download instructions and point to the documentation for that option.
Note that this means that the tutorial is problematic for Windows users but, at the end of this section, we say that the rest of the tutorial is really only for Linux/macOS so that seems appropriate.  We should probably put that caveat in Section 1.

stale


Extend  test_make_sure_we_dont_remove_any_command to cover upsert-validator show

Problem: test_make_sure_we_dont_remove_any_command needs to be aligned to cover upsert-validator show
Solution: Extended the test

Codecov Report

Merging #2499 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2499   +/-   ##
=======================================
  Coverage   91.48%   91.48%
=======================================
  Files          41       41
  Lines        2397     2397
=======================================
  Hits         2193     2193
  Misses        204      204


Milestone #4: Display number of likes

In this PR, the changes are as follows:

Add the number of likes per item using Involvement API
Improve Responsiveness
Improve UX
Fix typo
Improve SVG Like Icon

Hi @lordksix  ,
Your project is complete! There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
Highlights
Clear all completed tasks ✔
Used webpack✔
Used GitHub flow ✔
Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.
As described in the Code reviews limits policy you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.

Hi @lordksix  ,
Your project is complete! There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
Highlights
Clear all completed tasks ✔
Used webpack✔
Used GitHub flow ✔
Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.
As described in the Code reviews limits policy you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.


Fix typos in logging messages, possible leftover bug from #11765, and remove two unused functions in WMCore/Database

Impact of the new feature
While looking for typos in most of the python code (that's specifically printed out), I found a seemingly unfixed part of the PR #11765 , see https://github.com/dmwm/WMCore/pull/11765#discussion_r1369296362 which still seems to be there: https://github.com/dmwm/WMCore/compare/master...oljemark:WMCore:custom-typo
Should this be fixed, or was this a DB field that was created with a certain typo and now has to be kept as is?
I also found that WMCore/Database/CmsCouch.py had two seemingly totally unused functions, getPrevisousRevision() & non_rotating_commit(), that were suspected as such already in #6928 .
The log text typos I found are here: https://github.com/dmwm/WMCore/compare/master...oljemark:WMCore:typo and error message & handler watchword typos here https://github.com/dmwm/WMCore/compare/master...oljemark:WMCore:twiki-codes
In the latter case, can I assume WMCore/Agent/Flow/DefaultFlow.py is not used anywhere, but just an example to be filled in by the user?
Is your feature request related to a problem? Please describe.
Describe the solution you'd like
If you think any of these corrections could be acceptable, I could open up to three PR's based on the branches above, and https://github.com/dmwm/WMCore/compare/master...oljemark:WMCore:remove2old , one each.
Describe alternatives you've considered
Additional context

Hi @oljemark , the first typo is very much undesired and I would appreciate it if you could make a PR out of your development branch.
The other 2 changes would be nice to have as well (I think that revision function for CouchDB does not even work in CouchDB 3.x series). So feel free to open the relevant PRs at your convenience and link them to close this one ticket, please.
I will be sorting out Jenkins today, as now we will definitely need to see the outcome of the CI pipeline. You might be getting another GitHub email at some. Thanks

@oljemark you should have received an email to join "dmwm" GitHub organization. Can you please confirm once you have accepted that, such that I can re-trigger the CI pipeline?

Hello @amaltaro . I didn't see any email now, but I had got an invitation Thursday which I thought I had accepted. Clicking it now only went to dmwm/WMCore webpage, but now when I went to github.com/dmwm I saw an invitation that I accepted.

Great, I think we are all set and I see Jenkins results for one of the PRs. We can follow up on those PRs now, if needed.
I guess you will not be able to see details inside the jenkins jobs, but I can help you out as needed.
Even though we have moved our documentation over to CERN Gitlab, this Jenkins documentation should be mostly up-to-date: https://github.com/dmwm/WMCore/wiki/Understanding-Jenkins, in case you want to learn more about what we test in each pull request. Thanks Fredrik!


Add finalizer before creating clients

Signed-off-by: zhujian<EMAIL_ADDRESS>Fix: https://github.com/open-cluster-management-io/registration-operator/issues/215

/assign @qiujian16 @zhiweiyin318


From having worked with the PL test suite and debugging tests, I do heavily prefer test functions that are "as dumb as possible". Essentially it should be easy to isolate a test case without having to stitch various pieces of code together from different locations in the test file. I believe the core team has also adopted this strategy.

          From having worked with the PL test suite and debugging tests, I do heavily prefer test functions that are "as dumb as possible". Essentially it should be easy to isolate a test case without having to stitch various pieces of code together from different locations in the test file. I believe the core team has also adopted this strategy.

A few things to keep in mind when writing tests:

each piece of logic being tested should live in a separate test function
only parametrize tests if the same logic is tested with different input values
prefer self-contained test functions with duplicated code if necessary (within reason)
avoid test function with too much logic, that is overly modular / parametrized, or makes heavy use of fixtures or other setup code

Originally posted by @dime10 in https://github.com/PennyLaneAI/catalyst/pull/98#discussion_r1190445420

Why is this a GitHub issue? Also the title is way too long

Just pressed "reference in the new issue button". I believe having an issue would be a best way to both discuss and merge the guidelines.

We could certainly use a contribution guide, which we are currently lacking 👍


RePoE Data no longer maintained

https://github.com/brather1ng/RePoE is no longer maintained.
I looked into it and found a lot of promising tools to get the data myself, including crafting costs for harvest recipes (for future changes and cost of harvest crafts).
Lightweight tool to look around the data: https://snosme.github.io/poe-dat-viewer/
poe dat viewer also offers a library to extraxt data from the patch cdn of poe (the complete ggpk file is not needed with this method)
following https://github.com/poe-tool-dev/dat-schema it is possible to export via json.
Question is if the switch is necessary.
I could provide the additional tooling for the data, and a up to date repository for kalandralang to pull the data from.
Further more this would change the json structure of the files. Probably could do a hacky translation to RePoE Format, but it could break in the future or kalandralang uses the new format of the future repository (which will basically be the dat-schema format)
For the language files (mod stats) I have to cook my own thing, could reimplement RePoE schema or an entirely different one, depending on your input.

Thanks for the heads-up, I was not aware that it was no longer maintained!
I had a very quick look at dat-schema. If I understand correctly, it describes the format of GGPK files (more precisely, the data files inside the GGPK archive). It could be used to generate a parser for dat files in GGPK files, in any language. That sounds very useful. I didn't look at whether dat-schema had the information for all the files that Kalandralang needs though.
(Fun fact: a few years ago I had written OCaml code to extract and parse data files directly from the GGPK file. The code was not very good quality unfortunately so it's not really usable again. It was a lot of efforts though.)
In order of difficulty for me, from the easiest to the hardest:

someone starts maintaining something that looks like RePoE (same file names, same format), but it looks like this is also the least sustainable option if community efforts are now focusing on something else;
someone generates JSON files (or YAML or XML or whatever as long as there is a parser for it in OCaml) that contain the same data, with a similar format to RePoE, just different field names and perhaps some minor structural changes; and someone hosts those JSON files somewhere; all I have to do is fix field names, so it's easy;
same as the previous item, but with more structural changes (data in different places, need a transformation to get back what Kalandralang understands): depending on the changes and how intuitive the data format is, it can be easy or hard;
I use dat-schema to generate OCaml code that generates something that Kalandralang understands (whether I, or someone, then hosts this data for Kalandralang to fetch, or whether Kalandralang just asks the user for the location of the GGPK file, is also a question).

I would obviously have enough time for the first solution (it's just a URL to change). I would also have time for the second for sure. I would probably have time for the third solution, depending on how different the file structures are. I would probably have a lot of fun doing the fourth or fifth option, but I don't foresee myself having enough time for it in the near future.
I hope this answers your question. If you think you can hosts JSON files in some format somewhere, I'll be happy to discuss the file format to see how hard it would be to migrate Kalandralang.
There is also the question of long-term sustainability; do you happen to know:

why RePoE is no longer maintained?
whether dat-format will likely be maintained?
is there some community consensus about how to proceed?


feat: integrate boostlook in build.js task

Missing from #308

Not looking good because the changes in boostlook didn't make it to master


Modpack updater dialog has broken modality and window ownership

The modpack updater dialog does not correctly handle window ownership, especially when multiple modpacks are open.
Closing the modpack updater dialog will often select the modpack window that was first opened.

One thought is to make the modpack update dialog's modality be none then have it say that it's editing all the mods that it collected in the update collection process.

I haven't implemented the feature where mods collected in the update process are added to the list of editing mods, but I believe that 8f8d484 does enough to close this issue.


ContainerService 2019-06-01 API

Included in this PR:

Agent pool get versions API to get a list of available versions to use when creating agent pool
Agent pool get upgrade profile API to get a list of available versions to upgrade for an agent pool
Added node taints property to agent pool.
Added ScaleSetPriority and ScaleSetEvicitionPolicy to agent pool

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[x] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[x] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[x] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-js

Automation for azure-sdk-for-python

Automation for azure-sdk-for-java

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-go

Can one of the admins verify this patch?

@antmarti-microsoft if your issues have been addresses can you please approve?  Your request for changes is preventing merging of this PR.


[api] expose API to obtain service ALOC for given service ID

As discussed on the openthread-users group  a PR to expose API

@googlebot I signed it!

@pdgendt , thank you for this contribution! It looks like the Google CLA checker is not able to match the email address used in your commit with the one you used as part of the Google CLA agreement. Can you double check on that?

@googlebot I signed it!

@pdgendt , thank you for this contribution! It looks like the Google CLA checker is not able to match the email address used in your commit with the one you used as part of the Google CLA agreement. Can you double check on that?

Yes, I messed up with the CLA, that I signed with a non-authorized title, so it got voided. Our CEO will sign it, I just hope I didn't break the CLA check for my account.

@googlebot I signed it!

@googlebot I signed it!

@pdgendt , thank you for this contribution! It looks like the Google CLA checker is not able to match the email address used in your commit with the one you used as part of the Google CLA agreement. Can you double check on that?

Yes, I messed up with the CLA, that I signed with a non-authorized title, so it got voided. Our CEO will sign it, I just hope I didn't break the CLA check for my account.

@jwhui CLA fixed!


Fix #64 data race due to shallow copy.

fix #64, deep copy instance to avoid data race.

I also found this problem, your adjustment plan should be correct


[20.09] cantata: add perl for dynamic playlists

Cantata wants to check if perl is available at runtime, but we already patch the
script shebangs, making it unnecessary to be available at runtime --
thus, patch out this check.
(cherry picked from commit fca7cc57ad7ff3f72324636ed3a48d51ed29019f)

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

cc @worldofpeace

Thanks @cole-h :sparkles:


Do a power budget for the 12v and 24v DV batteries

Need to figure out what items we power off what battery

Follow up batteries

Solved by buying big batteries


Create Harness Delegate Architecture Best Practices.md

Harness Delegate Architecture Best practices
Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66ce89876d5a2ccfb009b827--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66ce98be215411dc0ade0b47--harness-developer.netlify.app


A Search feature like in Web Storm

Hi, I love VS Code for being smooth and fast without complicating things.
I am a full-stack developer and when I'm developing frontend applications I currently use Web Storm.
The one and the only reason why I am unable to switch to VSCode from Web Storm is your Search feature.
The way you show search results just does not help. IMO, It is not user-friendly enough.
I mean if anyone has used IntelliJ or webstorm for a while, he would spot the difference in a flash
If you can hopefully add something like this, or at least improve on what you guys have right now, that would be amazing.

Thanks and Cheers

We don't plan to implement language-specific search features or syntax highlighting in results
https://github.com/microsoft/vscode/issues/92781


[Silicon Labs] Add support for low-power async Serial

Added support for LEUART operation down to EM2.
Updated license headers

@0xc0170

I am on it, got scared yesterday seeing 64 files changed :smile:

LGTM.
My remarks from this review:

might be useful to provide more info in the commit message (for example commit https://github.com/stevew817/mbed/commit/c328fb068e0786bbe1ef68d9cebe97a05b60aa42 could provide some background for that fix)
if possible, send logical-feature-complete-changes via PR (easier to review, fix, integrate, test, ...).

@stevew817 Is this PR complete now?


Modularisation improvements

Reduce dependency on spring and make modules for creating their object factories.
Provide more fine grained control of db transactions.

Codecov Report

Merging #1093 into master will decrease coverage by 0.01%.
The diff coverage is 99.05%.

@@             Coverage Diff              @@
##             master    #1093      +/-   ##
============================================
- Coverage     99.39%   99.38%   -0.02%
- Complexity     2264     2319      +55
============================================
  Files           350      360      +10
  Lines          6298     6518     +220
  Branches        360      374      +14
============================================
+ Hits           6260     6478     +218
  Misses            5        5
- Partials         33       35       +2

Flag
Coverage Δ
Complexity Δ

#unittests
99.38% <99.05%> (-0.02%)
2319.00 <119.00> (+55.00)
:arrow_down:

Impacted Files
Coverage Δ
Complexity Δ

...m/quorum/tessera/data/EncryptedRawTransaction.java
96.77% <ø> (ø)
18.00 <0.00> (ø)

.../com/quorum/tessera/data/EncryptedTransaction.java
100.00% <ø> (ø)
12.00 <0.00> (ø)

.../tessera/transaction/resend/ResendManagerImpl.java
96.07% <93.54%> (ø)
9.00 <1.00> (?)

...com/quorum/tessera/data/EntityManagerTemplate.java
94.73% <94.73%> (ø)
4.00 <4.00> (?)

...orum/tessera/data/EncryptedTransactionDAOImpl.java
98.30% <98.24%> (-1.70%)
19.00 <17.00> (+12.00)
:arrow_down:

...m/quorum/tessera/enclave/DefaultEnclaveHolder.java
100.00% <100.00%> (ø)
5.00 <5.00> (?)

...ava/com/quorum/tessera/enclave/EnclaveFactory.java
100.00% <100.00%> (ø)
10.00 <5.00> (+2.00)

...java/com/quorum/tessera/enclave/EnclaveHolder.java
100.00% <100.00%> (ø)
1.00 <1.00> (?)

...n/java/com/quorum/tessera/enclave/EnclaveImpl.java
100.00% <100.00%> (ø)
18.00 <1.00> (+1.00)

...ava/com/quorum/tessera/enclave/EncodedPayload.java
100.00% <100.00%> (ø)
18.00 <12.00> (+11.00)

... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ebfce7...cf170f6. Read the comment docs.

PR open for almost 2 weeks for internally developed changes.


Feature: Batch Rescind Badge: All Badges of a Member

Easy way to remove all Badges from n specific Member(s) of an organization

As an organization owner, I may want to remove an individual completely from my organization.
This includes:

Revoking all Badges
Removing Manager access

Instead of going from Badge to Badge to remove the user, I want to say "get this 0x out of the org"

This will be included as the working app catches up to the design. This is a very low priority, although it is a needed feature.
When implementing, we will not show a list of all users. The flow will be:

Select badges on Organization page.
Click update holders.
Set if it is a mint or revoke.
Enter the addresses.

-- When the address is set the field should validate that the address can have all those tokens burned.

Archived in preference of #41


Callbacks

Right now PhotoView is very short on callbacks. I'm sure that with the exposition of the internal state through the controller, most of the internal events will be accessible. But I'm sure that some callbacks would make things easier.  React native's implementation of PhotoView has a lot of them.
So I want to hear from you which callbacks you would like to be included in PhotoView's options?

image loadErrorCallback


⚠ Fortnite has degraded performance

In 8ff2569, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1790 ms

Resolved: Fortnite performance has improved in b6be7ad.


Enable a test case - Create the workload from dev file

What does this PR do?
What issues does this PR fix or reference?
Is your PR tested? Consider putting some instruction how to test your changes

/retest

/test v4.7.console-e2e-gcp-console

/test v4.8.console-e2e-gcp-console

/retest


Refactor insert_c1c2 to use execute_concurrent

Fixes #488.
@mambocab and @josh-mckenzie  to review

This is excellent. Other than the parameters nitpick, which I'm not super adamant about, this looks good to me. I'd be happy if this were merged as-is, happier if the API were a little nicer.

After consideration, I'm persuaded. I'll make the change.

@mambocab Bump. I made your changes.

Since flake8 doesn't care either way, I'm preferring it as is.


"First Steps" links to nonexistent page

See this link.
Referred to in documentation but apparently doesn't exist.

What page is linking to that page?

https://github.com/sschmid/Entitas-CSharp/wiki/Code-Generator

The link should be good now.


Update custom words

ARM API Information (Control Plane)
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Azure 1st Party Service can try out the Shift Left experience to initiate API design review from ADO code repo.  If you are interested, may request engineering support by filling in with the form https://aka.ms/ShiftLeftSupportForm.
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[ ] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[x] Other, please clarify, update custom words to add Kubenet's, fix CI in PR #11666.

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month. N/A
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month. N/A
By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

[x] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review.  Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.
Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.
NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)
Please follow the link to find more details on PR review process.

@FumingZhang The change looks good to me. But why not add it in the original PR #11666?

Hey @archerzz, the original PR is a ShiftLeft PR (auto synced to GitHub from ADO). We could not update custom-words.txt located in the repo root. Instead, we could only update the content under specification/containerservice.

/azp run

Hey @archerzz, could you please help merge this PR, thanks!

Hey @archerzz, could you please help merge this PR, thanks!

@FumingZhang Done


RC for v1.12

New workflow "bulk revoke"
Minor fixes and improvements:

add colors to generic form buttons
add fcgi capable script for SCEP (old one still in place)
fix config wrapper to properly reload script-based config

I suggest to add use warnings; in core/server/OpenXPKI/Server/Workflow/Activity/Tools/StringToArray.pm


Submodule for testing utils

Finally, the main reason I've been contributing to this package. Added Neighborhood.Testing submodule with functions that test the return value of search and related funcs.
I'm pretty sure all searches should return identical results regardless of the search structure used (right?), so I think these should be just about all the tests you need when adding a new SS type.
One thing I'm not sure about - are results expected to be sorted by distance? If not I need to change that before this is merged.

are results expected to be sorted by distance?

Nope, not generally. If you have a look at the NearestNeighbors implementation here:  https://github.com/JuliaNeighbors/Neighborhood.jl/blob/master/src/kdtree.jl#L20 I added the possibility to search as an argument. Because depending on the implementation, sorting by distance most of the time costs additional processing, so therefore it should always be optional to not do it.

If someone wants to contribute a new package to Neighborhood.jl, do they have to extend the testing?

If you've implemented the Neighborhood.jl API, the following should be pretty comprehensive for your tests:
using MyPackage, Test, Neighborhood
using Neighborhood.Testing

metric = ...
data = ...
ss = searchstructure(MyStructure, data, metric)

queries = ...

searches = [NeighborNumber(k), WithinRange(r)]

@testset "Single search"
    for query in queries
	for t in searches
	    for skip in [nothing, myskipfunc]
		# Calls search(), isearch(), knn() and checks results match
		idxs, ds = result = search_allfuncs(ss, query, t, skip)

		# Exact test for when brute force method isn't too slow
		@test result = bruteforcesearch(data, metric, query, t, skip)

		# Quicker alternative which just runs some sanity checks on results
		check_search_results(data, metric, results, query, t, skip)
	    end
	end
    end
end

@testset "Bulk search" begin
    for t in searches
	for skip in [nothing, mybulkskipfunc]
	    # Calls bulksearch(), bulkisearch(), and then search() on each query and compares
	    test_bulksearch(ss, queries, t, skip)
	end
    end
end

bruteforcesearch is in my next PR. I'll also need to add a function to this one that compares results up to a reordering and use it instead of straight equality testing.

Okay. Meanwhile I gave you admin rights in this repo, so you can branch directly from here instead of maintining a different fork.

Thanks. This should be good to go. You can see usage in the bruteforce branch.

Awesome. We should put a note in the bruteforce PR saying something like "See the BruteForce tests for an example usage" at the dev docs.