Ntrip client + pyrtcm

RTCM data can also be obtained from RTCM sources using RTCM clients such as https://github.com/nunojpg/ntripclient or https://github.com/liukai-tech/NtripClient-Tools.
Is it possible to submit a stream from an RTCM client to pyrtcm?

Hi @foxittt Sorry, no. pyrtcm can parse RTCM3 data from any Serial (UART) data stream or binary datafile, but not (for example) an HTTP/TCP socket, so it would be necessary to convert any NTRIP data stream into one or other of these accepted formats before pyrtcm could parse it.
There are no current plans to add such functionality, though I'll try to look into what might be involved when I get some time and if there appears to be sufficient interest.

Re-opened as I think this merits further investigation,  but no guaarantee on timescale at this point.

Just found this library with a similar interest in mind. I'm writing a little robot navigation program, that's connecting to an NTRIP client, and then I want to feed the output of the NTRIP session into pyrtcm, and then yield complete RTCM messages that can be forwarded to a ublox GPS module.
It would be nice to be able to feed data into the RTCMReader, and then async await any complete messages. Will be taking a closer look shortly.

Hi @jakepoz  (and @foxittt)
FYI I'm working on a simple command line NTRIP client and hope to upload a preliminary version into the \examples folder in the next week or so. It will connect to a designated NTRIP server (caster), parse the header response and RTCM3 output from that server, and handle the uploading of NMEA GGA position records (using the companion pynmeagps library).
Watch this space.

Nice! I am also experimenting a bit on my side, just got myself a ublox F9P breakout board, and am connecting to a UNAVCO NTRIP server. You can see some of the code I wrote here: https://github.com/jakepoz/deweeder/blob/main/src/ntrip.py
My goal is to write everything using python async generators. So, with  my code you just await the ntrip_client and it will continuously yield messages that you can pass along to your serial port to the GPS. (In my case, I am also mixing a regular flow of message traffic to the GPS on the same serial port)

Hi @jakepoz Sounds interesting.
I've just posted a somewhat rudimentary NTRIP client to the \examples folder. It's clearly not a production-strength implementation but it hopefully demonstrates the basic concepts and provides a basis for further development.
I've just had a quick look at your code and it's clearly more advanced than mine; if you fancy collaborating on a more sophisticated implementation, I'd certainly welcome your contributions. Otherwise, I'd be more than happy to add a reference link to your project.
FYI I've only recently started to explore RTK / NTRIP as part of my SEMU GNSS library suite, as I wasn't sure if there was any significant appetite for Python implementations given the ubiquity of utilities like RTKLIB, gpsd etc. But it appears there is, so I'm hoping to commit more resources to this in future releases (including acquiring the definitive NTRIP protocol specifications rather than relying on public domain resources). As I say, contributions very welcome.

@jakepoz FYI I've just added a new branch add_parse_from_buffer which implements a new static method RTCMReader.parse_buffer() along the lines of the parse_rtcm() routine you use in your ntrip.py code. This supplements the existing RTCMReader.parse() method and allows users to parse data from both a serial stream or a buffer.
I've amended my simple ntripclient.py example to use this new method - hopefully others can do the same in their NTRIP client implementations.
I'll close this issue once this branch is successfully merged, but if you have any further comments or suggestions, by all means raise them.

Thank you! I'm still learning about the async stuff, this is my practical project in order to get the hang of it, so I'm not sure about your question. But I will be taking a look as I can here!

Hello.
Thanks for your work.
Sometimes the script ends with this error.

Hi @foxittt (and @jakepoz!)  This should now be fixed in pyrtcm version 0.2.1-beta. Please install this latest version of pyrtcm and ensure you are using the very latest version of ntripclient.pyin the \examples folder.
If you don't mind me asking, which NTRIP server, port and mountpoint were you using i.e. what does your command line look like? (no need to disclose user or password)
>>> python3 ntripclient.py server=??.??.??.?? port=???? mountpoint=???????? ...
If you have any further problems, feel free to raise another issue.

Please, check again.

If you don't mind me asking, which NTRIP server, port and mountpoint were you using i.e. what does your command line look like? (no need to disclose user or password)
>>> python3 ntripclient.py server=??.??.??.?? port=???? mountpoint=???????? ...

python ntripclient.py server=rt.igs.org port=2101 mountpoint=SSRC01IGS1

Please, check again.

Hi @foxittt you need to be running the very latest version of pyrtcm (0.2.1) - this error message suggests you're still running an older version (>=0.1.8) which doesn't have the parse_buffer() function. Try upgrading using pip:
> python3 -m pip install --upgrade pyrtcm
> python3 -m pip show pyrtcm
Name: pyrtcm
Version: 0.2.1
Summary: RTCM3 Protocol Parser
Home-page: https://github.com/semuconsulting/pyrtcm
Author: semuadmin
Author-email<EMAIL_ADDRESS>License: BSD 3-Clause 'Modified' License
Location: /Users/semuadmin/Library/Python/3.10/lib/python/site-packages
Requires: pyserial
Required-by: PyGPSClient

If you still get the same issue after this, it's possible you may have multiple versions of pyrtcm in different site_packages directories hanging over from previous installations. Run the following command repeatedly until all previous versions are removed and you get the 'not installed' warning, then run the installation command above again.
> python3 -m pip uninstall pyrtcm
WARNING: Skipping pyrtcm as it is not installed.

The script works.
But the previous error still appears.
It always appears in the order shown in the screenshot below.

Ok sorry you're still having problems, and thanks for your patience. I'm reopening this issue and will keep it open until you're happy with the resolution.
I believe message type 1042 is a relatively new (>=RTCM v3.3) message type containing Beidou Satellite Ephemeris Data which is not defined in the 3.2 standard (which was the original basis for my code). I'll get round to updating to RTCM 3.3 in due course, but this alone should not be causing the error you're seeing.
I'll need to investigate further and get back to you.

Hi @foxittt OK this should hopefully be properly fixed in v0.2.2, which you'll need to upgrade to before retrying.
I haven't yet got round to implementing the new v3.3 message types (1042-1046) so they'll still parse as 'Not_Yet_Implemented', but the IndexError should hopefully now be resolved.
I'll keep the issue open until you're able to confirm. Thanks again for your patience and support.

Hi
I've been testing the script for the last few days.
It works without errors.

Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to the PyGPSClient GUI application:
https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-client

Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to the PyGPSClient GUI application: https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-client

Is it possible to test new functionality?

Hi @foxittt Thanks for the update. I'll close this now.
FYI I'm working on adding basic NTRIP Client functionality to the PyGPSClient GUI application: https://github.com/semuconsulting/PyGPSClient/tree/add-NTRIP-client

Is it possible to test new functionality?

Always happy to receive feature requests for new functionality in any of the SEMU Python GNSS libraries. What did you have in mind?
In relation to NTRIP services, just be mindful that some commercial NTRIP services take a dim view of using their services to test or evaluate new software. There are alteratives, such as setting up a local SNIP LITE server on your own machine, but this requires some prior knowledge of the tech.

Please, check again.

Should be sorted now
From: foxittt @.>
Date: Wednesday, 29 June 2022 at 07:30
To: semuconsulting/pyrtcm @.>
Cc: semuadmin @.>, State change @.>
Subject: Re: [semuconsulting/pyrtcm] Ntrip client + pyrtcm (Issue #1)
Please, check again.
[image]https://user-images.githubusercontent.com/2026553/176366952-377a9010-9cfa-4bab-93df-5f7938b510c7.png
[image]https://user-images.githubusercontent.com/2026553/176367042-7d887a56-c324-4099-a48a-508d7a67ab72.png
—
Reply to this email directly, view it on GitHubhttps://github.com/semuconsulting/pyrtcm/issues/1#issuecomment-1169585931, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGZ7C3YYAALYNFD3BURIJRTVRPUQLANCNFSM5RKU4QOA.
You are receiving this because you modified the open/close state.Message ID: @.***>

Should be sorted now From: foxittt @.> Date: Wednesday, 29 June 2022 at 07:30 To: semuconsulting/pyrtcm @.> Cc: semuadmin @.>, State change @.> Subject: Re: [semuconsulting/pyrtcm] Ntrip client + pyrtcm (Issue #1) Please, check again. [image]https://user-images.githubusercontent.com/2026553/176366952-377a9010-9cfa-4bab-93df-5f7938b510c7.png [image]https://user-images.githubusercontent.com/2026553/176367042-7d887a56-c324-4099-a48a-508d7a67ab72.png — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGZ7C3YYAALYNFD3BURIJRTVRPUQLANCNFSM5RKU4QOA. You are receiving this because you modified the open/close state.Message ID: @.***>

Hi @semuadmin
Still an error
Please, check again.

Hi @foxittt
FYI I'm in the processing of moving some of the reader functionality which is common to pyubx2, pynmeagps and pyrtcm into a separate utility library pygnssutils. This contains more fully-featured RTK and NTRIP classes and examples.
You might like to take a look at:
https://github.com/semuconsulting/pygnssutils#gnssntripclient https://github.com/semuconsulting/pygnssutils/blob/main/examples/rtk_example.py
Some of the examples in this library will in due course be moving to pygnssutils, which will hopefully minimise the chances of this kind of issue regression happening.
Hope this helps

Hi @semuadmin
It is very interesting and useful library.
Is it possible to add such functionality:
to format rtcm output as table like this

Hi @foxittt
What error are you still getting with the ntripclient.py example? Are you definitely running the latest updated example code with the latest version of pyrtcm (v0.2.6)?
It seems to be working fine for me...
> python3 ntripclient.py server=rtk2go.com port=2101 mountpoint=WEBBPARTNERS user=anon password=anon
Connected to: rtk2go.com:2101
GET /WEBBPARTNERS HTTP/1.1
User-Agent: pyrtcm NTRIP client/0.1
Authorization: Basic YW5vbjphbm9u
Ntrip-Version: Ntrip/2.0

*** Start Of Header ***
HTTP/1.1 200 OK
Date: Thu, 30 June 2022 07:28:45 UTC
Server: SubCarrier Systems Corp SNIP simpleNTRIP_Caster_[3.05.00]RwPRO
Ntrip-Version: Ntrip/2.0
Cache-Control: no-store, no-cache, max-age=0
Pragma: no-cache
Connection: close
Content-Type: gnss/data
*** End Of Header ***

<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=372545000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=1765781494858514432, NSat=11, DF395=1073807360, NSig=2, DF396=4176895, DF405_01=0.00061272, DF405_02=0.00027956, DF405_03=0.00061271, DF405_04=0.00052751, DF405_05=0.00054932, DF405_06=0.0, DF405_07=3.7e-07, DF405_08=0.00040184, DF405_09=0.00084047, DF405_10=0.00097229, DF405_11=0.0008106, DF405_12=3.615e-05, DF405_13=0.0004822, DF405_14=0.00050736, DF405_15=-0.00036199, DF405_16=0.00020313, DF405_17=-0.00018435, DF405_18=3.147e-05, DF405_19=-0.00044269, DF405_20=-1.428e-05, DF405_21=-0.00017828, DF405_22=-0.0003604, DF406_01=-0.00019168, DF406_02=0.00243546, DF406_03=-0.00186527, DF406_04=-0.00341065, DF406_05=0.00045907, DF406_06=-0.00150083, DF406_07=0.00168963, DF406_08=0.00158228, DF406_09=-0.00048309, DF406_10=4.093e-05, DF406_11=-0.00120554, DF406_12=-0.0021954, DF406_13=0.00161642, DF406_14=0.00152614, DF406_15=0.00089166, DF406_16=-0.00012048, DF406_17=-0.00023344, DF406_18=0.00204755, DF406_19=0.00200356, DF406_20=-0.00368327, DF406_21=-0.0037881, DF406_22=0.00066355, DF407_01=298, DF407_02=139, DF407_03=17, DF407_04=745, DF407_05=1, DF407_06=773, DF407_07=163, DF407_08=867, DF407_09=58, DF407_10=503, DF407_11=908, DF407_12=32, DF407_13=66, DF407_14=692, DF407_15=4, DF407_16=182, DF407_17=835, DF407_18=354, DF407_19=616, DF407_20=237, DF407_21=303, DF407_22=255, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=1, DF420_07=1, DF420_08=0, DF420_09=0, DF420_10=1, DF420_11=0, DF420_12=1, DF420_13=0, DF420_14=1, DF420_15=1, DF420_16=0, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF408_01=10432, DF408_02=8000, DF408_03=2112, DF408_04=10128, DF408_05=8304, DF408_06=10224, DF408_07=6800, DF408_08=2960, DF408_09=2944, DF408_10=8576, DF408_11=8576, DF408_12=3456, DF408_13=3456, DF408_14=13968, DF408_15=2192, DF408_16=2192, DF408_17=416, DF408_18=1440, DF408_19=1680, DF408_20=6800, DF408_21=6736, DF408_22=9824, DF404_01=0.5435, DF404_02=1.0089, DF404_03=-0.7368, DF404_04=-1.6384, DF404_05=0.0328, DF404_06=0.9738, DF404_07=-0.7888, DF404_08=0.9481, DF404_09=0.036, DF404_10=1.1787, DF404_11=-1.6072, DF404_12=1.025, DF404_13=0.8576, DF404_14=1.1782, DF404_15=-1.616, DF404_16=1.1786, DF404_17=-0.7824, DF404_18=1.0764, DF404_19=-0.9348, DF404_20=-0.9166, DF404_21=0.557, DF404_22=0.5844)>
<RTCM(1087, DF002=1087, DF003=0, GNSSEpoch=574597912, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=9277679115173888000, NSat=7, DF395=1090519040, NSig=2, DF396=16127, DF405_01=0.00058938, DF405_02=-0.0008167, DF405_03=0.00049059, DF405_04=0.00061573, DF405_05=-0.00076875, DF405_06=-0.00033882, DF405_07=-0.00073091, DF405_08=-1.645e-05, DF405_09=-0.00094597, DF405_10=2.089e-05, DF405_11=0.00041193, DF405_12=-0.00036437, DF405_13=0.00039156, DF405_14=3.44e-05, DF406_01=-0.00198338, DF406_02=0.00070582, DF406_03=-0.00191314, DF406_04=0.00101879, DF406_05=0.00066027, DF406_06=-0.00355762, DF406_07=-0.00115241, DF406_08=-0.00086473, DF406_09=-7.867e-05, DF406_10=-0.00142049, DF406_11=-0.00162673, DF406_12=0.00015748, DF406_13=0.00155485, DF406_14=-0.00066995, DF407_01=894, DF407_02=688, DF407_03=477, DF407_04=408, DF407_05=26, DF407_06=386, DF407_07=700, DF407_08=685, DF407_09=755, DF407_10=807, DF407_11=90, DF407_12=527, DF407_13=995, DF407_14=433, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=0, DF420_07=0, DF420_08=0, DF420_09=1, DF420_10=0, DF420_11=1, DF420_12=1, DF420_13=0, DF420_14=0, DF408_01=5872, DF408_02=16064, DF408_03=2832, DF408_04=16336, DF408_05=2640, DF408_06=5664, DF408_07=15904, DF408_08=5728, DF408_09=608, DF408_10=544, DF408_11=4672, DF408_12=7744, DF408_13=7776, DF408_14=7776, DF404_01=1.5517, DF404_02=0.5215, DF404_03=0.3042, DF404_04=0.0005, DF404_05=0.4224, DF404_06=0.6277, DF404_07=-1.2108, DF404_08=0.5636, DF404_09=-0.8, DF404_10=0.5636, DF404_11=-1.622, DF404_12=0.4356, DF404_13=-0.7967, DF404_14=0.0737)>
Connection Terminated by User

If you're still getting errors with this after checking you're using the latest code, could I ask you to raise a new issue? This one is getting a little long now. Thanks.

Hi @foxittt
What error are you still getting with the ntripclient.py example? Are you definitely running the latest updated example code with the latest version of pyrtcm (v0.2.6)?
It seems to be working fine for me...
> python3 ntripclient.py server=rtk2go.com port=2101 mountpoint=WEBBPARTNERS user=anon password=anon
Connected to: rtk2go.com:2101
GET /WEBBPARTNERS HTTP/1.1
User-Agent: pyrtcm NTRIP client/0.1
Authorization: Basic YW5vbjphbm9u
Ntrip-Version: Ntrip/2.0

*** Start Of Header ***
HTTP/1.1 200 OK
Date: Thu, 30 June 2022 07:28:45 UTC
Server: SubCarrier Systems Corp SNIP simpleNTRIP_Caster_[3.05.00]RwPRO
Ntrip-Version: Ntrip/2.0
Cache-Control: no-store, no-cache, max-age=0
Pragma: no-cache
Connection: close
Content-Type: gnss/data
*** End Of Header ***

<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=372545000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=1765781494858514432, NSat=11, DF395=1073807360, NSig=2, DF396=4176895, DF405_01=0.00061272, DF405_02=0.00027956, DF405_03=0.00061271, DF405_04=0.00052751, DF405_05=0.00054932, DF405_06=0.0, DF405_07=3.7e-07, DF405_08=0.00040184, DF405_09=0.00084047, DF405_10=0.00097229, DF405_11=0.0008106, DF405_12=3.615e-05, DF405_13=0.0004822, DF405_14=0.00050736, DF405_15=-0.00036199, DF405_16=0.00020313, DF405_17=-0.00018435, DF405_18=3.147e-05, DF405_19=-0.00044269, DF405_20=-1.428e-05, DF405_21=-0.00017828, DF405_22=-0.0003604, DF406_01=-0.00019168, DF406_02=0.00243546, DF406_03=-0.00186527, DF406_04=-0.00341065, DF406_05=0.00045907, DF406_06=-0.00150083, DF406_07=0.00168963, DF406_08=0.00158228, DF406_09=-0.00048309, DF406_10=4.093e-05, DF406_11=-0.00120554, DF406_12=-0.0021954, DF406_13=0.00161642, DF406_14=0.00152614, DF406_15=0.00089166, DF406_16=-0.00012048, DF406_17=-0.00023344, DF406_18=0.00204755, DF406_19=0.00200356, DF406_20=-0.00368327, DF406_21=-0.0037881, DF406_22=0.00066355, DF407_01=298, DF407_02=139, DF407_03=17, DF407_04=745, DF407_05=1, DF407_06=773, DF407_07=163, DF407_08=867, DF407_09=58, DF407_10=503, DF407_11=908, DF407_12=32, DF407_13=66, DF407_14=692, DF407_15=4, DF407_16=182, DF407_17=835, DF407_18=354, DF407_19=616, DF407_20=237, DF407_21=303, DF407_22=255, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=1, DF420_07=1, DF420_08=0, DF420_09=0, DF420_10=1, DF420_11=0, DF420_12=1, DF420_13=0, DF420_14=1, DF420_15=1, DF420_16=0, DF420_17=1, DF420_18=1, DF420_19=1, DF420_20=1, DF420_21=1, DF420_22=1, DF408_01=10432, DF408_02=8000, DF408_03=2112, DF408_04=10128, DF408_05=8304, DF408_06=10224, DF408_07=6800, DF408_08=2960, DF408_09=2944, DF408_10=8576, DF408_11=8576, DF408_12=3456, DF408_13=3456, DF408_14=13968, DF408_15=2192, DF408_16=2192, DF408_17=416, DF408_18=1440, DF408_19=1680, DF408_20=6800, DF408_21=6736, DF408_22=9824, DF404_01=0.5435, DF404_02=1.0089, DF404_03=-0.7368, DF404_04=-1.6384, DF404_05=0.0328, DF404_06=0.9738, DF404_07=-0.7888, DF404_08=0.9481, DF404_09=0.036, DF404_10=1.1787, DF404_11=-1.6072, DF404_12=1.025, DF404_13=0.8576, DF404_14=1.1782, DF404_15=-1.616, DF404_16=1.1786, DF404_17=-0.7824, DF404_18=1.0764, DF404_19=-0.9348, DF404_20=-0.9166, DF404_21=0.557, DF404_22=0.5844)>
<RTCM(1087, DF002=1087, DF003=0, GNSSEpoch=574597912, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=9277679115173888000, NSat=7, DF395=1090519040, NSig=2, DF396=16127, DF405_01=0.00058938, DF405_02=-0.0008167, DF405_03=0.00049059, DF405_04=0.00061573, DF405_05=-0.00076875, DF405_06=-0.00033882, DF405_07=-0.00073091, DF405_08=-1.645e-05, DF405_09=-0.00094597, DF405_10=2.089e-05, DF405_11=0.00041193, DF405_12=-0.00036437, DF405_13=0.00039156, DF405_14=3.44e-05, DF406_01=-0.00198338, DF406_02=0.00070582, DF406_03=-0.00191314, DF406_04=0.00101879, DF406_05=0.00066027, DF406_06=-0.00355762, DF406_07=-0.00115241, DF406_08=-0.00086473, DF406_09=-7.867e-05, DF406_10=-0.00142049, DF406_11=-0.00162673, DF406_12=0.00015748, DF406_13=0.00155485, DF406_14=-0.00066995, DF407_01=894, DF407_02=688, DF407_03=477, DF407_04=408, DF407_05=26, DF407_06=386, DF407_07=700, DF407_08=685, DF407_09=755, DF407_10=807, DF407_11=90, DF407_12=527, DF407_13=995, DF407_14=433, DF420_01=0, DF420_02=0, DF420_03=0, DF420_04=0, DF420_05=0, DF420_06=0, DF420_07=0, DF420_08=0, DF420_09=1, DF420_10=0, DF420_11=1, DF420_12=1, DF420_13=0, DF420_14=0, DF408_01=5872, DF408_02=16064, DF408_03=2832, DF408_04=16336, DF408_05=2640, DF408_06=5664, DF408_07=15904, DF408_08=5728, DF408_09=608, DF408_10=544, DF408_11=4672, DF408_12=7744, DF408_13=7776, DF408_14=7776, DF404_01=1.5517, DF404_02=0.5215, DF404_03=0.3042, DF404_04=0.0005, DF404_05=0.4224, DF404_06=0.6277, DF404_07=-1.2108, DF404_08=0.5636, DF404_09=-0.8, DF404_10=0.5636, DF404_11=-1.622, DF404_12=0.4356, DF404_13=-0.7967, DF404_14=0.0737)>
Connection Terminated by User

If you're still getting errors with this after checking you're using the latest code, could I ask you to raise a new issue? This one is getting a little long now. Thanks.

Hi @foxittt In answer to your question about pygnssutils, it should be perfectly possible to use the pygnssutils library as the basis for an application that formats the RTCM data in the tabular way you've illustrated, but you'd need to write the app yourself. I'm not in a position to write your application code for you.

I`ll try

If you're still getting the RTCMReader has no attribute 'parse_buffer' error message, you're still using the old version of ntripclient.py. Download the new version, updated yesterday, and use this instead.

Thanks.
Now everything is working.

A word of warning - pyrtcm can parse RTCM3 data, but if you're looking to write an application to analyse the RTCM3 data, you may struggle without access to the full RTCM3 specification. Unfortunately the RTCM3 protocol is proprietary and the specification costs $340 to download from here:
https://rtcm.myshopify.com/collections/differential-global-navigation-satellite-dgnss-standards/products/rtcm-10403-2-differential-gnss-global-navigation-satellite-systems-services-version-3-february-1-2013
For copyright reasons I'm not permitted to reproduce the full specification here.

Thanks for the info

Hi @foxittt
I've deleted your recent message as it contains a link to content which is almost certainly in breach of RTCM's copyright. It also referred to the older 3.2 standard - the latest is 3.3.
You can find all sorts of things on the internet, but they're not necessarily legal ;-)

Hi @semuadmin
One more question
I am emailing a question.
Is it possible to add the following to the library:
DF394 of MSM messages of RTCM allows you to determine the number of
satellites. This is implemented in the parsed_data - NSats
It also allows you to determine the numbers of satellites according to the
conversion table. I tried it myself, but I can't get the correct satellite
numbers.
Can you realize this in pyrtcm lib?
Thanks
ср, 6 лип. 2022 р. о 15:14 SEMU Admin @.***> пише:

Hi @foxittt https://github.com/foxittt
I've deleted your recent message as it contains a link to content which is
almost certainly in breach of RTCM's copyright. It also referred to the
older 3.2 standard - the latest is 3.3.
You can find all sorts of things on the internet, but they're not
necessarily legal ;-)
—
Reply to this email directly, view it on GitHub
https://github.com/semuconsulting/pyrtcm/issues/1#issuecomment-1176148505,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPOYOL4YHPPGXTBNAISW3DVSV2D5ANCNFSM5RKU4QOA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi @jakepoz ,
In ptyrtcm3, DF394 already rendered as the attribute NSat; similarly DS395 is rendered as NSig. The attribute NCells = NSat * NSig and this represents the number of repeating groups in the MSM messages. e.g.
<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=304671000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=11620426274396176384, NSat=9, DF395=1073807360, NSig=2, DF396=262063, DF405_01=0.00052882, ...)>

Hi @foxittt
In pyrtcm, the number of satellites contained as a bitmask in DF394 is already rendered as the attribute NSat for convenience; similarly DS395 is rendered as NSig. The attribute NCells = NSat * NSig and this represents the number of repeating groups in the MSM messages. e.g.
<RTCM(1077, DF002=1077, DF003=0, GNSSEpoch=304671000, DF393=1, DF409=0, DF001_7=0, DF411=0, DF412=0, DF417=0, DF418=0, DF394=11620426274396176384, NSat=9, DF395=1073807360, NSig=2, DF396=262063, DF405_01=0.00052882, ...)>

@foxittt
Your recent messages have been deleted AGAIN as a screenshot also potentially constitutes infringement of the copyright. If there is any further infringement I will have no choice but to block you from the repo.
In answer to your question, this is application code which you need to work out for yourself.
PLEASE - NO FURTHER QUERIES AGAINST THIS CLOSE ISSUE.

@foxittt - a friendly gesture, hope this helps but no follow-ups please :-) ...
If I understand what you're after correctly, try something like this:
"""
prnsigmap.py
Illustration of how to map satellite PRN to signal ID from
values of DF394, DF395 and DF396 in RTCM3 MSM7 messages (e.g. 1077).

I can't swear that this is 100% correct, but this is my interpretation
of the RTCM3 specification for MSM messages.
"""

# from RTCM3 specification; similar maps are given for GAL, GLO, etc.
GPS_PRN_MAP = {
    1: "1",
    2: "2",
    3: "3",
    4: "4",
    5: "5",
    6: "6",
    7: "7",
    8: "8",
    9: "9",
    10: "10",
    11: "11",
    12: "12",
    13: "13",
    14: "14",
    15: "15",
    16: "16",
    17: "17",
    18: "18",
    19: "19",
    20: "20",
    21: "21",
    22: "22",
    23: "23",
    24: "24",
    25: "25",
    26: "26",
    27: "27",
    28: "28",
    29: "29",
    30: "30",
    31: "31",
    32: "32",
    33: "33",
    34: "34",
    35: "35",
    36: "36",
    37: "37",
    38: "38",
    39: "39",
    40: "40",
    41: "41",
    42: "42",
    43: "43",
    44: "44",
    45: "45",
    46: "46",
    47: "47",
    48: "48",
    49: "49",
    50: "50",
    51: "51",
    52: "52",
    53: "53",
    54: "54",
    55: "55",
    56: "56",
    57: "57",
    58: "58",
    59: "59",
    60: "60",
    61: "61",
    62: "62",
    63: "63",
    64: "Reserved",
}

# from RTCM3 specification; similar maps are given for GAL, GLO, etc.
GPS_SIG_MAP = {
    1: "Reserved",
    2: "1C",
    3: "1P",
    4: "1W",
    5: "Reserved",
    6: "Reserved",
    7: "Reserved",
    8: "2C",
    9: "2P",
    10: "2W",
    11: "Reserved",
    12: "Reserved",
    13: "Reserved",
    14: "Reserved",
    15: "2S",
    16: "2L",
    17: "2X",
    18: "Reserved",
    19: "Reserved",
    20: "Reserved",
    21: "Reserved",
    22: "5I",
    23: "5Q",
    24: "5X",
    25: "Reserved",
    26: "Reserved",
    27: "Reserved",
    28: "Reserved",
    29: "Reserved",
    30: "1S",
    31: "1L",
    32: "1X",
}

def cell_2_prn(prnmap: dict, sigmap: dict, df394: int, df395: int, df396: int):
    """
    Map Satellite PRN and Signal ID
    """

    print("\nSatellite PRN mapping from DF394...")
    sats = []
    nsat = 0
    for idx in range(64, 0, -1):
        if df394 & pow(2, idx):
            prn = prnmap[64 - idx]
            nsat += 1
            sats.append(prn)
    print(f"NSat: {nsat}; Satellite PRNs: {sats}\n")

    print("Signal ID mapping from DF395...")
    sigs = []
    nsig = 0
    for idx in range(32, 0, -1):
        if df395 & pow(2, idx):
            sig = sigmap[32 - idx]
            nsig += 1
            sigs.append(sig)
    print(f"NSig: {nsig}; Signal IDs: {sigs}\n")

    print("Mapping of Cell to PRN to Signal ID from DF396...")
    cln = nsat * nsig
    ncell = 0
    for prn in range(nsat):
        for sig in range(nsig):
            cln -= 1
            if df396 & pow(2, cln):
                print(f"cell: {ncell+1:02d} prn: {sats[prn]}  sig:  {sigs[sig]}")
            else:
                print(f"cell: {ncell+1:02d} prn: {sats[prn]}  sig:  n/a")
            ncell += 1

DF394 =<PHONE_NUMBER>4396176384
print(f"DF394 in binary: {DF394:0>64b}")
DF395 =<PHONE_NUMBER>
print(f"DF395 in binary: {DF395:0>32b}")
DF396 = 262063
print(f"DF396 in binary: {DF396:0>18b}")

cell_2_prn(GPS_PRN_MAP, GPS_SIG_MAP, DF394, DF395, DF396)

> python3 prnsigmap.py
DF394 in binary:<PHONE_NUMBER>000100000011000010000100000000000000000000000000000000
DF395 in binary:<PHONE_NUMBER>0000010000000000000000
DF396 in binary:<PHONE_NUMBER>10101111

Satellite PRN mapping from DF394...
NSat: 9; Satellite PRNs: ['1', '3', '8', '10', '14', '21', '22', '27', '32']

Signal ID mapping from DF395...
NSig: 2; Signal IDs: ['1C', '2L']

Mapping of Cell to PRN to Signal ID from DF396...
cell: 01 prn: 1  sig:  1C
cell: 02 prn: 1  sig:  2L
cell: 03 prn: 3  sig:  1C
cell: 04 prn: 3  sig:  2L
cell: 05 prn: 8  sig:  1C
cell: 06 prn: 8  sig:  2L
cell: 07 prn: 10  sig:  1C
cell: 08 prn: 10  sig:  2L
cell: 09 prn: 14  sig:  1C
cell: 10 prn: 14  sig:  2L
cell: 11 prn: 21  sig:  1C
cell: 12 prn: 21  sig:  n/a
cell: 13 prn: 22  sig:  1C
cell: 14 prn: 22  sig:  n/a
cell: 15 prn: 27  sig:  1C
cell: 16 prn: 27  sig:  2L
cell: 17 prn: 32  sig:  1C
cell: 18 prn: 32  sig:  2L

Thanks!

Hi @semuadmin
Sorry,
I won't post anymore such links.
ср, 6 лип. 2022 р. о 15:14 SEMU Admin @.***> пише:

Hi @foxittt https://github.com/foxittt
I've deleted your recent message as it contains a link to content which is
almost certainly in breach of RTCM's copyright. It also referred to the
older 3.2 standard - the latest is 3.3.
You can find all sorts of things on the internet, but they're not
necessarily legal ;-)
—
Reply to this email directly, view it on GitHub
https://github.com/semuconsulting/pyrtcm/issues/1#issuecomment-1176148505,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPOYOL4YHPPGXTBNAISW3DVSV2D5ANCNFSM5RKU4QOA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Properties metadata.manifest.main are no more available in json output

What happened:
Up to version 0.103.1 the metadata.manifest.main properties were also included in the output json. Instead, in recent versions when trying to launch syft on the same target input, they are not not inserted in the resulting json.
What you expected to happen:
Include metadata.manifest.main values into the json output
Steps to reproduce the issue:
Anything else we need to know?:
Environment:

Output of syft version: syft 1.0.0
OS (e.g: cat /etc/os-release or similar):

Assuming you're referring to the java manifest metadata type, which was:
https://github.com/anchore/syft/blob/fef0e54c0f04f8480ed40c40fe8de62c37dbb623/syft/pkg/java.go#L68-L71
And is now:
https://github.com/anchore/syft/blob/f4e18961b979f5e6d0cc3b1e4fce608c8ceb29d8/syft/pkg/java.go#L71-L74
This was a change made in https://github.com/anchore/syft/pull/2553 (which is the closest culprit for what you're describing).
That being said, I'm not seeing the behavior you're referring to:
$ syft cloudbees/cloudbees-core-mm:latest -o json > /tmp/core-mm.json
$ cat /tmp/core-mm.json | jq '.artifacts[] | select(.metadata.manifest.main != null)'

{
  "id": "f5da67d7d1f4d742",
  "name": "aws-java-sdk-codeguruprofiler",
  "version": "1.11.995",
  "type": "java-archive",
  "foundBy": "java-archive-cataloger",
  "locations": [
    {
      "path": "/usr/share/jenkins/jenkins.war",
      "layerID": "sha256:545b89fe82a087acefa3ce12783e1da0a1268224364c7310341e408d73985846",
      "accessPath": "/usr/share/jenkins/jenkins.war:WEB-INF/plugins/aws-java-sdk.hpi:WEB-INF/lib/aws-java-sdk-codeguruprofiler-1.11.995.jar",
      "annotations": {
        "evidence": "primary"
      }
    }
  ],
  "licenses": [],
  "language": "java",
  "cpes": [... snip...],
  "purl"<EMAIL_ADDRESS>  "metadataType": "java-archive",
  "metadata": {
    "virtualPath": "/usr/share/jenkins/jenkins.war:WEB-INF/plugins/aws-java-sdk.hpi:WEB-INF/lib/aws-java-sdk-codeguruprofiler-1.11.995.jar",
    "manifest": {
      "main": [
        {
          "key": "Manifest-Version",
          "value": "1.0"
        },
        {
          "key": "Archiver-Version",
          "value": "Plexus Archiver"
        },
        {
          "key": "Built-By",
          "value": "root"
        },
        {
          "key": "Created-By",
          "value": "Apache Maven 3.5.4"
        },
        {
          "key": "Build-Jdk",
          "value": "1.8.0_171"
        }
      ]
    },
    "pomProperties": {
      "path": "META-INF/maven/com.amazonaws/aws-java-sdk-codeguruprofiler/pom.properties",
      "name": "",
      "groupId": "com.amazonaws",
      "artifactId": "aws-java-sdk-codeguruprofiler",
      "version": "1.11.995"
    },
    "digest": [
      {
        "algorithm": "sha1",
        "value": "d16a17cdb41c76ef41b6172d903db751cccda395"
      }
    ]
  }
}
...

There are a lot more examples in that output too.
Do you have any example public docker images that you can share that show this behavior? (with java packages you were expecting to have a main manifest section that you are seeing as null?)

Since we haven't heard back, we'll go ahead and close this issue, but if you need more help, feel free to let us know! Thanks!


feat: migrate charts on import

SUMMARY

When importing a chart, migrate it to a new visualization type if possible.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

A legacy area chart:

When exported and imported into Superset running this branch we get a new echarts area chart:

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Ping @michael-s-molina, care to take another look? 🙇🏼


Allow use of SignalFd through shared reference

Like with many other file descriptors, concurrent use of signalfds is safe. Changing the signal mask of and reading signals from a signalfd can now be done with the SignalFd API even if other references to it exist.
Fixes #2366.
What does this PR do
Checklist:

[x] I have read CONTRIBUTING.md
[x] I have written necessary tests and rustdoc comments
[x] A change log has been added if this PR modifies nix's API

Could you please rebase your branch, the CI failure should be fixed in #2368

Rebased on latest master.

Thanks!


fix(bus_captain): alerts admins if a bus captain is removed

if a bus captain changes bus or declinces a rsvp he is removed as a bus
captain. All admins are alerted to the removal of a bus captain by email

Let's update update later, codeclimate can wait for after 2.0 😬


Without --strictNullChecks, accessing an array of string literal type returns empty string

TypeScript Version: 2.7.0-dev.20171024
Code
function first(x: Array<"a">): "a" {
    return x && x[0];
}

Test with --strictNullChecks off.
Expected behavior:
No error.
Actual behavior:
src/a.ts(2,5): error TS2322: Type '"" | "a"' is not assignable to type '"a"'.
  Type '""' is not assignable to type '"a"'.

This appears to be something to do with use of &&.
With Typescript 2.8.3, I get
TS2322: Type 'number | "" | null' is not assignable to type 'number | null'.
  Type '""' is not assignable to type 'number | null'.

when trying to compile:
function parseDate(optionalDate: string | null): number | null {
    return optionalDate && Date.parse(optionalDate);
}

--strictNullChecks enables stricter behavior for logical operators. something we could not do in the non-strictNullChecks mode to avoid breaking changes to existing code.

@mhegazy I've found this comment in the 2.0 release notes:

The && operator adds null and/or undefined to the type of the right operand depending on which are present in the type of the left operand, and the || operator removes both null and undefined from the type of the left operand in the resulting union type.

According to the example given, my code should not produce an error:
// Compiled with --strictNullChecks
interface Entity {
    name: string;
}
let x: Entity | null;
let s = x && x.name;  // s is of type string | null
let y = x || { name: "test" };  // y is of type Entity

@nealeu your issue is a diffrent issue from the OP.
your function:
function parseDate(optionalDate: string | null): number | null {
    return optionalDate && Date.parse(optionalDate);
}

Shows a correct error. let me explain:
the first check is that optionalDate is truthy, if it is then you will return number | null. if it is not then you will return optionalDate.  A truthy check in JS considers an empty string to be false. so calling this function with  parseDate("") actually returns "" and not number | null. and that is the error message..
I believe you wanted to implement it as:
function parseDate(optionalDate: string | null): number | null {
    return optionalDate?  Date.parse(optionalDate) : null;
}

@mhegazy Thanks for your generosity in fully explaining that.  It now makes perfect sense in that OMG! way that is Javascript.  This is why I tolerated JS and love TS!


Read and write permissions

This action also need the user to give the permission to the github-actions[bot] in your repo's by following the steps below:
Settings -> Actions -> General -> Workflow permissions -> Read and write permissions -> Save.


Add compiler and syntax_tools as dependencies

Add compiler and syntax_tools as dependencies in the application resource file for correct releases using Reltool.

When is is this useful? Are you running the parse transform after the release is built?

When I made a release with 'rebar generate' and run the release, if syntax_tools and compiler are not included, I get the error below. I.e. according to me lager requires those apps to be included (& started).
Including syntax_tools and compiler in the application resource resource file also generates a handy error message is they are omitted in reltool.config.
=INFO REPORT==== 24-Feb-2012::18:55:10 ===
application: lager
exited: {bad_return,
{{lager_app,start,[normal,[]]},
{'EXIT',
{undef,
[{erl_syntax,atom,[module],[]},
{lager_mochiglobal,term_to_abstract,3,
[{file,"src/lager_mochiglobal.erl"},{line,63}]},
{lager_mochiglobal,forms,2,
[{file,"src/lager_mochiglobal.erl"},{line,57}]},
{lager_mochiglobal,compile,2,
[{file,"src/lager_mochiglobal.erl"},{line,51}]},
{lager_mochiglobal,put,3,
[{file,"src/lager_mochiglobal.erl"},{line,31}]},
{lager_app,start,2,
[{file,"src/lager_app.erl"},{line,35}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},
{line,274}]}]}}}}


how to run an Adaptation Project in VSCode with the BAS Toolkit

Hello,
I would like to use Adaptation projects with the toolkit in VSCode but It doesn't work, see image below

The destination is working with A Fiori application, and the Adaptation projects in the BAS itself as well.
I can't see information about it, is it a missing feature or a detail that I have missed?
Denis Galand

@DenisGalandSAP ,
known issue, currently we have a problem with the getDestinations call not working in hybrid mode.


no such file crash when using umap2

[ALWAYS] Testing support: mtp
[INFO  ] Loading USB device mtp
[ERROR ] Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/umap2/apps/scan.py", line 51, in run
    device = self.load_device(device_name, phy)
  File "/usr/local/lib/python2.7/dist-packages/umap2/apps/base.py", line 91, in load_device
    dev = usb_device(self, phy, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/umap2/dev/mtp.py", line 184, in __init__
    USBMtpInterface(app, phy)
  File "/usr/local/lib/python2.7/dist-packages/umap2/dev/mtp.py", line 80, in __init__
    self.object = MtpObject.from_fs_recursive('mtp_fs')
  File "build/bdist.linux-x86_64/egg/mtpdevice/mtp_object.py", line 174, in from_fs_recursive
    obj = MtpObject.from_file(path)
  File "build/bdist.linux-x86_64/egg/mtpdevice/mtp_object.py", line 186, in from_file
    raise Exception('there is no file/dir at %s' % os.path.abspath(path))
Exception: there is no file/dir at /home/username/research/umap2/mtp_fs

solution: make the folder


Multi VM compatible Multi Signature Wallet

Project Abstract
A Multi-Signature Wallet that is user-friendly and simplifies the management of crypto assets across both Wasm and EVM.
Grant level

[ ] Level 1:  Up to $10,000, 2 approvals
[x] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[x] I have read the application guidelines.
[x] Payment details have been provided (Polkadot AssetHub (DOT, USDC & USDT) address in the application and bank details via email, if applicable).
[x] I understand that an agreed upon percentage of each milestone will be paid in vested DOT, to the Polkadot address listed in the application.
[x] I am aware that, in order to receive a grant, I (and the entity I represent) have to successfully complete a KYC/KYB check.
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).

Hey @ashWhiteHat ,
thank you for your interest in our grants program. The idea of a multisig solution that supports both substrate and evm chains sound great. Have you considered to extend existing multisig solutions that already cover the substrate side and have a great UX already? See https://polkadotmultisig.com/ which is powered by Signet from Talisman Wallet. Their code is open source.  I'm interested to hear what you think about extending existing projects.

Hey @PieWol
Thank you for your feedback.
As you mentioned, we considered extending the existing multisig wallet to support EVM, but this would result in high gas costs or exceed the EVM gas limit due to compatibility issues.
The current multisig wallet relies on BLS381, whose interface is not supported by EVM.
Implementing curve operations and signing algorithms from scratch for EVM compatibility would be inefficient and require substantial gas costs.
This would be user-unfriendly and impractical.
Instead of extending the existing multisig wallet, we propose implementing an EVM-compatible multisig as a Substrate pallet.
This approach will enable multi-VM compatibility while providing a user-friendly experience.

Hi @Noc2
Thank you for the question.
For instance, this is main project.
As I mentioned before, these applications have been already delivered.
https://github.com/w3f/Grants-Program/pull/1788#issuecomment-2196183753

Hi @ashWhiteHat thanks for the application. I agree that this sounds interesting, but I'm curious about the implications of using an EVM-compatible substrate solution vs. a typical EVM smart contract multi-sig. Would it possibly introduce any performance overhead or compatibility requirements? How would you ensure that the pallet functions correctly within the substrate runtime?

Hi @keeganquigley
Thank you for the review.
Firstly, the primary motivation(implication) of this proposal is to create a multi-signature wallet that supports both EVM and Wasm with a single private key. This simplifies private key management and allows for a single setup to create a multi-signature wallet for both EVM and Wasm.

Would it possibly introduce any performance overhead or compatibility requirements?

Implementing equivalent functionality in EVM with Substrate's existing multi-sig support results in overhead and performance degradation.
Supporting multi-sig compatible with EVM in Substrate's pallet introduces no overhead and, being in a Wasm environment, improves performance.

How would you ensure that the pallet functions correctly within the substrate runtime?

This proposal includes the implementation of a multi-sig contract wallet for EVM. Therefore, by conducting equivalent functional tests within the Substrate runtime pallet, we can ensure its proper functionality.

Hi @Noc2 Thank you for the question. This is main project. As I mentioned before, these applications have been already delivered. #1788 (comment)

I know that they have been delivered. My concern is that they are no longer maintained or never made it to production.

My concern is that they are no longer maintained or never made it to production.

All of our pallets are ready and available for developers as we describe in our organization README.
https://github.com/KogarashiNetwork#activities
We have been working on privacy and scaling so far.
You held the MTG and told me it was hard to support the privacy project.
Then, we changed the direction and focused on scaling the solution.
However, in the Substrate Builder Program, the scaling solution was not reasonable for the Polkadot because the overhead took more than the scaling benefits.
This was the MTG report slide.
https://docs.google.com/presentation/d/1QIgXMJ_OZPBUzTrpby0j02eR5pozXIYv/edit#slide=id.p1
And this was the research article.
https://medium.com/coinmonks/brief-introduction-of-latest-zero-knowledge-proof-such-as-zkvm-recursion-and-lookup-9cbc60215a47
Finally, we have changed the direction to enterprise service.
That's the background of this proposal.
I can talk about it more detail in the MTG if you would like.

This proposal aims to address the challenges faced when implementing blockchain into corporate business workflows.
Corporate needs typically involve utilizing EVM, which is widely used for transaction settlements, and Wasm for recording the evidence of agreements made on digital documents.
When setting up wallets, it is common practice to store privately generated keys locally, managed by in-house engineers or KYC providers, in cold wallets, rather than relying on seed generation from browser or app wallets.
In constructing multi-signature wallets, our objective is to minimize complex interactions and key management costs between corporations.
This proposal presents a multi-signature wallet designed to meet these requirements.
This use case envisions a business workflow where documents and evidence, agreed upon by a consortium of corporations, can be efficiently verified by anyone in the Wasm environment, while also being stored in a legally binding manner and used for settlements on the widely adopted EVM chain.
Compared to existing wallets, the advantages of this proposal include.

Implementing a multi-signature wallet compatible with both EVM and Wasm environments with minimal management and communication costs.
Providing a service that is well-suited to corporate business workflows.

This wallet enables the provision of corporate services leveraging the benefits of both EVM and Wasm, and we anticipate its application across various scenarios in the future.

Hi @Noc2
Thank you for the polite review.
I understand your opinion.
Hi @PieWol

I fail to understand the benefit of using this solution over deriving both an EVM and substrate style account from the same seed given that you would still have to send transactions to both chains with your solution

This solution is mainly for corporate use cases.
The legal basis for using private keys is that they must be registered by a business recognized by the government, and they are managed on a string basis rather than a seed basis.
Most countries use similar management methods, such as passports and ID cards.
When implementing multisig using that private key, the man-hours required to apply for the private key and set up multisig will double if different keys are used for both EVM and Wasm, depending on the number of participants in the consortium.
Because of the complexity of the process, it is difficult to adopt Wasm as a use case because it requires several months for the approval of private key applications, etc., since it is a collaboration of governments and large companies, and the construction of a multi-sig wallet also requires collaboration between the respective companies.
If this man-hour requirement could be significantly reduced, more corporate users would use Wasm because of the demand for VM efficiency and its function as secure storage.
Hi @coax1d
Thank you for the review.
Yes, the protocol based multisigs is also huge benefits for ecosystem.

Thank you for the detailed explanation @ashWhiteHat. Considering that you confirmed my understanding of the project I'm sorry to let you know that I will not approve it. I simply don't see the need for this project as interesting as it might sound. I hope you understand that we are trying to fund impactful projects within the grants program. Let's see what my colleagues will say. Best of luck.

Hi @ashWhiteHat I'm afraid I have to agree with @PieWol concerns above that I'm not quite convinced about the use case here. There is already a lot of development happening in the multi-sig area and imo it would be better to collaborate on an existing project. I also think that this project could be better funded by VCs, if enough corporations want something like this.
Now that three have voted against it, I will go ahead and close the application. Of course, this decision has nothing to do with your work, which has been great in the past. So let us know if you have other interesting ideas you want to work on.


Menu_And_Commands sample has incorrectly named file Example.MenusAndCommands.png

Installed products

Visual Studio Community 2017 Version 15.3.5

Description
The Menu_And_Commands sample has an incorrectly named file:
Example.MenusAndCommands.png
Should be:
Example.MenuAndCommands.png (singular 'Menu' not 'Menus')
Steps to recreate
Clone sample and build.
Current behavior
Build fails with: "Could not copy the file "C:\Source\VSSDK-Extensibility-Samples\Menu_And_Commands\C#\Example.MenuAndCommands.png" because it was not found."
Expected behavior
Sample should build correctly.
Notes
The solution built after I renamed the file as described above.

Fixed.  Thanks for reporting!


Hide context menu when window is resized

Version
1.2.0
Steps to reproduce
Open context menu via right click, change window's size - context menu still visible, in the wrong place.
Expected Behavior
Context menu should be hidden on window resize event.
Actual Behavior
Appears visible in the wrong place.

For now, you can handle this custom logic using monitor.hideMenu()

@vkbansal this should be handled without extra code

I've released v2.0.0-alpha.1 with this feature. Do check it out. Any feedback is welcome 😃


Comments get removed if a self-running function is directly below imports

Consider a basic React app, the code is formatted by Prettier using the config semi: false, singleQuote: true.
import { createRoot } from 'react-dom/client'
import App from './App'

/**
 * Bootstraps React.
 */
;(function startReact() {
  const root = createRoot(document.getElementById('root'))
  root.render(<App />)
})()

When running eslint --fix with this plugin, the comment is removed and the semicolon is moved. My configuration is 'import-newlines/enforce': [ 'error', { items: Infinity } ], but the bug happens with any settings, as far as I can tell. It happens regardless of using the default JS or TS parser.
import { createRoot } from 'react-dom/client'
import App from './App';

(function startReact() {
  const root = createRoot(document.getElementById('root'))
  root.render(<App />)
})()

The expected behavior is, of course, to keep the comment and preferably leave the semicolon where it was.

I have a sneaking suspicion that because you have a ; in front of the IIFE the comment is treated by the parser as part of the import statement. As far as I know there is no reason to have that there, you can safely remove it and formatting should work as expected.

You're right the semicolon doesn't need to be there at all. The thing is, Prettier always adds the semicolon in semi: false mode and eslint --fix then proceeds to remove the comment. Running Prettier+ESLint is a common use case with an official package that does just that https://github.com/prettier/prettier-eslint .
The actual file has multiple IIFEs, they're unaffected by eslint, except for the first one under the import statements block.

I have released version 1.3.2 with a workaround for this specific issue, please update to this version and reopen the issue if the problem persists.

Excellent work, just tested this and I can confirm it's working correctly now, comments and semicolon formatting being preserved.


Upgrade javac.src.version and javac.target.version to 1.7 or 1.8

The above is well justified. 1.6 is not supported and neither is 1.7 anymore. We need to think about upgrading javac compliance to at least 1.7.

+1 for 1.7

+1

@lewismc what do you mean by saying that 1.6 is not supported, CC won't run on Java v1.6 or v1.7?

I mean that Oracle do not publish updates to these development kits
anymore.
On Wednesday, September 2, 2015, Avi Hayun<EMAIL_ADDRESS>wrote:

@lewismc https://github.com/lewismc what do you mean by saying that 1.6
is not supported, CC won't run on Java v1.6 or v1.7?
—
Reply to this email directly or view it on GitHub
https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137142457
.

--
Lewis

Ahh, I see.
Well, we won't have many issues of backwards compatibility as we know most
of the people who are using CC.
But I don't think we should be only java8 compatible, as many people are
still using java7
So I am +1 for java7
On Wed, Sep 2, 2015 at 7:02 PM, Lewis John McGibbney <
<EMAIL_ADDRESS>wrote:

I mean that Oracle do not publish updates to these development kits
anymore.
On Wednesday, September 2, 2015, Avi Hayun<EMAIL_ADDRESS>wrote:

@lewismc https://github.com/lewismc what do you mean by saying that
1.6
is not supported, CC won't run on Java v1.6 or v1.7?
—
Reply to this email directly or view it on GitHub
<
https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137142457
.

--
Lewis
—
Reply to this email directly or view it on GitHub
https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137144647
.

Working on this now then folks.
PR coming up.
On Thu, Sep 3, 2015 at 3:48 AM, Avi Hayun<EMAIL_ADDRESS>wrote:

Ahh, I see.
Well, we won't have many issues of backwards compatibility as we know most
of the people who are using CC.
But I don't think we should be only java8 compatible, as many people are
still using java7
So I am +1 for java7
On Wed, Sep 2, 2015 at 7:02 PM, Lewis John McGibbney <
<EMAIL_ADDRESS>wrote:

I mean that Oracle do not publish updates to these development kits
anymore.
On Wednesday, September 2, 2015, Avi Hayun<EMAIL_ADDRESS>wrote:

@lewismc https://github.com/lewismc what do you mean by saying that
1.6
is not supported, CC won't run on Java v1.6 or v1.7?
—
Reply to this email directly or view it on GitHub
<

https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137142457

.

--
Lewis
—
Reply to this email directly or view it on GitHub
<
https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137144647
.

—
Reply to this email directly or view it on GitHub
https://github.com/crawler-commons/crawler-commons/issues/93#issuecomment-137366503
.

--
Lewis

Implemented in #94


Add additional weapon accessory requirements

Some weapon accessories require "Rail weapons & Smallarms" or "Melee & Smallarms" those have been added to the available options in weapon accessory details.

Before I merge, just a thought. Would it be a better idea to convert the setting into a checkboxable multi-select thing? Lest we end up creating half a dozen potential combinations over the course of more books?

I'll repeat my Discord comment, as it's been buried: there hasn’t been any accessories since they were introduced in armoury - so until we have a hint the concept will be expanded I don’t think it’s worth a big refactor As the current cases cover all combination But yeah if tech revolution is full of accessories we should refactor I think, maybe at the same time as vehicle type and weapon damage type

Alright, sounds reasonable. :)


Stringify keys in YAML

This PR changes the YAML output to use string keys instead of symbol keys.
I think this is a good idea because it is the default in rails and at least the Rubymine editor.
Garbage collection symbols is also not available (yet) to most people.

I really like this. Thank you!

Cool :+1:
This project continues to be really useful, so... thank you, too! :beers:

Included in the 1.1.0 release.


Cisdp 927

Pre-review checklist
I have ensured that:

[ ] Code runs successfully in the development environment.
[ ] New code is tested to prove that it works. If not, justify reasoning.
[ ] New code is documented using type hinting and following the numpydoc docstring format.
[ ] New code follows the project naming standards.

Overview of changes

Post-review checklist
As I merge these changes, I have:

[ ] Updated the related Jira ticket, including approximate time spent (including review).
[ ] Moved the related Jira ticket to Done.

From the test error here, you might need to use path.as_posix() to convert the pathlib object to a string before writing into the config.


Failed to compile to windows

Failed to compile to windows:
GOOS=windows GOARCH=386 go build <project_name>
# github.com/sirupsen/logrus
../src/github.com/sirupsen/logrus/terminal_windows.go:78: undefined: windows.Syscall

Duplicate of https://github.com/sirupsen/logrus/issues/599 and already fixed a week ago. Please update your dependencies.

I updated my dependencies and problem resolved. Strongly enough i wasn't able to find #599 on google.


Fixed inconsistent indentation in docs/ref/contrib/auth.txt.

It is inconsitent:

for ModelBackend the methods are indented and not prefixed with the class name
for RemoteUserBackend the methods are not indented and prefixed with the class name

Not a big issue. Feel free to close this request if you think it is not an improvement.

The downside is that translators will have to redo some work (I think), but I think it does look more consistent.

Yes, I was going to ask about that. (Exactly what triggers the need for re-translation...?)


Problem with Skype service on Franz 4.0.4 (Mac OS)

After logging in, it's giving me a "Page Not Found" response.

Same here :/

Me too!

Upvote

upvote

+1


Lines get cut (because of overflow) when they are too wide

Just did some sparklines to experiment with the lib.
I found that the lines get cropped when they overflow the inner rect of the chart.
Here is an example:

Notice the top and left side.

Removing the clip-path for the line path fixes this. Why do you have that set anyway?


What is wrong with rs/zerolog?

What is wrong with https://github.com/rs/zerolog?
No offense. Just to better understand your opinion.

Sorry for the late response,
slf4g is a logger made for adjust to your needs and integrate with many other existing frameworks. For example for:

logrus (using slf4g-logrus)
klog (using slf4g-klog)
logr (using slf4g-logr)
Or obviously also the default Go logger using the sdk/bridge

...often you're using libraries which does simply not like your logging library. In those cases you often ending up in different log outputs. This logger library is made in way to make it easier to adapt to those (if not already - as listed above - integrations are available).
On top of that you can customize as you want. JSON (obviously), Text (color and plain) and your own stuff.
Structures of logger: ROOT->A->B->C ... and so on...
In general, I hoped I was able to explain it in the motivation section.
Cheers


[Suggestion] Option to Toggle Cape for Rendering in FP

Rendering the third person model in first person also means rendering the cape (since it is part of the third person model).
This becomes a problem because the cape can sometimes clip into the camera.
This currently happens when sprint-swimming and looking down.

The problem becomes even worse when using cape physics from physics mod; you need not even look down, but straight ahead to see the problem:

Therefore, an option to prevent the cape from rendering whilst in "first person" would solve this issue at the user's discretion, similar to the approach towards arrows/stings being a togglable option.

That doesn't seem to be a normal cape. Never seen heard of the cape getting into the way, since then also the torso would be in the way.

That doesn't seem to be a normal cape. Never seen heard of the cape getting into the way, since then also the torso would be in the way.

Well, after further investigation that's mostly true. Using the mod version 2.2.0 on Fabric 1.19.2 alone, I could still sometimes get the cape to clip into the camera while sprint-swimming and looking down, but it wasn't nearly as bad. Any FOV at Normal or below should not see the problem, but I think the majority of players set their FOV higher than Normal.
It seems like the Capes mod is what's making the problem so bad (in the first two images, the third image is the behavior alongside the Physics mod's Pro Version capes).
It's obvious why the Physics mod causes this problem, but I'm not sure why the Capes mod makes it so bad. Capes Mod does not appear to change the cape's behavior or position on the model; only the texture. I'm guessing your mod already has some near-field culling code that the vanilla cape adheres to, but the Capes mod overwrites the vanilla cape and thus doesn't adhere to the culling?

That all sounds more like an issue with the swimming animation, that the offsets are not good enough for some angles/fov. Nothing to do with the Cape.


Fix broken colors on ButtonGroup

Hoist P/R Checklist
Pull request authors: Review and check off the below. Items that do not apply can also be
checked off to indicate they have been considered. If unclear if a step is relevant, please leave
unchecked and note in comments.

[x] Caught up with develop branch as of last change.
N/A Added CHANGELOG entry, or determined not required.
[x] Reviewed for breaking changes, added breaking-change label + CHANGELOG if so.
N/A Updated doc comments / prop-types, or determined not required.
[x] Reviewed and tested on Mobile, or determined not required.
N/A Created Toolbox branch / PR, or determined not required.

If your change is still a WIP, please use the "Create draft pull request" option in the split
button below to indicate it is not ready yet for a final review.

Pull request reviewers: when merging this P/R, please consider using a squash commit to
collapse multiple intermediate commits into a single commit representing the overall feature
change. This helps keep the commit log clean and easy to scan across releases. PRs containing a
single commit should be rebased when possible.

Took a look at this. This works great in terms of getting the style applied, and think we should take it over the other PR.
I also pushed some further tweaks to the colors:

In dark theme, show the selected button as "highlighted" rather than "depressed" (i.e. brighter). A lot of our struggle here has been trying to darken already dark buttons against a dark background.
In mobile, change the active color. They were previously grey, which I felt looked "disabled" rather than selected.
In all cases, mix in a subtle hint of our selection color to the active button.

Thanks Guys.  Merging.

Tom I looked at this earlier after we chatted and meant to say looked like a clear improvement to me as well.


Update hid-architecture.md

HID Usage Page and Usage ID are 16-bit values accodring to USB HID spec.

Hi @DJm00n,
Thank you for reporting this. 16-bit values are represented by 4 hexadecimal digits, so the table is correct. The USB spec is using abbreviated 8-bit values since the high-order bits are currently all zeros.
I did notice a typo in one of the values for HID UPS Battery, which I will fix.
Thanks,
~ Mark

Oh, you're right.

@DJm00n
Thank you for taking the time to submit this, it helped me discover the typo.
Best wishes,
~ Mark


fix: Make docker compose db init script executable

Fixes the following error when using the docker-compose.yml setup to launch a PostgreSQL instance:
ERROR:  no schema has been selected to create in at character 14

:tada: This PR is included in version 3.11.6 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add flyway hints

This commit contains the adjustments that were made in micronaut and
quarkus to use flyway in native binaries.
The files were copied/took over from the micronaut repository which
copied/took over the files from quarkus.
A minor adjustment was made to use the flyways logger factory instead of
slf4j.
The micronaut repository with the sources can be found here:
https://github.com/micronaut-projects/micronaut-flyway/
The quarkus repository with the sources can be found here:
https://github.com/quarkusio/quarkus/tree/main/extensions/flyway/
The location of flyway scripts can be customized by adding the
flyway.locations property to the buildArgs of the native build
maven/gradle task. It is recognized at build time:
nativeBuild {
  ...
  buildArgs.set(listOf(
    "-Dflyway.locations=classpath:db/migration/postgres"))
}

I am not sure if the location of the files is correct as it contains classes with
@Substitute/@Alias annotations. I haven't found them in the spring-native-configuration repository.
I haven't tested the files after copying them from my test project to this repo. Is it possible to use the build results of the PR build to test the change in a demo application? Or is there another way of testing it without configuring the maven-publish plugin in the spring-native-configuration repository?

I have copied the files from micronaut and did small adjustments as described above.
I've removed the @author and @since annotations from the javadoc as this relates probably mainly to the micronaut changes (at least partially they are identical to the ones in the quarkus repository). I'm not sure whether they have to be kept or whether it's sufficient to have the links to the files in the other repos in the javadoc.
Additionally I had the impression that the latest quarkus version is a little bit different. With the latest changes from the quarkus repo i couldn't start the application successfully (initially they probably had something more similar to the micronaut solution).
The changes that were taken over from micronaut are working successfully in my sample application.
The sample application can be found here:
https://github.com/cschaible/spring-native-intro-samples/tree/master/csc.native.jpa
The docker containers can be started by running the docker/up.sh script.
The application can be built by running ./gradlew clean nativeBuild.
The binary can be started by running: SPRING_PROFILES_ACTIVE=postgres ./csc.native.jpa.

Please add comments if you have refactoring wishes or feel free to adjust the code yourself.

Thanks for this initial work, but for this kind of contribution with code coming from various frameworks, a better home will be the GraalVM native configuration repository + asking agreement of the original authors would be better IMO. We will share more about how to contribute to this repository shortly, stay tuned. Also be aware that tests will have to be provided.


udev symlinks are missing

Symlinks such as /dev/disk/by-{id,label,uuid} are missing after boot. They do show up after login and manually run the /etc/init.d/udev-trigger coldplug init:

udevadm trigger --type=subsystems --action=add
udevadm trigger --type=devices --action=add

I'm having this problem with v0.19.4-dev.5 . No symlinks under /dev/disk/ until I manually run udevadm trigger ... as described here. The files from #427 are present. I'm more of a systemd person, so I'm not sure how to debug. :)
I'll try with v0.11.1.

Just faced this on v0.19.5-rc.1 as well, though the symlinks definitely showed up prior to a node reboot.


ARROW-17561: [Java][ORC] Move JNI build configuration from cpp/ to java/

@github-actions crossbow submit java-jars

+1


Additional datapoints: domain and inclusion criteria

This may not be very necessary for active learning, but it makes the data more meaningful, and accessible on its own. In a structured format, it can be read using scripts without needing to go to the source of the data.
Would very much prefer if the inclusion criteria is a list of the criteria all in boolean question format. This is important for a project I am working on.
And domain, the general field of the research so researchers can be selective.

Thanks for the example! I discussed this with the team, and this example helps. They think it is definitely interesting, and we will look into it.
Question: do you have a reference to the study you are referring to? It looks like a nice addition to our work (if the data is available as well)

I took a paper https://jcircadianrhythms.com/articles/10.5334/jcr.183 in our new collection and asked ChatGPT to extract the questions in the form you requested.

Hi, this is a part of an academic paper on a systematic review:
"Screening was performed in EROS (Early Review Organising Software; Institute of Clinical Effectiveness and Health Policy, Buenos Aires, Argentina) by two independent reviewers. Discrepancies were resolved by discussion among the reviewers. For our mapping review [17], we excluded (1) studies on other techniques than microdialysis (e.g. biosensors and microdialysis precursors such as push-pull perfusion), (2) studies measuring other substances than Hist and the amino acids Asn, Asp, GABA, Glu, Gln, Gly, Pro and Tau, (3) retro-dialysis studies, (4) microdialysis studies that did not report baseline values without the specified molecules in the perfusion fluid, (5) extra-cerebral microdialysis studies, (6) human and in vitro studies, and (7) papers not containing primary study data. Within publications, experiments using techniques other than microdialysis were also ignored (e.g. in [22]), as well as data on amino acids other than those we searched for (e.g. in [23]).
During screening, tags were added by KJ and CL to all studies on circadian rhythms, sleep and sleep deprivation. Tagged studies were subsequently screened for inclusion in this review based on the following criterion (besides being included in our mapping): studies measuring one or more of the molecules of interest during (1) naturally occurring sleep stages that were validated with polysomnographic measurements and/or (2) during sleep deprivation. Studies on e.g. carbachol-induced REM-sleep were thus excluded (e.g. in [23]).
To ensure capturing all relevant studies, we searched for studies with the terms “sleep”, “REM”, “rest”, “fatig”, and “somn” in the title within the studies included in our mapping review."
Can you help me to make a list of the inclusion criteria in the form of boolean questions?

The answer:

Sure! Here is a list of the inclusion criteria in the form of boolean questions:
Was the study conducted using microdialysis technique?
Did the study measure Hist and at least one of the following amino acids: Asn, Asp, GABA, Glu, Gln, Gly, Pro, or Tau?
Was the study a retro-dialysis study?
Did the study report baseline values without the specified molecules in the perfusion fluid?
Was the study an extra-cerebral microdialysis study?
Was the study a human or in vitro study?
Did the paper contain primary study data?
Did the study measure one or more of the molecules of interest during naturally occurring sleep stages that were validated with polysomnographic measurements?
Did the study measure one or more of the molecules of interest during sleep deprivation?
Did the study include any of the following terms in the title: "sleep", "REM", "rest", "fatig", or "somn"?
Was the study not using techniques other than microdialysis, or did it not report data on amino acids other than those we searched for?
I hope this helps! Let me know if you have any further questions.

So this looks quite good and interesting to add.
I also asked to make it into a JSON object, which is interesting:
{
  "criteria": [
    {
      "question": "Was the study conducted using microdialysis technique?",
      "exclusion": false
    },
    {
      "question": "Did the study measure Hist and at least one of the following amino acids: Asn, Asp, GABA, Glu, Gln, Gly, Pro, or Tau?",
      "exclusion": false
    },
    {
      "question": "Was the study a retro-dialysis study?",
      "exclusion": true
    },
    {
      "question": "Did the study report baseline values without the specified molecules in the perfusion fluid?",
      "exclusion": true
    },
    {
      "question": "Was the study an extra-cerebral microdialysis study?",
      "exclusion": true
    },
    {
      "question": "Was the study a human or in vitro study?",
      "exclusion": true
    },
    {
      "question": "Did the paper contain primary study data?",
      "exclusion": true
    },
    {
      "question": "Did the study measure one or more of the molecules of interest during naturally occurring sleep stages that were validated with polysomnographic measurements?",
      "exclusion": false
    },
    {
      "question": "Did the study measure one or more of the molecules of interest during sleep deprivation?",
      "exclusion": false
    },
    {
      "question": "Did the study include any of the following terms in the title: '*sleep*', '*REM*', '*rest*', '*fatig*', or '*somn*'",
      "exclusion": false
    },
    {
      "question": "Was the study not using techniques other than microdialysis, or did it not report data on amino acids other than those we searched for?",
      "exclusion": true
    }
  ]
}

I think your idea is very nice, and we might want to include this for all datasets or as an extension in the future.

ChatGPT to the rescue! I like the JSON format too
(My example was from https://dl.acm.org/doi/pdf/10.1145/3491102.3501875)

We would also very much appreciate this addition. Would it help to crowd-source this? If a bunch of people do one or two reviews each, this is a manageable effort. I'd volunteer to do two!
This would need some instructions though, so it contains what you'd like in the form that you think is best.

Please do not use ChatGPT or T5 or any other machine learning approach to create the inclusion criteria!
The synergy dataset serves an important purpose in that we can use it to test how well machine learning techniques work. It is therefore important that the data contained in it (including on inclusion criteria) is high quality and reliable.
The example given above looks OK on first glance, but read the last one:
    {
      "question": "Was the study not using techniques other than microdialysis, or did it not report data on amino acids other than those we searched for?",
      "exclusion": true
    }

If a study was using only microdialysis, then it was "not using techniques other than microdialysis", so it would pass this question and be excluded! Further, The original criteria included studies measuring one or more of the molecules of interest during (1) naturally occurring sleep stages that were validated with polysomnographic measurements and/or (2) during sleep deprivation. Yet the ChatGPT version splits these two into two separate criteria, making the and/or a simple and. It also mixes up inclusion criteria with the search strategy.
I think there are a couple of tasks involved here. The first one is to simply locate the text in the paper. The second is turn this into a structured form. I actually think just the first part would already be extremely helpful (and a task for humans not ChatGPT).  It may or not be a good idea to use ChatGPT to turn this text into a structured format. This first example shows that it may be more difficult, but if we want to use a model for this task, we should first create a hand-labelled dataset to be able to assess how well the model can perform this task. So in any case we would need to create by hand an ideal structured version of the inclusion criteria.
I think 26 papers is not such a large number that this would not be feasible (at least in a sample of cases), and I don't think this number of papers justifies fine-tuning a model.
I'd be happy to volunteer to extract inclusion criteria for 19% of your systematic reviews

@mcallaghan Thanks for your response!
I have copied and pasted the text containing the inclusion criteria for all SYNERGY papers into a table. The next step is to create a standardized format for presenting the criteria. Do you have ideas on how to present these?

I observed three types of criteria:
1) Meta Criteria
These are often also part of the search query but are once again checked during the screening phase. It are general requirements applicable to all studies being considered for inclusion, often used to ensure consistency in language, accessibility, and document type.
Examples I came across:

Works can be in English, German, Dutch, French, Spanish, or Portuguese. Or Non-English studies are excluded.
Full-text access required;
Articles with only abstract access are excluded.
Include journal papers and conference proceedings. Excludes reviews, editorials, case reports, letters, abstract-only publications, books, book chapters, and non-peer-reviewed sources.
restrictions on the publication date range of the included studies.

Generic Criteria
These are broad inclusion criteria applicable to many studies regardless of the specific research question or topic. They help define the general characteristics of the included studies.

Examples:

Include prospective /or retrospective studies, randomized controlled trials (RCTs), non-randomized controlled trials, and comparative observational studies.
Exclude case reports, cross-sectional studies, and non-controlled studies.
Include human subjects, specific populations such as adults with certain conditions (e.g., cancer, knee osteoarthritis). Excludes pediatric populations under a specific age, non-human organisms, and certain conditions like late-life MDD.
Include specified drugs or therapies (e.g., DPen, trientine for WD).  Comparators can include placebo, no treatment, or different treatments excluding the same drug in different doses.

Specific Criteria
These criteria are unique to the particular topic or question under investigation.

Thanks very much for this!
I think a useful first step would be all the inclusion criteria - exactly as they are written - in text form. For experiments with using LLMs for screening, it would be interesting to see if providing the criteria as they are written helps identify studies better.
I think it could also be interesting to list of each of these, perhaps in a form similar to that suggested above, although I would perhaps consider a simplification
{
  "inclusion_criteria": [
    "Study includes human subjects",
    "Study is written in English",
    "..."
  ],
  "exclusion_criteria": [
    "Study is a human/in vitro study",
    "Study is a case study"
  ]

This is at least easier to understand to my mind. I think it is also crucial to pay attention to double negatives. One could consider writing the exclusion criteria in negative form. "Study must not be a case study", but I think the positive form is slightly easier.
Written like this, one could think of looping through each of the inclusion criteria, and each of the exclusion criteria and asking an LLM to provide the answer for each. Then you could come to the decision based on the combination of these answers.
One final thing is the boolean logic. I am assuming that a study is included if ALL of the inclusion criteria apply, and a study is excluded if ANY of the exclusion criteria apply. However, this might not always be the case. If not, one might want to make the logic explicit.
{
  "inclusion_criteria": (
    "Study includes human subjects" &
    (
      "Study is written in English" |
      "Study is written in Spanish"
    )
  ),
  "exclusion_criteria": (
    "Study is a human/in vitro study" |
    "Study is a case study"
   )
  ]

It is also interesting that we have criteria like "Full-text access required".
Automated approaches will never be able to replicate this, unless we knew exactly which journals the reviewers had access to, and which papers had open fulltext available (on researchgate, requested through researchgate, on google scholar, etc.).
It makes me wonder if these criteria are all applied at the title and abstract level. If there is some distinction made, then it may be wise to separate title and abstract inclusion criteria from full text inclusion criteria.

It would be nice to make the criteria more easily accessible for more datasets! My suggestion would be to use YAML over JSON:

Written like this, one could think of looping through each of the inclusion criteria, and each of the exclusion criteria and asking an LLM to provide the answer for each. Then you could come to the decision based on the combination of these answers.

In our recent paper (to be presented at IAL@ECML-PKDD 2024), we did exactly this!
In our experiments, the criteria were formatted as follows:
criteria:
  - a: |
      Is the study a longitudinal/prospective study with at least three-time point assessments measuring posttraumatic stress disorder (PTSD)?
      Answer with YES if it is a longitudinal/prospective study with at least three-time point assessments measuring PTSD.
      Answer with NO if the study is not longitudinal or prospective or does not measure PTSD.
      Answer with UNKNOWN if the study is longitudinal but the number of time-point assessments is not mentioned
  - b: |
      Does the study assess PTSD symptoms as a continuous variable using an eligible PTSD scale?
      Here are some eligible PTSD scales (answer with YES if the scale is in this list) that measure PTSD as a continuous variable:
      * Clinician Administered PTSD Scale (CAPS)
      * PTSD Checklist (PCL)
     [...]

We transformed the screening protocol to a more verbose version to make it more suitable for the LLM. The idea behind this formatting is as follows:
We use YAML for better multiline support over JSON. Some criteria contain long lists of eligible methods/scales; entering those in JSON is not fun.
Next, every criterion has an identifier (in our work, $a, b, c, d$), but this can be any string.
In the case study of our paper, a study can be included if $a \wedge b \wedge c \wedge d$. For now, the calculation of this decision function was hard-coded in our Python code, but specifying the decision function (for example, in the YAML file) could also be a future option. The logic should also account for unknowns, as not every criterion can be decided based on the title and abstract. We used ternary logic (True, False, Unknown) for this in our paper.
To summarize, I would propose the following format (using the example of @mcallaghan):
criteria:
  - human:  "Study includes human subjects"
  - english: "Study is written in English"
  - spanish: "Study is written in Spanish"
  - invitro: "Study is a human/in vitro study"
  - casestudy: "Study is a case study"
decision_function:
  - main: "inclusion AND NOT exclusion"
  - inclusion: "human AND (english OR spanish)"
  - exclusion: "invitro OR casestudy"

Encoding the decision function fully in YAML may also be an option.

The addition of the eligibility criteria also enables us to study how well we would perform if we took the inclusion and exclusion criteria as a prior starting point compared to the current default of a 1+1 prior. We started making these comparisons in this repository. First preliminary results (using TFIDF and Naive Bayes) show that using these inclusion and exclusion criteria performs about equal with a 1+1 prior knowledge start.


boards: arduino_opta: add M7 core support

With the release of the schematic of the Arduino OPTA, it makes sense to support both cores to run Zephyr, this PR adds it, and exposes some of the features.
Please notice the schematic lacks of some information of the WIFI+BT combo, so they are not supported this time, on the other hand USB and Ethernet access are available.
For console purpose it is possible to use the side expansion connector and wire the UART pins from there, the documentation mentions that

Added some extra information, the OPTA has two applications scenarios:

Zephyr M7r + Zephyr M4  Images: the image on the M4 has the base address located to the beginning of the second flash bank
Arduino M7 + Zephyr M4: In this case Arduino code sets the boot address of the M7 to 0x8008000.

In the second case, it is not possible to use the same DTS configuration, we need to create a code partition with custom address, and it was done on the previous PR: https://github.com/zephyrproject-rtos/zephyr/pull/64101 which brought the support for the M4, the solution adopted here, is to make default Zephyr + Zephyr configurations, and point in the documentation how to create an overlay file with the code partition when mixing Arduino code with Zephyr code.
Other than that, tested on Hardware a set of samples on both cores.
It is unfortunate that the debug port is not exposed  on Opta.

I was not aware of that, thank you.
@pillo79 , do you think there is something here that can enhance the other PR?

Closing this one in favor of the existing original PR, I'm going to work on the additional support.


Initial changelog for v0.9.0

@nvkelso I scanned through the commits to generate this initial list. Can you make any necessary updates to it?

Yep, moving assignment to me.

Back to Rob for review.

Made some minor edits. :+1:

Good catches (not sure why Mou didn't preview some of those errors to me!). Merging.


SCP-1663: Do the debruijn transformation in the plugin

This way it's done at compile-time, rather than runtime, which should
shave some time off the tests.
Most of the noise is moving the de-bruijn passes to use prismatic errors so they're easier to slot in to other places.
I don't see much difference in the tests locally, but this should address Ulf's observation in https://github.com/input-output-hk/plutus/pull/2555.

Pre-submit checklist:

Branch

[x] Commit sequence broadly makes sense
[x] Key commits have useful messages
[x] Relevant tickets are mentioned in commit messages

PR

[x] Self-reviewed the diff
[x] Useful pull request description
[x] Reviewer requested

If you updated any cabal files or added Haskell packages:

[ ] $(nix-build default.nix -A pkgsLocal.updateMaterialized) to update the materialized Nix files
[ ] Update hie-*.yaml files if needed

If you changed any Haskell files:

[ ] $(nix-shell shell.nix --run fix-stylish-haskell) to fix any formatting issues

If you changed any Purescript files:

[ ] $(nix-shell shell.nix --run fix-purty) to fix any formatting issues

Pre-merge checklist:

[ ] Someone approved it
[ ] Commits have useful messages
[ ] Review clarifications made it into the code
[ ] History is moderately tidy; or going to squash-merge

Notably, the thing this doesn't handle is the compilation of the stuff that's made with Lift... but that's irredeemably a runtime thing, so there's nothing we can do about that.

Notably, the thing this doesn't handle is the compilation of the stuff that's made with Lift... but that's irredeemably a runtime thing, so there's nothing we can do about that.

I think this will break plutus-benchmark/nofib in a number of places.  Each benchmark has a function called mk<BenchmarkName>Term which calls Code.getPlc to get PLC from the Haskell version of the relevant benchmark, and the return type of that function has changed.  We'll probably need to call unDeBruijnTerm to convert things back to named form (we're not attempting to run de Bruijn-indexed code directly yet, right?), so we should probably just wrap getPlc in a function that does that too.
There also seems to be a problem with ambiguous type variables in plutus-core/exe/Main.hs.

I think this will break plutus-benchmark/nofib in a number of places.  Each benchmark has a function called mk<BenchmarkName>Term which calls Code.getPlc to get PLC from the Haskell version of the relevant benchmark, and the return type of that function has changed.  We'll probably need to call unDeBruijnTerm to convert things back to named form (we're not attempting to run de Bruijn-indexed code directly yet, right?), so we should probably just wrap getPlc in a function that does that too.
There also seems to be a problem with ambiguous type variables in plutus-core/exe/Main.hs.

There also seems to be a problem with ambiguous type variables in several places in plutus-core/exe/Main.hs.

I've fixed that with ... runExceptT @UPLC.FreeVariableError ... in a couple of places.

There also seems to be a problem with ambiguous type variables in several places in plutus-core/exe/Main.hs.

I've fixed that with ... runExceptT @UPLC.FreeVariableError ... in a couple of places.

Hmmm.  I was trying to fix the benchmarks, but I had to introduce monads all over the place and now I'm having trouble when the monadic code has to interact with Criterion.  I'll have to stop and think about that.

Hmmm.  I was trying to fix the benchmarks, but I had to introduce monads all over the place and now I'm having trouble when the monadic code has to interact with Criterion.  I'll have to stop and think about that.

I'm looking at this, leave it to me!

I'm looking at this, leave it to me!


Add arbitrary floors

Change floors to be denoted by objects named Этаж I-F (where I is instance number, and F is floor number) on the "этажи" layer on the map, allowing arbitrary sizes and locations of floors.

Closed by #33.


Disable REMOTE_TERMINAL capability for devices when Edgehog Forwarder is disabled

Currently, if devices supports the Astarte interfaces for Forwarder sessions but a Forwarder instance is not configured in Edgehog, the devices still reports a REMOTE_TERMINAL capability.
Since the presence of the capability controls whether the remote terminal functionality is enabled on Edgehog frontend, and it would be odd to display it enabled without a supporting Forwarder instance, this PR disables the capability on devices if a Forwarder instance is not configured.

I'm closing this as superseded by #467, which provides a more compelling solution.